DPDK patches and discussions
 help / color / mirror / Atom feed
From: Yuan Wang <yuanx.wang@intel.com>
To: dev@dpdk.org, Thomas Monjalon <thomas@monjalon.net>,
	Ferruh Yigit <ferruh.yigit@xilinx.com>,
	Andrew Rybchenko <andrew.rybchenko@oktetlabs.ru>
Cc: mdr@ashroe.eu, xiaoyun.li@intel.com, aman.deep.singh@intel.com,
	yuying.zhang@intel.com, qi.z.zhang@intel.com,
	qiming.yang@intel.com, jerinjacobk@gmail.com,
	viacheslavo@nvidia.com, stephen@networkplumber.org,
	xuan.ding@intel.com, hpothula@marvell.com, yaqi.tang@intel.com,
	Yuan Wang <yuanx.wang@intel.com>,
	Wenxuan Wu <wenxuanx.wu@intel.com>
Subject: [PATCH v5 2/4] ethdev: introduce protocol hdr based buffer split
Date: Mon, 26 Sep 2022 17:40:37 +0800	[thread overview]
Message-ID: <20220926094039.1572741-3-yuanx.wang@intel.com> (raw)
In-Reply-To: <20220926094039.1572741-1-yuanx.wang@intel.com>

Currently, Rx buffer split supports length based split. With Rx queue
offload RTE_ETH_RX_OFFLOAD_BUFFER_SPLIT enabled and Rx packet segment
configured, PMD will be able to split the received packets into
multiple segments.

However, length based buffer split is not suitable for NICs that do split
based on protocol headers. Given an arbitrarily variable length in Rx
packet segment, it is almost impossible to pass a fixed protocol header to
driver. Besides, the existence of tunneling results in the composition of
a packet is various, which makes the situation even worse.

This patch extends current buffer split to support protocol header based
buffer split. A new proto_hdr field is introduced in the reserved field
of rte_eth_rxseg_split structure to specify protocol header. The proto_hdr
field defines the split position of packet, splitting will always happen
after the protocol header defined in the Rx packet segment. When Rx queue
offload RTE_ETH_RX_OFFLOAD_BUFFER_SPLIT is enabled and corresponding
protocol header is configured, driver will split the ingress packets into
multiple segments.

Examples for proto_hdr field defines:
To split after ETH-IPV4-UDP, it should be defined as
RTE_PTYPE_L2_ETHER | RTE_PTYPE_L3_IPV4_EXT_UNKNOWN | RTE_PTYPE_L4_UDP

For inner ETH-IPV4-UDP, it should be defined as
RTE_PTYPE_TUNNEL_GRENAT | RTE_PTYPE_INNER_L2_ETHER |
RTE_PTYPE_INNER_L3_IPV4_EXT_UNKNOWN | RTE_PTYPE_INNER_L4_UDP

struct rte_eth_rxseg_split {
        struct rte_mempool *mp; /* memory pools to allocate segment from */
        uint16_t length; /* segment maximal data length,
                            configures split point */
        uint16_t offset; /* data offset from beginning
                            of mbuf data buffer */
        /**
	 * Proto_hdr defines a bit mask of the protocol sequence as
         * RTE_PTYPE_*, configures split point. The last RTE_PTYPE*
         * in the mask indicates the split position.
	 * For non-tunneling packets, the complete protocol sequence
         * should be defined.
	 * For tunneling packets, for simplicity, only the tunnel and
         * inner protocol sequence should be defined.
	 */
        uint32_t proto_hdr;
};

If protocol header split can be supported by a PMD, the
rte_eth_buffer_split_get_supported_hdr_ptypes function can
be use to obtain a list of these protocol headers.

For example, let's suppose we configured the Rx queue with the
following segments:
        seg0 - pool0, proto_hdr0=RTE_PTYPE_L2_ETHER | RTE_PTYPE_L3_IPV4,
               off0=2B
        seg1 - pool1, proto_hdr1=RTE_PTYPE_L2_ETHER | RTE_PTYPE_L3_IPV4
               | RTE_PTYPE_L4_UDP, off1=128B
        seg2 - pool2, off1=0B

The packet consists of ETH_IPV4_UDP_PAYLOAD will be split like
following:
        seg0 - ipv4 header @ RTE_PKTMBUF_HEADROOM + 2 in mbuf from pool0
        seg1 - udp header @ 128 in mbuf from pool1
        seg2 - payload @ 0 in mbuf from pool2

Note: NIC will only do split when the packets exactly match all the
protocol headers in the segments. For example, if ARP packets received
with above config, the NIC won't do split for ARP packets since
it does not contains ipv4 header and udp header. These packets will be put
into the last valid mempool, with zero offset.

Now buffer split can be configured in two modes. For length based
buffer split, the mp, length, offset field in Rx packet segment should
be configured, while the proto_hdr field will be ignored.
For protocol header based buffer split, the mp, offset, proto_hdr field
in Rx packet segment should be configured, while the length field will
be ignored.

The split limitations imposed by underlying driver is reported in the
rte_eth_dev_info->rx_seg_capa field. The memory attributes for the split
parts may differ either, dpdk memory and external memory, respectively.

Signed-off-by: Yuan Wang <yuanx.wang@intel.com>
Signed-off-by: Xuan Ding <xuan.ding@intel.com>
Signed-off-by: Wenxuan Wu <wenxuanx.wu@intel.com>
---
 doc/guides/rel_notes/release_22_11.rst |  7 +++
 lib/ethdev/rte_ethdev.c                | 74 ++++++++++++++++++++++----
 lib/ethdev/rte_ethdev.h                | 29 +++++++++-
 3 files changed, 98 insertions(+), 12 deletions(-)

diff --git a/doc/guides/rel_notes/release_22_11.rst b/doc/guides/rel_notes/release_22_11.rst
index 8e5bdde46a..cce1f6e50c 100644
--- a/doc/guides/rel_notes/release_22_11.rst
+++ b/doc/guides/rel_notes/release_22_11.rst
@@ -64,6 +64,13 @@ New Features
   Added ``rte_eth_buffer_split_get_supported_hdr_ptypes()``, to get supported
   header protocols of a PMD to split.
 
+* **Added protocol header based buffer split.**
+
+  Ethdev: The ``reserved`` field in the ``rte_eth_rxseg_split`` structure is
+  replaced with ``proto_hdr`` to support protocol header based buffer split.
+  User can choose length or protocol header to configure buffer split
+  according to NIC's capability.
+
 
 Removed Items
 -------------
diff --git a/lib/ethdev/rte_ethdev.c b/lib/ethdev/rte_ethdev.c
index 1f0a7f8f3f..27ec19faed 100644
--- a/lib/ethdev/rte_ethdev.c
+++ b/lib/ethdev/rte_ethdev.c
@@ -1649,9 +1649,10 @@ rte_eth_dev_is_removed(uint16_t port_id)
 }
 
 static int
-rte_eth_rx_queue_check_split(const struct rte_eth_rxseg_split *rx_seg,
-			     uint16_t n_seg, uint32_t *mbp_buf_size,
-			     const struct rte_eth_dev_info *dev_info)
+rte_eth_rx_queue_check_split(uint16_t port_id,
+			const struct rte_eth_rxseg_split *rx_seg,
+			uint16_t n_seg, uint32_t *mbp_buf_size,
+			const struct rte_eth_dev_info *dev_info)
 {
 	const struct rte_eth_rxseg_capa *seg_capa = &dev_info->rx_seg_capa;
 	struct rte_mempool *mp_first;
@@ -1674,6 +1675,7 @@ rte_eth_rx_queue_check_split(const struct rte_eth_rxseg_split *rx_seg,
 		struct rte_mempool *mpl = rx_seg[seg_idx].mp;
 		uint32_t length = rx_seg[seg_idx].length;
 		uint32_t offset = rx_seg[seg_idx].offset;
+		uint32_t proto_hdr = rx_seg[seg_idx].proto_hdr;
 
 		if (mpl == NULL) {
 			RTE_ETHDEV_LOG(ERR, "null mempool pointer\n");
@@ -1707,13 +1709,63 @@ rte_eth_rx_queue_check_split(const struct rte_eth_rxseg_split *rx_seg,
 		}
 		offset += seg_idx != 0 ? 0 : RTE_PKTMBUF_HEADROOM;
 		*mbp_buf_size = rte_pktmbuf_data_room_size(mpl);
-		length = length != 0 ? length : *mbp_buf_size;
-		if (*mbp_buf_size < length + offset) {
-			RTE_ETHDEV_LOG(ERR,
-				       "%s mbuf_data_room_size %u < %u (segment length=%u + segment offset=%u)\n",
-				       mpl->name, *mbp_buf_size,
-				       length + offset, length, offset);
-			return -EINVAL;
+
+		if (proto_hdr > 0) {
+			/* Split based on protocol headers. */
+
+			/* skip the payload */
+			if (proto_hdr == RTE_PTYPE_ALL_MASK)
+				continue;
+
+			int ptype_cnt;
+
+			ptype_cnt = rte_eth_buffer_split_get_supported_hdr_ptypes(port_id, NULL, 0);
+			if (ptype_cnt <= 0) {
+				RTE_ETHDEV_LOG(ERR,
+					"Port %u failed to supported buffer split header protocols\n",
+					port_id);
+				return -EINVAL;
+			}
+
+			uint32_t ptypes[ptype_cnt];
+			int i;
+
+			ptype_cnt = rte_eth_buffer_split_get_supported_hdr_ptypes(port_id,
+										ptypes, ptype_cnt);
+			if (ptype_cnt < 0) {
+				RTE_ETHDEV_LOG(ERR,
+					"Port %u failed to supported buffer split header protocols\n",
+					port_id);
+				return -EINVAL;
+			}
+
+			for (i = 0; i < ptype_cnt; i++)
+				if (ptypes[i] == proto_hdr)
+					break;
+			if (i == ptype_cnt) {
+				RTE_ETHDEV_LOG(ERR,
+					"Requested Rx split header protocols 0x%x is not supported.\n",
+					proto_hdr);
+				return -EINVAL;
+			}
+
+			if (*mbp_buf_size < offset) {
+				RTE_ETHDEV_LOG(ERR,
+						"%s mbuf_data_room_size %u < %u segment offset)\n",
+						mpl->name, *mbp_buf_size,
+						offset);
+				return -EINVAL;
+			}
+		} else {
+			/* Split at fixed length. */
+			length = length != 0 ? length : *mbp_buf_size;
+			if (*mbp_buf_size < length + offset) {
+				RTE_ETHDEV_LOG(ERR,
+					"%s mbuf_data_room_size %u < %u (segment length=%u + segment offset=%u)\n",
+					mpl->name, *mbp_buf_size,
+					length + offset, length, offset);
+				return -EINVAL;
+			}
 		}
 	}
 	return 0;
@@ -1793,7 +1845,7 @@ rte_eth_rx_queue_setup(uint16_t port_id, uint16_t rx_queue_id,
 		n_seg = rx_conf->rx_nseg;
 
 		if (rx_conf->offloads & RTE_ETH_RX_OFFLOAD_BUFFER_SPLIT) {
-			ret = rte_eth_rx_queue_check_split(rx_seg, n_seg,
+			ret = rte_eth_rx_queue_check_split(port_id, rx_seg, n_seg,
 							   &mbp_buf_size,
 							   &dev_info);
 			if (ret != 0)
diff --git a/lib/ethdev/rte_ethdev.h b/lib/ethdev/rte_ethdev.h
index c440e3863a..ba7c11f735 100644
--- a/lib/ethdev/rte_ethdev.h
+++ b/lib/ethdev/rte_ethdev.h
@@ -994,6 +994,9 @@ struct rte_eth_txmode {
  *   specified in the first array element, the second buffer, from the
  *   pool in the second element, and so on.
  *
+ * - The proto_hdrs in the elements define the split position of
+ *   received packets.
+ *
  * - The offsets from the segment description elements specify
  *   the data offset from the buffer beginning except the first mbuf.
  *   The first segment offset is added with RTE_PKTMBUF_HEADROOM.
@@ -1015,12 +1018,36 @@ struct rte_eth_txmode {
  *     - pool from the last valid element
  *     - the buffer size from this pool
  *     - zero offset
+ *
+ * - Length based buffer split:
+ *     - mp, length, offset should be configured.
+ *     - The proto_hdr field will be ignored.
+ *
+ * - Protocol header based buffer split:
+ *     - mp, offset, proto_hdr should be configured.
+ *     - The length field will be ignored.
+ *
+ * - For Protocol header based buffer split, if the received packets
+ *   don't exactly match all protocol headers in the elements, packets
+ *   will not be split.
+ *   These packets will be put into:
+ *     - pool from the last valid element
+ *     - the buffer size from this pool
+ *     - zero offset
  */
 struct rte_eth_rxseg_split {
 	struct rte_mempool *mp; /**< Memory pool to allocate segment from. */
 	uint16_t length; /**< Segment data length, configures split point. */
 	uint16_t offset; /**< Data offset from beginning of mbuf data buffer. */
-	uint32_t reserved; /**< Reserved field. */
+	/**
+	 * Proto_hdr defines a bit mask of the protocol sequence as RTE_PTYPE_*,
+	 * configures split point. The last RTE_PTYPE* in the mask indicates the
+	 * split position.
+	 * For non-tunneling packets, the complete protocol sequence should be defined.
+	 * For tunneling packets, for simplicity, only the tunnel and inner
+	 * protocol sequence should be defined.
+	 */
+	uint32_t proto_hdr;
 };
 
 /**
-- 
2.25.1


  parent reply	other threads:[~2022-09-26  1:57 UTC|newest]

Thread overview: 72+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2022-08-12 18:15 [PATCH 0/4] support protocol " Yuan Wang
2022-08-12 18:15 ` [PATCH 1/4] ethdev: introduce protocol header API Yuan Wang
2022-08-12 18:15 ` [PATCH 2/4] ethdev: introduce protocol hdr based buffer split Yuan Wang
2022-08-12 18:15 ` [PATCH 3/4] app/testpmd: add rxhdrs commands and parameters Yuan Wang
2022-08-12 18:15 ` [PATCH 4/4] net/ice: support buffer split in Rx path Yuan Wang
2022-09-01 22:33 ` [PATCH v2 0/4] support protocol based buffer split Yuan Wang
2022-09-01 22:34 ` [PATCH v2 1/4] ethdev: introduce protocol header API Yuan Wang
2022-09-01 22:35 ` [PATCH v2 2/4] ethdev: introduce protocol hdr based buffer split Yuan Wang
2022-09-01 22:36 ` [PATCH v2 3/4] app/testpmd: add rxhdrs commands and parameters Yuan Wang
2022-09-01 22:37 ` [PATCH v2 4/4] net/ice: support buffer split in Rx path Yuan Wang
2022-09-02 19:10 ` [PATCH v3 0/4] support protocol based buffer split Yuan Wang
2022-09-02 19:10   ` [PATCH v3 1/4] ethdev: introduce protocol header API Yuan Wang
2022-09-12 11:24     ` Andrew Rybchenko
2022-09-16  8:34       ` Wang, YuanX
2022-09-02 19:10   ` [PATCH v3 2/4] ethdev: introduce protocol hdr based buffer split Yuan Wang
2022-09-12 11:47     ` Andrew Rybchenko
2022-09-16  8:38       ` Wang, YuanX
2022-09-20  5:35         ` Andrew Rybchenko
2022-09-22  3:13           ` Wang, YuanX
2022-09-13  7:56     ` Suanming Mou
2022-09-16  8:39       ` Wang, YuanX
2022-09-02 19:10   ` [PATCH v3 3/4] app/testpmd: add rxhdrs commands and parameters Yuan Wang
2022-09-02 19:10   ` [PATCH v3 4/4] net/ice: support buffer split in Rx path Yuan Wang
2022-09-20 11:12 ` [PATCH v4 0/4] support protocol based buffer split Yuan Wang
2022-09-20 11:12   ` [PATCH v4 1/4] ethdev: introduce protocol header API Yuan Wang
2022-09-20 11:12   ` [PATCH v4 2/4] ethdev: introduce protocol hdr based buffer split Yuan Wang
2022-09-20 11:12   ` [PATCH v4 3/4] app/testpmd: add rxhdrs commands and parameters Yuan Wang
2022-09-20 11:12   ` [PATCH v4 4/4] net/ice: support buffer split in Rx path Yuan Wang
2022-09-26  9:40 ` [PATCH v5 0/4] support protocol based buffer split Yuan Wang
2022-09-26  9:40   ` [PATCH v5 1/4] ethdev: introduce protocol header API Yuan Wang
2022-09-26  9:40   ` Yuan Wang [this message]
2022-09-28 15:42     ` [PATCH v5 2/4] ethdev: introduce protocol hdr based buffer split Wang, YuanX
2022-09-26  9:40   ` [PATCH v5 3/4] app/testpmd: add rxhdrs commands and parameters Yuan Wang
2022-09-26  9:40   ` [PATCH v5 4/4] net/ice: support buffer split in Rx path Yuan Wang
2022-09-29 18:59 ` [PATCH v6 0/4] support protocol based buffer split Yuan Wang
2022-09-29 18:59   ` [PATCH v6 1/4] ethdev: introduce protocol header API Yuan Wang
2022-09-29 18:59   ` [PATCH v6 2/4] ethdev: introduce protocol hdr based buffer split Yuan Wang
2022-09-29 18:59   ` [PATCH v6 3/4] app/testpmd: add rxhdrs commands and parameters Yuan Wang
2022-09-29 18:59   ` [PATCH v6 4/4] net/ice: support buffer split in Rx path Yuan Wang
2022-09-30  6:45     ` Tang, Yaqi
2022-10-01 21:05 ` [PATCH v7 0/4] support protocol based buffer split Yuan Wang
2022-10-01 21:05   ` [PATCH v7 1/4] ethdev: introduce protocol header API Yuan Wang
2022-10-03  7:04     ` Andrew Rybchenko
2022-10-04  2:21       ` Wang, YuanX
2022-10-04  7:52         ` Andrew Rybchenko
2022-10-04 15:00           ` Wang, YuanX
2022-10-01 21:05   ` [PATCH v7 2/4] ethdev: introduce protocol hdr based buffer split Yuan Wang
2022-10-02  4:01     ` Wang, YuanX
2022-10-03  7:47     ` Andrew Rybchenko
2022-10-04  2:48       ` Wang, YuanX
2022-10-04  8:22         ` Andrew Rybchenko
2022-10-04 15:01           ` Wang, YuanX
2022-10-01 21:05   ` [PATCH v7 3/4] app/testpmd: add rxhdrs commands and parameters Yuan Wang
2022-10-01 21:05   ` [PATCH v7 4/4] net/ice: support buffer split in Rx path Yuan Wang
2022-10-05 23:18 ` [PATCH v8 0/4] support protocol based buffer split Yuan Wang
2022-10-05 23:18   ` [PATCH v8 1/4] ethdev: introduce protocol header API Yuan Wang
2022-10-06 10:11     ` Andrew Rybchenko
2022-10-05 23:18   ` [PATCH v8 2/4] ethdev: introduce protocol hdr based buffer split Yuan Wang
2022-10-06 10:11     ` Andrew Rybchenko
2022-10-08 14:30       ` Ding, Xuan
2022-10-05 23:18   ` [PATCH v8 3/4] app/testpmd: add rxhdrs commands and parameters Yuan Wang
2022-10-06 10:12     ` Andrew Rybchenko
2022-10-05 23:18   ` [PATCH v8 4/4] net/ice: support buffer split in Rx path Yuan Wang
2022-10-06 10:12     ` Andrew Rybchenko
2022-10-06 10:13   ` [PATCH v8 0/4] support protocol based buffer split Andrew Rybchenko
2022-10-09 20:25 ` [PATCH v9 " Yuan Wang
2022-10-09 14:58   ` Andrew Rybchenko
2022-10-10  2:45     ` Ding, Xuan
2022-10-09 20:25   ` [PATCH v9 1/4] ethdev: introduce protocol header API Yuan Wang
2022-10-09 20:25   ` [PATCH v9 2/4] ethdev: introduce protocol hdr based buffer split Yuan Wang
2022-10-09 20:25   ` [PATCH v9 3/4] app/testpmd: add rxhdrs commands and parameters Yuan Wang
2022-10-09 20:25   ` [PATCH v9 4/4] net/ice: support buffer split in Rx path Yuan Wang

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=20220926094039.1572741-3-yuanx.wang@intel.com \
    --to=yuanx.wang@intel.com \
    --cc=aman.deep.singh@intel.com \
    --cc=andrew.rybchenko@oktetlabs.ru \
    --cc=dev@dpdk.org \
    --cc=ferruh.yigit@xilinx.com \
    --cc=hpothula@marvell.com \
    --cc=jerinjacobk@gmail.com \
    --cc=mdr@ashroe.eu \
    --cc=qi.z.zhang@intel.com \
    --cc=qiming.yang@intel.com \
    --cc=stephen@networkplumber.org \
    --cc=thomas@monjalon.net \
    --cc=viacheslavo@nvidia.com \
    --cc=wenxuanx.wu@intel.com \
    --cc=xiaoyun.li@intel.com \
    --cc=xuan.ding@intel.com \
    --cc=yaqi.tang@intel.com \
    --cc=yuying.zhang@intel.com \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).