DPDK patches and discussions
 help / color / mirror / Atom feed
From: Kumara Parameshwaran <kumaraparamesh92@gmail.com>
To: hujiayu.hu@foxmail.com
Cc: dev@dpdk.org, Kumara Parameshwaran <kumaraparamesh92@gmail.com>
Subject: [PATCH v12] gro: fix reordering of packets in GRO layer
Date: Mon,  8 Jan 2024 21:20:06 +0530	[thread overview]
Message-ID: <20240108155006.722195-1-kumaraparamesh92@gmail.com> (raw)
In-Reply-To: <20231208181738.23931-1-kumaraparamesh92@gmail.com>

In the current implementation when a packet is received with
special TCP flag(s) set, only that packet is delivered out of order.
There could be already coalesced packets in the GRO table
belonging to the same flow but not delivered.
This fix makes sure that the entire segment is delivered with the
special flag(s) set which is how the Linux GRO is also implemented

Signed-off-by: Kumara Parameshwaran <kumaraparamesh92@gmail.com>
---
	If the received packet is not a pure ACK packet, we check if
	there are any previous packets in the flow, if present we indulge
	the received packet also in the coalescing logic and update the flags
	of the last recived packet to the entire segment which would avoid
	re-ordering.

	Lets say a case where P1(PSH), P2(ACK), P3(ACK)  are received in burst mode,
	P1 contains PSH flag and since it does not contain any prior packets in the flow
	we copy it to unprocess_packets and P2(ACK) and P3(ACK) are merged together.
	In the existing case the  P2,P3 would be delivered as single segment first and the
	unprocess_packets will be copied later which will cause reordering. With the patch
	copy the unprocess packets first and then the packets from the GRO table.

	Testing done
	The csum test-pmd was modified to support the following
	GET request of 10MB from client to server via test-pmd (static arp entries added in client
	and server). Enable GRO and TSO in test-pmd where the packets recived from the client mac
	would be sent to server mac and vice versa.
	In above testing, without the patch the client observerd re-ordering of 25 packets
	and with the patch there were no packet re-ordering observerd.

v2: 
	Fix warnings in commit and comment.
	Do not consider packet as candidate to merge if it contains SYN/RST flag.

v3:
	Fix warnings.

v4:
	Rebase with master.

v5:
	Adding co-author email
v6:
	Address review comments from the maintainer to restructure the code 
	and handle only special flags PSH,FIN

v7:
	Fix warnings and errors

v8:
	Fix warnings and errors

v9:
	Fix commit message 

v10:
    Update tcp header flags and address review comments 

v11: 
    Fix warnings

v12:
    Fix nit review comments

 lib/gro/gro_tcp.h          |  9 +++++++
 lib/gro/gro_tcp4.c         | 48 ++++++++++++++++++++++++--------------
 lib/gro/gro_tcp_internal.h |  2 +-
 lib/gro/gro_vxlan_tcp4.c   |  5 ++--
 4 files changed, 44 insertions(+), 20 deletions(-)

diff --git a/lib/gro/gro_tcp.h b/lib/gro/gro_tcp.h
index d926c4b8cc..2c68b5f23e 100644
--- a/lib/gro/gro_tcp.h
+++ b/lib/gro/gro_tcp.h
@@ -19,6 +19,8 @@
 #define INVALID_TCP_HDRLEN(len) \
 	(((len) < sizeof(struct rte_tcp_hdr)) || ((len) > MAX_TCP_HLEN))
 
+#define VALID_GRO_TCP_FLAGS (RTE_TCP_ACK_FLAG | RTE_TCP_PSH_FLAG | RTE_TCP_FIN_FLAG)
+
 struct cmn_tcp_key {
 	struct rte_ether_addr eth_saddr;
 	struct rte_ether_addr eth_daddr;
@@ -81,11 +83,13 @@ merge_two_tcp_packets(struct gro_tcp_item *item,
 		struct rte_mbuf *pkt,
 		int cmp,
 		uint32_t sent_seq,
+		uint8_t tcp_flags,
 		uint16_t ip_id,
 		uint16_t l2_offset)
 {
 	struct rte_mbuf *pkt_head, *pkt_tail, *lastseg;
 	uint16_t hdr_len, l2_len;
+	struct rte_tcp_hdr *tcp_hdr;
 
 	if (cmp > 0) {
 		pkt_head = item->firstseg;
@@ -128,6 +132,11 @@ merge_two_tcp_packets(struct gro_tcp_item *item,
 	/* update MBUF metadata for the merged packet */
 	pkt_head->nb_segs += pkt_tail->nb_segs;
 	pkt_head->pkt_len += pkt_tail->pkt_len;
+	if (tcp_flags != RTE_TCP_ACK_FLAG) {
+		tcp_hdr = rte_pktmbuf_mtod_offset(pkt, struct rte_tcp_hdr *,
+						l2_offset + pkt_head->l2_len + pkt_head->l3_len);
+		tcp_hdr->tcp_flags |= tcp_flags;
+	}
 
 	return 1;
 }
diff --git a/lib/gro/gro_tcp4.c b/lib/gro/gro_tcp4.c
index 6645de592b..ad9cf04dbe 100644
--- a/lib/gro/gro_tcp4.c
+++ b/lib/gro/gro_tcp4.c
@@ -126,6 +126,7 @@ gro_tcp4_reassemble(struct rte_mbuf *pkt,
 	uint32_t item_idx;
 	uint32_t i, max_flow_num, remaining_flow_num;
 	uint8_t find;
+	uint32_t item_start_idx;
 
 	/*
 	 * Don't process the packet whose TCP header length is greater
@@ -139,11 +140,8 @@ gro_tcp4_reassemble(struct rte_mbuf *pkt,
 	tcp_hdr = (struct rte_tcp_hdr *)((char *)ipv4_hdr + pkt->l3_len);
 	hdr_len = pkt->l2_len + pkt->l3_len + pkt->l4_len;
 
-	/*
-	 * Don't process the packet which has FIN, SYN, RST, PSH, URG, ECE
-	 * or CWR set.
-	 */
-	if (tcp_hdr->tcp_flags != RTE_TCP_ACK_FLAG)
+	/* Return early if the TCP flags are not handled in GRO layer */
+	if (tcp_hdr->tcp_flags & ~VALID_GRO_TCP_FLAGS)
 		return -1;
 
 	/* trim the tail padding bytes */
@@ -183,25 +181,43 @@ gro_tcp4_reassemble(struct rte_mbuf *pkt,
 		if (tbl->flows[i].start_index != INVALID_ARRAY_INDEX) {
 			if (is_same_tcp4_flow(tbl->flows[i].key, key)) {
 				find = 1;
+				item_start_idx = tbl->flows[i].start_index;
 				break;
 			}
 			remaining_flow_num--;
 		}
 	}
 
-	if (find == 0) {
+	if (find == 1) {
+		/*
+		 * Any packet with additional flags like PSH,FIN should be processed
+		 * and flushed immediately.
+		 * Hence marking the start time to 0, so that the packets will be flushed
+		 * immediately in timer mode.
+		 */
+		if (tcp_hdr->tcp_flags & (RTE_TCP_ACK_FLAG | RTE_TCP_PSH_FLAG | RTE_TCP_FIN_FLAG)) {
+			if (tcp_hdr->tcp_flags != RTE_TCP_ACK_FLAG)
+				tbl->items[item_start_idx].start_time = 0;
+			return process_tcp_item(pkt, tcp_hdr, tcp_dl, tbl->items, tbl->flows[i].start_index,
+						&tbl->item_num, tbl->max_item_num, ip_id, is_atomic, start_time);
+		} else {
+			return -1;
+		}
+	}
+	/*
+	 * Add new flow to the table only if contains ACK flag with data.
+	 * Do not add any packets with additional tcp flags to the GRO table
+	 */
+	if (tcp_hdr->tcp_flags == RTE_TCP_ACK_FLAG) {
 		sent_seq = rte_be_to_cpu_32(tcp_hdr->sent_seq);
-		item_idx = insert_new_tcp_item(pkt, tbl->items, &tbl->item_num,
-						tbl->max_item_num, start_time,
-						INVALID_ARRAY_INDEX, sent_seq, ip_id,
-						is_atomic);
+		item_idx = insert_new_tcp_item(pkt, tbl->items, &tbl->item_num, tbl->max_item_num,
+						start_time, INVALID_ARRAY_INDEX, sent_seq, ip_id, is_atomic);
 		if (item_idx == INVALID_ARRAY_INDEX)
 			return -1;
-		if (insert_new_flow(tbl, &key, item_idx) ==
-			INVALID_ARRAY_INDEX) {
+		if (insert_new_flow(tbl, &key, item_idx) == INVALID_ARRAY_INDEX) {
 			/*
-			 * Fail to insert a new flow, so delete the
-			 * stored packet.
+			* Fail to insert a new flow, so delete the
+			* stored packet.
 			*/
 			delete_tcp_item(tbl->items, item_idx, &tbl->item_num, INVALID_ARRAY_INDEX);
 			return -1;
@@ -209,9 +225,7 @@ gro_tcp4_reassemble(struct rte_mbuf *pkt,
 		return 0;
 	}
 
-	return process_tcp_item(pkt, tcp_hdr, tcp_dl, tbl->items, tbl->flows[i].start_index,
-						&tbl->item_num, tbl->max_item_num,
-						ip_id, is_atomic, start_time);
+	return -1;
 }
 
 /*
diff --git a/lib/gro/gro_tcp_internal.h b/lib/gro/gro_tcp_internal.h
index cc84abeaeb..e4855da1ad 100644
--- a/lib/gro/gro_tcp_internal.h
+++ b/lib/gro/gro_tcp_internal.h
@@ -101,7 +101,7 @@ process_tcp_item(struct rte_mbuf *pkt,
 				is_atomic);
 		if (cmp) {
 			if (merge_two_tcp_packets(&items[cur_idx],
-						pkt, cmp, sent_seq, ip_id, 0))
+						pkt, cmp, sent_seq, tcp_hdr->tcp_flags, ip_id, 0))
 				return 1;
 			/*
 			 * Fail to merge the two packets, as the packet
diff --git a/lib/gro/gro_vxlan_tcp4.c b/lib/gro/gro_vxlan_tcp4.c
index 6ab7001922..8dd62a949c 100644
--- a/lib/gro/gro_vxlan_tcp4.c
+++ b/lib/gro/gro_vxlan_tcp4.c
@@ -239,10 +239,11 @@ merge_two_vxlan_tcp4_packets(struct gro_vxlan_tcp4_item *item,
 		struct rte_mbuf *pkt,
 		int cmp,
 		uint32_t sent_seq,
+		uint8_t tcp_flags,
 		uint16_t outer_ip_id,
 		uint16_t ip_id)
 {
-	if (merge_two_tcp_packets(&item->inner_item, pkt, cmp, sent_seq,
+	if (merge_two_tcp_packets(&item->inner_item, pkt, cmp, sent_seq, tcp_flags,
 				ip_id, pkt->outer_l2_len +
 				pkt->outer_l3_len)) {
 		/* Update the outer IPv4 ID to the large value. */
@@ -413,7 +414,7 @@ gro_vxlan_tcp4_reassemble(struct rte_mbuf *pkt,
 				tcp_dl, outer_is_atomic, is_atomic);
 		if (cmp) {
 			if (merge_two_vxlan_tcp4_packets(&(tbl->items[cur_idx]),
-						pkt, cmp, sent_seq,
+						pkt, cmp, sent_seq, tcp_hdr->tcp_flags,
 						outer_ip_id, ip_id))
 				return 1;
 			/*
-- 
2.25.1


  parent reply	other threads:[~2024-01-08 15:50 UTC|newest]

Thread overview: 23+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2022-10-13 10:18 [PATCH] gro : fix reordering of packets in GRO library Kumara Parameshwaran
2022-10-13 10:20 ` kumaraparameshwaran rathinavel
2022-10-28  8:09 ` [PATCH v2] " Kumara Parameshwaran
2022-10-28  8:27 ` [PATCH v3] " Kumara Parameshwaran
2022-10-28  9:51 ` [PATCH v4] " Kumara Parameshwaran
2022-11-01  7:05   ` [PATCH v5] " Kumara Parameshwaran
2023-06-19 13:25     ` Thomas Monjalon
2023-06-20  7:35     ` Hu, Jiayu
2023-06-21  8:47       ` kumaraparameshwaran rathinavel
2023-06-30 11:32       ` kumaraparameshwaran rathinavel
2023-12-08 17:54     ` [PATCH v6] gro: fix reordering of packets in GRO layer Kumara Parameshwaran
2023-12-08 18:05     ` [PATCH v7] " Kumara Parameshwaran
2023-12-08 18:12     ` [PATCH v8] " Kumara Parameshwaran
2023-12-08 18:17     ` [PATCH v9] " Kumara Parameshwaran
2024-01-04 15:49       ` 胡嘉瑜
2024-01-07 11:21       ` [PATCH v10] " Kumara Parameshwaran
2024-01-07 11:29       ` [PATCH v11] " Kumara Parameshwaran
2024-01-07 17:20         ` Stephen Hemminger
2024-01-08 16:11           ` kumaraparameshwaran rathinavel
2024-01-08 15:50       ` Kumara Parameshwaran [this message]
2024-01-08 16:04       ` [PATCH v13] " Kumara Parameshwaran
2024-01-16 14:28         ` 胡嘉瑜
2024-02-12 14:30           ` Thomas Monjalon

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=20240108155006.722195-1-kumaraparamesh92@gmail.com \
    --to=kumaraparamesh92@gmail.com \
    --cc=dev@dpdk.org \
    --cc=hujiayu.hu@foxmail.com \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).