From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id A800F436AC; Fri, 8 Dec 2023 19:05:51 +0100 (CET) Received: from mails.dpdk.org (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 39ED043297; Fri, 8 Dec 2023 19:05:51 +0100 (CET) Received: from mail-pf1-f175.google.com (mail-pf1-f175.google.com [209.85.210.175]) by mails.dpdk.org (Postfix) with ESMTP id 2E2DA43001 for ; Fri, 8 Dec 2023 19:05:50 +0100 (CET) Received: by mail-pf1-f175.google.com with SMTP id d2e1a72fcca58-6ce76f0748fso1851718b3a.2 for ; Fri, 08 Dec 2023 10:05:50 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1702058749; x=1702663549; darn=dpdk.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=nbnNtkUupFuYANv6QGzCbTV76Be+LCTSlRBhWERvRlQ=; b=DCKe5k2WeBJjLZtxRFEQ0dZ53R9JX+m60PjWXFHLDv6acM5uOlXu09dwua9CJcnmtE k+rV7onAYSiQ4TQGHdJiyjlhs0J5Pq9I6P3APwMLFU6N/OY5HfMN7G81YPJSmDBxPVu0 CDia2b9+rdBHH8IDepjgJpY5mYqZvD36ETFkvKofh2AoM+zT6MydkGZuDPWkCjC9vbJ9 8qZhkuADmopO7jJ/qcHFSRuzBtw3yNk1aX7sbW5bJ1aiC4M6m2yu/6WjpjBCfMKUPaBj Z5QUuYbvbVYV4ZLjvTaijcHbGjFLRckZxqBLaBzPCIKMp9zhwGLKkSd6BXjOZfcPKN41 X5SQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1702058749; x=1702663549; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=nbnNtkUupFuYANv6QGzCbTV76Be+LCTSlRBhWERvRlQ=; b=EecI/jfvnXNbM01gL/XGRCkiZtNMXAh9ZW4GMC11nwe684WSBSSWzUEfzs0hCYTaMp +3drp4Ut3ZtAWS+z/LUYVc82omv8BqeBK54eFmgBaNt1CBojbs7AvTVgDX3TaNxAus2Y mTL3kbk3/05FaaPsYrXV15caXu+cSndMdEfv3C5MkGBGNm6/rjzleuz1OWX7wYwB8h79 luQ1aXWkgkyI3Sq5P8wLGD5PzMe/RxpCuAXl0g+wInW307lSbR5lVenElrn2rw3SWJ4G seYwPAMEqVVRpXaKiSwTaQVhgGgCsBeYgKs3muzdWWCKk1w1JbYt3JJoG9SNEUOWp0g+ TyjQ== X-Gm-Message-State: AOJu0Yz0v/UYbOtwnagYiXGGO9OFdw7IyYhDMhGkKueRYGHk0lq8LwKY raYao075bFKwAiJCiQxABs/elHp7Cks= X-Google-Smtp-Source: AGHT+IHlcrOxJRZBOxrl1gZn0DCdPR7ZKIn0lh3onjBsw4PgqI2oMD5sznPmljr4k4Q3DiGDdgv9+Q== X-Received: by 2002:a05:6a00:1894:b0:6ce:784a:579 with SMTP id x20-20020a056a00189400b006ce784a0579mr424230pfh.21.1702058748670; Fri, 08 Dec 2023 10:05:48 -0800 (PST) Received: from IDCREDBOX918.fareast.corp.microsoft.com ([167.220.238.132]) by smtp.gmail.com with ESMTPSA id k13-20020aa792cd000000b006ce273562fasm1865533pfa.40.2023.12.08.10.05.46 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 08 Dec 2023 10:05:48 -0800 (PST) From: Kumara Parameshwaran To: dev@dpdk.org Cc: hujiayu.hu@foxmail.com, Kumara Parameshwaran , Kumara Parameshwaran Subject: [PATCH v7] gro: fix reordering of packets in GRO layer Date: Fri, 8 Dec 2023 23:35:34 +0530 Message-Id: <20231208180534.23524-1-kumaraparamesh92@gmail.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20221101070557.58808-1-kumaraparmesh92@gmail.com> References: <20221101070557.58808-1-kumaraparmesh92@gmail.com> MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org In the current implementation when packets is received with special flag(s) set, only that packet is delivered out of order. There could be already coalesced packets in the GRO table beloging to the same flow but not handled. This fix makes sure that the entire segment is delivered with the special flag(s) set which is how the Linux GRO is also implemented Signed-off-by: Kumara Parameshwaran Co-authored-by: Kumara Parameshwaran --- If the received packet is not a pure ACK packet, we check if there are any previous packets in the flow, if present we indulge the received packet also in the coalescing logic and update the flags of the last recived packet to the entire segment which would avoid re-ordering. Lets say a case where P1(PSH), P2(ACK), P3(ACK) are received in burst mode, P1 contains PSH flag and since it does not contain any prior packets in the flow we copy it to unprocess_packets and P2(ACK) and P3(ACK) are merged together. In the existing case the P2,P3 would be delivered as single segment first and the unprocess_packets will be copied later which will cause reordering. With the patch copy the unprocess packets first and then the packets from the GRO table. Testing done The csum test-pmd was modified to support the following GET request of 10MB from client to server via test-pmd (static arp entries added in client and server). Enable GRO and TSO in test-pmd where the packets recived from the client mac would be sent to server mac and vice versa. In above testing, without the patch the client observerd re-ordering of 25 packets and with the patch there were no packet re-ordering observerd. v2: Fix warnings in commit and comment. Do not consider packet as candidate to merge if it contains SYN/RST flag. v3: Fix warnings. v4: Rebase with master. v5: Adding co-author email v6: Address review comments from the maintainer to restructure the code and handle only special flags PSH,FIN lib/gro/gro_tcp.h | 11 ++++++++ lib/gro/gro_tcp4.c | 65 +++++++++++++++++++++++++++++----------------- 2 files changed, 52 insertions(+), 24 deletions(-) diff --git a/lib/gro/gro_tcp.h b/lib/gro/gro_tcp.h index d926c4b8cc..137a03bc96 100644 --- a/lib/gro/gro_tcp.h +++ b/lib/gro/gro_tcp.h @@ -187,4 +187,15 @@ is_same_common_tcp_key(struct cmn_tcp_key *k1, struct cmn_tcp_key *k2) return (!memcmp(k1, k2, sizeof(struct cmn_tcp_key))); } +static inline void +update_tcp_hdr_flags(struct rte_tcp_hdr *tcp_hdr, struct rte_mbuf *pkt) +{ + struct rte_tcp_hdr *merged_tcp_hdr; + + merged_tcp_hdr = rte_pktmbuf_mtod_offset(pkt, struct rte_tcp_hdr *, pkt->l2_len + + pkt->l3_len); + merged_tcp_hdr->tcp_flags |= tcp_hdr->tcp_flags; + +} + #endif diff --git a/lib/gro/gro_tcp4.c b/lib/gro/gro_tcp4.c index 6645de592b..7a68615031 100644 --- a/lib/gro/gro_tcp4.c +++ b/lib/gro/gro_tcp4.c @@ -126,6 +126,7 @@ gro_tcp4_reassemble(struct rte_mbuf *pkt, uint32_t item_idx; uint32_t i, max_flow_num, remaining_flow_num; uint8_t find; + uint32_t item_start_idx; /* * Don't process the packet whose TCP header length is greater @@ -139,13 +140,6 @@ gro_tcp4_reassemble(struct rte_mbuf *pkt, tcp_hdr = (struct rte_tcp_hdr *)((char *)ipv4_hdr + pkt->l3_len); hdr_len = pkt->l2_len + pkt->l3_len + pkt->l4_len; - /* - * Don't process the packet which has FIN, SYN, RST, PSH, URG, ECE - * or CWR set. - */ - if (tcp_hdr->tcp_flags != RTE_TCP_ACK_FLAG) - return -1; - /* trim the tail padding bytes */ ip_tlen = rte_be_to_cpu_16(ipv4_hdr->total_length); if (pkt->pkt_len > (uint32_t)(ip_tlen + pkt->l2_len)) @@ -183,6 +177,7 @@ gro_tcp4_reassemble(struct rte_mbuf *pkt, if (tbl->flows[i].start_index != INVALID_ARRAY_INDEX) { if (is_same_tcp4_flow(tbl->flows[i].key, key)) { find = 1; + item_start_idx = tbl->flows[i].start_index; break; } remaining_flow_num--; @@ -190,28 +185,50 @@ gro_tcp4_reassemble(struct rte_mbuf *pkt, } if (find == 0) { - sent_seq = rte_be_to_cpu_32(tcp_hdr->sent_seq); - item_idx = insert_new_tcp_item(pkt, tbl->items, &tbl->item_num, - tbl->max_item_num, start_time, - INVALID_ARRAY_INDEX, sent_seq, ip_id, - is_atomic); - if (item_idx == INVALID_ARRAY_INDEX) + /* + * Add new flow to the table only if contains ACK flag with data. + * Do not add any packets with additional tcp flags to the GRO table + */ + if (tcp_hdr->tcp_flags == RTE_TCP_ACK_FLAG) { + sent_seq = rte_be_to_cpu_32(tcp_hdr->sent_seq); + item_idx = insert_new_tcp_item(pkt, tbl->items, &tbl->item_num, + tbl->max_item_num, start_time, + INVALID_ARRAY_INDEX, sent_seq, ip_id, + is_atomic); + if (item_idx == INVALID_ARRAY_INDEX) + return -1; + if (insert_new_flow(tbl, &key, item_idx) == + INVALID_ARRAY_INDEX) { + /* + * Fail to insert a new flow, so delete the + * stored packet. + */ + delete_tcp_item(tbl->items, item_idx, &tbl->item_num, INVALID_ARRAY_INDEX); + return -1; + } + return 0; + } else { return -1; - if (insert_new_flow(tbl, &key, item_idx) == - INVALID_ARRAY_INDEX) { - /* - * Fail to insert a new flow, so delete the - * stored packet. - */ - delete_tcp_item(tbl->items, item_idx, &tbl->item_num, INVALID_ARRAY_INDEX); + } + } else { + /* + * Any packet with additional flags like PSH,FIN should be processed + * and flushed immediately. + * Hence marking the start time to 0, so that the packets will be flushed + * immediately in timer mode. + */ + if (tcp_hdr->tcp_flags & (RTE_TCP_ACK_FLAG|RTE_TCP_PSH_FLAG|RTE_TCP_FIN_FLAG)) { + if (tcp_hdr->tcp_flags != RTE_TCP_ACK_FLAG) + tbl->items[item_start_idx].start_time = 0; + return process_tcp_item(pkt, tcp_hdr, tcp_dl, tbl->items, tbl->flows[i].start_index, + &tbl->item_num, tbl->max_item_num, + ip_id, is_atomic, start_time); + } else { return -1; } - return 0; } - return process_tcp_item(pkt, tcp_hdr, tcp_dl, tbl->items, tbl->flows[i].start_index, - &tbl->item_num, tbl->max_item_num, - ip_id, is_atomic, start_time); + return -1; } /* -- 2.34.1