From mboxrd@z Thu Jan  1 00:00:00 1970
Return-Path: <dev-bounces@dpdk.org>
Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124])
	by inbox.dpdk.org (Postfix) with ESMTP id 8D49942D04;
	Tue, 20 Jun 2023 09:36:24 +0200 (CEST)
Received: from mails.dpdk.org (localhost [127.0.0.1])
	by mails.dpdk.org (Postfix) with ESMTP id 1A9F64113F;
	Tue, 20 Jun 2023 09:36:24 +0200 (CEST)
Received: from mga18.intel.com (mga18.intel.com [134.134.136.126])
 by mails.dpdk.org (Postfix) with ESMTP id 8A56F4068E
 for <dev@dpdk.org>; Tue, 20 Jun 2023 09:36:21 +0200 (CEST)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple;
 d=intel.com; i=@intel.com; q=dns/txt; s=Intel;
 t=1687246581; x=1718782581;
 h=from:to:cc:subject:date:message-id:references:
 in-reply-to:content-transfer-encoding:mime-version;
 bh=w4Up1ungwtZnOu0RN9qjC7QRmbG4PE08LBiShfPHbTA=;
 b=DVdO01QO3+2rfiSKR4JtY9EkqUHeYXcbPpQHzjjf27YhnHTUnP1t+3vG
 eyRH5KYnJWfAFMjnXelXA6VcioZOW6pZPsybaffQUB1nNSIXlEOaDT8fP
 ttFMjmAsx24vQ6JxF2BohdTBq4CompLONjboDRG0+bcYGTKOFo0aKXksr
 /+2SNg7vJVmYv+Ob1Hg+CqmIdYytZtcF1DIg56HuT5+sbFMM/cCOf4wRp
 oH4J+5wpxbL1ZfTK5WHxwFamWccRHA5PweDGG5SKcMIhZ4CkPcklyVJxp
 ubzHKsk7zSfig6luV5ZESBlyBRBJK+2jnCnv7HiHb4bghf31kjRrTMs4J Q==;
X-IronPort-AV: E=McAfee;i="6600,9927,10746"; a="344534918"
X-IronPort-AV: E=Sophos;i="6.00,256,1681196400"; d="scan'208";a="344534918"
Received: from orsmga007.jf.intel.com ([10.7.209.58])
 by orsmga106.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384;
 20 Jun 2023 00:35:59 -0700
X-ExtLoop1: 1
X-IronPort-AV: E=McAfee;i="6600,9927,10746"; a="708156329"
X-IronPort-AV: E=Sophos;i="6.00,256,1681196400"; d="scan'208";a="708156329"
Received: from orsmsx603.amr.corp.intel.com ([10.22.229.16])
 by orsmga007.jf.intel.com with ESMTP; 20 Jun 2023 00:35:59 -0700
Received: from orsmsx611.amr.corp.intel.com (10.22.229.24) by
 ORSMSX603.amr.corp.intel.com (10.22.229.16) with Microsoft SMTP Server
 (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id
 15.1.2507.23; Tue, 20 Jun 2023 00:35:59 -0700
Received: from orsmsx611.amr.corp.intel.com (10.22.229.24) by
 ORSMSX611.amr.corp.intel.com (10.22.229.24) with Microsoft SMTP Server
 (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id
 15.1.2507.23; Tue, 20 Jun 2023 00:35:57 -0700
Received: from orsedg603.ED.cps.intel.com (10.7.248.4) by
 orsmsx611.amr.corp.intel.com (10.22.229.24) with Microsoft SMTP Server
 (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id
 15.1.2507.23 via Frontend Transport; Tue, 20 Jun 2023 00:35:57 -0700
Received: from NAM11-DM6-obe.outbound.protection.outlook.com (104.47.57.168)
 by edgegateway.intel.com (134.134.137.100) with Microsoft SMTP Server
 (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.1.2507.23; Tue, 20 Jun 2023 00:35:57 -0700
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=Roy+BdLHZC3/jVT0NNHklxBFziefW1XbimHZmQS59YxwUcqYpez0GUV482D6Zwoupdy5C5zTzApDYcfhEBJFUKl1ea4+DR81Ud9JRAw5/lgEAL/RPMP/6YyP/vNpPrfqZYpal7pf2eM1leBFSxOh1/9LbYs8Kg8fHfkSAmluttffXmU29CG9GGJ/bskr7OMQPhVud6QrGtgMsEg3tH6VOX0zMVc2iadWX76Qdfqyt0wIhbDknVDtWpN1x5+gV0MyPgQQSgkP1xqbP/jcmf8aUEJlbH5c69inRMHXhEcCKtgGFKZwfnK1y8myUncalsj6JnGlwYC3C7uDYzo6xFRqLQ==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; 
 s=arcselector9901;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1;
 bh=op5gnQa1QTsem7DNxH/xUCOwwlt7LRNAk+bseOeVdEs=;
 b=WGL7fD6aphEOLbJQ8k8vCjHEnyuvESazMDSF40MNjcn9NdrWzaVAo1vQLC2a7wlluteGOoBxP+FffgpYy3QzB5WkIHjSnWdPrjUshbRLQyn6x1dUSIT7FmI8gayI8+WNtD40RI4gLMraNz9KTzDdiu+7WiM58V7Q+RaEdzDOATeO4RVtYmrfwXchcVlovA0GCSQqMEoQBPkKsvVZPvpXEEY/J5hCSSE3wVh+sIesednTqCzrYzKdAwBzmgXmfw1QfUKLT81y7ACA72jF1rS1az9GR2fw+nFj3Kndh0YLB0KqPeeA03bfZdkzVvUbQEO4PbDJDSuo12CAoFuZzA6jxA==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=intel.com; dmarc=pass action=none header.from=intel.com;
 dkim=pass header.d=intel.com; arc=none
Received: from DS0PR11MB6494.namprd11.prod.outlook.com (2603:10b6:8:c2::14) by
 IA0PR11MB8334.namprd11.prod.outlook.com (2603:10b6:208:483::16) with
 Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.6500.29; Tue, 20 Jun
 2023 07:35:50 +0000
Received: from DS0PR11MB6494.namprd11.prod.outlook.com
 ([fe80::bbd0:7628:1b7c:78b0]) by DS0PR11MB6494.namprd11.prod.outlook.com
 ([fe80::bbd0:7628:1b7c:78b0%4]) with mapi id 15.20.6500.036; Tue, 20 Jun 2023
 07:35:49 +0000
From: "Hu, Jiayu" <jiayu.hu@intel.com>
To: Kumara Parameshwaran <kumaraparamesh92@gmail.com>
CC: "dev@dpdk.org" <dev@dpdk.org>, Kumara Parameshwaran
 <kparameshwar@vmware.com>, "thomas@monjalon.net" <thomas@monjalon.net>
Subject: RE: [PATCH v5] gro : fix reordering of packets in GRO library
Thread-Topic: [PATCH v5] gro : fix reordering of packets in GRO library
Thread-Index: AQHY7cBuFq3EALEykUaAHR2RKMn3DK+Uod4Q
Date: Tue, 20 Jun 2023 07:35:49 +0000
Message-ID: <DS0PR11MB6494520A13C8C1132DA9E6ED925CA@DS0PR11MB6494.namprd11.prod.outlook.com>
References: <20221028095114.17210-1-kumaraparmesh92@gmail.com>
 <20221101070557.58808-1-kumaraparmesh92@gmail.com>
In-Reply-To: <20221101070557.58808-1-kumaraparmesh92@gmail.com>
Accept-Language: en-US
Content-Language: en-US
X-MS-Has-Attach: 
X-MS-TNEF-Correlator: 
authentication-results: dkim=none (message not signed)
 header.d=none;dmarc=none action=none header.from=intel.com;
x-ms-publictraffictype: Email
x-ms-traffictypediagnostic: DS0PR11MB6494:EE_|IA0PR11MB8334:EE_
x-ms-office365-filtering-correlation-id: 12f18147-7058-4860-44d7-08db7160f7c9
x-ms-exchange-senderadcheck: 1
x-ms-exchange-antispam-relay: 0
x-microsoft-antispam: BCL:0;
x-microsoft-antispam-message-info: oUgKPHRg34bLl7Be6E4HtUfwntQmhEvfRmsYok/6M8YpKTEzii3wXJl+pPShiyX0cQoerEC1fbJDe7CiG417ipfg5V5o9WMgldSwtMoNz1d/YQzG42EYU129z/oE41AH+cEMGlaF7QUAQwnLFfW5UtYZSTgHOZ61Yc3V+4+1MnnUQ2pNGOdIRVAhzrWLtUIzkvA+RwzFa++Lf8Wi+ZS4EFDHxad67C9CztojieuOn8a4gJXoDdYxt65N5A1Jx5Dq1tFjKdk8MMoAds9VwXeoLa7wpjwDhWDqTHJnC+A1iwbdGcAeQwFlvCBF7FVCqbyDsazeqTISbeIAjoxbKlF71vE+zN0NZAhS1ouQs6FpnwsisNvtVaCfFpG5pNiYfZixoGmbzuKu/Kwe0icCSg1MG6Xdkdm1XvYGP4PYMJnjNIgYse3qdMX+wburLFNv69ihFZ49gWV61XdMptxzaSJUTHvdKboxj/Mjeqv4/y5J+4p187i5BQrpCSbY1e1WI+1jJXpSyfvjVliUINtihbB152hYvRdrify0nhUkLV4VB8KxTU4le1lk8rXuGD6tXPOO6D+BeWq8YmdK/evuJQ+jRYtkHMSFRRXlPjYoGOadPrm6ba6YmcrHgvvB2Lhxi3v2
x-forefront-antispam-report: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:;
 IPV:NLI; SFV:NSPM; H:DS0PR11MB6494.namprd11.prod.outlook.com; PTR:; CAT:NONE;
 SFS:(13230028)(396003)(136003)(366004)(346002)(39860400002)(376002)(451199021)(2906002)(64756008)(41300700001)(8676002)(55016003)(52536014)(66946007)(316002)(4326008)(5660300002)(8936002)(6916009)(66446008)(66556008)(76116006)(66476007)(33656002)(186003)(83380400001)(71200400001)(9686003)(6506007)(53546011)(26005)(38070700005)(82960400001)(122000001)(86362001)(478600001)(38100700002)(54906003)(7696005);
 DIR:OUT; SFP:1102; 
x-ms-exchange-antispam-messagedata-chunkcount: 1
x-ms-exchange-antispam-messagedata-0: =?us-ascii?Q?iBLFUSQW6n2pTL9txoy+B5egEQRrLpfH/tjpTrzhgcS+a1cpJ9fTvxvoTUOk?=
 =?us-ascii?Q?04umlRPXxl8wE0Zhw9oq1x2qLrQnlM0zuPL9JqPEoYPi0AXUI26oCPBLkeLx?=
 =?us-ascii?Q?Fx4CYbs0JRGhP8eectNTKnxi/7mAa5RXHA0q/j52e60wl0UWHfkU5IPVkef7?=
 =?us-ascii?Q?sTG4kcTdfbC8uWF+V0eM4YRdTslUJMZX6yb0hpK+6D35YanupsoQ7CFm6yzQ?=
 =?us-ascii?Q?rzoMQ2N/BpHPOvTVbctFZQLvwPGLmbG4W07AKZrdJUZMsEsObnBFE1Tgs7Sz?=
 =?us-ascii?Q?9rT82NPvaEDqOXYXoMLn5jGp+lSOnO88PIOPJIJiiBObd+84IWF3Z5hnOSbj?=
 =?us-ascii?Q?mYWePQs7jcHyerP15gHgj91MbYuZ/K9ajx5KmGSga+uKutMgVl0UVpbzoZFr?=
 =?us-ascii?Q?guMQfpigZzSKM1pDXxwQoTtduchSW/E7dJaKa0q5w8cLUH9tQVXoiFmhabkC?=
 =?us-ascii?Q?IbMhFkZNlAhVHToCdellfqa3oRyJDe46E/W4ZqEDB6JZ525x30IL9hyqJdA5?=
 =?us-ascii?Q?dqVb0weq/2/yD53yRWCyUncv1jHFwhZHF9s4C/hfntCVohAuHUEOMxulje5i?=
 =?us-ascii?Q?u0Oq39ETw63stQgiUQwzJkI0qBg7HRldYd98UsIbWRxlCPsEGT7GVj4glAjl?=
 =?us-ascii?Q?9fuEIn20SHiXOYmhQUtJluZ/3xckbYWTnjagR1P+hd7wZcxox202FVkl/rYa?=
 =?us-ascii?Q?EryTZTpSmMEdb6MMQR02yFJWPBECAoU+JR7XuHf6gNfIjBNhDyG9Y8loWquI?=
 =?us-ascii?Q?q5PuWSHKqwI047UrHfL25UcID2Xb57RQnglqqdpedjC57r86xWG5tw6AxBIH?=
 =?us-ascii?Q?95cUDgg7qjW6oGhqckNvl9J1hmtTLORm5dANyZ12nCFF3kUdooGkXXNf4f1g?=
 =?us-ascii?Q?Quj91yJSD5eikn5ckHPqzw4l7lVrHdAbQ5JvHnKTbsLgh6kLBVwa9+tcbTfq?=
 =?us-ascii?Q?/QOtKM8xbpFmj1uhd51ZvNt7Y9wMtL6a0AJBNu0yZSVrG75p5+DPvyuTuAR/?=
 =?us-ascii?Q?0ClQJ6/wBpcedr2Fg20JGRYbXAx8VVSWZQquQZ9bXx8QBuxJqAZvcZWIoMnp?=
 =?us-ascii?Q?mJkAAOKZ81WAGBJ37n6qOSf5LdzzaHnUq5WI0ulG+5oMCKZggdrJfdqmBmJe?=
 =?us-ascii?Q?SAr2Rfhr+uioAnj695B2L4bg8YcpkKJt+LyM6bFXXZb3DZOIKSwnrmMG3CKX?=
 =?us-ascii?Q?wwREzZ0D0CJxOba1Fn0ieDE/3FxQJj/H7/s1HH2tTBwmV+gv9rZQFTzF2ibl?=
 =?us-ascii?Q?7/WV0LuqlE4O6iz0v2s6c1vhT+GovHBB9KB63Fd4ubP30fkUGEGxsUknP7C+?=
 =?us-ascii?Q?WiEBIhD6B+6yY/FOtoKyjTSCSWUOg/F4gH9OB5yu4KMGOngWyXQMGHGdKdBZ?=
 =?us-ascii?Q?H7Dp/xYf/OYIII+kxttPhZ5SalZFP1TFX6/86c/VlfM/FceuzWiLzl4s13vd?=
 =?us-ascii?Q?0UYmrd4nbEfmdRLNT0TYNsTPPAS5lqDj41Zk+G2CafzFr3YEo2O4VPx4RP42?=
 =?us-ascii?Q?15q6eqrSRRmVKdMioY458EaMdRgVBRogAYVTvSxu8Mbn5o9Tcy7eI2U/ot0Y?=
 =?us-ascii?Q?mdNFj3ZSoTY1yRjId78f+8DSHaG2YYssF/Ri5v5l?=
Content-Type: text/plain; charset="us-ascii"
Content-Transfer-Encoding: quoted-printable
MIME-Version: 1.0
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-AuthSource: DS0PR11MB6494.namprd11.prod.outlook.com
X-MS-Exchange-CrossTenant-Network-Message-Id: 12f18147-7058-4860-44d7-08db7160f7c9
X-MS-Exchange-CrossTenant-originalarrivaltime: 20 Jun 2023 07:35:49.0534 (UTC)
X-MS-Exchange-CrossTenant-fromentityheader: Hosted
X-MS-Exchange-CrossTenant-id: 46c98d88-e344-4ed4-8496-4ed7712e255d
X-MS-Exchange-CrossTenant-mailboxtype: HOSTED
X-MS-Exchange-CrossTenant-userprincipalname: bZ13RPwX3GLZyNFsk3otgGJNMoGOmoTFWZigxZjzqdntTpWxT6OwBboGh3nEUZNYeYyYo6DBP737UBlUgoNufA==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: IA0PR11MB8334
X-OriginatorOrg: intel.com
X-BeenThere: dev@dpdk.org
X-Mailman-Version: 2.1.29
Precedence: list
List-Id: DPDK patches and discussions <dev.dpdk.org>
List-Unsubscribe: <https://mails.dpdk.org/options/dev>,
 <mailto:dev-request@dpdk.org?subject=unsubscribe>
List-Archive: <http://mails.dpdk.org/archives/dev/>
List-Post: <mailto:dev@dpdk.org>
List-Help: <mailto:dev-request@dpdk.org?subject=help>
List-Subscribe: <https://mails.dpdk.org/listinfo/dev>,
 <mailto:dev-request@dpdk.org?subject=subscribe>
Errors-To: dev-bounces@dpdk.org

Hi Kumara,

Please see replies inline.

Thanks,
Jiayu

> -----Original Message-----
> From: Kumara Parameshwaran <kumaraparamesh92@gmail.com>
> Sent: Tuesday, November 1, 2022 3:06 PM
> To: Hu, Jiayu <jiayu.hu@intel.com>
> Cc: dev@dpdk.org; Kumara Parameshwaran
> <kumaraparamesh92@gmail.com>; Kumara Parameshwaran
> <kparameshwar@vmware.com>
> Subject: [PATCH v5] gro : fix reordering of packets in GRO library
>=20
> From: Kumara Parameshwaran <kumaraparamesh92@gmail.com>
>=20
> When a TCP packet contains flags like PSH it is returned immediately to t=
he
> application though there might be packets of the same flow in the GRO tab=
le.
> If PSH flag is set on a segment packets up to the segment should be deliv=
ered
> immediately. But the current implementation delivers the last arrived pac=
ket
> with PSH flag set causing re-ordering
>=20
> With this patch, if a packet does not contain only ACK flag and if there =
are no
> previous packets for the flow the packet would be returned immediately,
> else will be merged with the previous segment and the flag on the last
> segment will be set on the entire segment.
> This is the behaviour with linux stack as well.
>=20
> Signed-off-by: Kumara Parameshwaran <kumaraparamesh92@gmail.com>
> Co-authored-by: Kumara Parameshwaran <kparameshwar@vmware.com>
> ---
> v1:
> 	If the received packet is not a pure ACK packet, we check if
> 	there are any previous packets in the flow, if present we indulge
> 	the received packet also in the coalescing logic and update the flags
> 	of the last recived packet to the entire segment which would avoid
> 	re-ordering.
>=20
> 	Lets say a case where P1(PSH), P2(ACK), P3(ACK)  are received in
> burst mode,
> 	P1 contains PSH flag and since it does not contain any prior packets in
> the flow
> 	we copy it to unprocess_packets and P2(ACK) and P3(ACK) are
> merged together.
> 	In the existing case the  P2,P3 would be delivered as single segment
> first and the
> 	unprocess_packets will be copied later which will cause reordering.
> With the patch
> 	copy the unprocess packets first and then the packets from the GRO
> table.
>=20
> 	Testing done
> 	The csum test-pmd was modifited to support the following
> 	GET request of 10MB from client to server via test-pmd (static arp
> entries added in client
> 	and server). Enable GRO and TSO in test-pmd where the packets
> recived from the client mac
> 	would be sent to server mac and vice versa.
> 	In above testing, without the patch the client observerd re-ordering
> of 25 packets
> 	and with the patch there were no packet re-ordering observerd.
>=20
> v2:
> 	Fix warnings in commit and comment.
> 	Do not consider packet as candidate to merge if it contains SYN/RST
> flag.
>=20
> v3:
> 	Fix warnings.
>=20
> v4:
> 	Rebase with master.
>=20
> v5:
> 	Adding co-author email
>=20
>  lib/gro/gro_tcp4.c | 45 +++++++++++++++++++++++++++++++++++++--------
>  lib/gro/rte_gro.c  | 18 +++++++++---------
>  2 files changed, 46 insertions(+), 17 deletions(-)
>=20
> diff --git a/lib/gro/gro_tcp4.c b/lib/gro/gro_tcp4.c index
> 0014096e63..7363c5d540 100644
> --- a/lib/gro/gro_tcp4.c
> +++ b/lib/gro/gro_tcp4.c
> @@ -188,6 +188,19 @@ update_header(struct gro_tcp4_item *item)
>  			pkt->l2_len);
>  }
>=20
> +static inline void
> +update_tcp_hdr_flags(struct rte_tcp_hdr *tcp_hdr, struct rte_mbuf *pkt)
> +{
> +	struct rte_ether_hdr *eth_hdr;
> +	struct rte_ipv4_hdr *ipv4_hdr;
> +	struct rte_tcp_hdr *merged_tcp_hdr;
> +
> +	eth_hdr =3D rte_pktmbuf_mtod(pkt, struct rte_ether_hdr *);
> +	ipv4_hdr =3D (struct rte_ipv4_hdr *)((char *)eth_hdr + pkt->l2_len);
> +	merged_tcp_hdr =3D (struct rte_tcp_hdr *)((char *)ipv4_hdr + pkt-
> >l3_len);
> +	merged_tcp_hdr->tcp_flags |=3D tcp_hdr->tcp_flags; }

The Linux kernel updates the TCP flag via "tcp_flag_word(th2) |=3D flags & =
(TCP_FLAG_FIN | TCP_FLAG_PSH)",
which only adds FIN and PSH at most to the merge packet.

> +
>  int32_t
>  gro_tcp4_reassemble(struct rte_mbuf *pkt,
>  		struct gro_tcp4_tbl *tbl,
> @@ -206,6 +219,7 @@ gro_tcp4_reassemble(struct rte_mbuf *pkt,
>  	uint32_t i, max_flow_num, remaining_flow_num;
>  	int cmp;
>  	uint8_t find;
> +	uint32_t start_idx;
>=20
>  	/*
>  	 * Don't process the packet whose TCP header length is greater @@ -
> 219,13 +233,6 @@ gro_tcp4_reassemble(struct rte_mbuf *pkt,
>  	tcp_hdr =3D (struct rte_tcp_hdr *)((char *)ipv4_hdr + pkt->l3_len);
>  	hdr_len =3D pkt->l2_len + pkt->l3_len + pkt->l4_len;
>=20
> -	/*
> -	 * Don't process the packet which has FIN, SYN, RST, PSH, URG, ECE
> -	 * or CWR set.
> -	 */
> -	if (tcp_hdr->tcp_flags !=3D RTE_TCP_ACK_FLAG)
> -		return -1;
> -
>  	/* trim the tail padding bytes */
>  	ip_tlen =3D rte_be_to_cpu_16(ipv4_hdr->total_length);
>  	if (pkt->pkt_len > (uint32_t)(ip_tlen + pkt->l2_len)) @@ -264,12
> +271,30 @@ gro_tcp4_reassemble(struct rte_mbuf *pkt,
>  		if (tbl->flows[i].start_index !=3D INVALID_ARRAY_INDEX) {
>  			if (is_same_tcp4_flow(tbl->flows[i].key, key)) {
>  				find =3D 1;
> +				start_idx =3D tbl->flows[i].start_index;
>  				break;
>  			}
>  			remaining_flow_num--;
>  		}
>  	}
>=20
> +	if (tcp_hdr->tcp_flags !=3D RTE_TCP_ACK_FLAG) {
> +		/*
> +		 * Check and try merging the current TCP segment with the
> previous
> +		 * TCP segment if the TCP header does not contain RST and
> SYN flag
> +		 * There are cases where the last segment is sent with
> FIN|PSH|ACK
> +		 * which should also be considered for merging with previous
> segments.
> +		 */
> +		if (find && !(tcp_hdr->tcp_flags &
> (RTE_TCP_RST_FLAG|RTE_TCP_SYN_FLAG)))
> +			/*
> +			 * Since PSH flag is set, start time will be set to 0 so it
> will be flushed
> +			 * immediately.
> +			 */
> +			tbl->items[start_idx].start_time =3D 0;
> +		else
> +			return -1;
> +	}

The nested if-else check is not straightforward, and it's hard to read the =
condition-action of
different combinations of flag bits. In addition, are all flag bits conside=
red like Linux kernel?

> +
>  	/*
>  	 * Fail to find a matched flow. Insert a new flow and store the
>  	 * packet into the flow.
> @@ -304,8 +329,12 @@ gro_tcp4_reassemble(struct rte_mbuf *pkt,
>  				is_atomic);
>  		if (cmp) {
>  			if (merge_two_tcp4_packets(&(tbl->items[cur_idx]),
> -						pkt, cmp, sent_seq, ip_id, 0))
> +						pkt, cmp, sent_seq, ip_id, 0))
> {
> +				if (tbl->items[cur_idx].start_time =3D=3D 0)
> +					update_tcp_hdr_flags(tcp_hdr, tbl-
> >items[cur_idx].firstseg);
>  				return 1;
> +			}
> +
>  			/*
>  			 * Fail to merge the two packets, as the packet
>  			 * length is greater than the max value. Store diff --git
> a/lib/gro/rte_gro.c b/lib/gro/rte_gro.c index e35399fd42..87c5502dce
> 100644
> --- a/lib/gro/rte_gro.c
> +++ b/lib/gro/rte_gro.c
> @@ -283,10 +283,17 @@ rte_gro_reassemble_burst(struct rte_mbuf **pkts,
>  	if ((nb_after_gro < nb_pkts)
>  		 || (unprocess_num < nb_pkts)) {
>  		i =3D 0;
> +		/* Copy unprocessed packets */
> +		if (unprocess_num > 0) {
> +			memcpy(&pkts[i], unprocess_pkts,
> +					sizeof(struct rte_mbuf *) *
> +					unprocess_num);
> +			i =3D unprocess_num;
> +		}

Why copy unprocess pkts first? This is for avoiding out-of-order?

Thanks,
Jiayu
>  		/* Flush all packets from the tables */
>  		if (do_vxlan_tcp_gro) {
> -			i =3D gro_vxlan_tcp4_tbl_timeout_flush(&vxlan_tcp_tbl,
> -					0, pkts, nb_pkts);
> +			i +=3D
> gro_vxlan_tcp4_tbl_timeout_flush(&vxlan_tcp_tbl,
> +					0, &pkts[i], nb_pkts - i);
>  		}
>=20
>  		if (do_vxlan_udp_gro) {
> @@ -304,13 +311,6 @@ rte_gro_reassemble_burst(struct rte_mbuf **pkts,
>  			i +=3D gro_udp4_tbl_timeout_flush(&udp_tbl, 0,
>  					&pkts[i], nb_pkts - i);
>  		}
> -		/* Copy unprocessed packets */
> -		if (unprocess_num > 0) {
> -			memcpy(&pkts[i], unprocess_pkts,
> -					sizeof(struct rte_mbuf *) *
> -					unprocess_num);
> -		}
> -		nb_after_gro =3D i + unprocess_num;
>  	}
>=20
>  	return nb_after_gro;
> --
> 2.25.1