From mboxrd@z Thu Jan 1 00:00:00 1970
Return-Path: <dev-bounces@dpdk.org>
Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124])
by inbox.dpdk.org (Postfix) with ESMTP id CE75042CA8;
Tue, 13 Jun 2023 12:21:10 +0200 (CEST)
Received: from mails.dpdk.org (localhost [127.0.0.1])
by mails.dpdk.org (Postfix) with ESMTP id 699BC42D49;
Tue, 13 Jun 2023 12:20:38 +0200 (CEST)
Received: from EUR01-HE1-obe.outbound.protection.outlook.com
(mail-he1eur01on2075.outbound.protection.outlook.com [40.107.13.75])
by mails.dpdk.org (Postfix) with ESMTP id 80ABB42D40;
Tue, 13 Jun 2023 12:20:37 +0200 (CEST)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=armh.onmicrosoft.com;
s=selector2-armh-onmicrosoft-com;
h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
bh=Q1ywgohhGxPmVGaZBMK8cEVjEvcdX83WPwJrQdW4yRg=;
b=BHrIPQKOIDgpoZO9PTh5pge600Vpsl4z7BU0uk01lzq6oPb/4FqFaG6gUCJTaEK3qAwVfwrNJ4/v8jqIBoSlOkWFRULIBqBGOod77lX48eLnlrWMz23bLqSe6ZbEDe3Wvzcu0c6VTlFAmoorsYm8iBCM+yRT8igxCAd2oqK8C7A=
Received: from AS9PR07CA0011.eurprd07.prod.outlook.com (2603:10a6:20b:46c::18)
by PA4PR08MB5886.eurprd08.prod.outlook.com (2603:10a6:102:e2::19)
with Microsoft SMTP Server (version=TLS1_2,
cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.6477.29; Tue, 13 Jun
2023 10:20:28 +0000
Received: from AM7EUR03FT032.eop-EUR03.prod.protection.outlook.com
(2603:10a6:20b:46c:cafe::1c) by AS9PR07CA0011.outlook.office365.com
(2603:10a6:20b:46c::18) with Microsoft SMTP Server (version=TLS1_2,
cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.6500.22 via Frontend
Transport; Tue, 13 Jun 2023 10:20:28 +0000
X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 63.35.35.123)
smtp.mailfrom=arm.com; dkim=pass (signature was verified)
header.d=armh.onmicrosoft.com;dmarc=pass action=none header.from=arm.com;
Received-SPF: Pass (protection.outlook.com: domain of arm.com designates
63.35.35.123 as permitted sender) receiver=protection.outlook.com;
client-ip=63.35.35.123; helo=64aa7808-outbound-1.mta.getcheckrecipient.com;
pr=C
Received: from 64aa7808-outbound-1.mta.getcheckrecipient.com (63.35.35.123) by
AM7EUR03FT032.mail.protection.outlook.com (100.127.140.65) with
Microsoft
SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
15.20.6500.23 via Frontend Transport; Tue, 13 Jun 2023 10:20:28 +0000
Received: ("Tessian outbound 945aec65ec65:v136");
Tue, 13 Jun 2023 10:20:28 +0000
X-CheckRecipientChecked: true
X-CR-MTA-CID: 10d7115e5104e07e
X-CR-MTA-TID: 64aa7808
Received: from e2db92fa5c5b.1
by 64aa7808-outbound-1.mta.getcheckrecipient.com id
C5582B97-1559-4D50-9FB3-F3C19C21E34F.1;
Tue, 13 Jun 2023 10:20:21 +0000
Received: from EUR05-VI1-obe.outbound.protection.outlook.com
by 64aa7808-outbound-1.mta.getcheckrecipient.com with ESMTPS id e2db92fa5c5b.1
(version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384);
Tue, 13 Jun 2023 10:20:20 +0000
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
b=ecYkWPn03Bt5mnlkdiixUgvHMMaRGjolxq4gWiUEnj19G+V+YAYXEYjZzFr9R3uKM8WITIPo/U22TxvL3pYVS+E/HLM++PS4+5whKMrDJeSAvXbdGpU4HweZ9FbjtOmetNBqmBhWah+IpZWt57dzqSeFwSuboYmk9YlnpWfAIbLhqOzfU2T6vPB5kI0sssupTktaVqdWD5/js2Al3YxIM8ZWQWYC1gzlKsxOU2BbI7Ovoea3yOFzQxse+vFi/Nb4TStwJrFc8irpdveoiFmJeq5xZT1B9MQJh295TC5uLtJ5GOWrdc1I6bnEQjcdgTvUV0F5udzMhPW2D0x7q4afjQ==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
s=arcselector9901;
h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1;
bh=Q1ywgohhGxPmVGaZBMK8cEVjEvcdX83WPwJrQdW4yRg=;
b=eiP0c5x7sAby0Cr74hV57XfLT4LJ4xJ5Za2B1onaVe9xDRyG4Pa54Uhtf/KC7psidfTzA+zKC0bi5CIxG4ZopvpJkeKNElZt3hWdsdjmeT46DsXVyJ48dQDZs+uDQS6n6b+DQKWPJHWx2c8etkeIayiO0TaU1hfD7BsjWKh0o8FHhuvpghdD9NkA/aBSjjpwKAxhE9yTVYxFDO3QkhDoitGc2SNrBDOT3fFyBBTg08Xc3brWzKCB7dK0oqjNG/iwC/+2DXx1RB1jP+irPsRpfxYOpgqXs9HrzIExON+HWceOIyskW0BblUcQVjwbGI61ElJM31O1O84rlFO8qJohjg==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
smtp.mailfrom=arm.com; dmarc=pass action=none header.from=arm.com; dkim=pass
header.d=arm.com; arc=none
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=armh.onmicrosoft.com;
s=selector2-armh-onmicrosoft-com;
h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
bh=Q1ywgohhGxPmVGaZBMK8cEVjEvcdX83WPwJrQdW4yRg=;
b=BHrIPQKOIDgpoZO9PTh5pge600Vpsl4z7BU0uk01lzq6oPb/4FqFaG6gUCJTaEK3qAwVfwrNJ4/v8jqIBoSlOkWFRULIBqBGOod77lX48eLnlrWMz23bLqSe6ZbEDe3Wvzcu0c6VTlFAmoorsYm8iBCM+yRT8igxCAd2oqK8C7A=
Received: from AS8PR08MB7080.eurprd08.prod.outlook.com (2603:10a6:20b:401::19)
by DB8PR08MB5515.eurprd08.prod.outlook.com (2603:10a6:10:11f::9) with
Microsoft SMTP Server (version=TLS1_2,
cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.6477.29; Tue, 13 Jun
2023 10:20:17 +0000
Received: from AS8PR08MB7080.eurprd08.prod.outlook.com
([fe80::dd71:31fd:80b0:c4e0]) by AS8PR08MB7080.eurprd08.prod.outlook.com
([fe80::dd71:31fd:80b0:c4e0%4]) with mapi id 15.20.6477.028; Tue, 13 Jun 2023
10:20:17 +0000
From: Ruifeng Wang <Ruifeng.Wang@arm.com>
To: Min Zhou <zhoumin@loongson.cn>, "thomas@monjalon.net"
<thomas@monjalon.net>, "qi.z.zhang@intel.com" <qi.z.zhang@intel.com>,
"mb@smartsharesystems.com" <mb@smartsharesystems.com>,
"konstantin.v.ananyev@yandex.ru" <konstantin.v.ananyev@yandex.ru>,
"drc@linux.vnet.ibm.com" <drc@linux.vnet.ibm.com>,
"roretzla@linux.microsoft.com" <roretzla@linux.microsoft.com>,
"qiming.yang@intel.com" <qiming.yang@intel.com>, "wenjun1.wu@intel.com"
<wenjun1.wu@intel.com>
CC: "dev@dpdk.org" <dev@dpdk.org>, "stable@dpdk.org" <stable@dpdk.org>,
"maobibo@loongson.cn" <maobibo@loongson.cn>, "jiawenwu@trustnetic.com"
<jiawenwu@trustnetic.com>, nd <nd@arm.com>
Subject: RE: [PATCH v4] net/ixgbe: add proper memory barriers for some Rx
functions
Thread-Topic: [PATCH v4] net/ixgbe: add proper memory barriers for some Rx
functions
Thread-Index: AQHZnduxR1MtGHx/7kSmktCy+7b686+IhV5A
Date: Tue, 13 Jun 2023 10:20:17 +0000
Message-ID: <AS8PR08MB70808C84F77C6F68A6F8E71B9E55A@AS8PR08MB7080.eurprd08.prod.outlook.com>
References: <20230506102359.243462-1-zhoumin@loongson.cn>
<20230613094425.2469036-1-zhoumin@loongson.cn>
In-Reply-To: <20230613094425.2469036-1-zhoumin@loongson.cn>
Accept-Language: en-US
Content-Language: en-US
X-MS-Has-Attach:
X-MS-TNEF-Correlator:
x-ts-tracking-id: 22AA3665E1A4A44986CDCEB7F807298D.0
Authentication-Results-Original: dkim=none (message not signed)
header.d=none;dmarc=none action=none header.from=arm.com;
x-ms-traffictypediagnostic: AS8PR08MB7080:EE_|DB8PR08MB5515:EE_|AM7EUR03FT032:EE_|PA4PR08MB5886:EE_
X-MS-Office365-Filtering-Correlation-Id: 0e049e4b-5691-415b-ec38-08db6bf7cf8c
x-ld-processed: f34e5979-57d9-4aaa-ad4d-b122a662184d,ExtAddr
x-checkrecipientrouted: true
nodisclaimer: true
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam-Untrusted: BCL:0;
X-Microsoft-Antispam-Message-Info-Original: Wh8fnhXIvUamrjxue9YVrDZIx0OiZw7QlbcBRDtOnFcSloYoioxIBGXAZZpQOGXXKEgSCBliLZswBuP3VKIRupGJG9m++Xf0hBRZ1yhMGroSyiGUYyxreGUfsRt31dxenZzGoDGU20eBSUY2M0XmsftNjJxcLsDk1gqLDoIPYKAte4HZCS9rYIc0xWeNex1xmxRQMyh2VNAW1CE/NV91IkX0m7T6EfU0zGyBj2+E4wRZdwChrAw/QI0u/qhZagWaVCG4ThH67oKrgKBpJ2ldGeZaKeXDuVnryWRyNevU6KQftOlD2DUsMWlQpYvyv9rcUMtJTax243/z635WmZ6RyO7z7abAFJnw8HN5ZEI5Ncq0VvqVPlR7+5kJD6vHxXS4TqmpifOpIPeivC3lDw/22HMVo/oH6IH8iz/721e7qSq6G2o9pop7f7/35esZ24uEaXYTQjox2qrRKOoffOEduMDIULGXKMDg/MSxJ27V4QRYX5fEbDfHfGYB7cg74u12JI8hNTRnTE2yb8IkJhUqpu8/fLaHcaIGffN+7Sm+mlxB+UD/zwTSwrLyLiKqUkGEyGYi3BoS88f884R+OUq7Ak2KmbNaeg+VVbsC3cYoXow/hNBxyNOWhIrLgLmoCa0ORy5x55ho/hSOzxRx0gXv6Q==
X-Forefront-Antispam-Report-Untrusted: CIP:255.255.255.255; CTRY:; LANG:en;
SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:AS8PR08MB7080.eurprd08.prod.outlook.com;
PTR:; CAT:NONE;
SFS:(13230028)(4636009)(346002)(136003)(366004)(39860400002)(376002)(396003)(451199021)(86362001)(7416002)(7696005)(316002)(8676002)(41300700001)(5660300002)(83380400001)(26005)(55016003)(38100700002)(53546011)(6506007)(52536014)(33656002)(71200400001)(8936002)(122000001)(9686003)(38070700005)(66476007)(66446008)(64756008)(66556008)(66946007)(76116006)(4326008)(478600001)(186003)(54906003)(2906002)(110136005)(23180200003);
DIR:OUT; SFP:1101;
Content-Type: text/plain; charset="us-ascii"
Content-Transfer-Encoding: quoted-printable
MIME-Version: 1.0
X-MS-Exchange-Transport-CrossTenantHeadersStamped: DB8PR08MB5515
Original-Authentication-Results: dkim=none (message not signed)
header.d=none;dmarc=none action=none header.from=arm.com;
X-EOPAttributedMessage: 0
X-MS-Exchange-Transport-CrossTenantHeadersStripped: AM7EUR03FT032.eop-EUR03.prod.protection.outlook.com
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-Correlation-Id-Prvs: 3e3b5269-2233-4949-8c8e-08db6bf7c8fa
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info: f/wWGb76uR+gWdrkxHE2JhN+zeZdA7zoJ3bXG/Wwhmuny91UPgE8wdCbLKddAiraY06GwYHrnbabvApuoBmKfqJ3xRGU2dDKC38YFIBkPlCTuN20S99LQVyULB2GHxvSFXpz3PRozZTZV25xLVGunIq+X64lU2UPRKsANJXSzjiDxGXHai93h+yV1H5mIIQ6HC78GqzcMSQoydG14CKQazmMLkLnpZMsyUmdTigjsjYp8fH8BHD6RpuDr+t+3uJCB9QVlAy7AD3gfuFiIQjgvTGQe3z8zbfHlpYxfvTRgRkNfAyweXpzkhoAK8ArSTlnZi53zSSB9x0MZ3AU+NC4TGB1/lZDKkEws/98mMCD48qBPsXuWsvEjnC0tHgrYlBaIB39KzpviLoMLQt9YoCwSnI0kEgXUnHSnUU/6f4QkHdm1z57yEuFLqdllX/ghSXEpKSmneUKTwY3vseHOHWbEfkp06RqmmH59cYOn92D+2X/0o2iKr+pj1cZX+fnODu5QddkxQ/zOvszhCgpqIxubYqwN84SCV7UQ8u8jVvzH3/FO8zPrcSTpZqsinloSeZBj39g3d8a1GAIRWooe+9BaRRMP+RYO1PEGbKrTOmx8AUQIfMXovzkE7cKeI4mpskbh3R8T7t3ilP025prYRRKFRskcaAsOFENr04YsoXT+zoxnAzSzwk8BYIHpcq+EJNrDk6Xu9NjvNRHCa4AACcJJEFObIrJeiMKeLoEW2sZdm5X9vAVqMiqvLosfh4zmzEjM2SHtsxSpetncVFr6SWsuQ==
X-Forefront-Antispam-Report: CIP:63.35.35.123; CTRY:IE; LANG:en; SCL:1; SRV:;
IPV:CAL; SFV:NSPM; H:64aa7808-outbound-1.mta.getcheckrecipient.com;
PTR:ec2-63-35-35-123.eu-west-1.compute.amazonaws.com; CAT:NONE;
SFS:(13230028)(4636009)(396003)(376002)(39860400002)(136003)(346002)(451199021)(36840700001)(40470700004)(46966006)(4326008)(316002)(450100002)(70206006)(41300700001)(70586007)(186003)(54906003)(2906002)(110136005)(478600001)(8936002)(8676002)(5660300002)(52536014)(40460700003)(36860700001)(7696005)(9686003)(53546011)(33656002)(81166007)(82740400003)(356005)(6506007)(40480700001)(26005)(55016003)(336012)(83380400001)(47076005)(86362001)(82310400005)(23180200003);
DIR:OUT; SFP:1101;
X-OriginatorOrg: arm.com
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 13 Jun 2023 10:20:28.5333 (UTC)
X-MS-Exchange-CrossTenant-Network-Message-Id: 0e049e4b-5691-415b-ec38-08db6bf7cf8c
X-MS-Exchange-CrossTenant-Id: f34e5979-57d9-4aaa-ad4d-b122a662184d
X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=f34e5979-57d9-4aaa-ad4d-b122a662184d; Ip=[63.35.35.123];
Helo=[64aa7808-outbound-1.mta.getcheckrecipient.com]
X-MS-Exchange-CrossTenant-AuthSource: AM7EUR03FT032.eop-EUR03.prod.protection.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Anonymous
X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem
X-MS-Exchange-Transport-CrossTenantHeadersStamped: PA4PR08MB5886
X-BeenThere: dev@dpdk.org
X-Mailman-Version: 2.1.29
Precedence: list
List-Id: DPDK patches and discussions <dev.dpdk.org>
List-Unsubscribe: <https://mails.dpdk.org/options/dev>,
<mailto:dev-request@dpdk.org?subject=unsubscribe>
List-Archive: <http://mails.dpdk.org/archives/dev/>
List-Post: <mailto:dev@dpdk.org>
List-Help: <mailto:dev-request@dpdk.org?subject=help>
List-Subscribe: <https://mails.dpdk.org/listinfo/dev>,
<mailto:dev-request@dpdk.org?subject=subscribe>
Errors-To: dev-bounces@dpdk.org
> -----Original Message-----
> From: Min Zhou <zhoumin@loongson.cn>
> Sent: Tuesday, June 13, 2023 5:44 PM
> To: thomas@monjalon.net; qi.z.zhang@intel.com; mb@smartsharesystems.com;
> konstantin.v.ananyev@yandex.ru; Ruifeng Wang <Ruifeng.Wang@arm.com>;
> drc@linux.vnet.ibm.com; roretzla@linux.microsoft.com; qiming.yang@intel.c=
om;
> wenjun1.wu@intel.com; zhoumin@loongson.cn
> Cc: dev@dpdk.org; stable@dpdk.org; maobibo@loongson.cn; jiawenwu@trustnet=
ic.com
> Subject: [PATCH v4] net/ixgbe: add proper memory barriers for some Rx fun=
ctions
>=20
> Segmentation fault has been observed while running the
> ixgbe_recv_pkts_lro() function to receive packets on the Loongson 3C5000 =
processor which
> has 64 cores and 4 NUMA nodes.
>=20
> From the ixgbe_recv_pkts_lro() function, we found that as long as the fir=
st packet has the
> EOP bit set, and the length of this packet is less than or equal to rxq->=
crc_len, the
> segmentation fault will definitely happen even though on the other platfo=
rms. For example,
> if we made the first packet which had the EOP bit set had a zero length b=
y force, the
> segmentation fault would happen on X86.
>=20
> Because when processd the first packet the first_seg->next will be NULL, =
if at the same
> time this packet has the EOP bit set and its length is less than or equal=
to rxq->crc_len,
> the following loop will be executed:
>=20
> for (lp =3D first_seg; lp->next !=3D rxm; lp =3D lp->next)
> ;
>=20
> We know that the first_seg->next will be NULL under this condition. So th=
e expression of
> lp->next->next will cause the segmentation fault.
>=20
> Normally, the length of the first packet with EOP bit set will be greater=
than rxq-
> >crc_len. However, the out-of-order execution of CPU may make the read or=
dering of the
> status and the rest of the descriptor fields in this function not be corr=
ect. The related
> codes are as following:
>=20
> rxdp =3D &rx_ring[rx_id];
> #1 staterr =3D rte_le_to_cpu_32(rxdp->wb.upper.status_error);
>=20
> if (!(staterr & IXGBE_RXDADV_STAT_DD))
> break;
>=20
> #2 rxd =3D *rxdp;
>=20
> The sentence #2 may be executed before sentence #1. This action is likely=
to make the
> ready packet zero length. If the packet is the first packet and has the E=
OP bit set, the
> above segmentation fault will happen.
>=20
> So, we should add a proper memory barrier to ensure the read ordering be =
correct. We also
> did the same thing in the ixgbe_recv_pkts() function to make the rxd data=
be valid even
> though we did not find segmentation fault in this function.
>=20
> Fixes: 8eecb3295ae ("ixgbe: add LRO support")
> Cc: stable@dpdk.org
>=20
> Signed-off-by: Min Zhou <zhoumin@loongson.cn>
> ---
> v4:
> - Replace rte_smp_rmb() with rte_atomic_thread_fence() as the proper memo=
ry
> barrier
> ---
> v3:
> - Use rte_smp_rmb() as the proper memory barrier instead of rte_rmb()
> ---
> v2:
> - Make the calling of rte_rmb() for all platforms
> ---
>=20
> drivers/net/ixgbe/ixgbe_rxtx.c | 47 +++++++++++++++-------------------
> 1 file changed, 21 insertions(+), 26 deletions(-)
>=20
> diff --git a/drivers/net/ixgbe/ixgbe_rxtx.c b/drivers/net/ixgbe/ixgbe_rxt=
x.c index
> 6cbb992823..61f17cd90b 100644
> --- a/drivers/net/ixgbe/ixgbe_rxtx.c
> +++ b/drivers/net/ixgbe/ixgbe_rxtx.c
> @@ -1817,11 +1817,22 @@ ixgbe_recv_pkts(void *rx_queue, struct rte_mbuf *=
*rx_pkts,
> * of accesses cannot be reordered by the compiler. If they were
> * not volatile, they could be reordered which could lead to
> * using invalid descriptor fields when read from rxd.
> + *
> + * Meanwhile, to prevent the CPU from executing out of order, we
> + * need to use a proper memory barrier to ensure the memory
> + * ordering below.
> */
> rxdp =3D &rx_ring[rx_id];
> staterr =3D rxdp->wb.upper.status_error;
> if (!(staterr & rte_cpu_to_le_32(IXGBE_RXDADV_STAT_DD)))
> break;
> +
> + /*
> + * Use acquire fence to ensure that status_error which includes
> + * DD bit is loaded before loading of other descriptor words.
> + */
> + rte_atomic_thread_fence(__ATOMIC_ACQUIRE);
> +
> rxd =3D *rxdp;
>=20
> /*
> @@ -2088,32 +2099,10 @@ ixgbe_recv_pkts_lro(void *rx_queue, struct rte_mb=
uf **rx_pkts,
> uint16_t nb_pkts,
>=20
> next_desc:
> /*
> - * The code in this whole file uses the volatile pointer to
> - * ensure the read ordering of the status and the rest of the
> - * descriptor fields (on the compiler level only!!!). This is so
> - * UGLY - why not to just use the compiler barrier instead? DPDK
> - * even has the rte_compiler_barrier() for that.
> - *
> - * But most importantly this is just wrong because this doesn't
> - * ensure memory ordering in a general case at all. For
> - * instance, DPDK is supposed to work on Power CPUs where
> - * compiler barrier may just not be enough!
> - *
> - * I tried to write only this function properly to have a
> - * starting point (as a part of an LRO/RSC series) but the
> - * compiler cursed at me when I tried to cast away the
> - * "volatile" from rx_ring (yes, it's volatile too!!!). So, I'm
> - * keeping it the way it is for now.
> - *
> - * The code in this file is broken in so many other places and
> - * will just not work on a big endian CPU anyway therefore the
> - * lines below will have to be revisited together with the rest
> - * of the ixgbe PMD.
> - *
> - * TODO:
> - * - Get rid of "volatile" and let the compiler do its job.
> - * - Use the proper memory barrier (rte_rmb()) to ensure the
> - * memory ordering below.
> + * "Volatile" only prevents caching of the variable marked
> + * volatile. Most important, "volatile" cannot prevent the CPU
> + * from executing out of order. So, it is necessary to use a
> + * proper memory barrier to ensure the memory ordering below.
> */
> rxdp =3D &rx_ring[rx_id];
> staterr =3D rte_le_to_cpu_32(rxdp->wb.upper.status_error);
> @@ -2121,6 +2110,12 @@ ixgbe_recv_pkts_lro(void *rx_queue, struct rte_mbu=
f **rx_pkts,
> uint16_t nb_pkts,
> if (!(staterr & IXGBE_RXDADV_STAT_DD))
> break;
>=20
> + /*
> + * Use acquire fence to ensure that status_error which includes
> + * DD bit is loaded before loading of other descriptor words.
> + */
> + rte_atomic_thread_fence(__ATOMIC_ACQUIRE);
> +
> rxd =3D *rxdp;
>=20
> PMD_RX_LOG(DEBUG, "port_id=3D%u queue_id=3D%u rx_id=3D%u "
> --
> 2.31.1
Reviewed-by: Ruifeng Wang <ruifeng.wang@arm.com>