From mboxrd@z Thu Jan  1 00:00:00 1970
Return-Path: <dev-bounces@dpdk.org>
Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124])
	by inbox.dpdk.org (Postfix) with ESMTP id 89C4942CA9;
	Tue, 13 Jun 2023 14:11:47 +0200 (CEST)
Received: from mails.dpdk.org (localhost [127.0.0.1])
	by mails.dpdk.org (Postfix) with ESMTP id 66FAD40A8A;
	Tue, 13 Jun 2023 14:11:47 +0200 (CEST)
Received: from mga02.intel.com (mga02.intel.com [134.134.136.20])
 by mails.dpdk.org (Postfix) with ESMTP id C250E40698;
 Tue, 13 Jun 2023 14:11:45 +0200 (CEST)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple;
 d=intel.com; i=@intel.com; q=dns/txt; s=Intel;
 t=1686658306; x=1718194306;
 h=from:to:cc:subject:date:message-id:references:
 in-reply-to:content-transfer-encoding:mime-version;
 bh=jgciwfj0sobY9JXnJbyg0YDmRXV+CRm62vi0kijXPPs=;
 b=kXqmqDejyckbxmSzHgjispQ1kaByzbpiDkafiOS/qUr5ZiDKsys4wxVJ
 Jn9uGq3PhUXjoMMAEJh3WLrN3maAVsgIZLyVddCEAaO9TCpOjAIzHtdkq
 lMsxitgeaivkBJzWx6RdtrB2n9Ah5TY1fQzq9zlL/4rjSIkrJ1YD1mCgp
 XmFpFu1knm8VfdcOeP78NUmG9CAYiyzCYDnVKlaH7mqWngYBb4wwcRdeS
 f9JM93aCUwoHRKlSzdlGz80w/p01YeOoKfyaOaQNMT1rWEib0QLGCfjA3
 4vOUIGKSWSuVjkzQboYXMtv/uLsTyzR/MP/yuokqQNJTkb8/p46VVfEhJ A==;
X-IronPort-AV: E=McAfee;i="6600,9927,10739"; a="347966066"
X-IronPort-AV: E=Sophos;i="6.00,239,1681196400"; d="scan'208";a="347966066"
Received: from fmsmga007.fm.intel.com ([10.253.24.52])
 by orsmga101.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384;
 13 Jun 2023 05:11:44 -0700
X-ExtLoop1: 1
X-IronPort-AV: E=McAfee;i="6600,9927,10739"; a="714778945"
X-IronPort-AV: E=Sophos;i="6.00,239,1681196400"; d="scan'208";a="714778945"
Received: from fmsmsx602.amr.corp.intel.com ([10.18.126.82])
 by fmsmga007.fm.intel.com with ESMTP; 13 Jun 2023 05:11:44 -0700
Received: from fmsmsx611.amr.corp.intel.com (10.18.126.91) by
 fmsmsx602.amr.corp.intel.com (10.18.126.82) with Microsoft SMTP Server
 (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id
 15.1.2507.23; Tue, 13 Jun 2023 05:11:43 -0700
Received: from fmsmsx610.amr.corp.intel.com (10.18.126.90) by
 fmsmsx611.amr.corp.intel.com (10.18.126.91) with Microsoft SMTP Server
 (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id
 15.1.2507.23; Tue, 13 Jun 2023 05:11:43 -0700
Received: from FMSEDG603.ED.cps.intel.com (10.1.192.133) by
 fmsmsx610.amr.corp.intel.com (10.18.126.90) with Microsoft SMTP Server
 (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id
 15.1.2507.23 via Frontend Transport; Tue, 13 Jun 2023 05:11:43 -0700
Received: from NAM12-MW2-obe.outbound.protection.outlook.com (104.47.66.44) by
 edgegateway.intel.com (192.55.55.68) with Microsoft SMTP Server
 (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.1.2507.23; Tue, 13 Jun 2023 05:11:42 -0700
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=dqSd0MDv2uvU4pPRO+4RHE6UKHcuXdEhJ7HuHSK7786XOkNLOsehgqNE27R/RR4LIFtpRPa7F/NL5eIyMFfX5dnzUAp1JD+J0TSl/Bww22+zdajMyliQc3EPTgBGx8Ql/xJAJey4J23UCkBthqIiMItWWH+sq3/qBX63cyyTnbJka2UWolaAaH9fUfzgt76rq7RG56mUuzprlztVx/Tcx+xO1dc8Lzs0dnio/VssNKNpgnaidIDOErf9UU2ak8YQyLqnS3ZStcP/s931CpJYr6/vMY+AEOf5PwwaT9NqiQUtXsIUqjoDGoFEgb+pU3edNaTLjIzuZMJ7pstx79nUQQ==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; 
 s=arcselector9901;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1;
 bh=q1BY9dBVJl/vsGvScfubpu+mkL+j039LnW0FdzW5MhA=;
 b=RJwzME4qJAQBZ9iWGc1madE87O1Y3O2T0Ad1xjkD8ew8/0XJebgYSNCliju+zjJL8LWeYKlFs0dchfMIUKjgYKh8TIV0YGbE9hXuVbDTOExmK9t8m7w2+/mKjcfnjUJByi5RTaRyfv1cPavL2Hg98guOMI9IFP0Y1yArWkE+jwN972k6U2O+m3/MWAQ2Ncwe8nLwptyMJTmdXF+KIsLwGV+LoQgFu/wcPZISxfL5xEKgwtZATlX9hIx/FZlyMcoq3wEGd6HipY3/CEV6iXGkmhFivfRTFcCSadv8/TiQ0YcpSz/67F5QJYZrUF8XYUM/4Th0klAf4WB8X9cfBynrJQ==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=intel.com; dmarc=pass action=none header.from=intel.com;
 dkim=pass header.d=intel.com; arc=none
Received: from DM4PR11MB5994.namprd11.prod.outlook.com (2603:10b6:8:5d::20) by
 BL1PR11MB5415.namprd11.prod.outlook.com (2603:10b6:208:315::14) with
 Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.6477.29; Tue, 13 Jun
 2023 12:11:39 +0000
Received: from DM4PR11MB5994.namprd11.prod.outlook.com
 ([fe80::e570:d9a7:df1b:1589]) by DM4PR11MB5994.namprd11.prod.outlook.com
 ([fe80::e570:d9a7:df1b:1589%6]) with mapi id 15.20.6455.030; Tue, 13 Jun 2023
 12:11:39 +0000
From: "Zhang, Qi Z" <qi.z.zhang@intel.com>
To: Ruifeng Wang <Ruifeng.Wang@arm.com>, Min Zhou <zhoumin@loongson.cn>,
 "thomas@monjalon.net" <thomas@monjalon.net>, "mb@smartsharesystems.com"
 <mb@smartsharesystems.com>, "konstantin.v.ananyev@yandex.ru"
 <konstantin.v.ananyev@yandex.ru>, "drc@linux.vnet.ibm.com"
 <drc@linux.vnet.ibm.com>, "roretzla@linux.microsoft.com"
 <roretzla@linux.microsoft.com>, "Yang, Qiming" <qiming.yang@intel.com>, "Wu,
 Wenjun1" <wenjun1.wu@intel.com>
CC: "dev@dpdk.org" <dev@dpdk.org>, "stable@dpdk.org" <stable@dpdk.org>,
 "maobibo@loongson.cn" <maobibo@loongson.cn>, "jiawenwu@trustnetic.com"
 <jiawenwu@trustnetic.com>, nd <nd@arm.com>
Subject: RE: [PATCH v4] net/ixgbe: add proper memory barriers for some Rx
 functions
Thread-Topic: [PATCH v4] net/ixgbe: add proper memory barriers for some Rx
 functions
Thread-Index: AQHZndu3s6pgY6KY9U+L/PmGiKdNSq+IhZSAgAAfCBA=
Date: Tue, 13 Jun 2023 12:11:38 +0000
Message-ID: <DM4PR11MB599457EA8C3F53C5B462C5B8D755A@DM4PR11MB5994.namprd11.prod.outlook.com>
References: <20230506102359.243462-1-zhoumin@loongson.cn>
 <20230613094425.2469036-1-zhoumin@loongson.cn>
 <AS8PR08MB70808C84F77C6F68A6F8E71B9E55A@AS8PR08MB7080.eurprd08.prod.outlook.com>
In-Reply-To: <AS8PR08MB70808C84F77C6F68A6F8E71B9E55A@AS8PR08MB7080.eurprd08.prod.outlook.com>
Accept-Language: en-US
Content-Language: en-US
X-MS-Has-Attach: 
X-MS-TNEF-Correlator: 
authentication-results: dkim=none (message not signed)
 header.d=none;dmarc=none action=none header.from=intel.com;
x-ms-publictraffictype: Email
x-ms-traffictypediagnostic: DM4PR11MB5994:EE_|BL1PR11MB5415:EE_
x-ms-office365-filtering-correlation-id: b46a5592-a499-451e-8e4f-08db6c07575c
x-ms-exchange-senderadcheck: 1
x-ms-exchange-antispam-relay: 0
x-microsoft-antispam: BCL:0;
x-microsoft-antispam-message-info: zCnfFD6repDoS+zTPdxdt2HavLKcO4JrzeKhJYDmme3ktXLY+gmHxf+i4ZWaKKWTgeP4GxGmPS6GxoNTBcj0JmhvALh9kCwj1apbKLQhIj2vQ2oaha+tNl+d2F7m3RO2sHOqLAZgQs/cpdjofZl9DBBE5qPDRD/BpiYsxRmfCYtK7t0duHMld2fWxQpsYG3xKl+8HplY4ltr3PXT2z9Dwvt1ihfpLnfKcsFbem/C0kdbHNHxEpcAwP4uh14puA1q4brerURMEYJI9BXPP8MX4XQSVOmGN1sg3TMrz+NFXOgQnwajZAZ7783ToXC2ofVagKugKv1cbsv6VPiyefQuUD3fiIOkEFoKhSkxKbKE9EnzRQYqE36pSrX/B0xwLzR7Jic1GK3Tv5PVxTvVfxsMR9+bmcgQnpT+VZWuq017ZKk6GUZHdbHj5z9X1Eh04jRCUKBNxYC8GsGYzzDRxiTn51Ne/e7nfmwNnwRkmdZh3Dv3gNFgqtlBw7qWaPPTaYUlnYQogZkWArfOJf8Ou0AioafjcK8XRm3qZT/soY2vZqx7XQQl9l3fXXr+CmBElmPbbNHztagwJ3KfDRj8M1Y/29mYNUWrfWUD7bUpCe6bkZFA9iVjd3Xu6Mto0ChzuMKme/XsBnQtlzHLYlDLW8mfjDHnEEyn4E+npJ6BKRVY9M0=
x-forefront-antispam-report: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:;
 IPV:NLI; SFV:NSPM; H:DM4PR11MB5994.namprd11.prod.outlook.com; PTR:; CAT:NONE;
 SFS:(13230028)(346002)(136003)(376002)(39860400002)(396003)(366004)(451199021)(86362001)(7416002)(7696005)(8676002)(316002)(41300700001)(82960400001)(83380400001)(5660300002)(26005)(55016003)(38100700002)(53546011)(921005)(52536014)(33656002)(6506007)(71200400001)(122000001)(8936002)(9686003)(38070700005)(64756008)(66476007)(66556008)(66446008)(76116006)(4326008)(6636002)(66946007)(478600001)(186003)(54906003)(2906002)(110136005)(23180200003);
 DIR:OUT; SFP:1102; 
x-ms-exchange-antispam-messagedata-chunkcount: 1
x-ms-exchange-antispam-messagedata-0: =?us-ascii?Q?FHCtReZTrjN0y9NJBXS+hMfdDtpYHDCERGali1KNXR78s+ce9EG3ZQIGibUZ?=
 =?us-ascii?Q?F7N2aKDO3uOFT7Mf9gYHurh1CKO9aYRJmdha0xEovs6/aodqHp4r/dxSLMOZ?=
 =?us-ascii?Q?OxzFtq0g2CYJBEqNg3EJvhulWwPiMHmRn4bzUOTZcoCEenyV1NGBc3Yl6rdY?=
 =?us-ascii?Q?hKPqHMKKkLWbJx2PyCx8hPVjI8qwufOnp3rmGEUbQ8W02ZrHedJAvvyApGdb?=
 =?us-ascii?Q?GrXOUhtJot7pcubXHrrFgPiGz2iH4RDRUvws3GsosHkRrrL9GjoA80ojfEEW?=
 =?us-ascii?Q?sCS7sXhzY+mH4zqs0MNZnGfev/CB4gtaZgIOuyjjLaA0YCEx+kT6/gIfHdK7?=
 =?us-ascii?Q?JwTv+dYPvHv2dJDVW84NWvkO9oPib3pMpQ1r/pfPXBT0cJTU7J5c7bURXpQP?=
 =?us-ascii?Q?W4hO+pm+UFugIhQemhoGd3i+aYlifpyeGrw2CymBSMgMJR6A6OWwovkjyBb8?=
 =?us-ascii?Q?gPD5CC2yK28OlSbu4uW6dRshDJNsBSgBpZj/q8kqWE0zi+hdlP5V90nQof3/?=
 =?us-ascii?Q?5Myp8uxGsRfjR8/7Y8obTAdTMrj2AltjmspcdeIqaGDWuoGHuNca9hZVInW7?=
 =?us-ascii?Q?qhV9v3YXeRHF0L0K5zYtEFprGQ8LvU7/e83c6V0OTixTonpAnIJZ2hhLG+IV?=
 =?us-ascii?Q?BAhtP5fjo7LyO6Iuh40DW24+NbBwcM3D4hhAq9BG3OSH/qfVP5OOevbxCcQ+?=
 =?us-ascii?Q?tQqfBMGkCizoEBcgJOy/nCTVjgBd6f/UhGxBH6nzVQsnZb8FdBjdOH1D4Qp7?=
 =?us-ascii?Q?rXRpAw3uSlGWAFv/7629er4Pn1S3LmPWfpUmvCGPL8fl74w+eWDBV6GtqqVi?=
 =?us-ascii?Q?6qecSbDRR3RFv4c4EWe2fvLinFxysfMv3iuX6B+x3yddzAL546R6M9NLnvos?=
 =?us-ascii?Q?4P5fZTE/h/FJCyKtNXPlpIdT2AZ32MESbVr8EVvvlFrpRIst43AkJ32I8gx8?=
 =?us-ascii?Q?6baXtsksTLTAcmak/3u46m62yUqftFgsCv3xIC8cY/Q+MgTkLkwnkUfzYyIN?=
 =?us-ascii?Q?hGbPLPrHr2KAitXKB903dplOQyuNTuXhAqt9Mdn+DU28S0uSNtFYrsXAQ3ai?=
 =?us-ascii?Q?p7fUmWZwhVwX4+/5HZKQNJdkxvNzNIsPkMMgBV+eyZhOWKxunZUWuNbl9U7Y?=
 =?us-ascii?Q?WaJNZBMEhPBs18ZVyWbIx+DQHdaMKao9vyRouknRy+LLOQFRqJQfhFGr+R+f?=
 =?us-ascii?Q?qQDatwNE097uP0ThB4Mep2mRAMa+Ji6KF2Ep52ffF+ObJsdAo/TTqHDg3S2h?=
 =?us-ascii?Q?QIKr4cx2/mRbOdKGsrFWoChHDpqx1JKkgMCCyyB52GDUsR5E3kZ68PifjnPZ?=
 =?us-ascii?Q?pXjRwsoQkPEoT40RpDiTzKLBFupFedpx15ZCTcG1kbJiRyLJpnIEabZPrEUd?=
 =?us-ascii?Q?PDDEByZmgGcB1a3z5XIxJycckIbCcz+XdmJdAtpzIfIeY3ZfCXEQLdF7G3s2?=
 =?us-ascii?Q?8/CK8oDRYBsJzG1KKQ5c+VvDXJZ/aEFCYKEFstHxvsFHdjfPyrjVhT5v8u6u?=
 =?us-ascii?Q?wKKUtHcHgQIX/1OhHltqASyItH5Y+OYl2GqWYO0psytwUut8IxT7sV5v7iSm?=
 =?us-ascii?Q?rIdJBXkmfr66u4qL1aVqZAposSbqA+e1KUj9W0kp?=
Content-Type: text/plain; charset="us-ascii"
Content-Transfer-Encoding: quoted-printable
MIME-Version: 1.0
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-AuthSource: DM4PR11MB5994.namprd11.prod.outlook.com
X-MS-Exchange-CrossTenant-Network-Message-Id: b46a5592-a499-451e-8e4f-08db6c07575c
X-MS-Exchange-CrossTenant-originalarrivaltime: 13 Jun 2023 12:11:38.8855 (UTC)
X-MS-Exchange-CrossTenant-fromentityheader: Hosted
X-MS-Exchange-CrossTenant-id: 46c98d88-e344-4ed4-8496-4ed7712e255d
X-MS-Exchange-CrossTenant-mailboxtype: HOSTED
X-MS-Exchange-CrossTenant-userprincipalname: 1gkhdhPs7+86hrz8IMmlmFiT8zXVNrKkrQMtxdy9CgLPLHJQrOrv4FJAcNrH2YAF5yF6DwKJvJQQhocpNpg+IQ==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: BL1PR11MB5415
X-OriginatorOrg: intel.com
X-BeenThere: dev@dpdk.org
X-Mailman-Version: 2.1.29
Precedence: list
List-Id: DPDK patches and discussions <dev.dpdk.org>
List-Unsubscribe: <https://mails.dpdk.org/options/dev>,
 <mailto:dev-request@dpdk.org?subject=unsubscribe>
List-Archive: <http://mails.dpdk.org/archives/dev/>
List-Post: <mailto:dev@dpdk.org>
List-Help: <mailto:dev-request@dpdk.org?subject=help>
List-Subscribe: <https://mails.dpdk.org/listinfo/dev>,
 <mailto:dev-request@dpdk.org?subject=subscribe>
Errors-To: dev-bounces@dpdk.org



> -----Original Message-----
> From: Ruifeng Wang <Ruifeng.Wang@arm.com>
> Sent: Tuesday, June 13, 2023 6:20 PM
> To: Min Zhou <zhoumin@loongson.cn>; thomas@monjalon.net; Zhang, Qi Z
> <qi.z.zhang@intel.com>; mb@smartsharesystems.com;
> konstantin.v.ananyev@yandex.ru; drc@linux.vnet.ibm.com;
> roretzla@linux.microsoft.com; Yang, Qiming <qiming.yang@intel.com>; Wu,
> Wenjun1 <wenjun1.wu@intel.com>
> Cc: dev@dpdk.org; stable@dpdk.org; maobibo@loongson.cn;
> jiawenwu@trustnetic.com; nd <nd@arm.com>
> Subject: RE: [PATCH v4] net/ixgbe: add proper memory barriers for some Rx
> functions
>=20
> > -----Original Message-----
> > From: Min Zhou <zhoumin@loongson.cn>
> > Sent: Tuesday, June 13, 2023 5:44 PM
> > To: thomas@monjalon.net; qi.z.zhang@intel.com;
> > mb@smartsharesystems.com; konstantin.v.ananyev@yandex.ru; Ruifeng
> Wang
> > <Ruifeng.Wang@arm.com>; drc@linux.vnet.ibm.com;
> > roretzla@linux.microsoft.com; qiming.yang@intel.com;
> > wenjun1.wu@intel.com; zhoumin@loongson.cn
> > Cc: dev@dpdk.org; stable@dpdk.org; maobibo@loongson.cn;
> > jiawenwu@trustnetic.com
> > Subject: [PATCH v4] net/ixgbe: add proper memory barriers for some Rx
> > functions
> >
> > Segmentation fault has been observed while running the
> > ixgbe_recv_pkts_lro() function to receive packets on the Loongson
> > 3C5000 processor which has 64 cores and 4 NUMA nodes.
> >
> > From the ixgbe_recv_pkts_lro() function, we found that as long as the
> > first packet has the EOP bit set, and the length of this packet is
> > less than or equal to rxq->crc_len, the segmentation fault will
> > definitely happen even though on the other platforms. For example, if
> > we made the first packet which had the EOP bit set had a zero length by
> force, the segmentation fault would happen on X86.
> >
> > Because when processd the first packet the first_seg->next will be
> > NULL, if at the same time this packet has the EOP bit set and its
> > length is less than or equal to rxq->crc_len, the following loop will b=
e
> executed:
> >
> >     for (lp =3D first_seg; lp->next !=3D rxm; lp =3D lp->next)
> >         ;
> >
> > We know that the first_seg->next will be NULL under this condition. So
> > the expression of
> > lp->next->next will cause the segmentation fault.
> >
> > Normally, the length of the first packet with EOP bit set will be
> > greater than rxq-
> > >crc_len. However, the out-of-order execution of CPU may make the read
> > >ordering of the
> > status and the rest of the descriptor fields in this function not be
> > correct. The related codes are as following:
> >
> >         rxdp =3D &rx_ring[rx_id];
> >  #1     staterr =3D rte_le_to_cpu_32(rxdp->wb.upper.status_error);
> >
> >         if (!(staterr & IXGBE_RXDADV_STAT_DD))
> >             break;
> >
> >  #2     rxd =3D *rxdp;
> >
> > The sentence #2 may be executed before sentence #1. This action is
> > likely to make the ready packet zero length. If the packet is the
> > first packet and has the EOP bit set, the above segmentation fault will
> happen.
> >
> > So, we should add a proper memory barrier to ensure the read ordering
> > be correct. We also did the same thing in the ixgbe_recv_pkts()
> > function to make the rxd data be valid even though we did not find
> segmentation fault in this function.
> >
> > Fixes: 8eecb3295ae ("ixgbe: add LRO support")
> > Cc: stable@dpdk.org
> >
> > Signed-off-by: Min Zhou <zhoumin@loongson.cn>
> > ---
> > v4:
> > - Replace rte_smp_rmb() with rte_atomic_thread_fence() as the proper
> memory
> >   barrier
> > ---
> > v3:
> > - Use rte_smp_rmb() as the proper memory barrier instead of rte_rmb()
> > ---
> > v2:
> > - Make the calling of rte_rmb() for all platforms
> > ---
> >
> >  drivers/net/ixgbe/ixgbe_rxtx.c | 47
> > +++++++++++++++-------------------
> >  1 file changed, 21 insertions(+), 26 deletions(-)
> >
> > diff --git a/drivers/net/ixgbe/ixgbe_rxtx.c
> > b/drivers/net/ixgbe/ixgbe_rxtx.c index 6cbb992823..61f17cd90b 100644
> > --- a/drivers/net/ixgbe/ixgbe_rxtx.c
> > +++ b/drivers/net/ixgbe/ixgbe_rxtx.c
> > @@ -1817,11 +1817,22 @@ ixgbe_recv_pkts(void *rx_queue, struct
> rte_mbuf **rx_pkts,
> >  		 * of accesses cannot be reordered by the compiler. If they
> were
> >  		 * not volatile, they could be reordered which could lead to
> >  		 * using invalid descriptor fields when read from rxd.
> > +		 *
> > +		 * Meanwhile, to prevent the CPU from executing out of
> order, we
> > +		 * need to use a proper memory barrier to ensure the
> memory
> > +		 * ordering below.
> >  		 */
> >  		rxdp =3D &rx_ring[rx_id];
> >  		staterr =3D rxdp->wb.upper.status_error;
> >  		if (!(staterr & rte_cpu_to_le_32(IXGBE_RXDADV_STAT_DD)))
> >  			break;
> > +
> > +		/*
> > +		 * Use acquire fence to ensure that status_error which
> includes
> > +		 * DD bit is loaded before loading of other descriptor words.
> > +		 */
> > +		rte_atomic_thread_fence(__ATOMIC_ACQUIRE);
> > +
> >  		rxd =3D *rxdp;
> >
> >  		/*
> > @@ -2088,32 +2099,10 @@ ixgbe_recv_pkts_lro(void *rx_queue, struct
> > rte_mbuf **rx_pkts, uint16_t nb_pkts,
> >
> >  next_desc:
> >  		/*
> > -		 * The code in this whole file uses the volatile pointer to
> > -		 * ensure the read ordering of the status and the rest of the
> > -		 * descriptor fields (on the compiler level only!!!). This is so
> > -		 * UGLY - why not to just use the compiler barrier instead?
> DPDK
> > -		 * even has the rte_compiler_barrier() for that.
> > -		 *
> > -		 * But most importantly this is just wrong because this
> doesn't
> > -		 * ensure memory ordering in a general case at all. For
> > -		 * instance, DPDK is supposed to work on Power CPUs where
> > -		 * compiler barrier may just not be enough!
> > -		 *
> > -		 * I tried to write only this function properly to have a
> > -		 * starting point (as a part of an LRO/RSC series) but the
> > -		 * compiler cursed at me when I tried to cast away the
> > -		 * "volatile" from rx_ring (yes, it's volatile too!!!). So, I'm
> > -		 * keeping it the way it is for now.
> > -		 *
> > -		 * The code in this file is broken in so many other places and
> > -		 * will just not work on a big endian CPU anyway therefore
> the
> > -		 * lines below will have to be revisited together with the rest
> > -		 * of the ixgbe PMD.
> > -		 *
> > -		 * TODO:
> > -		 *    - Get rid of "volatile" and let the compiler do its job.
> > -		 *    - Use the proper memory barrier (rte_rmb()) to ensure
> the
> > -		 *      memory ordering below.
> > +		 * "Volatile" only prevents caching of the variable marked
> > +		 * volatile. Most important, "volatile" cannot prevent the
> CPU
> > +		 * from executing out of order. So, it is necessary to use a
> > +		 * proper memory barrier to ensure the memory ordering
> below.
> >  		 */
> >  		rxdp =3D &rx_ring[rx_id];
> >  		staterr =3D rte_le_to_cpu_32(rxdp->wb.upper.status_error);
> > @@ -2121,6 +2110,12 @@ ixgbe_recv_pkts_lro(void *rx_queue, struct
> > rte_mbuf **rx_pkts, uint16_t nb_pkts,
> >  		if (!(staterr & IXGBE_RXDADV_STAT_DD))
> >  			break;
> >
> > +		/*
> > +		 * Use acquire fence to ensure that status_error which
> includes
> > +		 * DD bit is loaded before loading of other descriptor words.
> > +		 */
> > +		rte_atomic_thread_fence(__ATOMIC_ACQUIRE);
> > +
> >  		rxd =3D *rxdp;
> >
> >  		PMD_RX_LOG(DEBUG, "port_id=3D%u queue_id=3D%u rx_id=3D%u "
> > --
> > 2.31.1
>=20
> Reviewed-by: Ruifeng Wang <ruifeng.wang@arm.com>

Applied to dpdk-next-net-intel.

Thanks
Qi