From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from dpdk.org (dpdk.org [92.243.14.124]) by inbox.dpdk.org (Postfix) with ESMTP id 279C9A2EFC for ; Sun, 13 Oct 2019 04:32:03 +0200 (CEST) Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id B0A211D15D; Sun, 13 Oct 2019 04:32:02 +0200 (CEST) Received: from EUR02-HE1-obe.outbound.protection.outlook.com (mail-eopbgr10081.outbound.protection.outlook.com [40.107.1.81]) by dpdk.org (Postfix) with ESMTP id 365511C066; Sun, 13 Oct 2019 04:31:59 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=armh.onmicrosoft.com; s=selector2-armh-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=rQZKiUlWsZmQuVHuVxExXu1DKSNXL45Vmf3zmczAWCs=; b=3YBE9mNsSYrjy9f8z4PN1RfI02ayGuEsqa2VQPn/R1qPUnGtQrP3t4FCVfAcU1TdRrobcF7209GvwKyC6p9/uEmuwJ6NO2GzViXteZUETgxhD7sQjG2nSQVj2FCO10kN8D7YmJBfuTAEyc3vF0M8Yw4F/MVK8L7anInc0h+3ClI= Received: from VI1PR0802CA0031.eurprd08.prod.outlook.com (2603:10a6:800:a9::17) by DB8PR08MB5068.eurprd08.prod.outlook.com (2603:10a6:10:e9::12) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.2347.18; Sun, 13 Oct 2019 02:31:56 +0000 Received: from DB5EUR03FT013.eop-EUR03.prod.protection.outlook.com (2a01:111:f400:7e0a::201) by VI1PR0802CA0031.outlook.office365.com (2603:10a6:800:a9::17) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384) id 15.20.2347.16 via Frontend Transport; Sun, 13 Oct 2019 02:31:55 +0000 Authentication-Results: spf=temperror (sender IP is 63.35.35.123) smtp.mailfrom=arm.com; dpdk.org; dkim=pass (signature was verified) header.d=armh.onmicrosoft.com;dpdk.org; dmarc=none action=none header.from=arm.com; Received-SPF: TempError (protection.outlook.com: error in processing during lookup of arm.com: DNS Timeout) Received: from 64aa7808-outbound-1.mta.getcheckrecipient.com (63.35.35.123) by DB5EUR03FT013.mail.protection.outlook.com (10.152.20.105) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384) id 15.20.2305.15 via Frontend Transport; Sun, 13 Oct 2019 02:31:53 +0000 Received: ("Tessian outbound 0cf06bf5c60e:v33"); Sun, 13 Oct 2019 02:31:51 +0000 X-CR-MTA-TID: 64aa7808 Received: from 101dfe152971.2 (ip-172-16-0-2.eu-west-1.compute.internal [104.47.10.52]) by 64aa7808-outbound-1.mta.getcheckrecipient.com id 627CB19C-F0E5-4AAB-87A3-DD36FE0DA079.1; Sun, 13 Oct 2019 02:31:46 +0000 Received: from EUR03-DB5-obe.outbound.protection.outlook.com (mail-db5eur03lp2052.outbound.protection.outlook.com [104.47.10.52]) by 64aa7808-outbound-1.mta.getcheckrecipient.com with ESMTPS id 101dfe152971.2 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-SHA384); Sun, 13 Oct 2019 02:31:46 +0000 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=n5Xh/WoU2RaCwsN54nKShsmGDCOQYDazQMcj7n081/FZitdd9hGsMVLQTS0Ob7X/hMRBbTB/amvGTop81SS6SiUOoo8LkQJLsbk050M922ysOqhoW3ZHj8tt/MN0nByp13beaQSxuLQjDwFMzOfNJjmwNfTRaqq4rGZHZIKwYz6YxnGuDMPcQc0h1TB4Xpc2LSVUOqZ5ywGXYUlv0kZJHxs7IELDiLFdYgAV5pBBDLqi8fvQiX9Kj5veYMbNid/cFDfU/MDgMHG91vfApVTRFZOq5Z3v0GEmT9bbZYIg3aHO8ij48Q1hYftsWnhdCiklWigpaMR8HeW5FORc3Pt7lA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=rQZKiUlWsZmQuVHuVxExXu1DKSNXL45Vmf3zmczAWCs=; b=hXmjGnJqtJHA8Ojxg6j7FI321V0UhDxmai2EhNFqXxFEMqB1KS8dnlh7UQzcVHFpr6kHGUEhSpYA3P9rlq/+asMNGnjMUixOVe2PCEqolkt3kRA/wMSSlji6/gIupUiZ6d0SnSW4Z1onHe1TFI5uRrBODEFgRPVaHPFiKQ/QWjVmFiEP+IqD/BpzQ6UXf5YYez8ny37o1hyQRx/6IcpXX9HajWMWJL6zltRscfAXY5TrU14k6yGpxuii6dc6X+qSfpHD5XS367TtcfMEjtdBIwHULORNsLBaHJb38DI7mqE4KFt/UFmjv08FpYupiKE5K/YplEMArley9qJmSI6BCA== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=arm.com; dmarc=pass action=none header.from=arm.com; dkim=pass header.d=arm.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=armh.onmicrosoft.com; s=selector2-armh-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=rQZKiUlWsZmQuVHuVxExXu1DKSNXL45Vmf3zmczAWCs=; b=3YBE9mNsSYrjy9f8z4PN1RfI02ayGuEsqa2VQPn/R1qPUnGtQrP3t4FCVfAcU1TdRrobcF7209GvwKyC6p9/uEmuwJ6NO2GzViXteZUETgxhD7sQjG2nSQVj2FCO10kN8D7YmJBfuTAEyc3vF0M8Yw4F/MVK8L7anInc0h+3ClI= Received: from VE1PR08MB5149.eurprd08.prod.outlook.com (20.179.30.27) by VE1PR08MB4894.eurprd08.prod.outlook.com (10.255.113.211) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.2347.21; Sun, 13 Oct 2019 02:31:42 +0000 Received: from VE1PR08MB5149.eurprd08.prod.outlook.com ([fe80::8c82:8d9c:c78d:22a6]) by VE1PR08MB5149.eurprd08.prod.outlook.com ([fe80::8c82:8d9c:c78d:22a6%7]) with mapi id 15.20.2347.021; Sun, 13 Oct 2019 02:31:42 +0000 From: Honnappa Nagarahalli To: "Ruifeng Wang (Arm Technology China)" , "david.hunt@intel.com" CC: "dev@dpdk.org" , "hkalra@marvell.com" , "Gavin Hu (Arm Technology China)" , nd , "Ruifeng Wang (Arm Technology China)" , "stable@dpdk.org" , Honnappa Nagarahalli , nd Thread-Topic: [PATCH v2 1/2] lib/distributor: fix deadlock issue for aarch64 Thread-Index: AQHVgKcY/ihfMZfnjkG8skzlxP8PWqdX2sBg Date: Sun, 13 Oct 2019 02:31:42 +0000 Message-ID: References: <20191008095524.1585-1-ruifeng.wang@arm.com> <20191012024352.23545-1-ruifeng.wang@arm.com> <20191012024352.23545-2-ruifeng.wang@arm.com> In-Reply-To: <20191012024352.23545-2-ruifeng.wang@arm.com> Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-ts-tracking-id: f6c93bcc-c14c-4709-ac0b-4349cf7e32d8.0 x-checkrecipientchecked: true Authentication-Results-Original: spf=none (sender IP is ) smtp.mailfrom=Honnappa.Nagarahalli@arm.com; x-originating-ip: [217.140.111.135] x-ms-publictraffictype: Email X-MS-Office365-Filtering-Correlation-Id: e7fc39a1-6a36-4578-6604-08d74f858298 X-MS-Office365-Filtering-HT: Tenant X-MS-TrafficTypeDiagnostic: VE1PR08MB4894:|VE1PR08MB4894:|DB8PR08MB5068: x-ms-exchange-transport-forked: True X-Microsoft-Antispam-PRVS: x-checkrecipientrouted: true x-ms-oob-tlc-oobclassifiers: OLM:8273;OLM:8273; x-forefront-prvs: 01894AD3B8 X-Forefront-Antispam-Report-Untrusted: SFV:NSPM; SFS:(10009020)(4636009)(39860400002)(136003)(366004)(346002)(376002)(396003)(13464003)(189003)(199004)(2906002)(99286004)(256004)(7696005)(110136005)(76176011)(81156014)(81166006)(26005)(66476007)(66556008)(64756008)(66446008)(4326008)(102836004)(6506007)(53546011)(76116006)(66946007)(25786009)(8676002)(316002)(186003)(8936002)(54906003)(14444005)(478600001)(14454004)(74316002)(55016002)(6116002)(3846002)(71200400001)(305945005)(71190400001)(6246003)(446003)(11346002)(9686003)(229853002)(86362001)(30864003)(486006)(2501003)(5660300002)(66066001)(52536014)(33656002)(476003)(6436002)(7736002); DIR:OUT; SFP:1101; SCL:1; SRVR:VE1PR08MB4894; H:VE1PR08MB5149.eurprd08.prod.outlook.com; FPR:; SPF:None; LANG:en; PTR:InfoNoRecords; MX:1; A:1; received-spf: None (protection.outlook.com: arm.com does not designate permitted sender hosts) X-MS-Exchange-SenderADCheck: 1 X-Microsoft-Antispam-Untrusted: BCL:0; X-Microsoft-Antispam-Message-Info-Original: 1ucYr79uPGsg4EcKNAJJoGxMjPbeo85f0+Gzq7yj5619jEDr5iEnPueTAa/OgsvgF/7xk0/5XJFk/RVuqskYGC5JkhM8lkjBD3Obn41vUgUct3e5DhzXQBOBzbKe7+/WHugRS2YqitomfHsMEGV81l5OOWnkpigRlWdsasBjTawg2PSz6widAL46rV2REdyAkDMbTdSiZMbsdggCvfGGvJ5Ntw+On1WpgJk0DXpfK+K9BX1TMbP3vjyR80+Oegzq3bSWdETNrgTRHMTf+4Ij5Ogj4qbD7cRwdea/goO2wAyMqATE1/x0mFXRtmwiydJE63U/Um1hTFFyi822foepJojrVqhlc7Z4CoRBsABrPp0MxzargGyOCcYSjujLtNVDyB9n5h13avwhR4tTq+/yUtiiTIT3QnBKWQbJA4mfWeo= Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 X-MS-Exchange-Transport-CrossTenantHeadersStamped: VE1PR08MB4894 Original-Authentication-Results: spf=none (sender IP is ) smtp.mailfrom=Honnappa.Nagarahalli@arm.com; X-EOPAttributedMessage: 0 X-MS-Exchange-Transport-CrossTenantHeadersStripped: DB5EUR03FT013.eop-EUR03.prod.protection.outlook.com X-Forefront-Antispam-Report: CIP:63.35.35.123; IPV:CAL; SCL:-1; CTRY:IE; EFV:NLI; SFV:NSPM; SFS:(10009020)(4636009)(346002)(39860400002)(396003)(376002)(136003)(199004)(189003)(13464003)(6506007)(229853002)(9686003)(478600001)(81156014)(81166006)(8676002)(33656002)(316002)(110136005)(55016002)(486006)(476003)(54906003)(6246003)(3846002)(22756006)(26005)(102836004)(23726003)(186003)(336012)(70206006)(6116002)(86362001)(76130400001)(70586007)(305945005)(446003)(53546011)(11346002)(97756001)(63350400001)(74316002)(76176011)(7736002)(126002)(7696005)(26826003)(14444005)(2906002)(66066001)(14454004)(4326008)(99286004)(47776003)(25786009)(50466002)(52536014)(8936002)(8746002)(356004)(30864003)(2501003)(46406003)(5660300002)(450100002); DIR:OUT; SFP:1101; SCL:1; SRVR:DB8PR08MB5068; H:64aa7808-outbound-1.mta.getcheckrecipient.com; FPR:; SPF:TempError; LANG:en; PTR:ec2-63-35-35-123.eu-west-1.compute.amazonaws.com; A:1; MX:1; X-MS-Office365-Filtering-Correlation-Id-Prvs: 06d20ab7-884b-49ea-a9e5-08d74f857c01 NoDisclaimer: True X-Forefront-PRVS: 01894AD3B8 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: kjh1aIeo+1RfVYSS6GlnNMsX3ST/e4oKRq454DsS9j++ktdUiyg9+ZtWEvM8X8dEBODnKzjerFjVzqOD+PPM61YSZPwBazteo1QN9lwYqbPC2SvnnkvTCK5LvCjeHDqRA6HbjDg7HC4p0gEPaIZAQG2izxU/5vbqqw3A2xgVayqcL6kPE8agqFPszTNvwwJ9ZfXorHjRDi58yxebkHhfZlav65lcrF24LzlZH95nB9Enq6fd/LU5NyiP7Z2nXCqK8dQnk+fUJER8N0oYPVIku5jG+70ELsVr4+Ld+HjdsNsZ3U7KUQEBYZ+5x5olq7drj8AsOQZUxSvD/TZ3zIuSDRQar1lNrYEZrzQeDejx/4EeR7mpP0EE6Hd9v7ZmOq7Ugh5V6TI/zbcs+N4Yb2k2lIP6D7MoolcM+lodZXpr7b4= X-OriginatorOrg: arm.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 13 Oct 2019 02:31:53.5634 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: e7fc39a1-6a36-4578-6604-08d74f858298 X-MS-Exchange-CrossTenant-Id: f34e5979-57d9-4aaa-ad4d-b122a662184d X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=f34e5979-57d9-4aaa-ad4d-b122a662184d; Ip=[63.35.35.123]; Helo=[64aa7808-outbound-1.mta.getcheckrecipient.com] X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: DB8PR08MB5068 Subject: Re: [dpdk-stable] [PATCH v2 1/2] lib/distributor: fix deadlock issue for aarch64 X-BeenThere: stable@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: patches for DPDK stable branches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: stable-bounces@dpdk.org Sender: "stable" Hi Ruifeng, Typically, we have followed the convention of adding comments whenever C11= atomic APIs are used. Can you please add comments indicating why acquire o= r release semantics are used? > -----Original Message----- > From: Ruifeng Wang > Sent: Friday, October 11, 2019 9:44 PM > To: david.hunt@intel.com > Cc: dev@dpdk.org; hkalra@marvell.com; Gavin Hu (Arm Technology China) > ; Honnappa Nagarahalli > ; nd ; Ruifeng Wang (Arm > Technology China) ; stable@dpdk.org > Subject: [PATCH v2 1/2] lib/distributor: fix deadlock issue for aarch64 >=20 > Distributor and worker threads rely on data structs in cache line for > synchronization. The shared data structs were not protected. > This caused deadlock issue on weaker memory ordering platforms as aarch64= . > Fix this issue by adding memory barriers to ensure synchronization among > cores. >=20 > Bugzilla ID: 342 > Fixes: 775003ad2f96 ("distributor: add new burst-capable library") > Cc: stable@dpdk.org >=20 > Signed-off-by: Ruifeng Wang > Reviewed-by: Gavin Hu > --- > lib/librte_distributor/meson.build | 5 ++ > lib/librte_distributor/rte_distributor.c | 39 ++++++++++------ > lib/librte_distributor/rte_distributor_v20.c | 49 +++++++++++++------- > 3 files changed, 63 insertions(+), 30 deletions(-) >=20 > diff --git a/lib/librte_distributor/meson.build > b/lib/librte_distributor/meson.build > index dba7e3b2a..26577dbc1 100644 > --- a/lib/librte_distributor/meson.build > +++ b/lib/librte_distributor/meson.build > @@ -9,3 +9,8 @@ else > endif > headers =3D files('rte_distributor.h') > deps +=3D ['mbuf'] > + > +# for clang 32-bit compiles we need libatomic for 64-bit atomic ops if > +cc.get_id() =3D=3D 'clang' and dpdk_conf.get('RTE_ARCH_64') =3D=3D false > + ext_deps +=3D cc.find_library('atomic') > +endif > diff --git a/lib/librte_distributor/rte_distributor.c > b/lib/librte_distributor/rte_distributor.c > index 21eb1fb0a..b653146d0 100644 > --- a/lib/librte_distributor/rte_distributor.c > +++ b/lib/librte_distributor/rte_distributor.c > @@ -50,7 +50,8 @@ rte_distributor_request_pkt_v1705(struct > rte_distributor *d, >=20 > retptr64 =3D &(buf->retptr64[0]); > /* Spin while handshake bits are set (scheduler clears it) */ > - while (unlikely(*retptr64 & RTE_DISTRIB_GET_BUF)) { > + while (unlikely(__atomic_load_n(retptr64, __ATOMIC_ACQUIRE) > + & RTE_DISTRIB_GET_BUF)) { > rte_pause(); > uint64_t t =3D rte_rdtsc()+100; >=20 > @@ -76,7 +77,8 @@ rte_distributor_request_pkt_v1705(struct > rte_distributor *d, > * Finally, set the GET_BUF to signal to distributor that cache > * line is ready for processing > */ > - *retptr64 |=3D RTE_DISTRIB_GET_BUF; > + __atomic_store_n(retptr64, *retptr64 | RTE_DISTRIB_GET_BUF, > + __ATOMIC_RELEASE); > } > BIND_DEFAULT_SYMBOL(rte_distributor_request_pkt, _v1705, 17.05); > MAP_STATIC_SYMBOL(void rte_distributor_request_pkt(struct rte_distributor > *d, @@ -99,7 +101,8 @@ rte_distributor_poll_pkt_v1705(struct > rte_distributor *d, > } >=20 > /* If bit is set, return */ > - if (buf->bufptr64[0] & RTE_DISTRIB_GET_BUF) > + if (__atomic_load_n(&(buf->bufptr64[0]), __ATOMIC_ACQUIRE) > + & RTE_DISTRIB_GET_BUF) > return -1; >=20 > /* since bufptr64 is signed, this should be an arithmetic shift */ @@ - > 115,7 +118,8 @@ rte_distributor_poll_pkt_v1705(struct rte_distributor *d, > * mbuf pointers, so toggle the bit so scheduler can start working > * on the next cacheline while we're working. > */ > - buf->bufptr64[0] |=3D RTE_DISTRIB_GET_BUF; > + __atomic_store_n(&(buf->bufptr64[0]), > + buf->bufptr64[0] | RTE_DISTRIB_GET_BUF, > __ATOMIC_RELEASE); >=20 > return count; > } > @@ -174,6 +178,7 @@ rte_distributor_return_pkt_v1705(struct > rte_distributor *d, > return -EINVAL; > } >=20 > + __atomic_thread_fence(__ATOMIC_ACQUIRE); > for (i =3D 0; i < RTE_DIST_BURST_SIZE; i++) > /* Switch off the return bit first */ > buf->retptr64[i] &=3D ~RTE_DISTRIB_RETURN_BUF; @@ -183,7 > +188,8 @@ rte_distributor_return_pkt_v1705(struct rte_distributor *d, > RTE_DISTRIB_FLAG_BITS) | > RTE_DISTRIB_RETURN_BUF; >=20 > /* set the GET_BUF but even if we got no returns */ > - buf->retptr64[0] |=3D RTE_DISTRIB_GET_BUF; > + __atomic_store_n(&(buf->retptr64[0]), > + buf->retptr64[0] | RTE_DISTRIB_GET_BUF, > __ATOMIC_RELEASE); >=20 > return 0; > } > @@ -273,7 +279,8 @@ handle_returns(struct rte_distributor *d, unsigned > int wkr) > unsigned int count =3D 0; > unsigned int i; >=20 > - if (buf->retptr64[0] & RTE_DISTRIB_GET_BUF) { > + if (__atomic_load_n(&(buf->retptr64[0]), __ATOMIC_ACQUIRE) > + & RTE_DISTRIB_GET_BUF) { > for (i =3D 0; i < RTE_DIST_BURST_SIZE; i++) { > if (buf->retptr64[i] & RTE_DISTRIB_RETURN_BUF) { > oldbuf =3D ((uintptr_t)(buf->retptr64[i] >> @@ > -287,7 +294,7 @@ handle_returns(struct rte_distributor *d, unsigned int w= kr) > d->returns.start =3D ret_start; > d->returns.count =3D ret_count; > /* Clear for the worker to populate with more returns */ > - buf->retptr64[0] =3D 0; > + __atomic_store_n(&(buf->retptr64[0]), 0, > __ATOMIC_RELEASE); > } > return count; > } > @@ -307,7 +314,8 @@ release(struct rte_distributor *d, unsigned int wkr) > struct rte_distributor_buffer *buf =3D &(d->bufs[wkr]); > unsigned int i; >=20 > - while (!(d->bufs[wkr].bufptr64[0] & RTE_DISTRIB_GET_BUF)) > + while (!(__atomic_load_n(&(d->bufs[wkr].bufptr64[0]), > __ATOMIC_ACQUIRE) > + & RTE_DISTRIB_GET_BUF)) > rte_pause(); >=20 > handle_returns(d, wkr); > @@ -328,7 +336,8 @@ release(struct rte_distributor *d, unsigned int wkr) > d->backlog[wkr].count =3D 0; >=20 > /* Clear the GET bit */ > - buf->bufptr64[0] &=3D ~RTE_DISTRIB_GET_BUF; > + __atomic_store_n(&(buf->bufptr64[0]), > + buf->bufptr64[0] & ~RTE_DISTRIB_GET_BUF, > __ATOMIC_RELEASE); > return buf->count; >=20 > } > @@ -355,7 +364,8 @@ rte_distributor_process_v1705(struct rte_distributor > *d, > if (unlikely(num_mbufs =3D=3D 0)) { > /* Flush out all non-full cache-lines to workers. */ > for (wid =3D 0 ; wid < d->num_workers; wid++) { > - if (d->bufs[wid].bufptr64[0] & RTE_DISTRIB_GET_BUF) > { > + if (__atomic_load_n(&(d->bufs[wid].bufptr64[0]), > + __ATOMIC_ACQUIRE) & > RTE_DISTRIB_GET_BUF) { > release(d, wid); > handle_returns(d, wid); > } > @@ -367,7 +377,8 @@ rte_distributor_process_v1705(struct rte_distributor > *d, > uint16_t matches[RTE_DIST_BURST_SIZE]; > unsigned int pkts; >=20 > - if (d->bufs[wkr].bufptr64[0] & RTE_DISTRIB_GET_BUF) > + if (__atomic_load_n(&(d->bufs[wkr].bufptr64[0]), > + __ATOMIC_ACQUIRE) & RTE_DISTRIB_GET_BUF) > d->bufs[wkr].count =3D 0; >=20 > if ((num_mbufs - next_idx) < RTE_DIST_BURST_SIZE) @@ - > 465,7 +476,8 @@ rte_distributor_process_v1705(struct rte_distributor *d, >=20 > /* Flush out all non-full cache-lines to workers. */ > for (wid =3D 0 ; wid < d->num_workers; wid++) > - if ((d->bufs[wid].bufptr64[0] & RTE_DISTRIB_GET_BUF)) > + if ((__atomic_load_n(&(d->bufs[wid].bufptr64[0]), > + __ATOMIC_ACQUIRE) & RTE_DISTRIB_GET_BUF)) > release(d, wid); >=20 > return num_mbufs; > @@ -574,7 +586,8 @@ rte_distributor_clear_returns_v1705(struct > rte_distributor *d) >=20 > /* throw away returns, so workers can exit */ > for (wkr =3D 0; wkr < d->num_workers; wkr++) > - d->bufs[wkr].retptr64[0] =3D 0; > + __atomic_store_n(&(d->bufs[wkr].retptr64[0]), 0, > + __ATOMIC_RELEASE); > } > BIND_DEFAULT_SYMBOL(rte_distributor_clear_returns, _v1705, 17.05); > MAP_STATIC_SYMBOL(void rte_distributor_clear_returns(struct > rte_distributor *d), diff --git a/lib/librte_distributor/rte_distributor_= v20.c > b/lib/librte_distributor/rte_distributor_v20.c > index cdc0969a8..41411e3c1 100644 > --- a/lib/librte_distributor/rte_distributor_v20.c > +++ b/lib/librte_distributor/rte_distributor_v20.c > @@ -34,9 +34,10 @@ rte_distributor_request_pkt_v20(struct > rte_distributor_v20 *d, > union rte_distributor_buffer_v20 *buf =3D &d->bufs[worker_id]; > int64_t req =3D (((int64_t)(uintptr_t)oldpkt) << RTE_DISTRIB_FLAG_BITS) > | RTE_DISTRIB_GET_BUF; > - while (unlikely(buf->bufptr64 & RTE_DISTRIB_FLAGS_MASK)) > + while (unlikely(__atomic_load_n(&(buf->bufptr64), > __ATOMIC_ACQUIRE) > + & RTE_DISTRIB_FLAGS_MASK)) > rte_pause(); > - buf->bufptr64 =3D req; > + __atomic_store_n(&(buf->bufptr64), req, __ATOMIC_RELEASE); > } > VERSION_SYMBOL(rte_distributor_request_pkt, _v20, 2.0); >=20 > @@ -45,7 +46,8 @@ rte_distributor_poll_pkt_v20(struct rte_distributor_v20 > *d, > unsigned worker_id) > { > union rte_distributor_buffer_v20 *buf =3D &d->bufs[worker_id]; > - if (buf->bufptr64 & RTE_DISTRIB_GET_BUF) > + if (__atomic_load_n(&(buf->bufptr64), __ATOMIC_ACQUIRE) > + & RTE_DISTRIB_GET_BUF) > return NULL; >=20 > /* since bufptr64 is signed, this should be an arithmetic shift */ @@ - > 73,7 +75,7 @@ rte_distributor_return_pkt_v20(struct rte_distributor_v20 *= d, > union rte_distributor_buffer_v20 *buf =3D &d->bufs[worker_id]; > uint64_t req =3D (((int64_t)(uintptr_t)oldpkt) << > RTE_DISTRIB_FLAG_BITS) > | RTE_DISTRIB_RETURN_BUF; > - buf->bufptr64 =3D req; > + __atomic_store_n(&(buf->bufptr64), req, __ATOMIC_RELEASE); > return 0; > } > VERSION_SYMBOL(rte_distributor_return_pkt, _v20, 2.0); @@ -117,7 +119,7 > @@ handle_worker_shutdown(struct rte_distributor_v20 *d, unsigned int > wkr) { > d->in_flight_tags[wkr] =3D 0; > d->in_flight_bitmask &=3D ~(1UL << wkr); > - d->bufs[wkr].bufptr64 =3D 0; > + __atomic_store_n(&(d->bufs[wkr].bufptr64), 0, __ATOMIC_RELEASE); > if (unlikely(d->backlog[wkr].count !=3D 0)) { > /* On return of a packet, we need to move the > * queued packets for this core elsewhere. > @@ -165,18 +167,23 @@ process_returns(struct rte_distributor_v20 *d) > const int64_t data =3D d->bufs[wkr].bufptr64; > uintptr_t oldbuf =3D 0; >=20 > - if (data & RTE_DISTRIB_GET_BUF) { > + if (__atomic_load_n(&data, __ATOMIC_ACQUIRE) > + & RTE_DISTRIB_GET_BUF) { > flushed++; > if (d->backlog[wkr].count) > - d->bufs[wkr].bufptr64 =3D > - backlog_pop(&d- > >backlog[wkr]); > + __atomic_store_n(&(d->bufs[wkr].bufptr64), > + backlog_pop(&d->backlog[wkr]), > + __ATOMIC_RELEASE); > else { > - d->bufs[wkr].bufptr64 =3D > RTE_DISTRIB_GET_BUF; > + __atomic_store_n(&(d->bufs[wkr].bufptr64), > + RTE_DISTRIB_GET_BUF, > + __ATOMIC_RELEASE); > d->in_flight_tags[wkr] =3D 0; > d->in_flight_bitmask &=3D ~(1UL << wkr); > } > oldbuf =3D data >> RTE_DISTRIB_FLAG_BITS; > - } else if (data & RTE_DISTRIB_RETURN_BUF) { > + } else if (__atomic_load_n(&data, __ATOMIC_ACQUIRE) > + & RTE_DISTRIB_RETURN_BUF) { > handle_worker_shutdown(d, wkr); > oldbuf =3D data >> RTE_DISTRIB_FLAG_BITS; > } > @@ -251,21 +258,26 @@ rte_distributor_process_v20(struct > rte_distributor_v20 *d, > } > } >=20 > - if ((data & RTE_DISTRIB_GET_BUF) && > + if ((__atomic_load_n(&data, __ATOMIC_ACQUIRE) > + & RTE_DISTRIB_GET_BUF) && > (d->backlog[wkr].count || next_mb)) { >=20 > if (d->backlog[wkr].count) > - d->bufs[wkr].bufptr64 =3D > - backlog_pop(&d- > >backlog[wkr]); > + __atomic_store_n(&(d->bufs[wkr].bufptr64), > + backlog_pop(&d- > >backlog[wkr]), > + __ATOMIC_RELEASE); >=20 > else { > - d->bufs[wkr].bufptr64 =3D next_value; > + __atomic_store_n(&(d->bufs[wkr].bufptr64), > + next_value, > + __ATOMIC_RELEASE); > d->in_flight_tags[wkr] =3D new_tag; > d->in_flight_bitmask |=3D (1UL << wkr); > next_mb =3D NULL; > } > oldbuf =3D data >> RTE_DISTRIB_FLAG_BITS; > - } else if (data & RTE_DISTRIB_RETURN_BUF) { > + } else if (__atomic_load_n(&data, __ATOMIC_ACQUIRE) > + & RTE_DISTRIB_RETURN_BUF) { > handle_worker_shutdown(d, wkr); > oldbuf =3D data >> RTE_DISTRIB_FLAG_BITS; > } > @@ -280,13 +292,16 @@ rte_distributor_process_v20(struct > rte_distributor_v20 *d, > * if they are ready */ > for (wkr =3D 0; wkr < d->num_workers; wkr++) > if (d->backlog[wkr].count && > - (d->bufs[wkr].bufptr64 & > RTE_DISTRIB_GET_BUF)) { > + (__atomic_load_n(&(d->bufs[wkr].bufptr64), > + __ATOMIC_ACQUIRE) & > RTE_DISTRIB_GET_BUF)) { >=20 > int64_t oldbuf =3D d->bufs[wkr].bufptr64 >> > RTE_DISTRIB_FLAG_BITS; > store_return(oldbuf, d, &ret_start, &ret_count); >=20 > - d->bufs[wkr].bufptr64 =3D backlog_pop(&d- > >backlog[wkr]); > + __atomic_store_n(&(d->bufs[wkr].bufptr64), > + backlog_pop(&d->backlog[wkr]), > + __ATOMIC_RELEASE); > } >=20 > d->returns.start =3D ret_start; > -- > 2.17.1