From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from dpdk.org (dpdk.org [92.243.14.124]) by inbox.dpdk.org (Postfix) with ESMTP id 39558A00E6 for ; Tue, 9 Jul 2019 06:43:23 +0200 (CEST) Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id 078B03237; Tue, 9 Jul 2019 06:43:22 +0200 (CEST) Received: from EUR01-HE1-obe.outbound.protection.outlook.com (mail-eopbgr130072.outbound.protection.outlook.com [40.107.13.72]) by dpdk.org (Postfix) with ESMTP id C66FB1D7 for ; Tue, 9 Jul 2019 06:43:20 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=armh.onmicrosoft.com; s=selector2-armh-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=NAC4TfefA/JhB22bAbz2amK4KgsZr5wd965JPFrpDgI=; b=Arag3uNIYZPaqkhZIzOC7d4VDUYtJ5REpuLeJ+CMor35oVX0Q7a5n//ezOlb0ABTgI0yLoX8Sy0zb1FK4obvA8vJi3jpIYrpY0BTsCEp7fN+ZrmhGrbIysExrhYfnP9wF/4jjr/DkhMfcrC/GIh+7jrMHMuHSTWedU9/qwLvoVA= Received: from VE1PR08MB5149.eurprd08.prod.outlook.com (20.179.30.152) by VE1PR08MB4733.eurprd08.prod.outlook.com (10.255.112.14) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.2052.19; Tue, 9 Jul 2019 04:43:19 +0000 Received: from VE1PR08MB5149.eurprd08.prod.outlook.com ([fe80::a89e:33:fbda:ed35]) by VE1PR08MB5149.eurprd08.prod.outlook.com ([fe80::a89e:33:fbda:ed35%4]) with mapi id 15.20.2052.020; Tue, 9 Jul 2019 04:43:19 +0000 From: Honnappa Nagarahalli To: "Ruifeng Wang (Arm Technology China)" , "vladimir.medvedkin@intel.com" , "bruce.richardson@intel.com" CC: "dev@dpdk.org" , "Gavin Hu (Arm Technology China)" , Honnappa Nagarahalli , nd , nd Thread-Topic: [PATCH v4 3/3] lib/lpm: use atomic store to avoid partial update Thread-Index: AQHVMWJ8xcVaFMB+nUu7+JOs7yV9CqbALtOggAATfYCAAKQOUA== Date: Tue, 9 Jul 2019 04:43:19 +0000 Message-ID: References: <20190703054441.30162-1-ruifeng.wang@arm.com> <20190703054441.30162-3-ruifeng.wang@arm.com> In-Reply-To: Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-ts-tracking-id: bc3abaab-0bd1-4040-979d-2ed898bced9a.0 x-checkrecipientchecked: true authentication-results: spf=none (sender IP is ) smtp.mailfrom=Honnappa.Nagarahalli@arm.com; x-originating-ip: [217.140.111.135] x-ms-publictraffictype: Email x-ms-office365-filtering-correlation-id: 328d60be-c014-4304-b08b-08d70427f72c x-ms-office365-filtering-ht: Tenant x-microsoft-antispam: BCL:0; PCL:0; RULEID:(2390118)(7020095)(4652040)(8989299)(4534185)(4627221)(201703031133081)(201702281549075)(8990200)(5600148)(711020)(4605104)(1401327)(4618075)(2017052603328)(7193020); SRVR:VE1PR08MB4733; x-ms-traffictypediagnostic: VE1PR08MB4733: x-microsoft-antispam-prvs: nodisclaimer: True x-ms-oob-tlc-oobclassifiers: OLM:8273; x-forefront-prvs: 0093C80C01 x-forefront-antispam-report: SFV:NSPM; SFS:(10009020)(4636009)(136003)(376002)(366004)(346002)(396003)(39860400002)(199004)(189003)(25786009)(8936002)(8676002)(64756008)(33656002)(478600001)(81166006)(66556008)(81156014)(66946007)(2501003)(73956011)(486006)(11346002)(446003)(66446008)(52536014)(476003)(76116006)(68736007)(66476007)(4326008)(6116002)(3846002)(5660300002)(14454004)(2201001)(53936002)(86362001)(71190400001)(54906003)(71200400001)(76176011)(14444005)(6246003)(110136005)(6436002)(66066001)(7696005)(6506007)(99286004)(256004)(2906002)(229853002)(9686003)(74316002)(55016002)(102836004)(26005)(15650500001)(305945005)(7736002)(72206003)(186003)(316002); DIR:OUT; SFP:1101; SCL:1; SRVR:VE1PR08MB4733; H:VE1PR08MB5149.eurprd08.prod.outlook.com; FPR:; SPF:None; LANG:en; PTR:InfoNoRecords; A:1; MX:1; received-spf: None (protection.outlook.com: arm.com does not designate permitted sender hosts) x-ms-exchange-senderadcheck: 1 x-microsoft-antispam-message-info: cfq46zOp3ABLhvd6JWdWUadhOYi7qClZyUy8xy3KskcjR/rOKqOXud9jqgmU1CXNRwtfXrv3SCiNvdd+4ZuRgcDpnPbIcTqHx/WA0jkrsh+8aGg3pFSb5/JabGd7ixJ/GYkxQFhh73qeTAS3/YmcVz8PsUV/PA/a4k4rZwTSnYscFvgdGRJomikISjIuOALBdBFha55ExF9nxkYnFp1iqdI8R0YDEV+6ni//m6ghjvOuH4WixvMVTIAyvBYA0liNsLsfkLguzjL/Ge6hL00JG7cPY2DTn636HRc0ZIb9Fzz721U8LsE1VBGd9wfQirmKirETtdjlX0LHYmtcaAF2LYOhrn/EhsBexbBe1ghMRc/7gpIeo/lnCMNbP7E/g2npl861y269ZsfUodJBonq6nlXfQvo+rTOWDceWj9bz+QU= Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 X-OriginatorOrg: arm.com X-MS-Exchange-CrossTenant-Network-Message-Id: 328d60be-c014-4304-b08b-08d70427f72c X-MS-Exchange-CrossTenant-originalarrivaltime: 09 Jul 2019 04:43:19.1632 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Hosted X-MS-Exchange-CrossTenant-id: f34e5979-57d9-4aaa-ad4d-b122a662184d X-MS-Exchange-CrossTenant-mailboxtype: HOSTED X-MS-Exchange-CrossTenant-userprincipalname: Honnappa.Nagarahalli@arm.com X-MS-Exchange-Transport-CrossTenantHeadersStamped: VE1PR08MB4733 Subject: Re: [dpdk-dev] [PATCH v4 3/3] lib/lpm: use atomic store to avoid partial update X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" > > > > > > > > Compiler could generate non-atomic stores for whole table entry updat= ing. > > > This may cause incorrect nexthop to be returned, if the byte with > > > valid flag is updated prior to the byte with next hot is updated. > > ^^^^^^^ > > Should be nexthop > > > > > > > > Changed to use atomic store to update whole table entry. > > > > > > Suggested-by: Medvedkin Vladimir > > > Signed-off-by: Ruifeng Wang > > > Reviewed-by: Gavin Hu > > > --- > > > v4: initial version > > > > > > lib/librte_lpm/rte_lpm.c | 34 ++++++++++++++++++++++++---------- > > > 1 file changed, 24 insertions(+), 10 deletions(-) > > > > > > diff --git a/lib/librte_lpm/rte_lpm.c b/lib/librte_lpm/rte_lpm.c > > > index > > > baa6e7460..5d1dbd7e6 100644 > > > --- a/lib/librte_lpm/rte_lpm.c > > > +++ b/lib/librte_lpm/rte_lpm.c > > > @@ -767,7 +767,9 @@ add_depth_small_v20(struct rte_lpm_v20 *lpm, > > > uint32_t ip, uint8_t depth, > > > * Setting tbl8 entry in one go to avoid > > > * race conditions > > > */ > > > - lpm->tbl8[j] =3D new_tbl8_entry; > > > + __atomic_store(&lpm->tbl8[j], > > > + &new_tbl8_entry, > > > + __ATOMIC_RELAXED); > > > > > > continue; > > > } > > > @@ -837,7 +839,9 @@ add_depth_small_v1604(struct rte_lpm *lpm, > > > uint32_t ip, uint8_t depth, > > > * Setting tbl8 entry in one go to avoid > > > * race conditions > > > */ > > > - lpm->tbl8[j] =3D new_tbl8_entry; > > > + __atomic_store(&lpm->tbl8[j], > > > + &new_tbl8_entry, > > > + __ATOMIC_RELAXED); > > > > > > continue; > > > } > > > @@ -965,7 +969,8 @@ add_depth_big_v20(struct rte_lpm_v20 *lpm, > > > uint32_t ip_masked, uint8_t depth, > > > * Setting tbl8 entry in one go to avoid race > > > * condition > > > */ > > > - lpm->tbl8[i] =3D new_tbl8_entry; > > > + __atomic_store(&lpm->tbl8[i], > > > &new_tbl8_entry, > > > + __ATOMIC_RELAXED); > > > > > > continue; > > > } > > > @@ -1100,7 +1105,8 @@ add_depth_big_v1604(struct rte_lpm *lpm, > > > uint32_t ip_masked, uint8_t depth, > > > * Setting tbl8 entry in one go to avoid race > > > * condition > > > */ > > > - lpm->tbl8[i] =3D new_tbl8_entry; > > > + __atomic_store(&lpm->tbl8[i], > > > &new_tbl8_entry, > > > + __ATOMIC_RELAXED); > > > > > > continue; > > > } > > > @@ -1393,7 +1399,9 @@ delete_depth_small_v20(struct rte_lpm_v20 > > *lpm, > > > uint32_t ip_masked, > > > > > > RTE_LPM_TBL8_GROUP_NUM_ENTRIES); j++) { > > > > > > if (lpm->tbl8[j].depth <=3D depth) > > > - lpm->tbl8[j] =3D > > > new_tbl8_entry; > > > + __atomic_store(&lpm- > > >tbl8[j], > > > + &new_tbl8_entry, > > > + __ATOMIC_RELAXED); > > > } > > > } > > > } > > > @@ -1490,7 +1498,9 @@ delete_depth_small_v1604(struct rte_lpm *lpm, > > > uint32_t ip_masked, > > > > > > RTE_LPM_TBL8_GROUP_NUM_ENTRIES); j++) { > > > > > > if (lpm->tbl8[j].depth <=3D depth) > > > - lpm->tbl8[j] =3D > > > new_tbl8_entry; > > > + __atomic_store(&lpm- > > >tbl8[j], > > > + &new_tbl8_entry, > > > + __ATOMIC_RELAXED); > > > } > > > } > > > } > > > @@ -1646,7 +1656,8 @@ delete_depth_big_v20(struct rte_lpm_v20 *lpm, > > > uint32_t ip_masked, > > > */ > > > for (i =3D tbl8_index; i < (tbl8_index + tbl8_range); i++) { > > > if (lpm->tbl8[i].depth <=3D depth) > > > - lpm->tbl8[i] =3D new_tbl8_entry; > > > + __atomic_store(&lpm->tbl8[i], > > > &new_tbl8_entry, > > > + __ATOMIC_RELAXED); > > > } > > > } > > > > > > @@ -1677,7 +1688,8 @@ delete_depth_big_v20(struct rte_lpm_v20 *lpm, > > > uint32_t ip_masked, > > > /* Set tbl24 before freeing tbl8 to avoid race condition. > > > * Prevent the free of the tbl8 group from hoisting. > > > */ > > > - lpm->tbl24[tbl24_index] =3D new_tbl24_entry; > > > + __atomic_store(&lpm->tbl24[tbl24_index], > > > &new_tbl24_entry, > > > + __ATOMIC_RELAXED); > > > __atomic_thread_fence(__ATOMIC_RELEASE); > > > tbl8_free_v20(lpm->tbl8, tbl8_group_start); > > tbl8_alloc_v20/tbl8_free_v20 need to be updated to use __atomic_store > > > tbl8_alloc_v20/tbl8_free_v20 updates a single field of table entry. The p= rocess > is already atomic. Do we really need to use __atomic_store? I thought we agreed that all the tbl8 stores will use __atomic_store. IMO, it is better to use C11 atomic built-ins entirely, at least for the da= ta structures used in reader-writer scenario. Otherwise, the code does not = follow C11 memory model completely. (I do not know what to call such a mode= l). >=20 > > > } > > > @@ -1730,7 +1742,8 @@ delete_depth_big_v1604(struct rte_lpm *lpm, > > > uint32_t ip_masked, > > > */ > > > for (i =3D tbl8_index; i < (tbl8_index + tbl8_range); i++) { > > > if (lpm->tbl8[i].depth <=3D depth) > > > - lpm->tbl8[i] =3D new_tbl8_entry; > > > + __atomic_store(&lpm->tbl8[i], > > > &new_tbl8_entry, > > > + __ATOMIC_RELAXED); > > > } > > > } > > > > > > @@ -1761,7 +1774,8 @@ delete_depth_big_v1604(struct rte_lpm *lpm, > > > uint32_t ip_masked, > > > /* Set tbl24 before freeing tbl8 to avoid race condition. > > > * Prevent the free of the tbl8 group from hoisting. > > > */ > > > - lpm->tbl24[tbl24_index] =3D new_tbl24_entry; > > > + __atomic_store(&lpm->tbl24[tbl24_index], > > > &new_tbl24_entry, > > > + __ATOMIC_RELAXED); > > > __atomic_thread_fence(__ATOMIC_RELEASE); > > > tbl8_free_v1604(lpm->tbl8, tbl8_group_start); > > tbl8_alloc_v1604 /tbl8_free_v1604 need to be updated to use > > __atomic_store > Ditto. >=20 > > > > > } > > > -- > > > 2.17.1