From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from dpdk.org (dpdk.org [92.243.14.124]) by inbox.dpdk.org (Postfix) with ESMTP id 5474FA2EDB for ; Fri, 6 Sep 2019 21:45:12 +0200 (CEST) Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id 2ADE31F40D; Fri, 6 Sep 2019 21:45:12 +0200 (CEST) Received: from EUR02-AM5-obe.outbound.protection.outlook.com (mail-eopbgr00050.outbound.protection.outlook.com [40.107.0.50]) by dpdk.org (Postfix) with ESMTP id 1AEA71F40A for ; Fri, 6 Sep 2019 21:45:11 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=armh.onmicrosoft.com; s=selector2-armh-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=+J2zapcptknXL2hmxy7dS+8LDJPcKmi+mEa2wHbajpo=; b=F65LZ4dMEZSX6FPZzz8dbgXhSAYIHK/8T5wBcDE5r5MyLjhC1lV8R3A8p7uDczj9pf+yxtxR5QQ3oeQwpX4lXtjMjq+mUqnWOYg1nMXOtShSy0uL3JJFDqA8FmtIm887uNkzLnFNrj7qZFcCkCo7h01of98lv80LEJFV+XcLcSs= Received: from VI1PR08CA0127.eurprd08.prod.outlook.com (2603:10a6:800:d4::29) by AM0PR08MB5361.eurprd08.prod.outlook.com (2603:10a6:208:180::9) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.2241.15; Fri, 6 Sep 2019 19:45:08 +0000 Received: from AM5EUR03FT058.eop-EUR03.prod.protection.outlook.com (2a01:111:f400:7e08::205) by VI1PR08CA0127.outlook.office365.com (2603:10a6:800:d4::29) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.2241.14 via Frontend Transport; Fri, 6 Sep 2019 19:45:08 +0000 Authentication-Results: spf=temperror (sender IP is 63.35.35.123) smtp.mailfrom=arm.com; dpdk.org; dkim=pass (signature was verified) header.d=armh.onmicrosoft.com;dpdk.org; dmarc=temperror action=none header.from=arm.com; Received-SPF: TempError (protection.outlook.com: error in processing during lookup of arm.com: DNS Timeout) Received: from 64aa7808-outbound-1.mta.getcheckrecipient.com (63.35.35.123) by AM5EUR03FT058.mail.protection.outlook.com (10.152.17.48) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.2241.14 via Frontend Transport; Fri, 6 Sep 2019 19:45:06 +0000 Received: ("Tessian outbound d33df262a6a7:v27"); Fri, 06 Sep 2019 19:45:02 +0000 X-CR-MTA-TID: 64aa7808 Received: from 3bd440417744.3 (cr-mta-lb-1.cr-mta-net [104.47.9.59]) by 64aa7808-outbound-1.mta.getcheckrecipient.com id A6C6A4C1-E11E-47D8-9232-D3129DC26159.1; Fri, 06 Sep 2019 19:44:57 +0000 Received: from EUR03-VE1-obe.outbound.protection.outlook.com (mail-ve1eur03lp2059.outbound.protection.outlook.com [104.47.9.59]) by 64aa7808-outbound-1.mta.getcheckrecipient.com with ESMTPS id 3bd440417744.3 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384); Fri, 06 Sep 2019 19:44:57 +0000 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=FS9FGmEThHczJhDd7fRULfui6hjxp50Mv/ky3h5HdvJxrcqdBkSf6/7AtKJktg/uqMsDMstva5fC1ri9Fh4iw7No7KXUPwpRTavEnzfWIEwcWXtdf91DxnYph1AxywNL4Hlh0cnGyzsAOJC+nc8n7hp4tiCI6F1wx2ap1au8HCszELwHDNk4KT13sWvYib0ePtGsrZdnBdWJjvX/CZrIz9w1AbZIV3pZKKebHMtpAXNjSGGoEGxDdpVKceXf89IEHaffhf1uTViL8HssLVYfPwQ5XDaNBdf1loCq/SJz2ggKLsYfKmPf3t/BOL7IoVKnlhmwWMTQhc0g35p68SB6Mg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=+J2zapcptknXL2hmxy7dS+8LDJPcKmi+mEa2wHbajpo=; b=lpDkpjM2W8TqYjoOPMImH0vHxcozuCv0zi7y1XOD5GK7cebY6l0yqLYsQSlS4w4ja69MQ6Fvl4LYrKzEstA6wupq4rbDjt/7XjB+o2sz68bObs44x0Uszk1b/t0V1DQSgR7yWSYe2JtskWxR8C19X/Evo5gmEQzeCBiJXK1UExnC7jfIt0qLcw4ShR2sQUib5MzWea0vzJNEWFglyWkZa6vBeizCv8paPMOKNmikV2KCIlTiCISNY1GcFVuhbJSMVC+h5aCm+dp6yVZKxStWbNbXPayumD54ZvfX/hEH+OqbbEoUumnKzZgFCjKdkP36dn3WyacLL9KTsEAC11BCdQ== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=arm.com; dmarc=pass action=none header.from=arm.com; dkim=pass header.d=arm.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=armh.onmicrosoft.com; s=selector2-armh-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=+J2zapcptknXL2hmxy7dS+8LDJPcKmi+mEa2wHbajpo=; b=F65LZ4dMEZSX6FPZzz8dbgXhSAYIHK/8T5wBcDE5r5MyLjhC1lV8R3A8p7uDczj9pf+yxtxR5QQ3oeQwpX4lXtjMjq+mUqnWOYg1nMXOtShSy0uL3JJFDqA8FmtIm887uNkzLnFNrj7qZFcCkCo7h01of98lv80LEJFV+XcLcSs= Received: from VE1PR08MB5149.eurprd08.prod.outlook.com (20.179.30.152) by VE1PR08MB4976.eurprd08.prod.outlook.com (10.255.158.145) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.2241.18; Fri, 6 Sep 2019 19:44:55 +0000 Received: from VE1PR08MB5149.eurprd08.prod.outlook.com ([fe80::a8af:a9b8:4597:4128]) by VE1PR08MB5149.eurprd08.prod.outlook.com ([fe80::a8af:a9b8:4597:4128%3]) with mapi id 15.20.2178.023; Fri, 6 Sep 2019 19:44:55 +0000 From: Honnappa Nagarahalli To: "Ruifeng Wang (Arm Technology China)" , "bruce.richardson@intel.com" , "vladimir.medvedkin@intel.com" , "olivier.matz@6wind.com" CC: "dev@dpdk.org" , "stephen@networkplumber.org" , "konstantin.ananyev@intel.com" , "Gavin Hu (Arm Technology China)" , Dharmik Thakkar , nd , "Ruifeng Wang (Arm Technology China)" , "paulmck@linux.ibm.com" , nd Thread-Topic: [PATCH v2 3/6] lib/lpm: integrate RCU QSBR Thread-Index: AQHVZOuOVB2Rjprc1kGY8bizM9HAfw== Date: Fri, 6 Sep 2019 19:44:55 +0000 Message-ID: References: <20190822063457.41596-1-ruifeng.wang@arm.com> <20190906094534.36060-1-ruifeng.wang@arm.com> <20190906094534.36060-4-ruifeng.wang@arm.com> In-Reply-To: <20190906094534.36060-4-ruifeng.wang@arm.com> Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-ts-tracking-id: 1e72f0dc-4e58-4f74-bce1-57639dbb7ae4.0 x-checkrecipientchecked: true Authentication-Results-Original: spf=none (sender IP is ) smtp.mailfrom=Honnappa.Nagarahalli@arm.com; x-originating-ip: [217.140.111.135] x-ms-publictraffictype: Email X-MS-Office365-Filtering-Correlation-Id: bd9c5781-9821-4312-7672-08d73302b809 X-MS-Office365-Filtering-HT: Tenant X-Microsoft-Antispam-Untrusted: BCL:0; PCL:0; RULEID:(2390118)(7020095)(4652040)(8989299)(4534185)(4627221)(201703031133081)(201702281549075)(8990200)(5600166)(711020)(4605104)(1401327)(4618075)(2017052603328)(7193020); SRVR:VE1PR08MB4976; X-MS-TrafficTypeDiagnostic: VE1PR08MB4976:|VE1PR08MB4976:|AM0PR08MB5361: x-ms-exchange-transport-forked: True X-Microsoft-Antispam-PRVS: x-checkrecipientrouted: true x-ms-oob-tlc-oobclassifiers: OLM:8273;OLM:8273; x-forefront-prvs: 0152EBA40F X-Forefront-Antispam-Report-Untrusted: SFV:NSPM; SFS:(10009020)(4636009)(346002)(136003)(366004)(376002)(39860400002)(396003)(199004)(189003)(13464003)(4326008)(2201001)(86362001)(66066001)(5660300002)(25786009)(446003)(76176011)(11346002)(476003)(2501003)(486006)(14444005)(71200400001)(6246003)(71190400001)(53946003)(14454004)(256004)(478600001)(99286004)(66476007)(7696005)(6506007)(53546011)(102836004)(52536014)(30864003)(66946007)(64756008)(26005)(66446008)(186003)(53936002)(66556008)(55016002)(6436002)(81166006)(81156014)(8936002)(8676002)(74316002)(305945005)(110136005)(54906003)(3846002)(6116002)(2906002)(9686003)(76116006)(316002)(229853002)(33656002)(7736002); DIR:OUT; SFP:1101; SCL:1; SRVR:VE1PR08MB4976; H:VE1PR08MB5149.eurprd08.prod.outlook.com; FPR:; SPF:None; LANG:en; PTR:InfoNoRecords; A:1; MX:1; received-spf: None (protection.outlook.com: arm.com does not designate permitted sender hosts) X-MS-Exchange-SenderADCheck: 1 X-Microsoft-Antispam-Message-Info-Original: 9GPwXPlESncJJBkx+FfStBZez+pR4HYQ7cxtA3bxQTstyeNV4lZJsAoDfoHSP34b6pAGO9CRuEvbXydTeHN4tEIolA3y7lfy8+QeB0qojxu80sUgp1fCkAG+sRytHtjTbSW0o0gq6K4Fm4omyyp03A/flYHrZUuemjQsAXpnKeq+g4zPS7QgVlqZ+qVtkgEVv9U+RrgsqgkcW6PdCLNSZjhBTtT7Vfjh9zGWq9Tgmztv9UF3f26+qIUBNbUK5Xr7bnVlbpRTIkvwTlfXls8vkAQc+hLMGHdbWw/Ht2KnC0vFcoN8ph9R3Z0Y3muG5/1Dk+0z90vx3nc2lqiMp6EvFCCRGGLyMojP2BnJb3HCMBEpKshy4ke1YQTCLp4OnSkgcPZwb3kk9y3C32qjCR09RQ1BCXk3VN+gIv5053bZV1M= Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 X-MS-Exchange-Transport-CrossTenantHeadersStamped: VE1PR08MB4976 Original-Authentication-Results: spf=none (sender IP is ) smtp.mailfrom=Honnappa.Nagarahalli@arm.com; X-EOPAttributedMessage: 0 X-MS-Exchange-Transport-CrossTenantHeadersStripped: AM5EUR03FT058.eop-EUR03.prod.protection.outlook.com X-Forefront-Antispam-Report: CIP:63.35.35.123; IPV:CAL; SCL:-1; CTRY:IE; EFV:NLI; SFV:NSPM; SFS:(10009020)(4636009)(396003)(39860400002)(376002)(136003)(346002)(2980300002)(189003)(199004)(13464003)(356004)(25786009)(97756001)(478600001)(50466002)(336012)(2201001)(26826003)(63370400001)(86362001)(36906005)(76176011)(99286004)(46406003)(63350400001)(6246003)(486006)(9686003)(81156014)(14454004)(4326008)(2501003)(8746002)(81166006)(7696005)(8936002)(8676002)(76130400001)(102836004)(7736002)(55016002)(70206006)(5660300002)(6506007)(53546011)(52536014)(6116002)(26005)(316002)(22756006)(3846002)(14444005)(70586007)(229853002)(110136005)(23726003)(11346002)(446003)(2906002)(30864003)(126002)(33656002)(476003)(186003)(74316002)(54906003)(66066001)(305945005)(47776003); DIR:OUT; SFP:1101; SCL:1; SRVR:AM0PR08MB5361; H:64aa7808-outbound-1.mta.getcheckrecipient.com; FPR:; SPF:TempError; LANG:en; PTR:ec2-63-35-35-123.eu-west-1.compute.amazonaws.com; MX:1; A:1; X-MS-Office365-Filtering-Correlation-Id-Prvs: 97bf7f4a-6e90-4427-3056-08d73302b17a X-Microsoft-Antispam: BCL:0; PCL:0; RULEID:(2390118)(7020095)(4652040)(8989299)(5600166)(710020)(711020)(4605104)(1401327)(4618075)(4534185)(4627221)(201703031133081)(201702281549075)(8990200)(2017052603328)(7193020); SRVR:AM0PR08MB5361; NoDisclaimer: True X-Forefront-PRVS: 0152EBA40F X-Microsoft-Antispam-Message-Info: TKAUls3U22t6EkbuOyhVZdgao1pkyq1wPj7DYB046vskLJ9qDny8m+2mMiZEjrW8JBq/MRJl1j9YhVtXrooquXwzJxq+jS+5fYQkjxrLD2XctMMy+vXrGMbjID9E/KMg5cghgu7ElU5ERvClzS3LU/lkU6UaitGqqibuJf7hecdQrGZPvQCYpYbG67V4w5iHquPyYR5JU+PsdkC1y3VUZvdxAzau0cgq6k38pRjo0GuwRXtB90Z5IInEC1Nga25xeF4uF6T+MbxRki1JX/bJ6yDlTZD5bvLcj1bdFEfGURjVxM+5lxfXqZMo7fiyrEhAK3mxDCM7NDuFhfsWBGaLZgG1VWvYBeyijqTDW7VSzOno5WvUYPDoQbhPvA6RjVlavO8AZZ7AYFCJe0HRV5iOndz9WIZXLQZvfFtGXF54Ct0= X-OriginatorOrg: arm.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 06 Sep 2019 19:45:06.5552 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: bd9c5781-9821-4312-7672-08d73302b809 X-MS-Exchange-CrossTenant-Id: f34e5979-57d9-4aaa-ad4d-b122a662184d X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=f34e5979-57d9-4aaa-ad4d-b122a662184d; Ip=[63.35.35.123]; Helo=[64aa7808-outbound-1.mta.getcheckrecipient.com] X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: AM0PR08MB5361 Subject: Re: [dpdk-dev] [PATCH v2 3/6] lib/lpm: integrate RCU QSBR X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" Adding Paul for feedback > -----Original Message----- > From: Ruifeng Wang > Sent: Friday, September 6, 2019 4:46 AM > To: bruce.richardson@intel.com; vladimir.medvedkin@intel.com; > olivier.matz@6wind.com > Cc: dev@dpdk.org; stephen@networkplumber.org; > konstantin.ananyev@intel.com; Gavin Hu (Arm Technology China) > ; Honnappa Nagarahalli > ; Dharmik Thakkar > ; nd ; Ruifeng Wang (Arm > Technology China) > Subject: [PATCH v2 3/6] lib/lpm: integrate RCU QSBR >=20 > Currently, the tbl8 group is freed even though the readers might be using= the > tbl8 group entries. The freed tbl8 group can be reallocated quickly. This= results > in incorrect lookup results. >=20 > RCU QSBR process is integrated for safe tbl8 group reclaim. > Refer to RCU documentation to understand various aspects of integrating R= CU > library into other libraries. >=20 > Signed-off-by: Ruifeng Wang > Reviewed-by: Honnappa Nagarahalli > --- > lib/librte_lpm/Makefile | 3 +- > lib/librte_lpm/meson.build | 2 + > lib/librte_lpm/rte_lpm.c | 223 +++++++++++++++++++++++++++-- > lib/librte_lpm/rte_lpm.h | 22 +++ > lib/librte_lpm/rte_lpm_version.map | 6 + > lib/meson.build | 3 +- > 6 files changed, 244 insertions(+), 15 deletions(-) >=20 > diff --git a/lib/librte_lpm/Makefile b/lib/librte_lpm/Makefile index > a7946a1c5..ca9e16312 100644 > --- a/lib/librte_lpm/Makefile > +++ b/lib/librte_lpm/Makefile > @@ -6,9 +6,10 @@ include $(RTE_SDK)/mk/rte.vars.mk # library name LIB = =3D > librte_lpm.a >=20 > +CFLAGS +=3D -DALLOW_EXPERIMENTAL_API > CFLAGS +=3D -O3 > CFLAGS +=3D $(WERROR_FLAGS) -I$(SRCDIR) > -LDLIBS +=3D -lrte_eal -lrte_hash > +LDLIBS +=3D -lrte_eal -lrte_hash -lrte_rcu >=20 > EXPORT_MAP :=3D rte_lpm_version.map >=20 > diff --git a/lib/librte_lpm/meson.build b/lib/librte_lpm/meson.build inde= x > a5176d8ae..19a35107f 100644 > --- a/lib/librte_lpm/meson.build > +++ b/lib/librte_lpm/meson.build > @@ -2,9 +2,11 @@ > # Copyright(c) 2017 Intel Corporation >=20 > version =3D 2 > +allow_experimental_apis =3D true > sources =3D files('rte_lpm.c', 'rte_lpm6.c') headers =3D files('rte_lpm= .h', > 'rte_lpm6.h') # since header files have different names, we can install = all vector > headers # without worrying about which architecture we actually need > headers +=3D files('rte_lpm_altivec.h', 'rte_lpm_neon.h', 'rte_lpm_sse.h'= ) deps +=3D > ['hash'] > +deps +=3D ['rcu'] > diff --git a/lib/librte_lpm/rte_lpm.c b/lib/librte_lpm/rte_lpm.c index > 3a929a1b1..9764b8de6 100644 > --- a/lib/librte_lpm/rte_lpm.c > +++ b/lib/librte_lpm/rte_lpm.c > @@ -1,5 +1,6 @@ > /* SPDX-License-Identifier: BSD-3-Clause > * Copyright(c) 2010-2014 Intel Corporation > + * Copyright(c) 2019 Arm Limited > */ >=20 > #include > @@ -22,6 +23,7 @@ > #include > #include > #include > +#include >=20 > #include "rte_lpm.h" >=20 > @@ -39,6 +41,11 @@ enum valid_flag { > VALID > }; >=20 > +struct __rte_lpm_qs_item { > + uint64_t token; /**< QSBR token.*/ > + uint32_t index; /**< tbl8 group index.*/ > +}; > + > /* Macro to enable/disable run-time checks. */ #if > defined(RTE_LIBRTE_LPM_DEBUG) #include @@ -381,6 > +388,7 @@ rte_lpm_free_v1604(struct rte_lpm *lpm) >=20 > rte_mcfg_tailq_write_unlock(); >=20 > + rte_ring_free(lpm->qs_fifo); > rte_free(lpm->tbl8); > rte_free(lpm->rules_tbl); > rte_free(lpm); > @@ -390,6 +398,147 @@ BIND_DEFAULT_SYMBOL(rte_lpm_free, _v1604, > 16.04); MAP_STATIC_SYMBOL(void rte_lpm_free(struct rte_lpm *lpm), > rte_lpm_free_v1604); >=20 > +/* Add an item into FIFO. > + * return: 0 - success > + */ > +static int > +__rte_lpm_rcu_qsbr_fifo_push(struct rte_ring *fifo, > + struct __rte_lpm_qs_item *item) > +{ > + if (rte_ring_free_count(fifo) < 2) { > + RTE_LOG(ERR, LPM, "QS FIFO full\n"); > + rte_errno =3D ENOSPC; > + return 1; > + } > + > + (void)rte_ring_sp_enqueue(fifo, (void *)(uintptr_t)item->token); > + (void)rte_ring_sp_enqueue(fifo, (void *)(uintptr_t)item->index); > + > + return 0; > +} > + > +/* Remove item from FIFO. > + * Used when data observed by rte_ring_peek. > + */ > +static void > +__rte_lpm_rcu_qsbr_fifo_pop(struct rte_ring *fifo, > + struct __rte_lpm_qs_item *item) > +{ > + void *obj_token =3D NULL; > + void *obj_index =3D NULL; > + > + (void)rte_ring_sc_dequeue(fifo, &obj_token); > + (void)rte_ring_sc_dequeue(fifo, &obj_index); > + > + if (item) { > + item->token =3D (uint64_t)((uintptr_t)obj_token); > + item->index =3D (uint32_t)((uintptr_t)obj_index); > + } > +} > + > +/* Max number of tbl8 groups to reclaim at one time. */ > +#define RCU_QSBR_RECLAIM_SIZE 8 > + > +/* When RCU QSBR FIFO usage is above 1/(2^RCU_QSBR_RECLAIM_LEVEL), > + * reclaim will be triggered by tbl8_free. > + */ > +#define RCU_QSBR_RECLAIM_LEVEL 3 > + > +/* Reclaim some tbl8 groups based on quiescent state check. > + * RCU_QSBR_RECLAIM_SIZE groups will be reclaimed at max. > + * Params: lpm - lpm object handle > + * index - (onput) one of successfully reclaimed tbl8 groups > + * return: 0 - success, 1 - no group reclaimed. > + */ > +static uint32_t > +__rte_lpm_rcu_qsbr_reclaim_chunk(struct rte_lpm *lpm, uint32_t *index) > +{ > + struct __rte_lpm_qs_item qs_item; > + struct rte_lpm_tbl_entry *tbl8_entry =3D NULL; > + void *obj_token; > + uint32_t cnt =3D 0; > + > + RTE_LOG(DEBUG, LPM, "RCU QSBR reclaimation triggered.\n"); > + /* Check reader threads quiescent state and > + * reclaim as much tbl8 groups as possible. > + */ > + while ((cnt < RCU_QSBR_RECLAIM_SIZE) && > + (rte_ring_peek(lpm->qs_fifo, &obj_token) =3D=3D 0) && > + (rte_rcu_qsbr_check(lpm->qsv, > (uint64_t)((uintptr_t)obj_token), > + false) =3D=3D 1)) { > + __rte_lpm_rcu_qsbr_fifo_pop(lpm->qs_fifo, &qs_item); > + > + tbl8_entry =3D &lpm->tbl8[qs_item.index * > + > RTE_LPM_TBL8_GROUP_NUM_ENTRIES]; > + memset(&tbl8_entry[0], 0, > + RTE_LPM_TBL8_GROUP_NUM_ENTRIES * > + sizeof(tbl8_entry[0])); > + cnt++; > + } > + > + RTE_LOG(DEBUG, LPM, "RCU QSBR reclaimed %u groups.\n", cnt); > + if (cnt) { > + if (index) > + *index =3D qs_item.index; > + return 0; > + } > + return 1; > +} > + > +/* Trigger tbl8 group reclaim when necessary. > + * Reclaim happens when RCU QSBR queue usage > + * is over 1/(2^RCU_QSBR_RECLAIM_LEVEL). > + */ > +static void > +__rte_lpm_rcu_qsbr_try_reclaim(struct rte_lpm *lpm) { > + if (lpm->qsv =3D=3D NULL) > + return; > + > + if (rte_ring_count(lpm->qs_fifo) < > + (rte_ring_get_capacity(lpm->qs_fifo) >> > RCU_QSBR_RECLAIM_LEVEL)) > + return; > + > + (void)__rte_lpm_rcu_qsbr_reclaim_chunk(lpm, NULL); } > + > +/* Associate QSBR variable with an LPM object. > + */ > +int > +rte_lpm_rcu_qsbr_add(struct rte_lpm *lpm, struct rte_rcu_qsbr *v) { > + uint32_t qs_fifo_size; > + char rcu_ring_name[RTE_RING_NAMESIZE]; > + > + if ((lpm =3D=3D NULL) || (v =3D=3D NULL)) { > + rte_errno =3D EINVAL; > + return 1; > + } > + > + if (lpm->qsv) { > + rte_errno =3D EEXIST; > + return 1; > + } > + > + /* round up qs_fifo_size to next power of two that is not less than > + * number_tbl8s. Will store 'token' and 'index'. > + */ > + qs_fifo_size =3D rte_align32pow2((2 * lpm->number_tbl8s) + 1); > + > + /* Init QSBR reclaiming FIFO. */ > + snprintf(rcu_ring_name, sizeof(rcu_ring_name), "LPM_RCU_%s", lpm- > >name); > + lpm->qs_fifo =3D rte_ring_create(rcu_ring_name, qs_fifo_size, > + SOCKET_ID_ANY, 0); > + if (lpm->qs_fifo =3D=3D NULL) { > + RTE_LOG(ERR, LPM, "LPM QS FIFO memory allocation > failed\n"); > + rte_errno =3D ENOMEM; > + return 1; > + } > + lpm->qsv =3D v; > + > + return 0; > +} > + > /* > * Adds a rule to the rule table. > * > @@ -640,6 +789,35 @@ rule_find_v1604(struct rte_lpm *lpm, uint32_t > ip_masked, uint8_t depth) > return -EINVAL; > } >=20 > +static int32_t > +tbl8_alloc_reclaimed(struct rte_lpm *lpm) { > + struct rte_lpm_tbl_entry *tbl8_entry =3D NULL; > + uint32_t index; > + > + if (lpm->qsv !=3D NULL) { > + if (__rte_lpm_rcu_qsbr_reclaim_chunk(lpm, &index) =3D=3D 0) { > + /* Set the last reclaimed tbl8 group as VALID. */ > + struct rte_lpm_tbl_entry new_tbl8_entry =3D { > + .next_hop =3D 0, > + .valid =3D INVALID, > + .depth =3D 0, > + .valid_group =3D VALID, > + }; > + > + tbl8_entry =3D &lpm->tbl8[index * > + > RTE_LPM_TBL8_GROUP_NUM_ENTRIES]; > + __atomic_store(tbl8_entry, &new_tbl8_entry, > + __ATOMIC_RELAXED); > + > + /* Return group index for reclaimed tbl8 group. */ > + return index; > + } > + } > + > + return -ENOSPC; > +} > + > /* > * Find, clean and allocate a tbl8. > */ > @@ -679,14 +857,15 @@ tbl8_alloc_v20(struct rte_lpm_tbl_entry_v20 > *tbl8) } >=20 > static int32_t > -tbl8_alloc_v1604(struct rte_lpm_tbl_entry *tbl8, uint32_t number_tbl8s) > +tbl8_alloc_v1604(struct rte_lpm *lpm) > { > uint32_t group_idx; /* tbl8 group index. */ > struct rte_lpm_tbl_entry *tbl8_entry; >=20 > /* Scan through tbl8 to find a free (i.e. INVALID) tbl8 group. */ > - for (group_idx =3D 0; group_idx < number_tbl8s; group_idx++) { > - tbl8_entry =3D &tbl8[group_idx * > RTE_LPM_TBL8_GROUP_NUM_ENTRIES]; > + for (group_idx =3D 0; group_idx < lpm->number_tbl8s; group_idx++) { > + tbl8_entry =3D &lpm->tbl8[group_idx * > + > RTE_LPM_TBL8_GROUP_NUM_ENTRIES]; > /* If a free tbl8 group is found clean it and set as VALID. */ > if (!tbl8_entry->valid_group) { > struct rte_lpm_tbl_entry new_tbl8_entry =3D { @@ - > 708,8 +887,8 @@ tbl8_alloc_v1604(struct rte_lpm_tbl_entry *tbl8, uint32_t > number_tbl8s) > } > } >=20 > - /* If there are no tbl8 groups free then return error. */ > - return -ENOSPC; > + /* If there are no tbl8 groups free then check reclaim queue. */ > + return tbl8_alloc_reclaimed(lpm); > } >=20 > static void > @@ -728,13 +907,31 @@ tbl8_free_v20(struct rte_lpm_tbl_entry_v20 *tbl8, > uint32_t tbl8_group_start) } >=20 > static void > -tbl8_free_v1604(struct rte_lpm_tbl_entry *tbl8, uint32_t tbl8_group_star= t) > +tbl8_free_v1604(struct rte_lpm *lpm, uint32_t tbl8_group_start) > { > - /* Set tbl8 group invalid*/ > + struct __rte_lpm_qs_item qs_item; > struct rte_lpm_tbl_entry zero_tbl8_entry =3D {0}; >=20 > - __atomic_store(&tbl8[tbl8_group_start], &zero_tbl8_entry, > - __ATOMIC_RELAXED); > + if (lpm->qsv !=3D NULL) { > + /* Push into QSBR FIFO. */ > + qs_item.token =3D rte_rcu_qsbr_start(lpm->qsv); > + qs_item.index =3D > + tbl8_group_start / > RTE_LPM_TBL8_GROUP_NUM_ENTRIES; > + if (__rte_lpm_rcu_qsbr_fifo_push(lpm->qs_fifo, &qs_item) !=3D 0) > + /* This should never happen as FIFO size is big enough > + * to hold all tbl8 groups. > + */ > + RTE_LOG(ERR, LPM, "Failed to push QSBR FIFO\n"); > + > + /* Speculatively reclaim tbl8 groups. > + * Help spread the reclaim work load across multiple calls. > + */ > + __rte_lpm_rcu_qsbr_try_reclaim(lpm); > + } else { > + /* Set tbl8 group invalid*/ > + __atomic_store(&lpm->tbl8[tbl8_group_start], > &zero_tbl8_entry, > + __ATOMIC_RELAXED); > + } > } >=20 > static __rte_noinline int32_t > @@ -1037,7 +1234,7 @@ add_depth_big_v1604(struct rte_lpm *lpm, > uint32_t ip_masked, uint8_t depth, >=20 > if (!lpm->tbl24[tbl24_index].valid) { > /* Search for a free tbl8 group. */ > - tbl8_group_index =3D tbl8_alloc_v1604(lpm->tbl8, lpm- > >number_tbl8s); > + tbl8_group_index =3D tbl8_alloc_v1604(lpm); >=20 > /* Check tbl8 allocation was successful. */ > if (tbl8_group_index < 0) { > @@ -1083,7 +1280,7 @@ add_depth_big_v1604(struct rte_lpm *lpm, > uint32_t ip_masked, uint8_t depth, > } /* If valid entry but not extended calculate the index into Table8. *= / > else if (lpm->tbl24[tbl24_index].valid_group =3D=3D 0) { > /* Search for free tbl8 group. */ > - tbl8_group_index =3D tbl8_alloc_v1604(lpm->tbl8, lpm- > >number_tbl8s); > + tbl8_group_index =3D tbl8_alloc_v1604(lpm); >=20 > if (tbl8_group_index < 0) { > return tbl8_group_index; > @@ -1818,7 +2015,7 @@ delete_depth_big_v1604(struct rte_lpm *lpm, > uint32_t ip_masked, > */ > lpm->tbl24[tbl24_index].valid =3D 0; > __atomic_thread_fence(__ATOMIC_RELEASE); > - tbl8_free_v1604(lpm->tbl8, tbl8_group_start); > + tbl8_free_v1604(lpm, tbl8_group_start); > } else if (tbl8_recycle_index > -1) { > /* Update tbl24 entry. */ > struct rte_lpm_tbl_entry new_tbl24_entry =3D { @@ -1834,7 > +2031,7 @@ delete_depth_big_v1604(struct rte_lpm *lpm, uint32_t > ip_masked, > __atomic_store(&lpm->tbl24[tbl24_index], &new_tbl24_entry, > __ATOMIC_RELAXED); > __atomic_thread_fence(__ATOMIC_RELEASE); > - tbl8_free_v1604(lpm->tbl8, tbl8_group_start); > + tbl8_free_v1604(lpm, tbl8_group_start); > } > #undef group_idx > return 0; > diff --git a/lib/librte_lpm/rte_lpm.h b/lib/librte_lpm/rte_lpm.h index > 906ec4483..5079fb262 100644 > --- a/lib/librte_lpm/rte_lpm.h > +++ b/lib/librte_lpm/rte_lpm.h > @@ -1,5 +1,6 @@ > /* SPDX-License-Identifier: BSD-3-Clause > * Copyright(c) 2010-2014 Intel Corporation > + * Copyright(c) 2019 Arm Limited > */ >=20 > #ifndef _RTE_LPM_H_ > @@ -21,6 +22,7 @@ > #include > #include > #include > +#include >=20 > #ifdef __cplusplus > extern "C" { > @@ -186,6 +188,8 @@ struct rte_lpm { > __rte_cache_aligned; /**< LPM tbl24 table. */ > struct rte_lpm_tbl_entry *tbl8; /**< LPM tbl8 table. */ > struct rte_lpm_rule *rules_tbl; /**< LPM rules. */ > + struct rte_rcu_qsbr *qsv; /**< RCU QSBR variable for tbl8 > group.*/ > + struct rte_ring *qs_fifo; /**< RCU QSBR reclaiming queue. */ > }; >=20 > /** > @@ -248,6 +252,24 @@ rte_lpm_free_v20(struct rte_lpm_v20 *lpm); void > rte_lpm_free_v1604(struct rte_lpm *lpm); >=20 > +/** > + * Associate RCU QSBR variable with an LPM object. > + * > + * @param lpm > + * the lpm object to add RCU QSBR > + * @param v > + * RCU QSBR variable > + * @return > + * On success - 0 > + * On error - 1 with error code set in rte_errno. > + * Possible rte_errno codes are: > + * - EINVAL - invalid pointer > + * - EEXIST - already added QSBR > + * - ENOMEM - memory allocation failure > + */ > +__rte_experimental > +int rte_lpm_rcu_qsbr_add(struct rte_lpm *lpm, struct rte_rcu_qsbr *v); > + > /** > * Add a rule to the LPM table. > * > diff --git a/lib/librte_lpm/rte_lpm_version.map > b/lib/librte_lpm/rte_lpm_version.map > index 90beac853..b353aabd2 100644 > --- a/lib/librte_lpm/rte_lpm_version.map > +++ b/lib/librte_lpm/rte_lpm_version.map > @@ -44,3 +44,9 @@ DPDK_17.05 { > rte_lpm6_lookup_bulk_func; >=20 > } DPDK_16.04; > + > +EXPERIMENTAL { > + global: > + > + rte_lpm_rcu_qsbr_add; > +}; > diff --git a/lib/meson.build b/lib/meson.build index e5ff83893..3a96f005d > 100644 > --- a/lib/meson.build > +++ b/lib/meson.build > @@ -11,6 +11,7 @@ > libraries =3D [ > 'kvargs', # eal depends on kvargs > 'eal', # everything depends on eal > + 'rcu', # hash and lpm depends on this > 'ring', 'mempool', 'mbuf', 'net', 'meter', 'ethdev', 'pci', # core > 'cmdline', > 'metrics', # bitrate/latency stats depends on this @@ -22,7 +23,7 @@ > libraries =3D [ > 'gro', 'gso', 'ip_frag', 'jobstats', > 'kni', 'latencystats', 'lpm', 'member', > 'power', 'pdump', 'rawdev', > - 'rcu', 'reorder', 'sched', 'security', 'stack', 'vhost', > + 'reorder', 'sched', 'security', 'stack', 'vhost', > # ipsec lib depends on net, crypto and security > 'ipsec', > # add pkt framework libs which use other libs from above > -- > 2.17.1