From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from dpdk.org (dpdk.org [92.243.14.124]) by inbox.dpdk.org (Postfix) with ESMTP id F2465A0613 for ; Mon, 26 Aug 2019 07:33:00 +0200 (CEST) Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id 582FB1BF4D; Mon, 26 Aug 2019 07:33:00 +0200 (CEST) Received: from EUR04-VI1-obe.outbound.protection.outlook.com (mail-eopbgr80053.outbound.protection.outlook.com [40.107.8.53]) by dpdk.org (Postfix) with ESMTP id D1C561BF49 for ; Mon, 26 Aug 2019 07:32:58 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=armh.onmicrosoft.com; s=selector2-armh-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=+DQrj6ON+CxNPAakc+5E7g8ruL2SlrTnIwOVBH0w4CI=; b=h8NKSd/WzJ1RLTtZBRojpxpgJIlpbMn8OZ1ZKxuoxz2K4iXIUDK+Lhflb+n8XUuj/SZZbpK+mYbvHJHuhOBM2a4fyWsXZISVqEq6wxeXhgJ6rrqI6std3kF7XkhNu5Zh+Jbr9u14V2tJJQXb5ubNHR92yj3jBnKqqv4zBfQbo3U= Received: from VI1PR08CA0168.eurprd08.prod.outlook.com (10.175.227.22) by VI1PR0801MB1854.eurprd08.prod.outlook.com (10.168.68.15) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.2199.21; Mon, 26 Aug 2019 05:32:56 +0000 Received: from VE1EUR03FT026.eop-EUR03.prod.protection.outlook.com (2a01:111:f400:7e09::206) by VI1PR08CA0168.outlook.office365.com (2603:10a6:800:d1::22) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384) id 15.20.2199.14 via Frontend Transport; Mon, 26 Aug 2019 05:32:56 +0000 Authentication-Results: spf=temperror (sender IP is 63.35.35.123) smtp.mailfrom=arm.com; dpdk.org; dkim=pass (signature was verified) header.d=armh.onmicrosoft.com;dpdk.org; dmarc=temperror action=none header.from=arm.com; Received-SPF: TempError (protection.outlook.com: error in processing during lookup of arm.com: DNS Timeout) Received: from 64aa7808-outbound-1.mta.getcheckrecipient.com (63.35.35.123) by VE1EUR03FT026.mail.protection.outlook.com (10.152.18.148) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384) id 15.20.2199.13 via Frontend Transport; Mon, 26 Aug 2019 05:32:54 +0000 Received: ("Tessian outbound 4f2e8f9f1994:v27"); Mon, 26 Aug 2019 05:32:49 +0000 X-CR-MTA-TID: 64aa7808 Received: from 24ad46b2f648.1 (ip-172-16-0-2.eu-west-1.compute.internal [104.47.8.55]) by 64aa7808-outbound-1.mta.getcheckrecipient.com id 7286D709-3B12-4C3E-BBFC-E928B1D6C034.1; Mon, 26 Aug 2019 05:32:44 +0000 Received: from EUR03-AM5-obe.outbound.protection.outlook.com (mail-am5eur03lp2055.outbound.protection.outlook.com [104.47.8.55]) by 64aa7808-outbound-1.mta.getcheckrecipient.com with ESMTPS id 24ad46b2f648.1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384); Mon, 26 Aug 2019 05:32:44 +0000 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=MO5iwiQEne9nk6re4tJpJkOl5t2eFICAWQw4p08HEEWqlonAB5VEoJJU2FAnGotYw9TKIuVx3qWRMAk/6Qn0HHTpBZDehL0/OnM4MHmP1iRzCiWbeA0wCcN4C1M2r7KsWkWv2n0LERpnoKFMt1ilbsxbM9MI9wwOINjWe2rc0oR7syUzYGeJBz2onojOkhhnNitYMM/aO1y4J3zV/dWbvFZgr/aDr01xB6JtqDIFERtEyN6pqNhOAq82kVsVy3tAnLPiku62cwZD8MpTsm+pGXQ3np+E+YwvMgI6+BRgNks1ywP3PXPpwnjCMtr6N9GanJFfqoSAdDlWrrfezbXziA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=+DQrj6ON+CxNPAakc+5E7g8ruL2SlrTnIwOVBH0w4CI=; b=ZtHjuEcYjKCo3sOAEhVZNCt0kIXoanfQw/RCm1uvJWDhzZHtnAvCu49g3drjak1FSKSBpd4I4p8yay/0YaXgIJGn8E1JE6J18s0CXDpsmSWBoo6XQX4lrKKkpVpIWKb5iDyCnmiT1CwnvYQCy7/3HHxJ+HWoXhYf7tq3it0kAL9gakIcWs5h4jNGvhwlaq1mxLfFJa0QSarw8d9lsH6I2Bee/c1nsaSTqp5zroUWm6FULhNb7MoDiul8OqQOOk3u2PTvuMHS0Za+9ZaRaNbV5eKtw7DJnlireXK3CJRLVa46+gqFhirtN+RkL6jCOeQDcJx7NF3gNQfIdZiO28P0Jw== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=arm.com; dmarc=pass action=none header.from=arm.com; dkim=pass header.d=arm.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=armh.onmicrosoft.com; s=selector2-armh-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=+DQrj6ON+CxNPAakc+5E7g8ruL2SlrTnIwOVBH0w4CI=; b=h8NKSd/WzJ1RLTtZBRojpxpgJIlpbMn8OZ1ZKxuoxz2K4iXIUDK+Lhflb+n8XUuj/SZZbpK+mYbvHJHuhOBM2a4fyWsXZISVqEq6wxeXhgJ6rrqI6std3kF7XkhNu5Zh+Jbr9u14V2tJJQXb5ubNHR92yj3jBnKqqv4zBfQbo3U= Received: from VE1PR08MB5149.eurprd08.prod.outlook.com (20.179.30.152) by VE1PR08MB4720.eurprd08.prod.outlook.com (10.255.115.87) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.2178.18; Mon, 26 Aug 2019 05:32:42 +0000 Received: from VE1PR08MB5149.eurprd08.prod.outlook.com ([fe80::a8af:a9b8:4597:4128]) by VE1PR08MB5149.eurprd08.prod.outlook.com ([fe80::a8af:a9b8:4597:4128%3]) with mapi id 15.20.2178.020; Mon, 26 Aug 2019 05:32:42 +0000 From: Honnappa Nagarahalli To: "Ruifeng Wang (Arm Technology China)" , Stephen Hemminger CC: "bruce.richardson@intel.com" , "vladimir.medvedkin@intel.com" , "olivier.matz@6wind.com" , "dev@dpdk.org" , Dharmik Thakkar , Honnappa Nagarahalli , nd , nd Thread-Topic: [dpdk-dev] [RFC PATCH 3/3] lib/lpm: integrate RCU QSBR Thread-Index: AQHVWLPmHHr5a8S2lkuqc1vtt/U+P6cH8TqAgATVRICAAAg3MA== Date: Mon, 26 Aug 2019 05:32:42 +0000 Message-ID: References: <20190822063457.41596-1-ruifeng.wang@arm.com> <20190822063457.41596-4-ruifeng.wang@arm.com> <20190822212258.1a8f7292@xps13> In-Reply-To: Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-ts-tracking-id: 0981b79b-7c75-488f-98d2-efaaa761c87a.0 x-checkrecipientchecked: true Authentication-Results-Original: spf=none (sender IP is ) smtp.mailfrom=Honnappa.Nagarahalli@arm.com; x-originating-ip: [217.140.111.135] x-ms-publictraffictype: Email X-MS-Office365-Filtering-Correlation-Id: d2544a6a-5a8f-4229-d987-08d729e6d874 X-MS-Office365-Filtering-HT: Tenant X-Microsoft-Antispam-Untrusted: BCL:0; PCL:0; RULEID:(2390118)(7020095)(4652040)(8989299)(4534185)(4627221)(201703031133081)(201702281549075)(8990200)(5600166)(711020)(4605104)(1401327)(4618075)(2017052603328)(7193020); SRVR:VE1PR08MB4720; X-MS-TrafficTypeDiagnostic: VE1PR08MB4720:|VI1PR0801MB1854: x-ms-exchange-transport-forked: True X-Microsoft-Antispam-PRVS: x-checkrecipientrouted: true x-ms-oob-tlc-oobclassifiers: OLM:1728;OLM:1728; x-forefront-prvs: 01415BB535 X-Forefront-Antispam-Report-Untrusted: SFV:NSPM; SFS:(10009020)(4636009)(396003)(366004)(136003)(346002)(39850400004)(376002)(199004)(189003)(55016002)(110136005)(102836004)(53936002)(64756008)(8936002)(6116002)(6506007)(81156014)(14454004)(26005)(229853002)(76116006)(66556008)(478600001)(33656002)(66446008)(316002)(6436002)(66946007)(25786009)(11346002)(486006)(99286004)(476003)(5660300002)(446003)(81166006)(4326008)(74316002)(7736002)(76176011)(256004)(14444005)(52536014)(305945005)(71190400001)(71200400001)(7696005)(2906002)(54906003)(6246003)(8676002)(66066001)(9686003)(3846002)(186003)(86362001)(66476007); DIR:OUT; SFP:1101; SCL:1; SRVR:VE1PR08MB4720; H:VE1PR08MB5149.eurprd08.prod.outlook.com; FPR:; SPF:None; LANG:en; PTR:InfoNoRecords; A:1; MX:1; received-spf: None (protection.outlook.com: arm.com does not designate permitted sender hosts) X-MS-Exchange-SenderADCheck: 1 X-Microsoft-Antispam-Message-Info-Original: KmSWMHLC/9Fa5xgjDVJl0m4ZnC7a8zRLpZd4cZ2WlKswS10AKzLYBvOczJ69ly/Q2sJpLUUgtSQt0ruxIOTNySb+jQDyDHVgFuJNt/jjsYtaSSt1YAm13iMzUe8jGEJ3IIA51GvG1jEBEUkGai/bp3iiyxQvs0dqYqgvDD4nTxs4OJDLp/ajewXadwDuHbsIAmW3g7zgeT4XF09r7Nvk6+MvQeQr0V2U7W8rw/9rRk7Nx+VI7uhue5b588I65Oj7YLa+ReWF6HVOBwoiu6nhcY1Ew61Q3ivPJmrjdkIiXeu1uwQkD7V6wvGu3kTVXRgskUp+SGnOpHp3Fyulo5d7WN6q2kY2Gfg4opGH5+Ie6Jzbq/5c9P9vVEqTPXw3gi2C0SpuXzh5pEo2A2bwL8LFMrNTMfMlAGWppyjkSPm/Gmc= Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 X-MS-Exchange-Transport-CrossTenantHeadersStamped: VE1PR08MB4720 Original-Authentication-Results: spf=none (sender IP is ) smtp.mailfrom=Honnappa.Nagarahalli@arm.com; X-EOPAttributedMessage: 0 X-MS-Exchange-Transport-CrossTenantHeadersStripped: VE1EUR03FT026.eop-EUR03.prod.protection.outlook.com X-Forefront-Antispam-Report: CIP:63.35.35.123; IPV:CAL; SCL:-1; CTRY:IE; EFV:NLI; SFV:NSPM; SFS:(10009020)(4636009)(39850400004)(136003)(376002)(346002)(396003)(2980300002)(189003)(199004)(4326008)(99286004)(478600001)(6116002)(8676002)(14444005)(74316002)(102836004)(46406003)(86362001)(25786009)(356004)(9686003)(2906002)(81156014)(50466002)(26826003)(70206006)(70586007)(229853002)(76130400001)(52536014)(6246003)(81166006)(97756001)(486006)(66066001)(316002)(36906005)(22756006)(54906003)(110136005)(126002)(8746002)(33656002)(305945005)(476003)(5660300002)(23726003)(76176011)(7696005)(186003)(446003)(63350400001)(63370400001)(11346002)(26005)(336012)(7736002)(8936002)(14454004)(47776003)(3846002)(55016002)(6506007); DIR:OUT; SFP:1101; SCL:1; SRVR:VI1PR0801MB1854; H:64aa7808-outbound-1.mta.getcheckrecipient.com; FPR:; SPF:TempError; LANG:en; PTR:ec2-63-35-35-123.eu-west-1.compute.amazonaws.com; MX:1; A:1; X-MS-Office365-Filtering-Correlation-Id-Prvs: 66342e94-e826-4115-7999-08d729e6d117 X-Microsoft-Antispam: BCL:0; PCL:0; RULEID:(2390118)(7020095)(4652040)(8989299)(4534185)(4627221)(201703031133081)(201702281549075)(8990200)(5600166)(710020)(711020)(4605104)(1401327)(4618075)(2017052603328)(7193020); SRVR:VI1PR0801MB1854; NoDisclaimer: True X-Forefront-PRVS: 01415BB535 X-Microsoft-Antispam-Message-Info: WWwJG0XKU+zX+bP0OM552wpZltm4Uti4x8eMP9PuVXZbPjnk1L0pJWAo+5JRwffiz67/RwGjdd482QdB3yloIzC3wTAgFWXEzb7IsIfN1HFkvnj4Yk8Hiwnt97Cf1OnydTRWrtDbW6MUmpqTCtWe0DpY4YYBlrgG7aPGH+EJDZDAV2G0qikpQRgg1KsNwQwdx3QtsbC6eSDRjln8aJfBB1BBfR7rRaoMKZDhuilQyb9gmPO+Y7PXTVq+7IJ3eQg0HAsFM9PL0s3QDs4671IlxGhCHQytGvKb+nNFB9ycO68T1ofpCSHoxmFuKMSCGCZz1N7kmJCG72aGemsVLt9Wj+z62Jy9kw6UfRH9jXb49t7/b6w0fcneLTGKXSRHgyFFzXjqLj9Yv3UpVmMbYXKq7/OYkLV6bxKgubXi+/p6LSw= X-OriginatorOrg: arm.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 26 Aug 2019 05:32:54.5656 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: d2544a6a-5a8f-4229-d987-08d729e6d874 X-MS-Exchange-CrossTenant-Id: f34e5979-57d9-4aaa-ad4d-b122a662184d X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=f34e5979-57d9-4aaa-ad4d-b122a662184d; Ip=[63.35.35.123]; Helo=[64aa7808-outbound-1.mta.getcheckrecipient.com] X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: VI1PR0801MB1854 Subject: Re: [dpdk-dev] [RFC PATCH 3/3] lib/lpm: integrate RCU QSBR X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" Thank you Stephen for your comments, appreciate your inputs. > > On Thu, 22 Aug 2019 14:34:57 +0800 > > Ruifeng Wang wrote: > > > > > Currently, the tbl8 group is freed even though the readers might be > > > using the tbl8 group entries. The freed tbl8 group can be > > > reallocated quickly. This results in incorrect lookup results. > > > > > > RCU QSBR process is integrated for safe tbl8 group reclaim. > > > Refer to RCU documentation to understand various aspects of > > > integrating RCU library into other libraries. > > > > > > Signed-off-by: Ruifeng Wang > > > Reviewed-by: Honnappa Nagarahalli > > > Reviewed-by: Gavin Hu > > > > Having RCU in LPM is a good idea but difficult to find out how to do it= in > DPDK. > > Not everyone wants to use RCU, so making a required part of how LPM is > > used will impact users. >=20 > LPM users will not be imposed to use RCU. New API is provided to enable t= he > RCU functionality in LPM library. For users not using RCU, code path is i= ntact, > and there will be no performance drop. >=20 > > > > Also, it looks like DPDK RCU lacks a good generic way to handle deferre= d > free. Both rcu_defer and call_rcu from 'userspace RCU library' are wrappers on to= p of the underlying basic mechanisms. Such wrappers can be added. However, = I would prefer to integrate RCU into couple of libraries to clearly show th= e need for wrappers. Integrating RCU in the libraries also removes some bur= den from the application. > > Having to introduce a ring to handle is adding more complexity when a > > generic solution would be better (see userspace RCU library for example= ). A ring is required in rcu_defer as well as call_rcu since the pointer needs= to be stored while waiting for quiescent state updates. The ring is used i= n the proposed solution for the same purpose. I briefly looked through rcu_defer. The solution proposed here seems to be = similar to rcu_defer. However, there are several differences. 1) rcu_defer uses a single queue for each updater thread, the proposed solu= tion uses per data structure ring. IMO, this provides a better control over= the resources to reclaim. Note that currently, the ring usage itself is no= t optimized (intentionally) to keep the patches focused on the understandin= g the design. 2) rcu_defer also launches another thread which wakes up periodically and r= eclaims the resources in the ring (along with the updater thread calling sy= nchronize_rcu, which blocks, when the queue is full). This requires additio= nal synchronization between the updater thread and the reclaimer thread. Th= e solution proposed here does not need another thread as the DPDK RCU libra= ry provides non-blocking reclaiming mechanism, reclaiming is done in the co= ntext of the update thread. > > Other parts of DPDK would benefit if deferred free was done better. Which other parts are you talking about? The design proposed in 1/3 is a co= mmon solution that should apply to other libraries as well. >=20 > This requires support from RCU library. > Needs Honnappa's comment.