From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from dpdk.org (dpdk.org [92.243.14.124]) by dpdk.space (Postfix) with ESMTP id F3169A00E6 for ; Tue, 16 Apr 2019 18:56:39 +0200 (CEST) Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id 328CF1B52B; Tue, 16 Apr 2019 18:56:37 +0200 (CEST) Received: from EUR03-VE1-obe.outbound.protection.outlook.com (mail-eopbgr50051.outbound.protection.outlook.com [40.107.5.51]) by dpdk.org (Postfix) with ESMTP id 71D791B526 for ; Tue, 16 Apr 2019 18:56:35 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=armh.onmicrosoft.com; s=selector1-arm-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=ftdJ8fZv11oUyft8UjIeIKnHKCQ8bG1kY3sRGg8BZB8=; b=DFw8ooufcbOFdeuuKK3rFcRg14+/uCala0LprhTrWGOpo9ok4lVgKM+w3SEVButUrPdJOZu/+vcIzseQKffv1ErV0LTWOHNqSK1O6e8rvrtJZAuJ7cd8+8jc58KdAInIobjDw3qzC1uv6YoeeXNXdeQFs3QcbumfjW+Ap6lxdmU= Received: from VE1PR08MB5149.eurprd08.prod.outlook.com (20.179.30.152) by VE1PR08MB5007.eurprd08.prod.outlook.com (10.255.159.32) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.1792.16; Tue, 16 Apr 2019 16:56:33 +0000 Received: from VE1PR08MB5149.eurprd08.prod.outlook.com ([fe80::e0ae:ecad:ec5:8177]) by VE1PR08MB5149.eurprd08.prod.outlook.com ([fe80::e0ae:ecad:ec5:8177%2]) with mapi id 15.20.1792.018; Tue, 16 Apr 2019 16:56:33 +0000 From: Honnappa Nagarahalli To: Stephen Hemminger CC: "Ananyev, Konstantin" , "paulmck@linux.ibm.com" , "Kovacevic, Marko" , "dev@dpdk.org" , "Gavin Hu (Arm Technology China)" , Dharmik Thakkar , Malvika Gupta , nd , nd Thread-Topic: [PATCH v5 1/3] rcu: add RCU library supporting QSBR mechanism Thread-Index: AQHU8XwL9ZW9vcwT20K3DuZSwCCMIqY5FWOAgASrdlSAAIGtIIAAowaAgAAhIcA= Date: Tue, 16 Apr 2019 16:56:32 +0000 Message-ID: References: <20181122033055.3431-1-honnappa.nagarahalli@arm.com> <20190412202039.46902-1-honnappa.nagarahalli@arm.com> <20190412202039.46902-2-honnappa.nagarahalli@arm.com> <20190412150650.3709358e@shemminger-XPS-13-9360> <20190412160629.670eacd1@shemminger-XPS-13-9360> <2601191342CEEE43887BDE71AB9772580148A97E53@irsmsx105.ger.corp.intel.com> <20190415083834.31b38ed3@shemminger-XPS-13-9360> <2601191342CEEE43887BDE71AB9772580148A98064@irsmsx105.ger.corp.intel.com> <20190415142631.4c250248@shemminger-XPS-13-9360> <20190416075415.76c9d64a@xps13.lan> In-Reply-To: <20190416075415.76c9d64a@xps13.lan> Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: authentication-results: spf=none (sender IP is ) smtp.mailfrom=Honnappa.Nagarahalli@arm.com; x-originating-ip: [217.140.111.135] x-ms-publictraffictype: Email x-ms-office365-filtering-correlation-id: 6f233a7a-f2e4-448d-59e9-08d6c28c7ad4 x-ms-office365-filtering-ht: Tenant x-microsoft-antispam: BCL:0; PCL:0; RULEID:(2390118)(7020095)(4652040)(8989299)(5600140)(711020)(4605104)(4618075)(4534185)(4627221)(201703031133081)(201702281549075)(8990200)(2017052603328)(7193020); SRVR:VE1PR08MB5007; x-ms-traffictypediagnostic: VE1PR08MB5007: x-ms-exchange-purlcount: 3 nodisclaimer: True x-microsoft-antispam-prvs: x-forefront-prvs: 000947967F x-forefront-antispam-report: SFV:NSPM; SFS:(10009020)(366004)(39860400002)(136003)(346002)(376002)(396003)(189003)(199004)(33656002)(53936002)(9686003)(55016002)(6306002)(6436002)(71190400001)(106356001)(3846002)(2906002)(229853002)(71200400001)(105586002)(6116002)(66066001)(14444005)(74316002)(7736002)(305945005)(256004)(97736004)(6246003)(72206003)(347745004)(52536014)(81156014)(4326008)(966005)(8676002)(99286004)(16799955002)(476003)(6916009)(54906003)(5660300002)(8936002)(6506007)(486006)(76176011)(186003)(7696005)(25786009)(68736007)(26005)(93886005)(14454004)(102836004)(446003)(478600001)(316002)(86362001)(11346002)(81166006); DIR:OUT; SFP:1101; SCL:1; SRVR:VE1PR08MB5007; H:VE1PR08MB5149.eurprd08.prod.outlook.com; FPR:; SPF:None; LANG:en; PTR:InfoNoRecords; MX:1; A:1; received-spf: None (protection.outlook.com: arm.com does not designate permitted sender hosts) x-ms-exchange-senderadcheck: 1 x-microsoft-antispam-message-info: Z4futd2PJJdAnGjnF+aTDXqJ+PbbWyMuzjgCH9J2FPGSJM1UVMSpKM05hTZKHy4dSiZNntErVUIciIrB+1z7hoLNshhRoBE39Pf9Hsu82yoduHeQngOxiVtOIE9AFUTNfzYuhm5/ON2uIFmbm6uq3WMUoTzZic3yACD2oSeyG5O7+50gyJBkKNODA4q1M73kx6Is62aKbzW7xNlXa4dIf+P08/1PcSiG+aJdlM26Mrem9HwKmXY0ayVC9YXfXpGk/42LU7DeFTe8lT33AdzZt8DZslSnI+6fxb1JcgXBWL3qsV15am+EqkI0csgxdRPHXqHXiSpqrtYI84PfOgAnwTYRyP40yA+v1l80mQ7L1vMwODh/h0oeofrKpaaPZ3T+wdoOkxkFj0B9f1rH5grtxEeb5g+ZScvNFdVLEQxGwE4= Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 X-OriginatorOrg: arm.com X-MS-Exchange-CrossTenant-Network-Message-Id: 6f233a7a-f2e4-448d-59e9-08d6c28c7ad4 X-MS-Exchange-CrossTenant-originalarrivaltime: 16 Apr 2019 16:56:33.0250 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Hosted X-MS-Exchange-CrossTenant-id: f34e5979-57d9-4aaa-ad4d-b122a662184d X-MS-Exchange-CrossTenant-mailboxtype: HOSTED X-MS-Exchange-Transport-CrossTenantHeadersStamped: VE1PR08MB5007 Subject: Re: [dpdk-dev] [PATCH v5 1/3] rcu: add RCU library supporting QSBR mechanism X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" Message-ID: <20190416165632.W7z0GY5RzWLxlR89NVwmqun3h_AuY30NXLaguP4zETg@z> >=20 > > > > > > > > > On Fri, 12 Apr 2019 15:20:37 -0500 Honnappa Nagarahalli > > > > > > > > > wrote: > > > > > > > > > > > > > > > > > > > Add RCU library supporting quiescent state based > > > > > > > > > > memory reclamation > > > > > > > > > method. > > > > > > > > > > This library helps identify the quiescent state of the > > > > > > > > > > reader threads so that the writers can free the memory > > > > > > > > > > associated with the lock less data structures. > > > > > > > > > > > > > > > > > > > > Signed-off-by: Honnappa Nagarahalli > > > > > > > > > > > > > > > > > > > > Reviewed-by: Steve Capper > > > > > > > > > > Reviewed-by: Gavin Hu > > > > > > > > > > Reviewed-by: Ola Liljedahl > > > > > > > > > > Acked-by: Konstantin Ananyev > > > > > > > > > > > > > > > > > > > > > > > > > > > > After evaluating long term API/ABI issues, I think you > > > > > > > > > need to get rid of almost all use of inline and visible > > > > > > > > > structures. Yes it might be marginally slower, but you > > > > > > > > > thank me > > > the first time you have to fix something. > > > > > > > > > > > > > > > > > Agree, I was planning on another version to address this > > > > > > > > (I am yet > > > to take a look at your patch addressing the ABI). > > > > > > > > The structure visibility definitely needs to be addressed. > > > > > > > > For the inline functions, is the plan to convert all the > > > > > > > > inline functions in DPDK? If yes, I think we need to > > > > > > > > consider the performance > > > > > > > difference. May be consider L3-fwd application, change all > > > > > > > the > > > inline functions in its path and run a test? > > > > > > > > > > > > > > Every function that is not in the direct datapath should not > > > > > > > be > > > inline. > > > > > > > Exceptions or things like rx/tx burst, ring enqueue/dequeue, > > > > > > > and packet alloc/free > > I do not understand how DPDK can claim ABI compatibility if we have > inline functions (unless we freeze any development in these inline functi= ons > forever). > > > > > > > > > > > > > > Plus synchronization routines: spin/rwlock/barrier, etc. > > > > > > I think rcu should be one of such exceptions - it is just > > > > > > another synchronization mechanism after all (just a bit more > sophisticated). > > > > > > Konstantin > > > > > > > > > > If you look at the other userspace RCU, you wil see that the > > > > > only inlines are the rcu_read_lock,rcu_read_unlock and > > > rcu_reference/rcu_assign_pointer. > > > > > > > > > > The synchronization logic is all real functions. > > > > > > > > In fact, I think urcu provides both flavors: > > > > https://github.com/urcu/userspace- > > > rcu/blob/master/include/urcu/static/ > > > > urcu-qsbr.h I still don't understand why we have to treat it > > > > differently then let say spin-lock/ticket-lock or rwlock. > > > > If we gone all the way to create our own version of rcu, we > > > > probably want it to be as fast as possible (I know that main > > > > speedup should come from the fact that readers don't have to wait > > > > for writer to finish, but still...) > > > > > > > > Konstantin > > > > > > > > > > Having locking functions inline is already a problem in current relea= ses. > > > The implementation can not be improved without breaking ABI (or > > > doing special workarounds like lock v2) > > I think ABI and inline function discussion needs to be taken up in a > different thread. > > > > Currently, I am looking to hide the structure visibility. I looked at y= our > patch [1], it is a different case than what I have in this patch. It is a= pretty > generic use case as well (similar situation exists in other libraries). I= think a > generic solution should be agreed upon. > > > > If we have to hide the structure content, the handle to QS variable > returned to the application needs to be opaque. I suggest using 'void *' > behind which any structure can be used. > > > > typedef void * rte_rcu_qsbr_t; > > typedef void * rte_hash_t; > > > > But it requires typecasting. > > > > [1] http://patchwork.dpdk.org/cover/52609/ >=20 > C allows structure to be defined without knowing what is in it therefore. >=20 > typedef struct rte_rcu_qsbr rte_rcu_qsbr_t; >=20 > is preferred (or do it without typedef) >=20 > struct rte_rcu_qsbr; I see that rte_hash library uses the same approach (struct rte_hash in rte_= hash.h, though it is marking as internal). But the ABI Laboratory tool [1] = seems to be reporting incorrect numbers for this library even though the in= ternal structure is changed. [1] https://abi-laboratory.pro/index.php?view=3Dcompat_report&l=3Ddpdk&v1= =3D19.02&v2=3Dcurrent&obj=3D66794&kind=3Dabi