From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from NAM03-BY2-obe.outbound.protection.outlook.com (mail-by2nam03on0049.outbound.protection.outlook.com [104.47.42.49]) by dpdk.org (Postfix) with ESMTP id 6984423B for ; Tue, 24 Apr 2018 05:49:13 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=CAVIUMNETWORKS.onmicrosoft.com; s=selector1-cavium-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version; bh=SiHcDvcgCkL4mnhDoqWesTnfrJXGjk4S0GtY6I9lWto=; b=PLpdpFTiR3f/8GfoPC8gmis2zmorRliVbiMIQd8HEBCtlXM8W5LBWI6+Mab0xL0tlthE28eg7RopUVuJbfW9fyW+faehAAMoQ/3urypvN4cZ34/aK6S8J2W5PoSHB9ZXZRvkbU9AuZgFciKu28UJ2yN8jTD1brUUbZBauUUPncA= Received: from jerin (115.113.156.3) by CO2PR07MB2518.namprd07.prod.outlook.com (2603:10b6:102:12::24) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384_P256) id 15.20.696.15; Tue, 24 Apr 2018 03:49:07 +0000 Date: Tue, 24 Apr 2018 09:18:54 +0530 From: Jerin Jacob To: Jim Murphy Cc: Stephen Hemminger , Brijesh Singh , dev@dpdk.org Message-ID: <20180424034853.GA4546@jerin> References: <20180423165039.51393aad@xeon-e3> <20180423173034.7086b772@xeon-e3> MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Disposition: inline Content-Transfer-Encoding: 8bit In-Reply-To: User-Agent: Mutt/1.9.4 (2018-02-28) X-Originating-IP: [115.113.156.3] X-ClientProxiedBy: BM1PR0101CA0004.INDPRD01.PROD.OUTLOOK.COM (2603:1096:b00:18::14) To CO2PR07MB2518.namprd07.prod.outlook.com (2603:10b6:102:12::24) X-MS-PublicTrafficType: Email X-Microsoft-Antispam: UriScan:; BCL:0; PCL:0; RULEID:(7020095)(4652020)(5600026)(4534165)(7168020)(4627221)(201703031133081)(201702281549075)(2017052603328)(7153060)(7193020); SRVR:CO2PR07MB2518; X-Microsoft-Exchange-Diagnostics: 1; CO2PR07MB2518; 3:dVWNJ/6wph9sQTTGugHHPL4LcfyxfBmQVmLNDo3CYwFc0yvB8FxS6g0e3sxohUxEL9Efw0RIWU43vjFZDxl0QN1xs65uam+xhDWrnzeYmzE75L7q247VYqATfNF2joDX96ng0CgfNrL2uSfeCIOuJZjRemRdGtPAqLg3iGVbIn66YjpBJcIkwmr7yGvTtahObuG0LksIVjqku1b6OTsEBNCtreFlSLH/bq3pdXQFPPXVDd5CRH9O4nF+pL8Lp7TE; 25:clJYrKUjX7jda150HCZ6H6QwJoZV+lV8KR8dZ54uv6EMmsk5sX37BLJBKcveSq1dDFSjdQKAAzsNzTcXwtCnP/M+GmPT4b1MXnMuGHuN7r2cVVqxXAO4d29O1ErstPPmyQ4NLYZj8nuKsLq7M3ysKfNL5cobjgtmANwEFaT+dntvOgl76OBHjwcfFjNOIs//NQG78AkjUQ9E1lu7SS64JEdYCR0ISk2suEJJQQRu/XV7st1c9wSJozdG8xphdqRhTCD9KYCdyM2nAmAlLLkQ/uC7aMCHrc+BUJQIVvpIA1roEv8qjizckRZa3+HAwKJnY1nKbjcTgo6t08V8BWusJA==; 31:QSwl8tP4oHZxINEFLbW3UuR8bokBrb8f6PNaNsDY8fpcSWFQkYHWw8BW1X+GadZKMxn10UGlbHXum2OBLh8Lpjk3QUKgwwSluP2XY5rU1ncuRfBb7kqK7NCv3lMy4fEjXYPSu4Loi905Qt8FzAWgvBdFFYaLbZBKYf5vXZ/JOsSKjItUdoDliF4kaJzvJr+8yo/BcW6iDMWDrMmrr18S3d7FyQb44Adanj7F6syfVCc= X-MS-TrafficTypeDiagnostic: CO2PR07MB2518: Authentication-Results: spf=none (sender IP is ) smtp.mailfrom=Jerin.JacobKollanukkaran@cavium.com; X-Microsoft-Exchange-Diagnostics: 1; CO2PR07MB2518; 20:SZL3ypf9DWo123ZG79kQItmD1cOduaxqyxJPvDSJMUGZ5O6rlKn82NIXaMd4cgR0yo29MNadp2q55XI/WFkV6PkT1V0/fZE4i34opua02tL9EQ57kHufl/xoCGMAUYekiTUB1dV81a7dx6sLZtvEaSY9gyYT02vg7fDuSIByw5apXHyQJmitPx5xEr2muDxNYvC2hj2wopOKOymdj1pzi4qUdGtePWdyCQ3JEHp432jb9b4kG7U9tcZSRmv8QwE8MqOTFBfD5Q95xxDHwdd/b1j+Kar9Rdl6S7nPGVyQ9Sx3n8iSkxGyniEIG0SqIZalv5K67/yL1fTVQCxC3T24uKMw4eZFsVB1Gh1w0SGEbeuqqQjeIdjjaRU85pqhmiXTDK2VawSn7/8UQ1OJGik3Tj0CGPiiKd53KpipHKr6Mi35jCYxDm4VT90fqODZQskLA7ca40T63fsNK2dFDgjmp8XNoqsevhf5nBDtv9HNPgJ2QQNeUpClMCfYukh4KuR/D6jJvW+vX0FseTycPb9QDuxWkql7K7hJeaCMjX0+OEyT0n67nWUXM4xzZfBJ6xMjGQcb/4b5NGjkCYOShl1VV2PR2v0SZWcBV5ASto8BppY=; 4:SWrvK/GmVWSxeTntAbAlPh5L5YQJSRuhlCNVZHx8Nyd2ETUHTpMIbmHIX5yUaqH18xDfHKq9OZi2mcK65rj/1l5K9G4zxNn5XR0hrOOLkaxlJY/+Kcvuwrl0Ia/vifvrheVErLQyQx4d1plPcrCYxyVKRh38R5f6bdEUc5xfVhcc0AmE05LWBgjekxzxszqDbUn6QKcsuKWnnb3zq502mWVASgrEaQJwpoiJl4yIM0k1j+U77UPql2cXK0jm6u4yonCJ8vLyscHk1F4OLlULBWClcz/KWMO/84o5MGuI384yPGkCg/kHpXqsAjAJiehY X-Microsoft-Antispam-PRVS: X-Exchange-Antispam-Report-Test: UriScan:(85827821059158); X-Exchange-Antispam-Report-CFA-Test: BCL:0; PCL:0; RULEID:(8211001083)(6040522)(2401047)(8121501046)(5005006)(10201501046)(93006095)(3231232)(944501410)(52105095)(3002001)(6041310)(20161123560045)(20161123558120)(20161123564045)(201703131423095)(201702281528075)(20161123555045)(201703061421075)(201703061406153)(20161123562045)(6072148)(201708071742011); SRVR:CO2PR07MB2518; BCL:0; PCL:0; RULEID:; SRVR:CO2PR07MB2518; X-Forefront-PRVS: 0652EA5565 X-Forefront-Antispam-Report: SFV:NSPM; SFS:(10009020)(396003)(39850400004)(39380400002)(346002)(366004)(376002)(13464003)(33716001)(47776003)(53936002)(33656002)(66066001)(7736002)(39060400002)(93886005)(305945005)(33896004)(4326008)(386003)(54906003)(59450400001)(23676004)(50466002)(58126008)(53376002)(6246003)(76176011)(53546011)(55236004)(316002)(25786009)(6666003)(229853002)(2906002)(16526019)(72206003)(2870700001)(8676002)(6916009)(966005)(6306002)(9686003)(81166006)(55016002)(52116002)(5660300001)(2486003)(6496006)(52146003)(8936002)(478600001)(956004)(186003)(11346002)(42882007)(476003)(26005)(6116002)(1076002)(446003)(3846002)(44832011)(18370500001); DIR:OUT; SFP:1101; SCL:1; SRVR:CO2PR07MB2518; H:jerin; FPR:; SPF:None; LANG:en; MLV:ovr; PTR:InfoNoRecords; Received-SPF: None (protection.outlook.com: cavium.com does not designate permitted sender hosts) X-Microsoft-Exchange-Diagnostics: =?utf-8?B?MTtDTzJQUjA3TUIyNTE4OzIzOlFYOFVWcGVacVd3OTRvbTlCd1JqOEduM2s5?= =?utf-8?B?MDdHOFYzejlaVFNtWlU5QUFoek02M25pZldoYlNGWFdxWUs0UkZWTEJ4VEJr?= =?utf-8?B?cDBVdURxZzVQeEx4cmNTc0s3ZVJROW40NFpONmJQTVJ0ZGtaVlFIZFYwZmhj?= =?utf-8?B?Z2FuUXdUMDEzRnUwV1duMHNTckpKdVlndWs2WGR5RnU5Y3VvVFJpM0h6VEpu?= =?utf-8?B?Y1NwMVVjSm1FS0l5NWU2SDZFSDlmVDhaWWdHK1k5WVdQOUxlR1lIUWp0TzlS?= =?utf-8?B?TDRNZ0JDaTdxUDMwalMzZk5uTHQ4cGdTYm1CdTFROWZ2YVZvZ05jVGNzV2ZL?= =?utf-8?B?YncyZmsxTW5reHQxK3A2NmFKSnRPN1lhYXNpTHZqdE1pQ3dFNnkrRW5GbUtC?= =?utf-8?B?UjRWM2UwMUFxVk82S21BcjBUUG8vaUZuSVMyaUgzdkU1WEVUc1NEbjB6Q0xQ?= =?utf-8?B?ZlJKSEw5cUxRbWdxUzFKZ2tMNmRUUGViSThzZFdwQ1g2aWE3SGVyak5qOENL?= =?utf-8?B?akR6aG96VU4yV1ZzN1c5Yk5ENFAwOGlMeXFzZldMQ0oxVERpdWZmNnE3Q1Yw?= =?utf-8?B?WjN1RlJzQVRNcnpnMmpRNUw1eUIwK1p1SVhxRVIxSzY1TEJBQlVXQ3hFbnJa?= =?utf-8?B?S3VRMFdZelh6RVF6Q1hFci84cUJ6ZndHaXFsM3dWeWdvLzlUM29rSUJZcjNw?= =?utf-8?B?UlBEYU5oQlcxUXdvZkNCb3hlZ3BtZHlRVzRMWUtZRkZ1VjlSQ0lFbjdRYit1?= =?utf-8?B?aFowbnZuSjUxV3ZQajhrU2FrUDNDT0t1YXg2aDl5TUtEcGcyYmVvdFpMK0ZG?= =?utf-8?B?anJ1cTR0MHozbVFuUVljOGVKUXN5ZG5DS1ptRzVHUkJ2WnVTZ050d1BuQzQr?= =?utf-8?B?d1Z2ekdNOWFkYzRrRllVL1h4cGZ0ckdtZjk4MlNJZGZFdjlGSUZJSWlEZEtG?= =?utf-8?B?T1lWdW9yaUg5NnJwL2RoOFlJSWJhYkVrUmFPeEg2aWpiN082UDU1NE5LSDU2?= =?utf-8?B?REFmUVFrZkcvY2J6TzYxRWlabkFpSngrRHN1RkhiclI0eU5YLzRRR3JmNklP?= =?utf-8?B?UWdMYUtVNENGT1NPMGw5RXAwSjducEh4dlFkOEpUMGR0WWRDRjlsZmUzQVZs?= =?utf-8?B?d0x6dEZTWlQ1NEtmNkpSaGh5Rjh0MUk1WWJ5UHd4QUFjQXozc0dWTWFHYk9M?= =?utf-8?B?bW5nWUlua1RvVlk4bGVHTFpTNUpIelVoU1ROTjMrWDQ1ZmMxQ1RSUW05ajFX?= =?utf-8?B?UWZWa0JoTFY2eDBmVGduTXZsdVlUU29LUDRITmZtRExkdHZRSlBFczlZTlkr?= =?utf-8?B?QUR4YXF2emYydTZrbjhyWmpMQTl4bFcyQTlLNzF3bm9Ib2VGSUZNaWhFaWkx?= =?utf-8?B?TmNMTjBqblhzY3gvanRPMFhvdHFtT0taU0xKaHZhR240Y2kxOUpnbEI1MEFC?= =?utf-8?B?bTI0ZE5NeHNRdmZMa0FNbEt2a3hmRXFNN1BCS1hIL3RNUUdIRVJSRWVaL3RT?= =?utf-8?B?SnMzTTlNbGw3Y0NIWDNFdFpGL0FjcmE2czMrdlJBaWtobGxJSStUK2lWbmJm?= =?utf-8?B?UWsxdjlsYStaYmUrekhieEpUcEpFaEFnbFFDWW95bHZnM0tBaGlucGx2NkpX?= =?utf-8?B?SlhwV0RHZ1Jlb1RIemErTW1qL3g5dXhYOTdaYUIvbFRuZHpuTjJhZ3U0cURo?= =?utf-8?B?ZlVPYTJ2OTZ2OG1qUWQ1RXZ2V0VTMXlxVGpqa05wTzBoMEJiZUU1WDh5Zzds?= =?utf-8?B?R1NUU3BCV29jRVNVeWpiQkNrUTBrc3JpWnV1YW13QzIvUTZPMGJjMjh6UEdU?= =?utf-8?B?d3ZQYVdpNUZqNXo2akZkL0NGK3RrSStTYWlHZC9UdC9XQVJHdXB5d1RhM2Y3?= =?utf-8?B?K3R6Yy9KUjdKNFZZdHg1UVd5aVk5SGxESGNuWUVOYlVOczJ3OEZqdFh4ZXNL?= =?utf-8?B?R1BoUmgxalBnPT0=?= X-Microsoft-Antispam-Message-Info: cn/iQ+3BCIsJ9HW4pdIU0Q+Rn8MmrvpMFIkxJIGxnvQpciNSAXVRMijyapbbRRSOaDggThRC3SBTTLojXRZEPuZOjVyPeZst5IQr/nFgZmty+N6vySAftUVD3k4MEeWQY9Le/bN7WApMYnoqdCEJ8qYdgihFivBVcsvnJic5XxcaXWsSgHcfnvxv+eEoYT7G X-Microsoft-Exchange-Diagnostics: 1; CO2PR07MB2518; 6:pJW3Lj4eoU6ZS0PEDnLc3maTnLhpHeYVUxxzBocFrtC/uFfjK4axq+NKR5HT4GA9RAb3ex+ozvY6aIydQtaIjeci0PjOCK+444i3yD4tC0fVOdY5nLNO/cuOerMUQFnIM4BNT/OBLdLciTME/evcRQphQ/tDLi1XFgqyMwZ5HLXUCVZlmoRJLx4FNY/2eeAgrQdTHGsd4BzgVxIPYqpClyyPHEclZp0Ubg3IWOVjMt7u5x7wy/U/6GReIvkakDfMMln7NfH7oiVJD/FFJ68S2uZvRp4ksRnP2MExniA745LeAYc5w7d8yNRl8sn2GfziCEbbVHJGBfHCbVTI/x6OrOMroYW4TjrLwO/EZ7RO8k+HjpDzKnVEYPDalwRW0FScbmJaq0/AtccxYdCrCJEYC23l55RfoEAJz/1JiEUNMtDTT1Z1U6qC9BYDYTZPqjr5/WNYNC7ypmDMJu7vll7LyQ==; 5:klB2LKt9IoW8a7VBCALelAx9lTQj6Rl9Ue08AQ87KhmlLhePPP/DHMT0LCo+E5vC72Kzxg8B35IgnpdwHzcx5zWbPpXHll1KVR+298W3jwWk8jZIs9gL/ErPGTRH1Lnw5MzYakz6+RCMKM++VQYEKJ33+7iGDiPfV4TDtkfDYk8=; 24:rfLJ3VEq9h79eZzIi1s88a0tCDsWh2PIKRKazLpOlOgLLOmKHNKeftAl9y1fsEYV5T6Nhnp2WzN9bEwiuPMoVnES4P3mCtucvlmXTMiSXJ4= SpamDiagnosticOutput: 1:99 SpamDiagnosticMetadata: NSPM X-Microsoft-Exchange-Diagnostics: 1; CO2PR07MB2518; 7:ukWlkKdzJMc2aeymiDRM1IVe37jBDMhKeRMgNvv1HEpDNZT2AgOWAM8hD+VD/+tskRGQFwGT+Qj/MC5fxdsxMbiB6wAtLb/xcINIsHxlH/hAAkA9aSBZBe+hr3xg7Qnz5WmAnkgcfybb7gWxnavfw72OviJ8y98E5KSUcYdYMZmiZdEubSnypzR/boMzYOHlLqiL5E0Lc9vCBOaPiJajjmlWl+AQoFZF5YwPadEnM71m6mZPwQZOBQoajgoDIwQ4 X-MS-Office365-Filtering-Correlation-Id: b0cd35df-9899-452d-e85a-08d5a9965642 X-OriginatorOrg: caviumnetworks.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 24 Apr 2018 03:49:07.9680 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: b0cd35df-9899-452d-e85a-08d5a9965642 X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 711e4ccf-2e9b-4bcf-a551-4094005b6194 X-MS-Exchange-Transport-CrossTenantHeadersStamped: CO2PR07MB2518 Subject: Re: [dpdk-dev] rte_hash thread safe X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 24 Apr 2018 03:49:14 -0000 -----Original Message----- > Date: Mon, 23 Apr 2018 17:48:50 -0700 > From: Jim Murphy > To: Stephen Hemminger > Cc: Brijesh Singh , dev@dpdk.org > Subject: Re: [dpdk-dev] rte_hash thread safe > > Anecdotally I've heard that the urcu hash implementation is slower than > rte_hash based on pure lookup performance. Has anyone considered adding RCU > hooks into rte_hash? For one of our internal project on arm64, we did try rte_hash vs URCU hash. Based on our results URCU lookup was much better. By default, URCU library does not allocate the memory from huge page, But it has some plugin based scheme to override the memory allocation scheme to choose hugepage using DPDK backend. > > > On Mon, Apr 23, 2018 at 5:30 PM, Stephen Hemminger < > stephen@networkplumber.org> wrote: > > > On Mon, 23 Apr 2018 17:21:15 -0700 > > Jim Murphy wrote: > > > > > Has anyone seen performance data comparing rte_hash (perhaps with r/w > > > locks) versus URCU hash? > > > > > > > > > On Mon, Apr 23, 2018 at 4:50 PM, Stephen Hemminger < > > > stephen@networkplumber.org> wrote: > > > > > > > On Mon, 23 Apr 2018 12:40:41 -0700 > > > > Brijesh Singh wrote: > > > > > > > > > A gentle reminder, > > > > > > > > > > I am curious to know if/how rte_hash is thread safe for lookups.It is > > > > > not obvious to me how following code is thread safe: > > > > > > > > > > _rte_hash_lookup_with_hash(const struct rte_hash *h, const void > > *key, > > > > > > > > > > hash_sig_t sig, void **data) > > > > > > > > > > { > > > > > > > > > > > > > > > > > > > > … > > > > > > > > > > if (rte_hash_cmp_eq(key, k->key, h) == 0) { > > > > > > > > > > if (data != NULL) > > > > > > > > > > *data = k->pdata; > > > > > > > > > > } > > > > > > > > > > a key could be deleted and another key inserted in its slot while the > > > > > lookup is happening. For example, in the following sequence of > > events: > > > > > The slot has Key1,V1 > > > > > Lookup Thread T1 compares the input key to Key1 and it matches. The > > > > > thread gets context switched out > > > > > Thread T2 deletes Key1. > > > > > Thread T2 inserts Key2 with value V2. > > > > > T1 reads the data from the slot and returns V2. This is incorrect. > > > > > > > > > > > > > > > Regards, > > > > > Brijesh > > > > > > > > > > On Wed, Apr 11, 2018 at 9:12 PM, Brijesh Singh > > > > > wrote: > > > > > > Hello, > > > > > > > > > > > > I want to use DPDK's rte_hash library to keep track of tcp flows. > > The > > > > > > lookups will be done by multiple threads but inserts will be done > > only > > > > > > on one thread. > > > > > > > > > > > > As per the documentation rte_hash library has thread safe lookups. > > Key > > > > > > /data inserts should be done on single thread, since those > > operations > > > > > > are not thread safe. Is this documentation still correct? > > > > > > > > > > > > The lookup code compares the key and returns the data if the key > > > > > > matches, this doesn't look like thread safe. Am I missing > > something? > > > > > > > > > > > > _rte_hash_lookup_with_hash(const struct rte_hash *h, const void > > *key, > > > > > > > > > > > > hash_sig_t sig, void > > **data) > > > > > > > > > > > > { > > > > > > > > > > > > > > > > > > > > > > > > … > > > > > > > > > > > > if (rte_hash_cmp_eq(key, k->key, h) == 0) { > > > > > > > > > > > > if (data != NULL) > > > > > > > > > > > > *data = k->pdata; > > > > > > > > > > > > } > > > > > > > > > > > > Regards, > > > > > > Brijesh > > > > > > > > The best way to handle this is to do some kind of Read Copy Update. > > > > Unfortunately, this is not possible in scope of DPDK since it requires > > > > cooperation from application threads. > > > > > > > > If you need thread safe hash table, my recommendation would be to > > > > skip the DPDK hash library and use userspace RCU instead: > > > > http://liburcu.org/ > > > > > > > > Note: URCU is LGPL versus BSD licensed. But then any non trivial > > > > Linux application needs to use LGPL libraries already. > > > > > > > > No. But R/W locks are really slow. Look at one of Paul Mc Kenney's many RCU > > talks and papers. > >