From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from dpdk.org (dpdk.org [92.243.14.124]) by inbox.dpdk.org (Postfix) with ESMTP id 17FA0A00C4; Thu, 4 Jun 2020 21:34:31 +0200 (CEST) Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id 174B11D5D5; Thu, 4 Jun 2020 21:34:30 +0200 (CEST) Received: from EUR03-DB5-obe.outbound.protection.outlook.com (mail-eopbgr40064.outbound.protection.outlook.com [40.107.4.64]) by dpdk.org (Postfix) with ESMTP id 2BFCC1D5C6 for ; Thu, 4 Jun 2020 21:34:28 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=armh.onmicrosoft.com; s=selector2-armh-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=frZ8rvVU4Y+8+ZfI8IzdkDFzpEpYQ9BDw489rQzofpA=; b=xK8Uv4qULnP5fizbIdk1DQTgbTIROidJfYsfMrPsay8P6Q71lefNfuDBft5MAHJZ11unQ1tCL9Gt59QFgPB/vOKb1fgcw2S8zLugef2cwMvdEnIt+kN1ap1AXsEQ8LvproB5P4su+JTRJbjF2M/0CTl6V6aBClYFBOvFC2BEtJM= Received: from AM5PR0101CA0022.eurprd01.prod.exchangelabs.com (2603:10a6:206:16::35) by AM5PR0802MB2596.eurprd08.prod.outlook.com (2603:10a6:203:98::8) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.3066.18; Thu, 4 Jun 2020 19:34:26 +0000 Received: from AM5EUR03FT004.eop-EUR03.prod.protection.outlook.com (2603:10a6:206:16:cafe::9c) by AM5PR0101CA0022.outlook.office365.com (2603:10a6:206:16::35) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.3066.18 via Frontend Transport; Thu, 4 Jun 2020 19:34:26 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 63.35.35.123) smtp.mailfrom=arm.com; dpdk.org; dkim=pass (signature was verified) header.d=armh.onmicrosoft.com;dpdk.org; dmarc=bestguesspass action=none header.from=arm.com; Received-SPF: Pass (protection.outlook.com: domain of arm.com designates 63.35.35.123 as permitted sender) receiver=protection.outlook.com; client-ip=63.35.35.123; helo=64aa7808-outbound-1.mta.getcheckrecipient.com; Received: from 64aa7808-outbound-1.mta.getcheckrecipient.com (63.35.35.123) by AM5EUR03FT004.mail.protection.outlook.com (10.152.16.163) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.3066.18 via Frontend Transport; Thu, 4 Jun 2020 19:34:26 +0000 Received: ("Tessian outbound fb809da9b456:v59"); Thu, 04 Jun 2020 19:34:26 +0000 X-CR-MTA-TID: 64aa7808 Received: from 94cac4b043ab.2 by 64aa7808-outbound-1.mta.getcheckrecipient.com id FDF166B5-954C-4D4B-9FA6-939A14087EC2.1; Thu, 04 Jun 2020 19:34:21 +0000 Received: from EUR03-AM5-obe.outbound.protection.outlook.com by 64aa7808-outbound-1.mta.getcheckrecipient.com with ESMTPS id 94cac4b043ab.2 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384); Thu, 04 Jun 2020 19:34:21 +0000 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=TzeWRGr3BrE7l5H9KXJkhcqvCf1PNjch3TTx0SR/BcvSqDYAoyaAJHeEd1LNj+aEoqzwjgkGMEBICDwjRBWHD6Zdsn4AI7iNxk4r4C7seP25i74hI4bqudIIS7OswtqIlbCPF0TxwJ9xih6mq2WVs/Tl5dQ/dJPN5c/1S903iP0oM5C085GUSdsOpm49rAkm62eHiCeXsS9tuGnQkTe9kV9OdX533DZAe3GNf4qK3TLR9ZeW1jOVhgVHCn0JtEA9oJUyx6PMEDH8j2+P8R532D8rGT40noFWIStLVYpzFQzDluAbF5Y59qUT+ss/+oXTxWxXLaE77xEqx6RiyqYYAQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=frZ8rvVU4Y+8+ZfI8IzdkDFzpEpYQ9BDw489rQzofpA=; b=MTg6wNwuOqmkvGGlk+XrKealGeOfHXs4ZMCsyUGQOVZVCyGqHS7AgGTob1u4NaYJ01wP2oMVtFUYKPKofj6ypLvUxqEF7Lfg16IKJnCQ/TnPa99E0CE22ypy3EZ5uAWO7PcNsMphPUJmnFnM+LhqeoNvNK62D6tKlqXjLFaHBAkOellUMHB+IXvy8zwPxrxLbcSbRdjN2ljw343NiL3PVPD4uKsra40a8pE6cr5daJPa3XP8fsWiwy4WOZjSwh/ztR2r74LxRkDku7dK4klbdBzqt7Eg8fW4NyA2fWWcJJm8+rEmfNwUHr/CTjCS/RbjxgINTglERag5wONlYb7uGg== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=arm.com; dmarc=pass action=none header.from=arm.com; dkim=pass header.d=arm.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=armh.onmicrosoft.com; s=selector2-armh-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=frZ8rvVU4Y+8+ZfI8IzdkDFzpEpYQ9BDw489rQzofpA=; b=xK8Uv4qULnP5fizbIdk1DQTgbTIROidJfYsfMrPsay8P6Q71lefNfuDBft5MAHJZ11unQ1tCL9Gt59QFgPB/vOKb1fgcw2S8zLugef2cwMvdEnIt+kN1ap1AXsEQ8LvproB5P4su+JTRJbjF2M/0CTl6V6aBClYFBOvFC2BEtJM= Received: from AM4PR0802MB2212.eurprd08.prod.outlook.com (2603:10a6:200:59::8) by AM4PR0802MB2132.eurprd08.prod.outlook.com (2603:10a6:200:5a::11) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.3066.18; Thu, 4 Jun 2020 19:34:20 +0000 Received: from AM4PR0802MB2212.eurprd08.prod.outlook.com ([fe80::9130:1e61:ef03:c4e1]) by AM4PR0802MB2212.eurprd08.prod.outlook.com ([fe80::9130:1e61:ef03:c4e1%3]) with mapi id 15.20.3066.019; Thu, 4 Jun 2020 19:34:19 +0000 From: Honnappa Nagarahalli To: "Wang, Yipeng1" , Stephen Hemminger CC: "Gobriel, Sameh" , "Richardson, Bruce" , "dev@dpdk.org" , "De Lara Guarch, Pablo" , nd , Honnappa Nagarahalli , nd Thread-Topic: [PATCH] hash: document breakage with multi-writer thread Thread-Index: AQHWOpQbSbTXGH7QCEO1ao+t3FmiRajIuzHAgAADGYCAAAbIYIAADW8AgAAEzpA= Date: Thu, 4 Jun 2020 19:34:19 +0000 Message-ID: References: <20200604171731.6738-1-stephen@networkplumber.org> <20200604105817.1a3a2749@hermes.lan> In-Reply-To: Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-ts-tracking-id: 909bb158-4f59-41fd-a8b2-1c5d542f41fd.0 x-checkrecipientchecked: true Authentication-Results-Original: intel.com; dkim=none (message not signed) header.d=none;intel.com; dmarc=none action=none header.from=arm.com; x-originating-ip: [70.113.25.165] x-ms-publictraffictype: Email X-MS-Office365-Filtering-HT: Tenant X-MS-Office365-Filtering-Correlation-Id: da2a7348-256b-43b8-ced7-08d808be4add x-ms-traffictypediagnostic: AM4PR0802MB2132:|AM5PR0802MB2596: x-ms-exchange-transport-forked: True X-Microsoft-Antispam-PRVS: x-checkrecipientrouted: true nodisclaimer: true x-ms-oob-tlc-oobclassifiers: OLM:10000;OLM:10000; x-forefront-prvs: 04244E0DC5 X-MS-Exchange-SenderADCheck: 1 X-Microsoft-Antispam-Untrusted: BCL:0; X-Microsoft-Antispam-Message-Info-Original: jWwQp5CtLjH2rfj0W9IX/jEUOuPlP8JyQq6R/DKZOGsRg8l5xRVe2Utt8Q7CmZnt9lsRB1JDSrfr7wnmNWFEKxVRWFu55CBjRTlMxhurtlq/x5KZsvANOrrwtd2Yj8tTMoVSz5/Rw4UwYbffuUZu2jp4iSog8RUxXZS/61NU8KRBTiIwE/l1KG0KMmdlRtB10HIsWlMh+Q8fFIvHV+9s+lM/xbcqYd1nmUgahy88dxFE0dMq1WYmR+eW1CTfOJlDUx+SqFYkPTs2yXNfGIygEO48uYHrcXFhHIyuF3YBRhd3nOYb53KmjqWSkYH7/Jej X-Forefront-Antispam-Report-Untrusted: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:AM4PR0802MB2212.eurprd08.prod.outlook.com; PTR:; CAT:NONE; SFTY:; SFS:(4636009)(396003)(136003)(366004)(39860400002)(346002)(376002)(5660300002)(76116006)(8936002)(54906003)(7696005)(6506007)(4326008)(478600001)(33656002)(316002)(110136005)(71200400001)(52536014)(83380400001)(66476007)(26005)(86362001)(186003)(8676002)(66446008)(9686003)(55016002)(66556008)(66946007)(64756008)(2906002); DIR:OUT; SFP:1101; x-ms-exchange-antispam-messagedata: Tge+r/szTsnj5iPLeG5CtpB8ZIuSKQDG67y0c7mbkFFpRRJxbR8GdFS429aY+QySAlh4VJ++7zHDo/TRcSZfbkXqH1MTUyvkHFrc4qHtQgnNLAwkXTJ4rDm07DIC/eMrOQOBIOy0+QihhgNyuSOkbWK9rRheJnKfbwn7dMwHrjgXL9eA4OXYI1CYPNEThBHPnVXUHFmCS+JYKUMMSmOBfPVKj4rY+VCWnEVQ7NzpBdm2q9CV7Ue0dunOeSKHwxPjj1XamF5f5Rr19jY044UVL+0J3q56KL5ll/h/wmkdzcGmuWYekQRoH8zL0t1VjDqYMCBgIYVjbWPFfV0CoGaZ9/ZNZb7cNh8gLlwmYlLAe8hupnC2ePmYSHMioNMJyrUUjxqWPi5UXh7i0yfbO+oll+0s4lAXOJtlpJ4OYFvbQFTu1dlsUnEiYKYzt1c9MraTSF1CMxpDDYKENHPnknjdax+BJRa7Q0GUXbhjoJpiainqFD0kpNUP3arZfL8GcqyH Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 X-MS-Exchange-Transport-CrossTenantHeadersStamped: AM4PR0802MB2132 Original-Authentication-Results: intel.com; dkim=none (message not signed) header.d=none;intel.com; dmarc=none action=none header.from=arm.com; X-EOPAttributedMessage: 0 X-MS-Exchange-Transport-CrossTenantHeadersStripped: AM5EUR03FT004.eop-EUR03.prod.protection.outlook.com X-Forefront-Antispam-Report: CIP:63.35.35.123; CTRY:IE; LANG:en; SCL:1; SRV:; IPV:CAL; SFV:NSPM; H:64aa7808-outbound-1.mta.getcheckrecipient.com; PTR:ec2-63-35-35-123.eu-west-1.compute.amazonaws.com; CAT:NONE; SFTY:; SFS:(4636009)(346002)(39860400002)(376002)(136003)(396003)(46966005)(2906002)(82740400003)(5660300002)(33656002)(336012)(52536014)(186003)(55016002)(86362001)(9686003)(7696005)(356005)(82310400002)(47076004)(6506007)(316002)(26005)(36906005)(8676002)(70586007)(110136005)(54906003)(70206006)(81166007)(4326008)(8936002)(478600001)(83380400001); DIR:OUT; SFP:1101; X-MS-Office365-Filtering-Correlation-Id-Prvs: 4f1018c4-f1b3-4ba1-75f5-08d808be46ee X-Forefront-PRVS: 04244E0DC5 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: AyaaU/dIx9o8v0u3QIiu75doL9VH4HGPJads53fClyWa1lQaFeShbW2/rALw8ntzdWOLmHSLn8zlAE4z/LwZt/33mcT7nSuBHUE+Gw/r9ux3b71Zm1K8XPM+KCfAziG1S9ziAZSQm/Wfxw1m8p1+rojfMgIA+KsXAoll821urzUCMziIhGGYNZttpyTup7XgK1gBEwfs574nxtiuXSWIsiTVxzdShSjPNoUQ291KqIUeAqFFE45KdPSNx6CXaKdr+6xrzLkq99+1gPZhETdqQvc2882TRK/gvNbzsRGXbn94kqwcHcpKSmp/U8pr8CByrVFBtdPOZ+i9nnhdmGOrMVlN/m5FRbgnuISZu48b2ZMFdOZFimvkn/1P91xzrQKW/Fo/buSTbVK+HPoRPyOP0Q== X-OriginatorOrg: arm.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 04 Jun 2020 19:34:26.5185 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: da2a7348-256b-43b8-ced7-08d808be4add X-MS-Exchange-CrossTenant-Id: f34e5979-57d9-4aaa-ad4d-b122a662184d X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=f34e5979-57d9-4aaa-ad4d-b122a662184d; Ip=[63.35.35.123]; Helo=[64aa7808-outbound-1.mta.getcheckrecipient.com] X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: AM5PR0802MB2596 Subject: Re: [dpdk-dev] [PATCH] hash: document breakage with multi-writer thread X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" > > > > > > > > > Subject: [PATCH] hash: document breakage with multi-writer > > > > > thread > > > > > > > > > > The code in rte_cuckoo_hash multi-writer support is broken if > > > > > write operations are called from a non-EAL thread. > > > > > > > > > > rte_lcore_id() wil return LCORE_ID_ANY (UINT32_MAX) for non EAL > > > > > thread and that leads to using wrong local cache. > > > > > > > > > > Add error checks and document the restriction. > > > > Having multiple non-EAL writer threads is a valid use case. Should > > > > we fix the > > > issue instead? > > > > > > Discovered this the hard way... > > > > > > Fixing is non-trivial. Basically, the local cache has to be take out > > > and that leads to having to do real locking or atomic operations. > > Looking at rte_hash_create function: > > > > if (params->extra_flag & > > RTE_HASH_EXTRA_FLAGS_MULTI_WRITER_ADD) { > > use_local_cache =3D 1; > > writer_takes_lock =3D 1; > > } > > > > The writer locks are in place already. The code to handle the case > > when local cache is taken out is also there. > > What we need is another input flag that says 'multi writer + non-eal > threads' > > which would set 'use_local_cache =3D 0' and 'writer_takes_lock =3D 1'. > > Not sure, it would be valuable addition. But looks like this is what > > you were expecting when you had enabled > > 'RTE_HASH_EXTRA_FLAGS_MULTI_WRITER_ADD'. Many other APIs in DPDK > do > > not provide this kind of MT safety. >=20 > [Wang, Yipeng] > If possible, we can try to not add new flags, because there are already a= lot of > flag options. > How about in the code, we check if the writer is a non-eal or not by chec= king > the rte_lcore_id, and operate on the global queue? > Could this work? > If(h->use_local_cache) { > lcore_id =3D rte_lcore_id(); > if(lcore_id =3D=3D LCORE_ID_ANY) { // this is non-eal threads > > } > Else { > > } > } The other thing I wanted to do was saving on the memory allocated for the l= ocal cache when the writers are non-eal threads. Without knowing the kind o= f threads upfront, we might have to create the local cache when a writer ad= ds the entry first time.