From mboxrd@z Thu Jan 1 00:00:00 1970
Return-Path: <dev-bounces@dpdk.org>
Received: from dpdk.org (dpdk.org [92.243.14.124])
by inbox.dpdk.org (Postfix) with ESMTP id 17FA0A00C4;
Thu, 4 Jun 2020 21:34:31 +0200 (CEST)
Received: from [92.243.14.124] (localhost [127.0.0.1])
by dpdk.org (Postfix) with ESMTP id 174B11D5D5;
Thu, 4 Jun 2020 21:34:30 +0200 (CEST)
Received: from EUR03-DB5-obe.outbound.protection.outlook.com
(mail-eopbgr40064.outbound.protection.outlook.com [40.107.4.64])
by dpdk.org (Postfix) with ESMTP id 2BFCC1D5C6
for <dev@dpdk.org>; Thu, 4 Jun 2020 21:34:28 +0200 (CEST)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=armh.onmicrosoft.com;
s=selector2-armh-onmicrosoft-com;
h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
bh=frZ8rvVU4Y+8+ZfI8IzdkDFzpEpYQ9BDw489rQzofpA=;
b=xK8Uv4qULnP5fizbIdk1DQTgbTIROidJfYsfMrPsay8P6Q71lefNfuDBft5MAHJZ11unQ1tCL9Gt59QFgPB/vOKb1fgcw2S8zLugef2cwMvdEnIt+kN1ap1AXsEQ8LvproB5P4su+JTRJbjF2M/0CTl6V6aBClYFBOvFC2BEtJM=
Received: from AM5PR0101CA0022.eurprd01.prod.exchangelabs.com
(2603:10a6:206:16::35) by AM5PR0802MB2596.eurprd08.prod.outlook.com
(2603:10a6:203:98::8) with Microsoft SMTP Server (version=TLS1_2,
cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.3066.18; Thu, 4 Jun
2020 19:34:26 +0000
Received: from AM5EUR03FT004.eop-EUR03.prod.protection.outlook.com
(2603:10a6:206:16:cafe::9c) by AM5PR0101CA0022.outlook.office365.com
(2603:10a6:206:16::35) with Microsoft SMTP Server (version=TLS1_2,
cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.3066.18 via Frontend
Transport; Thu, 4 Jun 2020 19:34:26 +0000
X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 63.35.35.123)
smtp.mailfrom=arm.com; dpdk.org; dkim=pass (signature was verified)
header.d=armh.onmicrosoft.com;dpdk.org; dmarc=bestguesspass action=none
header.from=arm.com;
Received-SPF: Pass (protection.outlook.com: domain of arm.com designates
63.35.35.123 as permitted sender) receiver=protection.outlook.com;
client-ip=63.35.35.123; helo=64aa7808-outbound-1.mta.getcheckrecipient.com;
Received: from 64aa7808-outbound-1.mta.getcheckrecipient.com (63.35.35.123) by
AM5EUR03FT004.mail.protection.outlook.com (10.152.16.163) with
Microsoft SMTP
Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
15.20.3066.18 via Frontend Transport; Thu, 4 Jun 2020 19:34:26 +0000
Received: ("Tessian outbound fb809da9b456:v59");
Thu, 04 Jun 2020 19:34:26 +0000
X-CR-MTA-TID: 64aa7808
Received: from 94cac4b043ab.2
by 64aa7808-outbound-1.mta.getcheckrecipient.com id
FDF166B5-954C-4D4B-9FA6-939A14087EC2.1;
Thu, 04 Jun 2020 19:34:21 +0000
Received: from EUR03-AM5-obe.outbound.protection.outlook.com
by 64aa7808-outbound-1.mta.getcheckrecipient.com with ESMTPS id 94cac4b043ab.2
(version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384);
Thu, 04 Jun 2020 19:34:21 +0000
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
b=TzeWRGr3BrE7l5H9KXJkhcqvCf1PNjch3TTx0SR/BcvSqDYAoyaAJHeEd1LNj+aEoqzwjgkGMEBICDwjRBWHD6Zdsn4AI7iNxk4r4C7seP25i74hI4bqudIIS7OswtqIlbCPF0TxwJ9xih6mq2WVs/Tl5dQ/dJPN5c/1S903iP0oM5C085GUSdsOpm49rAkm62eHiCeXsS9tuGnQkTe9kV9OdX533DZAe3GNf4qK3TLR9ZeW1jOVhgVHCn0JtEA9oJUyx6PMEDH8j2+P8R532D8rGT40noFWIStLVYpzFQzDluAbF5Y59qUT+ss/+oXTxWxXLaE77xEqx6RiyqYYAQ==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
s=arcselector9901;
h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
bh=frZ8rvVU4Y+8+ZfI8IzdkDFzpEpYQ9BDw489rQzofpA=;
b=MTg6wNwuOqmkvGGlk+XrKealGeOfHXs4ZMCsyUGQOVZVCyGqHS7AgGTob1u4NaYJ01wP2oMVtFUYKPKofj6ypLvUxqEF7Lfg16IKJnCQ/TnPa99E0CE22ypy3EZ5uAWO7PcNsMphPUJmnFnM+LhqeoNvNK62D6tKlqXjLFaHBAkOellUMHB+IXvy8zwPxrxLbcSbRdjN2ljw343NiL3PVPD4uKsra40a8pE6cr5daJPa3XP8fsWiwy4WOZjSwh/ztR2r74LxRkDku7dK4klbdBzqt7Eg8fW4NyA2fWWcJJm8+rEmfNwUHr/CTjCS/RbjxgINTglERag5wONlYb7uGg==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
smtp.mailfrom=arm.com; dmarc=pass action=none header.from=arm.com; dkim=pass
header.d=arm.com; arc=none
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=armh.onmicrosoft.com;
s=selector2-armh-onmicrosoft-com;
h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
bh=frZ8rvVU4Y+8+ZfI8IzdkDFzpEpYQ9BDw489rQzofpA=;
b=xK8Uv4qULnP5fizbIdk1DQTgbTIROidJfYsfMrPsay8P6Q71lefNfuDBft5MAHJZ11unQ1tCL9Gt59QFgPB/vOKb1fgcw2S8zLugef2cwMvdEnIt+kN1ap1AXsEQ8LvproB5P4su+JTRJbjF2M/0CTl6V6aBClYFBOvFC2BEtJM=
Received: from AM4PR0802MB2212.eurprd08.prod.outlook.com (2603:10a6:200:59::8)
by AM4PR0802MB2132.eurprd08.prod.outlook.com (2603:10a6:200:5a::11)
with Microsoft SMTP Server (version=TLS1_2,
cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.3066.18; Thu, 4 Jun
2020 19:34:20 +0000
Received: from AM4PR0802MB2212.eurprd08.prod.outlook.com
([fe80::9130:1e61:ef03:c4e1]) by AM4PR0802MB2212.eurprd08.prod.outlook.com
([fe80::9130:1e61:ef03:c4e1%3]) with mapi id 15.20.3066.019; Thu, 4 Jun 2020
19:34:19 +0000
From: Honnappa Nagarahalli <Honnappa.Nagarahalli@arm.com>
To: "Wang, Yipeng1" <yipeng1.wang@intel.com>, Stephen Hemminger
<stephen@networkplumber.org>
CC: "Gobriel, Sameh" <sameh.gobriel@intel.com>, "Richardson, Bruce"
<bruce.richardson@intel.com>, "dev@dpdk.org" <dev@dpdk.org>, "De Lara Guarch,
Pablo" <pablo.de.lara.guarch@intel.com>, nd <nd@arm.com>, Honnappa
Nagarahalli <Honnappa.Nagarahalli@arm.com>, nd <nd@arm.com>
Thread-Topic: [PATCH] hash: document breakage with multi-writer thread
Thread-Index: AQHWOpQbSbTXGH7QCEO1ao+t3FmiRajIuzHAgAADGYCAAAbIYIAADW8AgAAEzpA=
Date: Thu, 4 Jun 2020 19:34:19 +0000
Message-ID: <AM4PR0802MB2212599ACC592EC4A42965A298890@AM4PR0802MB2212.eurprd08.prod.outlook.com>
References: <20200604171731.6738-1-stephen@networkplumber.org>
<DB6PR0802MB221691379E0AAD8CAC76F3C398890@DB6PR0802MB2216.eurprd08.prod.outlook.com>
<20200604105817.1a3a2749@hermes.lan>
<AM4PR0802MB22129FFB7504E7161E68B6B998890@AM4PR0802MB2212.eurprd08.prod.outlook.com>
<BYAPR11MB3494A1653D9DD9B4958F6BD7C3890@BYAPR11MB3494.namprd11.prod.outlook.com>
In-Reply-To: <BYAPR11MB3494A1653D9DD9B4958F6BD7C3890@BYAPR11MB3494.namprd11.prod.outlook.com>
Accept-Language: en-US
Content-Language: en-US
X-MS-Has-Attach:
X-MS-TNEF-Correlator:
x-ts-tracking-id: 909bb158-4f59-41fd-a8b2-1c5d542f41fd.0
x-checkrecipientchecked: true
Authentication-Results-Original: intel.com; dkim=none (message not signed)
header.d=none;intel.com; dmarc=none action=none header.from=arm.com;
x-originating-ip: [70.113.25.165]
x-ms-publictraffictype: Email
X-MS-Office365-Filtering-HT: Tenant
X-MS-Office365-Filtering-Correlation-Id: da2a7348-256b-43b8-ced7-08d808be4add
x-ms-traffictypediagnostic: AM4PR0802MB2132:|AM5PR0802MB2596:
x-ms-exchange-transport-forked: True
X-Microsoft-Antispam-PRVS: <AM5PR0802MB259696C29FC7C96562A7FB0898890@AM5PR0802MB2596.eurprd08.prod.outlook.com>
x-checkrecipientrouted: true
nodisclaimer: true
x-ms-oob-tlc-oobclassifiers: OLM:10000;OLM:10000;
x-forefront-prvs: 04244E0DC5
X-MS-Exchange-SenderADCheck: 1
X-Microsoft-Antispam-Untrusted: BCL:0;
X-Microsoft-Antispam-Message-Info-Original: jWwQp5CtLjH2rfj0W9IX/jEUOuPlP8JyQq6R/DKZOGsRg8l5xRVe2Utt8Q7CmZnt9lsRB1JDSrfr7wnmNWFEKxVRWFu55CBjRTlMxhurtlq/x5KZsvANOrrwtd2Yj8tTMoVSz5/Rw4UwYbffuUZu2jp4iSog8RUxXZS/61NU8KRBTiIwE/l1KG0KMmdlRtB10HIsWlMh+Q8fFIvHV+9s+lM/xbcqYd1nmUgahy88dxFE0dMq1WYmR+eW1CTfOJlDUx+SqFYkPTs2yXNfGIygEO48uYHrcXFhHIyuF3YBRhd3nOYb53KmjqWSkYH7/Jej
X-Forefront-Antispam-Report-Untrusted: CIP:255.255.255.255; CTRY:; LANG:en;
SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:AM4PR0802MB2212.eurprd08.prod.outlook.com;
PTR:; CAT:NONE; SFTY:;
SFS:(4636009)(396003)(136003)(366004)(39860400002)(346002)(376002)(5660300002)(76116006)(8936002)(54906003)(7696005)(6506007)(4326008)(478600001)(33656002)(316002)(110136005)(71200400001)(52536014)(83380400001)(66476007)(26005)(86362001)(186003)(8676002)(66446008)(9686003)(55016002)(66556008)(66946007)(64756008)(2906002);
DIR:OUT; SFP:1101;
x-ms-exchange-antispam-messagedata: Tge+r/szTsnj5iPLeG5CtpB8ZIuSKQDG67y0c7mbkFFpRRJxbR8GdFS429aY+QySAlh4VJ++7zHDo/TRcSZfbkXqH1MTUyvkHFrc4qHtQgnNLAwkXTJ4rDm07DIC/eMrOQOBIOy0+QihhgNyuSOkbWK9rRheJnKfbwn7dMwHrjgXL9eA4OXYI1CYPNEThBHPnVXUHFmCS+JYKUMMSmOBfPVKj4rY+VCWnEVQ7NzpBdm2q9CV7Ue0dunOeSKHwxPjj1XamF5f5Rr19jY044UVL+0J3q56KL5ll/h/wmkdzcGmuWYekQRoH8zL0t1VjDqYMCBgIYVjbWPFfV0CoGaZ9/ZNZb7cNh8gLlwmYlLAe8hupnC2ePmYSHMioNMJyrUUjxqWPi5UXh7i0yfbO+oll+0s4lAXOJtlpJ4OYFvbQFTu1dlsUnEiYKYzt1c9MraTSF1CMxpDDYKENHPnknjdax+BJRa7Q0GUXbhjoJpiainqFD0kpNUP3arZfL8GcqyH
Content-Type: text/plain; charset="us-ascii"
Content-Transfer-Encoding: quoted-printable
MIME-Version: 1.0
X-MS-Exchange-Transport-CrossTenantHeadersStamped: AM4PR0802MB2132
Original-Authentication-Results: intel.com; dkim=none (message not signed)
header.d=none;intel.com; dmarc=none action=none header.from=arm.com;
X-EOPAttributedMessage: 0
X-MS-Exchange-Transport-CrossTenantHeadersStripped: AM5EUR03FT004.eop-EUR03.prod.protection.outlook.com
X-Forefront-Antispam-Report: CIP:63.35.35.123; CTRY:IE; LANG:en; SCL:1; SRV:;
IPV:CAL; SFV:NSPM; H:64aa7808-outbound-1.mta.getcheckrecipient.com;
PTR:ec2-63-35-35-123.eu-west-1.compute.amazonaws.com; CAT:NONE; SFTY:;
SFS:(4636009)(346002)(39860400002)(376002)(136003)(396003)(46966005)(2906002)(82740400003)(5660300002)(33656002)(336012)(52536014)(186003)(55016002)(86362001)(9686003)(7696005)(356005)(82310400002)(47076004)(6506007)(316002)(26005)(36906005)(8676002)(70586007)(110136005)(54906003)(70206006)(81166007)(4326008)(8936002)(478600001)(83380400001);
DIR:OUT; SFP:1101;
X-MS-Office365-Filtering-Correlation-Id-Prvs: 4f1018c4-f1b3-4ba1-75f5-08d808be46ee
X-Forefront-PRVS: 04244E0DC5
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info: AyaaU/dIx9o8v0u3QIiu75doL9VH4HGPJads53fClyWa1lQaFeShbW2/rALw8ntzdWOLmHSLn8zlAE4z/LwZt/33mcT7nSuBHUE+Gw/r9ux3b71Zm1K8XPM+KCfAziG1S9ziAZSQm/Wfxw1m8p1+rojfMgIA+KsXAoll821urzUCMziIhGGYNZttpyTup7XgK1gBEwfs574nxtiuXSWIsiTVxzdShSjPNoUQ291KqIUeAqFFE45KdPSNx6CXaKdr+6xrzLkq99+1gPZhETdqQvc2882TRK/gvNbzsRGXbn94kqwcHcpKSmp/U8pr8CByrVFBtdPOZ+i9nnhdmGOrMVlN/m5FRbgnuISZu48b2ZMFdOZFimvkn/1P91xzrQKW/Fo/buSTbVK+HPoRPyOP0Q==
X-OriginatorOrg: arm.com
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 04 Jun 2020 19:34:26.5185 (UTC)
X-MS-Exchange-CrossTenant-Network-Message-Id: da2a7348-256b-43b8-ced7-08d808be4add
X-MS-Exchange-CrossTenant-Id: f34e5979-57d9-4aaa-ad4d-b122a662184d
X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=f34e5979-57d9-4aaa-ad4d-b122a662184d; Ip=[63.35.35.123];
Helo=[64aa7808-outbound-1.mta.getcheckrecipient.com]
X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem
X-MS-Exchange-Transport-CrossTenantHeadersStamped: AM5PR0802MB2596
Subject: Re: [dpdk-dev] [PATCH] hash: document breakage with multi-writer
thread
X-BeenThere: dev@dpdk.org
X-Mailman-Version: 2.1.15
Precedence: list
List-Id: DPDK patches and discussions <dev.dpdk.org>
List-Unsubscribe: <https://mails.dpdk.org/options/dev>,
<mailto:dev-request@dpdk.org?subject=unsubscribe>
List-Archive: <http://mails.dpdk.org/archives/dev/>
List-Post: <mailto:dev@dpdk.org>
List-Help: <mailto:dev-request@dpdk.org?subject=help>
List-Subscribe: <https://mails.dpdk.org/listinfo/dev>,
<mailto:dev-request@dpdk.org?subject=subscribe>
Errors-To: dev-bounces@dpdk.org
Sender: "dev" <dev-bounces@dpdk.org>
<snip>
> > > >
> > > > > Subject: [PATCH] hash: document breakage with multi-writer
> > > > > thread
> > > > >
> > > > > The code in rte_cuckoo_hash multi-writer support is broken if
> > > > > write operations are called from a non-EAL thread.
> > > > >
> > > > > rte_lcore_id() wil return LCORE_ID_ANY (UINT32_MAX) for non EAL
> > > > > thread and that leads to using wrong local cache.
> > > > >
> > > > > Add error checks and document the restriction.
> > > > Having multiple non-EAL writer threads is a valid use case. Should
> > > > we fix the
> > > issue instead?
> > >
> > > Discovered this the hard way...
> > >
> > > Fixing is non-trivial. Basically, the local cache has to be take out
> > > and that leads to having to do real locking or atomic operations.
> > Looking at rte_hash_create function:
> >
> > if (params->extra_flag &
> > RTE_HASH_EXTRA_FLAGS_MULTI_WRITER_ADD) {
> > use_local_cache =3D 1;
> > writer_takes_lock =3D 1;
> > }
> >
> > The writer locks are in place already. The code to handle the case
> > when local cache is taken out is also there.
> > What we need is another input flag that says 'multi writer + non-eal
> threads'
> > which would set 'use_local_cache =3D 0' and 'writer_takes_lock =3D 1'.
> > Not sure, it would be valuable addition. But looks like this is what
> > you were expecting when you had enabled
> > 'RTE_HASH_EXTRA_FLAGS_MULTI_WRITER_ADD'. Many other APIs in DPDK
> do
> > not provide this kind of MT safety.
>=20
> [Wang, Yipeng]
> If possible, we can try to not add new flags, because there are already a=
lot of
> flag options.
> How about in the code, we check if the writer is a non-eal or not by chec=
king
> the rte_lcore_id, and operate on the global queue?
> Could this work?
> If(h->use_local_cache) {
> lcore_id =3D rte_lcore_id();
> if(lcore_id =3D=3D LCORE_ID_ANY) { // this is non-eal threads
> <call rte_ring_mp/mc_* to directly operate on global queue>
> }
> Else {
> <original path>
> }
> }
The other thing I wanted to do was saving on the memory allocated for the l=
ocal cache when the writers are non-eal threads. Without knowing the kind o=
f threads upfront, we might have to create the local cache when a writer ad=
ds the entry first time.