From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from dpdk.org (dpdk.org [92.243.14.124]) by inbox.dpdk.org (Postfix) with ESMTP id 44D78A00C4; Thu, 4 Jun 2020 22:22:15 +0200 (CEST) Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id 062DF1D5E2; Thu, 4 Jun 2020 22:22:14 +0200 (CEST) Received: from mga07.intel.com (mga07.intel.com [134.134.136.100]) by dpdk.org (Postfix) with ESMTP id 796221D5C6 for ; Thu, 4 Jun 2020 22:22:12 +0200 (CEST) IronPort-SDR: aVzUko0BCcaSXzhKaLxKexN7VwDKwLWX1qnLkVw/S9PKOkC1BSCsn08NZYckSbbS4NPcnQiiDZ 0zeV+H/zfuUA== X-Amp-Result: SKIPPED(no attachment in message) X-Amp-File-Uploaded: False Received: from fmsmga002.fm.intel.com ([10.253.24.26]) by orsmga105.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 04 Jun 2020 13:22:10 -0700 IronPort-SDR: 6g9J+zfbAs12fW72hri+54E7lObfrSyoijFfH/remHy7uiAJoiMJL2ir4QvbnqKr/FViQC1czw 0Bfa0ZRMWhyg== X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.73,472,1583222400"; d="scan'208";a="304830662" Received: from orsmsx110.amr.corp.intel.com ([10.22.240.8]) by fmsmga002.fm.intel.com with ESMTP; 04 Jun 2020 13:22:09 -0700 Received: from orsmsx601.amr.corp.intel.com (10.22.229.14) by ORSMSX110.amr.corp.intel.com (10.22.240.8) with Microsoft SMTP Server (TLS) id 14.3.439.0; Thu, 4 Jun 2020 13:22:09 -0700 Received: from orsmsx605.amr.corp.intel.com (10.22.229.18) by ORSMSX601.amr.corp.intel.com (10.22.229.14) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.1713.5; Thu, 4 Jun 2020 13:22:08 -0700 Received: from ORSEDG002.ED.cps.intel.com (10.7.248.5) by orsmsx605.amr.corp.intel.com (10.22.229.18) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256) id 15.1.1713.5 via Frontend Transport; Thu, 4 Jun 2020 13:22:08 -0700 Received: from NAM11-BN8-obe.outbound.protection.outlook.com (104.47.58.172) by edgegateway.intel.com (134.134.137.101) with Microsoft SMTP Server (TLS) id 14.3.439.0; Thu, 4 Jun 2020 13:22:08 -0700 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=jPQpWrUNWmHzmJrOGO8f2u6B2JiKKZbOSoZ2fzKCOeX/qp7yEPtlKB99zRO9Gkth6AG9tmuwN/9cARtdLo19JlLkOAAIttopok6OTFmSJIkzyA9qOg+iM8Ecd6f4SQhqPiVnpKbR/F14MLBdpUxMxQXtJejzbwTYtJcdWJxKHq5RkStKhr4hQIuPdd4QrjiuvEQl0abDza5lCM0rqldFjTpooZHW73uFqNfd6vMUjzW2dr90O5YqsdaNadTHBzy0mxOHyNZx6tAauPBiyq+muo6kZXNTv5NjSvHReAgnzbjAKXrEMPxSD4s7sC9Pr4cvyr9wEb7MRGXSgZJ9GcpPZA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=x9u6vaA73PGLh7wBazLBPLGdMw8IO85uq6V5J/lvnpA=; b=YUK9d0QfJY47Ml+IMs+zbnYDwmxlMYlkaJmjj2mQSb+M4m2THikhB8my8Qm3SiVZzRTfyjvs1uMwPVGmwRs7iXJFbtqnS/ZerFqIaeaYR0RIwA/RNzQHK4FoKKctlXC/fqsEMlOKaCrp9s4WXfm7I0us/jyp4Ndeb3ylAcvf8Ew2sTobtY/25yW03eLrPhJFiQuZrIM+nJKUqN5ldnOWFlmUvnisVLg7wcAhjMSS26XzxuvUBgT1iakNR7Yr53WfEzGKbxNCEX5AtBk1Eu/JwsjhyvHITfoOFesoNVMcC0keyU9VDxx7Jw4bpTQ4UxvBVukpqGXgqlKxstXGICXLsA== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=intel.com; dmarc=pass action=none header.from=intel.com; dkim=pass header.d=intel.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=intel.onmicrosoft.com; s=selector2-intel-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=x9u6vaA73PGLh7wBazLBPLGdMw8IO85uq6V5J/lvnpA=; b=FnPXP/VJlwE1PuZ4Emf30jsUcakByVXaW5yMSeaYSurIzbKza2fuVP+DTrTggJNCML4ZaALm6SCgBIUuJuzl3Jkzm3ZqTwSVj3V6ymHI+Luvxex1WsSrDVbsaokrSIQkBYjWZldcut9AMAkd68SJHB4D55pwPdRjdgQvqzl/SMA= Received: from BYAPR11MB3494.namprd11.prod.outlook.com (2603:10b6:a03:86::15) by BYAPR11MB2599.namprd11.prod.outlook.com (2603:10b6:a02:c6::20) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.3066.18; Thu, 4 Jun 2020 20:22:06 +0000 Received: from BYAPR11MB3494.namprd11.prod.outlook.com ([fe80::7d9c:1c6d:8919:c0ad]) by BYAPR11MB3494.namprd11.prod.outlook.com ([fe80::7d9c:1c6d:8919:c0ad%5]) with mapi id 15.20.3066.018; Thu, 4 Jun 2020 20:22:06 +0000 From: "Wang, Yipeng1" To: Honnappa Nagarahalli , Stephen Hemminger CC: "Gobriel, Sameh" , "Richardson, Bruce" , "dev@dpdk.org" , "De Lara Guarch, Pablo" , nd , nd Thread-Topic: [PATCH] hash: document breakage with multi-writer thread Thread-Index: AQHWOpQb4KRZqCaNA06+IpNpxNMQqKjIvHWAgAAB1YCAAAyqAIAABUFAgAAI6oCAAAMEQA== Date: Thu, 4 Jun 2020 20:22:06 +0000 Message-ID: References: <20200604171731.6738-1-stephen@networkplumber.org> <20200604105817.1a3a2749@hermes.lan> In-Reply-To: Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: dlp-reaction: no-action dlp-version: 11.2.0.6 dlp-product: dlpe-windows authentication-results: arm.com; dkim=none (message not signed) header.d=none;arm.com; dmarc=none action=none header.from=intel.com; x-originating-ip: [192.55.52.220] x-ms-publictraffictype: Email x-ms-office365-filtering-correlation-id: ae9279df-8989-4f7b-fc58-08d808c4f38e x-ms-traffictypediagnostic: BYAPR11MB2599: x-ld-processed: 46c98d88-e344-4ed4-8496-4ed7712e255d,ExtAddr x-ms-exchange-transport-forked: True x-microsoft-antispam-prvs: x-ms-oob-tlc-oobclassifiers: OLM:10000; x-forefront-prvs: 04244E0DC5 x-ms-exchange-senderadcheck: 1 x-microsoft-antispam: BCL:0; x-microsoft-antispam-message-info: bp7Zm4Ua28nMjbmC9V6dfJjCQ4vOsnVNyBhRLbVGD+A+VYJ4kvmEdKj8GT+9n4gh4g2nJlHdXvK0ksMwdYwFGETQxSk+IXVhjZA0WD2Jr9Zw28fNlBv4UQBTu05/YgRX2jVirdCUEQIy3GmFahA4C4bt93D6e+bO9Kj64JSSnvGkvgtaJRGkkRgUJML/jt5qJUQgpNqQatQYVC+k+Nc0uhv1vyPFIhpX5wkU9zOTULlK4yCpb36MTlh+bn+0cNVAa35XAiNZeqygIF5lMzbmug4n1Ae8NiW+z961WYY52tuIe9op/IGuFjxN+6O2RFYE x-forefront-antispam-report: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:BYAPR11MB3494.namprd11.prod.outlook.com; PTR:; CAT:NONE; SFTY:; SFS:(136003)(396003)(376002)(346002)(366004)(39860400002)(52536014)(110136005)(5660300002)(7696005)(86362001)(316002)(6506007)(53546011)(26005)(54906003)(8676002)(71200400001)(8936002)(64756008)(76116006)(66476007)(66446008)(186003)(55016002)(2906002)(66946007)(66556008)(478600001)(33656002)(83380400001)(4326008)(9686003); DIR:OUT; SFP:1102; x-ms-exchange-antispam-messagedata: 5FMj7pdS07AChWnQ4mRskqdqFoii0WbLp/Fru+s8rRxp5DP9eeVNLUZJ59YRRZ5v/ZupimgmiJUjGXo3KdLzQHZWhr9I/8BHPbfaMuaGR+We3v/lax2tyx2Qz4a5fQcCrytULMz+UAyfxrOycf7aZr3AC1NOSYUsPHj1MDmpz/pk6zyv8GansdKkSTysLOT1VaDx8McDWfdA1VnVvzmcle9E7ZJavVWkC+2eecXehwywlzxqIjAwmz3mdOxwzJinZycfuZs/L8sIn5C9A5h5yQBSztjabatd3xNLB7Tq5WQ8wX9ZmfVw8tYZLCS53uENL5KN9C42PrSTHOr0M8IkC9tvb0QeXzcwDPdyH5WhQiQvrOpUd+T0DE8C4pt0RDSdGVWxmJ0OIUtX8PhBn8mDefH+HcDUlD6f5kt+wCsLVEVOhtdq338iUwsWRQm/g2++4yw+VkWGE0p6pqPGMCX+ubtepBcZOO0fTrhRs9wCT2yOtl3ctv9/cOQ44XkcuqU3 Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 X-MS-Exchange-CrossTenant-Network-Message-Id: ae9279df-8989-4f7b-fc58-08d808c4f38e X-MS-Exchange-CrossTenant-originalarrivaltime: 04 Jun 2020 20:22:06.4672 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Hosted X-MS-Exchange-CrossTenant-id: 46c98d88-e344-4ed4-8496-4ed7712e255d X-MS-Exchange-CrossTenant-mailboxtype: HOSTED X-MS-Exchange-CrossTenant-userprincipalname: No7wL7DH07EeIp4Eul7AgTHQbDy5TadpPaDhI9YLXFo9B4yiH/hVYRHwfGY8AACMjJ0fLwEobNHAE4Cdffop0Q== X-MS-Exchange-Transport-CrossTenantHeadersStamped: BYAPR11MB2599 X-OriginatorOrg: intel.com Subject: Re: [dpdk-dev] [PATCH] hash: document breakage with multi-writer thread X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" > -----Original Message----- > From: Honnappa Nagarahalli > Sent: Thursday, June 4, 2020 12:34 PM > To: Wang, Yipeng1 ; Stephen Hemminger > > Cc: Gobriel, Sameh ; Richardson, Bruce > ; dev@dpdk.org; De Lara Guarch, Pablo > ; nd ; Honnappa > Nagarahalli ; nd > Subject: RE: [PATCH] hash: document breakage with multi-writer thread >=20 > >=20 > > > > > > > > > > > Subject: [PATCH] hash: document breakage with multi-writer > > > > > > thread > > > > > > > > > > > > The code in rte_cuckoo_hash multi-writer support is broken if > > > > > > write operations are called from a non-EAL thread. > > > > > > > > > > > > rte_lcore_id() wil return LCORE_ID_ANY (UINT32_MAX) for non > > > > > > EAL thread and that leads to using wrong local cache. > > > > > > > > > > > > Add error checks and document the restriction. > > > > > Having multiple non-EAL writer threads is a valid use case. > > > > > Should we fix the > > > > issue instead? > > > > > > > > Discovered this the hard way... > > > > > > > > Fixing is non-trivial. Basically, the local cache has to be take > > > > out and that leads to having to do real locking or atomic operation= s. > > > Looking at rte_hash_create function: > > > > > > if (params->extra_flag & > > > RTE_HASH_EXTRA_FLAGS_MULTI_WRITER_ADD) { > > > use_local_cache =3D 1; > > > writer_takes_lock =3D 1; > > > } > > > > > > The writer locks are in place already. The code to handle the case > > > when local cache is taken out is also there. > > > What we need is another input flag that says 'multi writer + non-eal > > threads' > > > which would set 'use_local_cache =3D 0' and 'writer_takes_lock =3D 1'= . > > > Not sure, it would be valuable addition. But looks like this is what > > > you were expecting when you had enabled > > > 'RTE_HASH_EXTRA_FLAGS_MULTI_WRITER_ADD'. Many other APIs in > DPDK > > do > > > not provide this kind of MT safety. > > > > [Wang, Yipeng] > > If possible, we can try to not add new flags, because there are > > already a lot of flag options. > > How about in the code, we check if the writer is a non-eal or not by > > checking the rte_lcore_id, and operate on the global queue? > > Could this work? > > If(h->use_local_cache) { > > lcore_id =3D rte_lcore_id(); > > if(lcore_id =3D=3D LCORE_ID_ANY) { // this is non-eal threads > > > > } > > Else { > > > > } > > } > The other thing I wanted to do was saving on the memory allocated for the > local cache when the writers are non-eal threads. Without knowing the kin= d > of threads upfront, we might have to create the local cache when a writer > adds the entry first time. I got what you mean. If people only use non-eal threads, we could save the= space of local cache completely.=20 Creating local cache during the first write is one solution. But the curren= t rte_hash always allocate things during table creation time. This provides guarantee that the program won't fail in= the middle due to memory allocation issue. Meanwhile I would rather be wasting some space than adding another option f= lag related to multi-threading. In my opinion, all those flags are already confusing enough. It would also = be harder to maintain in future. =20