From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from dpdk.org (dpdk.org [92.243.14.124]) by inbox.dpdk.org (Postfix) with ESMTP id B626DA00C4; Thu, 4 Jun 2020 23:33:06 +0200 (CEST) Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id BCB2A1D5F4; Thu, 4 Jun 2020 23:33:05 +0200 (CEST) Received: from mga04.intel.com (mga04.intel.com [192.55.52.120]) by dpdk.org (Postfix) with ESMTP id 44D6A1D5E6 for ; Thu, 4 Jun 2020 23:33:03 +0200 (CEST) IronPort-SDR: pQVheV2oDrVWd5Aqkk1ZB/QAcTDbAp7h9fMnHJledIvKW/x6Mvx+uxkfl1HGSHqjZ+Poixu87m 03wn8wKd7fMw== X-Amp-Result: SKIPPED(no attachment in message) X-Amp-File-Uploaded: False Received: from orsmga002.jf.intel.com ([10.7.209.21]) by fmsmga104.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 04 Jun 2020 14:33:02 -0700 IronPort-SDR: RZZb8zIRjyPOyYurwSvuFYcdB3FXd33+kJIMF8HIODceQPzLJmmsIluusWRgudbD1jCimYpIoj jcT/IOVI7DgQ== X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.73,472,1583222400"; d="scan'208";a="287517911" Received: from fmsmsx108.amr.corp.intel.com ([10.18.124.206]) by orsmga002.jf.intel.com with ESMTP; 04 Jun 2020 14:33:02 -0700 Received: from fmsmsx101.amr.corp.intel.com (10.18.124.199) by FMSMSX108.amr.corp.intel.com (10.18.124.206) with Microsoft SMTP Server (TLS) id 14.3.439.0; Thu, 4 Jun 2020 14:33:01 -0700 Received: from FMSEDG001.ED.cps.intel.com (10.1.192.133) by fmsmsx101.amr.corp.intel.com (10.18.124.199) with Microsoft SMTP Server (TLS) id 14.3.439.0; Thu, 4 Jun 2020 14:33:01 -0700 Received: from NAM10-DM6-obe.outbound.protection.outlook.com (104.47.58.105) by edgegateway.intel.com (192.55.55.68) with Microsoft SMTP Server (TLS) id 14.3.439.0; Thu, 4 Jun 2020 14:33:00 -0700 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=SRcx6iW7Mn0+iSO8OAfzk8XxJjofxfT/cmB70P+uQBorzv84CYuqa1ZgLNlfVNpJ/ABe3dNxV7E7MtV1hcYrxNGO+fRE8J3HikNdeYKKRG4Uvh4hd6/K+WD2m6l7SEuWZguRvACWuhIQvmLml/nzZxFnJxqgjrAJy5nNaa1aMHAx05gw1PdCeL9SBcUQOSyZGOEKrOhdhCboPzhyu11D4b9tX2Mbo13/GJtVgc9BdYz8Oqh53rUPLcG95iEg0/gsWkSozZRTueGID6P6tqpShq6rAIEgBlENp+8vFlx1u07Tzqc2dxuSEGKi7Rg93qqWQ51glKqLnsMXjvNkx/UBoQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=5Bh4jMth2jXd3KwCsYZHtFQ3s8RKBLk9CdpRQYhjMPc=; b=I3f2hYa3JYLRgVPhmsKOZF0p/Q/VjwkIXGnhrAAt8TAEE2zJdZy5inPOp2DhtpDain+AbthbpPtw054GGiHIMXzoYfunhXpUXt3JXFG7hLivQWnVKe9KWF6qSGBvatWsE76+vBEf9ZWv558wSu9MZLeOJyuzk8UBuCb+T6boQ4pHG/XLc1N5BHoCvUzeAyo51VM8UrhwtCzkgkyAURWGgLSIZ/rbvEzA/oGe3aZSTxoXgYVqyIbHUVihziNAlnMRZMsbuY91QJL2yJIaaRXzKElcqqPac+DwDgwvV2ffwS5l5AWTQgmaMX7wgk7ACbEpupehU7vxyjmF3XR627xVcg== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=intel.com; dmarc=pass action=none header.from=intel.com; dkim=pass header.d=intel.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=intel.onmicrosoft.com; s=selector2-intel-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=5Bh4jMth2jXd3KwCsYZHtFQ3s8RKBLk9CdpRQYhjMPc=; b=gSP6R0LUAjDlrJfMWBB3wSQP9E+juSs2a74FIVHdjAwnR3o5zZPf3wigKVxDCCx2wCnPdj6b7oDBrPmZmb12xF1LDytqUtwM9vjhgN0UZ2WymOBCCk2Q3MaqGz7HnAWaSHFz9H7fjlPSFkUT2Q2ghNYdZiJq7C3ID4a26cuZ1Uk= Received: from BYAPR11MB3494.namprd11.prod.outlook.com (2603:10b6:a03:86::15) by BYAPR11MB2790.namprd11.prod.outlook.com (2603:10b6:a02:c4::29) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.3066.20; Thu, 4 Jun 2020 21:32:58 +0000 Received: from BYAPR11MB3494.namprd11.prod.outlook.com ([fe80::7d9c:1c6d:8919:c0ad]) by BYAPR11MB3494.namprd11.prod.outlook.com ([fe80::7d9c:1c6d:8919:c0ad%5]) with mapi id 15.20.3066.018; Thu, 4 Jun 2020 21:32:58 +0000 From: "Wang, Yipeng1" To: Stephen Hemminger , "Gobriel, Sameh" , "Richardson, Bruce" CC: "dev@dpdk.org" , "honnappa.nagarahalli@arm.com" , "De Lara Guarch, Pablo" Thread-Topic: [PATCH] hash: document breakage with multi-writer thread Thread-Index: AQHWOpQb4KRZqCaNA06+IpNpxNMQqKjI+QyA Date: Thu, 4 Jun 2020 21:32:58 +0000 Message-ID: References: <20200604171731.6738-1-stephen@networkplumber.org> In-Reply-To: <20200604171731.6738-1-stephen@networkplumber.org> Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: dlp-reaction: no-action dlp-version: 11.2.0.6 dlp-product: dlpe-windows authentication-results: networkplumber.org; dkim=none (message not signed) header.d=none;networkplumber.org; dmarc=none action=none header.from=intel.com; x-originating-ip: [192.55.52.220] x-ms-publictraffictype: Email x-ms-office365-filtering-correlation-id: 5a960081-4f84-4583-9455-08d808ceda1e x-ms-traffictypediagnostic: BYAPR11MB2790: x-ld-processed: 46c98d88-e344-4ed4-8496-4ed7712e255d,ExtAddr x-ms-exchange-transport-forked: True x-microsoft-antispam-prvs: x-ms-oob-tlc-oobclassifiers: OLM:2958; x-forefront-prvs: 04244E0DC5 x-ms-exchange-senderadcheck: 1 x-microsoft-antispam: BCL:0; x-microsoft-antispam-message-info: Gxam7znngb5vs/Ewi9KqF/S1Fl9M+Ll6E3Q0+5Lx+MJPKrWwofJvB+U8vCQvJ3zrrPbLjot2xpbCUxaNSL5d4LIQeK1FUyu8YmhgZb7zsz0OSf/l5HFScvnCLgM9/YmtP1Pulj6wJrgc1S26kPeA5w+1fuRfX1FMAY17V2LHzTlDvH2jUCZJiLVtkXTDxDKJb9Pa+2BBAVTAmYkWPm9FUyUOOA5wPu5+0fL1an/MoJE5hHP/rE/QbnBr7YBG+mzWOCx3oHjzozaSi4GUALA5AVfXIuaGZxqj7RoTV0nYJPxzxJ5R+dSMRFftcRylpVTijR6wbw65nrClqKnYuGGS/Q== x-forefront-antispam-report: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:BYAPR11MB3494.namprd11.prod.outlook.com; PTR:; CAT:NONE; SFTY:; SFS:(376002)(346002)(39860400002)(136003)(366004)(396003)(107886003)(53546011)(2906002)(186003)(4326008)(478600001)(110136005)(54906003)(26005)(33656002)(52536014)(83380400001)(6506007)(55016002)(5660300002)(66476007)(8936002)(66446008)(66556008)(71200400001)(66946007)(9686003)(76116006)(6636002)(86362001)(316002)(7696005)(64756008)(8676002); DIR:OUT; SFP:1102; x-ms-exchange-antispam-messagedata: Mtl5AIj7hby5oK3yWxgowY6aeEqk0EIAWIQ2ps8sdt4WChR9IoZIHd3irMnM6NYG49SgOuah3cL9sdHsnl0S0NHbMupdFs3+frCiVc1E8R+AnzDWQJR9xrGotvwe6CXPOaxSd7AcG5jnqD8ssoq+/WxLRsZsZaUrnHVN7y8etU8suoXcSKWKLbnT8VD2ftTRFRBCvrfVA3PPKJxGfm03Xz31Y0Cz2nHn4CNzF/QyevI0YKuVRP9ODbVmnYMm8bAeOJjdJiWFQ+c4OBTJsFUnmJ0Zr/sqsxEe31DG6mOmuQ9NGoEPSH4NdWQ3N4gQtqxeGfW98BWjf+JwgAfBZ/GLclMd+DztPmoJIVPL9D289b2E79fzXhULiUHa3HJcLT+J5Z7bKwkgcvLxz1W4KQVfsrgK219Mo+PAKMIR8bA6GcUqBxuDwk0lt3b33rAJCKe3MyC5tli40YED2tOvDsR9VWIKr83L3FSdJWeyn6EPOHcQgAMH/Faw2+JqOIoBtcNo Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 X-MS-Exchange-CrossTenant-Network-Message-Id: 5a960081-4f84-4583-9455-08d808ceda1e X-MS-Exchange-CrossTenant-originalarrivaltime: 04 Jun 2020 21:32:58.7328 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Hosted X-MS-Exchange-CrossTenant-id: 46c98d88-e344-4ed4-8496-4ed7712e255d X-MS-Exchange-CrossTenant-mailboxtype: HOSTED X-MS-Exchange-CrossTenant-userprincipalname: DOqU4AHx6P4CmYdQVa98A8T157H9VZbGU/NTNtDawO5PwVM+T13kSibiqG2A5/HgjSBUj/j2Na31zYk+sx9QPg== X-MS-Exchange-Transport-CrossTenantHeadersStamped: BYAPR11MB2790 X-OriginatorOrg: intel.com Subject: Re: [dpdk-dev] [PATCH] hash: document breakage with multi-writer thread X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" > -----Original Message----- > From: Stephen Hemminger > Sent: Thursday, June 4, 2020 10:18 AM > To: Wang, Yipeng1 ; Gobriel, Sameh > ; Richardson, Bruce > > Cc: dev@dpdk.org; Stephen Hemminger ; > honnappa.nagarahalli@arm.com; De Lara Guarch, Pablo > > Subject: [PATCH] hash: document breakage with multi-writer thread >=20 > The code in rte_cuckoo_hash multi-writer support is broken if write > operations are called from a non-EAL thread. >=20 > rte_lcore_id() wil return LCORE_ID_ANY (UINT32_MAX) for non EAL thread > and that leads to using wrong local cache. >=20 > Add error checks and document the restriction. >=20 > Fixes: 9d033dac7d7c ("hash: support no free on delete") > Fixes: 5915699153d7 ("hash: fix scaling by reducing contention") > Signed-off-by: Stephen Hemminger > Cc: honnappa.nagarahalli@arm.com > Cc: pablo.de.lara.guarch@intel.com > --- > doc/guides/prog_guide/hash_lib.rst | 1 + > lib/librte_hash/rte_cuckoo_hash.c | 9 +++++++++ > 2 files changed, 10 insertions(+) >=20 > diff --git a/doc/guides/prog_guide/hash_lib.rst > b/doc/guides/prog_guide/hash_lib.rst > index d06c7de2ead1..29b41a425a43 100644 > --- a/doc/guides/prog_guide/hash_lib.rst > +++ b/doc/guides/prog_guide/hash_lib.rst > @@ -85,6 +85,7 @@ For concurrent writes, and concurrent reads and writes > the following flag values >=20 > * If the multi-writer flag (RTE_HASH_EXTRA_FLAGS_MULTI_WRITER_ADD) > is set, multiple threads writing to the table is allowed. > Key add, delete, and table reset are protected from other writer thre= ads. > With only this flag set, readers are not protected from ongoing writes. > + The writer threads must be EAL threads, it is not safe to write to a = multi- > writer hash table from an interrupt, control or other threads. >=20 > * If the read/write concurrency > (RTE_HASH_EXTRA_FLAGS_RW_CONCURRENCY) is set, multithread > read/write operation is safe > (i.e., application does not need to stop the readers from accessing t= he hash > table until writers finish their updates. Readers and writers can operate= on > the table concurrently). > diff --git a/lib/librte_hash/rte_cuckoo_hash.c > b/lib/librte_hash/rte_cuckoo_hash.c > index 90cb99b0eef8..79c94107a582 100644 > --- a/lib/librte_hash/rte_cuckoo_hash.c > +++ b/lib/librte_hash/rte_cuckoo_hash.c > @@ -979,6 +979,9 @@ __rte_hash_add_key_with_hash(const struct > rte_hash *h, const void *key, > /* Did not find a match, so get a new slot for storing the new key */ > if (h->use_local_cache) { > lcore_id =3D rte_lcore_id(); > + if (lcore_id =3D=3D LCORE_ID_ANY) > + return -EINVAL; > + > cached_free_slots =3D &h->local_free_slots[lcore_id]; > /* Try to get a free slot from the local cache */ > if (cached_free_slots->len =3D=3D 0) { > @@ -1382,6 +1385,10 @@ remove_entry(const struct rte_hash *h, struct > rte_hash_bucket *bkt, unsigned i) >=20 > if (h->use_local_cache) { > lcore_id =3D rte_lcore_id(); > + ERR_IF_TRUE((lcore_id =3D=3D LCORE_ID_ANY), > + "%s: attempt to remove entry from non EAL > thread\n", > + __func__); > + > cached_free_slots =3D &h->local_free_slots[lcore_id]; > /* Cache full, need to free it. */ > if (cached_free_slots->len =3D=3D LCORE_CACHE_SIZE) { @@ - > 1637,6 +1644,8 @@ rte_hash_free_key_with_position(const struct rte_hash > *h, >=20 > if (h->use_local_cache) { > lcore_id =3D rte_lcore_id(); > + RETURN_IF_TRUE((lcore_id =3D=3D LCORE_ID_ANY), -EINVAL); > + > cached_free_slots =3D &h->local_free_slots[lcore_id]; > /* Cache full, need to free it. */ > if (cached_free_slots->len =3D=3D LCORE_CACHE_SIZE) { > -- > 2.26.2 [Wang, Yipeng]=20 Since no conclusion on a better fix yet, I acked this fix. Acked-by: Yipeng Wang