From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from dpdk.org (dpdk.org [92.243.14.124]) by inbox.dpdk.org (Postfix) with ESMTP id DC51DA00C4; Thu, 4 Jun 2020 21:10:48 +0200 (CEST) Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id CBFA41D5C7; Thu, 4 Jun 2020 21:10:47 +0200 (CEST) Received: from mga07.intel.com (mga07.intel.com [134.134.136.100]) by dpdk.org (Postfix) with ESMTP id 0FAD31D5AA for ; Thu, 4 Jun 2020 21:10:45 +0200 (CEST) IronPort-SDR: 3nbxcgrzpGczKoOnJcPLkKizivpu5ITQGlvDwQbQEZSbIcThQf0d4HcyM9OrPY6K34hvBJB1SX Mcb9dUZpbizA== X-Amp-Result: SKIPPED(no attachment in message) X-Amp-File-Uploaded: False Received: from orsmga004.jf.intel.com ([10.7.209.38]) by orsmga105.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 04 Jun 2020 12:10:44 -0700 IronPort-SDR: AWpGoNQNWWA5CP7MOAejqT31NvpRbiYDgWy/Yl/Zkfqkc7y7MTaK4LnJJQ5NDhclBaKwXT0+Kk rYDeD40asqBQ== X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.73,472,1583222400"; d="scan'208";a="417013683" Received: from fmsmsx103.amr.corp.intel.com ([10.18.124.201]) by orsmga004.jf.intel.com with ESMTP; 04 Jun 2020 12:10:44 -0700 Received: from fmsmsx117.amr.corp.intel.com (10.18.116.17) by FMSMSX103.amr.corp.intel.com (10.18.124.201) with Microsoft SMTP Server (TLS) id 14.3.439.0; Thu, 4 Jun 2020 12:10:44 -0700 Received: from FMSEDG001.ED.cps.intel.com (10.1.192.133) by fmsmsx117.amr.corp.intel.com (10.18.116.17) with Microsoft SMTP Server (TLS) id 14.3.439.0; Thu, 4 Jun 2020 12:10:44 -0700 Received: from NAM11-DM6-obe.outbound.protection.outlook.com (104.47.57.173) by edgegateway.intel.com (192.55.55.68) with Microsoft SMTP Server (TLS) id 14.3.439.0; Thu, 4 Jun 2020 12:10:43 -0700 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=SE0CHn8FYX89Y5q7OdR1ewLYBnzZt4CZaTasoxkeW8HRIu2XlW/i8uBV0SG18mPd+HYW7Je/BEo5uj+HVuuNb8OR2qLHCe1IqE3PiLyQLsBWZxrTkyip+fHtfJDpNmQ5Z0DnSa2u0nIzOCfucrkbimhTNV07xip4J9qcnAdGny9B1nFed9CeYcnq/eVqjDkYfunb56pAeMyuVfCMpD36HcMSB+jiIyP74uYhhdSHfJljlPyfLKS6Vx369iv8N7Ni6AYjlpfLsrK6YtKZrulhX5FTJzjx20CQykVmg88XeiGIhBfJdCDGMEOvvZAOViawFFoWngqERHpRUN5VTTuzkg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=aj9z6XcOvjr1phZ9GEXmzpGGYXAixwJVb4ojFb5S560=; b=mDsGSaOoYrA5oRushfEKxi+DFkFQZ1BoyQUVgA3M4cwddEICBb/toqo/KXr7eH32YM6wynKMBkK7T/iothQao2xRvW6kQIURcfu5e3IY8Q/BBkDm0TMMXZtIFukdFytMqznJRCVn797yF6DVwoKveWDszub1j2+z35adNkyt6nlN7MNZFIPyKtyBkThu6MFwPDKQn/TNjUZMFnRyTiB4+O8kH30B6I6R7L9zgY3GKsPJDTehexDLLaKDl0eruZXRr5kJiGtRWM2Y7j3vBF1W5YgrWHbdRQgMWOCoDRsxP2foM8yKCUNsSbQNEPpxOyBaNjAaUxg4UxBU4fyHiXlt/A== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=intel.com; dmarc=pass action=none header.from=intel.com; dkim=pass header.d=intel.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=intel.onmicrosoft.com; s=selector2-intel-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=aj9z6XcOvjr1phZ9GEXmzpGGYXAixwJVb4ojFb5S560=; b=Y+LG5x+rBJJ9S+3b7wOIeSZrffNlm532WXJzdBeaQJJQhLSm+kGyal5F1StLmf1Za+LUgaTge+WGQhZQQFM2hpzQGYrJs7aT6UwBSYB4Abr78wAGIQe9OkZhFwjypdfUqghWX43Ve49WVpm36TjXCZbDB1gfuClYf7r/VXocD8o= Received: from BYAPR11MB3494.namprd11.prod.outlook.com (2603:10b6:a03:86::15) by BYAPR11MB3254.namprd11.prod.outlook.com (2603:10b6:a03:7c::19) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.3045.17; Thu, 4 Jun 2020 19:10:38 +0000 Received: from BYAPR11MB3494.namprd11.prod.outlook.com ([fe80::7d9c:1c6d:8919:c0ad]) by BYAPR11MB3494.namprd11.prod.outlook.com ([fe80::7d9c:1c6d:8919:c0ad%5]) with mapi id 15.20.3066.018; Thu, 4 Jun 2020 19:10:38 +0000 From: "Wang, Yipeng1" To: Honnappa Nagarahalli , Stephen Hemminger CC: "Gobriel, Sameh" , "Richardson, Bruce" , "dev@dpdk.org" , "De Lara Guarch, Pablo" , nd , nd Thread-Topic: [PATCH] hash: document breakage with multi-writer thread Thread-Index: AQHWOpQb4KRZqCaNA06+IpNpxNMQqKjIvHWAgAAB1YCAAAyqAIAABUFA Date: Thu, 4 Jun 2020 19:10:38 +0000 Message-ID: References: <20200604171731.6738-1-stephen@networkplumber.org> <20200604105817.1a3a2749@hermes.lan> In-Reply-To: Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: dlp-reaction: no-action dlp-version: 11.2.0.6 dlp-product: dlpe-windows authentication-results: arm.com; dkim=none (message not signed) header.d=none;arm.com; dmarc=none action=none header.from=intel.com; x-originating-ip: [192.55.52.220] x-ms-publictraffictype: Email x-ms-office365-filtering-correlation-id: 0041b898-4d00-4500-b89e-08d808baf7d6 x-ms-traffictypediagnostic: BYAPR11MB3254: x-ld-processed: 46c98d88-e344-4ed4-8496-4ed7712e255d,ExtAddr x-ms-exchange-transport-forked: True x-microsoft-antispam-prvs: x-ms-oob-tlc-oobclassifiers: OLM:10000; x-forefront-prvs: 04244E0DC5 x-ms-exchange-senderadcheck: 1 x-microsoft-antispam: BCL:0; x-microsoft-antispam-message-info: /j4f7lml75buw2NAX6wJTWusuC8jzarSiLHyTj4khGrBT90R5BTg+Mv9cf4DHLFmWhmZQKAYiYW1OhqPudRwSaG918A+czv5NyBEVBe8IXY0yuq5q7ndB9V2DjWVGPAs/EfncpzRe1fVPzuA1Nn2nw4mrbfJ1OtnfU2ow16T0WhjYEQZ5roa28KEy4kHmJfTsW5SJgZYAwNIo/LIz7dL50l8piwqp1ov0y6r/NEa44YCSt0lm8M9WiTbUJPBoAl6bbDahsUA7FWPTtNJ2I7mN4sDVOdIkC3OlaTyU+5Q1gJ92VuZV/d22Q69hJuVrev7 x-forefront-antispam-report: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:BYAPR11MB3494.namprd11.prod.outlook.com; PTR:; CAT:NONE; SFTY:; SFS:(366004)(136003)(396003)(376002)(39860400002)(346002)(33656002)(478600001)(83380400001)(316002)(7696005)(6506007)(2906002)(5660300002)(26005)(186003)(71200400001)(110136005)(54906003)(64756008)(66946007)(76116006)(66556008)(4326008)(55016002)(66446008)(66476007)(8936002)(52536014)(8676002)(9686003)(86362001); DIR:OUT; SFP:1102; x-ms-exchange-antispam-messagedata: XnVLJ2UQSTLDFg16Jo5Jv6LUNKSc8cKq1ixvo2sK3B9XVgRxutP3K/0AQ5WlPBnkAQHVeBG+XnA7WC/8AR17rFgEYJspvPud0doWWA7KZB4FPpvGflEMwIgTavQ9C/JBuIshrzjXtAVqeZvG0cK9edw1cpAV9iDVW1sOMr/0m//kiaAFoRyuKJZPhEKyuZfegniSpCN92gKRAbaaxotKTT7tMtlbQJYTSk7yY3L7PqRBMU7PFF+Ydb/iZeb7i+gnILhZQ2Jor9vEl8hZd1/HWSWs4q35+dQTHV11ls2+3KC4MGsGEtyopACl3zLl51KcbQkZRl0JMCR5/ao13b7lmI6OX7Gy20xFFXnXPYWtgy3drvteb0nSJjBPLA+9Pze/k/h+sfiYoO8+I+V2K7q/FjfWnYFfmOoV/gNhMwcUdrL36yD5O4E5ZstpnCtXpqwl//hJEVwSGYidmj+n864P2JTdSCT+uuDNl3sMNk2/c+A= Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 X-MS-Exchange-CrossTenant-Network-Message-Id: 0041b898-4d00-4500-b89e-08d808baf7d6 X-MS-Exchange-CrossTenant-originalarrivaltime: 04 Jun 2020 19:10:38.6559 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Hosted X-MS-Exchange-CrossTenant-id: 46c98d88-e344-4ed4-8496-4ed7712e255d X-MS-Exchange-CrossTenant-mailboxtype: HOSTED X-MS-Exchange-CrossTenant-userprincipalname: PY3/XLa8HRSA7cM+F4yH9jDa1NUT+q6bSRkp5O5/DifrDRXy76/dYzEln3i2T9QPlQ19ZR4gIEjFca2m5BAaOg== X-MS-Exchange-Transport-CrossTenantHeadersStamped: BYAPR11MB3254 X-OriginatorOrg: intel.com Subject: Re: [dpdk-dev] [PATCH] hash: document breakage with multi-writer thread X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" > > > > > > > > > > Subject: [PATCH] hash: document breakage with multi-writer thread > > > > > > > > The code in rte_cuckoo_hash multi-writer support is broken if > > > > write operations are called from a non-EAL thread. > > > > > > > > rte_lcore_id() wil return LCORE_ID_ANY (UINT32_MAX) for non EAL > > > > thread and that leads to using wrong local cache. > > > > > > > > Add error checks and document the restriction. > > > Having multiple non-EAL writer threads is a valid use case. Should > > > we fix the > > issue instead? > > > > Discovered this the hard way... > > > > Fixing is non-trivial. Basically, the local cache has to be take out > > and that leads to having to do real locking or atomic operations. > Looking at rte_hash_create function: >=20 > if (params->extra_flag & > RTE_HASH_EXTRA_FLAGS_MULTI_WRITER_ADD) { > use_local_cache =3D 1; > writer_takes_lock =3D 1; > } >=20 > The writer locks are in place already. The code to handle the case when l= ocal > cache is taken out is also there. > What we need is another input flag that says 'multi writer + non-eal thre= ads' > which would set 'use_local_cache =3D 0' and 'writer_takes_lock =3D 1'. > Not sure, it would be valuable addition. But looks like this is what you = were > expecting when you had enabled > 'RTE_HASH_EXTRA_FLAGS_MULTI_WRITER_ADD'. Many other APIs in DPDK > do not provide this kind of MT safety. [Wang, Yipeng] If possible, we can try to not add new flags, because there are already a l= ot of flag options. How about in the code, we check if the writer is a non-eal or not by checki= ng the rte_lcore_id, and operate on the global queue? Could this work? If(h->use_local_cache) { lcore_id =3D rte_lcore_id(); if(lcore_id =3D=3D LCORE_ID_ANY) { // this is non-eal threads } Else { } }