From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from EUR01-HE1-obe.outbound.protection.outlook.com (mail-he1eur01on0062.outbound.protection.outlook.com [104.47.0.62]) by dpdk.org (Postfix) with ESMTP id 7F32C49E0 for ; Fri, 9 Nov 2018 16:37:51 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=armh.onmicrosoft.com; s=selector1-arm-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=H5uGFyNbqUM2EtGR+TLPoVGOm2+FpGUCCJZR+AqsKcM=; b=Est3rqrVn7rqlAM2MNthcMFAk7LrXX/+RFoG4OK9Z6GVEgriN8BvB7aaUn5cKywSev7IfFmA7I+D+fH9KKiLBd3BoCu8OtOl3glwvsxoPQ87XxS+9Ea8zJ9Ane/gTHJ0apa3glWgmvjNkC1V5yy0AqqZwvWdGsrPjSW9/HkUDf8= Received: from AM6PR08MB3672.eurprd08.prod.outlook.com (20.177.115.29) by AM6PR08MB3685.eurprd08.prod.outlook.com (20.177.199.158) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.1294.20; Fri, 9 Nov 2018 15:37:48 +0000 Received: from AM6PR08MB3672.eurprd08.prod.outlook.com ([fe80::c1a0:51bf:cd33:2b27]) by AM6PR08MB3672.eurprd08.prod.outlook.com ([fe80::c1a0:51bf:cd33:2b27%6]) with mapi id 15.20.1294.034; Fri, 9 Nov 2018 15:37:48 +0000 From: Honnappa Nagarahalli To: Jerin Jacob CC: "bruce.richardson@intel.com" , "pablo.de.lara.guarch@intel.com" , "dev@dpdk.org" , "yipeng1.wang@intel.com" , Dharmik Thakkar , "Gavin Hu (Arm Technology China)" , nd , "thomas@monjalon.net" , "ferruh.yigit@intel.com" , "hemant.agrawal@nxp.com" , "chaozhu@linux.vnet.ibm.com" , "Kapoor, Prasun" , nd Thread-Topic: [dpdk-dev] [PATCH v7 4/5] hash: add lock-free read-write concurrency Thread-Index: AQHUbO4hC1hG0V5Pe0K6GLHhoSzYDKU9/ZaAgAA/xYCAA9sPQIAAbsAAgAQrzkCAAJBbgIAAZLtw Date: Fri, 9 Nov 2018 15:37:48 +0000 Message-ID: References: <1540532253-112591-1-git-send-email-honnappa.nagarahalli@arm.com> <1540532253-112591-5-git-send-email-honnappa.nagarahalli@arm.com> <20181103115240.GA3608@jerin> <20181103154039.GA25488@jerin> <20181106090953.GA5671@jerin> <20181109092821.GB4934@jerin> In-Reply-To: <20181109092821.GB4934@jerin> Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: authentication-results: spf=none (sender IP is ) smtp.mailfrom=Honnappa.Nagarahalli@arm.com; x-originating-ip: [217.140.111.135] x-ms-publictraffictype: Email x-microsoft-exchange-diagnostics: 1; AM6PR08MB3685; 6:GegkYvdhkvjMc3roezEXbw/Zj/mDAj58S7M8OOaEqI8MuP2w8VDGPuGmLEA+TMxYkrAAaFrbdf0tmKjmILcyz12iMDUHj78cfoX5aoW0LZlEOpmy58YgZS7cUM8P7oX9gi9t5TvCoifSmZdfrPVWfGh86WCIYN5sN0tRWYyiEKwCa3RDkaBnw2GgL+VqHu277HjEQrDCSLPTgTdtb/J/IUrXyPgPKkCmyW8KTSi3UFdUFnfQzHway+KEpDYi9fTFbzKZEP/ShipaVHotkqmDeT2K0C/3hp/FKskY5rJuiP8ghaCk62Xh0HY3RLhp7l6Y7vhlGLIw43vpGUwJKAMwNHDy8ghhn+Q9hIn7Tq8H/pxaeRP3bbbXeoU+7rBY0LWUuxqEpJRNACPpZ4wp5d9x8NeObur1N5H+okVt0GWf1cUsJg2i+epMKvI252vdJcRsxyFt14tMrmbpFUo1/xOPBw==; 5:wDNQXEFojOa19s53fWwmW0LbXSqQFis4i99AB8IQw6/qAR1ZZsMxfYA8o52jFUJXzSRQph2H+Hp0Btx+jzoiyAWq44tWz9DJ/kISnvG+6kIcjiGF9giC1vLg8RBk0UnzJ1A3bILqG1s33GYJKYQ8H1Z0wzOgPZiaw3hLEY04X/Y=; 7:eAq4ckp94bpV+lpSgrfWaYpKbvb5gfegHK6Yasm1Poe1cZY5N/e7U9ljMtBUgSx0eoSmgQcPiO0c3T5zgUEsTkBRQA4Loil8xGy4Tspr1HB6LmG9STuiVjGQ8URGot/IUEqx1zB/hi+xmbTsWPMe4g== x-ms-exchange-antispam-srfa-diagnostics: SOS;SOR; x-ms-office365-filtering-correlation-id: 7c219fd0-d8da-4763-ead2-08d646594db4 x-ms-office365-filtering-ht: Tenant x-microsoft-antispam: BCL:0; PCL:0; RULEID:(7020095)(4652040)(8989299)(5600074)(711020)(4618075)(4534185)(4627221)(201703031133081)(201702281549075)(8990200)(2017052603328)(7153060)(7193020); SRVR:AM6PR08MB3685; x-ms-traffictypediagnostic: AM6PR08MB3685: nodisclaimer: True x-microsoft-antispam-prvs: x-exchange-antispam-report-test: UriScan:(17755550239193); x-ms-exchange-senderadcheck: 1 x-exchange-antispam-report-cfa-test: BCL:0; PCL:0; RULEID:(8211001083)(6040522)(2401047)(8121501046)(5005006)(93006095)(93001095)(3231382)(944501410)(52105095)(10201501046)(3002001)(6055026)(148016)(149066)(150057)(6041310)(20161123560045)(20161123558120)(20161123562045)(201703131423095)(201702281528075)(20161123555045)(201703061421075)(201703061406153)(20161123564045)(201708071742011)(7699051)(76991095); SRVR:AM6PR08MB3685; BCL:0; PCL:0; RULEID:; SRVR:AM6PR08MB3685; x-forefront-prvs: 08512C5403 x-forefront-antispam-report: SFV:NSPM; SFS:(10009020)(396003)(366004)(39860400002)(376002)(346002)(136003)(43544003)(199004)(189003)(6436002)(81156014)(54906003)(9686003)(6306002)(53936002)(6246003)(446003)(11346002)(486006)(229853002)(476003)(2906002)(55016002)(68736007)(6916009)(7736002)(3846002)(81166006)(6116002)(86362001)(66066001)(8676002)(8936002)(186003)(5660300001)(256004)(102836004)(74316002)(71190400001)(305945005)(14444005)(76176011)(2900100001)(6506007)(106356001)(33656002)(7416002)(71200400001)(4326008)(26005)(105586002)(99286004)(14454004)(93886005)(72206003)(97736004)(966005)(316002)(25786009)(478600001)(7696005); DIR:OUT; SFP:1101; SCL:1; SRVR:AM6PR08MB3685; H:AM6PR08MB3672.eurprd08.prod.outlook.com; FPR:; SPF:None; LANG:en; PTR:InfoNoRecords; A:1; MX:1; received-spf: None (protection.outlook.com: arm.com does not designate permitted sender hosts) x-microsoft-antispam-message-info: 1Lr1O0tZuZHTGWZfj2Gvq3vYOG1G7y2Nc+h+P5aAQ2RhIykxPUUNpU7vIj1PdQKlU3VUrArPKcGhWxJ+RvF3afzAiAaxTbFPQkH846EyIzzR9ND7VoZJAN8Rc6iUYgAl7PduzL/Km3cGdXJCrIi/lMzyHU5AX3dLxZsVwq+oMpRYx20VmW+GHwcPBOQNhCIqpAyFt7AR+KklxwiqyUoZyOYiBwtuPpohrDLGvwwVq5yTRvHlnA5wTmK+7w6gBfXIFHwEAeGuiuZakjSQuEIaS20TiJu1U6xkf9rp9vvlq6Rh0JdB+v0rMx3hY+YKqwpTCQdBrDAVIz/CHy3wna/15fIz6gYCSLYyVMYpr8XsN/I= spamdiagnosticoutput: 1:99 spamdiagnosticmetadata: NSPM Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 X-OriginatorOrg: arm.com X-MS-Exchange-CrossTenant-Network-Message-Id: 7c219fd0-d8da-4763-ead2-08d646594db4 X-MS-Exchange-CrossTenant-originalarrivaltime: 09 Nov 2018 15:37:48.7978 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Hosted X-MS-Exchange-CrossTenant-id: f34e5979-57d9-4aaa-ad4d-b122a662184d X-MS-Exchange-Transport-CrossTenantHeadersStamped: AM6PR08MB3685 Subject: Re: [dpdk-dev] [PATCH v7 4/5] hash: add lock-free read-write concurrency X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 09 Nov 2018 15:37:51 -0000 > > > > Agree. There are multiple micro-architectures in Arm eco-system. > > > > We > > > should establish few simple rules to make sure algorithms perform > > > well on all the available platforms. I established few rules in VPP > > > and they are working fine so far. > > > > > > Can you share that rules for everyone's benefit? > > > > > These are just few simple rules anyone can think of, but avoid the surp= rises. > > We identified a owner for each platform (we have this already in DPDK, > > even across platforms) Each patch submitted for Arm platforms is > > marked with -2 (VPP uses Gerrit) Every platform owner tests on her/his > platform. -2 will be removed only if it does not cause regression on any > platforms. Platform owner helps out with optimization where required as t= hey > understand their micro-architecture best. I guess this is what is suppose= d to > happen through the review process in DPDK. But making sure everyone tests= it > before it gets merged avoids the surprises. >=20 > I think, The very same philosophy can be implemented with exiting mailing= list > method, if >=20 > 1) Author Cc all the architecture maintainers and platform owners for any= fast > path change where it can introduce performance regression. > 2) Author can CC the same list to request for performance check along wit= h > test command if area of performance regression known before. >=20 > I agree with last minute surprises are bad for both Author and platform o= wner. > I think, it can fixed by above scheme. >=20 Makes sense to me. > > > > > > > > > > > > IMO, This scheme won't work. I think, we are introducing such > > > > > performance critical feature, we need to put under function > > > > > pointer scheme so that if an application does not need such > > > > > feature it can use > > > plain loads. > > > > > > > > > IMO, we should do some more debugging before going into exploring > > > > other > > > options. > > > > > > Yes. But, how do we align with v18.11 release? > > > > > I think I have spent enough time optimizing the code. Please provide th= e > feedback and I will work on completing the fix. > > > > However, if the new patch is not satisfactory enough, we need another p= lan. >=20 > Based on the public release meeting held yesterday, RC3 date is on next > Monday. >=20 > I would suggest: > - Send your exiting tested patch in mailing list for review. In my > setup, The regression reduced to 5.7% from 24% > - Extend the patch for hash bulk case as well and check > NO_HASH_MULTI_LOOKUP > as zero with EM_HASH_LOOKUP_COUNT value as 16 or 8 for arm64. >=20 Thank you Jerin for testing. Unfortunately, when I looked at making the cor= responding changes for the hash_add API, they look more involved. They may = not be acceptable for RC3. I also need more time to spend on bulk case. For= RC3, we will split the lookup path for RW-lock and Lock-free. I will add t= he rest in a follow up patch. > > > > You had mentioned about using function pointers. I suggest, we use the > function pointer only for lookup function. Otherwise, it will be too much= of > code duplication. > > When lock-free is not used, the function with no memory-orderings will = be > called. However, I am not sure about the function pointer overhead. But t= his > will be a simple change. >=20 > It may not be very simple change as we need to take care secondary proces= s > case as well, see struct rte_mempool::ops_index scheme. >=20 Yes, I realized it while making changes. I have used a if statement with an= existing lock-free flag. Will send out the patch soon. > Since rte_hash_lookup() already NOT a inline function, so making it as in= line > and calling a function pointer inside may not attract much overhead. But = we > can tell only after testing(Which may be not possible for > RC3) >=20 > I think, in future it make sense to have function pointer scheme to avoid= new > APIs for different hash library and we can plugin other proven hash libra= ry like > urcu based one etc to DPDK. >=20 > https://lwn.net/Articles/573431/