From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from dpdk.org (dpdk.org [92.243.14.124]) by inbox.dpdk.org (Postfix) with ESMTP id 301EFA00C5; Thu, 30 Apr 2020 00:49:11 +0200 (CEST) Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id A47061D9D6; Thu, 30 Apr 2020 00:49:10 +0200 (CEST) Received: from EUR05-AM6-obe.outbound.protection.outlook.com (mail-am6eur05on2052.outbound.protection.outlook.com [40.107.22.52]) by dpdk.org (Postfix) with ESMTP id C1D291D9AD; Thu, 30 Apr 2020 00:49:08 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=armh.onmicrosoft.com; s=selector2-armh-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=QNjXLzirUqgL41pdcHhVcJ2cwuof92hjlXOqfUCsyjM=; b=AWUjnImexvqyvYJRRIG/gUxSWHg6PFtUqo96RQKPiuVmTurUax4JG9PF2TWh2uZ+GJAS0LvjUyUkzksnX8n6C7WRZLRfe9YGW15bOlWIMF1F2cgXF/5jCOLWIZ9gBPhZnEAWgLUKCJmoySUHHrlOqjfJh6COorNOhehbWSP7OcU= Received: from AM5PR0201CA0015.eurprd02.prod.outlook.com (2603:10a6:203:3d::25) by AM6PR08MB3141.eurprd08.prod.outlook.com (2603:10a6:209:4a::31) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.2937.13; Wed, 29 Apr 2020 22:49:07 +0000 Received: from VE1EUR03FT061.eop-EUR03.prod.protection.outlook.com (2603:10a6:203:3d:cafe::28) by AM5PR0201CA0015.outlook.office365.com (2603:10a6:203:3d::25) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.2958.20 via Frontend Transport; Wed, 29 Apr 2020 22:49:07 +0000 Authentication-Results: spf=pass (sender IP is 63.35.35.123) smtp.mailfrom=arm.com; dpdk.org; dkim=pass (signature was verified) header.d=armh.onmicrosoft.com;dpdk.org; dmarc=bestguesspass action=none header.from=arm.com; Received-SPF: Pass (protection.outlook.com: domain of arm.com designates 63.35.35.123 as permitted sender) receiver=protection.outlook.com; client-ip=63.35.35.123; helo=64aa7808-outbound-1.mta.getcheckrecipient.com; Received: from 64aa7808-outbound-1.mta.getcheckrecipient.com (63.35.35.123) by VE1EUR03FT061.mail.protection.outlook.com (10.152.19.220) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.2958.20 via Frontend Transport; Wed, 29 Apr 2020 22:49:06 +0000 Received: ("Tessian outbound 4cdf5642225a:v54"); Wed, 29 Apr 2020 22:49:06 +0000 X-CR-MTA-TID: 64aa7808 Received: from f643c5c0b085.1 by 64aa7808-outbound-1.mta.getcheckrecipient.com id 863B5E00-643E-4EFE-A8F8-8F4FD921D170.1; Wed, 29 Apr 2020 22:49:01 +0000 Received: from EUR05-DB8-obe.outbound.protection.outlook.com by 64aa7808-outbound-1.mta.getcheckrecipient.com with ESMTPS id f643c5c0b085.1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384); Wed, 29 Apr 2020 22:49:01 +0000 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=KxoHR5ivlTdW8sCMZS4+wFfWNArbIX8PdE2JgJFx+wMq8mKF6630NWswMhkLo0gj+D8KhtBJ4U4VS+FBLmqslM+Il+k5LDDS8PHQSzdmnbBFgvBg7oOtkVw7YrfVVdPVOBEsP+Kqsosd8UrYdNGx/eZlSuG9d9Ss4XoJECEHGJVtL87WCaY+n5Bq8We1B90jlyPs77MEVRnk5TdDygxncSEWj/Mv3BPUENhGSShlzpjjryjspS7GK5OlkoZu3Jy2+OcCrB8/dC7Go9mlDQDvCui3C0ywgasPsCcjDsE3HogeSBzjMGb4CNHq6UasOLSoMYt6pFw+TmqvybjeamjIlQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=QNjXLzirUqgL41pdcHhVcJ2cwuof92hjlXOqfUCsyjM=; b=L5zs2ky5GWiNWcCBh40E/sjPtV36w9rxtQjQ/jNCwxy2rC92HKtGfsou4ZSQlsBIkODhXoY1ZTh8A7Gb7/rvs9aFa/Sj65HD8XcZXKM8d/b7/bZzlV4RCjsufZSljU6YciNsxy+M1m17fswieJ76/t7hG64D96KrEIMRJba5GvdJ0TFCyfEcARe5BRAJbglLaPpp46km+xl6DhnKZouj+kiadWiU5M1dSMec5t7hSXYDzmhN9W6FclTcv7twh22AmRznlqu9YwS5xOgdkZ0rtMurHvt6df27ajxpfNSwxPMDNu8sV0F2kuvNGhSWzJhPz8EBMXOabtrdiVkG4CyG0A== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=arm.com; dmarc=pass action=none header.from=arm.com; dkim=pass header.d=arm.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=armh.onmicrosoft.com; s=selector2-armh-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=QNjXLzirUqgL41pdcHhVcJ2cwuof92hjlXOqfUCsyjM=; b=AWUjnImexvqyvYJRRIG/gUxSWHg6PFtUqo96RQKPiuVmTurUax4JG9PF2TWh2uZ+GJAS0LvjUyUkzksnX8n6C7WRZLRfe9YGW15bOlWIMF1F2cgXF/5jCOLWIZ9gBPhZnEAWgLUKCJmoySUHHrlOqjfJh6COorNOhehbWSP7OcU= Received: from DBBPR08MB4646.eurprd08.prod.outlook.com (2603:10a6:10:f5::16) by DBBPR08MB4442.eurprd08.prod.outlook.com (2603:10a6:10:c9::16) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.2937.13; Wed, 29 Apr 2020 22:49:00 +0000 Received: from DBBPR08MB4646.eurprd08.prod.outlook.com ([fe80::1870:afc4:b90f:609d]) by DBBPR08MB4646.eurprd08.prod.outlook.com ([fe80::1870:afc4:b90f:609d%5]) with mapi id 15.20.2958.020; Wed, 29 Apr 2020 22:49:00 +0000 From: Honnappa Nagarahalli To: "Van Haaren, Harry" , Phil Yang , "dev@dpdk.org" CC: "thomas@monjalon.net" , "david.marchand@redhat.com" , "Ananyev, Konstantin" , "jerinj@marvell.com" , "hemant.agrawal@nxp.com" , Gavin Hu , nd , "stable@dpdk.org" , "Eads, Gage" , "Richardson, Bruce" , Honnappa Nagarahalli , nd Thread-Topic: [PATCH v2 1/6] service: fix race condition for MT unsafe service Thread-Index: AQHWGYy9tt0UKX2nJkGWwRwyooSeg6iQWc2AgABcStA= Date: Wed, 29 Apr 2020 22:48:59 +0000 Message-ID: References: <1584407863-774-8-git-send-email-phil.yang@arm.com> <1587659482-27133-1-git-send-email-phil.yang@arm.com> <1587659482-27133-2-git-send-email-phil.yang@arm.com> In-Reply-To: Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-ts-tracking-id: 07be32ce-aa61-4b5b-bf94-1879a04b66b1.0 x-checkrecipientchecked: true Authentication-Results-Original: intel.com; dkim=none (message not signed) header.d=none;intel.com; dmarc=none action=none header.from=arm.com; x-originating-ip: [217.140.111.135] x-ms-publictraffictype: Email X-MS-Office365-Filtering-HT: Tenant X-MS-Office365-Filtering-Correlation-Id: b21179b1-5c5b-4463-0028-08d7ec8f85db x-ms-traffictypediagnostic: DBBPR08MB4442:|DBBPR08MB4442:|AM6PR08MB3141: x-ld-processed: f34e5979-57d9-4aaa-ad4d-b122a662184d,ExtAddr x-ms-exchange-transport-forked: True X-Microsoft-Antispam-PRVS: x-checkrecipientrouted: true nodisclaimer: true x-ms-oob-tlc-oobclassifiers: OLM:4125;OLM:4125; x-forefront-prvs: 03883BD916 X-Forefront-Antispam-Report-Untrusted: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:DBBPR08MB4646.eurprd08.prod.outlook.com; PTR:; CAT:NONE; SFTY:; SFS:(4636009)(39860400002)(376002)(346002)(366004)(396003)(136003)(186003)(4326008)(66556008)(64756008)(76116006)(66476007)(66446008)(86362001)(8936002)(33656002)(7416002)(9686003)(71200400001)(66946007)(55016002)(52536014)(5660300002)(478600001)(7696005)(316002)(110136005)(54906003)(2906002)(26005)(8676002)(6506007); DIR:OUT; SFP:1101; X-MS-Exchange-SenderADCheck: 1 X-Microsoft-Antispam-Untrusted: BCL:0; X-Microsoft-Antispam-Message-Info-Original: n/0HnPGUqSaEhS/1hUG3j2XhHZxj0NZC4F79DN0E1vknC7gZHGqQkjjawDRiiC7yZY33GO4HNSpXccB4oCEpH5CRVrIoXXYgwMVDM1dankfGEYDun15q94tFexypYtReLO8If5JqYumCSWk7zkfkHuzwVCBONdy7BXqjP/mNEAagFvBo1EBSYYUN9+t2wjpyMad8hlyGTuxlddlJQs+MCuAOsAKy6UqnoGEgaY0vYxig5Iw0eeKD/D08DBU4P3eeGNILt2THp5pYpMaDcYEyKdN12RICKPn2+c4LFnHM1YNQDC/lm9jaGRk9pbqQ5/QWubcsfJ+DnYdQM13h2QOURkOp7l/5shJkt2k/ZYCubI5GqwQLkygBuDK5qrJwX25dsCVCgLfKvOaZgwFrmw9BuvzPPFXZB1RizccEAEEqI1hgGwd3AmFq0J0DaTWCDFEp x-ms-exchange-antispam-messagedata: IwSiUCXEPoi9etLeA6c57QgYyZk1x5WQw7itb2VGAYU3kcGIaIF3Z3yAQL+/xrmeRZUfXFFzQOiVhEZjC6rtRTFaM3rHDGxhX93PZksA6FsFrba11lwQvvDtkQb/Y+GC78p+MXOBzK33nyPK5fJZs3FMSONrIWTpGz5JILU20uQ1z+9nVd3kaGtdZ/O3SXU2PlJVVXVHUTMCgM0Ln4ruEkTK+iJyzlQrh8Ie/Tu1XQH1P0RBGg9RSjAdShN3RShpnoXds1XaDuULss1cHBbUVGkhp2ShTkG1kDmcQ5H9XSkiq3CxW/lzj0l6zf7qzcoHNh4ybCDKLlY2qel7FbjzvlRMI/ozMB/2n1jWJyL65/2RKpRFJm/Uh4todNI7Y3PbhbBdE/r6mzv+J3roN8uuOCfh46XYal6EdFwRg3DjIkbQd5JcXyvp3qDC+I9rGXWhcjRy6v5zQQBpfFNYmU+yQ0dsxYf6U2sjaR47PsXOwB2VKf8SPFyE4C6SWxTVI9meFqhIQVc1uTJ+B43c0GRGTmv6iiW331E8/P1sijVeMjN0UgLgRGnoMe55JWF4/EmfaqzPZ8vMuQrqLqWw0Rrcfmq/EBS1Y8C2yz+7GZwo3f9y18mMsBiq9VKO01YVp95ht5XI4EgqrNoETgqaiKvPZMqyeBa570kBNDrVoQXkP4I/v2GwuIUxml0drUrlp3GxZ1BVyP3BTCZHOMl0q9KuRXckBaNh9LedfTkRZ7ps0SjMQnVxBVrTbDLu8E8fCMUCaF2LISCWlLhjs4jv8fhNpXxw1LDSowVyWRW/d0/Q4oJYTIcEvHaht/GYnpGxroS2 Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 X-MS-Exchange-Transport-CrossTenantHeadersStamped: DBBPR08MB4442 Original-Authentication-Results: intel.com; dkim=none (message not signed) header.d=none;intel.com; dmarc=none action=none header.from=arm.com; X-EOPAttributedMessage: 0 X-MS-Exchange-Transport-CrossTenantHeadersStripped: VE1EUR03FT061.eop-EUR03.prod.protection.outlook.com X-Forefront-Antispam-Report: CIP:63.35.35.123; CTRY:IE; LANG:en; SCL:1; SRV:; IPV:CAL; SFV:NSPM; H:64aa7808-outbound-1.mta.getcheckrecipient.com; PTR:ec2-63-35-35-123.eu-west-1.compute.amazonaws.com; CAT:NONE; SFTY:; SFS:(4636009)(39860400002)(396003)(346002)(376002)(136003)(46966005)(110136005)(36906005)(82310400002)(316002)(54906003)(7696005)(86362001)(8936002)(81166007)(47076004)(336012)(186003)(478600001)(356005)(2906002)(52536014)(5660300002)(70206006)(26005)(82740400003)(70586007)(450100002)(9686003)(55016002)(33656002)(8676002)(6506007)(4326008); DIR:OUT; SFP:1101; X-MS-Office365-Filtering-Correlation-Id-Prvs: ddeee3ce-c458-48e4-4ff9-08d7ec8f81fa X-Forefront-PRVS: 03883BD916 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: E9UnSjHxE+BzaIBnKf19ZIfBij7qZMgw/nMydj2w3YQjLqXUthWcjk7dnXdLJDHPxtkuqdVoeZH+XV4/85cFWQcaB3tHIcqdjM0mX/xJzaL2rk8O8HaZKYbGrXJUlTvPC6eJ7IiIzSE7gVS0cl6cWNvoH/Jwl//P221DaXefA42mkM8RA3d999qpksv/8N2tpCuf2f98Mr1b4z6ZvrXu9ZMMLZDGkv3hX1fAacje5mrugyQGgkgVvdAt2eI65DQ7EmXp0pBu5TT7ojRDL+pYGVNuigCfkckukFy1EpBg/meEkn22DHj0ZoLEtNCksTs2Pj+QOWtxFAS0+Qqcy6Nnr6ZtGIzQZGvTaq46fpsi84MEFcZTA2wOuV4atbHEWYMmkm2EkKh0ehAJIy4Xg/IP9ugpCjfjRNIKyR9Sk1OaJFlYUb8DRLUKqbhej4xZdvffT6vhknY9LU5No75mBxICSSXg4yZaWxYSu7n7l6V9QZsq+8XZtUoBQ9WwCb7QOX6Bzj0/TmlsjMNJyz1DnoX6mA== X-OriginatorOrg: arm.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 29 Apr 2020 22:49:06.4975 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: b21179b1-5c5b-4463-0028-08d7ec8f85db X-MS-Exchange-CrossTenant-Id: f34e5979-57d9-4aaa-ad4d-b122a662184d X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=f34e5979-57d9-4aaa-ad4d-b122a662184d; Ip=[63.35.35.123]; Helo=[64aa7808-outbound-1.mta.getcheckrecipient.com] X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: AM6PR08MB3141 Subject: Re: [dpdk-dev] [PATCH v2 1/6] service: fix race condition for MT unsafe service X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" Hi Harry, Thanks for getting back on this. > > Subject: [PATCH v2 1/6] service: fix race condition for MT unsafe > > service > > > > From: Honnappa Nagarahalli > > > > The MT unsafe service might get configured to run on another core > > while the service is running currently. This might result in the MT > > unsafe service running on multiple cores simultaneously. Use > > 'execute_lock' always when the service is MT unsafe. > > > > Fixes: e9139a32f6e8 ("service: add function to run on app lcore") > > Cc: stable@dpdk.org > > > > Signed-off-by: Honnappa Nagarahalli > > Reviewed-by: Phil Yang > > --- >=20 > Thanks for spinning a new revision - based on ML discussion previously, i= t > seems like the "use service-run-count" to avoid this race would be a comp= lex > solution. >=20 > Suggesting the following; > 1) Take the approach as per this patch, to always take the atomic, fixing= the > race condition. Ok > 2) Add an API to service-cores, which allows "committing" of mappings. > Committing the mapping would imply that the mappings will not be changed > in future. With runtime-remapping being removed from the equation, the > existing branch-over-atomic optimization is valid again. Ok. Just to make sure I understand this: a) on the data plane, if commit API is called (probably a new state variabl= e) and num_mapped_cores is set to 1, there is no need to take the lock. b) possible implementation of the commit API would check if num_mapped_core= s for the service is set to 1 and set a variable to indicate that the lock = is not required. What do you think about asking the application to set the service capabili= ty to MT_SAFE if it knows that the service will run on a single core? This = would require us to change the documentation and does not require additiona= l code. >=20 > So this would offer applications two situations > A) No application change: possible performance regression due to atomic > always taken. > B) Call "commit" API, and regain the performance as per previous DPDK > versions. >=20 > Thoughts/opinions on the above? I've flagged the rest of the patchset fo= r > review ASAP. Regards, -Harry >=20 > > lib/librte_eal/common/rte_service.c | 11 +++++------ > > 1 file changed, 5 insertions(+), 6 deletions(-) > > > > diff --git a/lib/librte_eal/common/rte_service.c > > b/lib/librte_eal/common/rte_service.c > > index 70d17a5..b8c465e 100644 > > --- a/lib/librte_eal/common/rte_service.c > > +++ b/lib/librte_eal/common/rte_service.c > > @@ -50,6 +50,10 @@ struct rte_service_spec_impl { > > uint8_t internal_flags; > > > > /* per service statistics */ > > + /* Indicates how many cores the service is mapped to run on. > > + * It does not indicate the number of cores the service is running > > + * on currently. > > + */ > > rte_atomic32_t num_mapped_cores; > > uint64_t calls; > > uint64_t cycles_spent; > > @@ -370,12 +374,7 @@ service_run(uint32_t i, struct core_state *cs, > > uint64_t service_mask, > > > > cs->service_active_on_lcore[i] =3D 1; > > > > - /* check do we need cmpset, if MT safe or <=3D 1 core > > - * mapped, atomic ops are not required. > > - */ > > - const int use_atomics =3D (service_mt_safe(s) =3D=3D 0) && > > - (rte_atomic32_read(&s- > >num_mapped_cores) > 1); > > - if (use_atomics) { > > + if (service_mt_safe(s) =3D=3D 0) { > > if (!rte_atomic32_cmpset((uint32_t *)&s->execute_lock, 0, 1)) > > return -EBUSY; > > > > -- > > 2.7.4