From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from dpdk.org (dpdk.org [92.243.14.124]) by inbox.dpdk.org (Postfix) with ESMTP id 80B3AA04AD; Fri, 1 May 2020 16:56:26 +0200 (CEST) Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id 668711DA83; Fri, 1 May 2020 16:56:26 +0200 (CEST) Received: from EUR05-DB8-obe.outbound.protection.outlook.com (mail-db8eur05on2089.outbound.protection.outlook.com [40.107.20.89]) by dpdk.org (Postfix) with ESMTP id 983D51DA71; Fri, 1 May 2020 16:56:24 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=armh.onmicrosoft.com; s=selector2-armh-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=hdgeRRdwG1YHJexNFPoCZtzpepDdqFwIM+7odxBzu0E=; b=f9JDnB6DSNF1sXGrR7thUNZXglaQxot+UYtUXT5MikcQjNbPTAG2zndqEAt9y58diKE0+9xUVL6nesNVyDyDp384r0hTcfI0HPPPlUY/CZ/hHjOccJlDlS2NhjFpBj+RJCAI3QnU8lkL47TehMtsv2cj9NtWnVKkLTl8J+Qr+4A= Received: from AM6P193CA0098.EURP193.PROD.OUTLOOK.COM (2603:10a6:209:88::39) by AM6PR08MB3480.eurprd08.prod.outlook.com (2603:10a6:20b:44::29) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.2937.13; Fri, 1 May 2020 14:56:22 +0000 Received: from VE1EUR03FT034.eop-EUR03.prod.protection.outlook.com (2603:10a6:209:88:cafe::36) by AM6P193CA0098.outlook.office365.com (2603:10a6:209:88::39) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.2958.20 via Frontend Transport; Fri, 1 May 2020 14:56:22 +0000 Authentication-Results: spf=pass (sender IP is 63.35.35.123) smtp.mailfrom=arm.com; dpdk.org; dkim=pass (signature was verified) header.d=armh.onmicrosoft.com;dpdk.org; dmarc=bestguesspass action=none header.from=arm.com; Received-SPF: Pass (protection.outlook.com: domain of arm.com designates 63.35.35.123 as permitted sender) receiver=protection.outlook.com; client-ip=63.35.35.123; helo=64aa7808-outbound-1.mta.getcheckrecipient.com; Received: from 64aa7808-outbound-1.mta.getcheckrecipient.com (63.35.35.123) by VE1EUR03FT034.mail.protection.outlook.com (10.152.18.85) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.2958.20 via Frontend Transport; Fri, 1 May 2020 14:56:22 +0000 Received: ("Tessian outbound 11763d234d54:v54"); Fri, 01 May 2020 14:56:21 +0000 X-CR-MTA-TID: 64aa7808 Received: from 6cb37b5e2cf0.1 by 64aa7808-outbound-1.mta.getcheckrecipient.com id 12D7C7D6-B9F7-460C-80E2-95AE5411E9D1.1; Fri, 01 May 2020 14:56:16 +0000 Received: from EUR01-VE1-obe.outbound.protection.outlook.com by 64aa7808-outbound-1.mta.getcheckrecipient.com with ESMTPS id 6cb37b5e2cf0.1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384); Fri, 01 May 2020 14:56:16 +0000 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=MB11AMGTx4T3LNpf1nUiXXnGAqpmgaBD1iKdX+nph5iXSmVk5NCJqtL6aN6Ex1gAYUmisIpSb5LdfUupXI8XLHL8HUtOBLLzSqNdyCFU6/s3IpWBneOCUJ3V3LhoCpMRUXF7Km1KSL78uqtXnK7gIARlDM4/yTufAoAMhiJllE0k2yt1McgyFYiHC/tomd0IW146tw/4YlTUnLt2lQ0GZkr35V0Jdzr3JuQPJrvhbBkEqoKWY8EBH4i/aD7KnO/ZCl+HLD+fPEjZD8hBd32ZpfleDMFmPNk91HuOMYv98PAOckltj42DJikX6HFvwLhYWJ1cxK16yOiY7ylrih/brg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=hdgeRRdwG1YHJexNFPoCZtzpepDdqFwIM+7odxBzu0E=; b=NOd/9WdhTeXJhQR5husg6Ol026Ic9jVfdxkaTT2WKbYi1VUS+ZCsTICCD+WBqGAbDUAn+dHeUD7++VPYenyDX4d0WK4rsrUFvPJiZQdRdjAsubrqh1YMsyYp6+oS/YIoAlw3/Fbti4Ov/Oe4ebqr3b6JFY+UHhCccDLUMKdU+Zy0w8EnuhDjXz4eNRs1ST6b8RWOu0DmXb6KebFEiPoD5d3IcpA1KY6qdWGXMQxh+WC0WI4lrmwmmn4aubIZq/okgXtTwNCa9jphdACiRsH/P1lbYrb4Uo876r49m3zUM0OgPYTPAKhTIyTiVgHhG8iPA8w6FKoiO2q9WayT4l7VkQ== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=arm.com; dmarc=pass action=none header.from=arm.com; dkim=pass header.d=arm.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=armh.onmicrosoft.com; s=selector2-armh-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=hdgeRRdwG1YHJexNFPoCZtzpepDdqFwIM+7odxBzu0E=; b=f9JDnB6DSNF1sXGrR7thUNZXglaQxot+UYtUXT5MikcQjNbPTAG2zndqEAt9y58diKE0+9xUVL6nesNVyDyDp384r0hTcfI0HPPPlUY/CZ/hHjOccJlDlS2NhjFpBj+RJCAI3QnU8lkL47TehMtsv2cj9NtWnVKkLTl8J+Qr+4A= Received: from DBBPR08MB4646.eurprd08.prod.outlook.com (2603:10a6:10:f5::16) by DBBPR08MB4490.eurprd08.prod.outlook.com (2603:10a6:10:c5::21) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.2958.20; Fri, 1 May 2020 14:56:12 +0000 Received: from DBBPR08MB4646.eurprd08.prod.outlook.com ([fe80::1870:afc4:b90f:609d]) by DBBPR08MB4646.eurprd08.prod.outlook.com ([fe80::1870:afc4:b90f:609d%5]) with mapi id 15.20.2958.020; Fri, 1 May 2020 14:56:12 +0000 From: Honnappa Nagarahalli To: "Van Haaren, Harry" , Phil Yang , "dev@dpdk.org" CC: "thomas@monjalon.net" , "david.marchand@redhat.com" , "Ananyev, Konstantin" , "jerinj@marvell.com" , "hemant.agrawal@nxp.com" , Gavin Hu , nd , "stable@dpdk.org" , "Eads, Gage" , "Richardson, Bruce" , nd , Honnappa Nagarahalli , nd Thread-Topic: [PATCH v2 1/6] service: fix race condition for MT unsafe service Thread-Index: AQHWGYy9tt0UKX2nJkGWwRwyooSeg6iQWc2AgABcStCAAp6IgIAAAqzA Date: Fri, 1 May 2020 14:56:12 +0000 Message-ID: References: <1584407863-774-8-git-send-email-phil.yang@arm.com> <1587659482-27133-1-git-send-email-phil.yang@arm.com> <1587659482-27133-2-git-send-email-phil.yang@arm.com> In-Reply-To: Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-ts-tracking-id: 30bfe385-c7a0-4af1-9414-ce929cd5de18.0 x-checkrecipientchecked: true Authentication-Results-Original: intel.com; dkim=none (message not signed) header.d=none;intel.com; dmarc=none action=none header.from=arm.com; x-originating-ip: [217.140.111.135] x-ms-publictraffictype: Email X-MS-Office365-Filtering-HT: Tenant X-MS-Office365-Filtering-Correlation-Id: cdb40232-17d7-4fb3-9a42-08d7eddfd024 x-ms-traffictypediagnostic: DBBPR08MB4490:|DBBPR08MB4490:|AM6PR08MB3480: x-ld-processed: f34e5979-57d9-4aaa-ad4d-b122a662184d,ExtAddr x-ms-exchange-transport-forked: True X-Microsoft-Antispam-PRVS: x-checkrecipientrouted: true nodisclaimer: true x-ms-oob-tlc-oobclassifiers: OLM:10000;OLM:10000; x-forefront-prvs: 0390DB4BDA X-MS-Exchange-SenderADCheck: 1 X-Microsoft-Antispam-Untrusted: BCL:0; X-Microsoft-Antispam-Message-Info-Original: NvCl8QlLsOynODSTO0ydBtRPyXt1cEKCpBSa9o0tfi0potaFbuAl551K30WiDhmxghN3HUJymnYDpIBizsEEKNGwJZwGOaqE/qrZ9lj0NlvxdhAWGUG822AdkmIHi0xpNBnyXNW5iQnMY+vEBQcqJ+QR9+GvPZ4YQ+EaOFhoF3yG9GGeoeUAQ2Bak+C8ZinDwp8GVmyNRTKcfkeBNFCSXnUZdsIq/TSTucfZH0Ky7EaTXFOBMJUVGFWwYV6EtlNT9aKezuaa0XjJbhqEYp8h784b+v6p4NOBUmDSxpJ8okNLUIxIxlFXnNh65k1T7011EcUkFTWO86m8EWl9ApgD507EZR5fNJS6gY0eLhtnhmVhmh7evO5ZrieIt2SO9FoIK7ZvXbYyMzXCZU3BlLm5VpnOIh526GCUuBMSXGWbLvs0iKHrXMKZHEUyi6Sj3/Hm X-Forefront-Antispam-Report-Untrusted: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:DBBPR08MB4646.eurprd08.prod.outlook.com; PTR:; CAT:NONE; SFTY:; SFS:(4636009)(366004)(7696005)(110136005)(8936002)(54906003)(86362001)(52536014)(8676002)(4326008)(76116006)(66446008)(9686003)(66556008)(55016002)(66946007)(66476007)(64756008)(26005)(6506007)(7416002)(186003)(33656002)(2906002)(71200400001)(498600001)(5660300002); DIR:OUT; SFP:1101; x-ms-exchange-antispam-messagedata: WK3alNTcC7IY0M0EmGxEXYYHPqEeXzLJt8v8PiqC6pvQJQzeLnJuHemHyTOiNwlNKzciy39JwgdgDgSKPOvFndYlN+OfTv4QUhn2ki6ZrXFGOxBleiUf4A1CBmwvMP52N/NUutQjyywnkvMBwFdkonhaGrLrGowtpx/kX8aKAZOaYCKTxjbuUs290VvDSd+RmAsJlhESH+Ft3K3YaXtbMsnnnxxXAkXpfaIjN4B91tG8RpJl74rXAoluwWUr4xpEExg614IEKdKAm4+LNLhCWQ5bjMU6V5mZFFn88rZC7eZf2HhyJNviATz42eO3ncDDa+h3SmXM7E7aFPXkbkgaiOwWgAFRQn1FTsCMOYHnI+xhvL78MGa3BAJvx3ydZxGPgrH6GwsvuX/b404KR9Qhxy7aT2ZlEIjikPE+dC299mlgd80nwKwbINh707p9jcPycZ+IGEw9ulk3oMf2rN6boFklulvlaowRSnNyuhKexx6smZvY9yyZYirX9CulV7TTMFClvlVv4UE7T9J/MZFW0kiCG1yQ1gzr/4hjxWAF2omUWh4asWmO3Fn4ZIbIGcrcykSeHZ+a3SIXdZ4qkon6xacb4kQgtGZC9xMipwz7VZiX3ug1cQAmF/3Py2Y7lrzCoMS0X3LcXvpMufRYZ8hJX33hzQ2Ngb7gZxa+/wVoKYOjVCzGTf9oo8+dsZFf9JSIgN6RwQAagVYeUBqJISlsVMQD+jB3fjIzDa9PI0Bq4FZVrTTt82CRIaydyf2g8jKOgFYA8Q56+6Pn8zeTnjE7QqEHnM+4yMjtlkMZIcyZAVwAAUnWy67Sbmch1sTIIKU0 Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 X-MS-Exchange-Transport-CrossTenantHeadersStamped: DBBPR08MB4490 Original-Authentication-Results: intel.com; dkim=none (message not signed) header.d=none;intel.com; dmarc=none action=none header.from=arm.com; X-EOPAttributedMessage: 0 X-MS-Exchange-Transport-CrossTenantHeadersStripped: VE1EUR03FT034.eop-EUR03.prod.protection.outlook.com X-Forefront-Antispam-Report: CIP:63.35.35.123; CTRY:IE; LANG:en; SCL:1; SRV:; IPV:CAL; SFV:NSPM; H:64aa7808-outbound-1.mta.getcheckrecipient.com; PTR:ec2-63-35-35-123.eu-west-1.compute.amazonaws.com; CAT:NONE; SFTY:; SFS:(4636009)(346002)(39860400002)(136003)(376002)(396003)(46966005)(70586007)(8936002)(70206006)(52536014)(8676002)(26005)(6506007)(82740400003)(186003)(82310400002)(336012)(356005)(47076004)(81166007)(5660300002)(36906005)(33656002)(4326008)(7696005)(316002)(110136005)(2906002)(478600001)(450100002)(55016002)(9686003)(54906003)(86362001); DIR:OUT; SFP:1101; X-MS-Office365-Filtering-Correlation-Id-Prvs: b0c7db87-bf0b-4318-9c18-08d7eddfca41 X-Forefront-PRVS: 0390DB4BDA X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: JuISETtZaH0CMCnRRT4lTLyV8PsPdFXcweZn14h2eux2RrmqivMtgpw9mXpo05rJ8Q6g5rs1f2LfXlgo122AfvXZSmaJK04FeTTm3fS4pLediw4bfdRwCIqEVDcwc5YCw7vZvjvvcicCUARxi5bRVZCI+PbAo9gQhLEDOVkhoqfbROgQNrpaDSOoms2C63704e/iXoeCvoarxzZZHKXvbx1amuhSEUpIUx7gHbLsUsRpqlJalF29CE+yiGwOtf0NFcAzbsFCl6qmek5ykDN+BbGSDO/5Q8pAqs5kXe9vIfGISZiMwlI+c8SIX/YOATLpf0Ak44AyaMpQviEeQcSgVT7fxc9pWOKKgB7e6/EGsN53LQPCitOP/0nOreatucRI5HzkJiqyPyLDXTK7FHe/IoKVELvwXlz1wauz7oubraCo5e+KjHtyxIdxxGhKRqpdTpMkIu7BgTjzWPFKL6jRqvxBVraKsfZ+dQjMN6BKBEGAzxS9j21YcO0HhWrrp9DZhqhrGeVsV4Dlb0ORcGfgOA== X-OriginatorOrg: arm.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 01 May 2020 14:56:22.0354 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: cdb40232-17d7-4fb3-9a42-08d7eddfd024 X-MS-Exchange-CrossTenant-Id: f34e5979-57d9-4aaa-ad4d-b122a662184d X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=f34e5979-57d9-4aaa-ad4d-b122a662184d; Ip=[63.35.35.123]; Helo=[64aa7808-outbound-1.mta.getcheckrecipient.com] X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: AM6PR08MB3480 Subject: Re: [dpdk-dev] [PATCH v2 1/6] service: fix race condition for MT unsafe service X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" > > > > > > Subject: [PATCH v2 1/6] service: fix race condition for MT unsafe > > > > service > > > > > > > > From: Honnappa Nagarahalli > > > > > > > > The MT unsafe service might get configured to run on another core > > > > while the service is running currently. This might result in the > > > > MT unsafe service running on multiple cores simultaneously. Use > > > > 'execute_lock' always when the service is MT unsafe. > > > > > > > > Fixes: e9139a32f6e8 ("service: add function to run on app lcore") > > > > Cc: stable@dpdk.org > > > > > > > > Signed-off-by: Honnappa Nagarahalli > > > > Reviewed-by: Phil Yang > > > > --- > > > > > > Thanks for spinning a new revision - based on ML discussion > > > previously, it seems like the "use service-run-count" to avoid this > > > race would be a complex solution. > > > > > > Suggesting the following; > > > 1) Take the approach as per this patch, to always take the atomic, > > > fixing the race condition. > > Ok >=20 > I've micro-benchmarked this code change inside the service cores autotest= , > and it introduces around 35 cycles of overhead per service call. This is= not > ideal, but given it's a bugfix, and by far the simplest method to fix thi= s race- > condition. Having discussed and investigated multiple other solutions, I > believe this is the right solution. > Thanks Honnappa and Phil for identifying and driving a solution. You are welcome. Thank you for your timely responses. >=20 > I suggest to post the benchmarking unit-test addition patch, and integrat= e > that > *before* the series under review here gets merged? This makes > benchmarking of the "before bugfix" performance in future easier should i= t be > required. I do not see any issues, would be happy to review. I think we still have ti= me to catch up with RC2 (May 8th). You had also mentioned about calling out that, the control plane APIs are n= ot MT safe. Should I add that to this patch? >=20 >=20 > > > 2) Add an API to service-cores, which allows "committing" of mappings= . > > > Committing the mapping would imply that the mappings will not be > > > changed in future. With runtime-remapping being removed from the > > > equation, the existing branch-over-atomic optimization is valid again= . > > Ok. Just to make sure I understand this: > > a) on the data plane, if commit API is called (probably a new state > > variable) and num_mapped_cores is set to 1, there is no need to take th= e > lock. > > b) possible implementation of the commit API would check if > > num_mapped_cores for the service is set to 1 and set a variable to > > indicate that the lock is not required. > > > > What do you think about asking the application to set the service > > capability to MT_SAFE if it knows that the service will run on a > > single core? This would require us to change the documentation and does > not require additional code. >=20 > That's a nice idea - I like that if applications want to micro-optimize a= round > the atomic, that they have a workaround/solution to do so, particularly t= hat it > doesn't require code-changes and backporting. >=20 > Will send review and send feedback on the patches themselves. > Regards, -Harry >=20 > > > So this would offer applications two situations > > > A) No application change: possible performance regression due to > > > atomic always taken. > > > B) Call "commit" API, and regain the performance as per previous > > > DPDK versions. > > > > > > Thoughts/opinions on the above? I've flagged the rest of the > > > patchset for review ASAP. Regards, -Harry > > > > > > > lib/librte_eal/common/rte_service.c | 11 +++++------ > > > > 1 file changed, 5 insertions(+), 6 deletions(-) >