From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 88145A00C5; Fri, 8 Jul 2022 18:34:15 +0200 (CEST) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 69C62406B4; Fri, 8 Jul 2022 18:34:15 +0200 (CEST) Received: from EUR04-VI1-obe.outbound.protection.outlook.com (mail-eopbgr80085.outbound.protection.outlook.com [40.107.8.85]) by mails.dpdk.org (Postfix) with ESMTP id 967B54021E for ; Fri, 8 Jul 2022 18:34:13 +0200 (CEST) ARC-Seal: i=2; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=pass; b=DaGmBSGztPdql3gsu7NVtbbiPCR9lrw8VumbSWLYXQk+rZSPymr7Y64nWtI5dN/2N3QuYMiqkenHtG8Cujeo2IXLKLAnL4OaezjYUJy62irO3/iLA18jCjPIi7hkiCQT+GavhIsV5nNewFDFoBZWtZjs7/8d7iQ5vbeI1LjyqQNk8UUirum1kOZzucilJcLpdZR789xTWSo0srwU2CRITozYpi6my84y9F+U2Bu9pJAu2P1rQz2Vk01P1U21BaUuSPhFoEAUYMktNJyJ73/gigGREaxwMX0PkoqUuhTglRLpSfUIWqCP5YGuknOEFeBBoZhT4g89zafZNamZtE5A7w== ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=qyYGrHSyzNLrt5tPq5Ma41k1UoUiK9i3xIVNMCfHmSo=; b=BOAcv01cEy2bllaFkYHLfL8i0sn2DJ/MpijzY3YKIeV7JYvPN3aEMH22ahMKKbvDGE++Dhi7UCArUd0H8gmPVDKSYqavDN1AyZRn27zChVxQ70sUzJJya4+mm3NVJIeeGOnMypCDqGON2vbRQulFOH5U4PPjZQesXFpSov3QRJ6x3Jesn/eiAcaGSK4zjfAi9MJy2EBzAbF+zyCAGOcbEzB1pKspApEgJOUA5vbluNvvyFWX16AWTws0M43l996jKLJPltp/nqO6wThhioA18aZJbzaaQ5YwRxUUCWi7Q4+VS/3DrpsQoThUOT8s4qzciFISiC5DwmJJijREsT/9lg== ARC-Authentication-Results: i=2; mx.microsoft.com 1; spf=pass (sender ip is 63.35.35.123) smtp.rcpttodomain=dpdk.org smtp.mailfrom=arm.com; dmarc=pass (p=none sp=none pct=100) action=none header.from=arm.com; dkim=pass (signature was verified) header.d=armh.onmicrosoft.com; arc=pass (0 oda=1 ltdi=1 spf=[1,1,smtp.mailfrom=arm.com] dkim=[1,1,header.d=arm.com] dmarc=[1,1,header.from=arm.com]) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=armh.onmicrosoft.com; s=selector2-armh-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=qyYGrHSyzNLrt5tPq5Ma41k1UoUiK9i3xIVNMCfHmSo=; b=0de1cZkHyN/OHE94F7YuDRuHVNzoodf6DE3e1OpKnL8QxDe6xI+kjNZcKu07Hhlda3aBq5Q5s+LcDGTRirvHZCxLj0PIeGUYawVTioVEWojFtEo5lz0tHtU1XG9Pg0jrzBvxoBd1PqjBVsPhf/jIZhNFJSnzdUKDICWGLlvT4cE= Received: from DB6PR0202CA0041.eurprd02.prod.outlook.com (2603:10a6:4:a5::27) by VI1PR0802MB2222.eurprd08.prod.outlook.com (2603:10a6:800:9b::19) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5417.16; Fri, 8 Jul 2022 16:34:01 +0000 Received: from DBAEUR03FT012.eop-EUR03.prod.protection.outlook.com (2603:10a6:4:a5:cafe::1f) by DB6PR0202CA0041.outlook.office365.com (2603:10a6:4:a5::27) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5417.20 via Frontend Transport; Fri, 8 Jul 2022 16:34:01 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 63.35.35.123) smtp.mailfrom=arm.com; dkim=pass (signature was verified) header.d=armh.onmicrosoft.com;dmarc=pass action=none header.from=arm.com; Received-SPF: Pass (protection.outlook.com: domain of arm.com designates 63.35.35.123 as permitted sender) receiver=protection.outlook.com; client-ip=63.35.35.123; helo=64aa7808-outbound-1.mta.getcheckrecipient.com; pr=C Received: from 64aa7808-outbound-1.mta.getcheckrecipient.com (63.35.35.123) by DBAEUR03FT012.mail.protection.outlook.com (100.127.142.126) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5417.15 via Frontend Transport; Fri, 8 Jul 2022 16:34:00 +0000 Received: ("Tessian outbound 8e3d5168572a:v122"); Fri, 08 Jul 2022 16:34:00 +0000 X-CR-MTA-TID: 64aa7808 Received: from 8b3b979eb880.2 by 64aa7808-outbound-1.mta.getcheckrecipient.com id 6C9ADA0B-7390-4C4B-B2CC-D2AAB10759DF.1; Fri, 08 Jul 2022 16:33:53 +0000 Received: from EUR05-DB8-obe.outbound.protection.outlook.com by 64aa7808-outbound-1.mta.getcheckrecipient.com with ESMTPS id 8b3b979eb880.2 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384); Fri, 08 Jul 2022 16:33:53 +0000 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=U4Vvnm1s5f8BUwwV/hCph7a6+0lNwrZ/+urbnvm1s6fm9OZ2mA+2/8urfIVU1b6YJUQPgTvItJiY2t7DKqihbK8JQrVI67xvKaIHTQoEzomlzvm67hWkMLHUBe8ZGF4+HnyMr24kr174CEajs2tKlHqNb+U2vIYBEujIcIwyLXZxK07VUsFYPrE6fbNCoMKVY5KSKVIZI31ZN8kGVtzcSqjiOZ6c+lIenv03spsQGABfSxiGCVSREYLkWGSYAUfZmoHcd2Hv/tR2fTpWhQEibDPvO24D/4i2yC3UMRUqTHS4U9Mn4OZvEYooKLuF7AyZAQpcUOebdbcc5KACLIXfAQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=qyYGrHSyzNLrt5tPq5Ma41k1UoUiK9i3xIVNMCfHmSo=; b=gjN+pkLYhO+Lg27VsmpW8/1tVsF1fFCLiesmxQ5AVglwLXFkQCNi1uTa1TefxoE/JxW0GY3gKxx+Vmh2M9moG+6ZxECLOS6QXnaBCJIGZLdUlN9v+V76dmshBMp+F9fKCX5a/bNXwkH3xKU4r66O/LMo3CXdrDuvt1xMmo/PGTLf6gOH+7tuRlyWJFZHUuLyw67uJVsQBGnYHjDsqRrz+2xHXi43eSreQEPBVUdFLo4fnA+ORuR4NFmA+kGXGZc6tZiQK4MtItAir9DoBKcT/2dCYqK83kHrMF9xJjSjn7dzNHBARS/ieW2vhG/SZ6U2gEEFKNBcxuzJliusflml7Q== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=arm.com; dmarc=pass action=none header.from=arm.com; dkim=pass header.d=arm.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=armh.onmicrosoft.com; s=selector2-armh-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=qyYGrHSyzNLrt5tPq5Ma41k1UoUiK9i3xIVNMCfHmSo=; b=0de1cZkHyN/OHE94F7YuDRuHVNzoodf6DE3e1OpKnL8QxDe6xI+kjNZcKu07Hhlda3aBq5Q5s+LcDGTRirvHZCxLj0PIeGUYawVTioVEWojFtEo5lz0tHtU1XG9Pg0jrzBvxoBd1PqjBVsPhf/jIZhNFJSnzdUKDICWGLlvT4cE= Received: from DBAPR08MB5814.eurprd08.prod.outlook.com (2603:10a6:10:1b1::6) by PA4PR08MB5951.eurprd08.prod.outlook.com (2603:10a6:102:ee::14) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5395.15; Fri, 8 Jul 2022 16:33:50 +0000 Received: from DBAPR08MB5814.eurprd08.prod.outlook.com ([fe80::1c7f:6a8d:b518:f972]) by DBAPR08MB5814.eurprd08.prod.outlook.com ([fe80::1c7f:6a8d:b518:f972%3]) with mapi id 15.20.5417.016; Fri, 8 Jul 2022 16:33:50 +0000 From: Honnappa Nagarahalli To: Bruce Richardson , "Van Haaren, Harry" CC: "dev@dpdk.org" , mattias.ronnblom , =?iso-8859-1?Q?Morten_Br=F8rup?= , nd , nd Subject: RE: [PATCH 2/2] service: fix potential stats race-condition on MT services Thread-Topic: [PATCH 2/2] service: fix potential stats race-condition on MT services Thread-Index: AQHYkso9CqvSXQ5m5kCgd1gUAmDVCq10lTCQgAAEsICAAA3+AIAAAWyQ Date: Fri, 8 Jul 2022 16:33:50 +0000 Message-ID: References: <20220708125645.3141464-1-harry.van.haaren@intel.com> <20220708125645.3141464-2-harry.van.haaren@intel.com> In-Reply-To: Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-ts-tracking-id: 8925475CAD3EC447BB3AB9CE99EE4081.0 x-checkrecipientchecked: true Authentication-Results-Original: dkim=none (message not signed) header.d=none;dmarc=none action=none header.from=arm.com; X-MS-Office365-Filtering-Correlation-Id: 55d4e836-fb5a-4cad-4d94-08da60ffa9d8 x-ms-traffictypediagnostic: PA4PR08MB5951:EE_|DBAEUR03FT012:EE_|VI1PR0802MB2222:EE_ x-checkrecipientrouted: true nodisclaimer: true X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam-Untrusted: BCL:0; X-Microsoft-Antispam-Message-Info-Original: JGqalSXoy/8N08dTiiVzW6+p7MiWMHSuhZhpt1O+HJOD58fwdQfxP+AkP9I6G0UPVXiGgwSRneSx1N0OnOdL+P1zw/ijOxUKIxgczkzO48P2t2c+cNUpkF9z4neRFygaCf8uaDm/uLHMk3OjbjJpTZfQ0rWvNiV8WBp2egckWbeCxjPc8Zcpyr2iaKGr7ZvOxlXYXomLbomGZXLGrkNzAQCMTGuUOeeH69mOfGr3NW3KRoqdkIcZfV5cFtlUJGKwWTamyELy0H8pAHbPUkqzV6nWgOCUa0YhFPZY226NZ9z50v9iYj4qkHbWTmkRkZNoEYX0j/JzaBIFF559IEtZaiEJ8dWjYxcrDLW+zbvdrLdzPjMvSgCK6aYFmMUbkuAifGl9c6fi2BgF59v3zwdajngIXzOh45DmkXvGrU0EKlsPO95sS1AZxXEkjD7URgcQ2yc/sXehAYSH27HHrc9Bvsfi2/uiDs49mBeFw3A+ukiEYBOuI8+bQKSlnWHBEV62L0U8k/w7avRfr7nD0XcDaTsXOtKVdAIbrXB48jMlan81VI6N1ciumVBzrJo6xd5PU8aqsruQ1+5EmIxPEamW+YyK2/FLW9kde2WRCXGAq6DuFKRrpY/RPpS6uWvdaEVD0PfKRo73NQYto8ng/tp+KnWloCKPVJym6cEhoyNP0wn/R+v58mqyRlCpjX4nyzK7z8L9UoLtQbs5qYA/xmGp4bqmM0NB31mDzYpfme3WPOVHsrO5I/DaC6lKPwvCpLKMvuCHyTD38h1DiV6q/BwZZpgRgjsFIsvvUxfX/KpttTbdBz/VdWYox1CGgnNeK6Okk9yBucOUcmzKsKJ0r6wZwg== X-Forefront-Antispam-Report-Untrusted: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:DBAPR08MB5814.eurprd08.prod.outlook.com; PTR:; CAT:NONE; SFS:(13230016)(4636009)(376002)(396003)(366004)(346002)(136003)(39860400002)(316002)(966005)(8676002)(54906003)(71200400001)(83380400001)(41300700001)(186003)(110136005)(33656002)(76116006)(26005)(7696005)(66556008)(52536014)(38100700002)(66476007)(66946007)(8936002)(6506007)(122000001)(86362001)(478600001)(55016003)(4326008)(2906002)(9686003)(38070700005)(66446008)(5660300002)(64756008); DIR:OUT; SFP:1101; Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 X-MS-Exchange-Transport-CrossTenantHeadersStamped: PA4PR08MB5951 Original-Authentication-Results: dkim=none (message not signed) header.d=none;dmarc=none action=none header.from=arm.com; X-EOPAttributedMessage: 0 X-MS-Exchange-Transport-CrossTenantHeadersStripped: DBAEUR03FT012.eop-EUR03.prod.protection.outlook.com X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id-Prvs: 2b13e07b-bf2b-4e98-66b3-08da60ffa3b5 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: FHFMj7vZCOOZDkKh5kl4h9StuAHsixvVhNcYXPnaY04oH2RrVNjZLlqfKpIZl8JhwGEMDjE1CDlHnMl8lqZwTau+/5/XOwniHJbvAOWvKJBR+2X6B7m8ftUlSjUagENu1iEHxGDUjTiDcAYPz+7zSK5RZ3hxvF5WEq/sEVc80J/lbunZ8IOFJc+bPChoNIHFfu1FRCCxyMxBD6vCN+uvmHATRVPPw0jyrS5b2so7bYZYWTqddlm5if4OYjkxvBUDAgikfgMtazpCMJ4WCI+ExwXdEfT5N0gXqbtmWw7US6AK5gNki0Ute5WlATEnASw1z3hIL6O4+lEaLhDgQly5wLtKvbEPK80pna6v/jEXMTHWuzzvJSQrwAhecgj1H4OS8zwVHAmI2brwjCU8St5K/LrgFY0oHDbQrHiCJ+3EMykiUi+eclw2ZKjsNzqxYhpXW/8kfezSw4CZw5afEJCkHeY/CNcT6RyqEaqiDA60hPAMqkAq4ZAi0+kRgKRq7yE2cBiZx1N8diAw7KvObw7lrImERdxRitWO2j346W0gTT7SF7C9TlEyWszsbybE898g6NQOH6XGQluQP5Pr4wJ2o0BEZlfMWRfIQnXj6BUkgNk5eUWMHJ5yGbC2MfB34ZP1cicdBG9ENaXfOD38tlp6xvs1Ah0eVZFwsut7V7X9+9GzxEojP89Uwaj/8qmHILg1/gCmAnEpoy3WINrari6I0PIZ23epe3fXpF3rpXK35MzZnKE5bKr4n/0lAhtr3AlZbhA6upLMM47+9Rd0zZIlP3zmtEh+KzbP/GPiK1NMSp7OB/+CDKh5TYL9RALVnPGqDJyCbrglwoE6BxI4tUjuGf6JB/ukuQJF6lHCEU6gLBk= X-Forefront-Antispam-Report: CIP:63.35.35.123; CTRY:IE; LANG:en; SCL:1; SRV:; IPV:CAL; SFV:NSPM; H:64aa7808-outbound-1.mta.getcheckrecipient.com; PTR:ec2-63-35-35-123.eu-west-1.compute.amazonaws.com; CAT:NONE; SFS:(13230016)(4636009)(376002)(396003)(136003)(39860400002)(346002)(36840700001)(40470700004)(46966006)(6506007)(26005)(41300700001)(7696005)(478600001)(966005)(9686003)(47076005)(83380400001)(2906002)(336012)(186003)(55016003)(40460700003)(82310400005)(40480700001)(54906003)(316002)(110136005)(8936002)(8676002)(4326008)(356005)(70586007)(52536014)(70206006)(81166007)(86362001)(5660300002)(82740400003)(33656002)(36860700001); DIR:OUT; SFP:1101; X-OriginatorOrg: arm.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 08 Jul 2022 16:34:00.8798 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: 55d4e836-fb5a-4cad-4d94-08da60ffa9d8 X-MS-Exchange-CrossTenant-Id: f34e5979-57d9-4aaa-ad4d-b122a662184d X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=f34e5979-57d9-4aaa-ad4d-b122a662184d; Ip=[63.35.35.123]; Helo=[64aa7808-outbound-1.mta.getcheckrecipient.com] X-MS-Exchange-CrossTenant-AuthSource: DBAEUR03FT012.eop-EUR03.prod.protection.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: VI1PR0802MB2222 X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org > > > > > > > > diff --git a/lib/eal/common/rte_service.c > > > > b/lib/eal/common/rte_service.c index ef31b1f63c..f045e74ef3 100644 > > > > --- a/lib/eal/common/rte_service.c > > > > +++ b/lib/eal/common/rte_service.c > > > > @@ -363,9 +363,15 @@ service_runner_do_callback(struct > > > > rte_service_spec_impl *s, > > > > uint64_t start =3D rte_rdtsc(); > > > > s->spec.callback(userdata); > > > > uint64_t end =3D rte_rdtsc(); > > > > - s->cycles_spent +=3D end - start; > > > > + uint64_t cycles =3D end - start; > > > > cs->calls_per_service[service_idx]++; > > > > - s->calls++; > > > > + if (service_mt_safe(s)) { > > > > + __atomic_fetch_add(&s->cycles_spent, cycles, > > > > __ATOMIC_RELAXED); > > > > + __atomic_fetch_add(&s->calls, 1, > > > > __ATOMIC_RELAXED); > > > > + } else { > > > > + s->cycles_spent +=3D cycles; > > > > + s->calls++; > > > This is still a problem from a reader perspective. It is possible > > > that the writes could be split while a reader is reading the stats. T= hese > need to be atomic adds. > > > > Thanks for pointing out; I do "think" in x86 in terms of load/store > > tearing; and on x86 naturally aligned load/stores will not tear. Apolog= ies for > missing the ARM angle here. Arm architecture has similar things as well. I believe compiler does not pr= ovide any guarantees that it will only generate non-tearing instructions. R= efer to a discussion in the past [1] [2] where it was thought that the comp= iler would generate a non-tearing store (this is a slightly different scena= rio). [1] http://inbox.dpdk.org/dev/d5d563ab-0411-3faf-39ec-4994f2bc9f6f@intel.co= m/ [2] Refer to commit '316095eb41ed22da808b5826404c398917c83c89' > > > > I'm not sure how to best encode the difference between tearing & "locke= d > instructions" > > to make things multi-writer safe. But they're not the same thing, and > > I'd prefer not pay the penalty for LOCK instructions (multi-writer) onl= y to > satisfy the non-tearing requirements. > > > > Is there an rte_atomic-* type that is guaranteed as non-tearing? Nothing that I know of. > > > > In that case, changing the type of the calls/cycles_spent variables to = such a > type to ensure "non-tearing" > > single-reader, single-writer behaviour is enough, instead of forcing > __atomic_fetch_add() everywhere? >=20 > Regular read, increment and then atomic store should work without locks > where alignment is correct on most architectures, and doing the store > atomically should prevent any tearing. Agree, we could do this, will provide more flexibility for the micro-archit= ecture to work with. Good to understand the perf benefits vs complexity and= the branch cost.