From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from dpdk.org (dpdk.org [92.243.14.124]) by inbox.dpdk.org (Postfix) with ESMTP id 3C6E3A0526; Tue, 21 Jul 2020 21:43:19 +0200 (CEST) Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id DFCE01C068; Tue, 21 Jul 2020 21:43:17 +0200 (CEST) Received: from EUR05-DB8-obe.outbound.protection.outlook.com (mail-db8eur05on2048.outbound.protection.outlook.com [40.107.20.48]) by dpdk.org (Postfix) with ESMTP id A911C1BFCD for ; Tue, 21 Jul 2020 21:43:16 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=armh.onmicrosoft.com; s=selector2-armh-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=hOoAt5UggpUD3R9mqJlMhEGq2ENhcDY8NeVbkbmPmbQ=; b=dFfAx+KaRd9hVzjObEVJv9sHjiH3EaF/nbuG3WidLmLaVtm6oKhV5RSu32oN2OIjtjeJn8ZXesZhgbnQNKMCj6dAlLy/n6eEEv4VWGv94OesOR2Zu0TCcH5dvpnBtGCG/oKZLSX21pWfxbf5XajjHf4wrtpeUW6nkn+IBh9G8ao= Received: from AM5PR0402CA0011.eurprd04.prod.outlook.com (2603:10a6:203:90::21) by AM0PR08MB5329.eurprd08.prod.outlook.com (2603:10a6:208:185::19) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.3195.18; Tue, 21 Jul 2020 19:43:14 +0000 Received: from VE1EUR03FT051.eop-EUR03.prod.protection.outlook.com (2603:10a6:203:90:cafe::3d) by AM5PR0402CA0011.outlook.office365.com (2603:10a6:203:90::21) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.3216.21 via Frontend Transport; Tue, 21 Jul 2020 19:43:14 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 63.35.35.123) smtp.mailfrom=arm.com; dpdk.org; dkim=pass (signature was verified) header.d=armh.onmicrosoft.com;dpdk.org; dmarc=bestguesspass action=none header.from=arm.com; Received-SPF: Pass (protection.outlook.com: domain of arm.com designates 63.35.35.123 as permitted sender) receiver=protection.outlook.com; client-ip=63.35.35.123; helo=64aa7808-outbound-1.mta.getcheckrecipient.com; Received: from 64aa7808-outbound-1.mta.getcheckrecipient.com (63.35.35.123) by VE1EUR03FT051.mail.protection.outlook.com (10.152.19.75) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.3195.18 via Frontend Transport; Tue, 21 Jul 2020 19:43:14 +0000 Received: ("Tessian outbound c4059ed8d7bf:v62"); Tue, 21 Jul 2020 19:43:13 +0000 X-CR-MTA-TID: 64aa7808 Received: from 138a2db29627.3 by 64aa7808-outbound-1.mta.getcheckrecipient.com id 3366F25F-0500-46DB-9B71-D47289F92A19.1; Tue, 21 Jul 2020 19:43:08 +0000 Received: from EUR04-VI1-obe.outbound.protection.outlook.com by 64aa7808-outbound-1.mta.getcheckrecipient.com with ESMTPS id 138a2db29627.3 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384); Tue, 21 Jul 2020 19:43:08 +0000 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=Z/R3Xosf+LYNidUie93EN+hMbZw5wogIzTuQWXV34yytPXKzfiap9QekyJpDHEuDngPQK9oCSso8QoliJLqOXl/rMoygbqo4APjo2UWI2FniH/Mo8V/3d70x1apcQA/kH93Q5mZ0tL3SMgEizQ3TH61U836+9kdy+YRAPf9DVVr+Op53RQjPC4ICuH3HmS+z+z9Q574lANirQqIvi4bbh4lFCUmUZ71RAGLUF4ahesJGx2cdEqyuLz6wDnfbIiTxUNtgd8G0ObEnsboocs0uQ0OcjDL7O6zH7rGFMdZOC6dSj4y4OzdkH/01tFXa975NQ614UJQfCKoce7Lxa4BcYw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=hOoAt5UggpUD3R9mqJlMhEGq2ENhcDY8NeVbkbmPmbQ=; b=lixztk3K2aEGiFChaXM4ndHTqqZwDQ7EQhc69Sh4wkalyb7xs40VkygC8B3IzMNG+BPvym9kR8NAzAmFruEbRWDa45tVPjJNPioDofOzsdo3a9RGJqRjWapr9ae2mvOS3aQUqH9XzmY0TO/prpPhQKtyJM0SsTNQvg/JU+5JwZaCgGRzS7GSotn2TFruECNtCccdJ7DIoK1kwEpFWk68kTZskP71fVod3IUKV6rvrILTLFSblpgrtSolt1kduAxTwdDhjXW+SFir3h11H4/LhavsF8hv/xcPX/4nWZaN/FlJEwLNtpzbB/v3qCLYKKQOo3Ka+4QWE4VEQiPEB1nLeQ== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=arm.com; dmarc=pass action=none header.from=arm.com; dkim=pass header.d=arm.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=armh.onmicrosoft.com; s=selector2-armh-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=hOoAt5UggpUD3R9mqJlMhEGq2ENhcDY8NeVbkbmPmbQ=; b=dFfAx+KaRd9hVzjObEVJv9sHjiH3EaF/nbuG3WidLmLaVtm6oKhV5RSu32oN2OIjtjeJn8ZXesZhgbnQNKMCj6dAlLy/n6eEEv4VWGv94OesOR2Zu0TCcH5dvpnBtGCG/oKZLSX21pWfxbf5XajjHf4wrtpeUW6nkn+IBh9G8ao= Received: from DB6PR0802MB2216.eurprd08.prod.outlook.com (2603:10a6:4:85::9) by DB8PR08MB5196.eurprd08.prod.outlook.com (2603:10a6:10:ee::31) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.3195.23; Tue, 21 Jul 2020 19:43:05 +0000 Received: from DB6PR0802MB2216.eurprd08.prod.outlook.com ([fe80::9d1d:207b:e89d:199d]) by DB6PR0802MB2216.eurprd08.prod.outlook.com ([fe80::9d1d:207b:e89d:199d%10]) with mapi id 15.20.3195.026; Tue, 21 Jul 2020 19:43:05 +0000 From: Honnappa Nagarahalli To: Harry van Haaren , "dev@dpdk.org" CC: "david.marchand@redhat.com" , "igor.romanov@oktetlabs.ru" , "ferruh.yigit@intel.com" , nd , "aconole@redhat.com" , "l.wojciechow@partner.samsung.com" , Honnappa Nagarahalli , Phil Yang , nd Thread-Topic: [PATCH v2 1/2] service: add API to retrieve service core active Thread-Index: AQHWXqNVej80cd/qgEGtgKmAQ2W1m6kSZL2A Date: Tue, 21 Jul 2020 19:43:05 +0000 Message-ID: References: <20200720120938.34660-1-harry.van.haaren@intel.com> <20200720143829.46280-1-harry.van.haaren@intel.com> In-Reply-To: <20200720143829.46280-1-harry.van.haaren@intel.com> Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-ts-tracking-id: 692a78f4-da33-466e-a8a7-9a058a81ed83.0 x-checkrecipientchecked: true Authentication-Results-Original: intel.com; dkim=none (message not signed) header.d=none;intel.com; dmarc=none action=none header.from=arm.com; x-originating-ip: [107.77.221.153] x-ms-publictraffictype: Email X-MS-Office365-Filtering-HT: Tenant X-MS-Office365-Filtering-Correlation-Id: ec8f3904-fa2f-480e-5177-08d82dae4ec9 x-ms-traffictypediagnostic: DB8PR08MB5196:|AM0PR08MB5329: x-ms-exchange-transport-forked: True X-Microsoft-Antispam-PRVS: x-checkrecipientrouted: true nodisclaimer: true x-ms-oob-tlc-oobclassifiers: OLM:10000;OLM:10000; X-MS-Exchange-SenderADCheck: 1 X-Microsoft-Antispam-Untrusted: BCL:0; X-Microsoft-Antispam-Message-Info-Original: YNGlLOSnxK5YI7mwrfOkoyzy6fDrwOzsAaje3fS8zozgcwCq42eWLKjde5iPtIhXbERqGJ1nEEUNrsC5wwX87s7xc+UmboIvKSzlD2hxJIwlz4Z9xp2KUZZ83kUjQi6KnF+Tuo+Xx6lfl2fVwZP2+reotaZde1SuDysScPhQ7NbSdZqY4u+ggtFAMVYnIOtZEIKDQUxtqmoyltGjKbJANtJa2aHtxQnsAYdHQPL/B0CRIqFsVlUWdn+T3uXtPaOohXHOe9ioE+WMg+Jw1zVSGs/Ck7wpiNhlQInERYoccgIUzoSqFJ83sw05OwfIqA1IDR3PzZDtrYQga1aNKlL5+iT4myo12ioQ1B8J5ZQpdnsiMGk0nn1cPB0OB0xE0pFK X-Forefront-Antispam-Report-Untrusted: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:DB6PR0802MB2216.eurprd08.prod.outlook.com; PTR:; CAT:NONE; SFTY:; SFS:(4636009)(346002)(136003)(39860400002)(376002)(366004)(396003)(55016002)(8936002)(4326008)(9686003)(478600001)(8676002)(2906002)(83380400001)(5660300002)(186003)(33656002)(86362001)(66946007)(71200400001)(66556008)(110136005)(6506007)(7696005)(316002)(54906003)(52536014)(64756008)(66446008)(66476007)(26005)(76116006)(87944003); DIR:OUT; SFP:1101; x-ms-exchange-antispam-messagedata: ol7qSSdU/Nj4Qmm396FFNkEcHEZmVvy71Nhvk07EfPohhRb/ow0F/5FqzkEfX++TWHajZucw22lM7rwdpErPa8DGAHTAzcwIgbLLaP7n2Q+dWA/Ja4Fl+nhmV+MhtptRJG3uOkAi3FSfqzP4OvOm29KS7DJO/+bYVHojSKLzjVI10R46l8aP/zTURiW5vGFPv2mc1M3R8p6lBe2OLZ+QELdqktUWV+GUVhesKArkQC+15YFZ/7xq+JBkHj3W/5HtcN8AHt9TZtSsvyEpXZhg8HFg+6ffdaqwZ8RteXVBAtlQtFOeIjyrrgLOsiq0rb1XRsSJ9YacN4ZaBLc/TD+ujDH2ZyfZes1ngeQlf1e0q3ds3tFeTAxfMVzcSAbYwhCrfxxUKM35oAqU9Hsnfn+GjJZjW9uiSwXevFrT1EhHJuf/J5XCbFQI7qPM5vFPNYl21UCoSotK66TH25nyrlDlu8J9Zz0x/IcVRAHLZaoLIro/H73ZONNcLhewGFzkTO9x Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 X-MS-Exchange-Transport-CrossTenantHeadersStamped: DB8PR08MB5196 Original-Authentication-Results: intel.com; dkim=none (message not signed) header.d=none;intel.com; dmarc=none action=none header.from=arm.com; X-EOPAttributedMessage: 0 X-MS-Exchange-Transport-CrossTenantHeadersStripped: VE1EUR03FT051.eop-EUR03.prod.protection.outlook.com X-Forefront-Antispam-Report: CIP:63.35.35.123; CTRY:IE; LANG:en; SCL:1; SRV:; IPV:CAL; SFV:NSPM; H:64aa7808-outbound-1.mta.getcheckrecipient.com; PTR:ec2-63-35-35-123.eu-west-1.compute.amazonaws.com; CAT:NONE; SFTY:; SFS:(4636009)(136003)(376002)(396003)(39860400002)(346002)(46966005)(52536014)(36906005)(5660300002)(82310400002)(2906002)(9686003)(33656002)(86362001)(55016002)(8936002)(356005)(81166007)(316002)(336012)(6506007)(82740400003)(70586007)(478600001)(7696005)(54906003)(26005)(186003)(110136005)(8676002)(70206006)(47076004)(4326008)(83380400001)(87944003); DIR:OUT; SFP:1101; X-MS-Office365-Filtering-Correlation-Id-Prvs: d20658aa-7665-4f80-8846-08d82dae4999 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: iEMbQiC8ryrjumwwQd7JYhrCgdY/uhei3/zbtM6MYz+RHuUyRqboHaITVjUqAETrgI76q1nGFNNBPksGY0XovmDJMVa8L5Z3EXx+R1ZwA1me+Hx5c8pMvDYFdt6lw6aSZ0T7k839BYCW5GDLEkvgUKF/LuNmezXjwRGHn8neZievN6l3/u12CChF9orYml68aiCuEeQK5nT74WnBUOtvNOvyXqivuPgQ8lYpebYt37M+PZIe0d42foByR/zAltVOiDqR4DKlAs9mpCVeNYyf4Q/UUnpJmrr0gCueBtV+cmOjg4XNTlWnwm+dZYPIf7QX203ew8Lxzsl0aJpdzr3jiSu6nb6s2uMSfak/PmetccJwSOZInZ40tQ/u3Boz4BHJIF4YuIZzTTaA0DgZdH6Zc8Xb/2N+a4jOnZPZ2XZ8CDw= X-OriginatorOrg: arm.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 21 Jul 2020 19:43:14.1138 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: ec8f3904-fa2f-480e-5177-08d82dae4ec9 X-MS-Exchange-CrossTenant-Id: f34e5979-57d9-4aaa-ad4d-b122a662184d X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=f34e5979-57d9-4aaa-ad4d-b122a662184d; Ip=[63.35.35.123]; Helo=[64aa7808-outbound-1.mta.getcheckrecipient.com] X-MS-Exchange-CrossTenant-AuthSource: VE1EUR03FT051.eop-EUR03.prod.protection.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: AM0PR08MB5329 Subject: Re: [dpdk-dev] [PATCH v2 1/2] service: add API to retrieve service core active X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" > Subject: [PATCH v2 1/2] service: add API to retrieve service core active >=20 > This commit adds a new experimental API which allows the user to retrieve > the active state of an lcore. Knowing when the service lcore is completed= its > polling loop can be useful to applications to avoid race conditions when = e.g. > finalizing statistics. >=20 > The service thread itself now has a variable to indicate if its thread is= active. > When zero the service thread has completed its service, and has returned > from the service_runner_func() function. >=20 > Suggested-by: Lukasz Wojciechowski > Signed-off-by: Harry van Haaren >=20 > --- >=20 > Thanks for feedback Lukasz, please have a look and see if this was what y= ou > were expecting? >=20 > Honnappa/Phil, are the __atomic_load/store's correct? >=20 > --- > lib/librte_eal/common/rte_service.c | 14 ++++++++++++++ > lib/librte_eal/include/rte_service.h | 9 +++++++++ > lib/librte_eal/rte_eal_version.map | 1 + > 3 files changed, 24 insertions(+) >=20 > diff --git a/lib/librte_eal/common/rte_service.c > b/lib/librte_eal/common/rte_service.c > index 6a0e0ff65..4c276b006 100644 > --- a/lib/librte_eal/common/rte_service.c > +++ b/lib/librte_eal/common/rte_service.c > @@ -65,6 +65,7 @@ struct core_state { > /* map of services IDs are run on this core */ > uint64_t service_mask; > uint8_t runstate; /* running or stopped */ > + uint8_t thread_active; /* indicates when thread is in service_run() */ Nit, if possible would be good to describe the difference between 'runstate= ' and 'thread_active'. > uint8_t is_service_core; /* set if core is currently a service core */ > uint8_t service_active_on_lcore[RTE_SERVICE_NUM_MAX]; > uint64_t loops; > @@ -457,6 +458,8 @@ service_runner_func(void *arg) > const int lcore =3D rte_lcore_id(); > struct core_state *cs =3D &lcore_states[lcore]; >=20 > + __atomic_store_n(&cs->thread_active, 1, __ATOMIC_RELAXED); Essentially, we have to ensure that all the memory operations are contained= within both stores (this one and the one below) to 'thread_active'. We should use __ATOMIC_SEQ_CST, which will avoid any memory operations from= getting hoisted above this line. Performance should not be an issue as it will get executed only when the se= rvice core is started. It would be good to add comment reasoning the memory order used. Also, what happens if the user calls 'rte_service_lcore_active' just before= the above statement is executed? It might not be a valid use case, but it = is good to document the race conditions and correct sequence of API calls. > + > /* runstate act as the guard variable. Use load-acquire > * memory order here to synchronize with store-release > * in runstate update functions. > @@ -475,9 +478,20 @@ service_runner_func(void *arg) > cs->loops++; > } >=20 > + __atomic_store_n(&cs->thread_active, 0, __ATOMIC_RELAXED); __ATOMIC_RELEASE would be enough. But, __ATOMIC_SEQ_CST should not cause an= y performance issues. > return 0; > } >=20 > +int32_t > +rte_service_lcore_active(uint32_t lcore) { > + if (lcore >=3D RTE_MAX_LCORE || !lcore_states[lcore].is_service_core) > + return -EINVAL; > + > + return __atomic_load_n(&lcore_states[lcore].thread_active, > + __ATOMIC_RELAXED); > +} > + > int32_t > rte_service_lcore_count(void) > { > diff --git a/lib/librte_eal/include/rte_service.h > b/lib/librte_eal/include/rte_service.h > index e2d0a6dd3..f7134b5c0 100644 > --- a/lib/librte_eal/include/rte_service.h > +++ b/lib/librte_eal/include/rte_service.h > @@ -261,6 +261,15 @@ int32_t rte_service_lcore_start(uint32_t lcore_id); > */ > int32_t rte_service_lcore_stop(uint32_t lcore_id); >=20 > +/** > + * Reports if a service lcore is currently running. > + * @retval 0 Service thread is not active, and has been returned to EAL. > + * @retval 1 Service thread is in the service core polling loop. > + * @retval -EINVAL Invalid *lcore_id* provided. > + */ > +__rte_experimental > +int32_t rte_service_lcore_active(uint32_t lcore_id); Would 'rte_service_lcore_may_be_active' better? It would be inline with 'rt= e_service_may_be_active'? I think we need additional documentation for 'rte_service_lcore_stop' to in= dicate that the caller should not assume that the service thread on the lco= re has stopped running and should call the above API to check. > + > /** > * Adds lcore to the list of service cores. > * > diff --git a/lib/librte_eal/rte_eal_version.map > b/lib/librte_eal/rte_eal_version.map > index bf0c17c23..d53d5d5b9 100644 > --- a/lib/librte_eal/rte_eal_version.map > +++ b/lib/librte_eal/rte_eal_version.map > @@ -401,6 +401,7 @@ EXPERIMENTAL { > rte_lcore_dump; > rte_lcore_iterate; > rte_mp_disable; > + rte_service_lcore_active; > rte_thread_register; > rte_thread_unregister; > }; > -- > 2.17.1