From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from EUR04-DB3-obe.outbound.protection.outlook.com (mail-eopbgr60065.outbound.protection.outlook.com [40.107.6.65]) by dpdk.org (Postfix) with ESMTP id 9A6002C2B for ; Tue, 11 Dec 2018 07:40:46 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=armh.onmicrosoft.com; s=selector1-arm-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=kNWU/iG40+4wxfTliUywmhIji7jHOGMOO4yIlz5uPtw=; b=KFGY7IefxdEDFt5KFZELA6fFw6WPwAREBOxX5Jwj3ABij4ddVKRB/AXVdC5dmYjoeIy+YR9mP8/HCAdrgnQ7hLHbvuAu0UuSWDyPpPoBsHAvWwFH6Y1+syP+iKeZ+TMRxkAH3xmw5B82OF0j16S9r5pIS3Valf0odeZNIOAbN4I= Received: from AM6PR08MB3672.eurprd08.prod.outlook.com (20.177.115.29) by AM6PR08MB3766.eurprd08.prod.outlook.com (20.178.88.219) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.1404.23; Tue, 11 Dec 2018 06:40:45 +0000 Received: from AM6PR08MB3672.eurprd08.prod.outlook.com ([fe80::78ab:2bf4:5476:6c3e]) by AM6PR08MB3672.eurprd08.prod.outlook.com ([fe80::78ab:2bf4:5476:6c3e%2]) with mapi id 15.20.1425.016; Tue, 11 Dec 2018 06:40:45 +0000 From: Honnappa Nagarahalli To: Stephen Hemminger CC: "Ananyev, Konstantin" , "dev@dpdk.org" , nd , Dharmik Thakkar , Malvika Gupta , "Gavin Hu (Arm Technology China)" , nd Thread-Topic: [dpdk-dev] [RFC 2/3] tqs: add thread quiescent state library Thread-Index: AQHUghQLRVNcuR2lRU2RH6S2NG2yyqVeB0ZggAUcBqCAAiq/IIANopHQgACwFACABZE3EA== Date: Tue, 11 Dec 2018 06:40:45 +0000 Message-ID: References: <20181122033055.3431-1-honnappa.nagarahalli@arm.com> <20181122033055.3431-3-honnappa.nagarahalli@arm.com> <2601191342CEEE43887BDE71AB977258010CEBBBA9@IRSMSX106.ger.corp.intel.com> <2601191342CEEE43887BDE71AB977258010CEBD5D4@IRSMSX106.ger.corp.intel.com> <20181207092936.17bf2887@xeon-e3> In-Reply-To: <20181207092936.17bf2887@xeon-e3> Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: authentication-results: spf=none (sender IP is ) smtp.mailfrom=Honnappa.Nagarahalli@arm.com; x-originating-ip: [217.140.111.135] x-ms-publictraffictype: Email x-microsoft-exchange-diagnostics: 1; AM6PR08MB3766; 6:i/hRHoIB6DI3JVAlNfPDNmZkmghPZdRqAXY9kSxYJrA/PPtLDRnUxmftASfA3lNPGVuPkionQZbGWhNqC2bLgCxa+GCjog1GyC/QeYS7eN91APRCLeBfO21nNfIlVPSZj29wqehMqBUObwmxVS0ChsKEMUdLEFkRc4UpmLcdtPqQThBWMXCDJm6zp5sqbOCa50oUfpWIBqwUO31KvCjW4gs4PYqzBitlzM+0Zk0MxxlLw2aF2EVTtUyJYQYZoB2alMsbIEUiUAVF3LC0tCY6nBoueGNxKML91hHpBWIMiWNBVjRSgqhvhR4FHETNk62p04YO39OtF/vyYYQh9WYFlwZ4seYUM4vo3AQDI7rwYqmepUIJd4vXXHaHDAxhfKT8kxaic2m4vzMHWA+nVZWR0wp6OqMkF64V/L1ZmRLbwa9skyEwcOMlcmtW8tMAfH4XkG5ClxmHh5zUCtAugLZc0Q==; 5:xhiyJFFcBvviB7ijzyVB+3QbFdOUYCdSiapEZZjZEPaGHO17lZHGu0Z5+xyk3Bd7yh2jufN5c7SCe2f2cwJKBPIDzgsoxqos7dnv1zwV74X6xcDxn+Gs2cFjRwvGFe/1QA+YtdxHCXwoVZUjG+2fshAoBr3d/Zf2VjblUmjkf4Q=; 7:wcm+tnrVzkfiV7G2Ev6cQQNYsGxJi36huSiZSAv7Tz+IrWDZDPfwFspxzuacl5Ic+zHmbQEtIkbBBpQzFo3PD/DFf8oLMtMJfJmX3XyNFM0u7fsfW4eMwmQzTgEyf9T/LbkCL/5P9PWkVduct2cArw== x-ms-exchange-antispam-srfa-diagnostics: SOS;SOR; x-ms-office365-filtering-correlation-id: 5f85c1ec-9405-4e6b-4b78-08d65f339431 x-ms-office365-filtering-ht: Tenant x-microsoft-antispam: BCL:0; PCL:0; RULEID:(2390098)(7020095)(4652040)(8989299)(5600074)(711020)(4618075)(4534185)(4627221)(201703031133081)(201702281549075)(8990200)(2017052603328)(7153060)(7193020); SRVR:AM6PR08MB3766; x-ms-traffictypediagnostic: AM6PR08MB3766: nodisclaimer: True x-microsoft-antispam-prvs: x-ms-exchange-senderadcheck: 1 x-exchange-antispam-report-cfa-test: BCL:0; PCL:0; RULEID:(8211001083)(6040522)(2401047)(5005006)(8121501046)(3231455)(999002)(944501520)(52105112)(10201501046)(3002001)(93006095)(93001095)(6055026)(148016)(149066)(150057)(6041310)(20161123562045)(201703131423095)(201702281528075)(20161123555045)(201703061421075)(201703061406153)(20161123558120)(20161123564045)(20161123560045)(201708071742011)(7699051)(76991095); SRVR:AM6PR08MB3766; BCL:0; PCL:0; RULEID:; SRVR:AM6PR08MB3766; x-forefront-prvs: 08831F51DC x-forefront-antispam-report: SFV:NSPM; SFS:(10009020)(396003)(346002)(376002)(366004)(136003)(39860400002)(199004)(189003)(54906003)(102836004)(93886005)(7736002)(486006)(6506007)(5660300001)(305945005)(74316002)(26005)(68736007)(72206003)(229853002)(6916009)(316002)(7696005)(99286004)(25786009)(14454004)(11346002)(86362001)(2906002)(476003)(446003)(76176011)(71200400001)(3846002)(71190400001)(66066001)(97736004)(6116002)(256004)(9686003)(33656002)(6246003)(6436002)(105586002)(186003)(478600001)(8936002)(55016002)(81166006)(53936002)(81156014)(8676002)(4326008)(106356001)(14444005); DIR:OUT; SFP:1101; SCL:1; SRVR:AM6PR08MB3766; H:AM6PR08MB3672.eurprd08.prod.outlook.com; FPR:; SPF:None; LANG:en; PTR:InfoNoRecords; A:1; MX:1; received-spf: None (protection.outlook.com: arm.com does not designate permitted sender hosts) x-microsoft-antispam-message-info: yRlMDtXtXXFJk8qO2QJRC227XlZB7PVbcWA4quaavTMPdaxW6kUVgZI7pnBax4J0QMtnHYjGg0jdSrC0VmErkLqz4NCo3FnvYX3Rg8BjSsoiOb87bIncM2RZv+wWicxSpen0MV73rI2eZdMe0ra9lTRr2IHnTkllNUEzUz1mO+YbAMjXCzxSUgjlhIDMY51heJcBgYPsJtqEpl3ktC5RhzeT6yT3/HPc5jVwTGUS6Ha2C818c/QtdN+34isk7vseJ8F7srnY4tk8MKqRPel/gYXSOAo860TjO3oD/zXPDqE04/3U+JLS1prerpPNW3IL spamdiagnosticoutput: 1:99 spamdiagnosticmetadata: NSPM Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 X-OriginatorOrg: arm.com X-MS-Exchange-CrossTenant-Network-Message-Id: 5f85c1ec-9405-4e6b-4b78-08d65f339431 X-MS-Exchange-CrossTenant-originalarrivaltime: 11 Dec 2018 06:40:45.2122 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Hosted X-MS-Exchange-CrossTenant-id: f34e5979-57d9-4aaa-ad4d-b122a662184d X-MS-Exchange-Transport-CrossTenantHeadersStamped: AM6PR08MB3766 Subject: Re: [dpdk-dev] [RFC 2/3] tqs: add thread quiescent state library X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 11 Dec 2018 06:40:46 -0000 >=20 > > > > > > > > > > + > > > > > > +/* Add a reader thread, running on an lcore, to the list of > > > > > > +threads > > > > > > + * reporting their quiescent state on a TQS variable. > > > > > > + */ > > > > > > +int __rte_experimental > > > > > > +rte_tqs_register_lcore(struct rte_tqs *v, unsigned int lcore_i= d) { > > > > > > + TQS_RETURN_IF_TRUE((v =3D=3D NULL || lcore_id >=3D > > > > > RTE_TQS_MAX_LCORE), > > > > > > + -EINVAL); > > > > > > > > > > It is not very good practice to make function return different > > > > > values and behave in a different way in debug/non-debug mode. > > > > > I'd say that for slow-path (functions in .c) it is always good > > > > > to check input parameters. > > > > > For fast-path (functions in .h) we sometimes skip such checking, > > > > > but debug mode can probably use RTE_ASSERT() or so. > > > > Makes sense, I will change this in the next version. > > > > > > > > > > > > > > > > > > > lcore_id >=3D RTE_TQS_MAX_LCORE > > > > > > > > > > Is this limitation really necessary? > > > > I added this limitation because currently DPDK application cannot > > > > take a mask more than 64bit wide. Otherwise, this should be as big > > > > as > > > RTE_MAX_LCORE. > > > > I see that in the case of '-lcores' option, the number of lcores > > > > can be more than the number of PEs. In this case, we still need a > > > > MAX limit (but > > > can be bigger than 64). > > > > > > > > > First it means that only lcores can use that API (at least > > > > > data-path part), second even today many machines have more than 6= 4 > cores. > > > > > I think you can easily avoid such limitation, if instead of > > > > > requiring lcore_id as input parameter, you'll just make it > > > > > return index of > > > next available entry in w[]. > > > > > Then tqs_update() can take that index as input parameter. > > > > I had thought about a similar approach based on IDs. I was > > > > concerned that ID will be one more thing to manage for the > > > > application. But, I see the > > > limitations of the current approach now. I will change it to allocati= on > based. > > > This will support even non-EAL pthreads as well. > > > > > > Yes, with such approach non-lcore threads will be able to use it also= . > > > > > I realized that rte_tqs_register_lcore/ rte_tqs_unregister_lcore need t= o be > efficient as they can be called from the worker's packet processing loop > (rte_event_dequeue_burst allows blocking. So, the worker thread needs to > call rte_tqs_unregister_lcore before calling rte_event_dequeue_burst and > rte_tqs_register_lcore before starting packet processing). Allocating the > thread ID in these functions will make them more complex. > > > > I suggest that we change the argument 'lcore_id' to 'thread_id'. The > application could use 'lcore_id' as 'thread_id' if threads are mapped to > physical cores 1:1. > > > > If the threads are not mapped 1:1 to physical cores, the threads need t= o use > a thread_id in the range of 0 - RTE_TQS_MAX_THREADS. I do not see that > DPDK has a thread_id concept. For TQS, the thread IDs are global (i.e. no= t per > TQS variable). I could provide APIs to do the thread ID allocation, but I= think > the thread ID allocation should not be part of this library. Such thread = ID > might be useful for other libraries. > > > > =20 >=20 > Thread id is problematic since Glibc doesn't want to give it out. > You have to roll your own function to do gettid(). > It is not as easy as just that. Plus what about preemption? Agree. I looked into this further. The rte_gettid function uses a system ca= ll (BSD and Linux). I am not clear on the space of the ID returned (as well= ). I do not think it is guaranteed that it will be with in a narrow range t= hat is required here. My suggestion would be to add a set of APIs that would allow for allocation= of thread IDs which are within a given range of 0 to