From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 825EA42A02; Wed, 26 Apr 2023 13:27:19 +0200 (CEST) Received: from mails.dpdk.org (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 1AEB6410FA; Wed, 26 Apr 2023 13:27:19 +0200 (CEST) Received: from NAM10-MW2-obe.outbound.protection.outlook.com (mail-mw2nam10on2080.outbound.protection.outlook.com [40.107.94.80]) by mails.dpdk.org (Postfix) with ESMTP id 7091D40DDA for ; Wed, 26 Apr 2023 13:27:17 +0200 (CEST) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=ntP8f6sI0hGJlQCkm2TnY0xmPCGm/S3aHwqn+WYEdzKRVatArsfWjjAMGJIPl5MZ1Emn8wYArCZdCBpOo5LTAux2LGJ9ANeq//s9hCYW0ZULZY259lHr86V3gyWsltMfwB/PUR6szkcXV+xIqiw2rKgIICRE2sWrNfx4lAxd799MUfIieTUsFvccCDa09NuCggH16AJLQB6vYfoCKOzqefZswn0FRJ1H2qrpI6iBX2e72KP/FLfHq/2Qno8jGqIp19Kx3ea4iLehTIwS5W7GYuXhlRQP7//2l76rA1iJZb0O5n7d2OqRdCTVWvoHrnoMeKfn/hY8R8A9xx1r47GLMA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=EGSHxEnLDEmOpVkcuMfbWnCyRi98DpFBF2nSawtyt3Y=; b=BzlO05HHdR0LUZiDS9JxSvJ2ZKgG9iPVp1Mh4WokVULWOEKcprlUKnXYyppLayY1glgYX8+5VMR4R4wQJsatojytGhXFA9xwhwtfDmyRyBoIGhbCt1REk2BoAK7Fb4TCM7KxDANF6fzfncVJbn1KxkiSlhPAvhnIQmM3MY25owMeEKVpJJmwKFfoKbEfxACRlADZI11QYoq+R3AXYq5PWPzXJP2RbJaaVHjlfdZSXKy3trQo079s81olml71XbNDiVdYXoULwhn8bx0W214r8L8j+co5Zo4+xAQEcbiecuNYGo+TZUO52SdqB/FL2UapuAmku+882ws4Ga9CYKbUWg== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=amd.com; dmarc=pass action=none header.from=amd.com; dkim=pass header.d=amd.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amd.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=EGSHxEnLDEmOpVkcuMfbWnCyRi98DpFBF2nSawtyt3Y=; b=sNj92VSlmtwMj/xLDr4FaPvuq0fgA5LXdkFmbrSx/7cOdTPVU4Qnrrmnc3tjXLt0cVpwBNRk3i6zfXNFN7qP0VXX/copw1FAUwNHThhdlGBJ1GqL5AqlTPjymqH9j+VF4aS8J/8uUazAidgJsPdt1hXZbCd3evXWyM7YyXmHivA= Authentication-Results: dkim=none (message not signed) header.d=none;dmarc=none action=none header.from=amd.com; Received: from MN2PR12MB4301.namprd12.prod.outlook.com (2603:10b6:208:1d4::22) by PH0PR12MB7469.namprd12.prod.outlook.com (2603:10b6:510:1e9::10) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.6340.21; Wed, 26 Apr 2023 11:27:13 +0000 Received: from MN2PR12MB4301.namprd12.prod.outlook.com ([fe80::6f2d:9990:c8b5:714a]) by MN2PR12MB4301.namprd12.prod.outlook.com ([fe80::6f2d:9990:c8b5:714a%6]) with mapi id 15.20.6340.021; Wed, 26 Apr 2023 11:27:13 +0000 Message-ID: <5875f8cf-8a37-2e81-de38-f88d66e0662f@amd.com> Date: Wed, 26 Apr 2023 12:27:06 +0100 User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:102.0) Gecko/20100101 Thunderbird/102.10.1 Content-Language: en-US To: fengchengwen , Stephen Hemminger , thomas@monjalon.net Cc: dev@dpdk.org References: <20230424130208.9517-1-fengchengwen@huawei.com> <20230424090836.0550e14b@hermes.local> From: Ferruh Yigit Subject: Re: [RFC 0/3] introduce coroutine library In-Reply-To: Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 7bit X-ClientProxiedBy: LNXP265CA0020.GBRP265.PROD.OUTLOOK.COM (2603:10a6:600:5e::32) To MN2PR12MB4301.namprd12.prod.outlook.com (2603:10b6:208:1d4::22) MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: MN2PR12MB4301:EE_|PH0PR12MB7469:EE_ X-MS-Office365-Filtering-Correlation-Id: d06199bc-4929-44fc-cc89-08db46492e4e X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: w6M9BWkFBe0Vp/PhwiOgj5z0laF178Wy0OIEwBzsCc6usQlsCGy0ZOp1zub3mX5lnf6JdSAu7V3frf7NnORkZXNmwjgSzRaxFNAj6ajnQamsqKP/NLhUTMz7N3zMI+3Yv0YziZKjP5yTKtBz5TghQWFRWz/7Y36j4oftx7926RQwfbur3RGL5ndGQRalypO3X4eKo/ZCqSQzPpR9AyKseOV3lUt/vjEbIevPQUhRg0M2HHvN8PainyHslGVqPhl3ICRhyMinpmUthSJy13ESLjVdQrom7sbJ/cOdpADLl1SPcaTG4yqtFkkt2ZJzIMRz9In795FNeNTTOxU4r7gYeNxIHFWlhwNUIpyURU7ri3UYgUkYJsT6wuywNh0iePv5/Z6aJokUq4Lu644SG/buvxQHR4Zuo+1sw+5YYJvy4cWNJ1yWqdhj92pOHUCFTw+SZOP69NkiPyFJ+N9SLaev+4XMERqxnw2fraakkAUE5nQr+rydBu2o0NOYYBWJdUVnFGv36bmFPA8Xt6oaTylo1QgNFEwuU60aRS024mAel4kBkBXVPFOmV33L+wruI8oZsdEcab+pQaMLxL6wgCGXcxf8/6U6PtFDwC4le0zmxMTE7H11he2PQlJkrpo2PtYbEtJiiOfIMxHoKnGHqryKmA== X-Forefront-Antispam-Report: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:MN2PR12MB4301.namprd12.prod.outlook.com; PTR:; CAT:NONE; SFS:(13230028)(4636009)(346002)(136003)(39860400002)(396003)(376002)(366004)(451199021)(53546011)(6506007)(31696002)(316002)(44832011)(86362001)(66476007)(66946007)(66556008)(2906002)(8676002)(38100700002)(8936002)(41300700001)(5660300002)(186003)(110136005)(6512007)(26005)(83380400001)(4326008)(36756003)(2616005)(31686004)(6666004)(6486002)(478600001)(66899021)(45980500001)(43740500002); DIR:OUT; SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: =?utf-8?B?VEFGQlRPT3hhSmxMOTJwNFJnZlNucXRyRytON2dld0pTRE5VdzNmejVwd3U1?= =?utf-8?B?bkNHRTVxbkZrNWhod2lmbFZoc1pnUVljZ0xTVnoySVpxYXdIVEo0OE5IVUNS?= =?utf-8?B?cTRSOWhwWTBKaVlrSkpYWUxlVnlZS1VkM2hUYUZ2ZDBWbEo0dGZhOEJodkMv?= =?utf-8?B?V0JIWmpyL0xEN1JSVXdic25QQUt0Ui9nc3AyWWFnWlNRL3R2RlpUbXJIdTZM?= =?utf-8?B?RWQ4Ulh3VjA1UW9lOXBqMFJpR3hlK0FtU2k0RHdYSWowRVEyMmZML0dDczdL?= =?utf-8?B?aTVQanRBdEl5K1c3UzlXTjRxcGdIbHZKZmw5d1k4MUg5KzljQ2ZnRUx0dnU4?= =?utf-8?B?RkQzVWhlWnEvc0pWTitEZTBRblE0K1hCc1JmMUVpYUJCazgxbzhXS3gzalhM?= =?utf-8?B?SGxTK3JlWmN6TU91c25HVjhZRTBFTnVFOXIyWktKWnMvSFJpMVIzUWVSeTVx?= =?utf-8?B?TTRJL2FBbnd0MzNtNkVvY241YzJVU1A3RzZhL1prWWJJOVZEMTlIN1Q0Wkkr?= =?utf-8?B?U0tDREZzQ2wwS0o3UEtDT3VBRlhDWGtobUhqcnpQM2tZT0tjZW9UOXdGZXp0?= =?utf-8?B?SlZzZ3ZCR1JaQzV0TXNGcXNlQkpXUTJtTGVzdjltVkM4NnJKVjNhNEFaSERC?= =?utf-8?B?dUh3bTdHYWQreXd3bHU2YlZ2bUI1N0JJa21hTHBKMmJGdE5LeUN1elNVVUlK?= =?utf-8?B?QWRaMWNPZ2lDdnlvRDREZENKMUFMWDcwRVcxdVRPdWt5bE83VVJjOVF1Rm9I?= =?utf-8?B?U1hkZFgvU3lIaGlXMGo3bEk3Rko2NnNabXl2eUdVb2xCcEErdW5OTkZSK2Yr?= =?utf-8?B?MldjL3JWaVFSVDk5NFFZUEQ0cXg0Uks2TjZRWUNZbDU4MGc4ZXhVdERkcHBw?= =?utf-8?B?VkZkWUYzWWVCZzk5dklMS2owWnlKTU90WFVIdHJLb0ppejYyRkd1Y2V3WFl2?= =?utf-8?B?ZTI1RjBVUkhTVndBeGpKRW9WSWFpU2h3Q1ZLWVE0M241cHZkQlJZNHFaaU1o?= =?utf-8?B?d3pxMXB5N2FtYngvdmpsdFVUZ251c0V2cU5Mb1JHUUlvMFE0YWp6NGVjOFkz?= =?utf-8?B?UnY0M244TVd6am5XZVRBM1ZyWlpVaTlZYVRqTjBaMUgyY3l1SXRpWElWOHBx?= =?utf-8?B?NGxTNk9mOHV2bmJYOEVqZHo0T2s0Qlp6MXYreFZjcWowTk9qeDhVaWw3ZGVa?= =?utf-8?B?bEpqdEMwN3VOVU9IemwyYmk2dU8wWGcrU2M4dlRiZ3l4c0VnODZjbDZUdXdQ?= =?utf-8?B?T0g0UU55bTV5YnZYL1BDaDlUZnFXZjZONkZjRWtSV2dnaENIdkZBdjFtZHdX?= =?utf-8?B?SFAvNXg0MGNYNlhodUluR2hacTNLaGcvdzY5Ykh4Mmo0b2plOTlHcFA2Y1pz?= =?utf-8?B?T1dyOXhkUlpoWVU4VXEvLzFsYWRTaWNZQTBlZjlZK0x5dC9pZlZOZVpQRmtN?= =?utf-8?B?dUFzSHp5UDFEemc1bU92d0RsckdVVWhxTkNJNmwwWTU5TlJlbW8yZUhyWTgy?= =?utf-8?B?OEY5YnNzTll0NDVOK1dwQ0NVUDBGTVdaK2lnWkE2Zy9EaUdoNlBtRmkrNHhj?= =?utf-8?B?YjdCQUZCOU4zNmhoMEo0OFFBT2NyWm9OZm9LNmtqa3pCeE83aEp4VmMvNlJL?= =?utf-8?B?VkplOFhYdmIySzZjOU5JNWxZdytiS2psVGdvRGJpZFFjbnIyeVBEUjVZYWI0?= =?utf-8?B?ZWJmNFE1UDgwZTl2cm5BR3JOQVJLdXhxTlFhMktFY3RTeDJmdktaaldKT0Rk?= =?utf-8?B?U3JNWUp2UW5DcHdqdWxOWU0wOTQ4NzZOV2xDbVB4ZzlFdFRidnVyd0NGdml2?= =?utf-8?B?MXNtZUpKTUpxYWduRG84QVR5WUYxT3RUVmpUeit2Y0xFcHpZZUVDZ3QvdEo4?= =?utf-8?B?eHRMRytSZEFpZ01lOUl3dWNyOVRDYjluVkk2UEltM09VQXhPNzE5Z0RLbStQ?= =?utf-8?B?SE5IOWlicGtSU3A4UjBDRWRGUzZoek1WOFlGTm40OThYVndxLzNaVm5mU2s5?= =?utf-8?B?ZU5TMzVwQkpsQTVCb2o1Tnhva2NwVjhOMHJZUS8zUTdHUDdVSlRiS2JsdVMy?= =?utf-8?B?SlZGaXhockF1U2NQWmhRa3Z2ajdiN0pDL045cS9OZnJQY1EyeDZ6QVg4SWdE?= =?utf-8?Q?bScutJQjnf+0wP9mELpsOeH1R?= X-OriginatorOrg: amd.com X-MS-Exchange-CrossTenant-Network-Message-Id: d06199bc-4929-44fc-cc89-08db46492e4e X-MS-Exchange-CrossTenant-AuthSource: MN2PR12MB4301.namprd12.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 26 Apr 2023 11:27:12.9127 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: P0iRLTWB66RKC7OW+2stJxSNLsFnnZ5HO/fJ1gFqPmpvKNKBNy+3T4pXFifydlvd X-MS-Exchange-Transport-CrossTenantHeadersStamped: PH0PR12MB7469 X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org On 4/25/2023 3:11 AM, fengchengwen wrote: > On 2023/4/25 0:08, Stephen Hemminger wrote: >> On Mon, 24 Apr 2023 13:02:05 +0000 >> Chengwen Feng wrote: >> >>> This patchset introduces the coroutine library which will help refactor >>> the hns3 PMD's reset process. >>> >>> The hns3 single function reset process consists of the following steps: >>> 1.stop_service(); >>> 2.prepare_reset(); >>> 3.delay(100ms); >>> 4.notify_hw(); >>> 5.wait_hw_reset_done(); // multiple sleep waits are involved. >>> 6.reinit(); >>> 7.restore_conf(); >>> >>> If the DPDK process take over multiple hns3 functions (e.g. 100), >>> it's impractical to reset and restore functions in sequence: >>> 1.proc_func(001); // will completed in 100+ms range. >>> 2.proc_func(002); // will completed in 100~200+ms range. >>> ... >>> x.proc_func(100); // will completed in 9900~10000+ms range. >>> The later functions will process fail because it's too late to deal with. >>> >>> One solution is that create a reset thread for each function, and it >>> will lead to large number of threads if the DPDK process take over >>> multiple hns3 functions. >>> >>> So the current hns3 driver uses asynchronous mechanism, for examples, it >>> use rte_eal_alarm_set() when process delay(100ms), it splits a serial >>> process into multiple asynchronous processes, and the code is complex >>> and difficult to understand. >>> >>> The coroutine is a good mechanism to provide programmers with the >>> simplicity of keeping serial processes within a limited number of >>> threads. >>> >>> This patchset use to build the coroutine framework, and it >>> just provides a demo. More APIs maybe added in the future. >>> >>> In addition, we would like to ask the community whether it it possible >>> to accept the library. If not, whether it is allowed to provide the >>> library in hns3 PMD. >>> >>> Chengwen Feng (3): >>> lib/coroutine: add coroutine library >>> examples/coroutine: support coroutine examples >>> net/hns3: refactor reset process with coroutine >> >> Interesting, but the DPDK really is not the right place for this. >> Also, why so much sleeping. Can't this device be handled with an event based >> model. Plus any complexity like this introduces more bugs into already fragile >> interaction of DPDK userspace applications and threads. > > A event base model will function as: > event-handler() { > for (...) { > event = get_next_event(); > proc_event(); > } > } > The root cause is that the proc_event() take too many time, and it will lead to other > function can't be processed timely. > > For which proc_event() may wait a lot of time, the coroutine could also used to optimize > it. > >> >> Not only that, coroutines add to the pre-existing problems with locking. >> If coroutine 1 acquires a lock, the coroutine 2 will deadlock itself. >> And someone will spend days figuring that out. And the existing analyzer >> tools will not know about the magic coroutine library. > > Analyzer tools like lock annotations maybe a problem. > > Locks in DPDK APIs are mostly no-blocking. We can add some restrictions(by reviewer), such > as once holding a lock, you can't invoke rte_co_yield() or rte_co_delay() API. > > > In addition, any technology has two sides, the greatest advantage of coroutine I think is > removes a large number of callbacks in asychronous programming. And also high-level languages > generally provide coroutines (e.g. C++/Python). With the development, the analyzer tools maybe > evolved to support detect. > > > And one more, if not acceptable as public library, whether it is allowed intergration of this > library in hns3 PMD ? Our internal evaluation solution (use coroutine refactor) is feasible, > but the code needs to be upstream, hope to listen to community's comments. > Hi Chengwen, For having library in hns3 PMD, my concern is ucontext.h portability. If this breaks the build in a specific platform, a user even doesn't have an intention to use hns3 driver will be impacted and will need to debug/fix this issue. If the dependency properly managed in the meson and for unsupported platforms coroutine can be disabled, I don't see any reason to not have this in the driver (except than the reasons already mentioned to not have coroutine as a public library applies to here). But this means you will need to maintain and support reset method without coroutine, or disable driver on the platform that ucontext.h not supported, are you OK with this? And if you continue continue to have coroutine in the driver can you please group into a subfolder etc in the driver and document it (again in a subsection of the driver) to help users that found it useful and try for themselves. I put a few comments to the implementation as well. Thanks, ferruh