From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id A91CF432D6; Wed, 8 Nov 2023 12:52:08 +0100 (CET) Received: from mails.dpdk.org (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 9508940A73; Wed, 8 Nov 2023 12:52:08 +0100 (CET) Received: from mgamail.intel.com (mgamail.intel.com [192.55.52.136]) by mails.dpdk.org (Postfix) with ESMTP id 22DE7402DA for ; Wed, 8 Nov 2023 12:52:06 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1699444327; x=1730980327; h=date:from:to:cc:subject:message-id:references: content-transfer-encoding:in-reply-to:mime-version; bh=3NcNoZymWZypnBN/bZgSWnMHmqxRIr2qVE3ivyk/nUU=; b=WDH9TlfiH+HBdo7orNQQTu+vdLs885qGN+Jg0H6P6e9q916TR8NwYMOs Syz2MY0+U7KaoAufNswHdyhCvmeNA3BJ4l9trQnF4+xfUSJLeKUf47JL2 zyhApUpCGJTosfuebAOHrdqgsXyVN6rZ4QzXxPLIS4bMqDzQU5hT8GjNA hzfaf7/HWEMR4ViU25DKnZ31rjVSLpE0nmv4H5fDlhw7h2rkp4SZJHP+b ooIqTkyysAO5OdRa++Pt4gR8pbtp71SjpfPp7ECiK/0VzbHnMzlb3Cnae kL3JI77RsMCobOhZfGLUBpQtqZg1uFvz74Up+nMQyfhIN/2o+YiRdzNqZ g==; X-IronPort-AV: E=McAfee;i="6600,9927,10887"; a="369085492" X-IronPort-AV: E=Sophos;i="6.03,286,1694761200"; d="scan'208";a="369085492" Received: from fmviesa001.fm.intel.com ([10.60.135.141]) by fmsmga106.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 08 Nov 2023 03:52:06 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="6.03,286,1694761200"; d="scan'208";a="11163267" Received: from fmsmsx601.amr.corp.intel.com ([10.18.126.81]) by fmviesa001.fm.intel.com with ESMTP/TLS/AES256-GCM-SHA384; 08 Nov 2023 03:52:06 -0800 Received: from fmsmsx612.amr.corp.intel.com (10.18.126.92) by fmsmsx601.amr.corp.intel.com (10.18.126.81) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.34; Wed, 8 Nov 2023 03:52:05 -0800 Received: from fmsmsx602.amr.corp.intel.com (10.18.126.82) by fmsmsx612.amr.corp.intel.com (10.18.126.92) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.34; Wed, 8 Nov 2023 03:52:05 -0800 Received: from FMSEDG603.ED.cps.intel.com (10.1.192.133) by fmsmsx602.amr.corp.intel.com (10.18.126.82) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.34 via Frontend Transport; Wed, 8 Nov 2023 03:52:05 -0800 Received: from NAM11-CO1-obe.outbound.protection.outlook.com (104.47.56.169) by edgegateway.intel.com (192.55.55.68) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.1.2507.34; Wed, 8 Nov 2023 03:52:05 -0800 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=AVHGVMs6G9T+QTZRMwN6PUtNtZpmQdApqbOvCdKSsJ1lg/zkY8UgKSuV/51OJup4GTtMMkVyeTRc/YBID5859lbWVcVhnJZG35dj0uo3GtzFqw4fI/9ypKhOoDVbNSg8kIbifqNGiBnUuplDIPseXmXeBFaCFm2yhwf5jwIhnxwiDSkRewOKL3dYJHfCJwAy8NtQfLRIPoFPQ9W9WZTtZHsFeWzqztuczdgS38P7WNNPYx4Din0cTV7bDIIMTToZT+CW51T4JoX2W5a30Vg9k2MD+Y7bieJCo9lN165RHP4wZcQxkIHpCu9u6KCQ5n8NfkieSHE6T6EsB3YDqdrhXg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=6lC9OMeGLCpJv6hvXixMsL1NvWSZ5mj0kELs703sCnU=; b=X0cdbQ883b7uMm6jULJlrATjsbOJVYkj1b/cFJNbwcNzrj3gjtFtZn80c3UAPaxWVNfpV4BZKKw5qfiQ4g+ey+LGXK/B7SJT8OIEh9dq5dUX2VcPpYzvLZYyWNEwIq6RqRDGHscJe46FvMWuYARvFtaJIYizL/bIGGITVmjP/P9zk1cSTuXBDPbKwqotnFYZrtejC9OkH2LjMdmivf+UQrHp6QYsxZNTK7RMNET5dqp6Hz5yoVMS4is3uqJDEhMqk3bbb/qFZ9vNiT01QBuACrHksvJvnciqGKsidiTxw5nw26omw0lFzF2GBeQqDU1L1vFwvMpJ6lBdRhUu+UKA1g== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=intel.com; dmarc=pass action=none header.from=intel.com; dkim=pass header.d=intel.com; arc=none Authentication-Results: dkim=none (message not signed) header.d=none;dmarc=none action=none header.from=intel.com; Received: from DS0PR11MB7309.namprd11.prod.outlook.com (2603:10b6:8:13e::17) by SA1PR11MB7109.namprd11.prod.outlook.com (2603:10b6:806:2ba::17) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.6954.27; Wed, 8 Nov 2023 11:52:04 +0000 Received: from DS0PR11MB7309.namprd11.prod.outlook.com ([fe80::8645:d921:ce8a:12ba]) by DS0PR11MB7309.namprd11.prod.outlook.com ([fe80::8645:d921:ce8a:12ba%5]) with mapi id 15.20.6954.028; Wed, 8 Nov 2023 11:52:03 +0000 Date: Wed, 8 Nov 2023 11:51:57 +0000 From: Bruce Richardson To: Rahul Gupta CC: , , , , , , , Rahul Gupta Subject: Re: [RFC] eal: RFC to refactor rte_eal_init into sub-functions Message-ID: References: <1698949164-20287-1-git-send-email-rahulgupt@linux.microsoft.com> Content-Type: text/plain; charset="utf-8" Content-Disposition: inline Content-Transfer-Encoding: 8bit In-Reply-To: <1698949164-20287-1-git-send-email-rahulgupt@linux.microsoft.com> X-ClientProxiedBy: DUZPR01CA0325.eurprd01.prod.exchangelabs.com (2603:10a6:10:4ba::22) To DS0PR11MB7309.namprd11.prod.outlook.com (2603:10b6:8:13e::17) MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: DS0PR11MB7309:EE_|SA1PR11MB7109:EE_ X-MS-Office365-Filtering-Correlation-Id: 60eefd28-3c07-4d12-e4ed-08dbe0512008 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: arY+8duHh4lFuOCMktOusD/Dk39PFiGN7sIMu8WoJk6CUB0TTSTRnsQrDqN2aoC9veU+EuvV+I6xL+KLVxe00HwCxtTRnSFgrh+OUXgjrCkUZjS3iHA/bEgsyAQvqW6cwlRjUOyUZAR5ttI/liqN5OdF0r+uUMvKVCs6nWugfyQC8ddY++AnfWVoJT5oIInviUYpfxV5eN4KxqFTclodks3sAK7iSXg6DY5c/fG1MMilJt13+twrXL1Zwe46+U83H9iJbJ+fIctZDQanNH7H0K4t8HtjqotsKZCrGhWrNQKk3a3M0tF0NYIBchHsRkWA16sw4vv8Lq1EtSbRMj5VehjPz2UMipT9GOl2T2bq8M63LmxmAk+rZyqp0TPCp6rZu/mgv7UCXq/so3Eul2b0qfcT4wMXRjQo7+LhWR/FQolu6UoAQsUFFNPl8JLMPj7dJ0z4JHb68SQYVdzA7AYfk+rT+skrz547YU0VVHzL2+hSwrM26GTImIN9EtcxmYe7WEABnbxgksVwOgEf+b8Aq3UlVDOUfdVFTYxFCtE4wP7k3poTZBl3SZy2LNzyFTfO X-Forefront-Antispam-Report: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:DS0PR11MB7309.namprd11.prod.outlook.com; PTR:; CAT:NONE; SFS:(13230031)(39860400002)(346002)(136003)(376002)(396003)(366004)(230922051799003)(186009)(64100799003)(1800799009)(451199024)(83380400001)(6916009)(82960400001)(6666004)(478600001)(6506007)(45080400002)(6486002)(6512007)(66946007)(26005)(66476007)(38100700002)(5660300002)(66556008)(2906002)(86362001)(8676002)(4326008)(41300700001)(8936002)(44832011)(316002); DIR:OUT; SFP:1102; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: =?utf-8?B?MGpRZDRFUEFaWDVJWjllQ3YxOXhsb1FjQzNNY0dGc1dhSEtrdm8wUHlKZ3Ar?= =?utf-8?B?NWcwSkhMN2UydXk3ZS8rVFVBcGdTdTQ0bys0b1lxYnBmaklVcmpydU9QbGJ2?= =?utf-8?B?WHpUU3AwZEZNRlJDMklIZ3ZreWRWUVUvYTZoU1RxSVRnMks2YWFxMjFZcCty?= =?utf-8?B?ZTNHMU5JZng1bjJ0cHZrM29SeDN5STY2Y2x6SlZIVHZweUtRYnZ1VDJIUlRl?= =?utf-8?B?aldvUEsvR2Zrbi81ZjQvRXk3SkZ3clhhZUZWdlBSU2ptcjh5MWdDS1AralhV?= =?utf-8?B?cjVxUEJ6bkwrbmtGYUIwMDE1c0tFWDdpbWc0bTdiZmU3RVpSOVQ2VU41VFll?= =?utf-8?B?bVZ4aDJBdWtyWWNoQ3BzRUJrd0dBcVN4MUUreXI2K0FoVFZ5bVl2OXB4K3BL?= =?utf-8?B?a3gybmdqSG00T2Z3YXdudzFwY05lNFhHSVQ4eHVQTytQV2FJRklJTEE5VXh6?= =?utf-8?B?ejE5dmNsY3hlaG5VekVsV24xWFFjcjM5TkVYUW1mbUpSVVhxaVNLYUlId3Uy?= =?utf-8?B?M25UanZQK2wrcHo2YVl5NHhpYVhJRHlaOUthOXFiTS9DMmFFaElFWE85aDNs?= =?utf-8?B?SU1KWHBsL1V0ZWl4WEZBYTc0WEJzbVk3a3FqaVRhWUUza1NPKzk5aURHMG5a?= =?utf-8?B?QkdVanQ2R0FoWXNwa3ZxQ2dpWjEwdFpSL0piT2Q2aFQzVlMwMnd1M01kaDg3?= =?utf-8?B?VkliNG8vYVBpS2N4UjZjZUxoQ2lZSTdCRC91M3FpWXlVdWVpekRsbTVZZjYv?= =?utf-8?B?U3FkY3FuMXFKK2NOcTN2ZkVaekM5S1g2MTRVV2d3WHlQa1BtMndxSCswWHNE?= =?utf-8?B?SEV0SHNTQ29rcU1oaUlFTWcrS0lOOGZLRi9YZy9QTUJKaTR4UFRWZTVtN3JG?= =?utf-8?B?RGRMT2NWUWVMeWxyM0k1cUJjV05xNXdOWStKbzBjc1V5NlVDSCs0TFhLMUNo?= =?utf-8?B?U3AyeGNaYW9DU0Mwbk13eFZTZ1dLeEFJK3ZEVHN4a2lLVnVzdlFKYUp1RjV1?= =?utf-8?B?QWQ4ay9HMStzamFoZzRDSEUwdzdPT2sxS1htMnkzS09MZUZ4S0J2VXRSRG1S?= =?utf-8?B?a09OeUZvVEFGaENYRDNGZzIrTllvWUdRbnpOVDkyQnVjQWsvcDF1UUs2K3VX?= =?utf-8?B?RGkrVnh6cmk0RktkY2t0bTE3TGIxYkJRSE5VM0FEcitsQkFuajYrd292RkNW?= =?utf-8?B?Mi92MTVKaFNwbVloSHcwZWdqRkZNM2hUcWhJalRQSXJhb0lodURMeFpXODZ6?= =?utf-8?B?RDBudHg2SG9wbkxoYTJlWmt5Z1Z3Vk5OT3lhMnVMSjN6N0JhOVEvK1R1cWVE?= =?utf-8?B?QmNNMTdINVhRU1YvSGcrc1dwa0ppc0tKMUNmMDEvQWhPMDNONW1FV2xteVl2?= =?utf-8?B?enJqZENnU3pReUNaakJ5RTlwei9GbzFwakZSa09DMWw0OHlhNlp6WlViODhW?= =?utf-8?B?aStPeWZQYTZIbGUySVdpMExsdUZ5UzRvSW8ydWxKNjZENGp3NlRBMFE0c2dt?= =?utf-8?B?OWlEM1hXR0FFNVJmei9yMlZmNFUzMVZ2RzdFbGUwdzRiQmhmUHFXenN2MnpT?= =?utf-8?B?QzlhNGRtWVlYNzVVMjY5N0cwQW90WloyazRLTkYrVk0yM3NZWkZEUyszcW5Z?= =?utf-8?B?T0JqYWxVOEI1OXpHK0NMSC9oSHcyTTFyNWhnT1JwbC8xYXJIaDdKL0h3K3Zm?= =?utf-8?B?cUpuNXdkQTFzcDYra2h2VEwyUVFKdHFxUTMwQUhVeUNubis5elhLbXE5WHhp?= =?utf-8?B?VVBJcFpkS3daSEkwc0c3Zy8zODFsaVd5b0FEbG5ud0xCemhUTUwxb25HNTFr?= =?utf-8?B?ZUlDOHVYWEVSS1kxSzc3UFhjSUhBVnkvVWk2VzNhT1lwUy9zak1YWE9kU1Vz?= =?utf-8?B?NkJNV1R2VnBnMloxNE9RN3Fxa1FSYjNubkIrZmNhMklvYUlUdStXcDR5OVlr?= =?utf-8?B?c25mQStYZFl4K2FmT1VUVURDbFp1aU5JNHVNOGU0aGVSekhZUzA5ek5LYUZJ?= =?utf-8?B?a3dndkd6QldhdmpsZFdieS93b285K0l2U1RKZDRnb1ZWMkMzOFBlbmlBb09H?= =?utf-8?B?WDVHU29yb0hQbXhMN1BFaFN5d3ZCZ0JtZ29HU2RHNXlQTkc2dmxxdk9XR1N0?= =?utf-8?B?bG1vczVPOVhaL28yVEZVa0ZLZDNERllDbDRQK1p1UzVMWFl4ZEplV3piZ3NN?= =?utf-8?B?Z3c9PQ==?= X-MS-Exchange-CrossTenant-Network-Message-Id: 60eefd28-3c07-4d12-e4ed-08dbe0512008 X-MS-Exchange-CrossTenant-AuthSource: DS0PR11MB7309.namprd11.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 08 Nov 2023 11:52:03.9482 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 46c98d88-e344-4ed4-8496-4ed7712e255d X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: 8qbmIZLzHT9XVGHc5+ey3kGg4ADotOuWaPXFVozcyHLBLRJm1FIfKczO0TxeJOtE1aluaJDQrwSL/wlwk4j13PcqODnu1QAVqkHJUU8GCLk= X-MS-Exchange-Transport-CrossTenantHeadersStamped: SA1PR11MB7109 X-OriginatorOrg: intel.com X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org On Thu, Nov 02, 2023 at 11:19:24AM -0700, Rahul Gupta wrote: > From: Rahul Gupta > > Initialization often requires rte_eal_init + rte_pktmbuf_pool_create > which can consume a total time of 500-600 ms: > a) For many devices FLR may take a significant chunk of time > (200-250 ms in our use-case), this FLR is triggered during device > probe in rte_eal_init(). > b) rte_pktmbuf_pool_create() can consume upto 300-350 ms for > applications that require huge memory. > > This cost is incurred on each restart (which happens in our use-case > during binary updates for servicing). > This patch provides an optimization using pthreads that appplications > can use and which can save 200-230ms. > > In this patch, rte_eal_init() is refactored into two parts- > a) 1st part is dependent code ie- it’s a perquisite of the FLR and > mempool creation. So this code needs to be executed before any > pthreads. Its named as rte_eal_init_setup() > b) 2nd part of code is independent code ie- it can execute in parallel > to mempool creation in a pthread. Its named as rte_probe_and_ioctl(). > > Existing applications require no changes unless they wish to leverage > the optimization. > > If the application wants to use pthread functionality, it should call- > a) rte_eal_init_setup() then create two or more pthreads- > b) in one pthread call- rte_probe_and_ioctl(), > c) second pthread call- rte_pktmbuf_pool_create() > d) (optional) Other pthreads for any other independent function. > > Signed-off-by: Rahul Gupta Reading the description, this seems an interesting idea, and a good saving. If I may, I wonder if I can suggest a slight alternative. Rather than splitting EAL init into two functions like that, how about providing an "rte_eal_init_async()" function, which does part 1, and then spawns a thread for part 2, before returning. We can then provide an rte_eal_init_done() [or eal_init_async_done()] function to allow apps to resync and check for EAL being done. The reason for suggesting this is that the naming and purpose of the APIs may be a little clearer for the end user. Allowing the async init function to create threads also allows possible future parallelism in the function itself. For example, we could do probing of the devices themselves in parallel. Regards, /Bruce