From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: <dev-bounces@dpdk.org> Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 97C9B43901; Fri, 19 Jan 2024 18:26:53 +0100 (CET) Received: from mails.dpdk.org (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 6A72E402AE; Fri, 19 Jan 2024 18:26:53 +0100 (CET) Received: from mgamail.intel.com (mgamail.intel.com [192.55.52.120]) by mails.dpdk.org (Postfix) with ESMTP id CC39E4027A for <dev@dpdk.org>; Fri, 19 Jan 2024 18:26:51 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1705685212; x=1737221212; h=date:from:to:cc:subject:message-id:references: content-transfer-encoding:in-reply-to:mime-version; bh=Gcp5O8Zl4Rl/HDE+xHQPdWYetzjqZI6Kpxgun+V89fg=; b=P9ndqvbK5HOZtTXSqDL0LF7YC5DvpodQUCLN0P1Hq7zOCNYkKheofEmP uHSUeIOkuWs48lTRAEHVwYO4kJxZl6ESnvXzUjf8lRZIKvs0ZseGauXL7 Tg5B7p+Rm8+hC2FzJZZ91Iau734dtaW/9m/RBQrsBKKC0yjqcPBrsb2Tk m7y+Y++7MZqPg9pXeqKhkycH6Sya3q6QBNr6dknHOEu1VnisbsTmaazhV anyTZmg5vN0WcG00T1kn0aN9M19msXcKjCJ8jQM2pKe7moia5zEYT/tFu UYLeqcUm9gkNPMaCafpIEDHxbV+b7TqvnWjcRcxBUShSp3sBxzO7jdONF Q==; X-IronPort-AV: E=McAfee;i="6600,9927,10957"; a="399667037" X-IronPort-AV: E=Sophos;i="6.05,204,1701158400"; d="scan'208";a="399667037" Received: from orviesa005.jf.intel.com ([10.64.159.145]) by fmsmga104.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 19 Jan 2024 09:26:49 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="6.05,204,1701158400"; d="scan'208";a="634990" Received: from orsmsx602.amr.corp.intel.com ([10.22.229.15]) by orviesa005.jf.intel.com with ESMTP/TLS/AES256-GCM-SHA384; 19 Jan 2024 09:26:49 -0800 Received: from orsmsx610.amr.corp.intel.com (10.22.229.23) by ORSMSX602.amr.corp.intel.com (10.22.229.15) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.35; Fri, 19 Jan 2024 09:26:48 -0800 Received: from orsmsx610.amr.corp.intel.com (10.22.229.23) by ORSMSX610.amr.corp.intel.com (10.22.229.23) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.35; Fri, 19 Jan 2024 09:26:47 -0800 Received: from ORSEDG601.ED.cps.intel.com (10.7.248.6) by orsmsx610.amr.corp.intel.com (10.22.229.23) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.35 via Frontend Transport; Fri, 19 Jan 2024 09:26:47 -0800 Received: from NAM12-MW2-obe.outbound.protection.outlook.com (104.47.66.40) by edgegateway.intel.com (134.134.137.102) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.1.2507.35; Fri, 19 Jan 2024 09:26:47 -0800 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=gAqi0WnVPN4K5O4W8GNKuuQcdTnEH0Vza/v4hvDj0nrsOCrJ4R7LrR1Kp8zBaI/bMpk+KapLnt3szjEJ1CD8dmt9Q1khpiLYiw4AWJ5ex3MIGVy+or5PoT8/W0RRv8X2fvXAf+WbJs9hHjOB5IIvGb6aSYC0uQCaqmHcJ9n0N8nz+kqbebbiyP9e/m9yIjQPD7uXQMN/Of+vgE0uSxjNB3NxOtKDzG5PJJw8q1YVaBd7a8C8tF6uhg2Q4Fh80d2K60NSmw5i+da7+SKoLKEFYyb7eX+S5qROutu41SnMb0AEe48eWzdMBSrPyq8K0cVEiEAy/sltusSm27LfDsWYAw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=5D4Ob44STdfC9IhD2opRQVQj6Ix/QcpykkSbekytICg=; b=HJZHaYHFo9pp1UZtsxwQFzbS9/3uK3Xjql7JKPGUojp/9888agrISybd3oCuqCVYFJVF2eMnRFnxNrNounnQUgut2euBuHdmHVCZmEpSreYNOWGbIhX4/qM33u1OB1WXtknNXzev0dGqQ+/+cWkIp3f0K8sUeA8JIMCsBgwaZsx8o2MIQ69l9W2uNvC6snEZQ4QZQrv5KpwriQu0jWqnMVuGdMa1T+hf/2aYxW9LsrCCAaVroH892c1YzhCW75LDkGCvv2MvhcdrwROZgKVQZB2z9ZQW1dj8e/DZCyHMBHwWpwHayxeVqHvHI+MWpOaDfHW0p89/oBmIsWasrPNCLQ== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=intel.com; dmarc=pass action=none header.from=intel.com; dkim=pass header.d=intel.com; arc=none Authentication-Results: dkim=none (message not signed) header.d=none;dmarc=none action=none header.from=intel.com; Received: from DS0PR11MB7309.namprd11.prod.outlook.com (2603:10b6:8:13e::17) by CH3PR11MB7772.namprd11.prod.outlook.com (2603:10b6:610:120::5) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7202.26; Fri, 19 Jan 2024 17:26:45 +0000 Received: from DS0PR11MB7309.namprd11.prod.outlook.com ([fe80::df88:b743:97f8:516c]) by DS0PR11MB7309.namprd11.prod.outlook.com ([fe80::df88:b743:97f8:516c%5]) with mapi id 15.20.7202.024; Fri, 19 Jan 2024 17:26:44 +0000 Date: Fri, 19 Jan 2024 17:26:37 +0000 From: Bruce Richardson <bruce.richardson@intel.com> To: Rahul Gupta <rahulgupt@linux.microsoft.com> CC: <dev@dpdk.org>, <thomas@monjalon.net>, <dmitry.kozliuk@gmail.com>, <stephen@networkplumber.org>, <sovaradh@linux.microsoft.com>, <okaya@kernel.org>, <sujithsankar@microsoft.com>, <sowmini.varadhan@microsoft.com>, <krathinavel@microsoft.com>, <rahulrgupta27@gmail.com>, Rahul Gupta <rahulgupt@microsoft.com> Subject: Re: [dpdk-dev] [PATCH v3] eal: refactor rte_eal_init into sub-functions Message-ID: <ZaqwzXrxoPHxuH9d@bricha3-MOBL.ger.corp.intel.com> References: <1705683497-25074-1-git-send-email-rahulgupt@linux.microsoft.com> Content-Type: text/plain; charset="utf-8" Content-Disposition: inline Content-Transfer-Encoding: 8bit In-Reply-To: <1705683497-25074-1-git-send-email-rahulgupt@linux.microsoft.com> X-ClientProxiedBy: DB9PR06CA0005.eurprd06.prod.outlook.com (2603:10a6:10:1db::10) To DS0PR11MB7309.namprd11.prod.outlook.com (2603:10b6:8:13e::17) MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: DS0PR11MB7309:EE_|CH3PR11MB7772:EE_ X-MS-Office365-Filtering-Correlation-Id: ea1307e9-845e-471f-9b86-08dc1913cef4 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: OkutMkkrMjthng0eCiKEVcu5Ib+VM0UT9BXczHAB9Y92T7V06LVHXAjmgEexvY2XPC/kLpNojG+D/ngNrGFv3E2fW5qZST0b7GJidunKD9Bd1YaUddkPAt48lhA2oGlQyH1y7xli2LJtMVrLJEjvFOJRmkTqt28BkIYkYAKNgWusSrSOepPijmiA3CIqj16SY26frTxLQDhOcHbAr/IWPEJCcqFap8I3cnnwyHCgodHdoOUEZt700y4RzNpWmPQaNH65hq6Jeds6rvhEsSgvTy7s2+GpdTCuNQPmqZL9qXaNGWPl+TKQjr8MS3fLvWgYkVsAqkN8UKUQE5/6JwfbVHzPv8wabf0RTmJj0vQDYWlOChLncLOKgAeODt82r439PVfHgM1ZOEyleAh0z+p+q5sPOBswaKp9ZKi4y6jCjIVfB6hF5X/POsDM+rgWZ2lzI/QVdnvfWPwEXSjuPz1UWUoX+RbE5lN4jxU704I0LaZI8UN1NgG/bTgxm2BaIb1h2t+jF4oe/dt1Ic0/0SxJysCIeeJNI+Gsu/YPRLul5Wo= X-Forefront-Antispam-Report: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:DS0PR11MB7309.namprd11.prod.outlook.com; PTR:; CAT:NONE; SFS:(13230031)(396003)(39860400002)(346002)(376002)(366004)(136003)(230922051799003)(64100799003)(186009)(1800799012)(451199024)(6666004)(6506007)(6512007)(26005)(38100700002)(86362001)(82960400001)(8936002)(44832011)(4326008)(6486002)(8676002)(7416002)(5660300002)(83380400001)(41300700001)(2906002)(316002)(478600001)(45080400002)(6916009)(66556008)(66476007)(66946007); DIR:OUT; SFP:1102; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: =?utf-8?B?L0NTWXpnTFNOVzdJeUlkeXdGaXE5OHU4OWR5NkFpMkFnMzBSMEg4bGp0YkFF?= =?utf-8?B?TDI1b1BueG1XOU5OOFBxV2F0Z3hzd21veHNpWTRxZTRINFpuTklEVzBSM2FV?= =?utf-8?B?NER1QWlzSlFuTUFYblcyMjQvZFk5WlBCNUpyaSthVmJLa05HazNaVERMaTEx?= =?utf-8?B?YXlwV044aGlCbGpNdmJUL1FqZ3E2eTkrQTFIZEhhSzFGTU53djQ3ZFNJQmE4?= =?utf-8?B?RHhWNi94NXdGNkxLUHUvSzExMnlUcHBuT1kwUnZzNG9NMmRJRGtCSEM3d3Bk?= =?utf-8?B?azVTK1N3SE1qZnd4WjQrektQVXdrUXljb1RtRm5vNzVqNjl0K1ZCUGY0TGlC?= =?utf-8?B?aFZNUVBwTmtxZjZnVTZmem1xNEo5VVM1eFpvU202NTE3UG43Nk1vbUdqY3Ra?= =?utf-8?B?aU1VZGRXV0Q0KzhRa0lQWE9JakNuMEhOSEwvTFlzUkQ1UEJzV2VTMytBWEZV?= =?utf-8?B?bVdzZmQrTmtnQVRiaDloUGlteVdlVmpPZ05MZVBNQ1k0VkY5ZWw0SXpvTzRH?= =?utf-8?B?MncrRmVxdWg5T0ZhVEl1d2l1MXh5U215VEtnNFBid3NCU0txbmd3YjF1RnFF?= =?utf-8?B?WmtPbUJPYW9icVhBRWRmaHk0bnZkaUtOb0ZjL2Rvc010cUJRMzc0QzVOUCs2?= =?utf-8?B?UVBCNDk1UVZqT1l6YUFicEFzNFoweU5PM21ZMlBKNUk2K2prWW5oRVpqR3Jw?= =?utf-8?B?bHZvWHBJcHNaMDk0ZkVRcENDSjlFL0JHc05ScDFRV2tnQmlBVUg4TnRKZVNC?= =?utf-8?B?dEMyQ1kvU3YwcXN3OXNzY1N3bXVGK2F6Rk1TTXVGZ3k1bWw2RlFJemRIWnNV?= =?utf-8?B?VWJwcko4SllIRVIzTWtsK0RvZzFScHNGOS84NWpDc0NqenV4cWY1Q2QvOTFU?= =?utf-8?B?NDlBaXJMeER1bXpSY1FnN2NUaUpWTytlbitFTWRPWE9LclBHM3ZhY2I2Tmlj?= =?utf-8?B?T0ZJYXZHU2tHWW9OTlpsQlloMFh0U3prMTVWNC9TdmN2QWZsWnMyNkk3SkRU?= =?utf-8?B?SU9iclBhNE5BMGRLTHZoS0dQTWZ5cE45MmgvaGRHd0VKclJreGlhOUZVM3FU?= =?utf-8?B?ajdJRjczMWI5MzAxWm9WV1pFRlNCMXVMVU5vcE1tR2FmREdpQ2xsbnRhN0Y3?= =?utf-8?B?NEpvdklJRHJLMkFxRWRKdll0aGo1ZndCK1dUVjBXc1FidzFmalFGMWpybmF0?= =?utf-8?B?WjFpTXZ5UDRqNkFvQ3RPVUcrcTdOZlV5VlN3ZzlSWmpKNTBzbnArZGZ6OW4v?= =?utf-8?B?QjVpZkU0dGVGaWFKMjdrTmlkaVgxeGVSQVJqb09PYlRxb2ZPZmdPaHNqL21N?= =?utf-8?B?b2p3cjMzTHBncFR1Q1VLR3BhdGpIdzI4SnZrTWdUSVZCdU5UMDl3RkR4UHBU?= =?utf-8?B?UldWUXM1a3dzMXVaT0dhV1I0MWE4eXpWZ1ZodnRzKy9kYXlWdWR0M3hERUJy?= =?utf-8?B?MTJUSGFKeVlyQTVBemVzSTIxSElCNHZCZnoxMkNIS3FJZHNTMmJzQXBTblkz?= =?utf-8?B?VEJyb29yYTB2VWVuT0tuUitac0JWajJicFRXVU5YdDNmV3IvTXMxUzlmcVps?= =?utf-8?B?L0t6QVh4QlY5U3NNUTRKUzkwNS82T3F5dUZSMlFLZEhUQzlIQXIzUExDR0dq?= =?utf-8?B?Qyt5cnlXb2xRTVB5bFJlS2J0V09ORU9RbVp6c2hOTVFoMFc0dWplaE9GOCt3?= =?utf-8?B?cW81V2RYamh5R2trMUhDellvQ2FIY2pVZVdpcjhnbDVLMFFjS1JEZ0xzTXYr?= =?utf-8?B?bEMvQkhHL21ZeFp5SnduTFBtbHh2WW9uemxpeWk5QkdlUXc2TjlCM0Y5UERK?= =?utf-8?B?K2ZnL0MrbE5Qb0xpaktra0k0SDBqeGpPNVQrTDVBSld0MVlnSU5KWkpuWFNR?= =?utf-8?B?bnJDTjFPYmUweTFxaVRGVmRUYlRnRjNsR3dLVk9vdFFCbW95THhMVzFaNVhG?= =?utf-8?B?aXhYZitSWm41KytlVGJLZVQvVnZ0aXAxbE4rT01abXVrRlVvUXVEY210Yngy?= =?utf-8?B?RjRaNGc2R09NYUNCRkVpUWFFNUNHNUtzRE9ucXpTM3UvUE8rd0dEc01hQktP?= =?utf-8?B?RmgzZlh2SVd2OUhjUUowZmZoNzFGZXBBS3pDUFYvVnU2TExRRHZjZ21NT2FZ?= =?utf-8?B?UFdtTTdLaUNtTzRLMTFPN0Q1eFBNSmNQRytKRWJSQzhlU3l2RndUNTREYU5r?= =?utf-8?B?UlE9PQ==?= X-MS-Exchange-CrossTenant-Network-Message-Id: ea1307e9-845e-471f-9b86-08dc1913cef4 X-MS-Exchange-CrossTenant-AuthSource: DS0PR11MB7309.namprd11.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 19 Jan 2024 17:26:44.8888 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 46c98d88-e344-4ed4-8496-4ed7712e255d X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: zyo4BbO8B3CZmXCTHGj3MrD8caIdctDJ0N23YAsrP8oniI0dzTSw2667TgFDKIMFUf00/TuTG1skhIDKa8vhBBF+D6X8/psfD7y1IObiWsU= X-MS-Exchange-Transport-CrossTenantHeadersStamped: CH3PR11MB7772 X-OriginatorOrg: intel.com X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions <dev.dpdk.org> List-Unsubscribe: <https://mails.dpdk.org/options/dev>, <mailto:dev-request@dpdk.org?subject=unsubscribe> List-Archive: <http://mails.dpdk.org/archives/dev/> List-Post: <mailto:dev@dpdk.org> List-Help: <mailto:dev-request@dpdk.org?subject=help> List-Subscribe: <https://mails.dpdk.org/listinfo/dev>, <mailto:dev-request@dpdk.org?subject=subscribe> Errors-To: dev-bounces@dpdk.org On Fri, Jan 19, 2024 at 08:58:17AM -0800, Rahul Gupta wrote: > From: Rahul Gupta <rahulgupt@microsoft.com> > > In continuation to the following email, I am sending this patch. > (https://inbox.dpdk.org/dev/20231110172523.GA17466@microsoft.com/) > > Initialization requires rte_eal_init + rte_pktmbuf_pool_create which > can consume a total time of 500-600 ms: > a) For many devices FLR may take a significant chunk of time > (200-250 ms in our use-case), this FLR is triggered during device > probe in rte_eal_init(). > b) rte_pktmbuf_pool_create() can consume up to 300-350 ms for > applications that require huge memory. > > This cost is incurred on each restart (which happens in our use-case > during binary updates for servicing). > This patch provides an optimization using pthreads that applications > can use and which can save 200-230ms. > > In this patch, rte_eal_init() is refactored into two parts- > a) 1st part is dependent code ie- it’s a perquisite of the FLR and > mempool creation. So this code needs to be executed before any > pthreads. Its named as rte_eal_init_setup() > b) 2nd part of code is independent code ie- it can execute in parallel > to mempool creation in a pthread. Its named as rte_eal_init_async(). > > Existing applications requires to just call- > rte_eal_init_wait_async_complete() after rte_eal_init() unless they wish > to leverage the optimization. > This part here is a bit problematic. For something as ubiquitous as rte_eal_init(), we should not be requiring existing application change. Instead, can you create a new init function to replace rte_eal_init() in the async case - and then eal_init can call that and then the async_complete function, to avoid duplicating code. The goal should be to have existing apps unaffected, but to allow a path to faster startup for those that need it. In the original email you linked to above, we had such a proposal: rte_eal_init_async() & rte_eal_init_async_done(). Thanks, /Bruce