From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 5AC35439B8; Wed, 24 Jan 2024 18:49:29 +0100 (CET) Received: from mails.dpdk.org (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id C3F8940DF5; Wed, 24 Jan 2024 18:49:28 +0100 (CET) Received: from linux.microsoft.com (linux.microsoft.com [13.77.154.182]) by mails.dpdk.org (Postfix) with ESMTP id DBE76402A6 for ; Wed, 24 Jan 2024 18:49:26 +0100 (CET) Received: by linux.microsoft.com (Postfix, from userid 1086) id 0D60220E5641; Wed, 24 Jan 2024 09:49:26 -0800 (PST) DKIM-Filter: OpenDKIM Filter v2.11.0 linux.microsoft.com 0D60220E5641 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linux.microsoft.com; s=default; t=1706118566; bh=a7L0vF4ZmC6b8oeLCOLO1mtV68WNtItWn6j9nZPhQcM=; h=Date:From:To:Cc:Subject:References:In-Reply-To:From; b=LrBe8pRWYan6OTnwhP30Jh2nV8fK2wyQOxn7zBXUDRmjlQOlrAjHtRXs1SbJES8tN pJlhdfU8Ll4Zz/4sdrT0zrnNiS/ReR+YTNJqilFLaiA3LI3/gwzq7eG+e1UZX9q/LO ENnHvNrWauhUj24CsT1hXeLZ+dXyZ2wMV2Qcq2NY= Date: Wed, 24 Jan 2024 09:49:26 -0800 From: Tyler Retzlaff To: Rahul Gupta Cc: dev@dpdk.org, thomas@monjalon.net, bruce.richardson@intel.com, dmitry.kozliuk@gmail.com, stephen@networkplumber.org, sovaradh@linux.microsoft.com, okaya@kernel.org, sujithsankar@microsoft.com, sowmini.varadhan@microsoft.com, krathinavel@microsoft.com, rahulrgupta27@gmail.com, Rahul Gupta Subject: Re: [dpdk-dev] [PATCH v4] eal: refactor rte_eal_init into sub-functions Message-ID: <20240124174926.GA7395@linuxonhyperv3.guj3yctzbm1etfxqx2vob5hsef.xx.internal.cloudapp.net> References: <1706103911-6907-1-git-send-email-rahulgupt@linux.microsoft.com> MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Disposition: inline Content-Transfer-Encoding: 8bit In-Reply-To: <1706103911-6907-1-git-send-email-rahulgupt@linux.microsoft.com> User-Agent: Mutt/1.5.21 (2010-09-15) X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org On Wed, Jan 24, 2024 at 05:45:11AM -0800, Rahul Gupta wrote: > From: Rahul Gupta > > In continuation to the following email, I am sending this patch. > (https://inbox.dpdk.org/dev/20231110172523.GA17466@microsoft.com/) > > Initialization requires rte_eal_init + rte_pktmbuf_pool_create which > can consume a total time of 500-600 ms: > a) For many devices FLR may take a significant chunk of time > (200-250 ms in our use-case), this FLR is triggered during device > probe in rte_eal_init(). > b) rte_pktmbuf_pool_create() can consume up to 300-350 ms for > applications that require huge memory. > > This cost is incurred on each restart (which happens in our use-case > during binary updates for servicing). > This patch provides an optimization using pthreads that applications > can use and which can save 200-230ms. > > In this patch, rte_eal_init() is refactored into two parts- > a) 1st part is dependent code ie- it’s a perquisite of the FLR and > mempool creation. So this code needs to be executed before any > pthreads. Its named as rte_eal_init_setup() > b) 2nd part of code is independent code ie- it can execute in parallel > to mempool creation in a pthread. Its named as rte_eal_init_async_setup(). > > In existing applications no changes are required unless they wish to leverage > the optimization. > > If the application wants to leverage this optimization, then it needs to call > rte_eal_init_async() (instead of call rte_eal_init()), then it can create a > thread using rte_eal_remote_launch() to schedule a task it would like todo in > parallel rte_eal_init_async_setup(), this task can be a mbuf pool creation > using- rte_pktmbuf_pool_create() > > After this, if next operations require completion of above task, then > user can use rte_eal_init_wait_async_setup_complete(), > or if user wants to just check status of that thread, then use- > rte_eal_init_async_setup_done() > > --- > v2: Address Stephen Hemminger's comment > --- > v3: address support for single lcore > --- > v4: address Brue Richardson and Stephen Hemminger comment > Existing application need not do any changes if bootup optimization is not needed. > > app/test-pmd/testpmd.c | 24 ++++++++- > lib/eal/include/rte_eal.h | 107 ++++++++++++++++++++++++++++++++++++++ > lib/eal/linux/eal.c | 62 ++++++++++++++++++++-- > lib/eal/version.map | 7 +++ > 4 files changed, 196 insertions(+), 4 deletions(-) NAK changes to EAL need to cover all support OS/platforms, EAL is not Linux-only library.