From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id AD52443270; Thu, 2 Nov 2023 19:19:46 +0100 (CET) Received: from mails.dpdk.org (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 9181F40262; Thu, 2 Nov 2023 19:19:46 +0100 (CET) Received: from linux.microsoft.com (linux.microsoft.com [13.77.154.182]) by mails.dpdk.org (Postfix) with ESMTP id 3BB9740144 for ; Thu, 2 Nov 2023 19:19:45 +0100 (CET) Received: by linux.microsoft.com (Postfix, from userid 1179) id 949ED20B74C0; Thu, 2 Nov 2023 11:19:44 -0700 (PDT) DKIM-Filter: OpenDKIM Filter v2.11.0 linux.microsoft.com 949ED20B74C0 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linux.microsoft.com; s=default; t=1698949184; bh=McAqYfRJhqxX4+cN2fZcD4MRjD4hB29K+1xObF0i71U=; h=From:To:Cc:Subject:Date:From; b=DiD8lRbMB4SAoOLLgDDBrQGyfYGtE1DJdr1/rKNJD/bQulyc0+rM9dVNwv/KAUKEu wJiPRHfosppXV2dL4tIjLYSrjOn2BFv1GERQIp0bmUIJ1eQCvGJbS/zASFBPrZXExs sJFYlBttb7FK3zmwMs/8Wo4jvUkUgVarIY7vXTJM= From: Rahul Gupta To: dev@dpdk.org, thomas@monjalon.net Cc: sovaradh@linux.microsoft.com, okaya@kernel.org, sujithsankar@microsoft.com, sowmini.varadhan@microsoft.com, rahulrgupta27@gmail.com, Rahul Gupta , Rahul Gupta Subject: [RFC] eal: RFC to refactor rte_eal_init into sub-functions Date: Thu, 2 Nov 2023 11:19:24 -0700 Message-Id: <1698949164-20287-1-git-send-email-rahulgupt@linux.microsoft.com> X-Mailer: git-send-email 1.8.3.1 MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org From: Rahul Gupta Initialization often requires rte_eal_init + rte_pktmbuf_pool_create which can consume a total time of 500-600 ms: a) For many devices FLR may take a significant chunk of time (200-250 ms in our use-case), this FLR is triggered during device probe in rte_eal_init(). b) rte_pktmbuf_pool_create() can consume upto 300-350 ms for applications that require huge memory. This cost is incurred on each restart (which happens in our use-case during binary updates for servicing). This patch provides an optimization using pthreads that appplications can use and which can save 200-230ms. In this patch, rte_eal_init() is refactored into two parts- a) 1st part is dependent code ie- it’s a perquisite of the FLR and mempool creation. So this code needs to be executed before any pthreads. Its named as rte_eal_init_setup() b) 2nd part of code is independent code ie- it can execute in parallel to mempool creation in a pthread. Its named as rte_probe_and_ioctl(). Existing applications require no changes unless they wish to leverage the optimization. If the application wants to use pthread functionality, it should call- a) rte_eal_init_setup() then create two or more pthreads- b) in one pthread call- rte_probe_and_ioctl(), c) second pthread call- rte_pktmbuf_pool_create() d) (optional) Other pthreads for any other independent function. Signed-off-by: Rahul Gupta --- lib/librte_eal/common/include/rte_eal.h | 2 ++ lib/librte_eal/linux/eal/eal.c | 18 ++++++++++++++++-- lib/librte_eal/rte_eal_version.map | 2 ++ 3 files changed, 20 insertions(+), 2 deletions(-) diff --git a/lib/librte_eal/common/include/rte_eal.h b/lib/librte_eal/common/include/rte_eal.h index 2f9ed29..90db16b 100644 --- a/lib/librte_eal/common/include/rte_eal.h +++ b/lib/librte_eal/common/include/rte_eal.h @@ -129,6 +129,8 @@ enum rte_proc_type_t { * ENOEXEC indicates that a service core failed to launch successfully. */ int rte_eal_init(int argc, char **argv); +int rte_eal_init_setup(int argc, char **argv); +int rte_probe_and_ioctl(void); /** * Clean up the Environment Abstraction Layer (EAL) diff --git a/lib/librte_eal/linux/eal/eal.c b/lib/librte_eal/linux/eal/eal.c index 5cabba8..6f2a4d0 100644 --- a/lib/librte_eal/linux/eal/eal.c +++ b/lib/librte_eal/linux/eal/eal.c @@ -956,9 +956,8 @@ static void rte_eal_init_alert(const char *msg) return n > 2; } -/* Launch threads, called at application init(). */ int -rte_eal_init(int argc, char **argv) +rte_eal_init_setup(int argc, char **argv) { int i, fctret, ret; pthread_t thread_id; @@ -1246,7 +1245,13 @@ static void rte_eal_init_alert(const char *msg) */ rte_eal_mp_remote_launch(sync_func, NULL, SKIP_MASTER); rte_eal_mp_wait_lcore(); + return fctret; +} +int +rte_probe_and_ioctl(void) +{ + int ret = 0; /* initialize services so vdevs register service during bus_probe. */ ret = rte_service_init(); if (ret) { @@ -1296,7 +1301,16 @@ static void rte_eal_init_alert(const char *msg) /* Call each registered callback, if enabled */ rte_option_init(); + return 0; //return Success if control reaches here +} +/* Launch threads, called at application init(). */ +int +rte_eal_init(int argc, char **argv) +{ + int fctret = rte_eal_init_setup(argc, argv); + if (fctret>=0) + fctret = rte_probe_and_ioctl(); return fctret; } diff --git a/lib/librte_eal/rte_eal_version.map b/lib/librte_eal/rte_eal_version.map index 6482539..9e6054f 100644 --- a/lib/librte_eal/rte_eal_version.map +++ b/lib/librte_eal/rte_eal_version.map @@ -53,6 +53,8 @@ DPDK_20.0 { rte_eal_hotplug_remove; rte_eal_hpet_init; rte_eal_init; + rte_eal_init_setup; + rte_probe_and_ioctl; rte_eal_iopl_init; rte_eal_iova_mode; rte_eal_lcore_role; -- 1.8.3.1