From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id DB022432D7; Wed, 8 Nov 2023 14:53:35 +0100 (CET) Received: from mails.dpdk.org (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id B4790427E0; Wed, 8 Nov 2023 14:53:35 +0100 (CET) Received: from mail-lj1-f176.google.com (mail-lj1-f176.google.com [209.85.208.176]) by mails.dpdk.org (Postfix) with ESMTP id 06EBE402A7 for ; Wed, 8 Nov 2023 14:53:34 +0100 (CET) Received: by mail-lj1-f176.google.com with SMTP id 38308e7fff4ca-2c50906f941so99105061fa.2 for ; Wed, 08 Nov 2023 05:53:33 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1699451613; x=1700056413; darn=dpdk.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:subject:cc:to:from:date:from:to:cc:subject:date :message-id:reply-to; bh=wD98/XtKZoS/Ub3NvuVw6/0ZhbFjyNuMkYUfiGlsMQI=; b=Etwm7q7oFJcFB6aUwagwxktWQPft0cZeywK1YtMARd98LWcTUpHtVEGbdXhW8i/uJW x6Gx0HQrPLK/XXtm/7o3QyK+4JfkNkNFxWaOF9iyDxiwL6gzyYkzje5/18jsavrEaUfl OmEw3AFSxM739ZQvvT/krSyjvGrpXcvROcz0JoDfgHlxqjam4XUVTNa0nl0eculw11dP aGMreFoKABQ6bXQb2pCdbL9FgoGOR/jbTHbUjuK98YPqtajyiiheIVUApAdIQCpKzuTF pBK6g6J1kgmWMgEPjRKgZ37OJzxLoU1NL+ASKQqBgRKkWhijSvk3GT6LJHB7YfLbviNW W81g== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1699451613; x=1700056413; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:subject:cc:to:from:date:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=wD98/XtKZoS/Ub3NvuVw6/0ZhbFjyNuMkYUfiGlsMQI=; b=uN1hWEYPoBneyjSpt22IkYVlkxVvH72SuTUBeyb6KtP0OLkD/1rWG7Wewe4fz6P+ZO yc+HSmUT7sPEM2LJQZ04DZiQAbT4g7X4gU1nEJGNGwgs++K3ALqMWVbPaynMVQ4milDZ ea7ePDMF/4mh+snNxgzDEohPyS/XvVJGBtHvT6EPcYx2TqBJnEV8NdsDG42WJdUYJhGc 2/lbcLiWjROzhptfvEHueYhJPUEsk/DPglY2ZFViZqrZbMRe8SWzzPQclRl/5x7XV6dA jnQxo2TRN69vN9uKJ4BFncA9+hcJ4Sz1D/NVE95ajWcAMhn71kcqj8ioClS44s61izjV +xmA== X-Gm-Message-State: AOJu0Yy0+J+Ad5QXisrlByiW7XN3BPU3rA6rUplU3TTZpN2mvJquMd5L MLxM6bdqV+a3oOkxUL67Gf8= X-Google-Smtp-Source: AGHT+IFlKPsPGoJPybrwUXlgv4R8WUlnf9r/kR6mJB9Qohx+3EaIpSK3hBP4p81xlt2PdiDDCXsU9g== X-Received: by 2002:a05:6512:33c1:b0:503:2561:adbc with SMTP id d1-20020a05651233c100b005032561adbcmr1651190lfg.64.1699451613017; Wed, 08 Nov 2023 05:53:33 -0800 (PST) Received: from sovereign (broadband-109-173-110-33.ip.moscow.rt.ru. [109.173.110.33]) by smtp.gmail.com with ESMTPSA id f22-20020a05651232d600b00507a62cb135sm689297lfg.179.2023.11.08.05.53.31 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 08 Nov 2023 05:53:32 -0800 (PST) Date: Wed, 8 Nov 2023 16:53:31 +0300 From: Dmitry Kozlyuk To: rahul gupta Cc: Stephen Hemminger , Rahul Gupta , dev@dpdk.org, thomas@monjalon.net, sovaradh@linux.microsoft.com, okaya@kernel.org, sujithsankar@microsoft.com, sowmini.varadhan@microsoft.com, Rahul Gupta Subject: Re: [RFC] eal: RFC to refactor rte_eal_init into sub-functions Message-ID: <20231108165331.61eef3e0@sovereign> In-Reply-To: References: <1698949164-20287-1-git-send-email-rahulgupt@linux.microsoft.com> <20231102113759.341064ba@fedora> X-Mailer: Claws Mail 3.18.0 (GTK+ 2.24.33; x86_64-pc-linux-gnu) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org 2023-11-07 23:03 (UTC+0530), rahul gupta: > > > From: Rahul Gupta > > > To: dev@dpdk.org, thomas@monjalon.net > > > Cc: sovaradh@linux.microsoft.com, okaya@kernel.org, =20 > > sujithsankar@microsoft.com, sowmini.varadhan@microsoft.com, > > rahulrgupta27@gmail.com, Rahul Gupta , Rahul > > Gupta =20 > > > Subject: [RFC] eal: RFC to refactor rte_eal_init into sub-functions > > > Date: Thu, 2 Nov 2023 11:19:24 -0700 > > > X-Mailer: git-send-email 1.8.3.1 > > > > > > From: Rahul Gupta > > > > > > Initialization often requires rte_eal_init + rte_pktmbuf_pool_create > > > which can consume a total time of 500-600 ms: > > > a) For many devices FLR may take a significant chunk of time > > > (200-250 ms in our use-case), this FLR is triggered during device > > > probe in rte_eal_init(). > > > b) rte_pktmbuf_pool_create() can consume upto 300-350 ms for > > > applications that require huge memory. > > > > > > This cost is incurred on each restart (which happens in our use-case > > > during binary updates for servicing). > > > This patch provides an optimization using pthreads that appplications > > > can use and which can save 200-230ms. > > > > > > In this patch, rte_eal_init() is refactored into two parts- > > > a) 1st part is dependent code ie- it=E2=80=99s a perquisite of the FL= R and > > > mempool creation. So this code needs to be executed before any > > > pthreads. Its named as rte_eal_init_setup() > > > b) 2nd part of code is independent code ie- it can execute in parallel > > > to mempool creation in a pthread. Its named as rte_probe_and_ioctl= (). > > > > > > Existing applications require no changes unless they wish to leverage > > > the optimization. > > > > > > If the application wants to use pthread functionality, it should call- > > > a) rte_eal_init_setup() then create two or more pthreads- > > > b) in one pthread call- rte_probe_and_ioctl(), > > > c) second pthread call- rte_pktmbuf_pool_create() > > > d) (optional) Other pthreads for any other independent function. > > > > > > Signed-off-by: Rahul Gupta =20 I doubt that the new API is required. It is already possible to block all devices from automatic probing with EAL options and then probe explicitly in any threads desired. At the same time, this RFC shows a valuable optimization pattern, so maybe it is worth having in DPDK as an example. There are DPDK use cases when probing is completely unnecessary. Exposing the initialization process stages makes it harder to refactor and requires precise documentation of when and what is initialized (for example, in this RFC rte_eal_init_setup() does not make service core API usable yet). P. S. You may be also interested in using `--huge-unlink=3Dnever` to speed rte_pktmbuf_pool_create() during restarts: https://doc.dpdk.org/guides/linux_gsg/linux_eal_parameters.html#id3