Re: [dpdk-dev] [PATCH v4] eal: refactor rte_eal_init into sub-functions

[David Marchand <david.marchand@redhat.com>]

On Wed, Jan 24, 2024 at 2:45 PM Rahul Gupta
<rahulgupt@linux.microsoft.com> wrote:
>
> From: Rahul Gupta <rahulgupt@microsoft.com>
>
> In continuation to the following email, I am sending this patch.
> (https://inbox.dpdk.org/dev/20231110172523.GA17466@microsoft.com/)
>
> Initialization requires rte_eal_init + rte_pktmbuf_pool_create which
> can consume a total time of 500-600 ms:
> a) For many devices FLR may take a significant chunk of time
>    (200-250 ms in our use-case), this FLR is triggered during device
>    probe in rte_eal_init().
> b) rte_pktmbuf_pool_create() can consume up to 300-350 ms for
> applications that require huge memory.
>
> This cost is incurred on each restart (which happens in our use-case
> during binary updates for servicing).
> This patch provides an optimization using pthreads that applications
> can use and which can save 200-230ms.
>
> In this patch, rte_eal_init() is refactored into two parts-
> a) 1st part is dependent code ie- it’s a perquisite of the FLR and
>    mempool creation. So this code needs to be executed before any
>    pthreads. Its named as rte_eal_init_setup()
> b) 2nd part of code is independent code ie- it can execute in parallel
>    to mempool creation in a pthread. Its named as rte_eal_init_async_setup().
>
> In existing applications no changes are required unless they wish to leverage
> the optimization.
>
> If the application wants to leverage this optimization, then it needs to call
> rte_eal_init_async() (instead of call rte_eal_init()), then it can create a
> thread using rte_eal_remote_launch() to schedule a task it would like todo in
> parallel rte_eal_init_async_setup(), this task can be a mbuf pool creation
> using- rte_pktmbuf_pool_create()
>
> After this, if next operations require completion of above task, then
> user can use rte_eal_init_wait_async_setup_complete(),
> or if user wants to just check status of that thread, then use-
> rte_eal_init_async_setup_done()

Looking at what this patch does.. I am under the impression all you
really need is rte_eal_init without initial probing.
Such behavior can probably be achieved with a allowlist set to a non
existing device (like for example "-a 0000:00:00.0"), then later, use
device hotplug.


Some quick note on this patch.

- don't expose symbols externally if they are only for internal use in
the same library,
- current version is 24.03, not 24.01 (wrt comment in version.map),
- other OSes are not handled by this patch, please do the work for
FreeBSD and Windows,
- as a followup of the previous point, please check if we can share
code between OSes and, if so, move the shared code to lib/eal/common,
- did you test this series with multiprocess?
- why should telemetry and other parts of the current rte_eal_init()
be left in the second stage of this initialisation pipeline?


-- 
David Marchand