From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 17B21439B7; Wed, 24 Jan 2024 16:53:53 +0100 (CET) Received: from mails.dpdk.org (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 8C2AD40E72; Wed, 24 Jan 2024 16:53:52 +0100 (CET) Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) by mails.dpdk.org (Postfix) with ESMTP id B446940E0F for ; Wed, 24 Jan 2024 16:53:50 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1706111630; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=LUv3EL9n+efIHhTNxN8qwiezW4xeMk/OqttEwkyVg4A=; b=Gjt31ciKTSmSZqJkK0X1tWikfkd6Z3+V9BgqB14/ML47OzmkL2IN2Cj5VVOmv2iEJeWToH 4gWUIvJk9lrQTMzwPoJ0cvhoNUKdW1106Md6J/o7B3PxmIRrPGcyZbgvkfpuKC7RuRdNCc MHbVSBeILWycNcgf1TT9uBsTGFUhRPc= Received: from mail-lj1-f198.google.com (mail-lj1-f198.google.com [209.85.208.198]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-393-F5droj0DPUOZq7_T_XGyHQ-1; Wed, 24 Jan 2024 10:53:46 -0500 X-MC-Unique: F5droj0DPUOZq7_T_XGyHQ-1 Received: by mail-lj1-f198.google.com with SMTP id 38308e7fff4ca-2cf119d6c7aso11312661fa.1 for ; Wed, 24 Jan 2024 07:53:46 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1706111625; x=1706716425; h=content-transfer-encoding:cc:to:subject:message-id:date:from :in-reply-to:references:mime-version:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=LUv3EL9n+efIHhTNxN8qwiezW4xeMk/OqttEwkyVg4A=; b=fMW9bmkuJ2bAIMx3uV9TyVaOkfE4Vl3ySVF2ifAQv3xy3uY05nWaoCNJSfJaUI3hr6 /O1s+XAUpl+T0+VSmd7AJYAI4s6mCNv4B0uTSxvBFyuvU6lkKpLrF3WrE/A/Kj46uPQv U0BEXWLmqZ7wGml3OM10bjYL9hcfVrVlshErYZ5B0QXsGrqIYeAPnPEogvRdLKZ89BIP n87acUcIusGo1oF+ADJDrNJMRzQVyFfv9XqVkpvUZuRwcCTfKXYfTC32j7cqVqR7vnT8 dmYolHhb3mvlPjh1it24v2qrBGKAOWjJouweywrLfy0qeXkBFH+U7gZOnsMlxoLdLXWC 441w== X-Gm-Message-State: AOJu0YzdYl42JgNI4vMDz0h21lbzFo6OIlcoOvXftJIhnjKWBOU9BD2U 8XMQY/AozsSazM/nHoPorP+tsgdY/gtqwrsCMVrDe6S2crifFJK7wWXB02MxLHEQMaz27ICiATy rHKisiQeYQzuvIsfJiwD+oz1VJ5RDKMAoMot+XYxg+ucyhcu4igGk1NfFJ2DPRSw8lL1iX/Fcoj Zo77wxsd43m1gxX6A= X-Received: by 2002:a05:6512:3e05:b0:510:16b4:f579 with SMTP id i5-20020a0565123e0500b0051016b4f579mr313763lfv.23.1706111625356; Wed, 24 Jan 2024 07:53:45 -0800 (PST) X-Google-Smtp-Source: AGHT+IHY2e3fjqNleIBZzygfQavzcQg64OX6CfIwdDDjwo7FVkF9GteMwjicYdwBSDO4ZtvKVkevcBAmtXeC2dtZdQ4= X-Received: by 2002:a05:6512:3e05:b0:510:16b4:f579 with SMTP id i5-20020a0565123e0500b0051016b4f579mr313746lfv.23.1706111624983; Wed, 24 Jan 2024 07:53:44 -0800 (PST) MIME-Version: 1.0 References: <1706103911-6907-1-git-send-email-rahulgupt@linux.microsoft.com> In-Reply-To: <1706103911-6907-1-git-send-email-rahulgupt@linux.microsoft.com> From: David Marchand Date: Wed, 24 Jan 2024 16:53:33 +0100 Message-ID: Subject: Re: [dpdk-dev] [PATCH v4] eal: refactor rte_eal_init into sub-functions To: Rahul Gupta Cc: dev@dpdk.org, thomas@monjalon.net, bruce.richardson@intel.com, dmitry.kozliuk@gmail.com, stephen@networkplumber.org, sovaradh@linux.microsoft.com, okaya@kernel.org, sujithsankar@microsoft.com, sowmini.varadhan@microsoft.com, krathinavel@microsoft.com, rahulrgupta27@gmail.com, Rahul Gupta X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org On Wed, Jan 24, 2024 at 2:45=E2=80=AFPM Rahul Gupta wrote: > > From: Rahul Gupta > > In continuation to the following email, I am sending this patch. > (https://inbox.dpdk.org/dev/20231110172523.GA17466@microsoft.com/) > > Initialization requires rte_eal_init + rte_pktmbuf_pool_create which > can consume a total time of 500-600 ms: > a) For many devices FLR may take a significant chunk of time > (200-250 ms in our use-case), this FLR is triggered during device > probe in rte_eal_init(). > b) rte_pktmbuf_pool_create() can consume up to 300-350 ms for > applications that require huge memory. > > This cost is incurred on each restart (which happens in our use-case > during binary updates for servicing). > This patch provides an optimization using pthreads that applications > can use and which can save 200-230ms. > > In this patch, rte_eal_init() is refactored into two parts- > a) 1st part is dependent code ie- it=E2=80=99s a perquisite of the FLR an= d > mempool creation. So this code needs to be executed before any > pthreads. Its named as rte_eal_init_setup() > b) 2nd part of code is independent code ie- it can execute in parallel > to mempool creation in a pthread. Its named as rte_eal_init_async_setu= p(). > > In existing applications no changes are required unless they wish to leve= rage > the optimization. > > If the application wants to leverage this optimization, then it needs to = call > rte_eal_init_async() (instead of call rte_eal_init()), then it can create= a > thread using rte_eal_remote_launch() to schedule a task it would like tod= o in > parallel rte_eal_init_async_setup(), this task can be a mbuf pool creatio= n > using- rte_pktmbuf_pool_create() > > After this, if next operations require completion of above task, then > user can use rte_eal_init_wait_async_setup_complete(), > or if user wants to just check status of that thread, then use- > rte_eal_init_async_setup_done() Looking at what this patch does.. I am under the impression all you really need is rte_eal_init without initial probing. Such behavior can probably be achieved with a allowlist set to a non existing device (like for example "-a 0000:00:00.0"), then later, use device hotplug. Some quick note on this patch. - don't expose symbols externally if they are only for internal use in the same library, - current version is 24.03, not 24.01 (wrt comment in version.map), - other OSes are not handled by this patch, please do the work for FreeBSD and Windows, - as a followup of the previous point, please check if we can share code between OSes and, if so, move the shared code to lib/eal/common, - did you test this series with multiprocess? - why should telemetry and other parts of the current rte_eal_init() be left in the second stage of this initialisation pipeline? --=20 David Marchand