From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 14589A0548; Wed, 16 Jun 2021 14:59:41 +0200 (CEST) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 875A24067A; Wed, 16 Jun 2021 14:59:40 +0200 (CEST) Received: from mail-io1-f41.google.com (mail-io1-f41.google.com [209.85.166.41]) by mails.dpdk.org (Postfix) with ESMTP id 2846740140 for ; Wed, 16 Jun 2021 14:59:39 +0200 (CEST) Received: by mail-io1-f41.google.com with SMTP id q3so2868704iop.11 for ; Wed, 16 Jun 2021 05:59:39 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :cc:content-transfer-encoding; bh=I/BUEcK6NXAFYOtn+6HzypFpSSnmNfrSp5btKcGGlrw=; b=GUo2hEu9T79RqhTEAu4UCtY1luVK0nqTl/UAuUSK+8MfDS9rJtAXhKLOHygyDe7Lcw mjArs7UuiVCUXkvXNl6FcCqlcdJ02dogbSa2eGHB/XZmRW9NIBb5ncyqCSEsH9pd7qw+ a/XQRqo3snLlIG9epUWk0jBeHHb85RlXN2XF+jALoJph7XCSpXxJSiHJcMOOHGiwf37E va7cDeXDcsKDx61rntvJfidZ+5fTci+NRdD/iQZl0nRz9rauwVsCyNz+TtPQTBuHZQvJ FvCFFmEyZQe8BcFrJUF0irk364nCMW7ruiXk+k1Gx+6yJWujDX7NJlOIF8YeYIPN9Ir/ pcgg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc:content-transfer-encoding; bh=I/BUEcK6NXAFYOtn+6HzypFpSSnmNfrSp5btKcGGlrw=; b=WgF2mD2lIVFpUJhUqT2z2+Fvn2geIkRxLchDXJz8x7SVSKxxpQOPLYiUwR/so+dJHF Z6lKDqZ3nqfsW7/FLIoTXiOO6CS0Z9E+OSFX940TY4PBxw9F3G1wxOu3Wj6OAPEdI8AT iUlXwcyWpU60/e7a9rlcfX+WU1IA6tD0DgGgvLIZZWac+mAZfP3ABJR+Txyvi2/p69Wz JSuc7F16yVM+mnrAzRNUKb5V/ODQm+4fZNiESYvO+336R9OHHrV04sUE68meMXXJVvS/ 0tHPBlfbksdm+Xp0SEhBH81uVqOGavftUDIEhDMEnUJ2Sq9l85F5YDgYE0pC+Apt/G7z 6JJA== X-Gm-Message-State: AOAM530KIiPUKsOwB8nMlXH5mlmWQ/o6DBjUJdoMQd6wF0hnhwoXg+jb owXK2xmQMlmIy3rY66sfnM7z8dtCcoFtOYgU+wg= X-Google-Smtp-Source: ABdhPJzW1TY0QcU858bAI/0JLb4KZWEQDpCu6mWJKveKm4+Rc4fZXJnYwbjfVPvYvtKFqtsTzjvrYsGdsRyJ6k0vyiE= X-Received: by 2002:a02:3318:: with SMTP id c24mr4131561jae.112.1623848378353; Wed, 16 Jun 2021 05:59:38 -0700 (PDT) MIME-Version: 1.0 References: <20210614105839.3379790-1-thomas@monjalon.net> <2004320.XGyPsaEoyj@thomas> <98CBD80474FA8B44BF855DF32C47DC35C61851@smartserver.smartshare.dk> <1857954.7Ex43hCf9S@thomas> <01e1ade6-827f-83c7-5375-125b72888053@intel.com> In-Reply-To: <01e1ade6-827f-83c7-5375-125b72888053@intel.com> From: Jerin Jacob Date: Wed, 16 Jun 2021 18:29:22 +0530 Message-ID: To: "Burakov, Anatoly" Cc: Thomas Monjalon , =?UTF-8?Q?Morten_Br=C3=B8rup?= , Bruce Richardson , dpdk-dev , Olivier Matz , Andrew Rybchenko , Honnappa Nagarahalli , "Ananyev, Konstantin" , Ferruh Yigit , Jerin Jacob , Akhil Goyal , Dmitry Kozlyuk , Narcisa Ana Maria Vasile , "Dmitry Malloy (MESHCHANINOV)" , Pallavi Kadam Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable Subject: Re: [dpdk-dev] [PATCH] parray: introduce internal API for dynamic arrays X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" On Wed, Jun 16, 2021 at 5:52 PM Burakov, Anatoly wrote: > > On 16-Jun-21 10:42 AM, Jerin Jacob wrote: > > On Tue, Jun 15, 2021 at 12:18 PM Thomas Monjalon = wrote: > >> > >> 14/06/2021 17:48, Morten Br=C3=B8rup: > >>>> From: dev [mailto:dev-bounces@dpdk.org] On Behalf Of Thomas Monjalon > >>> It would be much simpler to just increase RTE_MAX_ETHPORTS to somethi= ng big enough to hold a sufficiently large array. And possibly add an rte_m= ax_ethports variable to indicate the number of populated entries in the arr= ay, for use when iterating over the array. > >>> > >>> Can we come up with another example than RTE_MAX_ETHPORTS where this = library provides a better benefit? > >> > >> What is big enough? > >> Is 640KB enough for RAM? ;) > > > > If I understand it correctly, Linux process allocates 640KB due to > > that fact currently > > struct rte_eth_dev rte_eth_devices[RTE_MAX_ETHPORTS] is global and it > > is from BSS. > > > > If we make this from heap i.e use malloc() to allocate this memory > > then in my understanding Linux > > really won't allocate the real page for backend memory until unless, > > someone write/read to this memory. > > > > i.e it will be free virtual memory using Linux memory management help. > > If so, we can keep large values for RTE_MAX_ETHPORTS > > without wasting any "real" memory even though the system has a few port= s. > > > > Thoughts? > > > > mmap works this way with anonymous memory, i'm not so sure about > malloc()'ed memory. Looking at online documentation scatters over the internet, sbrk(), is based on demand paging. So I am not sure as well. I am also not sure how we can write some test case to verify it. Allocating a huge memory through malloc() not failing, not sure it is due to demand pagging or Linux over commit feature or combination of both, if mmap works in this way, we could have EAL abstraction for such memory alloc like eal_malloc_demand_page() or so and if Windows also supports it. > Plus, we can't base these decisions on what Linux > does because we support other OS's. Do they do this as well? + Windows OS maintainers > > -- > Thanks, > Anatoly