From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 4BE04A0C46; Thu, 17 Jun 2021 00:58:11 +0200 (CEST) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id E07A94067A; Thu, 17 Jun 2021 00:58:10 +0200 (CEST) Received: from mail-lj1-f178.google.com (mail-lj1-f178.google.com [209.85.208.178]) by mails.dpdk.org (Postfix) with ESMTP id 1454040150 for ; Thu, 17 Jun 2021 00:58:10 +0200 (CEST) Received: by mail-lj1-f178.google.com with SMTP id x14so6055350ljp.7 for ; Wed, 16 Jun 2021 15:58:10 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=date:from:to:cc:subject:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=yeSbZTnx6Pi7W8mY6fqOTz2Qxtv1GtlG1M3dCk/pHZY=; b=FPzirg7C9JmLGxcq0fC9zvtFpl/k3da8Z+V9DsKnP/TJ4xwPUigqSRBJiQ/WAWr08K brSn4XSBcOt2wEsJTAVVJslPjcsw3EhmxCMnAPicff/IbUia4o1EUtKhEKCc7Py2sssJ 9g8m4fIaFg94DOpFTg2DGpEOr+0/QY0Q3stayZxM2GgLGmhwFS2i2piD3ovgYHryuhYE GfS9jKZ2mEpU2uVddHuE/xjV1MgakeS7gpL7sj4QWXd4tBPit7IMfvAozFgClnNwgZ46 UiTO0JA27fF4Iqus5GCpXO5p8CpkxKq9RtdJlYSV0woodSfjDKiSZzxJeN4AVMDw46gS +DJw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:from:to:cc:subject:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=yeSbZTnx6Pi7W8mY6fqOTz2Qxtv1GtlG1M3dCk/pHZY=; b=QEPTrksaEuKFLeWkzyOyrFofuueSEqZkQ4NAZmLO0OhwmBfAsFSqMAoj6JLtfP5Vz/ vEsH/dkPliy2EtducMU+TRAzk7j/UW/QWzZ1MZ9M/Wxrin5+olW0jCONQbiZ/qsuJWl7 EmPWKngujgsF38yq19i2EB0dCSWQoRJ1aod/Jfr7lda/6tURftapg3RNCX7OJ2PiMNFX pZija9AYpjaMoOfWludwElxpcJlBGFvdVVdz6s0Hc2Mtozs4V61MFLWEU3+QiKRXfI/L NFOngKlU+O7MJhWWXFPpaWMsmltEk+bDJDdYGJSFjlzixYWBQd1k5U4snvIYP6K3WakR 4HFg== X-Gm-Message-State: AOAM530C9pyobjQr/JpdwRp7z1AfJd3ZG1zie7lR4+f+bkIr4eFtmqYI q0L3tzxWtOHksmdD6/Kl9lk= X-Google-Smtp-Source: ABdhPJxTwB++XIZLlxCn9nLZch3IPu5xllRJvdvzcIH5bRXFMLL8b6BDJSyt6otdwswejnjbH9czdA== X-Received: by 2002:a05:651c:d7:: with SMTP id 23mr1901746ljr.207.1623884289522; Wed, 16 Jun 2021 15:58:09 -0700 (PDT) Received: from sovereign (broadband-37-110-65-23.ip.moscow.rt.ru. [37.110.65.23]) by smtp.gmail.com with ESMTPSA id b28sm382274lfp.197.2021.06.16.15.58.07 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 16 Jun 2021 15:58:08 -0700 (PDT) Date: Thu, 17 Jun 2021 01:58:06 +0300 From: Dmitry Kozlyuk To: Jerin Jacob Cc: "Burakov, Anatoly" , Thomas Monjalon , Morten =?UTF-8?B?QnLDuHJ1cA==?= , Bruce Richardson , dpdk-dev , Olivier Matz , Andrew Rybchenko , Honnappa Nagarahalli , "Ananyev, Konstantin" , Ferruh Yigit , Jerin Jacob , Akhil Goyal , Narcisa Ana Maria Vasile , "Dmitry Malloy (MESHCHANINOV)" , Pallavi Kadam Message-ID: <20210617015806.6452221e@sovereign> In-Reply-To: References: <20210614105839.3379790-1-thomas@monjalon.net> <2004320.XGyPsaEoyj@thomas> <98CBD80474FA8B44BF855DF32C47DC35C61851@smartserver.smartshare.dk> <1857954.7Ex43hCf9S@thomas> <01e1ade6-827f-83c7-5375-125b72888053@intel.com> X-Mailer: Claws Mail 3.17.6 (GTK+ 2.24.32; x86_64-pc-linux-gnu) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable Subject: Re: [dpdk-dev] [PATCH] parray: introduce internal API for dynamic arrays X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" 2021-06-16 18:29 (UTC+0530), Jerin Jacob: > On Wed, Jun 16, 2021 at 5:52 PM Burakov, Anatoly > wrote: > > > > On 16-Jun-21 10:42 AM, Jerin Jacob wrote: =20 > > > On Tue, Jun 15, 2021 at 12:18 PM Thomas Monjalon wrote: =20 > > >> > > >> 14/06/2021 17:48, Morten Br=C3=B8rup: =20 > > >>>> From: dev [mailto:dev-bounces@dpdk.org] On Behalf Of Thomas Monjal= on =20 > > >>> It would be much simpler to just increase RTE_MAX_ETHPORTS to somet= hing big enough to hold a sufficiently large array. And possibly add an rte= _max_ethports variable to indicate the number of populated entries in the a= rray, for use when iterating over the array. > > >>> > > >>> Can we come up with another example than RTE_MAX_ETHPORTS where thi= s library provides a better benefit? =20 > > >> > > >> What is big enough? > > >> Is 640KB enough for RAM? ;) =20 > > > > > > If I understand it correctly, Linux process allocates 640KB due to > > > that fact currently > > > struct rte_eth_dev rte_eth_devices[RTE_MAX_ETHPORTS] is global and it > > > is from BSS. > > > > > > If we make this from heap i.e use malloc() to allocate this memory > > > then in my understanding Linux > > > really won't allocate the real page for backend memory until unless, > > > someone write/read to this memory. > > > > > > i.e it will be free virtual memory using Linux memory management help. > > > If so, we can keep large values for RTE_MAX_ETHPORTS > > > without wasting any "real" memory even though the system has a few po= rts. > > > > > > Thoughts? > > > =20 > > > > mmap works this way with anonymous memory, i'm not so sure about > > malloc()'ed memory. =20 >=20 > Looking at online documentation scatters over the internet, sbrk(), is > based on demand paging. > So I am not sure as well. I am also not sure how we can write some > test case to verify it. > Allocating a huge memory through malloc() not failing, not sure it is > due to demand pagging > or Linux over commit feature or combination of both, >=20 > if mmap works in this way, we could have EAL abstraction for such > memory alloc like > eal_malloc_demand_page() or so and if Windows also supports it. >=20 >=20 >=20 > > Plus, we can't base these decisions on what Linux > > does because we support other OS's. Do they do this as well? =20 >=20 > + Windows OS maintainers Yes, Windows uses demand paging. Is it true that BSS is eagerly allocated (i. e. RAM consumed)? If not, and = it shouldn't be, malloc() isn't needed unless hugepages are required.