From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: <dev-bounces@dpdk.org> Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id EA709A0C44; Mon, 14 Jun 2021 15:32:10 +0200 (CEST) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 677614067E; Mon, 14 Jun 2021 15:32:10 +0200 (CEST) Received: from out1-smtp.messagingengine.com (out1-smtp.messagingengine.com [66.111.4.25]) by mails.dpdk.org (Postfix) with ESMTP id 7E8194067A for <dev@dpdk.org>; Mon, 14 Jun 2021 15:32:09 +0200 (CEST) Received: from compute2.internal (compute2.nyi.internal [10.202.2.42]) by mailout.nyi.internal (Postfix) with ESMTP id 21ADC5C00E0; Mon, 14 Jun 2021 09:32:09 -0400 (EDT) Received: from mailfrontend2 ([10.202.2.163]) by compute2.internal (MEProxy); Mon, 14 Jun 2021 09:32:09 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=monjalon.net; h= from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding:content-type; s=fm1; bh= YcgdhD8hBRhL7AzZ9QoDwXe/3P7a0IlExh/cCytxFqs=; b=CYJchzCTNIu5G+C7 JFIOwWm/6WhL4c1mMD8C5FsbRBJX8575MQJeCSfzpTyEMiZK21M1fHI+l4dyHHpt sdKwK//XXExvUJnBZUh0q+V/h7dZKvnwgRJaYRGmYkoShROVGL2joXDtRMjXA+l0 L3GQ+96Y7kpG6hv7y73q/HMnB8Lxic2gIGDURUUuV1w7aDqSD7SzRaRx2N+4RpPc MYvDNTjTM1pz+2bO4TDamKY2TzatJRQ4DEs8qqubzEJc7CesVM2+3TFe86s+HpTg oPPGng+sY6oyFYkPZmSBr8Qlc9zp5D2NEdzjwSWfD8BlkCIpZxsGAdoyJPageexu wPNabQ== DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d= messagingengine.com; h=cc:content-transfer-encoding:content-type :date:from:in-reply-to:message-id:mime-version:references :subject:to:x-me-proxy:x-me-proxy:x-me-sender:x-me-sender :x-sasl-enc; s=fm3; bh=YcgdhD8hBRhL7AzZ9QoDwXe/3P7a0IlExh/cCytxF qs=; b=IQWjJd2z0VAZQhxxbYNHHqvBkrfFQXc1iV/eJ2LKK5haLC3o0qUxL2XlW h/9YNCDY3g17b4qOsqNl/Gh3X42tHPaLHQ8MDvEnkBmejPyf1OBoI/oMP3D45OFj 9kKuIdRFmS7hUByRtst4OHgJeehOXfNlqUmytSbP7yxokKVrNYFi/mcr5W8Zdmuq 2MGR0LY2bb4QLlfTABM38ZXxGhD47IAQkhRcrzwBjY5XdC1fPiMY+KmsYxnXaDBm 7DGvHJXFyB+JuIjI3mAM/Crpt5w0FAF0uG3w5De1myyajKlQUiGUKf3l5tVOVbnk KIdbky/u1yYbi/Bvh5bH+HdMNkO1g== X-ME-Sender: <xms:V1rHYC6xN2pafHBdgNfEhz650pbsRUZ1vsd6k_p2oRVx1UWtfuMPlg> <xme:V1rHYL7gQ8XgYVhzXNoegYGMcMcLViWzae24v6eUngIax51scMuTuHJW63ck8e6Kh eOha25MOMV3th8H7w> X-ME-Received: <xmr:V1rHYBe9WeTE4qkgtd5JtSd6E6OOzx6enuVg5BUSwDu2RYhIJLg34l41lZKE6yQ7vDT6ThM5VNESaHij8lXps0FpGA> X-ME-Proxy-Cause: gggruggvucftvghtrhhoucdtuddrgeduledrfedvhedgiedvucetufdoteggodetrfdotf fvucfrrhhofhhilhgvmecuhfgrshhtofgrihhlpdfqfgfvpdfurfetoffkrfgpnffqhgen uceurghilhhouhhtmecufedttdenucesvcftvggtihhpihgvnhhtshculddquddttddmne cujfgurhephffvufffkfgjfhgggfgtsehtqhertddttddunecuhfhrohhmpefvhhhomhgr shcuofhonhhjrghlohhnuceothhhohhmrghssehmohhnjhgrlhhonhdrnhgvtheqnecugg ftrfgrthhtvghrnhepfeegffeihfeftedthfdvgfetkeffffdukeevtdevtddvgfevuedu veegvdeggedtnecuvehluhhsthgvrhfuihiivgeptdenucfrrghrrghmpehmrghilhhfrh homhepthhhohhmrghssehmohhnjhgrlhhonhdrnhgvth X-ME-Proxy: <xmx:V1rHYPKYL2KWvnhp4GEPjZQ9gDgUFJ-wZDQxi5oz7dl7VsgSIJ98Pw> <xmx:V1rHYGJPCamE0C4c5BJ5iVV74hI5oeQCpc1wOWiNcejTUWwCBm0mNw> <xmx:V1rHYAx1hwFoUotCcEkdvLght5EEvuziCsfePZ6q5-eEZkt0wLnocg> <xmx:WVrHYF8ZXjTwtGec1L6JqRsGyoGrEworXFcBIRLEqF742GgG26y4-g> Received: by mail.messagingengine.com (Postfix) with ESMTPA; Mon, 14 Jun 2021 09:32:06 -0400 (EDT) From: Thomas Monjalon <thomas@monjalon.net> To: Bruce Richardson <bruce.richardson@intel.com> Cc: Morten =?ISO-8859-1?Q?Br=F8rup?= <mb@smartsharesystems.com>, dev@dpdk.org, olivier.matz@6wind.com, andrew.rybchenko@oktetlabs.ru, honnappa.nagarahalli@arm.com, konstantin.ananyev@intel.com, ferruh.yigit@intel.com, jerinj@marvell.com, gakhil@marvell.com Date: Mon, 14 Jun 2021 15:32:05 +0200 Message-ID: <2004320.XGyPsaEoyj@thomas> In-Reply-To: <YMdWVzsk2+FQkLPJ@bricha3-MOBL.ger.corp.intel.com> References: <20210614105839.3379790-1-thomas@monjalon.net> <98CBD80474FA8B44BF855DF32C47DC35C6184E@smartserver.smartshare.dk> <YMdWVzsk2+FQkLPJ@bricha3-MOBL.ger.corp.intel.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="iso-8859-1" Subject: Re: [dpdk-dev] [PATCH] parray: introduce internal API for dynamic arrays X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions <dev.dpdk.org> List-Unsubscribe: <https://mails.dpdk.org/options/dev>, <mailto:dev-request@dpdk.org?subject=unsubscribe> List-Archive: <http://mails.dpdk.org/archives/dev/> List-Post: <mailto:dev@dpdk.org> List-Help: <mailto:dev-request@dpdk.org?subject=help> List-Subscribe: <https://mails.dpdk.org/listinfo/dev>, <mailto:dev-request@dpdk.org?subject=subscribe> Errors-To: dev-bounces@dpdk.org Sender: "dev" <dev-bounces@dpdk.org> 14/06/2021 15:15, Bruce Richardson: > On Mon, Jun 14, 2021 at 02:22:42PM +0200, Morten Br=F8rup wrote: > > > From: dev [mailto:dev-bounces@dpdk.org] On Behalf Of Thomas Monjalon > > > Sent: Monday, 14 June 2021 12.59 > > >=20 > > > Performance of access in a fixed-size array is very good > > > because of cache locality > > > and because there is a single pointer to dereference. > > > The only drawback is the lack of flexibility: > > > the size of such an array cannot be increase at runtime. > > >=20 > > > An approach to this problem is to allocate the array at runtime, > > > being as efficient as static arrays, but still limited to a maximum. > > >=20 > > > That's why the API rte_parray is introduced, > > > allowing to declare an array of pointer which can be resized > > > dynamically > > > and automatically at runtime while keeping a good read performance. > > >=20 > > > After resize, the previous array is kept until the next resize > > > to avoid crashs during a read without any lock. > > >=20 > > > Each element is a pointer to a memory chunk dynamically allocated. > > > This is not good for cache locality but it allows to keep the same > > > memory per element, no matter how the array is resized. > > > Cache locality could be improved with mempools. > > > The other drawback is having to dereference one more pointer > > > to read an element. > > >=20 > > > There is not much locks, so the API is for internal use only. > > > This API may be used to completely remove some compilation-time > > > maximums. > >=20 > > I get the purpose and overall intention of this library. > >=20 > > I probably already mentioned that I prefer "embedded style programming"= with fixed size arrays, rather than runtime configurability. It's my perso= nal opinion, and the DPDK Tech Board clearly prefers reducing the amount of= compile time configurability, so there is no way for me to stop this progr= ess, and I do not intend to oppose to this library. :-) > >=20 > > This library is likely to become a core library of DPDK, so I think it = is important getting it right. Could you please mention a few examples wher= e you think this internal library should be used, and where it should not b= e used. Then it is easier to discuss if the border line between control pat= h and data plane is correct. E.g. this library is not intended to be used f= or dynamically sized packet queues that grow and shrink in the fast path. > >=20 > > If the library becomes a core DPDK library, it should probably be publi= c instead of internal. E.g. if the library is used to make RTE_MAX_ETHPORTS= dynamic instead of compile time fixed, then some applications might also n= eed dynamically sized arrays for their application specific per-port runtim= e data, and this library could serve that purpose too. > >=20 >=20 > Thanks Thomas for starting this discussion and Morten for follow-up. >=20 > My thinking is as follows, and I'm particularly keeping in mind the cases > of e.g. RTE_MAX_ETHPORTS, as a leading candidate here. >=20 > While I dislike the hard-coded limits in DPDK, I'm also not convinced that > we should switch away from the flat arrays or that we need fully dynamic > arrays that grow/shrink at runtime for ethdevs. I would suggest a half-way > house here, where we keep the ethdevs as an array, but one allocated/sized > at runtime rather than statically. This would allow us to have a > compile-time default value, but, for use cases that need it, allow use of= a > flag e.g. "max-ethdevs" to change the size of the parameter given to the > malloc call for the array. This max limit could then be provided to apps > too if they want to match any array sizes. [Alternatively those apps could > check the provided size and error out if the size has been increased beyo= nd > what the app is designed to use?]. There would be no extra dereferences p= er > rx/tx burst call in this scenario so performance should be the same as > before (potentially better if array is in hugepage memory, I suppose). I think we need some benchmarks to decide what is the best tradeoff. I spent time on this implementation, but sorry I won't have time for benchm= arks. Volunteers?