From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id A7446A0C44; Mon, 14 Jun 2021 15:15:15 +0200 (CEST) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id D249C4067A; Mon, 14 Jun 2021 15:15:14 +0200 (CEST) Received: from mga03.intel.com (mga03.intel.com [134.134.136.65]) by mails.dpdk.org (Postfix) with ESMTP id 9CB254003F for ; Mon, 14 Jun 2021 15:15:12 +0200 (CEST) IronPort-SDR: q5FEUtsHC+Q+aeiKP3Z6Z/7A2krlYb8ATOm/FimUGV6/Pu5cuGi5FyG9lvh7yLINgWDRkbgb/d qSgbdeX7PHww== X-IronPort-AV: E=McAfee;i="6200,9189,10014"; a="205838517" X-IronPort-AV: E=Sophos;i="5.83,273,1616482800"; d="scan'208";a="205838517" Received: from orsmga005.jf.intel.com ([10.7.209.41]) by orsmga103.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 14 Jun 2021 06:15:10 -0700 IronPort-SDR: EzfBz+cQNKwZGdYDTJHNUJe6DOhleeEqSBB6Mr/Ch5af5hfHXc4uzmh0tQsHNgqjri4vAUf4ua O8L6GI6K472g== X-IronPort-AV: E=Sophos;i="5.83,273,1616482800"; d="scan'208";a="621033696" Received: from bricha3-mobl.ger.corp.intel.com ([10.252.11.163]) by orsmga005-auth.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-SHA; 14 Jun 2021 06:15:07 -0700 Date: Mon, 14 Jun 2021 14:15:03 +0100 From: Bruce Richardson To: Morten =?iso-8859-1?Q?Br=F8rup?= Cc: Thomas Monjalon , dev@dpdk.org, olivier.matz@6wind.com, andrew.rybchenko@oktetlabs.ru, honnappa.nagarahalli@arm.com, konstantin.ananyev@intel.com, ferruh.yigit@intel.com, jerinj@marvell.com, gakhil@marvell.com Message-ID: References: <20210614105839.3379790-1-thomas@monjalon.net> <98CBD80474FA8B44BF855DF32C47DC35C6184E@smartserver.smartshare.dk> MIME-Version: 1.0 Content-Type: text/plain; charset=iso-8859-1 Content-Disposition: inline Content-Transfer-Encoding: 8bit In-Reply-To: <98CBD80474FA8B44BF855DF32C47DC35C6184E@smartserver.smartshare.dk> Subject: Re: [dpdk-dev] [PATCH] parray: introduce internal API for dynamic arrays X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" On Mon, Jun 14, 2021 at 02:22:42PM +0200, Morten Brørup wrote: > > From: dev [mailto:dev-bounces@dpdk.org] On Behalf Of Thomas Monjalon > > Sent: Monday, 14 June 2021 12.59 > > > > Performance of access in a fixed-size array is very good > > because of cache locality > > and because there is a single pointer to dereference. > > The only drawback is the lack of flexibility: > > the size of such an array cannot be increase at runtime. > > > > An approach to this problem is to allocate the array at runtime, > > being as efficient as static arrays, but still limited to a maximum. > > > > That's why the API rte_parray is introduced, > > allowing to declare an array of pointer which can be resized > > dynamically > > and automatically at runtime while keeping a good read performance. > > > > After resize, the previous array is kept until the next resize > > to avoid crashs during a read without any lock. > > > > Each element is a pointer to a memory chunk dynamically allocated. > > This is not good for cache locality but it allows to keep the same > > memory per element, no matter how the array is resized. > > Cache locality could be improved with mempools. > > The other drawback is having to dereference one more pointer > > to read an element. > > > > There is not much locks, so the API is for internal use only. > > This API may be used to completely remove some compilation-time > > maximums. > > I get the purpose and overall intention of this library. > > I probably already mentioned that I prefer "embedded style programming" with fixed size arrays, rather than runtime configurability. It's my personal opinion, and the DPDK Tech Board clearly prefers reducing the amount of compile time configurability, so there is no way for me to stop this progress, and I do not intend to oppose to this library. :-) > > This library is likely to become a core library of DPDK, so I think it is important getting it right. Could you please mention a few examples where you think this internal library should be used, and where it should not be used. Then it is easier to discuss if the border line between control path and data plane is correct. E.g. this library is not intended to be used for dynamically sized packet queues that grow and shrink in the fast path. > > If the library becomes a core DPDK library, it should probably be public instead of internal. E.g. if the library is used to make RTE_MAX_ETHPORTS dynamic instead of compile time fixed, then some applications might also need dynamically sized arrays for their application specific per-port runtime data, and this library could serve that purpose too. > Thanks Thomas for starting this discussion and Morten for follow-up. My thinking is as follows, and I'm particularly keeping in mind the cases of e.g. RTE_MAX_ETHPORTS, as a leading candidate here. While I dislike the hard-coded limits in DPDK, I'm also not convinced that we should switch away from the flat arrays or that we need fully dynamic arrays that grow/shrink at runtime for ethdevs. I would suggest a half-way house here, where we keep the ethdevs as an array, but one allocated/sized at runtime rather than statically. This would allow us to have a compile-time default value, but, for use cases that need it, allow use of a flag e.g. "max-ethdevs" to change the size of the parameter given to the malloc call for the array. This max limit could then be provided to apps too if they want to match any array sizes. [Alternatively those apps could check the provided size and error out if the size has been increased beyond what the app is designed to use?]. There would be no extra dereferences per rx/tx burst call in this scenario so performance should be the same as before (potentially better if array is in hugepage memory, I suppose). Regards, /Bruce