From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 576CAA04FD; Thu, 6 Oct 2022 22:15:39 +0200 (CEST) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 0DA08400D6; Thu, 6 Oct 2022 22:15:39 +0200 (CEST) Received: from mail.lysator.liu.se (mail.lysator.liu.se [130.236.254.3]) by mails.dpdk.org (Postfix) with ESMTP id 8731A40042 for ; Thu, 6 Oct 2022 22:15:37 +0200 (CEST) Received: from mail.lysator.liu.se (localhost [127.0.0.1]) by mail.lysator.liu.se (Postfix) with ESMTP id 2564911D09 for ; Thu, 6 Oct 2022 22:15:37 +0200 (CEST) Received: by mail.lysator.liu.se (Postfix, from userid 1004) id 2455D11AB8; Thu, 6 Oct 2022 22:15:37 +0200 (CEST) X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on hermod.lysator.liu.se X-Spam-Level: X-Spam-Status: No, score=-2.2 required=5.0 tests=ALL_TRUSTED, AWL, NICE_REPLY_A autolearn=disabled version=3.4.6 X-Spam-Score: -2.2 Received: from [192.168.1.59] (h-62-63-215-114.A163.priv.bahnhof.se [62.63.215.114]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange ECDHE (P-256) server-signature RSA-PSS (2048 bits)) (No client certificate requested) by mail.lysator.liu.se (Postfix) with ESMTPSA id 3BA5711C0A; Thu, 6 Oct 2022 22:15:34 +0200 (CEST) Message-ID: <7ea65099-1b7b-8e43-a956-0969ee7425ee@lysator.liu.se> Date: Thu, 6 Oct 2022 22:15:34 +0200 MIME-Version: 1.0 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Thunderbird/102.2.2 Subject: Re: [PATCH v5 01/10] memarea: introduce memarea library Content-Language: en-US To: datshan , david.marchand@redhat.com, mb@smartsharesystems.com, anatoly.burakov@intel.com, dmitry.kozliuk@gmail.com, jerinjacobk@gmail.com Cc: thomas@monjalon.net, dev@dpdk.org, Chengwen Feng References: <20220721044648.6817-1-fengchengwen@huawei.com> <20221005040952.8166-1-datshan@qq.com> From: =?UTF-8?Q?Mattias_R=c3=b6nnblom?= In-Reply-To: Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 7bit X-Virus-Scanned: ClamAV using ClamSMTP X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org On 2022-10-05 06:09, datshan wrote: > From: Chengwen Feng > > The memarea library is an allocator of variable-size object which based > on a memory region. > > This patch provides create/destroy API. > > Signed-off-by: Chengwen Feng > --- > MAINTAINERS | 5 + > doc/api/doxy-api-index.md | 3 +- > doc/api/doxy-api.conf.in | 1 + > doc/guides/prog_guide/index.rst | 1 + > doc/guides/prog_guide/memarea_lib.rst | 39 ++++++ > doc/guides/rel_notes/release_22_11.rst | 6 + > lib/eal/common/eal_common_log.c | 1 + > lib/eal/include/rte_log.h | 1 + > lib/memarea/memarea_private.h | 30 +++++ > lib/memarea/meson.build | 16 +++ > lib/memarea/rte_memarea.c | 157 +++++++++++++++++++++++++ > lib/memarea/rte_memarea.h | 145 +++++++++++++++++++++++ > lib/memarea/version.map | 12 ++ > lib/meson.build | 1 + > 14 files changed, 417 insertions(+), 1 deletion(-) > create mode 100644 doc/guides/prog_guide/memarea_lib.rst > create mode 100644 lib/memarea/memarea_private.h > create mode 100644 lib/memarea/meson.build > create mode 100644 lib/memarea/rte_memarea.c > create mode 100644 lib/memarea/rte_memarea.h > create mode 100644 lib/memarea/version.map > > diff --git a/MAINTAINERS b/MAINTAINERS > index a55b379d73..b9c638221d 100644 > --- a/MAINTAINERS > +++ b/MAINTAINERS > @@ -1550,6 +1550,11 @@ F: app/test/test_lpm* > F: app/test/test_func_reentrancy.c > F: app/test/test_xmmt_ops.h > > +Memarea - EXPERIMENTAL > +M: Chengwen Feng > +F: lib/memarea > +F: doc/guides/prog_guide/memarea_lib.rst > + > Membership - EXPERIMENTAL > M: Yipeng Wang > M: Sameh Gobriel > diff --git a/doc/api/doxy-api-index.md b/doc/api/doxy-api-index.md > index de488c7abf..24456604f8 100644 > --- a/doc/api/doxy-api-index.md > +++ b/doc/api/doxy-api-index.md > @@ -62,7 +62,8 @@ The public API headers are grouped by topics: > [memzone](@ref rte_memzone.h), > [mempool](@ref rte_mempool.h), > [malloc](@ref rte_malloc.h), > - [memcpy](@ref rte_memcpy.h) > + [memcpy](@ref rte_memcpy.h), > + [memarea](@ref rte_memarea.h) > > - **timers**: > [cycles](@ref rte_cycles.h), > diff --git a/doc/api/doxy-api.conf.in b/doc/api/doxy-api.conf.in > index f0886c3bd1..8334ebcbd6 100644 > --- a/doc/api/doxy-api.conf.in > +++ b/doc/api/doxy-api.conf.in > @@ -53,6 +53,7 @@ INPUT = @TOPDIR@/doc/api/doxy-api-index.md \ > @TOPDIR@/lib/latencystats \ > @TOPDIR@/lib/lpm \ > @TOPDIR@/lib/mbuf \ > + @TOPDIR@/lib/memarea \ > @TOPDIR@/lib/member \ > @TOPDIR@/lib/mempool \ > @TOPDIR@/lib/meter \ > diff --git a/doc/guides/prog_guide/index.rst b/doc/guides/prog_guide/index.rst > index 8564883018..e9015d65e3 100644 > --- a/doc/guides/prog_guide/index.rst > +++ b/doc/guides/prog_guide/index.rst > @@ -37,6 +37,7 @@ Programmer's Guide > hash_lib > toeplitz_hash_lib > efd_lib > + memarea_lib > member_lib > lpm_lib > lpm6_lib > diff --git a/doc/guides/prog_guide/memarea_lib.rst b/doc/guides/prog_guide/memarea_lib.rst > new file mode 100644 > index 0000000000..b96dad15f6 > --- /dev/null > +++ b/doc/guides/prog_guide/memarea_lib.rst > @@ -0,0 +1,39 @@ > +.. SPDX-License-Identifier: BSD-3-Clause > + Copyright(c) 2022 HiSilicon Limited > + > +Memarea Library > +=============== > + > +Introduction > +------------ > + > +The memarea library provides an allocator of variable-size objects, it is > +oriented towards the application layer, which could provides 'region-based > +memory management' function [1]. > + > +The main features are as follows: > + > +* The default aligement size is ``RTE_CACHE_LINE_SIZE``. > + > +* The memory region can be initialized from the following memory sources: > + a) RTE memory: e.g. invoke ``rte_malloc_socket`` to obtain. b) System API: > + e.g. invoke posix_memalign to obtain. c) User provided address: it can be from > + extendedd memory as long as it is available. d) User provided memarea: it can > + be from another memarea. > + > +* It provides refcnt feature which could be useful in multi-reader scenario. > + > +* It supports MT-safe as long as it's specified at creation time. > + > +Library API Overview > +-------------------- > + > +The ``rte_memarea_create()`` function is used to create a memarea, the function > +returns the pointer to the created memarea or ``NULL`` if the creation failed. > + > +The ``rte_memarea_destroy()`` function is used to destroy a memarea. > + > +Reference > +--------- > + > +[1] https://en.wikipedia.org/wiki/Region-based_memory_management > diff --git a/doc/guides/rel_notes/release_22_11.rst b/doc/guides/rel_notes/release_22_11.rst > index 5d8ef669b8..4c1f760b98 100644 > --- a/doc/guides/rel_notes/release_22_11.rst > +++ b/doc/guides/rel_notes/release_22_11.rst > @@ -55,6 +55,12 @@ New Features > Also, make sure to start the actual text at the margin. > ======================================================= > > +* **Added memarea library.** > + > + The memarea library is an allocator of variable-size objects, it is oriented > + towards the application layer, which could provides 'region-based memory > + management' function. > + > * **Added configuration for asynchronous flow connection tracking.** > > Added connection tracking action number hint to ``rte_flow_configure`` > diff --git a/lib/eal/common/eal_common_log.c b/lib/eal/common/eal_common_log.c > index bd7b188ceb..3d62af59c6 100644 > --- a/lib/eal/common/eal_common_log.c > +++ b/lib/eal/common/eal_common_log.c > @@ -369,6 +369,7 @@ static const struct logtype logtype_strings[] = { > {RTE_LOGTYPE_EFD, "lib.efd"}, > {RTE_LOGTYPE_EVENTDEV, "lib.eventdev"}, > {RTE_LOGTYPE_GSO, "lib.gso"}, > + {RTE_LOGTYPE_MEMAREA, "lib.memarea"}, > {RTE_LOGTYPE_USER1, "user1"}, > {RTE_LOGTYPE_USER2, "user2"}, > {RTE_LOGTYPE_USER3, "user3"}, > diff --git a/lib/eal/include/rte_log.h b/lib/eal/include/rte_log.h > index 25ce42cdfc..708f3a39dd 100644 > --- a/lib/eal/include/rte_log.h > +++ b/lib/eal/include/rte_log.h > @@ -48,6 +48,7 @@ extern "C" { > #define RTE_LOGTYPE_EFD 18 /**< Log related to EFD. */ > #define RTE_LOGTYPE_EVENTDEV 19 /**< Log related to eventdev. */ > #define RTE_LOGTYPE_GSO 20 /**< Log related to GSO. */ > +#define RTE_LOGTYPE_MEMAREA 21 /**< Log related to memarea. */ > > /* these log types can be used in an application */ > #define RTE_LOGTYPE_USER1 24 /**< User-defined log type 1. */ > diff --git a/lib/memarea/memarea_private.h b/lib/memarea/memarea_private.h > new file mode 100644 > index 0000000000..c76392d3e6 > --- /dev/null > +++ b/lib/memarea/memarea_private.h > @@ -0,0 +1,30 @@ > +/* SPDX-License-Identifier: BSD-3-Clause > + * Copyright(c) 2022 HiSilicon Limited > + */ > + > +#ifndef MEMAREA_PRIVATE_H > +#define MEMAREA_PRIVATE_H > + > +#include > + > +#define MEMAREA_FREE_ELEM_COOKIE 0xFFFFFFFF > + > +struct memarea_elem { > + size_t size; > + uint32_t cookie; > + int32_t refcnt; /* Non-zero indicates that it has been allocated */ > + TAILQ_ENTRY(memarea_elem) elem_node; > + TAILQ_ENTRY(memarea_elem) free_node; > +} __rte_cache_aligned; > + Why is the elem type cache line aligned? Need the elem data start be cache line aligned? > +TAILQ_HEAD(memarea_elem_list, memarea_elem); > + > +struct rte_memarea { > + struct rte_memarea_param init; > + rte_spinlock_t lock; > + void *area_addr; > + struct memarea_elem_list elem_list; > + struct memarea_elem_list free_list; > +} __rte_cache_aligned; > + > +#endif /* MEMAREA_PRIVATE_H */ > diff --git a/lib/memarea/meson.build b/lib/memarea/meson.build > new file mode 100644 > index 0000000000..0a74fb4cd1 > --- /dev/null > +++ b/lib/memarea/meson.build > @@ -0,0 +1,16 @@ > +# SPDX-License-Identifier: BSD-3-Clause > +# Copyright(c) 2022 HiSilicon Limited > + > +if is_windows > + build = false > + reason = 'not supported on Windows' > + subdir_done() > +endif > + > +sources = files( > + 'rte_memarea.c', > +) > +headers = files( > + 'rte_memarea.h', > +) > +deps += [] > diff --git a/lib/memarea/rte_memarea.c b/lib/memarea/rte_memarea.c > new file mode 100644 > index 0000000000..868da7661d > --- /dev/null > +++ b/lib/memarea/rte_memarea.c > @@ -0,0 +1,157 @@ > +/* SPDX-License-Identifier: BSD-3-Clause > + * Copyright(c) 2022 HiSilicon Limited > + */ > + > +#include > +#include > + > +#include > +#include > +#include > +#include > + > +#include "rte_memarea.h" > +#include "memarea_private.h" > + > +static int > +memarea_check_param(const struct rte_memarea_param *init) > +{ > + size_t len; > + > + if (init == NULL) { > + RTE_LOG(ERR, MEMAREA, "memarea init param is NULL!\n"); > + return -EINVAL; > + } > + > + len = strnlen(init->name, RTE_MEMAREA_NAMESIZE); > + if (len == 0 || len >= RTE_MEMAREA_NAMESIZE) { > + RTE_LOG(ERR, MEMAREA, "memarea name size %zu invalid!\n", len); > + return -EINVAL; > + } > + > + if (init->source != RTE_MEMAREA_SOURCE_RTE_MEMORY && > + init->source != RTE_MEMAREA_SOURCE_SYSTEM_API && > + init->source != RTE_MEMAREA_SOURCE_USER_ADDR && > + init->source != RTE_MEMAREA_SOURCE_USER_MEMAREA) { > + RTE_LOG(ERR, MEMAREA, "memarea: %s source: %d not supported!\n", > + init->name, init->source); > + return -EINVAL; > + } > + > + if (init->total_sz <= sizeof(struct memarea_elem)) { > + RTE_LOG(ERR, MEMAREA, "memarea: %s total-size: %zu too small!\n", > + init->name, init->total_sz); > + return -EINVAL; > + } > + > + if (init->source == RTE_MEMAREA_SOURCE_USER_ADDR && init->user_addr == NULL) { > + RTE_LOG(ERR, MEMAREA, "memarea: %s user provided addr is NULL!\n", init->name); > + return -EINVAL; > + } > + > + if (init->source == RTE_MEMAREA_SOURCE_USER_ADDR && > + ((uintptr_t)init->user_addr & (RTE_CACHE_LINE_SIZE - 1))) { > + RTE_LOG(ERR, MEMAREA, "memarea: %s user provided addr should align: %d!\n", > + init->name, RTE_CACHE_LINE_SIZE); > + return -EINVAL; > + } > + > + if (init->source == RTE_MEMAREA_SOURCE_USER_MEMAREA && init->user_memarea == NULL) { > + RTE_LOG(ERR, MEMAREA, "memarea: %s user provided memarea is NULL!\n", init->name); > + return -EINVAL; > + } > + > + if (init->alg != RTE_MEMAREA_ALG_DEFAULT) { > + RTE_LOG(ERR, MEMAREA, "memarea: %s alg: %d not supported!\n", > + init->name, init->alg); > + return -EINVAL; > + } > + > + return 0; > +} > + > +static void * > +memarea_alloc_from_system_api(size_t size) > +{ > + void *ptr = NULL; > + int ret; > + > + ret = posix_memalign(&ptr, RTE_CACHE_LINE_SIZE, size); > + if (ret) > + return NULL; > + return ptr; > +} > + > +static void * > +memarea_alloc_area(const struct rte_memarea_param *init) > +{ > + void *ptr = NULL; > + > + if (init->source == RTE_MEMAREA_SOURCE_RTE_MEMORY) Delete MEMORY. Of course it's memory. What else? If you want to make it clear it's from the RTE heap, it should spell out HEAP. Or MALLOC. > + ptr = rte_malloc_socket(NULL, init->total_sz, RTE_CACHE_LINE_SIZE, > + init->numa_socket); > + else if (init->source == RTE_MEMAREA_SOURCE_SYSTEM_API) > + ptr = memarea_alloc_from_system_api(init->total_sz); "SYSTEM_API" doesn't strike me as a good name. RTE_MEMAREA_SOURCE_LIBC RTE_MEMAREA_SOURCE_STD_HEAP or at least remove API so it'll be RTE_MEMAREA_SOURCE_SYSTEM > + else if (init->source == RTE_MEMAREA_SOURCE_USER_ADDR) I would delete "ADDR". > + ptr = init->user_addr; > + > + if (ptr == NULL) > + RTE_LOG(ERR, MEMAREA, "memarea: %s alloc memory area fail!\n", init->name); > + > + return ptr; > +} > + > +struct rte_memarea * > +rte_memarea_create(const struct rte_memarea_param *init) > +{ > + struct memarea_elem *elem; > + struct rte_memarea *ma; > + void *addr; > + int ret; > + > + ret = memarea_check_param(init); > + if (ret) > + return NULL; > + > + addr = memarea_alloc_area(init); > + if (addr == NULL) > + return NULL; > + > + ma = rte_zmalloc(NULL, sizeof(struct rte_memarea), RTE_CACHE_LINE_SIZE); > + if (ma == NULL) { > + RTE_LOG(ERR, MEMAREA, "malloc memarea: %s management obj fail!\n", init->name); > + return NULL; > + } > + > + ma->init = *init; > + rte_spinlock_init(&ma->lock); > + TAILQ_INIT(&ma->elem_list); > + TAILQ_INIT(&ma->free_list); > + ma->area_addr = addr; > + elem = addr; > + elem->size = init->total_sz - sizeof(struct memarea_elem); > + elem->cookie = MEMAREA_FREE_ELEM_COOKIE; > + elem->refcnt = 0; > + TAILQ_INSERT_TAIL(&ma->elem_list, elem, elem_node); > + TAILQ_INSERT_TAIL(&ma->free_list, elem, free_node); > + > + return ma; > +} > + > +static void > +memarea_free_area(struct rte_memarea *ma) > +{ > + if (ma->init.source == RTE_MEMAREA_SOURCE_RTE_MEMORY) > + rte_free(ma->area_addr); > + else if (ma->init.source == RTE_MEMAREA_SOURCE_SYSTEM_API) > + free(ma->area_addr); > +} > + > +void > +rte_memarea_destroy(struct rte_memarea *ma) > +{ > + if (ma == NULL) > + return; > + memarea_free_area(ma); > + rte_free(ma); > +} > diff --git a/lib/memarea/rte_memarea.h b/lib/memarea/rte_memarea.h > new file mode 100644 > index 0000000000..543cda4cac > --- /dev/null > +++ b/lib/memarea/rte_memarea.h > @@ -0,0 +1,145 @@ > +/* SPDX-License-Identifier: BSD-3-Clause > + * Copyright(c) 2022 HiSilicon Limited > + */ > + > +#ifndef RTE_MEMAREA_H > +#define RTE_MEMAREA_H > + > +/** > + * @file > + * RTE Memarea. > + * > + * The memarea is an allocator of variable-size object which based on a memory > + * region. It has the following features: > + * > + * - The default alignment size is RTE_CACHE_LINE_SIZE. This can be read as two things: the object size is aligned, or the start address is aligned. > + * - The memory region can be initialized from the following memory sources: > + * 1. RTE memory: e.g. invoke rte_malloc_socket to obtain. Remove "to obtain", or add "memory" after "obtain". Do you really mean "e.g.", and not "i.e."? > + * 2. System API: e.g. invoke posix_memalign to obtain. > + * 3. User provided address: it can be from extended memory as long as it is > + * available. The address must be aligned to RTE_CACHE_LINE_SIZE. What is extended memory? > + * 4) User provided memarea: it can be from another memarea. So we can build > + * the following memory management structure: > + * \code{.unparsed} > + * ------------- > + * | memarea-1 | > + * ------------- > + * | > + * v > + * ------------------------------- > + * | | | > + * v v v > + * ------------- ------------- ---------- > + * | memarea-2 | | memarea-3 | | object | > + * ------------- ------------- ---------- > + * \endcode > + * As shown above, the memarea-2/3 both create from memarea-1's memory. > + * - It provides refcnt feature which could be useful in multi-reader scenario. > + * - It supports MT-safe as long as it's specified at creation time. If not > + * specified, all the functions of the memarea API are lock-free, and assume > + * to not be invoked in parallel on different logical cores to work on the > + * same memarea. > + */ > + > +#include > +#include > +#include > + > +#include > + > +#ifdef __cplusplus > +extern "C" { > +#endif > + > +#define RTE_MEMAREA_NAMESIZE 64 > + > +/** > + * Memarea memory source. > + */ > +enum rte_memarea_source { > + /** Memory source comes from rte memory. */ > + RTE_MEMAREA_SOURCE_RTE_MEMORY, > + /** Memory source comes from system API. */ > + RTE_MEMAREA_SOURCE_SYSTEM_API, > + /** Memory source comes from user-provided address. */ > + RTE_MEMAREA_SOURCE_USER_ADDR, > + /** Memory source comes from user-provided memarea. */ > + RTE_MEMAREA_SOURCE_USER_MEMAREA, > +}; > + > +/** > + * Memarea memory management algorithm. > + */ > +enum rte_memarea_alg { > + /* The default management algorithm is a variant of the next fit > + * algorithm. It uses a free-list to apply for memory and uses an > + * element-list in ascending order of address to support merging > + * upon free. > + */ > + RTE_MEMAREA_ALG_DEFAULT, > +}; > + > +struct rte_memarea; > + > +struct rte_memarea_param { Does this struct need to be public? > + char name[RTE_MEMAREA_NAMESIZE]; /**< Name of memarea. */ > + enum rte_memarea_source source; /**< Memory source of memarea. */ > + enum rte_memarea_alg alg; /**< Memory management algorithm. */ > + size_t total_sz; /**< total size (bytes) of memarea. */ > + /** Indicates whether the memarea API should be MT-safe. */ > + uint32_t mt_safe : 1; Why not bool? > + union { > + /** Numa socket from which to apply for memarea's memory, this > + * field is valid only when the source is set to be > + * RTE_MEMAREA_SOURCE_RTE_MEMORY. > + */ > + int numa_socket; > + /** User provided address, this field is valid only when the > + * source is set to be RTE_MEMAREA_SOURCE_USER_ADDR. > + * Note: the provided address must align at least > + * RTE_CACHE_LINE_SIZE. > + */ > + void *user_addr; > + /** User provided memarea, this field is valid only when the > + * source is set to be RTE_MEMAREA_SOURCE_USER_MEMAREA. > + */ > + struct rte_memarea *user_memarea; > + }; > +}; > + > +/** > + * @warning > + * @b EXPERIMENTAL: this API may change without prior notice. > + * > + * Create memarea. > + * > + * Create one new memarea. > + * > + * @param init > + * The init parameter of memarea. > + * > + * @return > + * Non-NULL on success. Otherwise NULL is returned. > + */ > +__rte_experimental > +struct rte_memarea *rte_memarea_create(const struct rte_memarea_param *init); > + > +/** > + * @warning > + * @b EXPERIMENTAL: this API may change without prior notice. > + * > + * Destroy memarea. > + * > + * Destroy the memarea. > + * > + * @param ma > + * The pointer of memarea. > + */ > +__rte_experimental > +void rte_memarea_destroy(struct rte_memarea *ma); > + > +#ifdef __cplusplus > +} > +#endif > + > +#endif /* RTE_MEMAREA_H */ > diff --git a/lib/memarea/version.map b/lib/memarea/version.map > new file mode 100644 > index 0000000000..f36a04d7cf > --- /dev/null > +++ b/lib/memarea/version.map > @@ -0,0 +1,12 @@ > +EXPERIMENTAL { > + global: > + > + rte_memarea_create; > + rte_memarea_destroy; > + > + local: *; > +}; > + > +INTERNAL { > + local: *; > +}; > diff --git a/lib/meson.build b/lib/meson.build > index c648f7d800..521a25d6c0 100644 > --- a/lib/meson.build > +++ b/lib/meson.build > @@ -42,6 +42,7 @@ libraries = [ > 'kni', > 'latencystats', > 'lpm', > + 'memarea', > 'member', > 'pcapng', > 'power',