Re: [PATCH v5 01/10] memarea: introduce memarea library

["Mattias Rönnblom" <hofors@lysator.liu.se>]

On 2022-10-11 01:33, fengchengwen wrote:
> On 2022/10/11 0:53, Mattias Rönnblom wrote:
>> On 2022-10-08 09:53, fengchengwen wrote:
>>> Hi Mattias, Thanks for your review, most will fix in v6.
>>>
>>> On 2022/10/7 4:15, Mattias Rönnblom wrote:
>>>> On 2022-10-05 06:09, datshan wrote:
>>>>> From: Chengwen Feng <fengchengwen@huawei.com>
>>>>>
>>>>> The memarea library is an allocator of variable-size object which 
>>>>> based
>>>>> on a memory region.
>>>>>
>>>>> This patch provides create/destroy API.
>>>>>
>>>>> Signed-off-by: Chengwen Feng <fengchengwen@huawei.com>
>>>>> ---
>>>>>    MAINTAINERS                            |   5 +
>>>>>    doc/api/doxy-api-index.md              |   3 +-
>>>>>    doc/api/doxy-api.conf.in               |   1 +
>>>>>    doc/guides/prog_guide/index.rst        |   1 +
>>>>>    doc/guides/prog_guide/memarea_lib.rst  |  39 ++++++
>>>>>    doc/guides/rel_notes/release_22_11.rst |   6 +
>>>>>    lib/eal/common/eal_common_log.c        |   1 +
>>>>>    lib/eal/include/rte_log.h              |   1 +
>>>>>    lib/memarea/memarea_private.h          |  30 +++++
>>>>>    lib/memarea/meson.build                |  16 +++
>>>>>    lib/memarea/rte_memarea.c              | 157 
>>>>> +++++++++++++++++++++++++
>>>>>    lib/memarea/rte_memarea.h              | 145 
>>>>> +++++++++++++++++++++++
>>>>>    lib/memarea/version.map                |  12 ++
>>>>>    lib/meson.build                        |   1 +
>>>>>    14 files changed, 417 insertions(+), 1 deletion(-)
>>>>>    create mode 100644 doc/guides/prog_guide/memarea_lib.rst
>>>>>    create mode 100644 lib/memarea/memarea_private.h
>>>>>    create mode 100644 lib/memarea/meson.build
>>>>>    create mode 100644 lib/memarea/rte_memarea.c
>>>>>    create mode 100644 lib/memarea/rte_memarea.h
>>>>>    create mode 100644 lib/memarea/version.map
>>>>>
>>>>> diff --git a/MAINTAINERS b/MAINTAINERS
>>>>> index a55b379d73..b9c638221d 100644
>>>>> --- a/MAINTAINERS
>>>>> +++ b/MAINTAINERS
>>>>> @@ -1550,6 +1550,11 @@ F: app/test/test_lpm*
>>>>>    F: app/test/test_func_reentrancy.c
>>>>>    F: app/test/test_xmmt_ops.h
>>>>>    +Memarea - EXPERIMENTAL
>>>>> +M: Chengwen Feng <fengchengwen@huawei.com>
>>>>> +F: lib/memarea
>>>>> +F: doc/guides/prog_guide/memarea_lib.rst
>>>>> +
>>>>>    Membership - EXPERIMENTAL
>>>>>    M: Yipeng Wang <yipeng1.wang@intel.com>
>>>>>    M: Sameh Gobriel <sameh.gobriel@intel.com>
>>>>> diff --git a/doc/api/doxy-api-index.md b/doc/api/doxy-api-index.md
>>>>> index de488c7abf..24456604f8 100644
>>>>> --- a/doc/api/doxy-api-index.md
>>>>> +++ b/doc/api/doxy-api-index.md
>>>>> @@ -62,7 +62,8 @@ The public API headers are grouped by topics:
>>>>>      [memzone](@ref rte_memzone.h),
>>>>>      [mempool](@ref rte_mempool.h),
>>>>>      [malloc](@ref rte_malloc.h),
>>>>> -  [memcpy](@ref rte_memcpy.h)
>>>>> +  [memcpy](@ref rte_memcpy.h),
>>>>> +  [memarea](@ref rte_memarea.h)
>>>>>      - **timers**:
>>>>>      [cycles](@ref rte_cycles.h),
>>>>> diff --git a/doc/api/doxy-api.conf.in b/doc/api/doxy-api.conf.in
>>>>> index f0886c3bd1..8334ebcbd6 100644
>>>>> --- a/doc/api/doxy-api.conf.in
>>>>> +++ b/doc/api/doxy-api.conf.in
>>>>> @@ -53,6 +53,7 @@ INPUT                   = 
>>>>> @TOPDIR@/doc/api/doxy-api-index.md \
>>>>>                              @TOPDIR@/lib/latencystats \
>>>>>                              @TOPDIR@/lib/lpm \
>>>>>                              @TOPDIR@/lib/mbuf \
>>>>> +                          @TOPDIR@/lib/memarea \
>>>>>                              @TOPDIR@/lib/member \
>>>>>                              @TOPDIR@/lib/mempool \
>>>>>                              @TOPDIR@/lib/meter \
>>>>> diff --git a/doc/guides/prog_guide/index.rst 
>>>>> b/doc/guides/prog_guide/index.rst
>>>>> index 8564883018..e9015d65e3 100644
>>>>> --- a/doc/guides/prog_guide/index.rst
>>>>> +++ b/doc/guides/prog_guide/index.rst
>>>>> @@ -37,6 +37,7 @@ Programmer's Guide
>>>>>        hash_lib
>>>>>        toeplitz_hash_lib
>>>>>        efd_lib
>>>>> +    memarea_lib
>>>>>        member_lib
>>>>>        lpm_lib
>>>>>        lpm6_lib
>>>>> diff --git a/doc/guides/prog_guide/memarea_lib.rst 
>>>>> b/doc/guides/prog_guide/memarea_lib.rst
>>>>> new file mode 100644
>>>>> index 0000000000..b96dad15f6
>>>>> --- /dev/null
>>>>> +++ b/doc/guides/prog_guide/memarea_lib.rst
>>>>> @@ -0,0 +1,39 @@
>>>>> +..  SPDX-License-Identifier: BSD-3-Clause
>>>>> +    Copyright(c) 2022 HiSilicon Limited
>>>>> +
>>>>> +Memarea Library
>>>>> +===============
>>>>> +
>>>>> +Introduction
>>>>> +------------
>>>>> +
>>>>> +The memarea library provides an allocator of variable-size 
>>>>> objects, it is
>>>>> +oriented towards the application layer, which could provides 
>>>>> 'region-based
>>>>> +memory management' function [1].
>>>>> +
>>>>> +The main features are as follows:
>>>>> +
>>>>> +* The default aligement size is ``RTE_CACHE_LINE_SIZE``.
>>>>> +
>>>>> +* The memory region can be initialized from the following memory 
>>>>> sources:
>>>>> +  a) RTE memory: e.g. invoke ``rte_malloc_socket`` to obtain. b) 
>>>>> System API:
>>>>> +  e.g. invoke posix_memalign to obtain. c) User provided address: 
>>>>> it can be from
>>>>> +  extendedd memory as long as it is available. d) User provided 
>>>>> memarea: it can
>>>>> +  be from another memarea.
>>>>> +
>>>>> +* It provides refcnt feature which could be useful in multi-reader 
>>>>> scenario.
>>>>> +
>>>>> +* It supports MT-safe as long as it's specified at creation time.
>>>>> +
>>>>> +Library API Overview
>>>>> +--------------------
>>>>> +
>>>>> +The ``rte_memarea_create()`` function is used to create a memarea, 
>>>>> the function
>>>>> +returns the pointer to the created memarea or ``NULL`` if the 
>>>>> creation failed.
>>>>> +
>>>>> +The ``rte_memarea_destroy()`` function is used to destroy a memarea.
>>>>> +
>>>>> +Reference
>>>>> +---------
>>>>> +
>>>>> +[1] https://en.wikipedia.org/wiki/Region-based_memory_management
>>>>> diff --git a/doc/guides/rel_notes/release_22_11.rst 
>>>>> b/doc/guides/rel_notes/release_22_11.rst
>>>>> index 5d8ef669b8..4c1f760b98 100644
>>>>> --- a/doc/guides/rel_notes/release_22_11.rst
>>>>> +++ b/doc/guides/rel_notes/release_22_11.rst
>>>>> @@ -55,6 +55,12 @@ New Features
>>>>>         Also, make sure to start the actual text at the margin.
>>>>> =======================================================
>>>>>    +* **Added memarea library.**
>>>>> +
>>>>> +  The memarea library is an allocator of variable-size objects, it 
>>>>> is oriented
>>>>> +  towards the application layer, which could provides 
>>>>> 'region-based memory
>>>>> +  management' function.
>>>>> +
>>>>>    * **Added configuration for asynchronous flow connection 
>>>>> tracking.**
>>>>>        Added connection tracking action number hint to 
>>>>> ``rte_flow_configure``
>>>>> diff --git a/lib/eal/common/eal_common_log.c 
>>>>> b/lib/eal/common/eal_common_log.c
>>>>> index bd7b188ceb..3d62af59c6 100644
>>>>> --- a/lib/eal/common/eal_common_log.c
>>>>> +++ b/lib/eal/common/eal_common_log.c
>>>>> @@ -369,6 +369,7 @@ static const struct logtype logtype_strings[] = {
>>>>>        {RTE_LOGTYPE_EFD,        "lib.efd"},
>>>>>        {RTE_LOGTYPE_EVENTDEV,   "lib.eventdev"},
>>>>>        {RTE_LOGTYPE_GSO,        "lib.gso"},
>>>>> +    {RTE_LOGTYPE_MEMAREA,    "lib.memarea"},
>>>>>        {RTE_LOGTYPE_USER1,      "user1"},
>>>>>        {RTE_LOGTYPE_USER2,      "user2"},
>>>>>        {RTE_LOGTYPE_USER3,      "user3"},
>>>>> diff --git a/lib/eal/include/rte_log.h b/lib/eal/include/rte_log.h
>>>>> index 25ce42cdfc..708f3a39dd 100644
>>>>> --- a/lib/eal/include/rte_log.h
>>>>> +++ b/lib/eal/include/rte_log.h
>>>>> @@ -48,6 +48,7 @@ extern "C" {
>>>>>    #define RTE_LOGTYPE_EFD       18 /**< Log related to EFD. */
>>>>>    #define RTE_LOGTYPE_EVENTDEV  19 /**< Log related to eventdev. */
>>>>>    #define RTE_LOGTYPE_GSO       20 /**< Log related to GSO. */
>>>>> +#define RTE_LOGTYPE_MEMAREA   21 /**< Log related to memarea. */
>>>>>      /* these log types can be used in an application */
>>>>>    #define RTE_LOGTYPE_USER1     24 /**< User-defined log type 1. */
>>>>> diff --git a/lib/memarea/memarea_private.h 
>>>>> b/lib/memarea/memarea_private.h
>>>>> new file mode 100644
>>>>> index 0000000000..c76392d3e6
>>>>> --- /dev/null
>>>>> +++ b/lib/memarea/memarea_private.h
>>>>> @@ -0,0 +1,30 @@
>>>>> +/* SPDX-License-Identifier: BSD-3-Clause
>>>>> + * Copyright(c) 2022 HiSilicon Limited
>>>>> + */
>>>>> +
>>>>> +#ifndef MEMAREA_PRIVATE_H
>>>>> +#define MEMAREA_PRIVATE_H
>>>>> +
>>>>> +#include <rte_memarea.h>
>>>>> +
>>>>> +#define MEMAREA_FREE_ELEM_COOKIE    0xFFFFFFFF
>>>>> +
>>>>> +struct memarea_elem {
>>>>> +    size_t   size;
>>>>> +    uint32_t cookie;
>>>>> +    int32_t  refcnt; /* Non-zero indicates that it has been 
>>>>> allocated */
>>>>> +    TAILQ_ENTRY(memarea_elem) elem_node;
>>>>> +    TAILQ_ENTRY(memarea_elem) free_node;
>>>>> +} __rte_cache_aligned;
>>>>> +
>>>>
>>>> Why is the elem type cache line aligned? Need the elem data start be 
>>>> cache line aligned?
>>>
>>> Yes, the elem data align at cache-line default.
>>>
>>>>
>>>>> +TAILQ_HEAD(memarea_elem_list, memarea_elem);
>>>>> +
>>>>> +struct rte_memarea {
>>>>> +    struct rte_memarea_param init;
>>>>> +    rte_spinlock_t           lock;
>>>>> +    void                    *area_addr;
>>>>> +    struct memarea_elem_list elem_list;
>>>>> +    struct memarea_elem_list free_list;
>>>>> +} __rte_cache_aligned;
>>>>> +
>>>>> +#endif /* MEMAREA_PRIVATE_H */
>>>>> diff --git a/lib/memarea/meson.build b/lib/memarea/meson.build
>>>>> new file mode 100644
>>>>> index 0000000000..0a74fb4cd1
>>>>> --- /dev/null
>>>>> +++ b/lib/memarea/meson.build
>>>>> @@ -0,0 +1,16 @@
>>>>> +# SPDX-License-Identifier: BSD-3-Clause
>>>>> +# Copyright(c) 2022 HiSilicon Limited
>>>>> +
>>>>> +if is_windows
>>>>> +    build = false
>>>>> +    reason = 'not supported on Windows'
>>>>> +    subdir_done()
>>>>> +endif
>>>>> +
>>>>> +sources = files(
>>>>> +        'rte_memarea.c',
>>>>> +)
>>>>> +headers = files(
>>>>> +        'rte_memarea.h',
>>>>> +)
>>>>> +deps += []
>>>>> diff --git a/lib/memarea/rte_memarea.c b/lib/memarea/rte_memarea.c
>>>>> new file mode 100644
>>>>> index 0000000000..868da7661d
>>>>> --- /dev/null
>>>>> +++ b/lib/memarea/rte_memarea.c
>>>>> @@ -0,0 +1,157 @@
>>>>> +/* SPDX-License-Identifier: BSD-3-Clause
>>>>> + * Copyright(c) 2022 HiSilicon Limited
>>>>> + */
>>>>> +
>>>>> +#include <stdio.h>
>>>>> +#include <stdlib.h>
>>>>> +
>>>>> +#include <rte_common.h>
>>>>> +#include <rte_log.h>
>>>>> +#include <rte_malloc.h>
>>>>> +#include <rte_spinlock.h>
>>>>> +
>>>>> +#include "rte_memarea.h"
>>>>> +#include "memarea_private.h"
>>>>> +
>>>>> +static int
>>>>> +memarea_check_param(const struct rte_memarea_param *init)
>>>>> +{
>>>>> +    size_t len;
>>>>> +
>>>>> +    if (init == NULL) {
>>>>> +        RTE_LOG(ERR, MEMAREA, "memarea init param is NULL!\n");
>>>>> +        return -EINVAL;
>>>>> +    }
>>>>> +
>>>>> +    len = strnlen(init->name, RTE_MEMAREA_NAMESIZE);
>>>>> +    if (len == 0 || len >= RTE_MEMAREA_NAMESIZE) {
>>>>> +        RTE_LOG(ERR, MEMAREA, "memarea name size %zu invalid!\n", 
>>>>> len);
>>>>> +        return -EINVAL;
>>>>> +    }
>>>>> +
>>>>> +    if (init->source != RTE_MEMAREA_SOURCE_RTE_MEMORY &&
>>>>> +        init->source != RTE_MEMAREA_SOURCE_SYSTEM_API &&
>>>>> +        init->source != RTE_MEMAREA_SOURCE_USER_ADDR &&
>>>>> +        init->source != RTE_MEMAREA_SOURCE_USER_MEMAREA) {
>>>>> +        RTE_LOG(ERR, MEMAREA, "memarea: %s source: %d not 
>>>>> supported!\n",
>>>>> +            init->name, init->source);
>>>>> +        return -EINVAL;
>>>>> +    }
>>>>> +
>>>>> +    if (init->total_sz <= sizeof(struct memarea_elem)) {
>>>>> +        RTE_LOG(ERR, MEMAREA, "memarea: %s total-size: %zu too 
>>>>> small!\n",
>>>>> +            init->name, init->total_sz);
>>>>> +        return -EINVAL;
>>>>> +    }
>>>>> +
>>>>> +    if (init->source == RTE_MEMAREA_SOURCE_USER_ADDR && 
>>>>> init->user_addr == NULL) {
>>>>> +        RTE_LOG(ERR, MEMAREA, "memarea: %s user provided addr is 
>>>>> NULL!\n", init->name);
>>>>> +        return -EINVAL;
>>>>> +    }
>>>>> +
>>>>> +    if (init->source == RTE_MEMAREA_SOURCE_USER_ADDR &&
>>>>> +        ((uintptr_t)init->user_addr & (RTE_CACHE_LINE_SIZE - 1))) {
>>>>> +        RTE_LOG(ERR, MEMAREA, "memarea: %s user provided addr 
>>>>> should align: %d!\n",
>>>>> +            init->name, RTE_CACHE_LINE_SIZE);
>>>>> +        return -EINVAL;
>>>>> +    }
>>>>> +
>>>>> +    if (init->source == RTE_MEMAREA_SOURCE_USER_MEMAREA && 
>>>>> init->user_memarea == NULL) {
>>>>> +        RTE_LOG(ERR, MEMAREA, "memarea: %s user provided memarea 
>>>>> is NULL!\n", init->name);
>>>>> +        return -EINVAL;
>>>>> +    }
>>>>> +
>>>>> +    if (init->alg != RTE_MEMAREA_ALG_DEFAULT) {
>>>>> +        RTE_LOG(ERR, MEMAREA, "memarea: %s alg: %d not supported!\n",
>>>>> +            init->name, init->alg);
>>>>> +        return -EINVAL;
>>>>> +    }
>>>>> +
>>>>> +    return 0;
>>>>> +}
>>>>> +
>>>>> +static void *
>>>>> +memarea_alloc_from_system_api(size_t size)
>>>>> +{
>>>>> +    void *ptr = NULL;
>>>>> +    int ret;
>>>>> +
>>>>> +    ret = posix_memalign(&ptr, RTE_CACHE_LINE_SIZE, size);
>>>>> +    if (ret)
>>>>> +        return NULL;
>>>>> +    return ptr;
>>>>> +}
>>>>> +
>>>>> +static void *
>>>>> +memarea_alloc_area(const struct rte_memarea_param *init)
>>>>> +{
>>>>> +    void *ptr = NULL;
>>>>> +
>>>>> +    if (init->source == RTE_MEMAREA_SOURCE_RTE_MEMORY)
>>>>
>>>> Delete MEMORY. Of course it's memory. What else? If you want to make 
>>>> it clear it's from the RTE heap, it should spell out HEAP. Or MALLOC.
>>>
>>> HEAP seem better.
>>>
>>>>
>>>>> +        ptr = rte_malloc_socket(NULL, init->total_sz, 
>>>>> RTE_CACHE_LINE_SIZE,
>>>>> +                    init->numa_socket);
>>>>> +    else if (init->source == RTE_MEMAREA_SOURCE_SYSTEM_API)
>>>>> +        ptr = memarea_alloc_from_system_api(init->total_sz);
>>>>
>>>> "SYSTEM_API" doesn't strike me as a good name.
>>>>
>>>> RTE_MEMAREA_SOURCE_LIBC
>>>
>>> LIBC seem better.
>>>
>>>> RTE_MEMAREA_SOURCE_STD_HEAP
>>>> or at least remove API so it'll be
>>>> RTE_MEMAREA_SOURCE_SYSTEM
>>>>
>>>>> +    else if (init->source == RTE_MEMAREA_SOURCE_USER_ADDR)
>>>>
>>>> I would delete "ADDR".
>>>
>>> +1
>>>
>>>>
>>>>> +        ptr = init->user_addr;
>>>>> +
>>>>> +    if (ptr == NULL)
>>>>> +        RTE_LOG(ERR, MEMAREA, "memarea: %s alloc memory area 
>>>>> fail!\n", init->name);
>>>>> +
>>>>> +    return ptr;
>>>>> +}
>>>>> +
>>>>> +struct rte_memarea *
>>>>> +rte_memarea_create(const struct rte_memarea_param *init)
>>>>> +{
>>>>> +    struct memarea_elem *elem;
>>>>> +    struct rte_memarea *ma;
>>>>> +    void *addr;
>>>>> +    int ret;
>>>>> +
>>>>> +    ret = memarea_check_param(init);
>>>>> +    if (ret)
>>>>> +        return NULL;
>>>>> +
>>>>> +    addr = memarea_alloc_area(init);
>>>>> +    if (addr == NULL)
>>>>> +        return NULL;
>>>>> +
>>>>> +    ma = rte_zmalloc(NULL, sizeof(struct rte_memarea), 
>>>>> RTE_CACHE_LINE_SIZE);
>>>>> +    if (ma == NULL) {
>>>>> +        RTE_LOG(ERR, MEMAREA, "malloc memarea: %s management obj 
>>>>> fail!\n", init->name);
>>>>> +        return NULL;
>>>>> +    }
>>>>> +
>>>>> +    ma->init = *init;
>>>>> +    rte_spinlock_init(&ma->lock);
>>>>> +    TAILQ_INIT(&ma->elem_list);
>>>>> +    TAILQ_INIT(&ma->free_list);
>>>>> +    ma->area_addr = addr;
>>>>> +    elem = addr;
>>>>> +    elem->size = init->total_sz - sizeof(struct memarea_elem);
>>>>> +    elem->cookie = MEMAREA_FREE_ELEM_COOKIE;
>>>>> +    elem->refcnt = 0;
>>>>> +    TAILQ_INSERT_TAIL(&ma->elem_list, elem, elem_node);
>>>>> +    TAILQ_INSERT_TAIL(&ma->free_list, elem, free_node);
>>>>> +
>>>>> +    return ma;
>>>>> +}
>>>>> +
>>>>> +static void
>>>>> +memarea_free_area(struct rte_memarea *ma)
>>>>> +{
>>>>> +    if (ma->init.source == RTE_MEMAREA_SOURCE_RTE_MEMORY)
>>>>> +        rte_free(ma->area_addr);
>>>>> +    else if (ma->init.source == RTE_MEMAREA_SOURCE_SYSTEM_API)
>>>>> +        free(ma->area_addr);
>>>>> +}
>>>>> +
>>>>> +void
>>>>> +rte_memarea_destroy(struct rte_memarea *ma)
>>>>> +{
>>>>> +    if (ma == NULL)
>>>>> +        return;
>>>>> +    memarea_free_area(ma);
>>>>> +    rte_free(ma);
>>>>> +}
>>>>> diff --git a/lib/memarea/rte_memarea.h b/lib/memarea/rte_memarea.h
>>>>> new file mode 100644
>>>>> index 0000000000..543cda4cac
>>>>> --- /dev/null
>>>>> +++ b/lib/memarea/rte_memarea.h
>>>>> @@ -0,0 +1,145 @@
>>>>> +/* SPDX-License-Identifier: BSD-3-Clause
>>>>> + * Copyright(c) 2022 HiSilicon Limited
>>>>> + */
>>>>> +
>>>>> +#ifndef RTE_MEMAREA_H
>>>>> +#define RTE_MEMAREA_H
>>>>> +
>>>>> +/**
>>>>> + * @file
>>>>> + * RTE Memarea.
>>>>> + *
>>>>> + * The memarea is an allocator of variable-size object which based 
>>>>> on a memory
>>>>> + * region. It has the following features:
>>>>> + *
>>>>> + * - The default alignment size is RTE_CACHE_LINE_SIZE.
>>>>
>>>> This can be read as two things: the object size is aligned, or the 
>>>> start address is aligned.
>>>
>>> It means the start address align, will define more clear in v6.
>>>
>>>>
>>>>> + * - The memory region can be initialized from the following 
>>>>> memory sources:
>>>>> + *   1. RTE memory: e.g. invoke rte_malloc_socket to obtain.
>>>>
>>>> Remove "to obtain", or add "memory" after "obtain". Do you really 
>>>> mean "e.g.", and not "i.e."?
>>>
>>> will fix in v6.
>>>
>>>>
>>>>> + *   2. System API: e.g. invoke posix_memalign to obtain.
>>>>> + *   3. User provided address: it can be from extended memory as 
>>>>> long as it is
>>>>> + *      available. The address must be aligned to 
>>>>> RTE_CACHE_LINE_SIZE.
>>>>
>>>> What is extended memory?
>>>
>>> Like rte_extmen
>>>
>>>>
>>>>> + *   4) User provided memarea: it can be from another memarea. So 
>>>>> we can build
>>>>> + *      the following memory management structure:
>>>>> + *         \code{.unparsed}
>>>>> + *                           -------------
>>>>> + *                           | memarea-1 |
>>>>> + *                           -------------
>>>>> + *                                 |
>>>>> + *                                 v
>>>>> + *                  -------------------------------
>>>>> + *                  |               |             |
>>>>> + *                  v               v             v
>>>>> + *            -------------   -------------   ----------
>>>>> + *            | memarea-2 |   | memarea-3 |   | object |
>>>>> + *            -------------   -------------   ----------
>>>>> + *         \endcode
>>>>> + *      As shown above, the memarea-2/3 both create from 
>>>>> memarea-1's memory.
>>>>> + * - It provides refcnt feature which could be useful in 
>>>>> multi-reader scenario.
>>>>> + * - It supports MT-safe as long as it's specified at creation 
>>>>> time. If not
>>>>> + *   specified, all the functions of the memarea API are 
>>>>> lock-free, and assume
>>>>> + *   to not be invoked in parallel on different logical cores to 
>>>>> work on the
>>>>> + *   same memarea.
>>>>> + */
>>>>> +
>>>>> +#include <stdbool.h>
>>>>> +#include <stdint.h>
>>>>> +#include <stdio.h>
>>>>> +
>>>>> +#include <rte_compat.h>
>>>>> +
>>>>> +#ifdef __cplusplus
>>>>> +extern "C" {
>>>>> +#endif
>>>>> +
>>>>> +#define RTE_MEMAREA_NAMESIZE    64
>>>>> +
>>>>> +/**
>>>>> + * Memarea memory source.
>>>>> + */
>>>>> +enum rte_memarea_source {
>>>>> +    /** Memory source comes from rte memory. */
>>>>> +    RTE_MEMAREA_SOURCE_RTE_MEMORY,
>>>>> +    /** Memory source comes from system API. */
>>>>> +    RTE_MEMAREA_SOURCE_SYSTEM_API,
>>>>> +    /** Memory source comes from user-provided address. */
>>>>> +    RTE_MEMAREA_SOURCE_USER_ADDR,
>>>>> +    /** Memory source comes from user-provided memarea. */
>>>>> +    RTE_MEMAREA_SOURCE_USER_MEMAREA,
>>>>> +};
>>>>> +
>>>>> +/**
>>>>> + * Memarea memory management algorithm.
>>>>> + */
>>>>> +enum rte_memarea_alg {
>>>>> +    /* The default management algorithm is a variant of the next fit
>>>>> +     * algorithm. It uses a free-list to apply for memory and uses an
>>>>> +     * element-list in ascending order of address to support merging
>>>>> +     * upon free.
>>>>> +     */
>>>>> +    RTE_MEMAREA_ALG_DEFAULT,
>>>>> +};
>>
>> Do you need to expose the algorithm/management scheme option in the 
>> public API, if there is only one implementation to choose from?
> 
> 
> Yes, we plan to support SLAB algorithm in future by adding a new alg.
> 
> 
>>
>> You can always have a rte_memarea_create_alg(/../) in the future, or 
>> just change the signature between ABI-breaking releases.
> 
> 
> I don't think add a new API is a good idea.
> 
> You can see that, we put all init parameter in one struct in this 
> design, and could extend function by adding new field .
> 
> 
>>
>>
>> Also, shouldn't the default algorithm have name? Rather than just 
>> DEFAULT. RTE_MEMAREA_ALG_NEXTFIT maybe.
> 
> 
> Yes, will fix in next version.
> 
> 
>>
>>>>> +
>>>>> +struct rte_memarea;
>>>>> +
>>>>> +struct rte_memarea_param {
>>>>
>>>> Does this struct need to be public?
>>>
>>> Yes, the rte_memarea_param contains create parameters for create 
>>> memarea which need be public.
>>>
>>
>> Why isn't the public API just a bunch of create-function parameters? 
>> Or, alternatively, you have an assortment of create functions, for 
>> different combinations of parameters. Then you don't have to think 
>> about having reserved bits or other kind of ABI-related issues due to 
>> the struct being public.
> 
> 
> These are two API design styles，and I prefer the one create-function 
> API style just it seem simple (by inner usage voice).
> 
> 

My reaction when I saw the test code, was that it was needlessly 
complicated to create a memarea. Intuitions vary, I suppose.

Anyway, the style you prefer is represented elsewhere in DPDK already, 
so I'm good with this.

>>
>>> And rte_memarea which pointer implementation struction which just a 
>>> declare here.
>>>
>>>>
>>>>> +    char name[RTE_MEMAREA_NAMESIZE]; /**< Name of memarea. */
>>>>> +    enum rte_memarea_source source;  /**< Memory source of 
>>>>> memarea. */
>>>>> +    enum rte_memarea_alg alg;        /**< Memory management 
>>>>> algorithm. */
>>>>> +    size_t total_sz;                 /**< total size (bytes) of 
>>>>> memarea. */
>>>>> +    /** Indicates whether the memarea API should be MT-safe. */
>>>>> +    uint32_t mt_safe : 1;
>>>>
>>>> Why not bool?
>>>
>>> Use bit field other than bool will provides more reserved field.
>>>
>>>>
>>>>> +    union {
>>>>> +        /** Numa socket from which to apply for memarea's memory, 
>>>>> this
>>>>> +         * field is valid only when the source is set to be
>>>>> +         * RTE_MEMAREA_SOURCE_RTE_MEMORY.
>>>>> +         */
>>>>> +        int numa_socket;
>>>>> +        /** User provided address, this field is valid only when the
>>>>> +         * source is set to be RTE_MEMAREA_SOURCE_USER_ADDR.
>>>>> +         * Note: the provided address must align at least
>>>>> +         * RTE_CACHE_LINE_SIZE.
>>>>> +         */
>>>>> +        void *user_addr;
>>>>> +        /** User provided memarea, this field is valid only when the
>>>>> +         * source is set to be RTE_MEMAREA_SOURCE_USER_MEMAREA.
>>>>> +         */
>>>>> +        struct rte_memarea *user_memarea;
>>>>> +    };
>>>>> +};
>>>>> +
>>>>> +/**
>>>>> + * @warning
>>>>> + * @b EXPERIMENTAL: this API may change without prior notice.
>>>>> + *
>>>>> + * Create memarea.
>>>>> + *
>>>>> + * Create one new memarea.
>>>>> + *
>>>>> + * @param init
>>>>> + *   The init parameter of memarea.
>>>>> + *
>>>>> + * @return
>>>>> + *   Non-NULL on success. Otherwise NULL is returned.
>>>>> + */
>>>>> +__rte_experimental
>>>>> +struct rte_memarea *rte_memarea_create(const struct 
>>>>> rte_memarea_param *init);
>>>>> +
>>>>> +/**
>>>>> + * @warning
>>>>> + * @b EXPERIMENTAL: this API may change without prior notice.
>>>>> + *
>>>>> + * Destroy memarea.
>>>>> + *
>>>>> + * Destroy the memarea.
>>>>> + *
>>>>> + * @param ma
>>>>> + *   The pointer of memarea.
>>>>> + */
>>>>> +__rte_experimental
>>>>> +void rte_memarea_destroy(struct rte_memarea *ma);
>>>>> +
>>>>> +#ifdef __cplusplus
>>>>> +}
>>>>> +#endif
>>>>> +
>>>>> +#endif /* RTE_MEMAREA_H */
>>>>> diff --git a/lib/memarea/version.map b/lib/memarea/version.map
>>>>> new file mode 100644
>>>>> index 0000000000..f36a04d7cf
>>>>> --- /dev/null
>>>>> +++ b/lib/memarea/version.map
>>>>> @@ -0,0 +1,12 @@
>>>>> +EXPERIMENTAL {
>>>>> +    global:
>>>>> +
>>>>> +    rte_memarea_create;
>>>>> +    rte_memarea_destroy;
>>>>> +
>>>>> +    local: *;
>>>>> +};
>>>>> +
>>>>> +INTERNAL {
>>>>> +    local: *;
>>>>> +};
>>>>> diff --git a/lib/meson.build b/lib/meson.build
>>>>> index c648f7d800..521a25d6c0 100644
>>>>> --- a/lib/meson.build
>>>>> +++ b/lib/meson.build
>>>>> @@ -42,6 +42,7 @@ libraries = [
>>>>>            'kni',
>>>>>            'latencystats',
>>>>>            'lpm',
>>>>> +        'memarea',
>>>>>            'member',
>>>>>            'pcapng',
>>>>>            'power',
>>>>
>>>> .
>>