Re: [RFC] memarea: introduce memory area library

[Jerin Jacob <jerinjacobk@gmail.com>]

On Thu, Jul 21, 2022 at 10:23 AM Chengwen Feng <fengchengwen@huawei.com> wrote:
>
> The memarea library is an allocator of variable-size object. It is a
> collection of allocated objects that can be efficiently alloc or free
> all at once, the main feature are as follows:
> a) it facilitate alloc and free of memory with low overhead.
>
> b) it provides refcnt feature which could be useful in some scenes.
>
> c) it supports MT-safe as long as it's specified at creation time.
>
> d) it's memory source could comes from:
> d.1) system API: malloc in C library.
> d.2) user provided address: it can be from the rte_malloc API series
> or extended memory as long as it is available.
> d.3) user provided memarea: it can be from another memarea.
>
> Note:
> a) the memarea is oriented towards the application layer, which could
> provides 'region-based memory management' [1] function.
> b) the eal library also provide memory zone/heap management, but these
> are tied to huge pages management.
>
> [1] https://en.wikipedia.org/wiki/Region-based_memory_management
>

Looks like a good feature to add to DPDK .


> Signed-off-by: Chengwen Feng <fengchengwen@huawei.com>
> ---
>  lib/memarea/meson.build   |  10 ++
>  lib/memarea/rte_memarea.c |  52 ++++++++++
>  lib/memarea/rte_memarea.h | 205 ++++++++++++++++++++++++++++++++++++++
>  lib/memarea/version.map   |  16 +++
>  lib/meson.build           |   1 +
>  5 files changed, 284 insertions(+)
>  create mode 100644 lib/memarea/meson.build
>  create mode 100644 lib/memarea/rte_memarea.c
>  create mode 100644 lib/memarea/rte_memarea.h
>  create mode 100644 lib/memarea/version.map
>
> diff --git a/lib/memarea/rte_memarea.h b/lib/memarea/rte_memarea.h
> new file mode 100644
> index 0000000000..dfbe845595
> --- /dev/null
> +++ b/lib/memarea/rte_memarea.h
> @@ -0,0 +1,205 @@
> +/* SPDX-License-Identifier: BSD-3-Clause
> + * Copyright(c) 2022 HiSilicon Limited
> + */
> +
> +#ifndef RTE_MEMAREA_H
> +#define RTE_MEMAREA_H
> +
> +/**
> + * @file
> + * RTE Memarea.
> + *
> + * A memory area is an allocator of variable-size object. It is identified
> + * by its name.
> + *
> + * The memarea is a collection of allocated objects that can be efficiently
> + * alloc or free all at once, the main feature are as follows:
> + *   a) it facilitate alloc and free of memory with low overhead.
> + *   b) it provides refcnt feature which could be useful in some scenes.
> + *   c) it supports MT-safe as long as it's specified at creation time.
> + *   d) it's memory source could comes from:
> + *      d.1) system API: malloc in C library.
> + *      d.2) user provided address: it can be from the rte_malloc API series
> + *           or extended memory as long as it is available.
> + *      d.3) user provided memarea: it can be from another memarea. So we can
> + *           build the following memory management structure:
> + *                         memarea-1
> + *                             |
> + *                             v
> + *                 ------------------------
> + *                 |           |          |
> + *                 v           v          v
> + *              memarea-2   memarea-3    obj
> + *
> + */
> +
> +#include <stdbool.h>
> +#include <stdint.h>
> +#include <stdio.h>
> +
> +#include <rte_compat.h>
> +
> +#ifdef __cplusplus
> +extern "C" {
> +#endif
> +
> +#define RTE_MEMAREA_NAMESIZE   64
> +
> +/**
> + * Memarea memory source.
> + */
> +enum rte_memarea_source {
> +       /** Memory source comes from sys api (e.g. malloc) */
> +       RTE_MEMAREA_SOURCE_SYSAPI,
> +       /** Memory source comes from user-provided address */
> +       RTE_MEMAREA_SOURCE_USER_ADDR,
> +       /** Memory source comes from user-provided memarea */
> +       RTE_MEMAREA_SOURCE_USER_MEMAREA,
> +};
> +
> +struct rte_memarea;
> +
> +struct rte_memarea_param {
> +       char name[RTE_MEMAREA_NAMESIZE]; /**< Name of memarea */
> +       enum rte_memarea_source source;  /**< Memory source of memarea */
> +       uint64_t size;                   /**< Size (byte) of memarea */
> +       uint32_t align;                  /**< Align of allocated object */
> +       /** Indicates whether the memarea should be MT-safe */
> +       bool mt_safe;
> +       /** Indicates whether the memarea is visible to multiple process.
> +        * If the memory source is RTE_MEMAREA_SOURCE_USER_ADDR, this filed
> +        * depends on user settings and must be set.
> +        * If the memory source is RTE_MEMAREA_SOURCE_SYSAPI or
> +        * RTE_MEMAREA_SOURCE_USER_MEMAREA, this filed does not need to be set.
> +        */
> +       bool mp_visible;
> +       /** User provided address, this field is valid only when source
> +        * is set to RTE_MEMAREA_SOURCE_USER_ADDR.
> +        */
> +       void *user_addr;
> +       /** User provided memarea, this field is valid only when source
> +        * is set to RTE_MEMAREA_SOURCE_MEMAREA.
> +        */
> +       struct rte_memarea *user_memarea;

Above two can be in union as it is  based on enum rte_memarea_source.

> +};
> +
> +struct rte_memarea {
> +       void *private_data; /**< private management data pointer*/
> +       struct rte_memarea_param init;
> +};


Make it opaque to application.

Also, In Terms of features, good to have following items

1)Stats
2)Detects the memory leak.