Re: [PATCH v2 1/6] eal: add static per-lcore memory allocation facility

[Jerin Jacob <jerinjacobk@gmail.com>]

On Thu, Sep 12, 2024 at 2:40 PM Morten Brørup <mb@smartsharesystems.com> wrote:
>
> > +#define LCORE_BUFFER_SIZE (RTE_MAX_LCORE_VAR * RTE_MAX_LCORE)
>
> Considering hugepages...
>
> Lcore variables may be allocated before DPDK's memory allocator (rte_malloc()) is ready, so rte_malloc() cannot be used for lcore variables.
>
> And lcore variables are not usable (shared) for DPDK multi-process, so the lcore_buffer could be allocated through the O/S APIs as anonymous hugepages, instead of using rte_malloc().
>
> The alternative, using rte_malloc(), would disallow allocating lcore variables before DPDK's memory allocator has been initialized, which I think is too late.

I thought it is not. A lot of the subsystems are initialized after the
memory subsystem is initialized.
[1] example given in documentation. I thought, RTE_INIT needs to
replaced if the subsystem called after memory initialized (which is
the case for most of the libraries)
Trace library had a similar situation. It is managed like [2]



[1]
 * struct foo_lcore_state {
 *         int a;
 *         long b;
 * };
 *
 * static RTE_LCORE_VAR_HANDLE(struct foo_lcore_state, lcore_states);
 *
 * long foo_get_a_plus_b(void)
 * {
 *         struct foo_lcore_state *state = RTE_LCORE_VAR_VALUE(lcore_states);
 *
 *         return state->a + state->b;
 * }
 *
 * RTE_INIT(rte_foo_init)
 * {
 *         RTE_LCORE_VAR_ALLOC(lcore_states);
 *
 *         struct foo_lcore_state *state;
 *         RTE_LCORE_VAR_FOREACH_VALUE(state, lcore_states) {
 *                 (initialize 'state')
 *         }
 *
 *         (other initialization)
 * }


[2]


        /* First attempt from huge page */
        header = eal_malloc_no_trace(NULL, trace_mem_sz(trace->buff_len), 8);
        if (header) {
                trace->lcore_meta[count].area = TRACE_AREA_HUGEPAGE;
                goto found;
        }

        /* Second attempt from heap */
        header = malloc(trace_mem_sz(trace->buff_len));
        if (header == NULL) {
                trace_crit("trace mem malloc attempt failed");
                header = NULL;
                goto fail;

        }