Re: DPDK 19.11.5 Legacy Memory Design Query

["Umakiran Godavarthi (ugodavar)" <ugodavar@cisco.com>]

[-- Attachment #1: Type: text/plain, Size: 4338 bytes --]

Hi Dmitry

My answer for your reply

You unmap memory, but you do not maintain DPDK memory management structures,
that is, DPDK does not know that this page is no longer usable.
Probably this is the reason for the crash.
You could print regions you're unmapping and the segfault address to confirm.

[Uma] :  Yes we are unmapping the entire range hoping all are free inside DPDK and DPDK heaps never use these pages.

Suppose we have 400 pages total free_hp, we want only 252 pages , so we reduce nr_pages to 252.

So we assume 253 to 400 inside DPDK are free as we nr_pages are made by my application as 252.

ms_idx = rte_fbarray_find_next_n_free(arr, 0, 2); -> 253 comes
ms_check_idx = rte_fbarray_find_next_n_free(arr, 0, RTE_PTR_DIFF(RTE_PTR_ADD(msl->base_va, msl->len), addr)/msl->page_sz); -> 253 comes (should be same as above)
ms_next_idx =  rte_fbarray_find_next_used(arr, ms_idx); -> This comes -1 as NO USED page is there (<0)

Hence we call unmap like -> munmap(addr, RTE_PTR_DIFF(RTE_PTR_ADD(msl->base_va, msl->len), addr));

Please let us know how to check in DPDK free heaps or FBARRAY that these pages we are freeing are really safe ? (253 to 400 unwanted pages by our application, other than above 3 checks)

If it’s not safe to free, how to inform DPDK to free those pages in FBARRAY and also clean up its heap list so that it never crashes !!

We are suspecting this code below  hits NULL crash or invalid address reference for us

static struct malloc_elem *
find_suitable_element(struct malloc_heap *heap, size_t size,
                unsigned int flags, size_t align, size_t bound, bool contig)
{
        size_t idx;
        struct malloc_elem *elem, *alt_elem = NULL;

        for (idx = malloc_elem_free_list_index(size);
                        idx < RTE_HEAP_NUM_FREELISTS; idx++) {
                for (elem = LIST_FIRST(&heap->free_head[idx]);   -> We are suspecting elem is invalid address and hence crashed !!!!
                                !!elem; elem = LIST_NEXT(elem, free_list)) {
                        if (malloc_elem_can_hold(elem, size, align, bound,
                                        contig)) {
Thanks
Umakiran


From: Dmitry Kozlyuk <dmitry.kozliuk@gmail.com>
Date: Thursday, 22 September 2022 at 2:31 PM
To: Umakiran Godavarthi (ugodavar) <ugodavar@cisco.com>
Cc: anatoly.burakov@intel.com <anatoly.burakov@intel.com>, dev@dpdk.org <dev@dpdk.org>, stephen@networkplumber.org <stephen@networkplumber.org>
Subject: Re: DPDK 19.11.5 Legacy Memory Design Query
Hi Umakiran,

> From: Umakiran Godavarthi (ugodavar) <ugodavar@cisco.com>
> Date: Wednesday, 14 September 2022 at 1:00 PM
[...]
>   1.  Then we go to DPDK Memory segment list walkthrough and for each FBARRAY , we find the used pages by DPDK and unmap the remaining pages by below code (Idea is to free the huge pages taken by DPDK process virtual memory) -> Free_HP will be 0 then, as X pages are used by DPDK and all unnecessary pages are freed in this step)
> Sample Code of 4 :
>
>               rte_memseg_list_walk_thread_unsafe(dpdk_find_and_free_unused, NULL); ->DPDK_FIND_AND_FREE_UNUSED is called for each Memory segment list (FBARRAY pointer is derived from MSL like below)
>
>               dpdk_find_and_free_unused(const struct rte_memseg_list *msl,
>                                           void *arg UNUSED)
>                {
>                       Int ms_idx;
>                        arr = (struct rte_fbarray *) &msl->memseg_arr;
>
>                         /*
>                          * use size of 2 instead of 1 to find the next free slot but
>                         * not hole.
>                         */
>                      ms_idx = rte_fbarray_find_next_n_free(arr, 0, 2);
>
>                      if (ms_idx >= 0) {
>                          addr = RTE_PTR_ADD(msl->base_va, ms_idx * msl->page_sz);
>                             munmap(addr, RTE_PTR_DIFF(RTE_PTR_ADD(msl->base_va, msl->len), addr));
>                       }
>                }

You unmap memory, but you do not maintain DPDK memory management structures,
that is, DPDK does not know that this page is no longer usable.
Probably this is the reason for the crash.
You could print regions you're unmapping and the segfault address to confirm.

[-- Attachment #2: Type: text/html, Size: 12046 bytes --]