Thanks Dmitry for quick turn around

Yes below are my answers


There still DPDK malloc internal structures that you cannot adjust.
I suggest going another way:
Instead of letting DPDK allocate all hugepages and unmapping some,
allow DPDK to allocate an absolute minimum (1 x 2MB page?)
and add all the rest you need via rte_extmem_*() API.

[Uma] :  Yes I agree if free_hp = 400, nr_hp = 252, we are expecting DPDK takes only 252 and keep the remaining pages free in its heap.
               As you have mentioned just boot DPDK with 1 page, and add pages we want later, is this the steps

  1.  NR_HP =1 , FREE_HP = 1
  2.  EAL INIT (DPDK boots up with 1 2 MB page)
  3.  What is the API for adding later on pages ? (rte_extmem_*, can you please give the full API details and how to call it with arguments )

We can do 1,2,3 there is a problem once we reduce pages to 1 , kernel will free the huge pages totally

So is there a way not to touch NR_HP, FREE_HP, and just pass arguments to boot DPDK with just 1 page ? Please let us know and later add pages we need to DPDK !!

Why do you need legacy mode in the first place?
Looks like you're painfully trying to achieve the same result
that dynamic mode would give you automatically.

[Uma] : Yes , we can’t avoid legacy memory design as secondary process mapped page by page to primary process , and physical addr space is same for both processes.  We have to stick to legacy memory design only for now !!

Thanks
Umakiran

From: Dmitry Kozlyuk <dmitry.kozliuk@gmail.com>
Date: Friday, 23 September 2022 at 5:17 PM
To: Umakiran Godavarthi (ugodavar) <ugodavar@cisco.com>
Cc: anatoly.burakov@intel.com <anatoly.burakov@intel.com>, dev@dpdk.org <dev@dpdk.org>, stephen@networkplumber.org <stephen@networkplumber.org>
Subject: Re: DPDK 19.11.5 Legacy Memory Design Query
2022-09-23 11:20 (UTC+0000), Umakiran Godavarthi (ugodavar):
> [Uma] :  Yes we are unmapping the entire range hoping all are free inside DPDK and DPDK heaps never use these pages.
>
> Suppose we have 400 pages total free_hp, we want only 252 pages , so we reduce nr_pages to 252.
>
> So we assume 253 to 400 inside DPDK are free as we nr_pages are made by my application as 252.
>
> ms_idx = rte_fbarray_find_next_n_free(arr, 0, 2); -> 253 comes
> ms_check_idx = rte_fbarray_find_next_n_free(arr, 0, RTE_PTR_DIFF(RTE_PTR_ADD(msl->base_va, msl->len), addr)/msl->page_sz); -> 253 comes (should be same as above)
> ms_next_idx =  rte_fbarray_find_next_used(arr, ms_idx); -> This comes -1 as NO USED page is there (<0)
>
> Hence we call unmap like -> munmap(addr, RTE_PTR_DIFF(RTE_PTR_ADD(msl->base_va, msl->len), addr));
>
> Please let us know how to check in DPDK free heaps or FBARRAY that these pages we are freeing are really safe ? (253 to 400 unwanted pages by our application, other than above 3 checks)
>
> If it’s not safe to free, how to inform DPDK to free those pages in FBARRAY and also clean up its heap list so that it never crashes !!

There still DPDK malloc internal structures that you cannot adjust.
I suggest going another way:
Instead of letting DPDK allocate all hugepages and unmapping some,
allow DPDK to allocate an absolute minimum (1 x 2MB page?)
and add all the rest you need via rte_extmem_*() API.

Why do you need legacy mode in the first place?
Looks like you're painfully trying to achieve the same result
that dynamic mode would give you automatically.