Hi Anatoly/DPDK-Developers
I am working on DPDK 19.11.5 Legacy Memory design and have a query about how to boot up in Legacy memory mode.
For example, if host has memory pattern huge pages like this for total 500 we get in step 1 kernel reservation.
250, 90, 80, 70 , 10 -> Sum is 500 (N pages)
We need only 350 pages (350 based on no of ports, queues dpdk application needs)
So we need 250, 90, 10.
So total 3 pools POOL_0 -> 250 pages, POOL_1 -> 90, POOL_2 -> 10 pages
Sample Code of 4 :
rte_memseg_list_walk_thread_unsafe(dpdk_find_and_free_unused,
NULL); ->DPDK_FIND_AND_FREE_UNUSED is called for each Memory segment list (FBARRAY pointer is derived from MSL like below)
dpdk_find_and_free_unused(const struct rte_memseg_list *msl,
void *arg UNUSED)
{
Int ms_idx;
arr = (struct rte_fbarray *) &msl->memseg_arr;
/*
* use size of 2 instead of 1 to find the next free slot but
* not hole.
*/
ms_idx = rte_fbarray_find_next_n_free(arr, 0, 2);
if (ms_idx >= 0) {
addr = RTE_PTR_ADD(msl->base_va, ms_idx * msl->page_sz);
munmap(addr, RTE_PTR_DIFF(RTE_PTR_ADD(msl->base_va, msl->len), addr));
}
}
struct rte_mempool *pool;
pool = rte_pktmbuf_pool_create(name, num_mbufs,
RTE_MIN(num_mbufs/4, MBUF_CACHE_SIZE),
MBUF_PRIV_SIZE,
frame_len + RTE_PKTMBUF_HEADROOM,
rte_socket_id()
DPDK EAL core comes with BT like this
#6 sigcrash
(signo=11, info=0x7fff1c1867f0, ctx=0x7fff1c1866c0)
#7
#8 malloc_elem_can_hold () from ./usr/lib64/dpdk-19/librte_eal.so.20.0
#9 find_suitable_element () from ./usr/lib64/dpdk-19/librte_eal.so.20.0
#10 malloc_heap_alloc () from ./usr/lib64/dpdk-19/librte_eal.so.20.0
#11 rte_malloc_socket () from ./usr/lib64/dpdk-19/librte_eal.so.20.0
#12 rte_mempool_create_empty () from ./usr/lib64/dpdk-19/librte_mempool.so.20.0
#13 rte_pktmbuf_pool_create_by_ops () from ./usr/lib64/dpdk-19/librte_mbuf.so.20.0
#14 rte_pktmbuf_pool_create () from ./usr/lib64/dpdk-19/librte_mbuf.so.20.0
#15 dpdk_create_mbuf_pool (mem_chunk_tbl=0x555d556de8e0 , num_mbufs=46080, frame_len=2048, name=0x7fff1c1873c0 "DPDK_POOL_0")
We see find suitable element does not able to find a suitable element in DPDK memory segment lists it searches for HEAP ALLOC and returns NULL and NULL
dereference crashes boot up process
Please let me know any comments on boot up process for 1-6 and any reason behind the crash ?
We are suspecting Step 4 where FBARRAY unused pages freeing at last should free the least contiguous memory segments right ? (munmap after finding 2 free pages , entire
length we unmap in step 4 to free virtual memory)
Please let me know thoughts on FBARRAY design, is it expected to map the most contiguous…..least contiguous in a virtual address space
right ?
So our most contiguous segments in Step 2 is safe even after Step 3, Step 4 we believe. Please correct my understanding if anything wrong.
Thanks
Umakiran