Re: [dpdk-dev] Questions about mmap in Asan APP

["Min Hu (Connor)" <humin29@huawei.com>]

在 2021/11/6 21:11, Thomas Monjalon 写道:
> 06/11/2021 10:27, Min Hu (Connor):
>> Hi, Jonas Pfefferle and all,
>> 	I met one questions:
>> 	When I ran APP on Kasan OS, secondary process occasionally init
>> failed.
>>
>> BTW:
>> testpmd is built with Asan, that is to add "CFLAGS="-fsanitize=address
>> -fno-omit-frame-pointer -pthread"".
>> Kasan OS is what opens this:
>> CONFIG_SLUB_DEBUG=y
>> CONFIG_KASAN=y
>>
>> The PRIMARY process APP(testpmd) log:
>> estuary:/home$ ./testpmd_kasan_debug    -w 0000:00:01.0
>> --file-prefix=test --proc-type=auto -l 0-1 -n 2 -- -i --rxq=4 --txq=4
>> EAL: Detected 16 lcore(s)
>> EAL: Detected 1 NUMA nodes
>> Option -w, --pci-whitelist is deprecated, use -a, --allow option instead
>> EAL: Auto-detected process type: PRIMARY
>> EAL: Detected static linkage of DPDK
>> EAL: Multi-process socket /var/run/dpdk/test/mp_socket
>> EAL: Selected IOVA mode 'PA'
>> EAL: No available 32768 kB hugepages reported
>> EAL: No available 64 kB hugepages reported
>> EAL: No available 1048576 kB hugepages reported
>> EAL: VFIO support initialized
>>
>> EAL: WARNING! Base virtual address hint (0xd01061000 != 0xfffb89e00000)
>> not respected!
>> EAL:    This may cause issues with mapping memory into secondary processes
>>
>> EAL:   Invalid NUMA socket, default to 0
>> EAL: Using IOMMU type 8 (No-IOMMU)
>> EAL: Probe PCI driver: net_hns3_vf (19e5:a22e) device: 0000:00:01.0
>> (socket 0)
>> TELEMETRY: No legacy callbacks, legacy socket not created
>> Interactive-mode selected
>> testpmd: create a new mbuf pool <mb_pool_0>: n=155456, size=2176, socket=0
>> testpmd: preferred mempool ops selected: ring_mp_mc
>>
>> Warning! port-topology=paired and odd forward ports number, the last
>> port will pair with itself.
>>
>> Configuring Port 0 (socket 0)
>> 0000:00:01.0 hns3_parse_rss_key(): Default RSS hash key to be set
>>
>> Port 0: link state change event
>> Port 0: 72:DE:F7:1B:F6:52
>> Checking link statuses...
>>
>> "EAL: WARNING! Base virtual address hint (0xd01061000 != 0xfffb89e00000)
>> not respected!  EAL:    This may cause issues with mapping memory into
>> secondary processes" should be noticed.
>>
>>
>>
>> The SECONDARY APP(proc-info) log is:
>> estuary:/home$ ./proc-info -w 0000:00:01.0 --file-prefix=test
>> --proc-type=auto -- --xstats-name rx_q0_packets
>> Option -w, --pci-whitelist is deprecated, use -a, --allow option instead
>> EAL: Cannot get a virtual area at requested address: 0xfffba6000000 (got
>> 0xfffb8ae3b000)
>>
>> EAL: Cannot reserve 17179869184 bytes at [0xfffba6000000] - please use
>> '--base-virtaddr' option
>>
>> EAL: Cannot preallocate VA space for hugepage memory
>> EAL: FATAL: Cannot init memory
>> EAL: Cannot init memory
>> PANIC in main():
>> Cannot init EAL
>> 4: [/lib/aarch64-linux-gnu/libc.so.6(__libc_start_main+0xe0)
>> [0xffff8be5a8a0]]
>> 3: [./proc-info(main+0x1098) [0x52a3a8]]
>> 2: [./proc-info(__rte_panic+0x9c) [0x48cfac]]
>> 1: [./proc-info(rte_dump_stack+0x20) [0x80d4b0]]
>> Aborted
>>
>> "EAL: Cannot get a virtual area at requested address: 0xfffba6000000
>> (got 0xfffb8ae3b000)" should be noticed.
>>
>> According to my analysis, the reason is that the system call "mmap"
>> return the address which is different with the hint address(first
>> parameter). That is to say：
>> return_addr = mmap(void* start,size_t length,int prot,int flags,int
>> fd,off_t offset);
>> return_addr and start is different.
>>
>> Why this happened? I guess Asan works based on shadow memory, shadow
>> memory will occupy the memory zone and it cannot be one for mmap.
>> So, it the shadow memory happens to be in the range of mmap region,
>> mmap will return a new address which is free(not occupied by shadow).
>>
>> BTW，I think DPDK multiple process is not supported in that situation,
>> such as "Asan APP on Kasan OS".
>>
>> What is your opinion? any will weclome, thanks.
> 
> Secondary process mapping is fragile.
> I am not surprised there are more issues when enabling
> some kernel features playing with memory ranges.
> You may try to find more hints to make it work,
> but it is OK stating this scenario is not supported.
Thank Thomas for your reply, I got it.
> 
> Another outcome to this trial could be to document the limitation
> in DPDK documentation for Linux.
> 
> Thanks
> 
> 
> .
>