DPDK patches and discussions
 help / color / mirror / Atom feed
* [dpdk-dev] Questions about mmap in Asan APP
@ 2021-11-06  9:27 Min Hu (Connor)
  2021-11-06 13:11 ` Thomas Monjalon
  0 siblings, 1 reply; 3+ messages in thread
From: Min Hu (Connor) @ 2021-11-06  9:27 UTC (permalink / raw)
  To: dev; +Cc: Thomas Monjalon, Ferruh Yigit, jpf

Hi, Jonas Pfefferle and all,
	I met one questions:
	When I ran APP on Kasan OS, secondary process occasionally init
failed.

BTW:
testpmd is built with Asan, that is to add "CFLAGS="-fsanitize=address 
-fno-omit-frame-pointer -pthread"".
Kasan OS is what opens this:
CONFIG_SLUB_DEBUG=y
CONFIG_KASAN=y

The PRIMARY process APP(testpmd) log:
estuary:/home$ ./testpmd_kasan_debug    -w 0000:00:01.0 
--file-prefix=test --proc-type=auto -l 0-1 -n 2 -- -i --rxq=4 --txq=4
EAL: Detected 16 lcore(s)
EAL: Detected 1 NUMA nodes
Option -w, --pci-whitelist is deprecated, use -a, --allow option instead
EAL: Auto-detected process type: PRIMARY
EAL: Detected static linkage of DPDK
EAL: Multi-process socket /var/run/dpdk/test/mp_socket
EAL: Selected IOVA mode 'PA'
EAL: No available 32768 kB hugepages reported
EAL: No available 64 kB hugepages reported
EAL: No available 1048576 kB hugepages reported
EAL: VFIO support initialized

EAL: WARNING! Base virtual address hint (0xd01061000 != 0xfffb89e00000) 
not respected!
EAL:    This may cause issues with mapping memory into secondary processes

EAL:   Invalid NUMA socket, default to 0
EAL: Using IOMMU type 8 (No-IOMMU)
EAL: Probe PCI driver: net_hns3_vf (19e5:a22e) device: 0000:00:01.0 
(socket 0)
TELEMETRY: No legacy callbacks, legacy socket not created
Interactive-mode selected
testpmd: create a new mbuf pool <mb_pool_0>: n=155456, size=2176, socket=0
testpmd: preferred mempool ops selected: ring_mp_mc

Warning! port-topology=paired and odd forward ports number, the last 
port will pair with itself.

Configuring Port 0 (socket 0)
0000:00:01.0 hns3_parse_rss_key(): Default RSS hash key to be set

Port 0: link state change event
Port 0: 72:DE:F7:1B:F6:52
Checking link statuses...

"EAL: WARNING! Base virtual address hint (0xd01061000 != 0xfffb89e00000) 
not respected!  EAL:    This may cause issues with mapping memory into
secondary processes" should be noticed.



The SECONDARY APP(proc-info) log is:
estuary:/home$ ./proc-info -w 0000:00:01.0 --file-prefix=test 
--proc-type=auto -- --xstats-name rx_q0_packets
Option -w, --pci-whitelist is deprecated, use -a, --allow option instead
EAL: Cannot get a virtual area at requested address: 0xfffba6000000 (got 
0xfffb8ae3b000)

EAL: Cannot reserve 17179869184 bytes at [0xfffba6000000] - please use 
'--base-virtaddr' option

EAL: Cannot preallocate VA space for hugepage memory
EAL: FATAL: Cannot init memory
EAL: Cannot init memory
PANIC in main():
Cannot init EAL
4: [/lib/aarch64-linux-gnu/libc.so.6(__libc_start_main+0xe0) 
[0xffff8be5a8a0]]
3: [./proc-info(main+0x1098) [0x52a3a8]]
2: [./proc-info(__rte_panic+0x9c) [0x48cfac]]
1: [./proc-info(rte_dump_stack+0x20) [0x80d4b0]]
Aborted

"EAL: Cannot get a virtual area at requested address: 0xfffba6000000 
(got 0xfffb8ae3b000)" should be noticed.

According to my analysis, the reason is that the system call "mmap"
return the address which is different with the hint address(first
parameter). That is to say:
return_addr = mmap(void* start,size_t length,int prot,int flags,int
fd,off_t offset);
return_addr and start is different.

Why this happened? I guess Asan works based on shadow memory, shadow
memory will occupy the memory zone and it cannot be one for mmap.
So, it the shadow memory happens to be in the range of mmap region,
mmap will return a new address which is free(not occupied by shadow).

BTW,I think DPDK multiple process is not supported in that situation,
such as "Asan APP on Kasan OS".

What is your opinion? any will weclome, thanks.

	Best regards.









^ permalink raw reply	[flat|nested] 3+ messages in thread

* Re: [dpdk-dev] Questions about mmap in Asan APP
  2021-11-06  9:27 [dpdk-dev] Questions about mmap in Asan APP Min Hu (Connor)
@ 2021-11-06 13:11 ` Thomas Monjalon
  2021-11-10  0:52   ` Min Hu (Connor)
  0 siblings, 1 reply; 3+ messages in thread
From: Thomas Monjalon @ 2021-11-06 13:11 UTC (permalink / raw)
  To: Min Hu (Connor); +Cc: dev, Ferruh Yigit, jpf

06/11/2021 10:27, Min Hu (Connor):
> Hi, Jonas Pfefferle and all,
> 	I met one questions:
> 	When I ran APP on Kasan OS, secondary process occasionally init
> failed.
> 
> BTW:
> testpmd is built with Asan, that is to add "CFLAGS="-fsanitize=address 
> -fno-omit-frame-pointer -pthread"".
> Kasan OS is what opens this:
> CONFIG_SLUB_DEBUG=y
> CONFIG_KASAN=y
> 
> The PRIMARY process APP(testpmd) log:
> estuary:/home$ ./testpmd_kasan_debug    -w 0000:00:01.0 
> --file-prefix=test --proc-type=auto -l 0-1 -n 2 -- -i --rxq=4 --txq=4
> EAL: Detected 16 lcore(s)
> EAL: Detected 1 NUMA nodes
> Option -w, --pci-whitelist is deprecated, use -a, --allow option instead
> EAL: Auto-detected process type: PRIMARY
> EAL: Detected static linkage of DPDK
> EAL: Multi-process socket /var/run/dpdk/test/mp_socket
> EAL: Selected IOVA mode 'PA'
> EAL: No available 32768 kB hugepages reported
> EAL: No available 64 kB hugepages reported
> EAL: No available 1048576 kB hugepages reported
> EAL: VFIO support initialized
> 
> EAL: WARNING! Base virtual address hint (0xd01061000 != 0xfffb89e00000) 
> not respected!
> EAL:    This may cause issues with mapping memory into secondary processes
> 
> EAL:   Invalid NUMA socket, default to 0
> EAL: Using IOMMU type 8 (No-IOMMU)
> EAL: Probe PCI driver: net_hns3_vf (19e5:a22e) device: 0000:00:01.0 
> (socket 0)
> TELEMETRY: No legacy callbacks, legacy socket not created
> Interactive-mode selected
> testpmd: create a new mbuf pool <mb_pool_0>: n=155456, size=2176, socket=0
> testpmd: preferred mempool ops selected: ring_mp_mc
> 
> Warning! port-topology=paired and odd forward ports number, the last 
> port will pair with itself.
> 
> Configuring Port 0 (socket 0)
> 0000:00:01.0 hns3_parse_rss_key(): Default RSS hash key to be set
> 
> Port 0: link state change event
> Port 0: 72:DE:F7:1B:F6:52
> Checking link statuses...
> 
> "EAL: WARNING! Base virtual address hint (0xd01061000 != 0xfffb89e00000) 
> not respected!  EAL:    This may cause issues with mapping memory into
> secondary processes" should be noticed.
> 
> 
> 
> The SECONDARY APP(proc-info) log is:
> estuary:/home$ ./proc-info -w 0000:00:01.0 --file-prefix=test 
> --proc-type=auto -- --xstats-name rx_q0_packets
> Option -w, --pci-whitelist is deprecated, use -a, --allow option instead
> EAL: Cannot get a virtual area at requested address: 0xfffba6000000 (got 
> 0xfffb8ae3b000)
> 
> EAL: Cannot reserve 17179869184 bytes at [0xfffba6000000] - please use 
> '--base-virtaddr' option
> 
> EAL: Cannot preallocate VA space for hugepage memory
> EAL: FATAL: Cannot init memory
> EAL: Cannot init memory
> PANIC in main():
> Cannot init EAL
> 4: [/lib/aarch64-linux-gnu/libc.so.6(__libc_start_main+0xe0) 
> [0xffff8be5a8a0]]
> 3: [./proc-info(main+0x1098) [0x52a3a8]]
> 2: [./proc-info(__rte_panic+0x9c) [0x48cfac]]
> 1: [./proc-info(rte_dump_stack+0x20) [0x80d4b0]]
> Aborted
> 
> "EAL: Cannot get a virtual area at requested address: 0xfffba6000000 
> (got 0xfffb8ae3b000)" should be noticed.
> 
> According to my analysis, the reason is that the system call "mmap"
> return the address which is different with the hint address(first
> parameter). That is to say:
> return_addr = mmap(void* start,size_t length,int prot,int flags,int
> fd,off_t offset);
> return_addr and start is different.
> 
> Why this happened? I guess Asan works based on shadow memory, shadow
> memory will occupy the memory zone and it cannot be one for mmap.
> So, it the shadow memory happens to be in the range of mmap region,
> mmap will return a new address which is free(not occupied by shadow).
> 
> BTW,I think DPDK multiple process is not supported in that situation,
> such as "Asan APP on Kasan OS".
> 
> What is your opinion? any will weclome, thanks.

Secondary process mapping is fragile.
I am not surprised there are more issues when enabling
some kernel features playing with memory ranges.
You may try to find more hints to make it work,
but it is OK stating this scenario is not supported.

Another outcome to this trial could be to document the limitation
in DPDK documentation for Linux.

Thanks



^ permalink raw reply	[flat|nested] 3+ messages in thread

* Re: [dpdk-dev] Questions about mmap in Asan APP
  2021-11-06 13:11 ` Thomas Monjalon
@ 2021-11-10  0:52   ` Min Hu (Connor)
  0 siblings, 0 replies; 3+ messages in thread
From: Min Hu (Connor) @ 2021-11-10  0:52 UTC (permalink / raw)
  To: Thomas Monjalon; +Cc: dev, Ferruh Yigit, jpf



在 2021/11/6 21:11, Thomas Monjalon 写道:
> 06/11/2021 10:27, Min Hu (Connor):
>> Hi, Jonas Pfefferle and all,
>> 	I met one questions:
>> 	When I ran APP on Kasan OS, secondary process occasionally init
>> failed.
>>
>> BTW:
>> testpmd is built with Asan, that is to add "CFLAGS="-fsanitize=address
>> -fno-omit-frame-pointer -pthread"".
>> Kasan OS is what opens this:
>> CONFIG_SLUB_DEBUG=y
>> CONFIG_KASAN=y
>>
>> The PRIMARY process APP(testpmd) log:
>> estuary:/home$ ./testpmd_kasan_debug    -w 0000:00:01.0
>> --file-prefix=test --proc-type=auto -l 0-1 -n 2 -- -i --rxq=4 --txq=4
>> EAL: Detected 16 lcore(s)
>> EAL: Detected 1 NUMA nodes
>> Option -w, --pci-whitelist is deprecated, use -a, --allow option instead
>> EAL: Auto-detected process type: PRIMARY
>> EAL: Detected static linkage of DPDK
>> EAL: Multi-process socket /var/run/dpdk/test/mp_socket
>> EAL: Selected IOVA mode 'PA'
>> EAL: No available 32768 kB hugepages reported
>> EAL: No available 64 kB hugepages reported
>> EAL: No available 1048576 kB hugepages reported
>> EAL: VFIO support initialized
>>
>> EAL: WARNING! Base virtual address hint (0xd01061000 != 0xfffb89e00000)
>> not respected!
>> EAL:    This may cause issues with mapping memory into secondary processes
>>
>> EAL:   Invalid NUMA socket, default to 0
>> EAL: Using IOMMU type 8 (No-IOMMU)
>> EAL: Probe PCI driver: net_hns3_vf (19e5:a22e) device: 0000:00:01.0
>> (socket 0)
>> TELEMETRY: No legacy callbacks, legacy socket not created
>> Interactive-mode selected
>> testpmd: create a new mbuf pool <mb_pool_0>: n=155456, size=2176, socket=0
>> testpmd: preferred mempool ops selected: ring_mp_mc
>>
>> Warning! port-topology=paired and odd forward ports number, the last
>> port will pair with itself.
>>
>> Configuring Port 0 (socket 0)
>> 0000:00:01.0 hns3_parse_rss_key(): Default RSS hash key to be set
>>
>> Port 0: link state change event
>> Port 0: 72:DE:F7:1B:F6:52
>> Checking link statuses...
>>
>> "EAL: WARNING! Base virtual address hint (0xd01061000 != 0xfffb89e00000)
>> not respected!  EAL:    This may cause issues with mapping memory into
>> secondary processes" should be noticed.
>>
>>
>>
>> The SECONDARY APP(proc-info) log is:
>> estuary:/home$ ./proc-info -w 0000:00:01.0 --file-prefix=test
>> --proc-type=auto -- --xstats-name rx_q0_packets
>> Option -w, --pci-whitelist is deprecated, use -a, --allow option instead
>> EAL: Cannot get a virtual area at requested address: 0xfffba6000000 (got
>> 0xfffb8ae3b000)
>>
>> EAL: Cannot reserve 17179869184 bytes at [0xfffba6000000] - please use
>> '--base-virtaddr' option
>>
>> EAL: Cannot preallocate VA space for hugepage memory
>> EAL: FATAL: Cannot init memory
>> EAL: Cannot init memory
>> PANIC in main():
>> Cannot init EAL
>> 4: [/lib/aarch64-linux-gnu/libc.so.6(__libc_start_main+0xe0)
>> [0xffff8be5a8a0]]
>> 3: [./proc-info(main+0x1098) [0x52a3a8]]
>> 2: [./proc-info(__rte_panic+0x9c) [0x48cfac]]
>> 1: [./proc-info(rte_dump_stack+0x20) [0x80d4b0]]
>> Aborted
>>
>> "EAL: Cannot get a virtual area at requested address: 0xfffba6000000
>> (got 0xfffb8ae3b000)" should be noticed.
>>
>> According to my analysis, the reason is that the system call "mmap"
>> return the address which is different with the hint address(first
>> parameter). That is to say:
>> return_addr = mmap(void* start,size_t length,int prot,int flags,int
>> fd,off_t offset);
>> return_addr and start is different.
>>
>> Why this happened? I guess Asan works based on shadow memory, shadow
>> memory will occupy the memory zone and it cannot be one for mmap.
>> So, it the shadow memory happens to be in the range of mmap region,
>> mmap will return a new address which is free(not occupied by shadow).
>>
>> BTW,I think DPDK multiple process is not supported in that situation,
>> such as "Asan APP on Kasan OS".
>>
>> What is your opinion? any will weclome, thanks.
> 
> Secondary process mapping is fragile.
> I am not surprised there are more issues when enabling
> some kernel features playing with memory ranges.
> You may try to find more hints to make it work,
> but it is OK stating this scenario is not supported.
Thank Thomas for your reply, I got it.
> 
> Another outcome to this trial could be to document the limitation
> in DPDK documentation for Linux.
> 
> Thanks
> 
> 
> .
> 

^ permalink raw reply	[flat|nested] 3+ messages in thread

end of thread, other threads:[~2021-11-10  0:52 UTC | newest]

Thread overview: 3+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2021-11-06  9:27 [dpdk-dev] Questions about mmap in Asan APP Min Hu (Connor)
2021-11-06 13:11 ` Thomas Monjalon
2021-11-10  0:52   ` Min Hu (Connor)

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).