* Re: [dpdk-users] Error in rte_eal_init using dpdk 16.04 and Qlogic (Broadcom) bnx2x driver in a SECONDARY process
@ 2016-05-04 19:07 dawid_jurek
2016-05-05 11:02 ` [dpdk-users] R: " David Coen
0 siblings, 1 reply; 4+ messages in thread
From: dawid_jurek @ 2016-05-04 19:07 UTC (permalink / raw)
To: users, David Coen
Hello David,
By looking on driver sources (bnx2x_ethdev.c) assertion `sc->bar[0].base_addr' fail is caused by pci_dev->mem_resource[0].addr == NULL. It means that MMIO mapping of region nr. 0 failed.
The reason of this may be print reported on ring level some time before crash:
"EAL: memzone_reserve_aligned_thread_unsafe(): memzone <RG_MP_log_history>
already exists
RING: Cannot reserve memory."
I suspect some memory allocation issue but not 100% sure.
Regards,
Dawid
> Hello,
>
> We are trying to run the
> "dpdk-16.04\examples\multi_process\client_server_mp" sample application
> on a RHEL6.7 linux server, with DPDK 16.04 and the following network
> interfaces:
>
> Network devices using DPDK-compatible driver
> ============================================
> 0000:10:00.0 'BCM57840 NetXtreme II 10 Gigabit Ethernet' drv=igb_uio
> unused=bnx2x
> 0000:10:00.1 'BCM57840 NetXtreme II 10 Gigabit Ethernet' drv=igb_uio
> unused=bnx2x
> ===================================
>
> PMD: bnx2x_print_adapter_info(): ASIC : 0x168d1010
> PMD: bnx2x_print_adapter_info(): Rev : B1
> PMD: bnx2x_print_adapter_info(): Bus PCIe : 8,
> PMD: bnx2x_print_adapter_info(): 5 Gbps
> PMD: bnx2x_print_adapter_info(): Flags :
> PMD: bnx2x_print_adapter_info(): MSI
> PMD: bnx2x_print_adapter_info(): |
> PMD: bnx2x_print_adapter_info(): MSI-X
> PMD: bnx2x_print_adapter_info(): Queues :
> PMD: bnx2x_print_adapter_info(): RSS : 1
> PMD: bnx2x_print_adapter_info(): DPDK : DPDK 16.04.0
> PMD: bnx2x_print_adapter_info(): Driver : BNX2X PMD 1.78.18_1.0.0
> PMD: bnx2x_print_adapter_info(): Firmware : 7.2.51
> PMD: bnx2x_print_adapter_info(): Bootcode : 7.13.24
> PMD: bnx2x_print_adapter_info():
>
> We can run the server process, without problems, with the following command.
>
> ./mp_server -c 6 -n 4 -- -p 3 -n 1
>
> .than, in a second shell, we run the client process with the following
> command .
>
> ./mp_client -c 8 -n 4 --proc-type=auto -- -n 0
>
> .and we always get this output:
>
> EAL: Detected lcore 0 as core 0 on socket 0
> EAL: Detected lcore 1 as core 1 on socket 0
> EAL: Detected lcore 2 as core 2 on socket 0
> EAL: Detected lcore 3 as core 3 on socket 0
> EAL: Detected lcore 4 as core 4 on socket 0
> EAL: Detected lcore 5 as core 5 on socket 0
> EAL: Detected lcore 6 as core 0 on socket 1
> EAL: Detected lcore 7 as core 1 on socket 1
> EAL: Detected lcore 8 as core 2 on socket 1
> EAL: Detected lcore 9 as core 3 on socket 1
> EAL: Detected lcore 10 as core 4 on socket 1
> EAL: Detected lcore 11 as core 5 on socket 1
> EAL: Support maximum 128 logical core(s) by configuration.
> EAL: Detected 12 lcore(s)
> EAL: Auto-detected process type: SECONDARY
> EAL: Setting up physically contiguous memory...
> EAL: Analysing 16 files
> EAL: Mapped segment 0 of size 0x200000000
> EAL: Mapped segment 1 of size 0x200000000
> EAL: memzone_reserve_aligned_thread_unsafe(): memzone <RG_MP_log_history>
> already exists
> RING: Cannot reserve memory
> EAL: TSC frequency is ~2400000 KHz
> EAL: Master lcore 3 is ready (tid=f7fec880;cpuset=[3])
> PMD: rte_bnx2x_pmd_init(): >>
> PMD: rte_bnx2xvf_pmd_init(): >>
> EAL: PCI device 0000:10:00.0 on NUMA socket 0
> EAL: probe driver: 14e4:16a1 rte_bnx2x_pmd
> EAL: PCI memory mapped at 0x7fffc0000000
> EAL: PCI memory mapped at 0x7fffc0800000
> EAL: PCI memory mapped at 0x7fffc1000000
> PMD: eth_bnx2x_dev_init(): >>
> PMD: bnx2x_common_dev_init(): >>
> mp_client: /root/dpdk/dpdk-16.04/drivers/net/bnx2x/bnx2x_ethdev.c:430:
> bnx2x_common_dev_init: Assertion `sc->bar[0].base_addr' failed.
> Aborted (core dumped)
>
> and this is the call stack from dump file:
>
> (gdb) bt
> #0 0x000000363c032625 in raise () from /lib64/libc.so.6
> #1 0x000000363c033e05 in abort () from /lib64/libc.so.6
> #2 0x000000363c02b74e in __assert_fail_base () from /lib64/libc.so.6
> #3 0x000000363c02b810 in __assert_fail () from /lib64/libc.so.6
> #4 0x00000000004b8ed3 in bnx2x_common_dev_init ()
> #5 0x0000000000473bfa in rte_eth_dev_init ()
> #6 0x0000000000482422 in rte_eal_pci_probe ()
> #7 0x00000000004779b8 in rte_eal_init ()
> #8 0x0000000000420e7c in main ()
>
> All this steps on different server with Intel NICs make the application
> (server side e client side) run with no problems at all.
>
> Is there someone that could help us to understand this issue with Qlogic
> (Broadcom) driver?
>
> Thanks in advance,
> David Coen
>
>
>
^ permalink raw reply [flat|nested] 4+ messages in thread
* [dpdk-users] R: Error in rte_eal_init using dpdk 16.04 and Qlogic (Broadcom) bnx2x driver in a SECONDARY process
2016-05-04 19:07 [dpdk-users] Error in rte_eal_init using dpdk 16.04 and Qlogic (Broadcom) bnx2x driver in a SECONDARY process dawid_jurek
@ 2016-05-05 11:02 ` David Coen
2016-05-05 20:21 ` Thomas Monjalon
0 siblings, 1 reply; 4+ messages in thread
From: David Coen @ 2016-05-05 11:02 UTC (permalink / raw)
To: users; +Cc: 'dawid_jurek'
Hello Dawid,
thank you for your answer.
Probably you are right, but I think that the issue is one or more steps before.
Looking at Intel and Mellanox driver sources ("ixgbe_ethdev.c" and "mlx4.c" respectively),
I found that, in the driver function called by rte_eal_init to initialize pci devs, is mandatory
to distinguish if the process is running as SECONDARY (--proc-type=secondary in EAL prameters).
For example:
eth_ixgbe_dev_init() function of the ixgbe Intel driver, at line 988 :
/*
* For secondary processes, we don't initialise any further as primary
* has already done this work. Only check we don't need a different
* RX and TX function.
*/
if (rte_eal_process_type() != RTE_PROC_PRIMARY){
... some code lines here ...
return 0;
}
While in Mellanox mlx4 driver there is a check in ALL driver functions that makes them return
immediatelly if the running process is of type "secondary".
That means that a driver has to take into account that the process could be a SECONDARY process...
but in Qlogic (Broadcom) bnx2x driver I didn't find anything about that.
Do you think I've missed something ?
Thank you for your help,
David
> -----Messaggio originale-----
> Da: dawid_jurek [mailto:dawid_jurek@vp.pl]
> Inviato: mercoledì 4 maggio 2016 21:08
> A: users@dpdk.org; David Coen
> Oggetto: Re: [dpdk-users] Error in rte_eal_init using dpdk 16.04 and Qlogic
> (Broadcom) bnx2x driver in a SECONDARY process
>
> Hello David,
>
> By looking on driver sources (bnx2x_ethdev.c) assertion `sc-
> >bar[0].base_addr' fail is caused by pci_dev->mem_resource[0].addr ==
> NULL. It means that MMIO mapping of region nr. 0 failed.
> The reason of this may be print reported on ring level some time before
> crash:
>
> "EAL: memzone_reserve_aligned_thread_unsafe(): memzone
> <RG_MP_log_history> already exists
> RING: Cannot reserve memory."
>
> I suspect some memory allocation issue but not 100% sure.
>
> Regards,
> Dawid
>
>
> > Hello,
> >
> > We are trying to run the
> > "dpdk-16.04\examples\multi_process\client_server_mp" sample
> > application on a RHEL6.7 linux server, with DPDK 16.04 and the
> > following network
> > interfaces:
> >
> > Network devices using DPDK-compatible driver
> > ============================================
> > 0000:10:00.0 'BCM57840 NetXtreme II 10 Gigabit Ethernet' drv=igb_uio
> > unused=bnx2x
> > 0000:10:00.1 'BCM57840 NetXtreme II 10 Gigabit Ethernet' drv=igb_uio
> > unused=bnx2x ===================================
> >
> > PMD: bnx2x_print_adapter_info(): ASIC : 0x168d1010
> > PMD: bnx2x_print_adapter_info(): Rev : B1
> > PMD: bnx2x_print_adapter_info(): Bus PCIe : 8,
> > PMD: bnx2x_print_adapter_info(): 5 Gbps
> > PMD: bnx2x_print_adapter_info(): Flags :
> > PMD: bnx2x_print_adapter_info(): MSI
> > PMD: bnx2x_print_adapter_info(): |
> > PMD: bnx2x_print_adapter_info(): MSI-X
> > PMD: bnx2x_print_adapter_info(): Queues :
> > PMD: bnx2x_print_adapter_info(): RSS : 1
> > PMD: bnx2x_print_adapter_info(): DPDK : DPDK 16.04.0
> > PMD: bnx2x_print_adapter_info(): Driver : BNX2X PMD 1.78.18_1.0.0
> > PMD: bnx2x_print_adapter_info(): Firmware : 7.2.51
> > PMD: bnx2x_print_adapter_info(): Bootcode : 7.13.24
> > PMD: bnx2x_print_adapter_info():
> >
> > We can run the server process, without problems, with the following
> command.
> >
> > ./mp_server -c 6 -n 4 -- -p 3 -n 1
> >
> > .than, in a second shell, we run the client process with the
> > following command .
> >
> > ./mp_client -c 8 -n 4 --proc-type=auto -- -n 0
> >
> > .and we always get this output:
> >
> > EAL: Detected lcore 0 as core 0 on socket 0
> > EAL: Detected lcore 1 as core 1 on socket 0
> > EAL: Detected lcore 2 as core 2 on socket 0
> > EAL: Detected lcore 3 as core 3 on socket 0
> > EAL: Detected lcore 4 as core 4 on socket 0
> > EAL: Detected lcore 5 as core 5 on socket 0
> > EAL: Detected lcore 6 as core 0 on socket 1
> > EAL: Detected lcore 7 as core 1 on socket 1
> > EAL: Detected lcore 8 as core 2 on socket 1
> > EAL: Detected lcore 9 as core 3 on socket 1
> > EAL: Detected lcore 10 as core 4 on socket 1
> > EAL: Detected lcore 11 as core 5 on socket 1
> > EAL: Support maximum 128 logical core(s) by configuration.
> > EAL: Detected 12 lcore(s)
> > EAL: Auto-detected process type: SECONDARY
> > EAL: Setting up physically contiguous memory...
> > EAL: Analysing 16 files
> > EAL: Mapped segment 0 of size 0x200000000
> > EAL: Mapped segment 1 of size 0x200000000
> > EAL: memzone_reserve_aligned_thread_unsafe(): memzone
> > <RG_MP_log_history> already exists
> > RING: Cannot reserve memory
> > EAL: TSC frequency is ~2400000 KHz
> > EAL: Master lcore 3 is ready (tid=f7fec880;cpuset=[3])
> > PMD: rte_bnx2x_pmd_init(): >>
> > PMD: rte_bnx2xvf_pmd_init(): >>
> > EAL: PCI device 0000:10:00.0 on NUMA socket 0
> > EAL: probe driver: 14e4:16a1 rte_bnx2x_pmd
> > EAL: PCI memory mapped at 0x7fffc0000000
> > EAL: PCI memory mapped at 0x7fffc0800000
> > EAL: PCI memory mapped at 0x7fffc1000000
> > PMD: eth_bnx2x_dev_init(): >>
> > PMD: bnx2x_common_dev_init(): >>
> > mp_client: /root/dpdk/dpdk-
> 16.04/drivers/net/bnx2x/bnx2x_ethdev.c:430:
> > bnx2x_common_dev_init: Assertion `sc->bar[0].base_addr' failed.
> > Aborted (core dumped)
> >
> > and this is the call stack from dump file:
> >
> > (gdb) bt
> > #0 0x000000363c032625 in raise () from /lib64/libc.so.6
> > #1 0x000000363c033e05 in abort () from /lib64/libc.so.6
> > #2 0x000000363c02b74e in __assert_fail_base () from /lib64/libc.so.6
> > #3 0x000000363c02b810 in __assert_fail () from /lib64/libc.so.6
> > #4 0x00000000004b8ed3 in bnx2x_common_dev_init ()
> > #5 0x0000000000473bfa in rte_eth_dev_init ()
> > #6 0x0000000000482422 in rte_eal_pci_probe ()
> > #7 0x00000000004779b8 in rte_eal_init ()
> > #8 0x0000000000420e7c in main ()
> >
> > All this steps on different server with Intel NICs make the
> > application (server side e client side) run with no problems at all.
> >
> > Is there someone that could help us to understand this issue with
> > Qlogic
> > (Broadcom) driver?
> >
> > Thanks in advance,
> > David Coen
> >
> >
> >
>
>
^ permalink raw reply [flat|nested] 4+ messages in thread
* Re: [dpdk-users] R: Error in rte_eal_init using dpdk 16.04 and Qlogic (Broadcom) bnx2x driver in a SECONDARY process
2016-05-05 11:02 ` [dpdk-users] R: " David Coen
@ 2016-05-05 20:21 ` Thomas Monjalon
0 siblings, 0 replies; 4+ messages in thread
From: Thomas Monjalon @ 2016-05-05 20:21 UTC (permalink / raw)
To: David Coen
Cc: users, 'dawid_jurek', sony.chacko, harish.patil, rasesh.mody
2016-05-05 13:02, David Coen:
> Looking at Intel and Mellanox driver sources ("ixgbe_ethdev.c" and "mlx4.c" respectively),
> I found that, in the driver function called by rte_eal_init to initialize pci devs, is mandatory
> to distinguish if the process is running as SECONDARY (--proc-type=secondary in EAL prameters).
>
> For example:
> eth_ixgbe_dev_init() function of the ixgbe Intel driver, at line 988 :
>
> /*
> * For secondary processes, we don't initialise any further as primary
> * has already done this work. Only check we don't need a different
> * RX and TX function.
> */
> if (rte_eal_process_type() != RTE_PROC_PRIMARY){
> ... some code lines here ...
> return 0;
> }
>
> While in Mellanox mlx4 driver there is a check in ALL driver functions that makes them return
> immediatelly if the running process is of type "secondary".
>
> That means that a driver has to take into account that the process could be a SECONDARY process...
This is called "Multiprocess aware" in doc/guides/nics/overview.rst.
> but in Qlogic (Broadcom) bnx2x driver I didn't find anything about that.
It seems the driver do not comply with this multiprocess requirement.
> Do you think I've missed something ?
You are probably right :)
^ permalink raw reply [flat|nested] 4+ messages in thread
* [dpdk-users] Error in rte_eal_init using dpdk 16.04 and Qlogic (Broadcom) bnx2x driver in a SECONDARY process
@ 2016-05-04 9:15 David Coen
0 siblings, 0 replies; 4+ messages in thread
From: David Coen @ 2016-05-04 9:15 UTC (permalink / raw)
To: users
Hello,
We are trying to run the
"dpdk-16.04\examples\multi_process\client_server_mp" sample application
on a RHEL6.7 linux server, with DPDK 16.04 and the following network
interfaces:
Network devices using DPDK-compatible driver
============================================
0000:10:00.0 'BCM57840 NetXtreme II 10 Gigabit Ethernet' drv=igb_uio
unused=bnx2x
0000:10:00.1 'BCM57840 NetXtreme II 10 Gigabit Ethernet' drv=igb_uio
unused=bnx2x
===================================
PMD: bnx2x_print_adapter_info(): ASIC : 0x168d1010
PMD: bnx2x_print_adapter_info(): Rev : B1
PMD: bnx2x_print_adapter_info(): Bus PCIe : 8,
PMD: bnx2x_print_adapter_info(): 5 Gbps
PMD: bnx2x_print_adapter_info(): Flags :
PMD: bnx2x_print_adapter_info(): MSI
PMD: bnx2x_print_adapter_info(): |
PMD: bnx2x_print_adapter_info(): MSI-X
PMD: bnx2x_print_adapter_info(): Queues :
PMD: bnx2x_print_adapter_info(): RSS : 1
PMD: bnx2x_print_adapter_info(): DPDK : DPDK 16.04.0
PMD: bnx2x_print_adapter_info(): Driver : BNX2X PMD 1.78.18_1.0.0
PMD: bnx2x_print_adapter_info(): Firmware : 7.2.51
PMD: bnx2x_print_adapter_info(): Bootcode : 7.13.24
PMD: bnx2x_print_adapter_info():
We can run the server process, without problems, with the following command.
./mp_server -c 6 -n 4 -- -p 3 -n 1
.than, in a second shell, we run the client process with the following
command .
./mp_client -c 8 -n 4 --proc-type=auto -- -n 0
.and we always get this output:
EAL: Detected lcore 0 as core 0 on socket 0
EAL: Detected lcore 1 as core 1 on socket 0
EAL: Detected lcore 2 as core 2 on socket 0
EAL: Detected lcore 3 as core 3 on socket 0
EAL: Detected lcore 4 as core 4 on socket 0
EAL: Detected lcore 5 as core 5 on socket 0
EAL: Detected lcore 6 as core 0 on socket 1
EAL: Detected lcore 7 as core 1 on socket 1
EAL: Detected lcore 8 as core 2 on socket 1
EAL: Detected lcore 9 as core 3 on socket 1
EAL: Detected lcore 10 as core 4 on socket 1
EAL: Detected lcore 11 as core 5 on socket 1
EAL: Support maximum 128 logical core(s) by configuration.
EAL: Detected 12 lcore(s)
EAL: Auto-detected process type: SECONDARY
EAL: Setting up physically contiguous memory...
EAL: Analysing 16 files
EAL: Mapped segment 0 of size 0x200000000
EAL: Mapped segment 1 of size 0x200000000
EAL: memzone_reserve_aligned_thread_unsafe(): memzone <RG_MP_log_history>
already exists
RING: Cannot reserve memory
EAL: TSC frequency is ~2400000 KHz
EAL: Master lcore 3 is ready (tid=f7fec880;cpuset=[3])
PMD: rte_bnx2x_pmd_init(): >>
PMD: rte_bnx2xvf_pmd_init(): >>
EAL: PCI device 0000:10:00.0 on NUMA socket 0
EAL: probe driver: 14e4:16a1 rte_bnx2x_pmd
EAL: PCI memory mapped at 0x7fffc0000000
EAL: PCI memory mapped at 0x7fffc0800000
EAL: PCI memory mapped at 0x7fffc1000000
PMD: eth_bnx2x_dev_init(): >>
PMD: bnx2x_common_dev_init(): >>
mp_client: /root/dpdk/dpdk-16.04/drivers/net/bnx2x/bnx2x_ethdev.c:430:
bnx2x_common_dev_init: Assertion `sc->bar[0].base_addr' failed.
Aborted (core dumped)
and this is the call stack from dump file:
(gdb) bt
#0 0x000000363c032625 in raise () from /lib64/libc.so.6
#1 0x000000363c033e05 in abort () from /lib64/libc.so.6
#2 0x000000363c02b74e in __assert_fail_base () from /lib64/libc.so.6
#3 0x000000363c02b810 in __assert_fail () from /lib64/libc.so.6
#4 0x00000000004b8ed3 in bnx2x_common_dev_init ()
#5 0x0000000000473bfa in rte_eth_dev_init ()
#6 0x0000000000482422 in rte_eal_pci_probe ()
#7 0x00000000004779b8 in rte_eal_init ()
#8 0x0000000000420e7c in main ()
All this steps on different server with Intel NICs make the application
(server side e client side) run with no problems at all.
Is there someone that could help us to understand this issue with Qlogic
(Broadcom) driver?
Thanks in advance,
David Coen
^ permalink raw reply [flat|nested] 4+ messages in thread
end of thread, other threads:[~2016-05-05 20:21 UTC | newest]
Thread overview: 4+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2016-05-04 19:07 [dpdk-users] Error in rte_eal_init using dpdk 16.04 and Qlogic (Broadcom) bnx2x driver in a SECONDARY process dawid_jurek
2016-05-05 11:02 ` [dpdk-users] R: " David Coen
2016-05-05 20:21 ` Thomas Monjalon
-- strict thread matches above, loose matches on Subject: below --
2016-05-04 9:15 [dpdk-users] " David Coen
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).