* [dpdk-dev] SR-IOV/DPDK/VPP with vfio-pci
@ 2016-06-01 18:21 Chandrasekar Kannan
2016-06-05 15:47 ` Zhang, Helin
0 siblings, 1 reply; 3+ messages in thread
From: Chandrasekar Kannan @ 2016-06-01 18:21 UTC (permalink / raw)
To: dev
I have been attempting to VPP (dpdk) application with SR-IOV enabled.
Using the vfio-pci driver/i40e/XL710nics/.
I'm encountering a SEGV on the i40e code path.
Has anyone else seen this behaviour ?.
full mail thread discussion with vpp-dev is here -
https://lists.fd.io/pipermail/vpp-dev/2016-June/001348.html
--------------------------------
here's a backtrace during vpp crash...
Program received signal SIGSEGV, Segmentation fault.
[Switching to Thread 0x7f2b811fd700 (LWP 8725)]
0x0000000000455539 in rx_recv_pkts ()
(gdb) backtrace
#0 0x0000000000455539 in rx_recv_pkts ()
#1 0x0000000000455e76 in i40e_recv_pkts_bulk_alloc ()
#2 0x00007f2be43d8bf7 in rte_eth_rx_burst (nb_pkts=256,
rx_pkts=0x7f2bb1e66a80, queue_id=1, port_id=<optimized out>)
at
/w/workspace/vpp-merge-master-centos7/build-root/install-vpp-native/dpdk/include/rte_ethdev.h:2641
#3 dpdk_rx_burst (dm=0x7f2be4673980 <dpdk_main>, queue_id=1,
xd=0x7f2bb1898540)
at
/w/workspace/vpp-merge-master-centos7/build-data/../vnet/vnet/devices/dpdk/dpdk_priv.h:64
#4 dpdk_device_input (dm=0x7f2be4673980 <dpdk_main>, queue_id=1,
cpu_index=<optimized out>, node=0x7f2bb1ef59fc, xd=<optimized out>)
at
/w/workspace/vpp-merge-master-centos7/build-data/../vnet/vnet/devices/dpdk/node.c:508
#5 dpdk_input_rss (vm=<optimized out>, node=0x7f2bb1ef59fc, f=<optimized
out>)
at
/w/workspace/vpp-merge-master-centos7/build-data/../vnet/vnet/devices/dpdk/node.c:821
#6 0x00007f2be47eb77a in dispatch_node (vm=vm@entry=0x7f2bb1f02cd4,
node=node@entry=0x7f2bb1ef59fc, type=type@entry=VLIB_NODE_TYPE_INPUT,
dispatch_state=dispatch_state@entry=VLIB_NODE_STATE_POLLING,
frame=frame@entry=0x0, last_time_stamp=1482097661494035)
at
/w/workspace/vpp-merge-master-centos7/build-data/../vlib/vlib/main.c:996
#7 0x00007f2be43db6d8 in dpdk_worker_thread_internal (have_io_threads=0,
callback=0x0, vm=0x7f2bb1f02cd4)
at
/w/workspace/vpp-merge-master-centos7/build-data/../vnet/vnet/devices/dpdk/threads.c:209
#8 dpdk_worker_thread (w=<optimized out>, io_name=<optimized out>,
callback=0x0)
at
/w/workspace/vpp-merge-master-centos7/build-data/../vnet/vnet/devices/dpdk/threads.c:265
#9 0x00007f2be3791530 in clib_calljmp () at
/w/workspace/vpp-merge-master-centos7/build-data/../vppinfra/vppinfra/longjmp.S:110
#10 0x00007f2b811fcc40 in ?? ()
#11 0x00000000004eb9c7 in eal_thread_loop ()
---Type <return> to continue, or q <return> to quit---
#12 0x0000000000000000 in ?? ()
(gdb)
(gdb) thread apply all bt
Thread 10 (Thread 0x7f2b8c685700 (LWP 8722)):
#0 0x00007f2be32584ad in accept () at ../sysdeps/unix/syscall-template.S:81
#1 0x00000000004ef178 in pci_vfio_mp_sync_thread ()
#2 0x00007f2be3251dc5 in start_thread (arg=0x7f2b8c685700) at
pthread_create.c:308
#3 0x00007f2be2992ced in clone () at
../sysdeps/unix/sysv/linux/x86_64/clone.S:113
Thread 9 (Thread 0x7f2b821ff700 (LWP 8723)):
#0 0x00007f2be29932c3 in epoll_wait () at
../sysdeps/unix/syscall-template.S:81
#1 0x00000000004efe04 in eal_intr_thread_main ()
#2 0x00007f2be3251dc5 in start_thread (arg=0x7f2b821ff700) at
pthread_create.c:308
#3 0x00007f2be2992ced in clone () at
../sysdeps/unix/sysv/linux/x86_64/clone.S:113
Thread 8 (Thread 0x7f2b819fe700 (LWP 8724)):
#0 0x000000000045526b in rx_recv_pkts ()
#1 0x0000000000455e76 in i40e_recv_pkts_bulk_alloc ()
#2 0x00007f2be43d8bf7 in rte_eth_rx_burst (nb_pkts=256,
rx_pkts=0x7f2bb1e75580, queue_id=0, port_id=<optimized out>)
at
/w/workspace/vpp-merge-master-centos7/build-root/install-vpp-native/dpdk/include/rte_ethdev.h:2641
#3 dpdk_rx_burst (dm=0x7f2be4673980 <dpdk_main>, queue_id=0,
xd=0x7f2bb1a1fd40)
at
/w/workspace/vpp-merge-master-centos7/build-data/../vnet/vnet/devices/dpdk/dpdk_priv.h:64
---Type <return> to continue, or q <return> to quit---
#4 dpdk_device_input (dm=0x7f2be4673980 <dpdk_main>, queue_id=0,
cpu_index=<optimized out>, node=0x7f2bb1ea18cc, xd=<optimized out>)
at
/w/workspace/vpp-merge-master-centos7/build-data/../vnet/vnet/devices/dpdk/node.c:508
#5 dpdk_input_rss (vm=<optimized out>, node=0x7f2bb1ea18cc, f=<optimized
out>)
at
/w/workspace/vpp-merge-master-centos7/build-data/../vnet/vnet/devices/dpdk/node.c:821
#6 0x00007f2be47eb77a in dispatch_node (vm=vm@entry=0x7f2bb1f237d4,
node=node@entry=0x7f2bb1ea18cc, type=type@entry=VLIB_NODE_TYPE_INPUT,
dispatch_state=dispatch_state@entry=VLIB_NODE_STATE_POLLING,
frame=frame@entry=0x0, last_time_stamp=1482097666654800)
at
/w/workspace/vpp-merge-master-centos7/build-data/../vlib/vlib/main.c:996
#7 0x00007f2be43db6d8 in dpdk_worker_thread_internal (have_io_threads=0,
callback=0x0, vm=0x7f2bb1f237d4)
at
/w/workspace/vpp-merge-master-centos7/build-data/../vnet/vnet/devices/dpdk/threads.c:209
#8 dpdk_worker_thread (w=<optimized out>, io_name=<optimized out>,
callback=0x0)
at
/w/workspace/vpp-merge-master-centos7/build-data/../vnet/vnet/devices/dpdk/threads.c:265
#9 0x00007f2be3791530 in clib_calljmp () at
/w/workspace/vpp-merge-master-centos7/build-data/../vppinfra/vppinfra/longjmp.S:110
#10 0x00007f2b819fdc40 in ?? ()
#11 0x00000000004eb9c7 in eal_thread_loop ()
#12 0x0000000000000000 in ?? ()
Thread 7 (Thread 0x7f2b811fd700 (LWP 8725)):
#0 0x0000000000455539 in rx_recv_pkts ()
#1 0x0000000000455e76 in i40e_recv_pkts_bulk_alloc ()
#2 0x00007f2be43d8bf7 in rte_eth_rx_burst (nb_pkts=256,
rx_pkts=0x7f2bb1e66a80, queue_id=1, port_id=<optimized out>)
---Type <return> to continue, or q <return> to quit---
at
/w/workspace/vpp-merge-master-centos7/build-root/install-vpp-native/dpdk/include/rte_ethdev.h:2641
#3 dpdk_rx_burst (dm=0x7f2be4673980 <dpdk_main>, queue_id=1,
xd=0x7f2bb1898540)
at
/w/workspace/vpp-merge-master-centos7/build-data/../vnet/vnet/devices/dpdk/dpdk_priv.h:64
#4 dpdk_device_input (dm=0x7f2be4673980 <dpdk_main>, queue_id=1,
cpu_index=<optimized out>, node=0x7f2bb1ef59fc, xd=<optimized out>)
at
/w/workspace/vpp-merge-master-centos7/build-data/../vnet/vnet/devices/dpdk/node.c:508
#5 dpdk_input_rss (vm=<optimized out>, node=0x7f2bb1ef59fc, f=<optimized
out>)
at
/w/workspace/vpp-merge-master-centos7/build-data/../vnet/vnet/devices/dpdk/node.c:821
#6 0x00007f2be47eb77a in dispatch_node (vm=vm@entry=0x7f2bb1f02cd4,
node=node@entry=0x7f2bb1ef59fc, type=type@entry=VLIB_NODE_TYPE_INPUT,
dispatch_state=dispatch_state@entry=VLIB_NODE_STATE_POLLING,
frame=frame@entry=0x0, last_time_stamp=1482097661494035)
at
/w/workspace/vpp-merge-master-centos7/build-data/../vlib/vlib/main.c:996
#7 0x00007f2be43db6d8 in dpdk_worker_thread_internal (have_io_threads=0,
callback=0x0, vm=0x7f2bb1f02cd4)
at
/w/workspace/vpp-merge-master-centos7/build-data/../vnet/vnet/devices/dpdk/threads.c:209
#8 dpdk_worker_thread (w=<optimized out>, io_name=<optimized out>,
callback=0x0)
at
/w/workspace/vpp-merge-master-centos7/build-data/../vnet/vnet/devices/dpdk/threads.c:265
#9 0x00007f2be3791530 in clib_calljmp () at
/w/workspace/vpp-merge-master-centos7/build-data/../vppinfra/vppinfra/longjmp.S:110
#10 0x00007f2b811fcc40 in ?? ()
#11 0x00000000004eb9c7 in eal_thread_loop ()
#12 0x0000000000000000 in ?? ()
Thread 6 (Thread 0x7f2b809fc700 (LWP 8726)):
---Type <return> to continue, or q <return> to quit---
#0 0x000000000045539e in rx_recv_pkts ()
#1 0x0000000000455e76 in i40e_recv_pkts_bulk_alloc ()
#2 0x00007f2be43d8bf7 in rte_eth_rx_burst (nb_pkts=256,
rx_pkts=0x7f2bb1da8a00, queue_id=2, port_id=<optimized out>)
at
/w/workspace/vpp-merge-master-centos7/build-root/install-vpp-native/dpdk/include/rte_ethdev.h:2641
#3 dpdk_rx_burst (dm=0x7f2be4673980 <dpdk_main>, queue_id=2,
xd=0x7f2bb1898540)
at
/w/workspace/vpp-merge-master-centos7/build-data/../vnet/vnet/devices/dpdk/dpdk_priv.h:64
#4 dpdk_device_input (dm=0x7f2be4673980 <dpdk_main>, queue_id=2,
cpu_index=<optimized out>, node=0x7f2bb1ebc7f8, xd=<optimized out>)
at
/w/workspace/vpp-merge-master-centos7/build-data/../vnet/vnet/devices/dpdk/node.c:508
#5 dpdk_input_rss (vm=<optimized out>, node=0x7f2bb1ebc7f8, f=<optimized
out>)
at
/w/workspace/vpp-merge-master-centos7/build-data/../vnet/vnet/devices/dpdk/node.c:821
#6 0x00007f2be47eb77a in dispatch_node (vm=vm@entry=0x7f2bb1ee29d4,
node=node@entry=0x7f2bb1ebc7f8, type=type@entry=VLIB_NODE_TYPE_INPUT,
dispatch_state=dispatch_state@entry=VLIB_NODE_STATE_POLLING,
frame=frame@entry=0x0, last_time_stamp=1482097658614609)
at
/w/workspace/vpp-merge-master-centos7/build-data/../vlib/vlib/main.c:996
#7 0x00007f2be43db6d8 in dpdk_worker_thread_internal (have_io_threads=0,
callback=0x0, vm=0x7f2bb1ee29d4)
at
/w/workspace/vpp-merge-master-centos7/build-data/../vnet/vnet/devices/dpdk/threads.c:209
#8 dpdk_worker_thread (w=<optimized out>, io_name=<optimized out>,
callback=0x0)
at
/w/workspace/vpp-merge-master-centos7/build-data/../vnet/vnet/devices/dpdk/threads.c:265
#9 0x00007f2be3791530 in clib_calljmp () at
/w/workspace/vpp-merge-master-centos7/build-data/../vppinfra/vppinfra/longjmp.S:110
#10 0x00007f2b809fbc40 in ?? ()
#11 0x00000000004eb9c7 in eal_thread_loop ()
---Type <return> to continue, or q <return> to quit---
#12 0x0000000000000000 in ?? ()
Thread 5 (Thread 0x7f2b801fb700 (LWP 8727)):
#0 0x00007f2be47eb728 in dispatch_node (vm=vm@entry=0x7f2bb1ec2614,
node=node@entry=0x7f2bb1da5e60, type=type@entry=VLIB_NODE_TYPE_INPUT,
dispatch_state=dispatch_state@entry=VLIB_NODE_STATE_POLLING,
frame=frame@entry=0x0, last_time_stamp=1482097666665442)
at
/w/workspace/vpp-merge-master-centos7/build-data/../vlib/vlib/main.c:924
#1 0x00007f2be43db6d8 in dpdk_worker_thread_internal (have_io_threads=0,
callback=0x0, vm=0x7f2bb1ec2614)
at
/w/workspace/vpp-merge-master-centos7/build-data/../vnet/vnet/devices/dpdk/threads.c:209
#2 dpdk_worker_thread (w=<optimized out>, io_name=<optimized out>,
callback=0x0)
at
/w/workspace/vpp-merge-master-centos7/build-data/../vnet/vnet/devices/dpdk/threads.c:265
#3 0x00007f2be3791530 in clib_calljmp () at
/w/workspace/vpp-merge-master-centos7/build-data/../vppinfra/vppinfra/longjmp.S:110
#4 0x00007f2b801fac40 in ?? ()
#5 0x00000000004eb9c7 in eal_thread_loop ()
#6 0x0000000000000000 in ?? ()
Thread 4 (Thread 0x7f2b7f9fa700 (LWP 8728)):
#0 0x00007f2be43db6cc in dpdk_worker_thread_internal (have_io_threads=0,
callback=0x0, vm=0x7f2bb1ea2294)
at
/w/workspace/vpp-merge-master-centos7/build-data/../vnet/vnet/devices/dpdk/threads.c:209
#1 dpdk_worker_thread (w=<optimized out>, io_name=<optimized out>,
callback=0x0)
at
/w/workspace/vpp-merge-master-centos7/build-data/../vnet/vnet/devices/dpdk/threads.c:265
---Type <return> to continue, or q <return> to quit---
#2 0x00007f2be3791530 in clib_calljmp () at
/w/workspace/vpp-merge-master-centos7/build-data/../vppinfra/vppinfra/longjmp.S:110
#3 0x00007f2b7f9f9c40 in ?? ()
#4 0x00000000004eb9c7 in eal_thread_loop ()
#5 0x0000000000000000 in ?? ()
Thread 3 (Thread 0x7f2b7f1f9700 (LWP 8729)):
#0 vlib_node_runtime_update_stats (n_clocks=205, n_vectors=0, n_calls=1,
node=0x7f2bb1d782d0, vm=0x7f2bb1e81f94)
at
/w/workspace/vpp-merge-master-centos7/build-data/../vlib/vlib/main.c:629
#1 dispatch_node (vm=vm@entry=0x7f2bb1e81f94, node=node@entry=0x7f2bb1d782d0,
type=type@entry=VLIB_NODE_TYPE_INPUT,
dispatch_state=dispatch_state@entry=VLIB_NODE_STATE_POLLING,
frame=frame@entry=0x0, last_time_stamp=1482097666680456)
at
/w/workspace/vpp-merge-master-centos7/build-data/../vlib/vlib/main.c:1005
#2 0x00007f2be43db6d8 in dpdk_worker_thread_internal (have_io_threads=0,
callback=0x0, vm=0x7f2bb1e81f94)
at
/w/workspace/vpp-merge-master-centos7/build-data/../vnet/vnet/devices/dpdk/threads.c:209
#3 dpdk_worker_thread (w=<optimized out>, io_name=<optimized out>,
callback=0x0)
at
/w/workspace/vpp-merge-master-centos7/build-data/../vnet/vnet/devices/dpdk/threads.c:265
#4 0x00007f2be3791530 in clib_calljmp () at
/w/workspace/vpp-merge-master-centos7/build-data/../vppinfra/vppinfra/longjmp.S:110
#5 0x00007f2b7f1f8c40 in ?? ()
#6 0x00000000004eb9c7 in eal_thread_loop ()
#7 0x0000000000000000 in ?? ()
---Type <return> to continue, or q <return> to quit---
Thread 2 (Thread 0x7f2b7e9f8700 (LWP 8730)):
#0 0x00007f2be325896d in nanosleep () at
../sysdeps/unix/syscall-template.S:81
#1 0x0000000000413916 in ip46_fib_stats_delay.isra.6 ()
#2 0x00000000004190b6 in stats_thread_fn ()
#3 0x00007f2be3791530 in clib_calljmp () at
/w/workspace/vpp-merge-master-centos7/build-data/../vppinfra/vppinfra/longjmp.S:110
#4 0x00007f2b7e9f7da0 in ?? ()
#5 0x00007f2be3251dc5 in start_thread (arg=0x0) at pthread_create.c:308
#6 0x0000000000000000 in ?? ()
Thread 1 (Thread 0x7f2be52898c0 (LWP 8721)):
#0 0x00007f2be2992f27 in __GI_epoll_pwait (epfd=5, events=0x7f2bb2155424,
maxevents=256, timeout=timeout@entry=10,
set=set@entry=0x7f2be4c522e0 <unblock_all_signals.16333>) at
../sysdeps/unix/sysv/linux/epoll_pwait.c:48
#1 0x00007f2be4a429fb in linux_epoll_input (vm=0x7e13c0
<vlib_global_main>, node=<optimized out>, frame=<optimized out>)
at
/w/workspace/vpp-merge-master-centos7/build-data/../vlib/vlib/unix/input.c:146
#2 0x00007f2be47eb77a in dispatch_node (vm=0x7e13c0 <vlib_global_main>,
node=0x7f2bb2254cc0, type=<optimized out>, dispatch_state=<optimized out>,
frame=<optimized out>, last_time_stamp=1482097636972836) at
/w/workspace/vpp-merge-master-centos7/build-data/../vlib/vlib/main.c:996
#3 0x00007f2be47ebea1 in vlib_main_loop (vm=0x7e13c0 <vlib_global_main>)
at /w/workspace/vpp-merge-master-centos7/build-data/../vlib/vlib/main.c:1403
#4 vlib_main (vm=vm@entry=0x7e13c0 <vlib_global_main>, input=input@entry
=0x7f2bb2786fa0)
at
/w/workspace/vpp-merge-master-centos7/build-data/../vlib/vlib/main.c:1615
#5 0x00007f2be4a435f3 in thread0 (arg=8262592) at
/w/workspace/vpp-merge-master-centos7/build-data/../vlib/vlib/unix/main.c:425
---Type <return> to continue, or q <return> to quit---
#6 0x00007f2be3791530 in clib_calljmp () at
/w/workspace/vpp-merge-master-centos7/build-data/../vppinfra/vppinfra/longjmp.S:110
#7 0x00007ffc98a68d60 in ?? ()
#8 0x00007f2be4a43e31 in vlib_unix_main (argc=<optimized out>,
argv=<optimized out>)
at
/w/workspace/vpp-merge-master-centos7/build-data/../vlib/vlib/unix/main.c:485
#9 0x0000000000000000 in ?? ()
(gdb)
(gdb)
(gdb)
(gdb)
-------------------------------------------------------------------
^ permalink raw reply [flat|nested] 3+ messages in thread
* Re: [dpdk-dev] SR-IOV/DPDK/VPP with vfio-pci
2016-06-01 18:21 [dpdk-dev] SR-IOV/DPDK/VPP with vfio-pci Chandrasekar Kannan
@ 2016-06-05 15:47 ` Zhang, Helin
2016-07-02 0:09 ` Chandrasekar Kannan
0 siblings, 1 reply; 3+ messages in thread
From: Zhang, Helin @ 2016-06-05 15:47 UTC (permalink / raw)
To: Chandrasekar Kannan, Wu, Jingjing, Tao, Zhe; +Cc: dev
Hi Kannan
Could you help to provide more details of your test cases? How to reproduce that issue? It would be better to try with DPDK example applications.
I think the kernel driver version, DPDK version, XL710 firmware version are the basic information we want to know.
If possible, I'd suggest you to try with latest version of above and try if the issue is still there.
Thank you very much!
Regards,
Helin
> -----Original Message-----
> From: dev [mailto:dev-bounces@dpdk.org] On Behalf Of Chandrasekar
> Kannan
> Sent: Thursday, June 2, 2016 2:22 AM
> To: dev@dpdk.org
> Subject: [dpdk-dev] SR-IOV/DPDK/VPP with vfio-pci
>
> I have been attempting to VPP (dpdk) application with SR-IOV enabled.
> Using the vfio-pci driver/i40e/XL710nics/.
> I'm encountering a SEGV on the i40e code path.
>
> Has anyone else seen this behaviour ?.
>
> full mail thread discussion with vpp-dev is here -
> https://lists.fd.io/pipermail/vpp-dev/2016-June/001348.html
>
>
> --------------------------------
> here's a backtrace during vpp crash...
>
> Program received signal SIGSEGV, Segmentation fault.
> [Switching to Thread 0x7f2b811fd700 (LWP 8725)]
> 0x0000000000455539 in rx_recv_pkts ()
> (gdb) backtrace
> #0 0x0000000000455539 in rx_recv_pkts ()
> #1 0x0000000000455e76 in i40e_recv_pkts_bulk_alloc ()
> #2 0x00007f2be43d8bf7 in rte_eth_rx_burst (nb_pkts=256,
> rx_pkts=0x7f2bb1e66a80, queue_id=1, port_id=<optimized out>)
> at
> /w/workspace/vpp-merge-master-centos7/build-root/install-vpp-
> native/dpdk/include/rte_ethdev.h:2641
> #3 dpdk_rx_burst (dm=0x7f2be4673980 <dpdk_main>, queue_id=1,
> xd=0x7f2bb1898540)
> at
> /w/workspace/vpp-merge-master-centos7/build-
> data/../vnet/vnet/devices/dpdk/dpdk_priv.h:64
> #4 dpdk_device_input (dm=0x7f2be4673980 <dpdk_main>, queue_id=1,
> cpu_index=<optimized out>, node=0x7f2bb1ef59fc, xd=<optimized out>)
> at
> /w/workspace/vpp-merge-master-centos7/build-
> data/../vnet/vnet/devices/dpdk/node.c:508
> #5 dpdk_input_rss (vm=<optimized out>, node=0x7f2bb1ef59fc,
> f=<optimized
> out>)
> at
> /w/workspace/vpp-merge-master-centos7/build-
> data/../vnet/vnet/devices/dpdk/node.c:821
> #6 0x00007f2be47eb77a in dispatch_node (vm=vm@entry=0x7f2bb1f02cd4,
> node=node@entry=0x7f2bb1ef59fc,
> type=type@entry=VLIB_NODE_TYPE_INPUT,
> dispatch_state=dispatch_state@entry=VLIB_NODE_STATE_POLLING,
> frame=frame@entry=0x0, last_time_stamp=1482097661494035)
> at
> /w/workspace/vpp-merge-master-centos7/build-data/../vlib/vlib/main.c:996
> #7 0x00007f2be43db6d8 in dpdk_worker_thread_internal
> (have_io_threads=0, callback=0x0, vm=0x7f2bb1f02cd4)
> at
> /w/workspace/vpp-merge-master-centos7/build-
> data/../vnet/vnet/devices/dpdk/threads.c:209
> #8 dpdk_worker_thread (w=<optimized out>, io_name=<optimized out>,
> callback=0x0)
> at
> /w/workspace/vpp-merge-master-centos7/build-
> data/../vnet/vnet/devices/dpdk/threads.c:265
> #9 0x00007f2be3791530 in clib_calljmp () at
> /w/workspace/vpp-merge-master-centos7/build-
> data/../vppinfra/vppinfra/longjmp.S:110
> #10 0x00007f2b811fcc40 in ?? ()
> #11 0x00000000004eb9c7 in eal_thread_loop () ---Type <return> to continue,
> or q <return> to quit---
> #12 0x0000000000000000 in ?? ()
> (gdb)
> (gdb) thread apply all bt
>
> Thread 10 (Thread 0x7f2b8c685700 (LWP 8722)):
> #0 0x00007f2be32584ad in accept () at ../sysdeps/unix/syscall-template.S:81
> #1 0x00000000004ef178 in pci_vfio_mp_sync_thread ()
> #2 0x00007f2be3251dc5 in start_thread (arg=0x7f2b8c685700) at
> pthread_create.c:308
> #3 0x00007f2be2992ced in clone () at
> ../sysdeps/unix/sysv/linux/x86_64/clone.S:113
>
> Thread 9 (Thread 0x7f2b821ff700 (LWP 8723)):
> #0 0x00007f2be29932c3 in epoll_wait () at
> ../sysdeps/unix/syscall-template.S:81
> #1 0x00000000004efe04 in eal_intr_thread_main ()
> #2 0x00007f2be3251dc5 in start_thread (arg=0x7f2b821ff700) at
> pthread_create.c:308
> #3 0x00007f2be2992ced in clone () at
> ../sysdeps/unix/sysv/linux/x86_64/clone.S:113
>
> Thread 8 (Thread 0x7f2b819fe700 (LWP 8724)):
> #0 0x000000000045526b in rx_recv_pkts ()
> #1 0x0000000000455e76 in i40e_recv_pkts_bulk_alloc ()
> #2 0x00007f2be43d8bf7 in rte_eth_rx_burst (nb_pkts=256,
> rx_pkts=0x7f2bb1e75580, queue_id=0, port_id=<optimized out>)
> at
> /w/workspace/vpp-merge-master-centos7/build-root/install-vpp-
> native/dpdk/include/rte_ethdev.h:2641
> #3 dpdk_rx_burst (dm=0x7f2be4673980 <dpdk_main>, queue_id=0,
> xd=0x7f2bb1a1fd40)
> at
> /w/workspace/vpp-merge-master-centos7/build-
> data/../vnet/vnet/devices/dpdk/dpdk_priv.h:64
> ---Type <return> to continue, or q <return> to quit---
> #4 dpdk_device_input (dm=0x7f2be4673980 <dpdk_main>, queue_id=0,
> cpu_index=<optimized out>, node=0x7f2bb1ea18cc, xd=<optimized out>)
> at
> /w/workspace/vpp-merge-master-centos7/build-
> data/../vnet/vnet/devices/dpdk/node.c:508
> #5 dpdk_input_rss (vm=<optimized out>, node=0x7f2bb1ea18cc,
> f=<optimized
> out>)
> at
> /w/workspace/vpp-merge-master-centos7/build-
> data/../vnet/vnet/devices/dpdk/node.c:821
> #6 0x00007f2be47eb77a in dispatch_node (vm=vm@entry=0x7f2bb1f237d4,
> node=node@entry=0x7f2bb1ea18cc,
> type=type@entry=VLIB_NODE_TYPE_INPUT,
> dispatch_state=dispatch_state@entry=VLIB_NODE_STATE_POLLING,
> frame=frame@entry=0x0, last_time_stamp=1482097666654800)
> at
> /w/workspace/vpp-merge-master-centos7/build-data/../vlib/vlib/main.c:996
> #7 0x00007f2be43db6d8 in dpdk_worker_thread_internal
> (have_io_threads=0, callback=0x0, vm=0x7f2bb1f237d4)
> at
> /w/workspace/vpp-merge-master-centos7/build-
> data/../vnet/vnet/devices/dpdk/threads.c:209
> #8 dpdk_worker_thread (w=<optimized out>, io_name=<optimized out>,
> callback=0x0)
> at
> /w/workspace/vpp-merge-master-centos7/build-
> data/../vnet/vnet/devices/dpdk/threads.c:265
> #9 0x00007f2be3791530 in clib_calljmp () at
> /w/workspace/vpp-merge-master-centos7/build-
> data/../vppinfra/vppinfra/longjmp.S:110
> #10 0x00007f2b819fdc40 in ?? ()
> #11 0x00000000004eb9c7 in eal_thread_loop ()
> #12 0x0000000000000000 in ?? ()
>
> Thread 7 (Thread 0x7f2b811fd700 (LWP 8725)):
> #0 0x0000000000455539 in rx_recv_pkts ()
> #1 0x0000000000455e76 in i40e_recv_pkts_bulk_alloc ()
> #2 0x00007f2be43d8bf7 in rte_eth_rx_burst (nb_pkts=256,
> rx_pkts=0x7f2bb1e66a80, queue_id=1, port_id=<optimized out>) ---Type
> <return> to continue, or q <return> to quit---
> at
> /w/workspace/vpp-merge-master-centos7/build-root/install-vpp-
> native/dpdk/include/rte_ethdev.h:2641
> #3 dpdk_rx_burst (dm=0x7f2be4673980 <dpdk_main>, queue_id=1,
> xd=0x7f2bb1898540)
> at
> /w/workspace/vpp-merge-master-centos7/build-
> data/../vnet/vnet/devices/dpdk/dpdk_priv.h:64
> #4 dpdk_device_input (dm=0x7f2be4673980 <dpdk_main>, queue_id=1,
> cpu_index=<optimized out>, node=0x7f2bb1ef59fc, xd=<optimized out>)
> at
> /w/workspace/vpp-merge-master-centos7/build-
> data/../vnet/vnet/devices/dpdk/node.c:508
> #5 dpdk_input_rss (vm=<optimized out>, node=0x7f2bb1ef59fc,
> f=<optimized
> out>)
> at
> /w/workspace/vpp-merge-master-centos7/build-
> data/../vnet/vnet/devices/dpdk/node.c:821
> #6 0x00007f2be47eb77a in dispatch_node (vm=vm@entry=0x7f2bb1f02cd4,
> node=node@entry=0x7f2bb1ef59fc,
> type=type@entry=VLIB_NODE_TYPE_INPUT,
> dispatch_state=dispatch_state@entry=VLIB_NODE_STATE_POLLING,
> frame=frame@entry=0x0, last_time_stamp=1482097661494035)
> at
> /w/workspace/vpp-merge-master-centos7/build-data/../vlib/vlib/main.c:996
> #7 0x00007f2be43db6d8 in dpdk_worker_thread_internal
> (have_io_threads=0, callback=0x0, vm=0x7f2bb1f02cd4)
> at
> /w/workspace/vpp-merge-master-centos7/build-
> data/../vnet/vnet/devices/dpdk/threads.c:209
> #8 dpdk_worker_thread (w=<optimized out>, io_name=<optimized out>,
> callback=0x0)
> at
> /w/workspace/vpp-merge-master-centos7/build-
> data/../vnet/vnet/devices/dpdk/threads.c:265
> #9 0x00007f2be3791530 in clib_calljmp () at
> /w/workspace/vpp-merge-master-centos7/build-
> data/../vppinfra/vppinfra/longjmp.S:110
> #10 0x00007f2b811fcc40 in ?? ()
> #11 0x00000000004eb9c7 in eal_thread_loop ()
> #12 0x0000000000000000 in ?? ()
>
> Thread 6 (Thread 0x7f2b809fc700 (LWP 8726)):
> ---Type <return> to continue, or q <return> to quit---
> #0 0x000000000045539e in rx_recv_pkts ()
> #1 0x0000000000455e76 in i40e_recv_pkts_bulk_alloc ()
> #2 0x00007f2be43d8bf7 in rte_eth_rx_burst (nb_pkts=256,
> rx_pkts=0x7f2bb1da8a00, queue_id=2, port_id=<optimized out>)
> at
> /w/workspace/vpp-merge-master-centos7/build-root/install-vpp-
> native/dpdk/include/rte_ethdev.h:2641
> #3 dpdk_rx_burst (dm=0x7f2be4673980 <dpdk_main>, queue_id=2,
> xd=0x7f2bb1898540)
> at
> /w/workspace/vpp-merge-master-centos7/build-
> data/../vnet/vnet/devices/dpdk/dpdk_priv.h:64
> #4 dpdk_device_input (dm=0x7f2be4673980 <dpdk_main>, queue_id=2,
> cpu_index=<optimized out>, node=0x7f2bb1ebc7f8, xd=<optimized out>)
> at
> /w/workspace/vpp-merge-master-centos7/build-
> data/../vnet/vnet/devices/dpdk/node.c:508
> #5 dpdk_input_rss (vm=<optimized out>, node=0x7f2bb1ebc7f8,
> f=<optimized
> out>)
> at
> /w/workspace/vpp-merge-master-centos7/build-
> data/../vnet/vnet/devices/dpdk/node.c:821
> #6 0x00007f2be47eb77a in dispatch_node (vm=vm@entry=0x7f2bb1ee29d4,
> node=node@entry=0x7f2bb1ebc7f8,
> type=type@entry=VLIB_NODE_TYPE_INPUT,
> dispatch_state=dispatch_state@entry=VLIB_NODE_STATE_POLLING,
> frame=frame@entry=0x0, last_time_stamp=1482097658614609)
> at
> /w/workspace/vpp-merge-master-centos7/build-data/../vlib/vlib/main.c:996
> #7 0x00007f2be43db6d8 in dpdk_worker_thread_internal
> (have_io_threads=0, callback=0x0, vm=0x7f2bb1ee29d4)
> at
> /w/workspace/vpp-merge-master-centos7/build-
> data/../vnet/vnet/devices/dpdk/threads.c:209
> #8 dpdk_worker_thread (w=<optimized out>, io_name=<optimized out>,
> callback=0x0)
> at
> /w/workspace/vpp-merge-master-centos7/build-
> data/../vnet/vnet/devices/dpdk/threads.c:265
> #9 0x00007f2be3791530 in clib_calljmp () at
> /w/workspace/vpp-merge-master-centos7/build-
> data/../vppinfra/vppinfra/longjmp.S:110
> #10 0x00007f2b809fbc40 in ?? ()
> #11 0x00000000004eb9c7 in eal_thread_loop () ---Type <return> to continue,
> or q <return> to quit---
> #12 0x0000000000000000 in ?? ()
>
> Thread 5 (Thread 0x7f2b801fb700 (LWP 8727)):
> #0 0x00007f2be47eb728 in dispatch_node (vm=vm@entry=0x7f2bb1ec2614,
> node=node@entry=0x7f2bb1da5e60,
> type=type@entry=VLIB_NODE_TYPE_INPUT,
> dispatch_state=dispatch_state@entry=VLIB_NODE_STATE_POLLING,
> frame=frame@entry=0x0, last_time_stamp=1482097666665442)
> at
> /w/workspace/vpp-merge-master-centos7/build-data/../vlib/vlib/main.c:924
> #1 0x00007f2be43db6d8 in dpdk_worker_thread_internal
> (have_io_threads=0, callback=0x0, vm=0x7f2bb1ec2614)
> at
> /w/workspace/vpp-merge-master-centos7/build-
> data/../vnet/vnet/devices/dpdk/threads.c:209
> #2 dpdk_worker_thread (w=<optimized out>, io_name=<optimized out>,
> callback=0x0)
> at
> /w/workspace/vpp-merge-master-centos7/build-
> data/../vnet/vnet/devices/dpdk/threads.c:265
> #3 0x00007f2be3791530 in clib_calljmp () at
> /w/workspace/vpp-merge-master-centos7/build-
> data/../vppinfra/vppinfra/longjmp.S:110
> #4 0x00007f2b801fac40 in ?? ()
> #5 0x00000000004eb9c7 in eal_thread_loop ()
> #6 0x0000000000000000 in ?? ()
>
> Thread 4 (Thread 0x7f2b7f9fa700 (LWP 8728)):
> #0 0x00007f2be43db6cc in dpdk_worker_thread_internal
> (have_io_threads=0, callback=0x0, vm=0x7f2bb1ea2294)
> at
> /w/workspace/vpp-merge-master-centos7/build-
> data/../vnet/vnet/devices/dpdk/threads.c:209
> #1 dpdk_worker_thread (w=<optimized out>, io_name=<optimized out>,
> callback=0x0)
> at
> /w/workspace/vpp-merge-master-centos7/build-
> data/../vnet/vnet/devices/dpdk/threads.c:265
> ---Type <return> to continue, or q <return> to quit---
> #2 0x00007f2be3791530 in clib_calljmp () at
> /w/workspace/vpp-merge-master-centos7/build-
> data/../vppinfra/vppinfra/longjmp.S:110
> #3 0x00007f2b7f9f9c40 in ?? ()
> #4 0x00000000004eb9c7 in eal_thread_loop ()
> #5 0x0000000000000000 in ?? ()
>
> Thread 3 (Thread 0x7f2b7f1f9700 (LWP 8729)):
> #0 vlib_node_runtime_update_stats (n_clocks=205, n_vectors=0, n_calls=1,
> node=0x7f2bb1d782d0, vm=0x7f2bb1e81f94)
> at
> /w/workspace/vpp-merge-master-centos7/build-data/../vlib/vlib/main.c:629
> #1 dispatch_node (vm=vm@entry=0x7f2bb1e81f94,
> node=node@entry=0x7f2bb1d782d0,
> type=type@entry=VLIB_NODE_TYPE_INPUT,
> dispatch_state=dispatch_state@entry=VLIB_NODE_STATE_POLLING,
> frame=frame@entry=0x0, last_time_stamp=1482097666680456)
> at
> /w/workspace/vpp-merge-master-centos7/build-
> data/../vlib/vlib/main.c:1005
> #2 0x00007f2be43db6d8 in dpdk_worker_thread_internal
> (have_io_threads=0, callback=0x0, vm=0x7f2bb1e81f94)
> at
> /w/workspace/vpp-merge-master-centos7/build-
> data/../vnet/vnet/devices/dpdk/threads.c:209
> #3 dpdk_worker_thread (w=<optimized out>, io_name=<optimized out>,
> callback=0x0)
> at
> /w/workspace/vpp-merge-master-centos7/build-
> data/../vnet/vnet/devices/dpdk/threads.c:265
> #4 0x00007f2be3791530 in clib_calljmp () at
> /w/workspace/vpp-merge-master-centos7/build-
> data/../vppinfra/vppinfra/longjmp.S:110
> #5 0x00007f2b7f1f8c40 in ?? ()
> #6 0x00000000004eb9c7 in eal_thread_loop ()
> #7 0x0000000000000000 in ?? ()
>
> ---Type <return> to continue, or q <return> to quit--- Thread 2 (Thread
> 0x7f2b7e9f8700 (LWP 8730)):
> #0 0x00007f2be325896d in nanosleep () at
> ../sysdeps/unix/syscall-template.S:81
> #1 0x0000000000413916 in ip46_fib_stats_delay.isra.6 ()
> #2 0x00000000004190b6 in stats_thread_fn ()
> #3 0x00007f2be3791530 in clib_calljmp () at
> /w/workspace/vpp-merge-master-centos7/build-
> data/../vppinfra/vppinfra/longjmp.S:110
> #4 0x00007f2b7e9f7da0 in ?? ()
> #5 0x00007f2be3251dc5 in start_thread (arg=0x0) at pthread_create.c:308
> #6 0x0000000000000000 in ?? ()
>
> Thread 1 (Thread 0x7f2be52898c0 (LWP 8721)):
> #0 0x00007f2be2992f27 in __GI_epoll_pwait (epfd=5,
> events=0x7f2bb2155424, maxevents=256, timeout=timeout@entry=10,
> set=set@entry=0x7f2be4c522e0 <unblock_all_signals.16333>) at
> ../sysdeps/unix/sysv/linux/epoll_pwait.c:48
> #1 0x00007f2be4a429fb in linux_epoll_input (vm=0x7e13c0
> <vlib_global_main>, node=<optimized out>, frame=<optimized out>)
> at
> /w/workspace/vpp-merge-master-centos7/build-
> data/../vlib/vlib/unix/input.c:146
> #2 0x00007f2be47eb77a in dispatch_node (vm=0x7e13c0
> <vlib_global_main>, node=0x7f2bb2254cc0, type=<optimized out>,
> dispatch_state=<optimized out>,
> frame=<optimized out>, last_time_stamp=1482097636972836) at
> /w/workspace/vpp-merge-master-centos7/build-data/../vlib/vlib/main.c:996
> #3 0x00007f2be47ebea1 in vlib_main_loop (vm=0x7e13c0
> <vlib_global_main>) at /w/workspace/vpp-merge-master-centos7/build-
> data/../vlib/vlib/main.c:1403
> #4 vlib_main (vm=vm@entry=0x7e13c0 <vlib_global_main>,
> input=input@entry
> =0x7f2bb2786fa0)
> at
> /w/workspace/vpp-merge-master-centos7/build-
> data/../vlib/vlib/main.c:1615
> #5 0x00007f2be4a435f3 in thread0 (arg=8262592) at
> /w/workspace/vpp-merge-master-centos7/build-
> data/../vlib/vlib/unix/main.c:425
> ---Type <return> to continue, or q <return> to quit---
> #6 0x00007f2be3791530 in clib_calljmp () at
> /w/workspace/vpp-merge-master-centos7/build-
> data/../vppinfra/vppinfra/longjmp.S:110
> #7 0x00007ffc98a68d60 in ?? ()
> #8 0x00007f2be4a43e31 in vlib_unix_main (argc=<optimized out>,
> argv=<optimized out>)
> at
> /w/workspace/vpp-merge-master-centos7/build-
> data/../vlib/vlib/unix/main.c:485
> #9 0x0000000000000000 in ?? ()
> (gdb)
> (gdb)
> (gdb)
> (gdb)
> -------------------------------------------------------------------
^ permalink raw reply [flat|nested] 3+ messages in thread
* Re: [dpdk-dev] SR-IOV/DPDK/VPP with vfio-pci
2016-06-05 15:47 ` Zhang, Helin
@ 2016-07-02 0:09 ` Chandrasekar Kannan
0 siblings, 0 replies; 3+ messages in thread
From: Chandrasekar Kannan @ 2016-07-02 0:09 UTC (permalink / raw)
To: Zhang, Helin; +Cc: Wu, Jingjing, Tao, Zhe, dev
[root@localhost ~]# modinfo i40e
filename:
/lib/modules/3.10.0-327.18.2.el7.x86_64/updates/drivers/net/ethernet/intel/i40e/i40e.ko
version: 1.4.25
[root@localhost ~]# modinfo i40evf
filename:
/lib/modules/3.10.0-327.18.2.el7.x86_64/updates/drivers/net/ethernet/intel/i40evf/i40evf.ko
version: 1.4.15
dpdk ver : 16.04
nic type : 83:00.0 Ethernet controller: Intel Corporation Ethernet
Controller XL710 for 40GbE QSFP+ (rev 02)
[root@localhost ~]# ethtool -i ens11
driver: i40e
version: 1.4.25
firmware-version: 5.04 0x8000253b 0.0.0
bus-info: 0000:83:00.0
supports-statistics: yes
supports-test: yes
supports-eeprom-access: yes
supports-register-dump: yes
supports-priv-flags: yes
[root@localhost ~]#
Testcase:
- Enable SRIOV
# echo 4 > /sys/class/net/ens11/device/sriov_numvfs
83:00.0 Ethernet controller: Intel Corporation Ethernet Controller XL710
for 40GbE QSFP+ (rev 02)
83:02.0 Ethernet controller: Intel Corporation XL710/X710 Virtual Function
(rev 02)
83:02.1 Ethernet controller: Intel Corporation XL710/X710 Virtual Function
(rev 02)
83:02.2 Ethernet controller: Intel Corporation XL710/X710 Virtual Function
(rev 02)
83:02.3 Ethernet controller: Intel Corporation XL710/X710 Virtual Function
(rev 02)
85:00.0 Ethernet controller: Intel Corporation Ethernet Controller XL710
for 40GbE QSFP+ (rev 02)
- modprobe vfio-pci
[root@localhost ~]#
/root/ckannan/vpp/build-root/build-vpp-native/dpdk/dpdk-16.04/tools/dpdk_nic_bind.py
--bind=vfio-pci 0000:83:02.0
[root@localhost ~]#
/root/ckannan/vpp/build-root/build-vpp-native/dpdk/dpdk-16.04/tools/dpdk_nic_bind.py
--bind=vfio-pci 0000:83:02.1
[root@localhost ~]#
/root/ckannan/vpp/build-root/build-vpp-native/dpdk/dpdk-16.04/tools/dpdk_nic_bind.py
--bind=vfio-pci 0000:83:02.2
[root@localhost ~]#
/root/ckannan/vpp/build-root/build-vpp-native/dpdk/dpdk-16.04/tools/dpdk_nic_bind.py
--bind=vfio-pci 0000:83:02.3
- launch testpmd
./testpmd -l 13,14,15,16 -n 4 -- -i -a --nb-cores=3 --nb-ports=1
I start seeing errors like these...
Jul 01 17:23:29 localhost.localdomain kernel: dmar: DRHD: handling fault
status reg 2
Jul 01 17:23:29 localhost.localdomain kernel: dmar: DMAR:[DMA Write]
Request device [83:02.0] fault addr 0
DMAR:[fault reason 05] PTE
Write access is not set
Jul 01 17:23:29 localhost.localdomain kernel: i40e 0000:83:00.0: TX driver
issue detected, PF reset issued
Jul 01 17:23:29 localhost.localdomain kernel: i40e 0000:83:00.0: TX driver
issue detected on VF 0
On Sun, Jun 5, 2016 at 8:47 AM, Zhang, Helin <helin.zhang@intel.com> wrote:
> Hi Kannan
>
> Could you help to provide more details of your test cases? How to
> reproduce that issue? It would be better to try with DPDK example
> applications.
> I think the kernel driver version, DPDK version, XL710 firmware version
> are the basic information we want to know.
> If possible, I'd suggest you to try with latest version of above and try
> if the issue is still there.
> Thank you very much!
>
> Regards,
> Helin
>
> > -----Original Message-----
> > From: dev [mailto:dev-bounces@dpdk.org] On Behalf Of Chandrasekar
> > Kannan
> > Sent: Thursday, June 2, 2016 2:22 AM
> > To: dev@dpdk.org
> > Subject: [dpdk-dev] SR-IOV/DPDK/VPP with vfio-pci
> >
> > I have been attempting to VPP (dpdk) application with SR-IOV enabled.
> > Using the vfio-pci driver/i40e/XL710nics/.
> > I'm encountering a SEGV on the i40e code path.
> >
> > Has anyone else seen this behaviour ?.
> >
> > full mail thread discussion with vpp-dev is here -
> > https://lists.fd.io/pipermail/vpp-dev/2016-June/001348.html
> >
> >
> > --------------------------------
> > here's a backtrace during vpp crash...
> >
> > Program received signal SIGSEGV, Segmentation fault.
> > [Switching to Thread 0x7f2b811fd700 (LWP 8725)]
> > 0x0000000000455539 in rx_recv_pkts ()
> > (gdb) backtrace
> > #0 0x0000000000455539 in rx_recv_pkts ()
> > #1 0x0000000000455e76 in i40e_recv_pkts_bulk_alloc ()
> > #2 0x00007f2be43d8bf7 in rte_eth_rx_burst (nb_pkts=256,
> > rx_pkts=0x7f2bb1e66a80, queue_id=1, port_id=<optimized out>)
> > at
> > /w/workspace/vpp-merge-master-centos7/build-root/install-vpp-
> > native/dpdk/include/rte_ethdev.h:2641
> > #3 dpdk_rx_burst (dm=0x7f2be4673980 <dpdk_main>, queue_id=1,
> > xd=0x7f2bb1898540)
> > at
> > /w/workspace/vpp-merge-master-centos7/build-
> > data/../vnet/vnet/devices/dpdk/dpdk_priv.h:64
> > #4 dpdk_device_input (dm=0x7f2be4673980 <dpdk_main>, queue_id=1,
> > cpu_index=<optimized out>, node=0x7f2bb1ef59fc, xd=<optimized out>)
> > at
> > /w/workspace/vpp-merge-master-centos7/build-
> > data/../vnet/vnet/devices/dpdk/node.c:508
> > #5 dpdk_input_rss (vm=<optimized out>, node=0x7f2bb1ef59fc,
> > f=<optimized
> > out>)
> > at
> > /w/workspace/vpp-merge-master-centos7/build-
> > data/../vnet/vnet/devices/dpdk/node.c:821
> > #6 0x00007f2be47eb77a in dispatch_node (vm=vm@entry=0x7f2bb1f02cd4,
> > node=node@entry=0x7f2bb1ef59fc,
> > type=type@entry=VLIB_NODE_TYPE_INPUT,
> > dispatch_state=dispatch_state@entry=VLIB_NODE_STATE_POLLING,
> > frame=frame@entry=0x0, last_time_stamp=1482097661494035)
> > at
> > /w/workspace/vpp-merge-master-centos7/build-data/../vlib/vlib/main.c:996
> > #7 0x00007f2be43db6d8 in dpdk_worker_thread_internal
> > (have_io_threads=0, callback=0x0, vm=0x7f2bb1f02cd4)
> > at
> > /w/workspace/vpp-merge-master-centos7/build-
> > data/../vnet/vnet/devices/dpdk/threads.c:209
> > #8 dpdk_worker_thread (w=<optimized out>, io_name=<optimized out>,
> > callback=0x0)
> > at
> > /w/workspace/vpp-merge-master-centos7/build-
> > data/../vnet/vnet/devices/dpdk/threads.c:265
> > #9 0x00007f2be3791530 in clib_calljmp () at
> > /w/workspace/vpp-merge-master-centos7/build-
> > data/../vppinfra/vppinfra/longjmp.S:110
> > #10 0x00007f2b811fcc40 in ?? ()
> > #11 0x00000000004eb9c7 in eal_thread_loop () ---Type <return> to
> continue,
> > or q <return> to quit---
> > #12 0x0000000000000000 in ?? ()
> > (gdb)
> > (gdb) thread apply all bt
> >
> > Thread 10 (Thread 0x7f2b8c685700 (LWP 8722)):
> > #0 0x00007f2be32584ad in accept () at
> ../sysdeps/unix/syscall-template.S:81
> > #1 0x00000000004ef178 in pci_vfio_mp_sync_thread ()
> > #2 0x00007f2be3251dc5 in start_thread (arg=0x7f2b8c685700) at
> > pthread_create.c:308
> > #3 0x00007f2be2992ced in clone () at
> > ../sysdeps/unix/sysv/linux/x86_64/clone.S:113
> >
> > Thread 9 (Thread 0x7f2b821ff700 (LWP 8723)):
> > #0 0x00007f2be29932c3 in epoll_wait () at
> > ../sysdeps/unix/syscall-template.S:81
> > #1 0x00000000004efe04 in eal_intr_thread_main ()
> > #2 0x00007f2be3251dc5 in start_thread (arg=0x7f2b821ff700) at
> > pthread_create.c:308
> > #3 0x00007f2be2992ced in clone () at
> > ../sysdeps/unix/sysv/linux/x86_64/clone.S:113
> >
> > Thread 8 (Thread 0x7f2b819fe700 (LWP 8724)):
> > #0 0x000000000045526b in rx_recv_pkts ()
> > #1 0x0000000000455e76 in i40e_recv_pkts_bulk_alloc ()
> > #2 0x00007f2be43d8bf7 in rte_eth_rx_burst (nb_pkts=256,
> > rx_pkts=0x7f2bb1e75580, queue_id=0, port_id=<optimized out>)
> > at
> > /w/workspace/vpp-merge-master-centos7/build-root/install-vpp-
> > native/dpdk/include/rte_ethdev.h:2641
> > #3 dpdk_rx_burst (dm=0x7f2be4673980 <dpdk_main>, queue_id=0,
> > xd=0x7f2bb1a1fd40)
> > at
> > /w/workspace/vpp-merge-master-centos7/build-
> > data/../vnet/vnet/devices/dpdk/dpdk_priv.h:64
> > ---Type <return> to continue, or q <return> to quit---
> > #4 dpdk_device_input (dm=0x7f2be4673980 <dpdk_main>, queue_id=0,
> > cpu_index=<optimized out>, node=0x7f2bb1ea18cc, xd=<optimized out>)
> > at
> > /w/workspace/vpp-merge-master-centos7/build-
> > data/../vnet/vnet/devices/dpdk/node.c:508
> > #5 dpdk_input_rss (vm=<optimized out>, node=0x7f2bb1ea18cc,
> > f=<optimized
> > out>)
> > at
> > /w/workspace/vpp-merge-master-centos7/build-
> > data/../vnet/vnet/devices/dpdk/node.c:821
> > #6 0x00007f2be47eb77a in dispatch_node (vm=vm@entry=0x7f2bb1f237d4,
> > node=node@entry=0x7f2bb1ea18cc,
> > type=type@entry=VLIB_NODE_TYPE_INPUT,
> > dispatch_state=dispatch_state@entry=VLIB_NODE_STATE_POLLING,
> > frame=frame@entry=0x0, last_time_stamp=1482097666654800)
> > at
> > /w/workspace/vpp-merge-master-centos7/build-data/../vlib/vlib/main.c:996
> > #7 0x00007f2be43db6d8 in dpdk_worker_thread_internal
> > (have_io_threads=0, callback=0x0, vm=0x7f2bb1f237d4)
> > at
> > /w/workspace/vpp-merge-master-centos7/build-
> > data/../vnet/vnet/devices/dpdk/threads.c:209
> > #8 dpdk_worker_thread (w=<optimized out>, io_name=<optimized out>,
> > callback=0x0)
> > at
> > /w/workspace/vpp-merge-master-centos7/build-
> > data/../vnet/vnet/devices/dpdk/threads.c:265
> > #9 0x00007f2be3791530 in clib_calljmp () at
> > /w/workspace/vpp-merge-master-centos7/build-
> > data/../vppinfra/vppinfra/longjmp.S:110
> > #10 0x00007f2b819fdc40 in ?? ()
> > #11 0x00000000004eb9c7 in eal_thread_loop ()
> > #12 0x0000000000000000 in ?? ()
> >
> > Thread 7 (Thread 0x7f2b811fd700 (LWP 8725)):
> > #0 0x0000000000455539 in rx_recv_pkts ()
> > #1 0x0000000000455e76 in i40e_recv_pkts_bulk_alloc ()
> > #2 0x00007f2be43d8bf7 in rte_eth_rx_burst (nb_pkts=256,
> > rx_pkts=0x7f2bb1e66a80, queue_id=1, port_id=<optimized out>) ---Type
> > <return> to continue, or q <return> to quit---
> > at
> > /w/workspace/vpp-merge-master-centos7/build-root/install-vpp-
> > native/dpdk/include/rte_ethdev.h:2641
> > #3 dpdk_rx_burst (dm=0x7f2be4673980 <dpdk_main>, queue_id=1,
> > xd=0x7f2bb1898540)
> > at
> > /w/workspace/vpp-merge-master-centos7/build-
> > data/../vnet/vnet/devices/dpdk/dpdk_priv.h:64
> > #4 dpdk_device_input (dm=0x7f2be4673980 <dpdk_main>, queue_id=1,
> > cpu_index=<optimized out>, node=0x7f2bb1ef59fc, xd=<optimized out>)
> > at
> > /w/workspace/vpp-merge-master-centos7/build-
> > data/../vnet/vnet/devices/dpdk/node.c:508
> > #5 dpdk_input_rss (vm=<optimized out>, node=0x7f2bb1ef59fc,
> > f=<optimized
> > out>)
> > at
> > /w/workspace/vpp-merge-master-centos7/build-
> > data/../vnet/vnet/devices/dpdk/node.c:821
> > #6 0x00007f2be47eb77a in dispatch_node (vm=vm@entry=0x7f2bb1f02cd4,
> > node=node@entry=0x7f2bb1ef59fc,
> > type=type@entry=VLIB_NODE_TYPE_INPUT,
> > dispatch_state=dispatch_state@entry=VLIB_NODE_STATE_POLLING,
> > frame=frame@entry=0x0, last_time_stamp=1482097661494035)
> > at
> > /w/workspace/vpp-merge-master-centos7/build-data/../vlib/vlib/main.c:996
> > #7 0x00007f2be43db6d8 in dpdk_worker_thread_internal
> > (have_io_threads=0, callback=0x0, vm=0x7f2bb1f02cd4)
> > at
> > /w/workspace/vpp-merge-master-centos7/build-
> > data/../vnet/vnet/devices/dpdk/threads.c:209
> > #8 dpdk_worker_thread (w=<optimized out>, io_name=<optimized out>,
> > callback=0x0)
> > at
> > /w/workspace/vpp-merge-master-centos7/build-
> > data/../vnet/vnet/devices/dpdk/threads.c:265
> > #9 0x00007f2be3791530 in clib_calljmp () at
> > /w/workspace/vpp-merge-master-centos7/build-
> > data/../vppinfra/vppinfra/longjmp.S:110
> > #10 0x00007f2b811fcc40 in ?? ()
> > #11 0x00000000004eb9c7 in eal_thread_loop ()
> > #12 0x0000000000000000 in ?? ()
> >
> > Thread 6 (Thread 0x7f2b809fc700 (LWP 8726)):
> > ---Type <return> to continue, or q <return> to quit---
> > #0 0x000000000045539e in rx_recv_pkts ()
> > #1 0x0000000000455e76 in i40e_recv_pkts_bulk_alloc ()
> > #2 0x00007f2be43d8bf7 in rte_eth_rx_burst (nb_pkts=256,
> > rx_pkts=0x7f2bb1da8a00, queue_id=2, port_id=<optimized out>)
> > at
> > /w/workspace/vpp-merge-master-centos7/build-root/install-vpp-
> > native/dpdk/include/rte_ethdev.h:2641
> > #3 dpdk_rx_burst (dm=0x7f2be4673980 <dpdk_main>, queue_id=2,
> > xd=0x7f2bb1898540)
> > at
> > /w/workspace/vpp-merge-master-centos7/build-
> > data/../vnet/vnet/devices/dpdk/dpdk_priv.h:64
> > #4 dpdk_device_input (dm=0x7f2be4673980 <dpdk_main>, queue_id=2,
> > cpu_index=<optimized out>, node=0x7f2bb1ebc7f8, xd=<optimized out>)
> > at
> > /w/workspace/vpp-merge-master-centos7/build-
> > data/../vnet/vnet/devices/dpdk/node.c:508
> > #5 dpdk_input_rss (vm=<optimized out>, node=0x7f2bb1ebc7f8,
> > f=<optimized
> > out>)
> > at
> > /w/workspace/vpp-merge-master-centos7/build-
> > data/../vnet/vnet/devices/dpdk/node.c:821
> > #6 0x00007f2be47eb77a in dispatch_node (vm=vm@entry=0x7f2bb1ee29d4,
> > node=node@entry=0x7f2bb1ebc7f8,
> > type=type@entry=VLIB_NODE_TYPE_INPUT,
> > dispatch_state=dispatch_state@entry=VLIB_NODE_STATE_POLLING,
> > frame=frame@entry=0x0, last_time_stamp=1482097658614609)
> > at
> > /w/workspace/vpp-merge-master-centos7/build-data/../vlib/vlib/main.c:996
> > #7 0x00007f2be43db6d8 in dpdk_worker_thread_internal
> > (have_io_threads=0, callback=0x0, vm=0x7f2bb1ee29d4)
> > at
> > /w/workspace/vpp-merge-master-centos7/build-
> > data/../vnet/vnet/devices/dpdk/threads.c:209
> > #8 dpdk_worker_thread (w=<optimized out>, io_name=<optimized out>,
> > callback=0x0)
> > at
> > /w/workspace/vpp-merge-master-centos7/build-
> > data/../vnet/vnet/devices/dpdk/threads.c:265
> > #9 0x00007f2be3791530 in clib_calljmp () at
> > /w/workspace/vpp-merge-master-centos7/build-
> > data/../vppinfra/vppinfra/longjmp.S:110
> > #10 0x00007f2b809fbc40 in ?? ()
> > #11 0x00000000004eb9c7 in eal_thread_loop () ---Type <return> to
> continue,
> > or q <return> to quit---
> > #12 0x0000000000000000 in ?? ()
> >
> > Thread 5 (Thread 0x7f2b801fb700 (LWP 8727)):
> > #0 0x00007f2be47eb728 in dispatch_node (vm=vm@entry=0x7f2bb1ec2614,
> > node=node@entry=0x7f2bb1da5e60,
> > type=type@entry=VLIB_NODE_TYPE_INPUT,
> > dispatch_state=dispatch_state@entry=VLIB_NODE_STATE_POLLING,
> > frame=frame@entry=0x0, last_time_stamp=1482097666665442)
> > at
> > /w/workspace/vpp-merge-master-centos7/build-data/../vlib/vlib/main.c:924
> > #1 0x00007f2be43db6d8 in dpdk_worker_thread_internal
> > (have_io_threads=0, callback=0x0, vm=0x7f2bb1ec2614)
> > at
> > /w/workspace/vpp-merge-master-centos7/build-
> > data/../vnet/vnet/devices/dpdk/threads.c:209
> > #2 dpdk_worker_thread (w=<optimized out>, io_name=<optimized out>,
> > callback=0x0)
> > at
> > /w/workspace/vpp-merge-master-centos7/build-
> > data/../vnet/vnet/devices/dpdk/threads.c:265
> > #3 0x00007f2be3791530 in clib_calljmp () at
> > /w/workspace/vpp-merge-master-centos7/build-
> > data/../vppinfra/vppinfra/longjmp.S:110
> > #4 0x00007f2b801fac40 in ?? ()
> > #5 0x00000000004eb9c7 in eal_thread_loop ()
> > #6 0x0000000000000000 in ?? ()
> >
> > Thread 4 (Thread 0x7f2b7f9fa700 (LWP 8728)):
> > #0 0x00007f2be43db6cc in dpdk_worker_thread_internal
> > (have_io_threads=0, callback=0x0, vm=0x7f2bb1ea2294)
> > at
> > /w/workspace/vpp-merge-master-centos7/build-
> > data/../vnet/vnet/devices/dpdk/threads.c:209
> > #1 dpdk_worker_thread (w=<optimized out>, io_name=<optimized out>,
> > callback=0x0)
> > at
> > /w/workspace/vpp-merge-master-centos7/build-
> > data/../vnet/vnet/devices/dpdk/threads.c:265
> > ---Type <return> to continue, or q <return> to quit---
> > #2 0x00007f2be3791530 in clib_calljmp () at
> > /w/workspace/vpp-merge-master-centos7/build-
> > data/../vppinfra/vppinfra/longjmp.S:110
> > #3 0x00007f2b7f9f9c40 in ?? ()
> > #4 0x00000000004eb9c7 in eal_thread_loop ()
> > #5 0x0000000000000000 in ?? ()
> >
> > Thread 3 (Thread 0x7f2b7f1f9700 (LWP 8729)):
> > #0 vlib_node_runtime_update_stats (n_clocks=205, n_vectors=0, n_calls=1,
> > node=0x7f2bb1d782d0, vm=0x7f2bb1e81f94)
> > at
> > /w/workspace/vpp-merge-master-centos7/build-data/../vlib/vlib/main.c:629
> > #1 dispatch_node (vm=vm@entry=0x7f2bb1e81f94,
> > node=node@entry=0x7f2bb1d782d0,
> > type=type@entry=VLIB_NODE_TYPE_INPUT,
> > dispatch_state=dispatch_state@entry=VLIB_NODE_STATE_POLLING,
> > frame=frame@entry=0x0, last_time_stamp=1482097666680456)
> > at
> > /w/workspace/vpp-merge-master-centos7/build-
> > data/../vlib/vlib/main.c:1005
> > #2 0x00007f2be43db6d8 in dpdk_worker_thread_internal
> > (have_io_threads=0, callback=0x0, vm=0x7f2bb1e81f94)
> > at
> > /w/workspace/vpp-merge-master-centos7/build-
> > data/../vnet/vnet/devices/dpdk/threads.c:209
> > #3 dpdk_worker_thread (w=<optimized out>, io_name=<optimized out>,
> > callback=0x0)
> > at
> > /w/workspace/vpp-merge-master-centos7/build-
> > data/../vnet/vnet/devices/dpdk/threads.c:265
> > #4 0x00007f2be3791530 in clib_calljmp () at
> > /w/workspace/vpp-merge-master-centos7/build-
> > data/../vppinfra/vppinfra/longjmp.S:110
> > #5 0x00007f2b7f1f8c40 in ?? ()
> > #6 0x00000000004eb9c7 in eal_thread_loop ()
> > #7 0x0000000000000000 in ?? ()
> >
> > ---Type <return> to continue, or q <return> to quit--- Thread 2 (Thread
> > 0x7f2b7e9f8700 (LWP 8730)):
> > #0 0x00007f2be325896d in nanosleep () at
> > ../sysdeps/unix/syscall-template.S:81
> > #1 0x0000000000413916 in ip46_fib_stats_delay.isra.6 ()
> > #2 0x00000000004190b6 in stats_thread_fn ()
> > #3 0x00007f2be3791530 in clib_calljmp () at
> > /w/workspace/vpp-merge-master-centos7/build-
> > data/../vppinfra/vppinfra/longjmp.S:110
> > #4 0x00007f2b7e9f7da0 in ?? ()
> > #5 0x00007f2be3251dc5 in start_thread (arg=0x0) at pthread_create.c:308
> > #6 0x0000000000000000 in ?? ()
> >
> > Thread 1 (Thread 0x7f2be52898c0 (LWP 8721)):
> > #0 0x00007f2be2992f27 in __GI_epoll_pwait (epfd=5,
> > events=0x7f2bb2155424, maxevents=256, timeout=timeout@entry=10,
> > set=set@entry=0x7f2be4c522e0 <unblock_all_signals.16333>) at
> > ../sysdeps/unix/sysv/linux/epoll_pwait.c:48
> > #1 0x00007f2be4a429fb in linux_epoll_input (vm=0x7e13c0
> > <vlib_global_main>, node=<optimized out>, frame=<optimized out>)
> > at
> > /w/workspace/vpp-merge-master-centos7/build-
> > data/../vlib/vlib/unix/input.c:146
> > #2 0x00007f2be47eb77a in dispatch_node (vm=0x7e13c0
> > <vlib_global_main>, node=0x7f2bb2254cc0, type=<optimized out>,
> > dispatch_state=<optimized out>,
> > frame=<optimized out>, last_time_stamp=1482097636972836) at
> > /w/workspace/vpp-merge-master-centos7/build-data/../vlib/vlib/main.c:996
> > #3 0x00007f2be47ebea1 in vlib_main_loop (vm=0x7e13c0
> > <vlib_global_main>) at /w/workspace/vpp-merge-master-centos7/build-
> > data/../vlib/vlib/main.c:1403
> > #4 vlib_main (vm=vm@entry=0x7e13c0 <vlib_global_main>,
> > input=input@entry
> > =0x7f2bb2786fa0)
> > at
> > /w/workspace/vpp-merge-master-centos7/build-
> > data/../vlib/vlib/main.c:1615
> > #5 0x00007f2be4a435f3 in thread0 (arg=8262592) at
> > /w/workspace/vpp-merge-master-centos7/build-
> > data/../vlib/vlib/unix/main.c:425
> > ---Type <return> to continue, or q <return> to quit---
> > #6 0x00007f2be3791530 in clib_calljmp () at
> > /w/workspace/vpp-merge-master-centos7/build-
> > data/../vppinfra/vppinfra/longjmp.S:110
> > #7 0x00007ffc98a68d60 in ?? ()
> > #8 0x00007f2be4a43e31 in vlib_unix_main (argc=<optimized out>,
> > argv=<optimized out>)
> > at
> > /w/workspace/vpp-merge-master-centos7/build-
> > data/../vlib/vlib/unix/main.c:485
> > #9 0x0000000000000000 in ?? ()
> > (gdb)
> > (gdb)
> > (gdb)
> > (gdb)
> > -------------------------------------------------------------------
>
^ permalink raw reply [flat|nested] 3+ messages in thread
end of thread, other threads:[~2016-07-02 0:09 UTC | newest]
Thread overview: 3+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2016-06-01 18:21 [dpdk-dev] SR-IOV/DPDK/VPP with vfio-pci Chandrasekar Kannan
2016-06-05 15:47 ` Zhang, Helin
2016-07-02 0:09 ` Chandrasekar Kannan
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).