From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mail-yw0-f182.google.com (mail-yw0-f182.google.com [209.85.161.182]) by dpdk.org (Postfix) with ESMTP id 64AE23794 for ; Wed, 1 Jun 2016 20:21:51 +0200 (CEST) Received: by mail-yw0-f182.google.com with SMTP id o16so26696052ywd.2 for ; Wed, 01 Jun 2016 11:21:51 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=console-to.20150623.gappssmtp.com; s=20150623; h=mime-version:date:message-id:subject:from:to; bh=tAUIPAu7cYyMznybO+uQJJHgmJH+yUoEPZJ6myYjBgw=; b=oZn+fmJCqGsC76InYeb2Zw7vQBU1KUGmXyh3x9Qfu4ImIlRqEyUU0S/I5A8uqNQ2xx mhEUnqali+K46eDgvVyLGqAz0Z+nspL/u7Ch1HbK8XwMy542drOtFqB51x6zBAsQAC20 U5zjxu+c1xfRmjHxE9K+A1x9dMwL0eJQIE6mbKz7HUAk/YSb60VOEhatYAQC/i4qpOlv y7XkkelSSE4GSDscX8qyv6wsPJQEsQVHXBz7SG0w0r1zYbdSHgXj/CnAXrJvldPLE6eh ZhsrtiFEt7hKIQ56+g48XYwjrdaza9R/dVRWCmc8Wy+3GOw60ZINoqTIlUycYH/SvG9Q cc4A== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:mime-version:date:message-id:subject:from:to; bh=tAUIPAu7cYyMznybO+uQJJHgmJH+yUoEPZJ6myYjBgw=; b=UDU04o9FwvsuZEw8w11+sjo8YwvPw0G3Gxrh18Ux2tmZWFiXflTCaiX7wPfjUG9cP2 fstyfCLeoyIoJ6eSFM4Xpn1GuzzQewKErlkGggKiFXvxlRvHIAd0y23wHFxtMAGG2kb5 PhxQiPrqTYnuV8xBMmWpumCEEwhBxEw0f+OCHR4npKdAi/XK93c2BloRzmXKg76LK54f WTp+oQETDMgADzs6hBVDcUzOBa0y/MqlwkHGJE2H90GBD6FruOq5vCerpUGUy0U4wMNM oq8MWSBq2fD9+vNtDH2nMc4yLqhW/UlwCLS85MZfQU7YSdkqJnfY8YtkeZ+hQ3ryCfob 5X8w== X-Gm-Message-State: ALyK8tI3D8WhWZ4Z3srervvtNkNrM5U5YZQjVOs6cfmtGsBu/c/Sm0JmSv/sja8Zr2ynfzACAsrGxg2M957S35te MIME-Version: 1.0 X-Received: by 10.13.253.131 with SMTP id n125mr3288935ywf.133.1464805310448; Wed, 01 Jun 2016 11:21:50 -0700 (PDT) Received: by 10.37.2.6 with HTTP; Wed, 1 Jun 2016 11:21:50 -0700 (PDT) Date: Wed, 1 Jun 2016 11:21:50 -0700 Message-ID: From: Chandrasekar Kannan To: dev@dpdk.org Content-Type: text/plain; charset=UTF-8 X-Content-Filtered-By: Mailman/MimeDel 2.1.15 Subject: [dpdk-dev] SR-IOV/DPDK/VPP with vfio-pci X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: patches and discussions about DPDK List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 01 Jun 2016 18:21:51 -0000 I have been attempting to VPP (dpdk) application with SR-IOV enabled. Using the vfio-pci driver/i40e/XL710nics/. I'm encountering a SEGV on the i40e code path. Has anyone else seen this behaviour ?. full mail thread discussion with vpp-dev is here - https://lists.fd.io/pipermail/vpp-dev/2016-June/001348.html -------------------------------- here's a backtrace during vpp crash... Program received signal SIGSEGV, Segmentation fault. [Switching to Thread 0x7f2b811fd700 (LWP 8725)] 0x0000000000455539 in rx_recv_pkts () (gdb) backtrace #0 0x0000000000455539 in rx_recv_pkts () #1 0x0000000000455e76 in i40e_recv_pkts_bulk_alloc () #2 0x00007f2be43d8bf7 in rte_eth_rx_burst (nb_pkts=256, rx_pkts=0x7f2bb1e66a80, queue_id=1, port_id=) at /w/workspace/vpp-merge-master-centos7/build-root/install-vpp-native/dpdk/include/rte_ethdev.h:2641 #3 dpdk_rx_burst (dm=0x7f2be4673980 , queue_id=1, xd=0x7f2bb1898540) at /w/workspace/vpp-merge-master-centos7/build-data/../vnet/vnet/devices/dpdk/dpdk_priv.h:64 #4 dpdk_device_input (dm=0x7f2be4673980 , queue_id=1, cpu_index=, node=0x7f2bb1ef59fc, xd=) at /w/workspace/vpp-merge-master-centos7/build-data/../vnet/vnet/devices/dpdk/node.c:508 #5 dpdk_input_rss (vm=, node=0x7f2bb1ef59fc, f=) at /w/workspace/vpp-merge-master-centos7/build-data/../vnet/vnet/devices/dpdk/node.c:821 #6 0x00007f2be47eb77a in dispatch_node (vm=vm@entry=0x7f2bb1f02cd4, node=node@entry=0x7f2bb1ef59fc, type=type@entry=VLIB_NODE_TYPE_INPUT, dispatch_state=dispatch_state@entry=VLIB_NODE_STATE_POLLING, frame=frame@entry=0x0, last_time_stamp=1482097661494035) at /w/workspace/vpp-merge-master-centos7/build-data/../vlib/vlib/main.c:996 #7 0x00007f2be43db6d8 in dpdk_worker_thread_internal (have_io_threads=0, callback=0x0, vm=0x7f2bb1f02cd4) at /w/workspace/vpp-merge-master-centos7/build-data/../vnet/vnet/devices/dpdk/threads.c:209 #8 dpdk_worker_thread (w=, io_name=, callback=0x0) at /w/workspace/vpp-merge-master-centos7/build-data/../vnet/vnet/devices/dpdk/threads.c:265 #9 0x00007f2be3791530 in clib_calljmp () at /w/workspace/vpp-merge-master-centos7/build-data/../vppinfra/vppinfra/longjmp.S:110 #10 0x00007f2b811fcc40 in ?? () #11 0x00000000004eb9c7 in eal_thread_loop () ---Type to continue, or q to quit--- #12 0x0000000000000000 in ?? () (gdb) (gdb) thread apply all bt Thread 10 (Thread 0x7f2b8c685700 (LWP 8722)): #0 0x00007f2be32584ad in accept () at ../sysdeps/unix/syscall-template.S:81 #1 0x00000000004ef178 in pci_vfio_mp_sync_thread () #2 0x00007f2be3251dc5 in start_thread (arg=0x7f2b8c685700) at pthread_create.c:308 #3 0x00007f2be2992ced in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:113 Thread 9 (Thread 0x7f2b821ff700 (LWP 8723)): #0 0x00007f2be29932c3 in epoll_wait () at ../sysdeps/unix/syscall-template.S:81 #1 0x00000000004efe04 in eal_intr_thread_main () #2 0x00007f2be3251dc5 in start_thread (arg=0x7f2b821ff700) at pthread_create.c:308 #3 0x00007f2be2992ced in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:113 Thread 8 (Thread 0x7f2b819fe700 (LWP 8724)): #0 0x000000000045526b in rx_recv_pkts () #1 0x0000000000455e76 in i40e_recv_pkts_bulk_alloc () #2 0x00007f2be43d8bf7 in rte_eth_rx_burst (nb_pkts=256, rx_pkts=0x7f2bb1e75580, queue_id=0, port_id=) at /w/workspace/vpp-merge-master-centos7/build-root/install-vpp-native/dpdk/include/rte_ethdev.h:2641 #3 dpdk_rx_burst (dm=0x7f2be4673980 , queue_id=0, xd=0x7f2bb1a1fd40) at /w/workspace/vpp-merge-master-centos7/build-data/../vnet/vnet/devices/dpdk/dpdk_priv.h:64 ---Type to continue, or q to quit--- #4 dpdk_device_input (dm=0x7f2be4673980 , queue_id=0, cpu_index=, node=0x7f2bb1ea18cc, xd=) at /w/workspace/vpp-merge-master-centos7/build-data/../vnet/vnet/devices/dpdk/node.c:508 #5 dpdk_input_rss (vm=, node=0x7f2bb1ea18cc, f=) at /w/workspace/vpp-merge-master-centos7/build-data/../vnet/vnet/devices/dpdk/node.c:821 #6 0x00007f2be47eb77a in dispatch_node (vm=vm@entry=0x7f2bb1f237d4, node=node@entry=0x7f2bb1ea18cc, type=type@entry=VLIB_NODE_TYPE_INPUT, dispatch_state=dispatch_state@entry=VLIB_NODE_STATE_POLLING, frame=frame@entry=0x0, last_time_stamp=1482097666654800) at /w/workspace/vpp-merge-master-centos7/build-data/../vlib/vlib/main.c:996 #7 0x00007f2be43db6d8 in dpdk_worker_thread_internal (have_io_threads=0, callback=0x0, vm=0x7f2bb1f237d4) at /w/workspace/vpp-merge-master-centos7/build-data/../vnet/vnet/devices/dpdk/threads.c:209 #8 dpdk_worker_thread (w=, io_name=, callback=0x0) at /w/workspace/vpp-merge-master-centos7/build-data/../vnet/vnet/devices/dpdk/threads.c:265 #9 0x00007f2be3791530 in clib_calljmp () at /w/workspace/vpp-merge-master-centos7/build-data/../vppinfra/vppinfra/longjmp.S:110 #10 0x00007f2b819fdc40 in ?? () #11 0x00000000004eb9c7 in eal_thread_loop () #12 0x0000000000000000 in ?? () Thread 7 (Thread 0x7f2b811fd700 (LWP 8725)): #0 0x0000000000455539 in rx_recv_pkts () #1 0x0000000000455e76 in i40e_recv_pkts_bulk_alloc () #2 0x00007f2be43d8bf7 in rte_eth_rx_burst (nb_pkts=256, rx_pkts=0x7f2bb1e66a80, queue_id=1, port_id=) ---Type to continue, or q to quit--- at /w/workspace/vpp-merge-master-centos7/build-root/install-vpp-native/dpdk/include/rte_ethdev.h:2641 #3 dpdk_rx_burst (dm=0x7f2be4673980 , queue_id=1, xd=0x7f2bb1898540) at /w/workspace/vpp-merge-master-centos7/build-data/../vnet/vnet/devices/dpdk/dpdk_priv.h:64 #4 dpdk_device_input (dm=0x7f2be4673980 , queue_id=1, cpu_index=, node=0x7f2bb1ef59fc, xd=) at /w/workspace/vpp-merge-master-centos7/build-data/../vnet/vnet/devices/dpdk/node.c:508 #5 dpdk_input_rss (vm=, node=0x7f2bb1ef59fc, f=) at /w/workspace/vpp-merge-master-centos7/build-data/../vnet/vnet/devices/dpdk/node.c:821 #6 0x00007f2be47eb77a in dispatch_node (vm=vm@entry=0x7f2bb1f02cd4, node=node@entry=0x7f2bb1ef59fc, type=type@entry=VLIB_NODE_TYPE_INPUT, dispatch_state=dispatch_state@entry=VLIB_NODE_STATE_POLLING, frame=frame@entry=0x0, last_time_stamp=1482097661494035) at /w/workspace/vpp-merge-master-centos7/build-data/../vlib/vlib/main.c:996 #7 0x00007f2be43db6d8 in dpdk_worker_thread_internal (have_io_threads=0, callback=0x0, vm=0x7f2bb1f02cd4) at /w/workspace/vpp-merge-master-centos7/build-data/../vnet/vnet/devices/dpdk/threads.c:209 #8 dpdk_worker_thread (w=, io_name=, callback=0x0) at /w/workspace/vpp-merge-master-centos7/build-data/../vnet/vnet/devices/dpdk/threads.c:265 #9 0x00007f2be3791530 in clib_calljmp () at /w/workspace/vpp-merge-master-centos7/build-data/../vppinfra/vppinfra/longjmp.S:110 #10 0x00007f2b811fcc40 in ?? () #11 0x00000000004eb9c7 in eal_thread_loop () #12 0x0000000000000000 in ?? () Thread 6 (Thread 0x7f2b809fc700 (LWP 8726)): ---Type to continue, or q to quit--- #0 0x000000000045539e in rx_recv_pkts () #1 0x0000000000455e76 in i40e_recv_pkts_bulk_alloc () #2 0x00007f2be43d8bf7 in rte_eth_rx_burst (nb_pkts=256, rx_pkts=0x7f2bb1da8a00, queue_id=2, port_id=) at /w/workspace/vpp-merge-master-centos7/build-root/install-vpp-native/dpdk/include/rte_ethdev.h:2641 #3 dpdk_rx_burst (dm=0x7f2be4673980 , queue_id=2, xd=0x7f2bb1898540) at /w/workspace/vpp-merge-master-centos7/build-data/../vnet/vnet/devices/dpdk/dpdk_priv.h:64 #4 dpdk_device_input (dm=0x7f2be4673980 , queue_id=2, cpu_index=, node=0x7f2bb1ebc7f8, xd=) at /w/workspace/vpp-merge-master-centos7/build-data/../vnet/vnet/devices/dpdk/node.c:508 #5 dpdk_input_rss (vm=, node=0x7f2bb1ebc7f8, f=) at /w/workspace/vpp-merge-master-centos7/build-data/../vnet/vnet/devices/dpdk/node.c:821 #6 0x00007f2be47eb77a in dispatch_node (vm=vm@entry=0x7f2bb1ee29d4, node=node@entry=0x7f2bb1ebc7f8, type=type@entry=VLIB_NODE_TYPE_INPUT, dispatch_state=dispatch_state@entry=VLIB_NODE_STATE_POLLING, frame=frame@entry=0x0, last_time_stamp=1482097658614609) at /w/workspace/vpp-merge-master-centos7/build-data/../vlib/vlib/main.c:996 #7 0x00007f2be43db6d8 in dpdk_worker_thread_internal (have_io_threads=0, callback=0x0, vm=0x7f2bb1ee29d4) at /w/workspace/vpp-merge-master-centos7/build-data/../vnet/vnet/devices/dpdk/threads.c:209 #8 dpdk_worker_thread (w=, io_name=, callback=0x0) at /w/workspace/vpp-merge-master-centos7/build-data/../vnet/vnet/devices/dpdk/threads.c:265 #9 0x00007f2be3791530 in clib_calljmp () at /w/workspace/vpp-merge-master-centos7/build-data/../vppinfra/vppinfra/longjmp.S:110 #10 0x00007f2b809fbc40 in ?? () #11 0x00000000004eb9c7 in eal_thread_loop () ---Type to continue, or q to quit--- #12 0x0000000000000000 in ?? () Thread 5 (Thread 0x7f2b801fb700 (LWP 8727)): #0 0x00007f2be47eb728 in dispatch_node (vm=vm@entry=0x7f2bb1ec2614, node=node@entry=0x7f2bb1da5e60, type=type@entry=VLIB_NODE_TYPE_INPUT, dispatch_state=dispatch_state@entry=VLIB_NODE_STATE_POLLING, frame=frame@entry=0x0, last_time_stamp=1482097666665442) at /w/workspace/vpp-merge-master-centos7/build-data/../vlib/vlib/main.c:924 #1 0x00007f2be43db6d8 in dpdk_worker_thread_internal (have_io_threads=0, callback=0x0, vm=0x7f2bb1ec2614) at /w/workspace/vpp-merge-master-centos7/build-data/../vnet/vnet/devices/dpdk/threads.c:209 #2 dpdk_worker_thread (w=, io_name=, callback=0x0) at /w/workspace/vpp-merge-master-centos7/build-data/../vnet/vnet/devices/dpdk/threads.c:265 #3 0x00007f2be3791530 in clib_calljmp () at /w/workspace/vpp-merge-master-centos7/build-data/../vppinfra/vppinfra/longjmp.S:110 #4 0x00007f2b801fac40 in ?? () #5 0x00000000004eb9c7 in eal_thread_loop () #6 0x0000000000000000 in ?? () Thread 4 (Thread 0x7f2b7f9fa700 (LWP 8728)): #0 0x00007f2be43db6cc in dpdk_worker_thread_internal (have_io_threads=0, callback=0x0, vm=0x7f2bb1ea2294) at /w/workspace/vpp-merge-master-centos7/build-data/../vnet/vnet/devices/dpdk/threads.c:209 #1 dpdk_worker_thread (w=, io_name=, callback=0x0) at /w/workspace/vpp-merge-master-centos7/build-data/../vnet/vnet/devices/dpdk/threads.c:265 ---Type to continue, or q to quit--- #2 0x00007f2be3791530 in clib_calljmp () at /w/workspace/vpp-merge-master-centos7/build-data/../vppinfra/vppinfra/longjmp.S:110 #3 0x00007f2b7f9f9c40 in ?? () #4 0x00000000004eb9c7 in eal_thread_loop () #5 0x0000000000000000 in ?? () Thread 3 (Thread 0x7f2b7f1f9700 (LWP 8729)): #0 vlib_node_runtime_update_stats (n_clocks=205, n_vectors=0, n_calls=1, node=0x7f2bb1d782d0, vm=0x7f2bb1e81f94) at /w/workspace/vpp-merge-master-centos7/build-data/../vlib/vlib/main.c:629 #1 dispatch_node (vm=vm@entry=0x7f2bb1e81f94, node=node@entry=0x7f2bb1d782d0, type=type@entry=VLIB_NODE_TYPE_INPUT, dispatch_state=dispatch_state@entry=VLIB_NODE_STATE_POLLING, frame=frame@entry=0x0, last_time_stamp=1482097666680456) at /w/workspace/vpp-merge-master-centos7/build-data/../vlib/vlib/main.c:1005 #2 0x00007f2be43db6d8 in dpdk_worker_thread_internal (have_io_threads=0, callback=0x0, vm=0x7f2bb1e81f94) at /w/workspace/vpp-merge-master-centos7/build-data/../vnet/vnet/devices/dpdk/threads.c:209 #3 dpdk_worker_thread (w=, io_name=, callback=0x0) at /w/workspace/vpp-merge-master-centos7/build-data/../vnet/vnet/devices/dpdk/threads.c:265 #4 0x00007f2be3791530 in clib_calljmp () at /w/workspace/vpp-merge-master-centos7/build-data/../vppinfra/vppinfra/longjmp.S:110 #5 0x00007f2b7f1f8c40 in ?? () #6 0x00000000004eb9c7 in eal_thread_loop () #7 0x0000000000000000 in ?? () ---Type to continue, or q to quit--- Thread 2 (Thread 0x7f2b7e9f8700 (LWP 8730)): #0 0x00007f2be325896d in nanosleep () at ../sysdeps/unix/syscall-template.S:81 #1 0x0000000000413916 in ip46_fib_stats_delay.isra.6 () #2 0x00000000004190b6 in stats_thread_fn () #3 0x00007f2be3791530 in clib_calljmp () at /w/workspace/vpp-merge-master-centos7/build-data/../vppinfra/vppinfra/longjmp.S:110 #4 0x00007f2b7e9f7da0 in ?? () #5 0x00007f2be3251dc5 in start_thread (arg=0x0) at pthread_create.c:308 #6 0x0000000000000000 in ?? () Thread 1 (Thread 0x7f2be52898c0 (LWP 8721)): #0 0x00007f2be2992f27 in __GI_epoll_pwait (epfd=5, events=0x7f2bb2155424, maxevents=256, timeout=timeout@entry=10, set=set@entry=0x7f2be4c522e0 ) at ../sysdeps/unix/sysv/linux/epoll_pwait.c:48 #1 0x00007f2be4a429fb in linux_epoll_input (vm=0x7e13c0 , node=, frame=) at /w/workspace/vpp-merge-master-centos7/build-data/../vlib/vlib/unix/input.c:146 #2 0x00007f2be47eb77a in dispatch_node (vm=0x7e13c0 , node=0x7f2bb2254cc0, type=, dispatch_state=, frame=, last_time_stamp=1482097636972836) at /w/workspace/vpp-merge-master-centos7/build-data/../vlib/vlib/main.c:996 #3 0x00007f2be47ebea1 in vlib_main_loop (vm=0x7e13c0 ) at /w/workspace/vpp-merge-master-centos7/build-data/../vlib/vlib/main.c:1403 #4 vlib_main (vm=vm@entry=0x7e13c0 , input=input@entry =0x7f2bb2786fa0) at /w/workspace/vpp-merge-master-centos7/build-data/../vlib/vlib/main.c:1615 #5 0x00007f2be4a435f3 in thread0 (arg=8262592) at /w/workspace/vpp-merge-master-centos7/build-data/../vlib/vlib/unix/main.c:425 ---Type to continue, or q to quit--- #6 0x00007f2be3791530 in clib_calljmp () at /w/workspace/vpp-merge-master-centos7/build-data/../vppinfra/vppinfra/longjmp.S:110 #7 0x00007ffc98a68d60 in ?? () #8 0x00007f2be4a43e31 in vlib_unix_main (argc=, argv=) at /w/workspace/vpp-merge-master-centos7/build-data/../vlib/vlib/unix/main.c:485 #9 0x0000000000000000 in ?? () (gdb) (gdb) (gdb) (gdb) -------------------------------------------------------------------