* [dpdk-users] seg fault in InterVM communication using virtio @ 2017-02-20 10:30 Priyanka 2017-02-22 8:43 ` [dpdk-users] [dpdk-dev] " Yuanhan Liu 0 siblings, 1 reply; 10+ messages in thread From: Priyanka @ 2017-02-20 10:30 UTC (permalink / raw) To: users, dev; +Cc: mitaliyada Hi All, We have 2 VM running on qemu-kvm hypervisor. We have provided vNIC to the VMs using macvtap device and virtio as the device model in bridge mode. We are running DPDK applications (udp client and server) on the VMs. We observe that the inter-VM communication is not happening using drivers i.e igb_uio and uio_pci_generic. We were able to communicate between the VMs properly using SRIOV. But we were not getting performance as expected so we switched to virtio. But using virtio the DPDK communication hangs after sending 128 packets. We also observe a seg fault in the dmesg of the VM. Please provide us some guideline to do the inter-VM communication using virtio. Are we correct if we are using rte_eth_tx/rx_burst to send and receive packets from the vNIC. Thanks, Priyanka ^ permalink raw reply [flat|nested] 10+ messages in thread
* Re: [dpdk-users] [dpdk-dev] seg fault in InterVM communication using virtio 2017-02-20 10:30 [dpdk-users] seg fault in InterVM communication using virtio Priyanka @ 2017-02-22 8:43 ` Yuanhan Liu 2017-02-22 8:55 ` mitali 0 siblings, 1 reply; 10+ messages in thread From: Yuanhan Liu @ 2017-02-22 8:43 UTC (permalink / raw) To: Priyanka; +Cc: users, dev, mitaliyada On Mon, Feb 20, 2017 at 04:00:12PM +0530, Priyanka wrote: > Hi All, > > We have 2 VM running on qemu-kvm hypervisor. We have provided vNIC to the > VMs using macvtap device and virtio as the device model in bridge mode. We > are running DPDK applications (udp client and server) on the VMs. We observe > that the inter-VM communication is not happening using drivers i.e igb_uio > and uio_pci_generic. We were able to communicate between the VMs properly > using SRIOV. But we were not getting performance as expected so we switched > to virtio. But using virtio the DPDK communication hangs after sending 128 > packets. We also observe a seg fault in the dmesg of the VM. > > Please provide us some guideline to do the inter-VM communication using > virtio. Are we correct if we are using rte_eth_tx/rx_burst to send and > receive packets from the vNIC. Would you provide more info, say how do you start the vm, and how do you run the dpdk apps? Providing all the commands that could reproduce your issue will be better. --yliu ^ permalink raw reply [flat|nested] 10+ messages in thread
* Re: [dpdk-users] [dpdk-dev] seg fault in InterVM communication using virtio 2017-02-22 8:43 ` [dpdk-users] [dpdk-dev] " Yuanhan Liu @ 2017-02-22 8:55 ` mitali 2017-02-23 6:27 ` Yuanhan Liu 0 siblings, 1 reply; 10+ messages in thread From: mitali @ 2017-02-22 8:55 UTC (permalink / raw) To: Yuanhan Liu, Priyanka; +Cc: users, dev Hi Yuanhan, We are using virt-manager to start the VMs. We are attaching the image which shows the vNIC setup for the VMs. We have written our own dpdk application to send and receive UDP packets. The command we are using is *./build/app/dpdkapi -c 1 -- -p 1* where dpdkapi is our application. Inside the application we are sending and receiving packets from vNIC using rte_eth_tx_burst() and rte_eth_rx_burst(). At the sender side after 127 packets the application stops and we observe a seg fault in dmesg. Using the print statements we observed that it faults in rte_eth_tx_burst(). All the 127 were received at the receiving VM. Thanks, Mitali On Wednesday 22 February 2017 02:13 PM, Yuanhan Liu wrote: > On Mon, Feb 20, 2017 at 04:00:12PM +0530, Priyanka wrote: >> Hi All, >> >> We have 2 VM running on qemu-kvm hypervisor. We have provided vNIC to the >> VMs using macvtap device and virtio as the device model in bridge mode. We >> are running DPDK applications (udp client and server) on the VMs. We observe >> that the inter-VM communication is not happening using drivers i.e igb_uio >> and uio_pci_generic. We were able to communicate between the VMs properly >> using SRIOV. But we were not getting performance as expected so we switched >> to virtio. But using virtio the DPDK communication hangs after sending 128 >> packets. We also observe a seg fault in the dmesg of the VM. >> >> Please provide us some guideline to do the inter-VM communication using >> virtio. Are we correct if we are using rte_eth_tx/rx_burst to send and >> receive packets from the vNIC. > Would you provide more info, say how do you start the vm, and how do you > run the dpdk apps? Providing all the commands that could reproduce your > issue will be better. > > --yliu -------------- next part -------------- A non-text attachment was scrubbed... Name: VM_virtio.png Type: image/png Size: 149226 bytes Desc: not available URL: <http://dpdk.org/ml/archives/users/attachments/20170222/2dbaeb06/attachment.png> ^ permalink raw reply [flat|nested] 10+ messages in thread
* Re: [dpdk-users] [dpdk-dev] seg fault in InterVM communication using virtio 2017-02-22 8:55 ` mitali @ 2017-02-23 6:27 ` Yuanhan Liu 2017-03-01 13:50 ` mitali 2017-03-02 11:13 ` ppnaik 0 siblings, 2 replies; 10+ messages in thread From: Yuanhan Liu @ 2017-02-23 6:27 UTC (permalink / raw) To: mitali; +Cc: Priyanka, users, dev On Wed, Feb 22, 2017 at 02:25:29PM +0530, mitali wrote: > Hi Yuanhan, > > We are using virt-manager to start the VMs. We are attaching the image which > shows the vNIC setup for the VMs. > > We have written our own dpdk application to send and receive UDP packets. The > command we are using is > > ./build/app/dpdkapi -c 1 -- -p 1 where dpdkapi is our application. > > Inside the application we are sending and receiving packets from vNIC using > rte_eth_tx_burst() and rte_eth_rx_burst(). > > At the sender side after 127 packets the application stops and we observe a seg > fault in dmesg. Using the print statements we observed that it faults in > rte_eth_tx_burst(). All the 127 were received at the receiving VM. Would you recompile dpdk with debug info enaled, which could be done by: make ... EXTRA_CFLAGS='-g3 -O0' And run it with gdb then dump the call stack? --yliu ^ permalink raw reply [flat|nested] 10+ messages in thread
* Re: [dpdk-users] [dpdk-dev] seg fault in InterVM communication using virtio 2017-02-23 6:27 ` Yuanhan Liu @ 2017-03-01 13:50 ` mitali 2017-03-02 12:23 ` Yuanhan Liu 2017-03-02 11:13 ` ppnaik 1 sibling, 1 reply; 10+ messages in thread From: mitali @ 2017-03-01 13:50 UTC (permalink / raw) To: Yuanhan Liu; +Cc: Priyanka, users, dev Hi, We have some doubt regarding virtio front end. As we are running dpdk application inside VM with virtio setup, do we need to use virtio specific functions like virtio_recv_pkts() and virtio_xmit_pkts() to receive and transmit packets, functions for port initialization like virtio_dev_queue_setup()? If yes, we are unable to find any application which uses these functions. Can you provide us some pointers on examples which uses these functions. Thanks, Mitali On Thursday 23 February 2017 11:57 AM, Yuanhan Liu wrote: > On Wed, Feb 22, 2017 at 02:25:29PM +0530, mitali wrote: >> Hi Yuanhan, >> >> We are using virt-manager to start the VMs. We are attaching the image which >> shows the vNIC setup for the VMs. >> >> We have written our own dpdk application to send and receive UDP packets. The >> command we are using is >> >> ./build/app/dpdkapi -c 1 -- -p 1 where dpdkapi is our application. >> >> Inside the application we are sending and receiving packets from vNIC using >> rte_eth_tx_burst() and rte_eth_rx_burst(). >> >> At the sender side after 127 packets the application stops and we observe a seg >> fault in dmesg. Using the print statements we observed that it faults in >> rte_eth_tx_burst(). All the 127 were received at the receiving VM. > Would you recompile dpdk with debug info enaled, which could be done by: > > make ... EXTRA_CFLAGS='-g3 -O0' > > And run it with gdb then dump the call stack? > > --yliu ^ permalink raw reply [flat|nested] 10+ messages in thread
* Re: [dpdk-users] [dpdk-dev] seg fault in InterVM communication using virtio 2017-03-01 13:50 ` mitali @ 2017-03-02 12:23 ` Yuanhan Liu 2017-03-02 12:32 ` mitali 0 siblings, 1 reply; 10+ messages in thread From: Yuanhan Liu @ 2017-03-02 12:23 UTC (permalink / raw) To: mitali; +Cc: Priyanka, users, dev On Wed, Mar 01, 2017 at 07:20:37PM +0530, mitali wrote: > Hi, > > We have some doubt regarding virtio front end. As we are running dpdk > application inside VM with virtio setup, do we need to use virtio specific > functions like virtio_recv_pkts() and virtio_xmit_pkts() to receive and > transmit packets, functions for port initialization like > virtio_dev_queue_setup()? If yes, we are unable to find any application > which uses these functions. Can you provide us some pointers on examples > which uses these functions. If you are running DPDK inside guest (with virtio), then the virtio is yet another PMD, meaning you should program it with the normal DPDK ethdev APIs, something like rte_eth_dev_configure, rte_eth_rx/tx_queue_setup, etc. --yliu > > Thanks, > Mitali > > On Thursday 23 February 2017 11:57 AM, Yuanhan Liu wrote: > >On Wed, Feb 22, 2017 at 02:25:29PM +0530, mitali wrote: > >>Hi Yuanhan, > >> > >>We are using virt-manager to start the VMs. We are attaching the image which > >>shows the vNIC setup for the VMs. > >> > >>We have written our own dpdk application to send and receive UDP packets. The > >>command we are using is > >> > >>./build/app/dpdkapi -c 1 -- -p 1 where dpdkapi is our application. > >> > >>Inside the application we are sending and receiving packets from vNIC using > >>rte_eth_tx_burst() and rte_eth_rx_burst(). > >> > >>At the sender side after 127 packets the application stops and we observe a seg > >>fault in dmesg. Using the print statements we observed that it faults in > >>rte_eth_tx_burst(). All the 127 were received at the receiving VM. > >Would you recompile dpdk with debug info enaled, which could be done by: > > > > make ... EXTRA_CFLAGS='-g3 -O0' > > > >And run it with gdb then dump the call stack? > > > > --yliu ^ permalink raw reply [flat|nested] 10+ messages in thread
* Re: [dpdk-users] [dpdk-dev] seg fault in InterVM communication using virtio 2017-03-02 12:23 ` Yuanhan Liu @ 2017-03-02 12:32 ` mitali 2017-03-02 12:59 ` Yuanhan Liu 0 siblings, 1 reply; 10+ messages in thread From: mitali @ 2017-03-02 12:32 UTC (permalink / raw) To: Yuanhan Liu; +Cc: Priyanka, users, dev Hi, Ok. But with other PMDs like igb, when we are providing sriov vf to VM and running our client/server dpdk application then there is no issue in communication between VMs. As soon as we use virtio setup for VMs and running same dpdk application, it stops after sending 127 packet and seg fault comes while executing rte_eth_tx_burst(). We are still not clear why with virtio its creating this issue but not with any other PMD. Mitali On Thursday 02 March 2017 05:53 PM, Yuanhan Liu wrote: > On Wed, Mar 01, 2017 at 07:20:37PM +0530, mitali wrote: >> Hi, >> >> We have some doubt regarding virtio front end. As we are running dpdk >> application inside VM with virtio setup, do we need to use virtio specific >> functions like virtio_recv_pkts() and virtio_xmit_pkts() to receive and >> transmit packets, functions for port initialization like >> virtio_dev_queue_setup()? If yes, we are unable to find any application >> which uses these functions. Can you provide us some pointers on examples >> which uses these functions. > If you are running DPDK inside guest (with virtio), then the virtio is > yet another PMD, meaning you should program it with the normal DPDK > ethdev APIs, something like rte_eth_dev_configure, rte_eth_rx/tx_queue_setup, > etc. > > --yliu >> Thanks, >> Mitali >> >> On Thursday 23 February 2017 11:57 AM, Yuanhan Liu wrote: >>> On Wed, Feb 22, 2017 at 02:25:29PM +0530, mitali wrote: >>>> Hi Yuanhan, >>>> >>>> We are using virt-manager to start the VMs. We are attaching the image which >>>> shows the vNIC setup for the VMs. >>>> >>>> We have written our own dpdk application to send and receive UDP packets. The >>>> command we are using is >>>> >>>> ./build/app/dpdkapi -c 1 -- -p 1 where dpdkapi is our application. >>>> >>>> Inside the application we are sending and receiving packets from vNIC using >>>> rte_eth_tx_burst() and rte_eth_rx_burst(). >>>> >>>> At the sender side after 127 packets the application stops and we observe a seg >>>> fault in dmesg. Using the print statements we observed that it faults in >>>> rte_eth_tx_burst(). All the 127 were received at the receiving VM. >>> Would you recompile dpdk with debug info enaled, which could be done by: >>> >>> make ... EXTRA_CFLAGS='-g3 -O0' >>> >>> And run it with gdb then dump the call stack? >>> >>> --yliu ^ permalink raw reply [flat|nested] 10+ messages in thread
* Re: [dpdk-users] [dpdk-dev] seg fault in InterVM communication using virtio 2017-03-02 12:32 ` mitali @ 2017-03-02 12:59 ` Yuanhan Liu 2017-03-02 14:00 ` mitali 0 siblings, 1 reply; 10+ messages in thread From: Yuanhan Liu @ 2017-03-02 12:59 UTC (permalink / raw) To: mitali; +Cc: Priyanka, users, dev On Thu, Mar 02, 2017 at 06:02:28PM +0530, mitali wrote: > Hi, > > Ok. But with other PMDs like igb, when we are providing sriov vf to VM and > running our client/server dpdk application then there is no issue in > communication between VMs. > As soon as we use virtio setup for VMs and running same dpdk application, it > stops after sending 127 packet and seg fault comes while executing > rte_eth_tx_burst(). We are still not clear why with virtio its creating this > issue but not with any other PMD. No idea. You don't provide any further details. Please, would you just enable the debug info, rebuild, run it with gdb and then dump the call stack after crash? --yliu > > Mitali > > On Thursday 02 March 2017 05:53 PM, Yuanhan Liu wrote: > >On Wed, Mar 01, 2017 at 07:20:37PM +0530, mitali wrote: > >>Hi, > >> > >>We have some doubt regarding virtio front end. As we are running dpdk > >>application inside VM with virtio setup, do we need to use virtio specific > >>functions like virtio_recv_pkts() and virtio_xmit_pkts() to receive and > >>transmit packets, functions for port initialization like > >>virtio_dev_queue_setup()? If yes, we are unable to find any application > >>which uses these functions. Can you provide us some pointers on examples > >>which uses these functions. > >If you are running DPDK inside guest (with virtio), then the virtio is > >yet another PMD, meaning you should program it with the normal DPDK > >ethdev APIs, something like rte_eth_dev_configure, rte_eth_rx/tx_queue_setup, > >etc. > > > > --yliu > >>Thanks, > >>Mitali > >> > >>On Thursday 23 February 2017 11:57 AM, Yuanhan Liu wrote: > >>>On Wed, Feb 22, 2017 at 02:25:29PM +0530, mitali wrote: > >>>>Hi Yuanhan, > >>>> > >>>>We are using virt-manager to start the VMs. We are attaching the image which > >>>>shows the vNIC setup for the VMs. > >>>> > >>>>We have written our own dpdk application to send and receive UDP packets. The > >>>>command we are using is > >>>> > >>>>./build/app/dpdkapi -c 1 -- -p 1 where dpdkapi is our application. > >>>> > >>>>Inside the application we are sending and receiving packets from vNIC using > >>>>rte_eth_tx_burst() and rte_eth_rx_burst(). > >>>> > >>>>At the sender side after 127 packets the application stops and we observe a seg > >>>>fault in dmesg. Using the print statements we observed that it faults in > >>>>rte_eth_tx_burst(). All the 127 were received at the receiving VM. > >>>Would you recompile dpdk with debug info enaled, which could be done by: > >>> > >>> make ... EXTRA_CFLAGS='-g3 -O0' > >>> > >>>And run it with gdb then dump the call stack? > >>> > >>> --yliu ^ permalink raw reply [flat|nested] 10+ messages in thread
* Re: [dpdk-users] [dpdk-dev] seg fault in InterVM communication using virtio 2017-03-02 12:59 ` Yuanhan Liu @ 2017-03-02 14:00 ` mitali 0 siblings, 0 replies; 10+ messages in thread From: mitali @ 2017-03-02 14:00 UTC (permalink / raw) To: Yuanhan Liu; +Cc: Priyanka, users, dev Hi, I rebuild dpdk with debug enabled and running the same dpdk application. Please go through the attached images showing the error as well as call stack. Mitali On Thursday 02 March 2017 06:29 PM, Yuanhan Liu wrote: > On Thu, Mar 02, 2017 at 06:02:28PM +0530, mitali wrote: >> Hi, >> >> Ok. But with other PMDs like igb, when we are providing sriov vf to VM and >> running our client/server dpdk application then there is no issue in >> communication between VMs. >> As soon as we use virtio setup for VMs and running same dpdk application, it >> stops after sending 127 packet and seg fault comes while executing >> rte_eth_tx_burst(). We are still not clear why with virtio its creating this >> issue but not with any other PMD. > No idea. You don't provide any further details. Please, would you just > enable the debug info, rebuild, run it with gdb and then dump the call > stack after crash? > > --yliu >> Mitali >> >> On Thursday 02 March 2017 05:53 PM, Yuanhan Liu wrote: >>> On Wed, Mar 01, 2017 at 07:20:37PM +0530, mitali wrote: >>>> Hi, >>>> >>>> We have some doubt regarding virtio front end. As we are running dpdk >>>> application inside VM with virtio setup, do we need to use virtio specific >>>> functions like virtio_recv_pkts() and virtio_xmit_pkts() to receive and >>>> transmit packets, functions for port initialization like >>>> virtio_dev_queue_setup()? If yes, we are unable to find any application >>>> which uses these functions. Can you provide us some pointers on examples >>>> which uses these functions. >>> If you are running DPDK inside guest (with virtio), then the virtio is >>> yet another PMD, meaning you should program it with the normal DPDK >>> ethdev APIs, something like rte_eth_dev_configure, rte_eth_rx/tx_queue_setup, >>> etc. >>> >>> --yliu >>>> Thanks, >>>> Mitali >>>> >>>> On Thursday 23 February 2017 11:57 AM, Yuanhan Liu wrote: >>>>> On Wed, Feb 22, 2017 at 02:25:29PM +0530, mitali wrote: >>>>>> Hi Yuanhan, >>>>>> >>>>>> We are using virt-manager to start the VMs. We are attaching the image which >>>>>> shows the vNIC setup for the VMs. >>>>>> >>>>>> We have written our own dpdk application to send and receive UDP packets. The >>>>>> command we are using is >>>>>> >>>>>> ./build/app/dpdkapi -c 1 -- -p 1 where dpdkapi is our application. >>>>>> >>>>>> Inside the application we are sending and receiving packets from vNIC using >>>>>> rte_eth_tx_burst() and rte_eth_rx_burst(). >>>>>> >>>>>> At the sender side after 127 packets the application stops and we observe a seg >>>>>> fault in dmesg. Using the print statements we observed that it faults in >>>>>> rte_eth_tx_burst(). All the 127 were received at the receiving VM. >>>>> Would you recompile dpdk with debug info enaled, which could be done by: >>>>> >>>>> make ... EXTRA_CFLAGS='-g3 -O0' >>>>> >>>>> And run it with gdb then dump the call stack? >>>>> >>>>> --yliu -------------- next part -------------- A non-text attachment was scrubbed... Name: gdb_error_call_stack.png Type: image/png Size: 213966 bytes Desc: not available URL: <http://dpdk.org/ml/archives/users/attachments/20170302/20fd6b7a/attachment.png> -------------- next part -------------- A non-text attachment was scrubbed... Name: vm_gdb_error.png Type: image/png Size: 304846 bytes Desc: not available URL: <http://dpdk.org/ml/archives/users/attachments/20170302/20fd6b7a/attachment-0001.png> ^ permalink raw reply [flat|nested] 10+ messages in thread
* Re: [dpdk-users] [dpdk-dev] seg fault in InterVM communication using virtio 2017-02-23 6:27 ` Yuanhan Liu 2017-03-01 13:50 ` mitali @ 2017-03-02 11:13 ` ppnaik 1 sibling, 0 replies; 10+ messages in thread From: ppnaik @ 2017-03-02 11:13 UTC (permalink / raw) To: Yuanhan Liu; +Cc: mitali, users, dev Hi, We have some doubt regarding virtio front end. As we are running dpdk application inside VM with virtio setup, do we need to use virtio specific functions like virtio_recv_pkts() and virtio_xmit_pkts() to receive and transmit packets, functions for port initialization like virtio_dev_queue_setup()? If yes, we are unable to find any application which uses these functions. Can you provide us some pointers on examples which uses these functions. Thanks, Priyanka On 2017-02-23 11:57, Yuanhan Liu wrote: > On Wed, Feb 22, 2017 at 02:25:29PM +0530, mitali wrote: >> Hi Yuanhan, >> >> We are using virt-manager to start the VMs. We are attaching the >> image which >> shows the vNIC setup for the VMs. >> >> We have written our own dpdk application to send and receive UDP >> packets. The >> command we are using is >> >> ./build/app/dpdkapi -c 1 -- -p 1 where dpdkapi is our application. >> >> Inside the application we are sending and receiving packets from >> vNIC using >> rte_eth_tx_burst() and rte_eth_rx_burst(). >> >> At the sender side after 127 packets the application stops and we >> observe a seg >> fault in dmesg. Using the print statements we observed that it >> faults in >> rte_eth_tx_burst(). All the 127 were received at the receiving VM. > > Would you recompile dpdk with debug info enaled, which could be done > by: > > make ... EXTRA_CFLAGS='-g3 -O0' > > And run it with gdb then dump the call stack? > > --yliu ^ permalink raw reply [flat|nested] 10+ messages in thread
end of thread, other threads:[~2017-03-02 14:02 UTC | newest] Thread overview: 10+ messages (download: mbox.gz / follow: Atom feed) -- links below jump to the message on this page -- 2017-02-20 10:30 [dpdk-users] seg fault in InterVM communication using virtio Priyanka 2017-02-22 8:43 ` [dpdk-users] [dpdk-dev] " Yuanhan Liu 2017-02-22 8:55 ` mitali 2017-02-23 6:27 ` Yuanhan Liu 2017-03-01 13:50 ` mitali 2017-03-02 12:23 ` Yuanhan Liu 2017-03-02 12:32 ` mitali 2017-03-02 12:59 ` Yuanhan Liu 2017-03-02 14:00 ` mitali 2017-03-02 11:13 ` ppnaik
This is a public inbox, see mirroring instructions for how to clone and mirror all data and code used for this inbox; as well as URLs for NNTP newsgroup(s).
