From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mail-wi0-f179.google.com (mail-wi0-f179.google.com [209.85.212.179]) by dpdk.org (Postfix) with ESMTP id 8310A5A30 for ; Fri, 22 May 2015 19:59:08 +0200 (CEST) Received: by wibt6 with SMTP id t6so54961153wib.0 for ; Fri, 22 May 2015 10:59:08 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:mime-version:in-reply-to:references:date :message-id:subject:from:to:cc:content-type; bh=NPv1Z7a3KkgnWcqrfN4A/6p7zmeAuued0NwR91EOwRM=; b=h1eAV6Ztowip2x+WBkU+rmiABJ66MxRYoRST+x6238d8RN8sKzg2CTmURk3jM/mvV0 gqgKi05LK4hKVgitzwRaLd7uN9b5WsC26OLiL6r5X5Amj+bDoVBRBfYRkGVCZ8/uvNEc 0FXhDR8TqO/+C/l9Ar/Bh02mH4ANBfabZLqcx6BPItDIwsowtdfRWUwrBe9F31stWhBO Q9p+raj1p3oIFKJm5NObTORpmjLV6157JqTD1q9M3ZnEbUJ3Oz6ATjIZjcmypVx9RuGl Znui3BkpdUBlA9kQ0MWYsVfFpMcvu8OO6e49M4B5T+WKTkM+P9EKxk5EswIApTUve/hI LcNw== X-Gm-Message-State: ALoCoQlGePMpMmmUMLP6FlLQ1DhvDUuyPWKGbms0CdnLWOFOX2v0Fqp1XPAU9/uMzpc3Zu5cm+1L MIME-Version: 1.0 X-Received: by 10.194.200.194 with SMTP id ju2mr16846878wjc.61.1432317548283; Fri, 22 May 2015 10:59:08 -0700 (PDT) Received: by 10.27.127.86 with HTTP; Fri, 22 May 2015 10:59:08 -0700 (PDT) In-Reply-To: References: <47CEA9C0E570484FBF22EF0D7EBCE5B534AC4253@szxema505-mbs.china.huawei.com> <47CEA9C0E570484FBF22EF0D7EBCE5B534AC4D90@szxema505-mbs.china.huawei.com> <555EFDDC.1010004@igel.co.jp> Date: Fri, 22 May 2015 19:59:08 +0200 Message-ID: From: Maciej Grochowski To: Andriy Berestovskyy Content-Type: text/plain; charset=UTF-8 X-Content-Filtered-By: Mailman/MimeDel 2.1.15 Cc: dev@dpdk.org Subject: Re: [dpdk-dev] Vhost user no connection vm2vm X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: patches and discussions about DPDK List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 22 May 2015 17:59:08 -0000 Thank You Andriy, You are right Below I put tested KVM configuration that put packets into vhost dpdk plane space: export TLBFS_DIR=/mnt/huge export UVH_PREFIX=/home/ubuntu/esi_ee/dpdk/examples/vhost/usvhost export VM1_MAC=00:01:04:00:01:00 kvm -cpu host -smp 2 -enable-kvm \ -drive if=virtio,file=debian_min_2.qcow2,cache=none \ -object memory-backend-file,id=mem,size=756M,mem-path=${TLBFS_DIR},share=on \ -numa node,memdev=mem \ -m 756 -nographic \ -chardev socket,id=charnet0,path=${UVH_PREFIX} \ -netdev type=vhost-user,id=hostnet0,chardev=charnet0 \ -device virtio-net-pci,netdev=hostnet0,mac=${VM1_MAC} Notice that: memory-backend-file must be with right -numa node, "memdev=mem" option So yeah right configuration is a essence :) On Fri, May 22, 2015 at 12:54 PM, Andriy Berestovskyy wrote: > Hi guys, > I guess you just miss the qemu flag to map the memory as shared, i.e.: > -object memory-backend-file,id=mem,size=1024M,mem-path=/mnt/huge,share=on > (the keyword is share=on) > > Here is an example script: > > https://github.com/srajag/contrail-vrouter/blob/dpdk-devel/examples/vms/VROUTER1/80.start-vm.sh > > Regards, > Andriy > > On Fri, May 22, 2015 at 12:04 PM, Maciej Grochowski > wrote: > > I checked this, results below > > > > #before script: > > root@# cat /sys/kernel/mm/hugepages/hugepages-2048kB/free_hugepages > > 494 > > #after 1 qemu script > > root@# cat /sys/kernel/mm/hugepages/hugepages-2048kB/free_hugepages > > 366 > > > > So qemu consume 262144k~262MB that is correct with script > > > > On Fri, May 22, 2015 at 11:58 AM, Tetsuya Mukawa > wrote: > > > >> Hi Maciej, > >> > >> I guess it's nice to make sure guest memory is actually allocated by > >> hugepages. > >> So please check like below. > >> > >> $ cat /sys/kernel/mm/hugepage/xxxxx/free_hugepages > >> $ ./start_qemu.sh > >> $ cat /sys/kernel/mm/hugepage/xxxxx/free_hugepages > >> > >> If qemu guest allocates memory from hugepages, 2nd cat command will > >> indicate it. > >> > >> Thanks, > >> Tetsuya > >> > >> > >> On 2015/05/22 18:28, Maciej Grochowski wrote: > >> > "Do you use some command I suggest before, > >> > In case of you miss the previous mail, just copy it again:" > >> > > >> > -Yes but it didn't help me ;/ > >> > > >> > I will describe step by step to esure that configuration is made by > right > >> > way > >> > > >> > > >> > I started vhost: > >> > > >> > ./build/app/vhost-switch -c f -n 4 --huge-dir /mnt/huge --socket-mem > >> 3712 > >> > -- -p 0x1 --dev-basename usvhost --vm2vm 1 --stats 9 > >> > > >> > Now I run two vm machines, with followed configuration > >> > > >> > VM1 __ __ VM2 > >> > eth0 > \/ > eth0 > >> > eth1 >__/\__> eth1 > >> > > >> > So I will connect VM1.eth0 with VM2.eth1 and VM1.eth1 with VM2.eth0 > >> > Because it is test env and I didn't have other network connection on > >> vhost > >> > I will create two networks 192.168.0.x and 192.168.1.x > >> > VM1.eth0 with VM2.eth1 will be placed in 192.168.0.x and VM1.eth1 > with > >> > VM2.eth0 in 192.168.1.x > >> > > >> > ## I started first VM1 as follow > >> > kvm -nographic -boot c -machine pc-i440fx-1.4,accel=kvm -name vm2 -cpu > >> host > >> > -smp 1 \ > >> > -hda /home/ubuntu/esi_ee/qemu/debian_min_1.qcow2 -m 256 -mem-path > >> /mnt/huge > >> > -mem-prealloc \ > >> > -chardev > >> > socket,id=char3,path=/home/ubuntu/esi_ee/dpdk/examples/vhost/usvhost \ > >> > -netdev type=vhost-user,id=hostnet3,chardev=char3 \ > >> > -device > >> > > >> > virtio-net-pci,netdev=hostnet3,id=net3,csum=off,gso=off,guest_tso4=off,guest_tso6=off,guest_ecn=off > >> > \ > >> > -chardev > >> > socket,id=char4,path=/home/ubuntu/esi_ee/dpdk/examples/vhost/usvhost \ > >> > -netdev type=vhost-user,id=hostnet4,chardev=char4 \ > >> > -device > >> > > >> > virtio-net-pci,netdev=hostnet4,id=net4,csum=off,gso=off,guest_tso4=off,guest_tso6=off,guest_ecn=off > >> > ## qemu give followed output > >> > qemu-system-x86_64: -netdev type=vhost-user,id=hostnet3,chardev=char3: > >> > chardev "char3" went up > >> > qemu-system-x86_64: -netdev type=vhost-user,id=hostnet4,chardev=char4: > >> > chardev "char4" went up > >> > > >> > ## second VM2 > >> > kvm -nographic -boot c -machine pc-i440fx-1.4,accel=kvm -name vm1 -cpu > >> host > >> > -smp 1 \ > >> > -hda /home/ubuntu/esi_ee/qemu/debian_min_2.qcow2 -m 256 -mem-path > >> /mnt/huge > >> > -mem-prealloc \ > >> > -chardev > >> > socket,id=char1,path=/home/ubuntu/esi_ee/dpdk/examples/vhost/usvhost \ > >> > -netdev type=vhost-user,id=hostnet1,chardev=char1 \ > >> > -device > >> > > >> > virtio-net-pci,netdev=hostnet1,id=net1,csum=off,gso=off,guest_tso4=off,guest_tso6=off,guest_ecn=off > >> > \ > >> > -chardev > >> > socket,id=char2,path=/home/ubuntu/esi_ee/dpdk/examples/vhost/usvhost \ > >> > -netdev type=vhost-user,id=hostnet2,chardev=char2 \ > >> > -device > >> > > >> > virtio-net-pci,netdev=hostnet2,id=net2,csum=off,gso=off,guest_tso4=off,guest_tso6=off,guest_ecn=off > >> > ## second output > >> > qemu-system-x86_64: -netdev type=vhost-user,id=hostnet1,chardev=char1: > >> > chardev "char1" went up > >> > qemu-system-x86_64: -netdev type=vhost-user,id=hostnet2,chardev=char2: > >> > chardev "char2" went up > >> > > >> > > >> > > >> > After that I had MAC conflict between VM2 and VM1 > >> > > >> > VM1: -ifconfig -a > >> > eth0 Link encap:Ethernet HWaddr 52:54:00:12:34:56 > >> > inet6 addr: fe80::5054:ff:fe12:3456/64 Scope:Link > >> > UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1 > >> > RX packets:0 errors:0 dropped:0 overruns:0 frame:0 > >> > TX packets:0 errors:0 dropped:0 overruns:0 carrier:0 > >> > collisions:0 txqueuelen:1000 > >> > RX bytes:0 (0.0 B) TX bytes:0 (0.0 B) > >> > > >> > eth1 Link encap:Ethernet HWaddr 52:54:00:12:34:57 > >> > BROADCAST MULTICAST MTU:1500 Metric:1 > >> > RX packets:0 errors:0 dropped:0 overruns:0 frame:0 > >> > TX packets:0 errors:0 dropped:0 overruns:0 carrier:0 > >> > collisions:0 txqueuelen:1000 > >> > RX bytes:0 (0.0 B) TX bytes:0 (0.0 B) > >> > > >> > > >> > VM2: -ifconfig -a > >> > eth0 Link encap:Ethernet HWaddr 52:54:00:12:34:56 > >> > inet6 addr: fe80::5054:ff:fe12:3456/64 Scope:Link > >> > UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1 > >> > RX packets:0 errors:0 dropped:0 overruns:0 frame:0 > >> > TX packets:0 errors:0 dropped:0 overruns:0 carrier:0 > >> > collisions:0 txqueuelen:1000 > >> > RX bytes:0 (0.0 B) TX bytes:0 (0.0 B) > >> > > >> > eth1 Link encap:Ethernet HWaddr 52:54:00:12:34:57 > >> > BROADCAST MULTICAST MTU:1500 Metric:1 > >> > RX packets:0 errors:0 dropped:0 overruns:0 frame:0 > >> > TX packets:0 errors:0 dropped:0 overruns:0 carrier:0 > >> > collisions:0 txqueuelen:1000 > >> > RX bytes:0 (0.0 B) TX bytes:0 (0.0 B) > >> > > >> > In KNI example I had something similar and also no packet flow and > >> solution > >> > was to change MAC addresses > >> > > >> > #VM1 > >> > /etc/init.d/networking stop > >> > ifconfig eth0 hw ether 00:01:04:00:01:00 > >> > ifconfig eth1 hw ether 00:01:04:00:01:01 > >> > /etc/init.d/networking start > >> > ifconfig eth0 > >> > ifconfig eth1 > >> > > >> > #VM2 > >> > /etc/init.d/networking stop > >> > ifconfig eth0 hw ether 00:01:04:00:02:00 > >> > ifconfig eth1 hw ether 00:01:04:00:02:01 > >> > /etc/init.d/networking start > >> > ifconfig eth0 > >> > ifconfig eth1 > >> > > >> > Then I make a configuration that You show: > >> > > >> > #VM1 > >> > ip addr add 192.168.0.100/24 dev eth0 > >> > ip addr add 192.168.1.100/24 dev eth1 > >> > ip neigh add 192.168.0.200 lladdr 00:01:04:00:02:01 dev eth0 > >> > ip link set dev eth0 up > >> > ip neigh add 192.168.1.200 lladdr 00:01:04:00:02:00 dev eth1 > >> > ip link set dev eth1 up > >> > > >> > eth0 Link encap:Ethernet HWaddr 00:01:04:00:01:00 > >> > inet addr:192.168.0.100 Bcast:0.0.0.0 Mask:255.255.255.0 > >> > inet6 addr: fe80::201:4ff:fe00:100/64 Scope:Link > >> > UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1 > >> > RX packets:0 errors:0 dropped:0 overruns:0 frame:0 > >> > TX packets:0 errors:0 dropped:0 overruns:0 carrier:0 > >> > collisions:0 txqueuelen:1000 > >> > RX bytes:0 (0.0 B) TX bytes:0 (0.0 B) > >> > > >> > eth1 Link encap:Ethernet HWaddr 00:01:04:00:01:01 > >> > inet addr:192.168.1.100 Bcast:0.0.0.0 Mask:255.255.255.0 > >> > inet6 addr: fe80::201:4ff:fe00:101/64 Scope:Link > >> > UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1 > >> > RX packets:0 errors:0 dropped:0 overruns:0 frame:0 > >> > TX packets:0 errors:0 dropped:0 overruns:0 carrier:0 > >> > collisions:0 txqueuelen:1000 > >> > RX bytes:0 (0.0 B) TX bytes:0 (0.0 B) > >> > > >> > > >> > #VM2 > >> > ip addr add 192.168.1.200/24 dev eth0 > >> > ip addr add 192.168.0.200/24 dev eth1 > >> > ip neigh add 192.168.1.100 lladdr 00:01:04:00:01:01 dev eth0 > >> > ip link set dev eth0 up > >> > ip neigh add 192.168.0.100 lladdr 00:01:04:00:01:00 dev eth1 > >> > ip link set dev eth1 up > >> > > >> > eth0 Link encap:Ethernet HWaddr 00:01:04:00:02:00 > >> > inet addr:192.168.1.200 Bcast:0.0.0.0 Mask:255.255.255.0 > >> > inet6 addr: fe80::201:4ff:fe00:200/64 Scope:Link > >> > UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1 > >> > RX packets:0 errors:0 dropped:0 overruns:0 frame:0 > >> > TX packets:0 errors:0 dropped:0 overruns:0 carrier:0 > >> > collisions:0 txqueuelen:1000 > >> > RX bytes:0 (0.0 B) TX bytes:0 (0.0 B) > >> > > >> > eth1 Link encap:Ethernet HWaddr 00:01:04:00:02:01 > >> > inet addr:192.168.0.200 Bcast:0.0.0.0 Mask:255.255.255.0 > >> > inet6 addr: fe80::201:4ff:fe00:201/64 Scope:Link > >> > UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1 > >> > RX packets:0 errors:0 dropped:0 overruns:0 frame:0 > >> > TX packets:0 errors:0 dropped:0 overruns:0 carrier:0 > >> > collisions:0 txqueuelen:1000 > >> > RX bytes:0 (0.0 B) TX bytes:0 (0.0 B) > >> > > >> > After that: > >> > > >> > VM1.eth0 ip=192.168.0.100-MAC=00:01:04:00:01:00 is connected to > VM2.eth1 > >> > ip=192.168.0.200-MAC=00:01:04:00:02:01 > >> > VM1.eth1 ip=192.168.1.100-MAC=00:01:04:00:01:01 is connected to > VM2.eth0 > >> > ip=192.168.1.200-MAC=00:01:04:00:02:00 > >> > > >> > That show my arp tables: > >> > > >> > #VM1 > >> > arp -a > >> > ? (192.168.0.200) at 00:01:04:00:02:01 [ether] PERM on eth0 > >> > ? (192.168.1.200) at 00:01:04:00:02:00 [ether] PERM on eth1 > >> > > >> > > >> > #VM2 > >> > arp -a > >> > ? (192.168.0.100) at 00:01:04:00:01:00 [ether] PERM on eth1 > >> > ? (192.168.1.100) at 00:01:04:00:01:01 [ether] PERM on eth0 > >> > > >> > > >> > #After this configuration I trying to ping from VM1 VM2 (both IP) > >> > > >> > root@debian-amd64:~# ping -I eth0 192.168.0.200 > >> > PING 192.168.0.200 (192.168.0.200) from 192.168.0.100 eth0: 56(84) > bytes > >> of > >> > data. > >> > ^C > >> > --- 192.168.0.200 ping statistics --- > >> > 5 packets transmitted, 0 received, 100% packet loss, time 4032ms > >> > > >> > root@debian-amd64:~# ping 192.168.0.200 > >> > PING 192.168.0.200 (192.168.0.200) 56(84) bytes of data. > >> > ^C > >> > --- 192.168.0.200 ping statistics --- > >> > 5 packets transmitted, 0 received, 100% packet loss, time 3999ms > >> > > >> > root@debian-amd64:~# ping -I eth1 192.168.1.200 > >> > PING 192.168.1.200 (192.168.1.200) from 192.168.1.100 eth1: 56(84) > bytes > >> of > >> > data. > >> > ^C > >> > --- 192.168.1.200 ping statistics --- > >> > 6 packets transmitted, 0 received, 100% packet loss, time 5040ms > >> > > >> > root@debian-amd64:~# ping 192.168.1.200 > >> > PING 192.168.1.200 (192.168.1.200) 56(84) bytes of data. > >> > ^C > >> > --- 192.168.1.200 ping statistics --- > >> > 5 packets transmitted, 0 received, 100% packet loss, time 4030ms > >> > > >> > #and from VM2 VM1 > >> > root@debian-amd64:~# ping 192.168.0.100 > >> > PING 192.168.0.100 (192.168.0.100) 56(84) bytes of data. > >> > ^C > >> > --- 192.168.0.100 ping statistics --- > >> > 3 packets transmitted, 0 received, 100% packet loss, time 2015ms > >> > > >> > root@debian-amd64:~# ping -I eth1 192.168.0.100 > >> > PING 192.168.0.100 (192.168.0.100) from 192.168.0.200 eth1: 56(84) > bytes > >> of > >> > data. > >> > ^C > >> > --- 192.168.0.100 ping statistics --- > >> > 5 packets transmitted, 0 received, 100% packet loss, time 4032ms > >> > > >> > root@debian-amd64:~# ping -I eth0 192.168.1.100 > >> > PING 192.168.1.100 (192.168.1.100) from 192.168.1.200 eth0: 56(84) > bytes > >> of > >> > data. > >> > ^C > >> > --- 192.168.1.100 ping statistics --- > >> > 4 packets transmitted, 0 received, 100% packet loss, time 3024ms > >> > > >> > root@debian-amd64:~# ping 192.168.1.100 > >> > PING 192.168.1.100 (192.168.1.100) 56(84) bytes of data. > >> > ^C > >> > --- 192.168.1.100 ping statistics --- > >> > 8 packets transmitted, 0 received, 100% packet loss, time 7055ms > >> > > >> > > >> > Also stats from vhost: > >> > Device statistics ==================================== > >> > Statistics for device 0 ------------------------------ > >> > TX total: 0 > >> > TX dropped: 0 > >> > TX successful: 0 > >> > RX total: 0 > >> > RX dropped: 0 > >> > RX successful: 0 > >> > Statistics for device 1 ------------------------------ > >> > TX total: 0 > >> > TX dropped: 0 > >> > TX successful: 0 > >> > RX total: 0 > >> > RX dropped: 0 > >> > RX successful: 0 > >> > Statistics for device 2 ------------------------------ > >> > TX total: 0 > >> > TX dropped: 0 > >> > TX successful: 0 > >> > RX total: 0 > >> > RX dropped: 0 > >> > RX successful: 0 > >> > Statistics for device 3 ------------------------------ > >> > TX total: 0 > >> > TX dropped: 0 > >> > TX successful: 0 > >> > RX total: 0 > >> > RX dropped: 0 > >> > RX successful: 0 > >> > ====================================================== > >> > > >> > My way of thinking was: "In vhost there are several function for L2 > that > >> > learn MAC-s and links them so why I see no received packets?" > >> > > >> > Maybe I'm doing some silly bug in network configuration but for me its > >> > looking like data flow issue especially that no function on the vhost > >> side > >> > did not see any packages. > >> > > >> > On Fri, May 22, 2015 at 10:54 AM, Gaohaifeng (A) < > >> gaohaifeng.gao@huawei.com> > >> > wrote: > >> > > >> >> Hi > >> >> > >> >> What kernel version are You using on host/guest? > >> >> > >> >>>> I use ubuntu 12.04(3.11.0-15-generic) in host. In vm I use ubuntu > >> 12.04 > >> >> and ubuntu14.04 both, but the result is same. > >> >> > >> >> > >> >> > >> >> Do you use some command I suggest before, In case of you miss the > >> previous > >> >> mail, just copy it again: > >> >> > >> >>>> I try it but the result is same > >> >> > >> >> > >> >> > >> >> > >> >> I use l2fwd in vm to do more test and found that virtio_xmit_pkts is > >> >> called and avail_idx is increasing in vm, but in host avail_idx(in > >> >> rte_vhost_dequeue_burst function) is always zero. It seems that the > host > >> >> see the different mem area. > >> >> > >> >> > >> >> > >> >> Init Logs below: > >> >> > >> >> VHOST_CONFIG: (0) Mergeable RX buffers disabled > >> >> > >> >> VHOST_CONFIG: read message VHOST_USER_SET_MEM_TABLE > >> >> > >> >> VHOST_CONFIG: mapped region 0 fd:24 to 0x2aaaaac00000 sz:0xa0000 > off:0x0 > >> >> > >> >> VHOST_CONFIG: REGION: 0 GPA: (nil) QEMU VA: 0x2aaaaac00000 SIZE > (655360) > >> >> > >> >> VHOST_CONFIG: mapped region 1 fd:26 to 0x2aaaaae00000 sz:0x40000000 > >> >> off:0xc0000 > >> >> > >> >> VHOST_CONFIG: REGION: 1 GPA: 0xc0000 QEMU VA: 0x2aaaaacc0000 SIZE > >> >> (1072955392) > >> >> > >> >> VHOST_CONFIG: read message VHOST_USER_SET_VRING_NUM > >> >> > >> >> VHOST_CONFIG: read message VHOST_USER_SET_VRING_BASE > >> >> > >> >> VHOST_CONFIG: read message VHOST_USER_SET_VRING_ADDR > >> >> > >> >> VHOST_CONFIG: (0) mapped address desc: 0x2aaae62f1000 > >> >> > >> >> VHOST_CONFIG: (0) mapped address avail: 0x2aaae62f2000 > >> >> > >> >> VHOST_CONFIG: (0) mapped address used: 0x2aaae62f3000 > >> >> > >> >> VHOST_CONFIG: read message VHOST_USER_SET_VRING_KICK > >> >> > >> >> VHOST_CONFIG: vring kick idx:0 file:23 > >> >> > >> >> VHOST_CONFIG: virtio isn't ready for processing. > >> >> > >> >> VHOST_CONFIG: read message VHOST_USER_SET_VRING_NUM > >> >> > >> >> VHOST_CONFIG: read message VHOST_USER_SET_VRING_BASE > >> >> > >> >> VHOST_CONFIG: read message VHOST_USER_SET_VRING_ADDR > >> >> > >> >> VHOST_CONFIG: (0) mapped address desc: 0x2aaae62f4000 > >> >> > >> >> VHOST_CONFIG: (0) mapped address avail: 0x2aaae62f5000 > >> >> > >> >> VHOST_CONFIG: (0) mapped address used: 0x2aaae62f6000 > >> >> > >> >> VHOST_CONFIG: read message VHOST_USER_SET_VRING_KICK > >> >> > >> >> VHOST_CONFIG: vring kick idx:1 file:28 > >> >> > >> >> VHOST_CONFIG: virtio is now ready for processing. > >> >> > >> >> > >> >> > >> >> > >> >> > >> >>> Unfortunately not, I have the same issue in rte_vhost_dequeue_burst > >> >> function. > >> >> > >> >> > >> >> > >> >>> What kernel version are You using on host/guest? In my case on host > I > >> >> had 3.13.0 and on guests old 3.2 debian. > >> >> > >> >> > >> >> > >> >>> I just looked deeper into virtio back-end (vhost) but at first > glace > >> it > >> >> seems like nothing coming from virtio. > >> >> > >> >> > >> >> > >> >>> What I'm going to do today is to compile newest kernel for vhost and > >> >> guest and debug where packet flow stuck, I will report the result > >> >> > >> >> > >> >> > >> >>> On Thu, May 21, 2015 at 11:12 AM, Gaohaifeng (A) < > >> >> gaohaifeng.gao@huawei.com> wrote: > >> >> > >> >>> Hi Maciej > >> >> >Did you solve your problem? I meet this problem as your > case. > >> >> And I found avail_idx(in rte_vhost_dequeue_burst function) is always > >> zero > >> >> although I do send packets in VM. > >> >> > >> >>> Thanks. > >> >> > >> >> > >> >>> Hello, I have strange issue with example/vhost app. > >> >>> > >> >>> I had compiled DPDK to run a vhost example app with followed flags > >> >>> > >> >>> CONFIG_RTE_LIBRTE_VHOST=y > >> >>> CONFIG_RTE_LIBRTE_VHOST_USER=y > >> >>> CONFIG_RTE_LIBRTE_VHOST_DEBUG=n > >> >>> > >> >>> then I run vhost app based on documentation: > >> >>> > >> >>> ./build/app/vhost-switch -c f -n 4 --huge-dir /mnt/huge > --socket-mem > >> >>> 3712 > >> >>> -- -p 0x1 --dev-basename usvhost --vm2vm 1 --stats 9 > >> >>> > >> >>> -I use this strange --socket-mem 3712 because of physical limit of > >> >>> memoryon device -with this vhost user I run two KVM machines with > >> >>> followed parameters > >> >>> > >> >>> kvm -nographic -boot c -machine pc-i440fx-1.4,accel=kvm -name vm1 > -cpu > >> >>> host -smp 2 -hda /home/ubuntu/qemu/debian_squeeze2_amd64.qcow2 -m > >> >>> 1024 -mem-path /mnt/huge -mem-prealloc -chardev > >> >>> socket,id=char1,path=/home/ubuntu/dpdk/examples/vhost/usvhost > >> >>> -netdev type=vhost-user,id=hostnet1,chardev=char1 > >> >>> -device virtio-net > >> >>> > pci,netdev=hostnet1,id=net1,csum=off,gso=off,guest_tso4=off,guest_tso6 > >> >>> = > >> >>> off,guest_ecn=off > >> >>> -chardev > >> >>> socket,id=char2,path=/home/ubuntu/dpdk/examples/vhost/usvhost > >> >>> -netdev type=vhost-user,id=hostnet2,chardev=char2 > >> >>> -device > >> >>> virtio-net- > >> >>> > pci,netdev=hostnet2,id=net2,csum=off,gso=off,guest_tso4=off,guest_tso6 > >> >>> = > >> >>> off,guest_ecn=off > >> >>> > >> >>> After running KVM virtio correctly starting (below logs from vhost > app) > >> >> ... > >> >>> VHOST_CONFIG: mapped region 0 fd:31 to 0x2aaabae00000 sz:0xa0000 > >> >>> off:0x0 > >> >>> VHOST_CONFIG: mapped region 1 fd:37 to 0x2aaabb000000 sz:0x10000000 > >> >>> off:0xc0000 > >> >>> VHOST_CONFIG: read message VHOST_USER_SET_VRING_NUM > >> >>> VHOST_CONFIG: read message VHOST_USER_SET_VRING_BASE > >> >>> VHOST_CONFIG: read message VHOST_USER_SET_VRING_ADDR > >> >>> VHOST_CONFIG: read message VHOST_USER_SET_VRING_KICK > >> >>> VHOST_CONFIG: vring kick idx:0 file:38 > >> >>> VHOST_CONFIG: virtio isn't ready for processing. > >> >>> VHOST_CONFIG: read message VHOST_USER_SET_VRING_NUM > >> >>> VHOST_CONFIG: read message VHOST_USER_SET_VRING_BASE > >> >>> VHOST_CONFIG: read message VHOST_USER_SET_VRING_ADDR > >> >>> VHOST_CONFIG: read message VHOST_USER_SET_VRING_KICK > >> >>> VHOST_CONFIG: vring kick idx:1 file:39 > >> >>> VHOST_CONFIG: virtio is now ready for processing. > >> >>> VHOST_DATA: (1) Device has been added to data core 2 > >> >>> > >> >>> So everything looking good. > >> >>> > >> >>> Maybe it is something trivial but using options: --vm2vm 1 (or) 2 > >> >>> --stats 9 it seems that I didn't have connection between VM2VM > >> >>> communication. I set manually IP for eth0 and eth1: > >> >>> > >> >>> on 1 VM > >> >>> ifconfig eth0 192.168.0.100 netmask 255.255.255.0 up ifconfig eth1 > >> >>> 192.168.1.101 netmask 255.255.255.0 up > >> >>> > >> >>> on 2 VM > >> >>> ifconfig eth0 192.168.1.200 netmask 255.255.255.0 up ifconfig eth1 > >> >>> 192.168.0.202 netmask 255.255.255.0 up > >> >>> > >> >>> I notice that in vhostapp are one directional rx/tx queue so I > tryied > >> >>> to ping between VM1 to VM2 using both interfaces ping -I eth0 > >> >>> 192.168.1.200 ping -I > >> >>> eth1 192.168.1.200 ping -I eth0 192.168.0.202 ping -I eth1 > >> >>> 192.168.0.202 > >> >>> > >> >>> on VM2 using tcpdump on both interfaces I didn't see any ICMP > requests > >> >>> or traffic > >> >>> > >> >>> And I cant ping between any IP/interfaces, moreover stats show me > that: > >> >>> > >> >>> Device statistics ==================================== > >> >>> Statistics for device 0 ------------------------------ > >> >>> TX total: 0 > >> >>> TX dropped: 0 > >> >>> TX successful: 0 > >> >>> RX total: 0 > >> >>> RX dropped: 0 > >> >>> RX successful: 0 > >> >>> Statistics for device 1 ------------------------------ > >> >>> TX total: 0 > >> >>> TX dropped: 0 > >> >>> TX successful: 0 > >> >>> RX total: 0 > >> >>> RX dropped: 0 > >> >>> RX successful: 0 > >> >>> Statistics for device 2 ------------------------------ > >> >>> TX total: 0 > >> >>> TX dropped: 0 > >> >>> TX successful: 0 > >> >>> RX total: 0 > >> >>> RX dropped: 0 > >> >>> RX successful: 0 > >> >>> Statistics for device 3 ------------------------------ > >> >>> TX total: 0 > >> >>> TX dropped: 0 > >> >>> TX successful: 0 > >> >>> RX total: 0 > >> >>> RX dropped: 0 > >> >>> RX successful: 0 > >> >>> ====================================================== > >> >>> > >> >>> So it seems like any packet didn't leave my VM. > >> >>> also arp table is empty on each VM. > >> >> > >> >> > >> > >> > > > > -- > Andriy Berestovskyy >