From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mail-pa0-f47.google.com (mail-pa0-f47.google.com [209.85.220.47]) by dpdk.org (Postfix) with ESMTP id 656E5592A for ; Fri, 22 May 2015 11:58:56 +0200 (CEST) Received: by paza2 with SMTP id a2so6508403paz.3 for ; Fri, 22 May 2015 02:58:55 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:message-id:date:from:user-agent:mime-version:to :cc:subject:references:in-reply-to:content-type :content-transfer-encoding; bh=Ihv44E30cKbTlOS5mUaWd2qLJx4yH6DGpLvWdtxpWo8=; b=kRu9k9BUysMyDnG1eZbUeyqGw5hdiBBv8wku+b7mBl0jHDWb9jnPfmfH+j31h/v7BW AT9EwZ1r8kCaTRtX/4EEnRk2ZVxNfbt14S1nZhRvTsgiOyiGIvdzgJKKd52cu4O7gENw FhXdcHSkmmduVM7PIXmAIJxMXiVAPAIuCuHgJc3LKf/im+nu++51XR+dksed1sPh4Bo0 RJzkKqvzkHCiZFvyRYKjm7d3Mu4uts4uXfWe5FCYLzfmKKhCWA4fhxDMRVRVmsZtv4iL wnTO/YWyYUBqi1075u7spmG/sK8XColEFDNk0+F3uqXMmjhvd8TAHb0QCLzYK9PMa+kd ZSrQ== X-Gm-Message-State: ALoCoQlxxUOwwKfQ1SDHaG4C2zltfLA8JTJbnlQcVC0cFWaI1No/6yb3pQDJL0+/fN7p0ptpduqp X-Received: by 10.68.69.35 with SMTP id b3mr14019771pbu.144.1432288735616; Fri, 22 May 2015 02:58:55 -0700 (PDT) Received: from [10.16.129.101] (napt.igel.co.jp. [219.106.231.132]) by mx.google.com with ESMTPSA id jt2sm1626252pbc.21.2015.05.22.02.58.53 (version=TLSv1.2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Fri, 22 May 2015 02:58:54 -0700 (PDT) Message-ID: <555EFDDC.1010004@igel.co.jp> Date: Fri, 22 May 2015 18:58:52 +0900 From: Tetsuya Mukawa User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:31.0) Gecko/20100101 Thunderbird/31.7.0 MIME-Version: 1.0 To: Maciej Grochowski References: <47CEA9C0E570484FBF22EF0D7EBCE5B534AC4253@szxema505-mbs.china.huawei.com> <47CEA9C0E570484FBF22EF0D7EBCE5B534AC4D90@szxema505-mbs.china.huawei.com> In-Reply-To: Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit Cc: "dev@dpdk.org" Subject: Re: [dpdk-dev] Vhost user no connection vm2vm X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: patches and discussions about DPDK List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 22 May 2015 09:58:57 -0000 Hi Maciej, I guess it's nice to make sure guest memory is actually allocated by hugepages. So please check like below. $ cat /sys/kernel/mm/hugepage/xxxxx/free_hugepages $ ./start_qemu.sh $ cat /sys/kernel/mm/hugepage/xxxxx/free_hugepages If qemu guest allocates memory from hugepages, 2nd cat command will indicate it. Thanks, Tetsuya On 2015/05/22 18:28, Maciej Grochowski wrote: > "Do you use some command I suggest before, > In case of you miss the previous mail, just copy it again:" > > -Yes but it didn't help me ;/ > > I will describe step by step to esure that configuration is made by right > way > > > I started vhost: > > ./build/app/vhost-switch -c f -n 4 --huge-dir /mnt/huge --socket-mem 3712 > -- -p 0x1 --dev-basename usvhost --vm2vm 1 --stats 9 > > Now I run two vm machines, with followed configuration > > VM1 __ __ VM2 > eth0 > \/ > eth0 > eth1 >__/\__> eth1 > > So I will connect VM1.eth0 with VM2.eth1 and VM1.eth1 with VM2.eth0 > Because it is test env and I didn't have other network connection on vhost > I will create two networks 192.168.0.x and 192.168.1.x > VM1.eth0 with VM2.eth1 will be placed in 192.168.0.x and VM1.eth1 with > VM2.eth0 in 192.168.1.x > > ## I started first VM1 as follow > kvm -nographic -boot c -machine pc-i440fx-1.4,accel=kvm -name vm2 -cpu host > -smp 1 \ > -hda /home/ubuntu/esi_ee/qemu/debian_min_1.qcow2 -m 256 -mem-path /mnt/huge > -mem-prealloc \ > -chardev > socket,id=char3,path=/home/ubuntu/esi_ee/dpdk/examples/vhost/usvhost \ > -netdev type=vhost-user,id=hostnet3,chardev=char3 \ > -device > virtio-net-pci,netdev=hostnet3,id=net3,csum=off,gso=off,guest_tso4=off,guest_tso6=off,guest_ecn=off > \ > -chardev > socket,id=char4,path=/home/ubuntu/esi_ee/dpdk/examples/vhost/usvhost \ > -netdev type=vhost-user,id=hostnet4,chardev=char4 \ > -device > virtio-net-pci,netdev=hostnet4,id=net4,csum=off,gso=off,guest_tso4=off,guest_tso6=off,guest_ecn=off > ## qemu give followed output > qemu-system-x86_64: -netdev type=vhost-user,id=hostnet3,chardev=char3: > chardev "char3" went up > qemu-system-x86_64: -netdev type=vhost-user,id=hostnet4,chardev=char4: > chardev "char4" went up > > ## second VM2 > kvm -nographic -boot c -machine pc-i440fx-1.4,accel=kvm -name vm1 -cpu host > -smp 1 \ > -hda /home/ubuntu/esi_ee/qemu/debian_min_2.qcow2 -m 256 -mem-path /mnt/huge > -mem-prealloc \ > -chardev > socket,id=char1,path=/home/ubuntu/esi_ee/dpdk/examples/vhost/usvhost \ > -netdev type=vhost-user,id=hostnet1,chardev=char1 \ > -device > virtio-net-pci,netdev=hostnet1,id=net1,csum=off,gso=off,guest_tso4=off,guest_tso6=off,guest_ecn=off > \ > -chardev > socket,id=char2,path=/home/ubuntu/esi_ee/dpdk/examples/vhost/usvhost \ > -netdev type=vhost-user,id=hostnet2,chardev=char2 \ > -device > virtio-net-pci,netdev=hostnet2,id=net2,csum=off,gso=off,guest_tso4=off,guest_tso6=off,guest_ecn=off > ## second output > qemu-system-x86_64: -netdev type=vhost-user,id=hostnet1,chardev=char1: > chardev "char1" went up > qemu-system-x86_64: -netdev type=vhost-user,id=hostnet2,chardev=char2: > chardev "char2" went up > > > > After that I had MAC conflict between VM2 and VM1 > > VM1: -ifconfig -a > eth0 Link encap:Ethernet HWaddr 52:54:00:12:34:56 > inet6 addr: fe80::5054:ff:fe12:3456/64 Scope:Link > UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1 > RX packets:0 errors:0 dropped:0 overruns:0 frame:0 > TX packets:0 errors:0 dropped:0 overruns:0 carrier:0 > collisions:0 txqueuelen:1000 > RX bytes:0 (0.0 B) TX bytes:0 (0.0 B) > > eth1 Link encap:Ethernet HWaddr 52:54:00:12:34:57 > BROADCAST MULTICAST MTU:1500 Metric:1 > RX packets:0 errors:0 dropped:0 overruns:0 frame:0 > TX packets:0 errors:0 dropped:0 overruns:0 carrier:0 > collisions:0 txqueuelen:1000 > RX bytes:0 (0.0 B) TX bytes:0 (0.0 B) > > > VM2: -ifconfig -a > eth0 Link encap:Ethernet HWaddr 52:54:00:12:34:56 > inet6 addr: fe80::5054:ff:fe12:3456/64 Scope:Link > UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1 > RX packets:0 errors:0 dropped:0 overruns:0 frame:0 > TX packets:0 errors:0 dropped:0 overruns:0 carrier:0 > collisions:0 txqueuelen:1000 > RX bytes:0 (0.0 B) TX bytes:0 (0.0 B) > > eth1 Link encap:Ethernet HWaddr 52:54:00:12:34:57 > BROADCAST MULTICAST MTU:1500 Metric:1 > RX packets:0 errors:0 dropped:0 overruns:0 frame:0 > TX packets:0 errors:0 dropped:0 overruns:0 carrier:0 > collisions:0 txqueuelen:1000 > RX bytes:0 (0.0 B) TX bytes:0 (0.0 B) > > In KNI example I had something similar and also no packet flow and solution > was to change MAC addresses > > #VM1 > /etc/init.d/networking stop > ifconfig eth0 hw ether 00:01:04:00:01:00 > ifconfig eth1 hw ether 00:01:04:00:01:01 > /etc/init.d/networking start > ifconfig eth0 > ifconfig eth1 > > #VM2 > /etc/init.d/networking stop > ifconfig eth0 hw ether 00:01:04:00:02:00 > ifconfig eth1 hw ether 00:01:04:00:02:01 > /etc/init.d/networking start > ifconfig eth0 > ifconfig eth1 > > Then I make a configuration that You show: > > #VM1 > ip addr add 192.168.0.100/24 dev eth0 > ip addr add 192.168.1.100/24 dev eth1 > ip neigh add 192.168.0.200 lladdr 00:01:04:00:02:01 dev eth0 > ip link set dev eth0 up > ip neigh add 192.168.1.200 lladdr 00:01:04:00:02:00 dev eth1 > ip link set dev eth1 up > > eth0 Link encap:Ethernet HWaddr 00:01:04:00:01:00 > inet addr:192.168.0.100 Bcast:0.0.0.0 Mask:255.255.255.0 > inet6 addr: fe80::201:4ff:fe00:100/64 Scope:Link > UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1 > RX packets:0 errors:0 dropped:0 overruns:0 frame:0 > TX packets:0 errors:0 dropped:0 overruns:0 carrier:0 > collisions:0 txqueuelen:1000 > RX bytes:0 (0.0 B) TX bytes:0 (0.0 B) > > eth1 Link encap:Ethernet HWaddr 00:01:04:00:01:01 > inet addr:192.168.1.100 Bcast:0.0.0.0 Mask:255.255.255.0 > inet6 addr: fe80::201:4ff:fe00:101/64 Scope:Link > UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1 > RX packets:0 errors:0 dropped:0 overruns:0 frame:0 > TX packets:0 errors:0 dropped:0 overruns:0 carrier:0 > collisions:0 txqueuelen:1000 > RX bytes:0 (0.0 B) TX bytes:0 (0.0 B) > > > #VM2 > ip addr add 192.168.1.200/24 dev eth0 > ip addr add 192.168.0.200/24 dev eth1 > ip neigh add 192.168.1.100 lladdr 00:01:04:00:01:01 dev eth0 > ip link set dev eth0 up > ip neigh add 192.168.0.100 lladdr 00:01:04:00:01:00 dev eth1 > ip link set dev eth1 up > > eth0 Link encap:Ethernet HWaddr 00:01:04:00:02:00 > inet addr:192.168.1.200 Bcast:0.0.0.0 Mask:255.255.255.0 > inet6 addr: fe80::201:4ff:fe00:200/64 Scope:Link > UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1 > RX packets:0 errors:0 dropped:0 overruns:0 frame:0 > TX packets:0 errors:0 dropped:0 overruns:0 carrier:0 > collisions:0 txqueuelen:1000 > RX bytes:0 (0.0 B) TX bytes:0 (0.0 B) > > eth1 Link encap:Ethernet HWaddr 00:01:04:00:02:01 > inet addr:192.168.0.200 Bcast:0.0.0.0 Mask:255.255.255.0 > inet6 addr: fe80::201:4ff:fe00:201/64 Scope:Link > UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1 > RX packets:0 errors:0 dropped:0 overruns:0 frame:0 > TX packets:0 errors:0 dropped:0 overruns:0 carrier:0 > collisions:0 txqueuelen:1000 > RX bytes:0 (0.0 B) TX bytes:0 (0.0 B) > > After that: > > VM1.eth0 ip=192.168.0.100-MAC=00:01:04:00:01:00 is connected to VM2.eth1 > ip=192.168.0.200-MAC=00:01:04:00:02:01 > VM1.eth1 ip=192.168.1.100-MAC=00:01:04:00:01:01 is connected to VM2.eth0 > ip=192.168.1.200-MAC=00:01:04:00:02:00 > > That show my arp tables: > > #VM1 > arp -a > ? (192.168.0.200) at 00:01:04:00:02:01 [ether] PERM on eth0 > ? (192.168.1.200) at 00:01:04:00:02:00 [ether] PERM on eth1 > > > #VM2 > arp -a > ? (192.168.0.100) at 00:01:04:00:01:00 [ether] PERM on eth1 > ? (192.168.1.100) at 00:01:04:00:01:01 [ether] PERM on eth0 > > > #After this configuration I trying to ping from VM1 VM2 (both IP) > > root@debian-amd64:~# ping -I eth0 192.168.0.200 > PING 192.168.0.200 (192.168.0.200) from 192.168.0.100 eth0: 56(84) bytes of > data. > ^C > --- 192.168.0.200 ping statistics --- > 5 packets transmitted, 0 received, 100% packet loss, time 4032ms > > root@debian-amd64:~# ping 192.168.0.200 > PING 192.168.0.200 (192.168.0.200) 56(84) bytes of data. > ^C > --- 192.168.0.200 ping statistics --- > 5 packets transmitted, 0 received, 100% packet loss, time 3999ms > > root@debian-amd64:~# ping -I eth1 192.168.1.200 > PING 192.168.1.200 (192.168.1.200) from 192.168.1.100 eth1: 56(84) bytes of > data. > ^C > --- 192.168.1.200 ping statistics --- > 6 packets transmitted, 0 received, 100% packet loss, time 5040ms > > root@debian-amd64:~# ping 192.168.1.200 > PING 192.168.1.200 (192.168.1.200) 56(84) bytes of data. > ^C > --- 192.168.1.200 ping statistics --- > 5 packets transmitted, 0 received, 100% packet loss, time 4030ms > > #and from VM2 VM1 > root@debian-amd64:~# ping 192.168.0.100 > PING 192.168.0.100 (192.168.0.100) 56(84) bytes of data. > ^C > --- 192.168.0.100 ping statistics --- > 3 packets transmitted, 0 received, 100% packet loss, time 2015ms > > root@debian-amd64:~# ping -I eth1 192.168.0.100 > PING 192.168.0.100 (192.168.0.100) from 192.168.0.200 eth1: 56(84) bytes of > data. > ^C > --- 192.168.0.100 ping statistics --- > 5 packets transmitted, 0 received, 100% packet loss, time 4032ms > > root@debian-amd64:~# ping -I eth0 192.168.1.100 > PING 192.168.1.100 (192.168.1.100) from 192.168.1.200 eth0: 56(84) bytes of > data. > ^C > --- 192.168.1.100 ping statistics --- > 4 packets transmitted, 0 received, 100% packet loss, time 3024ms > > root@debian-amd64:~# ping 192.168.1.100 > PING 192.168.1.100 (192.168.1.100) 56(84) bytes of data. > ^C > --- 192.168.1.100 ping statistics --- > 8 packets transmitted, 0 received, 100% packet loss, time 7055ms > > > Also stats from vhost: > Device statistics ==================================== > Statistics for device 0 ------------------------------ > TX total: 0 > TX dropped: 0 > TX successful: 0 > RX total: 0 > RX dropped: 0 > RX successful: 0 > Statistics for device 1 ------------------------------ > TX total: 0 > TX dropped: 0 > TX successful: 0 > RX total: 0 > RX dropped: 0 > RX successful: 0 > Statistics for device 2 ------------------------------ > TX total: 0 > TX dropped: 0 > TX successful: 0 > RX total: 0 > RX dropped: 0 > RX successful: 0 > Statistics for device 3 ------------------------------ > TX total: 0 > TX dropped: 0 > TX successful: 0 > RX total: 0 > RX dropped: 0 > RX successful: 0 > ====================================================== > > My way of thinking was: "In vhost there are several function for L2 that > learn MAC-s and links them so why I see no received packets?" > > Maybe I'm doing some silly bug in network configuration but for me its > looking like data flow issue especially that no function on the vhost side > did not see any packages. > > On Fri, May 22, 2015 at 10:54 AM, Gaohaifeng (A) > wrote: > >> Hi >> >> What kernel version are You using on host/guest? >> >>>> I use ubuntu 12.04(3.11.0-15-generic) in host. In vm I use ubuntu 12.04 >> and ubuntu14.04 both, but the result is same. >> >> >> >> Do you use some command I suggest before, In case of you miss the previous >> mail, just copy it again: >> >>>> I try it but the result is same >> >> >> >> >> I use l2fwd in vm to do more test and found that virtio_xmit_pkts is >> called and avail_idx is increasing in vm, but in host avail_idx(in >> rte_vhost_dequeue_burst function) is always zero. It seems that the host >> see the different mem area. >> >> >> >> Init Logs below: >> >> VHOST_CONFIG: (0) Mergeable RX buffers disabled >> >> VHOST_CONFIG: read message VHOST_USER_SET_MEM_TABLE >> >> VHOST_CONFIG: mapped region 0 fd:24 to 0x2aaaaac00000 sz:0xa0000 off:0x0 >> >> VHOST_CONFIG: REGION: 0 GPA: (nil) QEMU VA: 0x2aaaaac00000 SIZE (655360) >> >> VHOST_CONFIG: mapped region 1 fd:26 to 0x2aaaaae00000 sz:0x40000000 >> off:0xc0000 >> >> VHOST_CONFIG: REGION: 1 GPA: 0xc0000 QEMU VA: 0x2aaaaacc0000 SIZE >> (1072955392) >> >> VHOST_CONFIG: read message VHOST_USER_SET_VRING_NUM >> >> VHOST_CONFIG: read message VHOST_USER_SET_VRING_BASE >> >> VHOST_CONFIG: read message VHOST_USER_SET_VRING_ADDR >> >> VHOST_CONFIG: (0) mapped address desc: 0x2aaae62f1000 >> >> VHOST_CONFIG: (0) mapped address avail: 0x2aaae62f2000 >> >> VHOST_CONFIG: (0) mapped address used: 0x2aaae62f3000 >> >> VHOST_CONFIG: read message VHOST_USER_SET_VRING_KICK >> >> VHOST_CONFIG: vring kick idx:0 file:23 >> >> VHOST_CONFIG: virtio isn't ready for processing. >> >> VHOST_CONFIG: read message VHOST_USER_SET_VRING_NUM >> >> VHOST_CONFIG: read message VHOST_USER_SET_VRING_BASE >> >> VHOST_CONFIG: read message VHOST_USER_SET_VRING_ADDR >> >> VHOST_CONFIG: (0) mapped address desc: 0x2aaae62f4000 >> >> VHOST_CONFIG: (0) mapped address avail: 0x2aaae62f5000 >> >> VHOST_CONFIG: (0) mapped address used: 0x2aaae62f6000 >> >> VHOST_CONFIG: read message VHOST_USER_SET_VRING_KICK >> >> VHOST_CONFIG: vring kick idx:1 file:28 >> >> VHOST_CONFIG: virtio is now ready for processing. >> >> >> >> >> >>> Unfortunately not, I have the same issue in rte_vhost_dequeue_burst >> function. >> >> >> >>> What kernel version are You using on host/guest? In my case on host I >> had 3.13.0 and on guests old 3.2 debian. >> >> >> >>> I just looked deeper into virtio back-end (vhost) but at first glace it >> seems like nothing coming from virtio. >> >> >> >>> What I'm going to do today is to compile newest kernel for vhost and >> guest and debug where packet flow stuck, I will report the result >> >> >> >>> On Thu, May 21, 2015 at 11:12 AM, Gaohaifeng (A) < >> gaohaifeng.gao@huawei.com> wrote: >> >>> Hi Maciej >> >Did you solve your problem? I meet this problem as your case. >> And I found avail_idx(in rte_vhost_dequeue_burst function) is always zero >> although I do send packets in VM. >> >>> Thanks. >> >> >>> Hello, I have strange issue with example/vhost app. >>> >>> I had compiled DPDK to run a vhost example app with followed flags >>> >>> CONFIG_RTE_LIBRTE_VHOST=y >>> CONFIG_RTE_LIBRTE_VHOST_USER=y >>> CONFIG_RTE_LIBRTE_VHOST_DEBUG=n >>> >>> then I run vhost app based on documentation: >>> >>> ./build/app/vhost-switch -c f -n 4 --huge-dir /mnt/huge --socket-mem >>> 3712 >>> -- -p 0x1 --dev-basename usvhost --vm2vm 1 --stats 9 >>> >>> -I use this strange --socket-mem 3712 because of physical limit of >>> memoryon device -with this vhost user I run two KVM machines with >>> followed parameters >>> >>> kvm -nographic -boot c -machine pc-i440fx-1.4,accel=kvm -name vm1 -cpu >>> host -smp 2 -hda /home/ubuntu/qemu/debian_squeeze2_amd64.qcow2 -m >>> 1024 -mem-path /mnt/huge -mem-prealloc -chardev >>> socket,id=char1,path=/home/ubuntu/dpdk/examples/vhost/usvhost >>> -netdev type=vhost-user,id=hostnet1,chardev=char1 >>> -device virtio-net >>> pci,netdev=hostnet1,id=net1,csum=off,gso=off,guest_tso4=off,guest_tso6 >>> = >>> off,guest_ecn=off >>> -chardev >>> socket,id=char2,path=/home/ubuntu/dpdk/examples/vhost/usvhost >>> -netdev type=vhost-user,id=hostnet2,chardev=char2 >>> -device >>> virtio-net- >>> pci,netdev=hostnet2,id=net2,csum=off,gso=off,guest_tso4=off,guest_tso6 >>> = >>> off,guest_ecn=off >>> >>> After running KVM virtio correctly starting (below logs from vhost app) >> ... >>> VHOST_CONFIG: mapped region 0 fd:31 to 0x2aaabae00000 sz:0xa0000 >>> off:0x0 >>> VHOST_CONFIG: mapped region 1 fd:37 to 0x2aaabb000000 sz:0x10000000 >>> off:0xc0000 >>> VHOST_CONFIG: read message VHOST_USER_SET_VRING_NUM >>> VHOST_CONFIG: read message VHOST_USER_SET_VRING_BASE >>> VHOST_CONFIG: read message VHOST_USER_SET_VRING_ADDR >>> VHOST_CONFIG: read message VHOST_USER_SET_VRING_KICK >>> VHOST_CONFIG: vring kick idx:0 file:38 >>> VHOST_CONFIG: virtio isn't ready for processing. >>> VHOST_CONFIG: read message VHOST_USER_SET_VRING_NUM >>> VHOST_CONFIG: read message VHOST_USER_SET_VRING_BASE >>> VHOST_CONFIG: read message VHOST_USER_SET_VRING_ADDR >>> VHOST_CONFIG: read message VHOST_USER_SET_VRING_KICK >>> VHOST_CONFIG: vring kick idx:1 file:39 >>> VHOST_CONFIG: virtio is now ready for processing. >>> VHOST_DATA: (1) Device has been added to data core 2 >>> >>> So everything looking good. >>> >>> Maybe it is something trivial but using options: --vm2vm 1 (or) 2 >>> --stats 9 it seems that I didn't have connection between VM2VM >>> communication. I set manually IP for eth0 and eth1: >>> >>> on 1 VM >>> ifconfig eth0 192.168.0.100 netmask 255.255.255.0 up ifconfig eth1 >>> 192.168.1.101 netmask 255.255.255.0 up >>> >>> on 2 VM >>> ifconfig eth0 192.168.1.200 netmask 255.255.255.0 up ifconfig eth1 >>> 192.168.0.202 netmask 255.255.255.0 up >>> >>> I notice that in vhostapp are one directional rx/tx queue so I tryied >>> to ping between VM1 to VM2 using both interfaces ping -I eth0 >>> 192.168.1.200 ping -I >>> eth1 192.168.1.200 ping -I eth0 192.168.0.202 ping -I eth1 >>> 192.168.0.202 >>> >>> on VM2 using tcpdump on both interfaces I didn't see any ICMP requests >>> or traffic >>> >>> And I cant ping between any IP/interfaces, moreover stats show me that: >>> >>> Device statistics ==================================== >>> Statistics for device 0 ------------------------------ >>> TX total: 0 >>> TX dropped: 0 >>> TX successful: 0 >>> RX total: 0 >>> RX dropped: 0 >>> RX successful: 0 >>> Statistics for device 1 ------------------------------ >>> TX total: 0 >>> TX dropped: 0 >>> TX successful: 0 >>> RX total: 0 >>> RX dropped: 0 >>> RX successful: 0 >>> Statistics for device 2 ------------------------------ >>> TX total: 0 >>> TX dropped: 0 >>> TX successful: 0 >>> RX total: 0 >>> RX dropped: 0 >>> RX successful: 0 >>> Statistics for device 3 ------------------------------ >>> TX total: 0 >>> TX dropped: 0 >>> TX successful: 0 >>> RX total: 0 >>> RX dropped: 0 >>> RX successful: 0 >>> ====================================================== >>> >>> So it seems like any packet didn't leave my VM. >>> also arp table is empty on each VM. >> >>