From: "Gaohaifeng (A)" <gaohaifeng.gao@huawei.com>
To: Maciej Grochowski <maciej.grochowski@codilime.com>,
Andriy Berestovskyy <aber@semihalf.com>
Cc: "dev@dpdk.org" <dev@dpdk.org>
Subject: Re: [dpdk-dev] Vhost user no connection vm2vm
Date: Mon, 25 May 2015 04:15:45 +0000 [thread overview]
Message-ID: <47CEA9C0E570484FBF22EF0D7EBCE5B534AC5DE2@szxema505-mbs.china.huawei.com> (raw)
In-Reply-To: <CALPCkO-yzymwBXdV4Bnb2aHAUbfEYqApT0j0WSiYYBtXT3BZ9w@mail.gmail.com>
Much Thanks.
>Thank You Andriy, You are right
>Below I put tested KVM configuration that put packets into vhost dpdk plane
>space:
>export TLBFS_DIR=/mnt/huge
>export UVH_PREFIX=/home/ubuntu/esi_ee/dpdk/examples/vhost/usvhost
>export VM1_MAC=00:01:04:00:01:00
>kvm -cpu host -smp 2 -enable-kvm \
> -drive if=virtio,file=debian_min_2.qcow2,cache=none \
> -object
>memory-backend-file,id=mem,size=756M,mem-path=${TLBFS_DIR},share=on \
> -numa node,memdev=mem \
> -m 756 -nographic \
> -chardev socket,id=charnet0,path=${UVH_PREFIX} \
> -netdev type=vhost-user,id=hostnet0,chardev=charnet0 \
> -device virtio-net-pci,netdev=hostnet0,mac=${VM1_MAC}
>Notice that:
>memory-backend-file must be with right -numa node, "memdev=mem" option
> So yeah right configuration is a essence :)
> On Fri, May 22, 2015 at 12:54 PM, Andriy Berestovskyy <aber@semihalf.com>
> wrote:
> Hi guys,
> I guess you just miss the qemu flag to map the memory as shared, i.e.:
> -object
> memory-backend-file,id=mem,size=1024M,mem-path=/mnt/huge,share=on
> (the keyword is share=on)
>
> Here is an example script:
>
> https://github.com/srajag/contrail-vrouter/blob/dpdk-devel/examples/vm
> s/VROUTER1/80.start-vm.sh
>
> Regards,
> Andriy
>
> On Fri, May 22, 2015 at 12:04 PM, Maciej Grochowski
> <maciej.grochowski@codilime.com> wrote:
> > I checked this, results below
> >
> > #before script:
> > root@# cat /sys/kernel/mm/hugepages/hugepages-2048kB/free_hugepages
> > 494
> > #after 1 qemu script
> > root@# cat /sys/kernel/mm/hugepages/hugepages-2048kB/free_hugepages
> > 366
> >
> > So qemu consume 262144k~262MB that is correct with script
> >
> > On Fri, May 22, 2015 at 11:58 AM, Tetsuya Mukawa <mukawa@igel.co.jp>
> wrote:
> >
> >> Hi Maciej,
> >>
> >> I guess it's nice to make sure guest memory is actually allocated
> >> by hugepages.
> >> So please check like below.
> >>
> >> $ cat /sys/kernel/mm/hugepage/xxxxx/free_hugepages
> >> $ ./start_qemu.sh
> >> $ cat /sys/kernel/mm/hugepage/xxxxx/free_hugepages
> >>
> >> If qemu guest allocates memory from hugepages, 2nd cat command will
> >> indicate it.
> >>
> >> Thanks,
> >> Tetsuya
> >>
> >>
> >> On 2015/05/22 18:28, Maciej Grochowski wrote:
> >> > "Do you use some command I suggest before, In case of you miss
> >> > the previous mail, just copy it again:"
> >> >
> >> > -Yes but it didn't help me ;/
> >> >
> >> > I will describe step by step to esure that configuration is made
> >> > by
> right
> >> > way
> >> >
> >> >
> >> > I started vhost:
> >> >
> >> > ./build/app/vhost-switch -c f -n 4 --huge-dir /mnt/huge
> >> > --socket-mem
> >> 3712
> >> > -- -p 0x1 --dev-basename usvhost --vm2vm 1 --stats 9
> >> >
> >> > Now I run two vm machines, with followed configuration
> >> >
> >> > VM1 __ __ VM2
> >> > eth0 > \/ > eth0
> >> > eth1 >__/\__> eth1
> >> >
> >> > So I will connect VM1.eth0 with VM2.eth1 and VM1.eth1 with
> >> > VM2.eth0 Because it is test env and I didn't have other network
> >> > connection on
> >> vhost
> >> > I will create two networks 192.168.0.x and 192.168.1.x
> >> > VM1.eth0 with VM2.eth1 will be placed in 192.168.0.x and
> >> > VM1.eth1
> with
> >> > VM2.eth0 in 192.168.1.x
> >> >
> >> > ## I started first VM1 as follow
> >> > kvm -nographic -boot c -machine pc-i440fx-1.4,accel=kvm -name vm2
> >> > -cpu
> >> host
> >> > -smp 1 \
> >> > -hda /home/ubuntu/esi_ee/qemu/debian_min_1.qcow2 -m 256 -mem-path
> >> /mnt/huge
> >> > -mem-prealloc \
> >> > -chardev
> >> > socket,id=char3,path=/home/ubuntu/esi_ee/dpdk/examples/vhost/usvh
> >> > ost \ -netdev type=vhost-user,id=hostnet3,chardev=char3 \ -device
> >> >
> >>
> virtio-net-pci,netdev=hostnet3,id=net3,csum=off,gso=off,guest_tso4=off
> ,guest_tso6=off,guest_ecn=off
> >> > \
> >> > -chardev
> >> > socket,id=char4,path=/home/ubuntu/esi_ee/dpdk/examples/vhost/usvh
> >> > ost \ -netdev type=vhost-user,id=hostnet4,chardev=char4 \ -device
> >> >
> >>
> virtio-net-pci,netdev=hostnet4,id=net4,csum=off,gso=off,guest_tso4=off
> ,guest_tso6=off,guest_ecn=off
> >> > ## qemu give followed output
> >> > qemu-system-x86_64: -netdev type=vhost-user,id=hostnet3,chardev=char3:
> >> > chardev "char3" went up
> >> > qemu-system-x86_64: -netdev type=vhost-user,id=hostnet4,chardev=char4:
> >> > chardev "char4" went up
> >> >
> >> > ## second VM2
> >> > kvm -nographic -boot c -machine pc-i440fx-1.4,accel=kvm -name vm1
> >> > -cpu
> >> host
> >> > -smp 1 \
> >> > -hda /home/ubuntu/esi_ee/qemu/debian_min_2.qcow2 -m 256 -mem-path
> >> /mnt/huge
> >> > -mem-prealloc \
> >> > -chardev
> >> > socket,id=char1,path=/home/ubuntu/esi_ee/dpdk/examples/vhost/usvh
> >> > ost \ -netdev type=vhost-user,id=hostnet1,chardev=char1 \ -device
> >> >
> >>
> virtio-net-pci,netdev=hostnet1,id=net1,csum=off,gso=off,guest_tso4=off
> ,guest_tso6=off,guest_ecn=off
> >> > \
> >> > -chardev
> >> > socket,id=char2,path=/home/ubuntu/esi_ee/dpdk/examples/vhost/usvh
> >> > ost \ -netdev type=vhost-user,id=hostnet2,chardev=char2 \ -device
> >> >
> >>
> virtio-net-pci,netdev=hostnet2,id=net2,csum=off,gso=off,guest_tso4=off
> ,guest_tso6=off,guest_ecn=off
> >> > ## second output
> >> > qemu-system-x86_64: -netdev type=vhost-user,id=hostnet1,chardev=char1:
> >> > chardev "char1" went up
> >> > qemu-system-x86_64: -netdev type=vhost-user,id=hostnet2,chardev=char2:
> >> > chardev "char2" went up
> >> >
> >> >
> >> >
> >> > After that I had MAC conflict between VM2 and VM1
> >> >
> >> > VM1: -ifconfig -a
> >> > eth0 Link encap:Ethernet HWaddr 52:54:00:12:34:56
> >> > inet6 addr: fe80::5054:ff:fe12:3456/64 Scope:Link
> >> > UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
> >> > RX packets:0 errors:0 dropped:0 overruns:0 frame:0
> >> > TX packets:0 errors:0 dropped:0 overruns:0 carrier:0
> >> > collisions:0 txqueuelen:1000
> >> > RX bytes:0 (0.0 B) TX bytes:0 (0.0 B)
> >> >
> >> > eth1 Link encap:Ethernet HWaddr 52:54:00:12:34:57
> >> > BROADCAST MULTICAST MTU:1500 Metric:1
> >> > RX packets:0 errors:0 dropped:0 overruns:0 frame:0
> >> > TX packets:0 errors:0 dropped:0 overruns:0 carrier:0
> >> > collisions:0 txqueuelen:1000
> >> > RX bytes:0 (0.0 B) TX bytes:0 (0.0 B)
> >> >
> >> >
> >> > VM2: -ifconfig -a
> >> > eth0 Link encap:Ethernet HWaddr 52:54:00:12:34:56
> >> > inet6 addr: fe80::5054:ff:fe12:3456/64 Scope:Link
> >> > UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
> >> > RX packets:0 errors:0 dropped:0 overruns:0 frame:0
> >> > TX packets:0 errors:0 dropped:0 overruns:0 carrier:0
> >> > collisions:0 txqueuelen:1000
> >> > RX bytes:0 (0.0 B) TX bytes:0 (0.0 B)
> >> >
> >> > eth1 Link encap:Ethernet HWaddr 52:54:00:12:34:57
> >> > BROADCAST MULTICAST MTU:1500 Metric:1
> >> > RX packets:0 errors:0 dropped:0 overruns:0 frame:0
> >> > TX packets:0 errors:0 dropped:0 overruns:0 carrier:0
> >> > collisions:0 txqueuelen:1000
> >> > RX bytes:0 (0.0 B) TX bytes:0 (0.0 B)
> >> >
> >> > In KNI example I had something similar and also no packet flow
> >> > and
> >> solution
> >> > was to change MAC addresses
> >> >
> >> > #VM1
> >> > /etc/init.d/networking stop
> >> > ifconfig eth0 hw ether 00:01:04:00:01:00 ifconfig eth1 hw ether
> >> > 00:01:04:00:01:01 /etc/init.d/networking start ifconfig eth0
> >> > ifconfig eth1
> >> >
> >> > #VM2
> >> > /etc/init.d/networking stop
> >> > ifconfig eth0 hw ether 00:01:04:00:02:00
> >> > ifconfig eth1 hw ether 00:01:04:00:02:01
> >> > /etc/init.d/networking start
> >> > ifconfig eth0
> >> > ifconfig eth1
> >> >
> >> > Then I make a configuration that You show:
> >> >
> >> > #VM1
> >> > ip addr add 192.168.0.100/24 dev eth0
> >> > ip addr add 192.168.1.100/24 dev eth1
> >> > ip neigh add 192.168.0.200 lladdr 00:01:04:00:02:01 dev eth0
> >> > ip link set dev eth0 up
> >> > ip neigh add 192.168.1.200 lladdr 00:01:04:00:02:00 dev eth1
> >> > ip link set dev eth1 up
> >> >
> >> > eth0 Link encap:Ethernet HWaddr 00:01:04:00:01:00
> >> > inet addr:192.168.0.100 Bcast:0.0.0.0 Mask:255.255.255.0
> >> > inet6 addr: fe80::201:4ff:fe00:100/64 Scope:Link
> >> > UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
> >> > RX packets:0 errors:0 dropped:0 overruns:0 frame:0
> >> > TX packets:0 errors:0 dropped:0 overruns:0 carrier:0
> >> > collisions:0 txqueuelen:1000
> >> > RX bytes:0 (0.0 B) TX bytes:0 (0.0 B)
> >> >
> >> > eth1 Link encap:Ethernet HWaddr 00:01:04:00:01:01
> >> > inet addr:192.168.1.100 Bcast:0.0.0.0 Mask:255.255.255.0
> >> > inet6 addr: fe80::201:4ff:fe00:101/64 Scope:Link
> >> > UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
> >> > RX packets:0 errors:0 dropped:0 overruns:0 frame:0
> >> > TX packets:0 errors:0 dropped:0 overruns:0 carrier:0
> >> > collisions:0 txqueuelen:1000
> >> > RX bytes:0 (0.0 B) TX bytes:0 (0.0 B)
> >> >
> >> >
> >> > #VM2
> >> > ip addr add 192.168.1.200/24 dev eth0
> >> > ip addr add 192.168.0.200/24 dev eth1
> >> > ip neigh add 192.168.1.100 lladdr 00:01:04:00:01:01 dev eth0
> >> > ip link set dev eth0 up
> >> > ip neigh add 192.168.0.100 lladdr 00:01:04:00:01:00 dev eth1
> >> > ip link set dev eth1 up
> >> >
> >> > eth0 Link encap:Ethernet HWaddr 00:01:04:00:02:00
> >> > inet addr:192.168.1.200 Bcast:0.0.0.0 Mask:255.255.255.0
> >> > inet6 addr: fe80::201:4ff:fe00:200/64 Scope:Link
> >> > UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
> >> > RX packets:0 errors:0 dropped:0 overruns:0 frame:0
> >> > TX packets:0 errors:0 dropped:0 overruns:0 carrier:0
> >> > collisions:0 txqueuelen:1000
> >> > RX bytes:0 (0.0 B) TX bytes:0 (0.0 B)
> >> >
> >> > eth1 Link encap:Ethernet HWaddr 00:01:04:00:02:01
> >> > inet addr:192.168.0.200 Bcast:0.0.0.0 Mask:255.255.255.0
> >> > inet6 addr: fe80::201:4ff:fe00:201/64 Scope:Link
> >> > UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
> >> > RX packets:0 errors:0 dropped:0 overruns:0 frame:0
> >> > TX packets:0 errors:0 dropped:0 overruns:0 carrier:0
> >> > collisions:0 txqueuelen:1000
> >> > RX bytes:0 (0.0 B) TX bytes:0 (0.0 B)
> >> >
> >> > After that:
> >> >
> >> > VM1.eth0 ip=192.168.0.100-MAC=00:01:04:00:01:00 is connected to
> VM2.eth1
> >> > ip=192.168.0.200-MAC=00:01:04:00:02:01
> >> > VM1.eth1 ip=192.168.1.100-MAC=00:01:04:00:01:01 is connected to
> VM2.eth0
> >> > ip=192.168.1.200-MAC=00:01:04:00:02:00
> >> >
> >> > That show my arp tables:
> >> >
> >> > #VM1
> >> > arp -a
> >> > ? (192.168.0.200) at 00:01:04:00:02:01 [ether] PERM on eth0
> >> > ? (192.168.1.200) at 00:01:04:00:02:00 [ether] PERM on eth1
> >> >
> >> >
> >> > #VM2
> >> > arp -a
> >> > ? (192.168.0.100) at 00:01:04:00:01:00 [ether] PERM on eth1
> >> > ? (192.168.1.100) at 00:01:04:00:01:01 [ether] PERM on eth0
> >> >
> >> >
> >> > #After this configuration I trying to ping from VM1 VM2 (both IP)
> >> >
> >> > root@debian-amd64:~# ping -I eth0 192.168.0.200
> >> > PING 192.168.0.200 (192.168.0.200) from 192.168.0.100 eth0: 56(84)
> bytes
> >> of
> >> > data.
> >> > ^C
> >> > --- 192.168.0.200 ping statistics ---
> >> > 5 packets transmitted, 0 received, 100% packet loss, time 4032ms
> >> >
> >> > root@debian-amd64:~# ping 192.168.0.200
> >> > PING 192.168.0.200 (192.168.0.200) 56(84) bytes of data.
> >> > ^C
> >> > --- 192.168.0.200 ping statistics ---
> >> > 5 packets transmitted, 0 received, 100% packet loss, time 3999ms
> >> >
> >> > root@debian-amd64:~# ping -I eth1 192.168.1.200
> >> > PING 192.168.1.200 (192.168.1.200) from 192.168.1.100 eth1: 56(84)
> bytes
> >> of
> >> > data.
> >> > ^C
> >> > --- 192.168.1.200 ping statistics ---
> >> > 6 packets transmitted, 0 received, 100% packet loss, time 5040ms
> >> >
> >> > root@debian-amd64:~# ping 192.168.1.200
> >> > PING 192.168.1.200 (192.168.1.200) 56(84) bytes of data.
> >> > ^C
> >> > --- 192.168.1.200 ping statistics ---
> >> > 5 packets transmitted, 0 received, 100% packet loss, time 4030ms
> >> >
> >> > #and from VM2 VM1
> >> > root@debian-amd64:~# ping 192.168.0.100
> >> > PING 192.168.0.100 (192.168.0.100) 56(84) bytes of data.
> >> > ^C
> >> > --- 192.168.0.100 ping statistics ---
> >> > 3 packets transmitted, 0 received, 100% packet loss, time 2015ms
> >> >
> >> > root@debian-amd64:~# ping -I eth1 192.168.0.100
> >> > PING 192.168.0.100 (192.168.0.100) from 192.168.0.200 eth1: 56(84)
> bytes
> >> of
> >> > data.
> >> > ^C
> >> > --- 192.168.0.100 ping statistics ---
> >> > 5 packets transmitted, 0 received, 100% packet loss, time 4032ms
> >> >
> >> > root@debian-amd64:~# ping -I eth0 192.168.1.100
> >> > PING 192.168.1.100 (192.168.1.100) from 192.168.1.200 eth0: 56(84)
> bytes
> >> of
> >> > data.
> >> > ^C
> >> > --- 192.168.1.100 ping statistics ---
> >> > 4 packets transmitted, 0 received, 100% packet loss, time 3024ms
> >> >
> >> > root@debian-amd64:~# ping 192.168.1.100
> >> > PING 192.168.1.100 (192.168.1.100) 56(84) bytes of data.
> >> > ^C
> >> > --- 192.168.1.100 ping statistics ---
> >> > 8 packets transmitted, 0 received, 100% packet loss, time 7055ms
> >> >
> >> >
> >> > Also stats from vhost:
> >> > Device statistics ====================================
> >> > Statistics for device 0 ------------------------------
> >> > TX total: 0
> >> > TX dropped: 0
> >> > TX successful: 0
> >> > RX total: 0
> >> > RX dropped: 0
> >> > RX successful: 0
> >> > Statistics for device 1 ------------------------------
> >> > TX total: 0
> >> > TX dropped: 0
> >> > TX successful: 0
> >> > RX total: 0
> >> > RX dropped: 0
> >> > RX successful: 0
> >> > Statistics for device 2 ------------------------------
> >> > TX total: 0
> >> > TX dropped: 0
> >> > TX successful: 0
> >> > RX total: 0
> >> > RX dropped: 0
> >> > RX successful: 0
> >> > Statistics for device 3 ------------------------------
> >> > TX total: 0
> >> > TX dropped: 0
> >> > TX successful: 0
> >> > RX total: 0
> >> > RX dropped: 0
> >> > RX successful: 0
> >> > ======================================================
> >> >
> >> > My way of thinking was: "In vhost there are several function for L2
> that
> >> > learn MAC-s and links them so why I see no received packets?"
> >> >
> >> > Maybe I'm doing some silly bug in network configuration but for me its
> >> > looking like data flow issue especially that no function on the vhost
> >> side
> >> > did not see any packages.
> >> >
> >> > On Fri, May 22, 2015 at 10:54 AM, Gaohaifeng (A) <
> >> gaohaifeng.gao@huawei.com>
> >> > wrote:
> >> >
> >> >> Hi
> >> >>
> >> >> What kernel version are You using on host/guest?
> >> >>
> >> >>>> I use ubuntu 12.04(3.11.0-15-generic) in host. In vm I use ubuntu
> >> 12.04
> >> >> and ubuntu14.04 both, but the result is same.
> >> >>
> >> >>
> >> >>
> >> >> Do you use some command I suggest before, In case of you miss the
> >> previous
> >> >> mail, just copy it again:
> >> >>
> >> >>>> I try it but the result is same
> >> >>
> >> >>
> >> >>
> >> >>
> >> >> I use l2fwd in vm to do more test and found that virtio_xmit_pkts is
> >> >> called and avail_idx is increasing in vm, but in host avail_idx(in
> >> >> rte_vhost_dequeue_burst function) is always zero. It seems that the
> host
> >> >> see the different mem area.
> >> >>
> >> >>
> >> >>
> >> >> Init Logs below:
> >> >>
> >> >> VHOST_CONFIG: (0) Mergeable RX buffers disabled
> >> >>
> >> >> VHOST_CONFIG: read message VHOST_USER_SET_MEM_TABLE
> >> >>
> >> >> VHOST_CONFIG: mapped region 0 fd:24 to 0x2aaaaac00000 sz:0xa0000
> off:0x0
> >> >>
> >> >> VHOST_CONFIG: REGION: 0 GPA: (nil) QEMU VA: 0x2aaaaac00000 SIZE
> (655360)
> >> >>
> >> >> VHOST_CONFIG: mapped region 1 fd:26 to 0x2aaaaae00000 sz:0x40000000
> >> >> off:0xc0000
> >> >>
> >> >> VHOST_CONFIG: REGION: 1 GPA: 0xc0000 QEMU VA: 0x2aaaaacc0000 SIZE
> >> >> (1072955392)
> >> >>
> >> >> VHOST_CONFIG: read message VHOST_USER_SET_VRING_NUM
> >> >>
> >> >> VHOST_CONFIG: read message VHOST_USER_SET_VRING_BASE
> >> >>
> >> >> VHOST_CONFIG: read message VHOST_USER_SET_VRING_ADDR
> >> >>
> >> >> VHOST_CONFIG: (0) mapped address desc: 0x2aaae62f1000
> >> >>
> >> >> VHOST_CONFIG: (0) mapped address avail: 0x2aaae62f2000
> >> >>
> >> >> VHOST_CONFIG: (0) mapped address used: 0x2aaae62f3000
> >> >>
> >> >> VHOST_CONFIG: read message VHOST_USER_SET_VRING_KICK
> >> >>
> >> >> VHOST_CONFIG: vring kick idx:0 file:23
> >> >>
> >> >> VHOST_CONFIG: virtio isn't ready for processing.
> >> >>
> >> >> VHOST_CONFIG: read message VHOST_USER_SET_VRING_NUM
> >> >>
> >> >> VHOST_CONFIG: read message VHOST_USER_SET_VRING_BASE
> >> >>
> >> >> VHOST_CONFIG: read message VHOST_USER_SET_VRING_ADDR
> >> >>
> >> >> VHOST_CONFIG: (0) mapped address desc: 0x2aaae62f4000
> >> >>
> >> >> VHOST_CONFIG: (0) mapped address avail: 0x2aaae62f5000
> >> >>
> >> >> VHOST_CONFIG: (0) mapped address used: 0x2aaae62f6000
> >> >>
> >> >> VHOST_CONFIG: read message VHOST_USER_SET_VRING_KICK
> >> >>
> >> >> VHOST_CONFIG: vring kick idx:1 file:28
> >> >>
> >> >> VHOST_CONFIG: virtio is now ready for processing.
> >> >>
> >> >>
> >> >>
> >> >>
> >> >>
> >> >>> Unfortunately not, I have the same issue in rte_vhost_dequeue_burst
> >> >> function.
> >> >>
> >> >>
> >> >>
> >> >>> What kernel version are You using on host/guest? In my case on host
> I
> >> >> had 3.13.0 and on guests old 3.2 debian.
> >> >>
> >> >>
> >> >>
> >> >>> I just looked deeper into virtio back-end (vhost) but at first
> glace
> >> it
> >> >> seems like nothing coming from virtio.
> >> >>
> >> >>
> >> >>
> >> >>> What I'm going to do today is to compile newest kernel for vhost and
> >> >> guest and debug where packet flow stuck, I will report the result
> >> >>
> >> >>
> >> >>
> >> >>> On Thu, May 21, 2015 at 11:12 AM, Gaohaifeng (A) <
> >> >> gaohaifeng.gao@huawei.com> wrote:
> >> >>
> >> >>> Hi Maciej
> >> >> >Did you solve your problem? I meet this problem as your
> case.
> >> >> And I found avail_idx(in rte_vhost_dequeue_burst function) is always
> >> zero
> >> >> although I do send packets in VM.
> >> >>
> >> >>> Thanks.
> >> >>
> >> >>
> >> >>> Hello, I have strange issue with example/vhost app.
> >> >>>
> >> >>> I had compiled DPDK to run a vhost example app with followed flags
> >> >>>
> >> >>> CONFIG_RTE_LIBRTE_VHOST=y
> >> >>> CONFIG_RTE_LIBRTE_VHOST_USER=y
> >> >>> CONFIG_RTE_LIBRTE_VHOST_DEBUG=n
> >> >>>
> >> >>> then I run vhost app based on documentation:
> >> >>>
> >> >>> ./build/app/vhost-switch -c f -n 4 --huge-dir /mnt/huge
> --socket-mem
> >> >>> 3712
> >> >>> -- -p 0x1 --dev-basename usvhost --vm2vm 1 --stats 9
> >> >>>
> >> >>> -I use this strange --socket-mem 3712 because of physical limit of
> >> >>> memoryon device -with this vhost user I run two KVM machines with
> >> >>> followed parameters
> >> >>>
> >> >>> kvm -nographic -boot c -machine pc-i440fx-1.4,accel=kvm -name vm1
> -cpu
> >> >>> host -smp 2 -hda /home/ubuntu/qemu/debian_squeeze2_amd64.qcow2 -m
> >> >>> 1024 -mem-path /mnt/huge -mem-prealloc -chardev
> >> >>> socket,id=char1,path=/home/ubuntu/dpdk/examples/vhost/usvhost
> >> >>> -netdev type=vhost-user,id=hostnet1,chardev=char1
> >> >>> -device virtio-net
> >> >>>
> pci,netdev=hostnet1,id=net1,csum=off,gso=off,guest_tso4=off,guest_tso6
> >> >>> =
> >> >>> off,guest_ecn=off
> >> >>> -chardev
> >> >>> socket,id=char2,path=/home/ubuntu/dpdk/examples/vhost/usvhost
> >> >>> -netdev type=vhost-user,id=hostnet2,chardev=char2
> >> >>> -device
> >> >>> virtio-net-
> >> >>>
> pci,netdev=hostnet2,id=net2,csum=off,gso=off,guest_tso4=off,guest_tso6
> >> >>> =
> >> >>> off,guest_ecn=off
> >> >>>
> >> >>> After running KVM virtio correctly starting (below logs from vhost
> app)
> >> >> ...
> >> >>> VHOST_CONFIG: mapped region 0 fd:31 to 0x2aaabae00000 sz:0xa0000
> >> >>> off:0x0
> >> >>> VHOST_CONFIG: mapped region 1 fd:37 to 0x2aaabb000000 sz:0x10000000
> >> >>> off:0xc0000
> >> >>> VHOST_CONFIG: read message VHOST_USER_SET_VRING_NUM
> >> >>> VHOST_CONFIG: read message VHOST_USER_SET_VRING_BASE
> >> >>> VHOST_CONFIG: read message VHOST_USER_SET_VRING_ADDR
> >> >>> VHOST_CONFIG: read message VHOST_USER_SET_VRING_KICK
> >> >>> VHOST_CONFIG: vring kick idx:0 file:38
> >> >>> VHOST_CONFIG: virtio isn't ready for processing.
> >> >>> VHOST_CONFIG: read message VHOST_USER_SET_VRING_NUM
> >> >>> VHOST_CONFIG: read message VHOST_USER_SET_VRING_BASE
> >> >>> VHOST_CONFIG: read message VHOST_USER_SET_VRING_ADDR
> >> >>> VHOST_CONFIG: read message VHOST_USER_SET_VRING_KICK
> >> >>> VHOST_CONFIG: vring kick idx:1 file:39
> >> >>> VHOST_CONFIG: virtio is now ready for processing.
> >> >>> VHOST_DATA: (1) Device has been added to data core 2
> >> >>>
> >> >>> So everything looking good.
> >> >>>
> >> >>> Maybe it is something trivial but using options: --vm2vm 1 (or) 2
> >> >>> --stats 9 it seems that I didn't have connection between VM2VM
> >> >>> communication. I set manually IP for eth0 and eth1:
> >> >>>
> >> >>> on 1 VM
> >> >>> ifconfig eth0 192.168.0.100 netmask 255.255.255.0 up ifconfig eth1
> >> >>> 192.168.1.101 netmask 255.255.255.0 up
> >> >>>
> >> >>> on 2 VM
> >> >>> ifconfig eth0 192.168.1.200 netmask 255.255.255.0 up ifconfig eth1
> >> >>> 192.168.0.202 netmask 255.255.255.0 up
> >> >>>
> >> >>> I notice that in vhostapp are one directional rx/tx queue so I
> tryied
> >> >>> to ping between VM1 to VM2 using both interfaces ping -I eth0
> >> >>> 192.168.1.200 ping -I
> >> >>> eth1 192.168.1.200 ping -I eth0 192.168.0.202 ping -I eth1
> >> >>> 192.168.0.202
> >> >>>
> >> >>> on VM2 using tcpdump on both interfaces I didn't see any ICMP
> requests
> >> >>> or traffic
> >> >>>
> >> >>> And I cant ping between any IP/interfaces, moreover stats show me
> that:
> >> >>>
> >> >>> Device statistics ====================================
> >> >>> Statistics for device 0 ------------------------------
> >> >>> TX total: 0
> >> >>> TX dropped: 0
> >> >>> TX successful: 0
> >> >>> RX total: 0
> >> >>> RX dropped: 0
> >> >>> RX successful: 0
> >> >>> Statistics for device 1 ------------------------------
> >> >>> TX total: 0
> >> >>> TX dropped: 0
> >> >>> TX successful: 0
> >> >>> RX total: 0
> >> >>> RX dropped: 0
> >> >>> RX successful: 0
> >> >>> Statistics for device 2 ------------------------------
> >> >>> TX total: 0
> >> >>> TX dropped: 0
> >> >>> TX successful: 0
> >> >>> RX total: 0
> >> >>> RX dropped: 0
> >> >>> RX successful: 0
> >> >>> Statistics for device 3 ------------------------------
> >> >>> TX total: 0
> >> >>> TX dropped: 0
> >> >>> TX successful: 0
> >> >>> RX total: 0
> >> >>> RX dropped: 0
> >> >>> RX successful: 0
> >> >>> ======================================================
> >> >>>
> >> >>> So it seems like any packet didn't leave my VM.
> >> >>> also arp table is empty on each VM.
> >> >>
> >> >>
> >>
> >>
>
>
>
> --
> Andriy Berestovskyy
>
next prev parent reply other threads:[~2015-05-25 4:16 UTC|newest]
Thread overview: 13+ messages / expand[flat|nested] mbox.gz Atom feed top
2015-05-15 10:15 Maciej Grochowski
2015-05-17 14:41 ` Ouyang, Changchun
2015-05-21 9:12 ` [dpdk-dev] FW: " Gaohaifeng (A)
2015-05-22 8:05 ` Maciej Grochowski
2015-05-22 8:26 ` Ouyang, Changchun
2015-05-22 8:54 ` [dpdk-dev] " Gaohaifeng (A)
2015-05-22 9:28 ` Maciej Grochowski
2015-05-22 9:58 ` Tetsuya Mukawa
2015-05-22 10:04 ` Maciej Grochowski
2015-05-22 10:54 ` Andriy Berestovskyy
2015-05-22 17:59 ` Maciej Grochowski
2015-05-25 4:15 ` Gaohaifeng (A) [this message]
2015-05-22 9:27 ` [dpdk-dev] FW: " Luke Gorrie
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=47CEA9C0E570484FBF22EF0D7EBCE5B534AC5DE2@szxema505-mbs.china.huawei.com \
--to=gaohaifeng.gao@huawei.com \
--cc=aber@semihalf.com \
--cc=dev@dpdk.org \
--cc=maciej.grochowski@codilime.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).