[dpdk-dev] Questions about vm2vm vhost-user/virtio-net test

[dpdk-dev] Questions about vm2vm vhost-user/virtio-net test

[Min Hu (Connor)]

Hi, Coquelin, Chenbo, all,
	I  want to seek help about vm2vm vhost-user/virtio-net test from
you.
	When I set up vm2vm vhost-user/virtio-net test, I cannot ping
vm2 in vm1. That is, ping failed between vm1 and vm2.

	Detailed description are as flows:
host configuration:
Linux localhost 5.14.0-rc4+ #1 SMP PREEMPT Sun Sep 26 16:52:13 CST 2021 
aarch64 aarch64 aarch64 GNU/Linux

1, set up vhost on host
cd /home/humin/virtio_test
rm -rf ./vhost-net*

/usr/app/testpmd -l 2-4 -n 4 --no-pci --file-prefix=vhost --vdev 
'net_vhost0,iface=vhost-net0,queues=1' --vdev 
'net_vhost1,iface=vhost-net1,queues=1'  -- -i --nb-cores=2 --txd=1024 
--rxd=1024

testpmd> set fwd mac
testpmd> start

2, create vm1 and vm2
taskset -c 13 qemu-system-aarch64 \
-name us-vhost-vm1 \
-kernel /home/humin/virtio_test/Image \
-initrd /home/humin/virtio_test/rootfs.cpio.gz \
-machine virt,gic-version=3 -nographic \
-cpu host -enable-kvm -m 4096 \
-object 
memory-backend-file,id=mem,size=4096M,mem-path=/dev/hugepages,share=on \
-numa node,memdev=mem \
-mem-prealloc \
-smp cores=4,sockets=1 \
-monitor unix:/tmp/vm2_monitor.sock,server,nowait \
-net user,hostfwd=tcp:127.0.0.1:6004-:22 \
-chardev socket,id=char0,path=/home/humin/virtio_test/vhost-net0 \
-netdev type=vhost-user,id=mynet1,chardev=char0,vhostforce \
-device 
virtio-net-pci,mac=52:54:00:00:00:01,netdev=mynet1,mrg_rxbuf=on,csum=on,guest_csum=on,guest_ecn=on 
\
-vnc :15


taskset -c 15 qemu-system-aarch64 \
-name us-vhost-vm2 \
-kernel /home/humin/virtio_test/Image \
-initrd /home/humin/virtio_test/rootfs.cpio.gz \
-machine virt,gic-version=3 -nographic \
-cpu host -enable-kvm -m 4096 \
-object 
memory-backend-file,id=mem,size=4096M,mem-path=/dev/hugepages,share=on \
-numa node,memdev=mem \
-mem-prealloc \
-smp cores=4,sockets=1 \
-monitor unix:/tmp/vm2_monitor.sock,server,nowait \
-net user,hostfwd=tcp:127.0.0.1:6005-:22 \
-chardev socket,id=char1,path=/home/humin/virtio_test/vhost-net1 \
-netdev type=vhost-user,id=mynet2,chardev=char1,vhostforce \
-device 
virtio-net-pci,mac=52:54:00:00:00:02,netdev=mynet2,mrg_rxbuf=on,csum=on,guest_csum=on,guest_ecn=on 
\
-vnc :16


3, do test
in vm1:
ifconfig eth0 up
ifconfig eth0 1.1.1.2
arp -s 1.1.1.8 52:54:00:00:00:02

in vm2:
ifconfig eth0 up
ifconfig eth0 1.1.1.8
arp -s 1.1.1.2 52:54:00:00:00:01

Then in vm1:
ping 1.1.1.8

the result:
no icmp reply, ping failed. same for vm2.

4, Try to debug
using tcpdump to capture packets in vm2,  we can get only the ICMP
request packet which is from vm1, but no ICMP reply.
It looks like ping task does not send ICMP reply, but it is just guess
for it.

BTW, I also use iperf test:
UDP packet test:
vm1:
iperf -s  -i 1

vm2:
iperf -c 1.1.1.2 -u -i 1 -t 60

Then vm1 can get the packet. Same result if vm1 is used as client.

TCP packet test:
vm1:
iperf -s  -i 1

vm2:
iperf -c 1.1.1.2 -i 1 -t 60

Then vm1 can NOT get the packet. Same result if vm1 is used as client.

5, reference
I refre to the website
https://doc.dpdk.org/dts/test_plans/vm2vm_virtio_net_perf_test_plan.html

In 199.7. Test Case 5:
scp test success.

But my scp test also failed, any one could help me?  thanks very much.

Re: [dpdk-dev] Questions about vm2vm vhost-user/virtio-net test

[Min Hu (Connor)]

Hi, all,
	I got the answer: testpmd fwd mode should be set "io", then ping
OK.

	IO fwd mode, it will not change packet MAC address info, and
ping OK
	MAC fwd mode, it will change packet MAC address info, like:
	"rte_ether_addr_copy(&peer_eth_addrs[fs->peer_addr],
				&eth_hdr->d_addr);
	rte_ether_addr_copy(&ports[fs->tx_port].eth_addr,
				&eth_hdr->s_addr)"
	Then, ping failed.

	So, everyone, I got one question: how could this happen?
	router change packet src MAC and dst MAC, but ping OK,
	testpmd vhost, treated as switch, also changes packert src MAC
	and dst MAC, but ping failed ?	


在 2021/9/29 20:09, Min Hu (Connor) 写道:
> Hi, Coquelin, Chenbo, all,
>      I  want to seek help about vm2vm vhost-user/virtio-net test from
> you.
>      When I set up vm2vm vhost-user/virtio-net test, I cannot ping
> vm2 in vm1. That is, ping failed between vm1 and vm2.
> 
>      Detailed description are as flows:
> host configuration:
> Linux localhost 5.14.0-rc4+ #1 SMP PREEMPT Sun Sep 26 16:52:13 CST 2021 
> aarch64 aarch64 aarch64 GNU/Linux
> 
> 1, set up vhost on host
> cd /home/humin/virtio_test
> rm -rf ./vhost-net*
> 
> /usr/app/testpmd -l 2-4 -n 4 --no-pci --file-prefix=vhost --vdev 
> 'net_vhost0,iface=vhost-net0,queues=1' --vdev 
> 'net_vhost1,iface=vhost-net1,queues=1'  -- -i --nb-cores=2 --txd=1024 
> --rxd=1024
> 
> testpmd> set fwd mac
> testpmd> start
> 
> 2, create vm1 and vm2
> taskset -c 13 qemu-system-aarch64 \
> -name us-vhost-vm1 \
> -kernel /home/humin/virtio_test/Image \
> -initrd /home/humin/virtio_test/rootfs.cpio.gz \
> -machine virt,gic-version=3 -nographic \
> -cpu host -enable-kvm -m 4096 \
> -object 
> memory-backend-file,id=mem,size=4096M,mem-path=/dev/hugepages,share=on \
> -numa node,memdev=mem \
> -mem-prealloc \
> -smp cores=4,sockets=1 \
> -monitor unix:/tmp/vm2_monitor.sock,server,nowait \
> -net user,hostfwd=tcp:127.0.0.1:6004-:22 \
> -chardev socket,id=char0,path=/home/humin/virtio_test/vhost-net0 \
> -netdev type=vhost-user,id=mynet1,chardev=char0,vhostforce \
> -device 
> virtio-net-pci,mac=52:54:00:00:00:01,netdev=mynet1,mrg_rxbuf=on,csum=on,guest_csum=on,guest_ecn=on 
> \
> -vnc :15
> 
> 
> taskset -c 15 qemu-system-aarch64 \
> -name us-vhost-vm2 \
> -kernel /home/humin/virtio_test/Image \
> -initrd /home/humin/virtio_test/rootfs.cpio.gz \
> -machine virt,gic-version=3 -nographic \
> -cpu host -enable-kvm -m 4096 \
> -object 
> memory-backend-file,id=mem,size=4096M,mem-path=/dev/hugepages,share=on \
> -numa node,memdev=mem \
> -mem-prealloc \
> -smp cores=4,sockets=1 \
> -monitor unix:/tmp/vm2_monitor.sock,server,nowait \
> -net user,hostfwd=tcp:127.0.0.1:6005-:22 \
> -chardev socket,id=char1,path=/home/humin/virtio_test/vhost-net1 \
> -netdev type=vhost-user,id=mynet2,chardev=char1,vhostforce \
> -device 
> virtio-net-pci,mac=52:54:00:00:00:02,netdev=mynet2,mrg_rxbuf=on,csum=on,guest_csum=on,guest_ecn=on 
> \
> -vnc :16
> 
> 
> 3, do test
> in vm1:
> ifconfig eth0 up
> ifconfig eth0 1.1.1.2
> arp -s 1.1.1.8 52:54:00:00:00:02
> 
> in vm2:
> ifconfig eth0 up
> ifconfig eth0 1.1.1.8
> arp -s 1.1.1.2 52:54:00:00:00:01
> 
> Then in vm1:
> ping 1.1.1.8
> 
> the result:
> no icmp reply, ping failed. same for vm2.
> 
> 4, Try to debug
> using tcpdump to capture packets in vm2,  we can get only the ICMP
> request packet which is from vm1, but no ICMP reply.
> It looks like ping task does not send ICMP reply, but it is just guess
> for it.
> 
> BTW, I also use iperf test:
> UDP packet test:
> vm1:
> iperf -s  -i 1
> 
> vm2:
> iperf -c 1.1.1.2 -u -i 1 -t 60
> 
> Then vm1 can get the packet. Same result if vm1 is used as client.
> 
> TCP packet test:
> vm1:
> iperf -s  -i 1
> 
> vm2:
> iperf -c 1.1.1.2 -i 1 -t 60
> 
> Then vm1 can NOT get the packet. Same result if vm1 is used as client.
> 
> 5, reference
> I refre to the website
> https://doc.dpdk.org/dts/test_plans/vm2vm_virtio_net_perf_test_plan.html
> 
> In 199.7. Test Case 5:
> scp test success.
> 
> But my scp test also failed, any one could help me?  thanks very much.
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 
> .

Re: [dpdk-dev] Questions about vm2vm vhost-user/virtio-net test

[David Christensen]

On 9/30/21 2:07 AM, Min Hu (Connor) wrote:
> Hi, all,
>      I got the answer: testpmd fwd mode should be set "io", then ping
> OK.
> 
>      IO fwd mode, it will not change packet MAC address info, and
> ping OK
>      MAC fwd mode, it will change packet MAC address info, like:
>      "rte_ether_addr_copy(&peer_eth_addrs[fs->peer_addr],
>                  &eth_hdr->d_addr);
>      rte_ether_addr_copy(&ports[fs->tx_port].eth_addr,
>                  &eth_hdr->s_addr)"
>      Then, ping failed.
> 
>      So, everyone, I got one question: how could this happen?
>      router change packet src MAC and dst MAC, but ping OK,
>      testpmd vhost, treated as switch, also changes packert src MAC
>      and dst MAC, but ping failed ?

Running testpmd in this configuration is more like a virtual wire, not a 
switch, though in both cases the SRC/DST MAC addresses aren't typically 
modified by the wire/switch when the frame is in transit. Both endpoints 
reside in the same Ethernet broadcast domain and can talk to each other 
directly.

A router typically connects two different broadcast domains, which may 
or may not be using Ethernet, so the router usually changes the SRC/DST 
MAC address to work correctly as the frame transitions from one 
broadcast domain into another. See 
https://en.wikipedia.org/wiki/Broadcast_domain.

Dave

Re: [dpdk-dev] Questions about vm2vm vhost-user/virtio-net test

[Min Hu (Connor)]

Hi, David, thanks for your reply,
  other question as below,

在 2021/10/1 2:25, David Christensen 写道:
> 
> 
> On 9/30/21 2:07 AM, Min Hu (Connor) wrote:
>> Hi, all,
>>      I got the answer: testpmd fwd mode should be set "io", then ping
>> OK.
>>
>>      IO fwd mode, it will not change packet MAC address info, and
>> ping OK
>>      MAC fwd mode, it will change packet MAC address info, like:
>>      "rte_ether_addr_copy(&peer_eth_addrs[fs->peer_addr],
>>                  &eth_hdr->d_addr);
>>      rte_ether_addr_copy(&ports[fs->tx_port].eth_addr,
>>                  &eth_hdr->s_addr)"
>>      Then, ping failed.
>>
>>      So, everyone, I got one question: how could this happen?
>>      router change packet src MAC and dst MAC, but ping OK,
>>      testpmd vhost, treated as switch, also changes packert src MAC
>>      and dst MAC, but ping failed ?
> 
> Running testpmd in this configuration is more like a virtual wire, not a 
> switch, though in both cases the SRC/DST MAC addresses aren't typically 
Well, if SRC/DST MAC addresses modifed in some case, what will happen?
That is, A send ping request to B， while SRC/DST MAC addresses
modified，B can get the packet, but B cannot send ping reply.
This is the result of my test. Why?


> modified by the wire/switch when the frame is in transit. Both endpoints 
> reside in the same Ethernet broadcast domain and can talk to each other 
> directly.
> 
> A router typically connects two different broadcast domains, which may 
> or may not be using Ethernet, so the router usually changes the SRC/DST 
> MAC address to work correctly as the frame transitions from one 
> broadcast domain into another. See 
> https://en.wikipedia.org/wiki/Broadcast_domain.
> 
> Dave
> .