Re: [dts] [DTS][PATCH 2/3] virtio: update vhost user cases

["Fu, JingguoX" <jingguox.fu@intel.com>]

Update by version 2 patches

Thanks,
Jingguo Fu

-----Original Message-----
From: Xu, Qian Q 
Sent: Friday, January 22, 2016 9:55 AM
To: Fu, JingguoX; dts@dpdk.org
Cc: Liu, Yong; Xu, Qian Q
Subject: RE: [DTS][PATCH 2/3] virtio: update vhost user cases

Jingguo
I'm fine with item1 and item3, but as to item2, the logic is correct and it's similar as we test PF performance, all nic port will RX ixia traffic and TX the traffic back to IXIA, when testing performance, we default take traffic is 100% Linerate, and the traffic return to IXIA is called throughput, since virtio/NIC port will do RX and TX, so if you say it's TX, it's not accurate, so we can say it's throughput which will not cause confusion. Does it make sense? 

Thanks
Qian


-----Original Message-----
From: Fu, JingguoX
Sent: Thursday, January 21, 2016 5:03 PM
To: Xu, Qian Q; dts@dpdk.org
Cc: Liu, Yong
Subject: RE: [DTS][PATCH 2/3] virtio: update vhost user cases

Hi, qian

For self.dut.send_expect("rm -rf ./vhost-net", "#", 20), I found that sometimes it could not remove when the case failed at some different stages, so  I put it at set_up()  stage for each case, to make sure that when a new case start, it would not affect by the vhost-net from the prior case.

For recvpkt and send pkt, I think the logic as bellows:
For ixia, it will send packet to a virtio device, then the first virtio device will receive packets, and it will forward the packets to the second virtio device. At last, the second virtio will send out the packets it received, then from IXIA end, we see the receive rate. 
This logic based on the discussion with Ouyang Changchun originally. If we need to re-define it , maybe we can set a quickly discussion for it.

And at last, for the data table, originally we put the mode and case name to the table, we receive some comments for remove them. So now, it will disappear from the table. For my part, I don't think it necessary to put them into the table. Every  case has its own table, and it seems too much redundant info if put them in the table.  


Thanks,
Jingguo Fu
-----Original Message-----
From: Xu, Qian Q
Sent: Thursday, January 21, 2016 3:58 PM
To: Fu, JingguoX; dts@dpdk.org
Cc: Liu, Yong
Subject: RE: [DTS][PATCH 2/3] virtio: update vhost user cases

See my comments below. 

Thanks
Qian


-----Original Message-----
From: Fu, JingguoX
Sent: Thursday, January 21, 2016 2:43 PM
To: dts@dpdk.org
Cc: Liu, Yong; Xu, Qian Q; Fu, JingguoX
Subject: [DTS][PATCH 2/3] virtio: update vhost user cases

    1: optimize case
    2: add get rx rate for generator
    3: better way to check vhost.out file

Signed-off-by: Jingguo Fu <jingguox.fu@intel.com>
---
 tests/TestSuite_vhost_user_one_copy_one_vm.py | 202 ++++++++++++--------------
 1 file changed, 94 insertions(+), 108 deletions(-)

diff --git a/tests/TestSuite_vhost_user_one_copy_one_vm.py b/tests/TestSuite_vhost_user_one_copy_one_vm.py
index 32e47ad..971d9c2 100644
--- a/tests/TestSuite_vhost_user_one_copy_one_vm.py
+++ b/tests/TestSuite_vhost_user_one_copy_one_vm.py
@@ -47,13 +47,18 @@ from etgen import IxiaPacketGenerator  from qemu_kvm import QEMUKvm
 
 
-class TestVhostSample(TestCase, IxiaPacketGenerator):
+class TestVhostUserOneCopyOneVm(TestCase, IxiaPacketGenerator):
 
     def set_up_all(self):
         # To Extend IXIA packet generator method, call the tester's method.
-        self.tester.extend_external_packet_generator(TestVhostSample, self)
+        
+ self.tester.extend_external_packet_generator(TestVhostUserOneCopyOneVm
+ , self)
 
         # Build target with modified config file
+        self.dut.send_expect(
+            "sed -i -e 's/CONFIG_RTE_LIBRTE_VHOST_USER=.*$/CONFIG_RTE_LIBRTE"
+            "_VHOST_USER=y/' ./config/common_linuxapp",
+            "# ",
+            30)
         self.dut.build_install_dpdk(self.target)
 
         # Get and verify the ports
@@ -78,10 +83,16 @@ class TestVhostSample(TestCase, IxiaPacketGenerator):
             " -c %s -n %d --socket-mem 1024,1024 -- -p 0x1 --mergeable %d" + \
             " --zero-copy %d --vm2vm %d 2 > ./vhost.out &"
         # build the vhost sample in vhost-user mode.
-        self.dut.send_expect(
-            "sed -i -e 's/define MAX_QUEUES 512/define MAX_QUEUES 128/'"
-            " ./examples/vhost/main.c",
-            "#")
+        if self.nic in ['niantic']:
+            self.dut.send_expect(
+                "sed -i -e 's/#define MAX_QUEUES.*$/#define MAX_QUEUES 128/' "
+                "./examples/vhost/main.c",
+                "#", 10)
+        else:
+            self.dut.send_expect(
+                "sed -i -e 's/#define MAX_QUEUES.*$/#define MAX_QUEUES 512/' "
+                "./examples/vhost/main.c",
+                "#", 10)
         out = self.dut.send_expect("make -C examples/vhost", "#")
         self.verify("Error" not in out, "compilation error")
         self.verify("No such file" not in out, "Not found file error") @@ -97,7 +108,7 @@ class TestVhostSample(TestCase, IxiaPacketGenerator):
         self.dst2 = "192.168.4.1"
         self.vm_dut = None
 
-        self.header_row = ["Test", "Mode", "Frame", "Mpps", "% linerate"]
+        self.header_row = ["FrameSize(B)", "RecvPackets(Mpps)", 
+ "SendPackets(Mpps)", "LineRate(%)"]

In fact, Recvpkt is the ixia sending traffic rate, sendpkt is the ixia received packet rate and the virtio/vhost throughput, the naming recvpkt is a little confusing, since we know we will send line rate traffic, no need list here. Suggest to remove recvpkt, and change sendpkt name as throughput, and if removed Test and mode, how do we know which case has the perf and which mode is using in the result check? 


         self.memory_channel = 4
 
     def set_up(self):
@@ -108,30 +119,19 @@ class TestVhostSample(TestCase, IxiaPacketGenerator):
 
         if "jumbo" in self.running_case:
             self.jumbo = 1
-            self.frame_sizes = [
-                68,
-                128,
-                256,
-                512,
-                1024,
-                1280,
-                1518,
-                2048,
-                5000,
-                9000]
+            self.frame_sizes = [64, 128, 256, 512, 1024, 1280, 1518, 
+ 2048, 5000, 9000]
             self.vm_testpmd = "./x86_64-native-linuxapp-gcc/app/testpmd -c 0x3 -n 3" \
-                + \
-                " -- -i --txqflags=0xf00 --disable-hw-vlan-filter --max-pkt-len 9600"
+                + " -- -i --txqflags=0xf00 " \
+                + "--disable-hw-vlan-filter --max-pkt-len 9600"
         else:
             self.jumbo = 0
-            self.frame_sizes = [68, 128, 256, 512, 1024, 1280, 1518]
+            self.frame_sizes = [64, 128, 256, 512, 1024, 1280, 1518]
             self.vm_testpmd = "./x86_64-native-linuxapp-gcc/app/testpmd -c 0x3 -n 3" \
                 + " -- -i --txqflags=0xf00 --disable-hw-vlan-filter"
         self.dut.send_expect("rm -rf ./vhost.out", "#")
 
         self.launch_vhost_sample()
 
-        print "Start VM with 2virtio\n"
         # start VM with 2virtio
         self.start_onevm()
 
@@ -146,15 +146,14 @@ class TestVhostSample(TestCase, IxiaPacketGenerator):
         time.sleep(30)
         try:
             print "Launch vhost sample\n"
-            out = self.dut.send_expect(
-                "cat ./vhost.out",
-                "VHOST_CONFIG: bind to vhost-net",
-                20)
+            self.dut.session.copy_file_from("/root/dpdk/vhost.out")
+            fp = open('./vhost.out', 'r')
+            out = fp.read()
+            fp.close()
             if "Error" in out:
                 raise Exception("Launch vhost sample failed")
         except Exception as e:
             print dts.RED("Failed to launch vhost sample: %s" % str(e))
-            self.dut.send_expect("rm -rf ./vhost-net", "#", 20)
Why remove this line? We need remove the module and ensure next time the app can be launched successfully, when exception happened, the sample will not remove the module. 
 
     def start_onevm(self):
         #
@@ -172,7 +171,7 @@ class TestVhostSample(TestCase, IxiaPacketGenerator):
             self.vm.set_vm_device(**vm_params)
 
         try:
-            self.vm_dut = self.vm.start(auto_portmap=False)
+            self.vm_dut = self.vm.start()
             if self.vm_dut is None:
                 raise Exception("Set up VM ENV failed")
         except Exception as e:
@@ -267,23 +266,76 @@ class TestVhostSample(TestCase, IxiaPacketGenerator):
             self.vm_dut.send_expect("ip route show", "#")
             print self.vm_dut.send_expect("arp -a", "#")
 
+    def get_transmission_results(self, rx_port_list, tx_port_list, delay=5):
+        time.sleep(delay)
+        recvbpsRate = 0
+        recvRate = 0
+        txbpsRate = 0
+        txRate = 0
+        for port in tx_port_list:
+            self.stat_get_rate_stat_all_stats(port)
+            out = self.send_expect('stat cget -framesSent', '%', 10)
+            txRate += int(out.strip())
+            self.logger.info("Port %s: TX %f Mpps" % (port, (txRate * 1.0 / 1000000)))
+            out = self.send_expect('stat cget -bitsSent', '%', 10)
+            txbpsRate += int(out.strip())
+            self.logger.info("Port %s: TX %f Mbps" % (port, (txbpsRate * 1.0 / 1000000)))
+        for port in rx_port_list:
+            self.stat_get_rate_stat_all_stats(port)
+            out = self.send_expect('stat cget -framesReceived', '%', 10)
+            recvRate += int(out.strip())
+            out = self.send_expect('stat cget -oversize', '%', 10)
+            recvRate += int(out.strip())
+            self.logger.info("Port %s: RX %f Mpps" % (port, (recvRate * 1.0 / 1000000)))
+            out = self.send_expect('stat cget -bitsReceived', '%', 10)
+            recvbpsRate += int(out.strip())
+            self.logger.info("Port %s: RX %f Mbps" % (port, 
+ (recvbpsRate * 1.0 / 1000000)))
+

+        return (txRate,recvRate)
+
+    def send_verify(self, case, frame_sizes, vlan_id1=0):
+        dts.results_table_add_header(self.header_row)
+        for frame_size in frame_sizes:
+            info = "Running test %s, and %d frame size.\n" % (case, frame_size)
+            self.logger.info(info)
+            payload = frame_size - HEADER_SIZE['eth'] - HEADER_SIZE['ip']
+            flow = '[Ether(dst="%s")/Dot1Q(vlan=%s)/IP(src="%s",dst="%s")/("X"*%d)]' % (
+                self.virtio1_mac, vlan_id1, self.src1, self.dst1, payload)
+            self.tester.scapy_append('wrpcap("flow.pcap", %s)' % flow)
+            self.tester.scapy_execute()
+
+            tgenInput = []
+            port = self.tester.get_local_port(self.pf)
+            tgenInput.append((port, port, "flow.pcap"))
+
+            recvpkt, sendpkt = self.tester.traffic_generator_throughput(
+                tgenInput, delay=15)
+            recvpkt /= 1000000.0
+            sendpkt /= 1000000.0
+            pct = sendpkt * 100 / recvpkt
+            data_row = [frame_size, str(recvpkt), str(sendpkt), 
+ str(pct)]

+            dts.results_table_add_row(data_row)
+        dts.results_table_print()
+
     def test_perf_user_one_vm_legacy_fwd(self):
         #
         # Test the performance of one vm with 2virtio devices in legacy fwd
         #
         # Restore the virtio interfaces to use legacy driver
         self.vm_dut.restore_interfaces()
-
         self.set_legacy_disablefw()
         # Set the legacy fwd rules then get the VLAN id from vhost sample
         # output
         self.set_onevm_legacy_fwd()
-
         time.sleep(5)
-        out_clean = self.dut.get_session_output(timeout=2)
-        out = self.dut.send_expect("cat ./vhost.out", "# ", 20)
+
+        self.dut.get_session_output(timeout=2)
+        self.dut.session.copy_file_from("/root/dpdk/vhost.out")
+        fp = open('./vhost.out', 'r')
+        out = fp.read()
+        fp.close()
         # Get the VLAN ID for virtio
-        print out, "\ncheck the vlan info: \n"
+        print "Check the vlan info: \n"
         l1 = re.findall(
             'MAC_ADDRESS.*?%s.*?and.*?VLAN_TAG.*?(\d+).*?registered' %
             (str(self.virtio1_mac)), out) @@ -297,41 +349,7 @@ class TestVhostSample(TestCase, IxiaPacketGenerator):
             vlan_id2 = l2[0]
             print "vlan_id2 is ", vlan_id2
 
-        dts.results_table_add_header(self.header_row)
-
-        # Create pcap file and ixia traffic
-        for frame_size in self.frame_sizes:
-            info = "Running test %s, and %d frame size.\n" % (
-                self.running_case, frame_size)
-            self.logger.info(info)
-
-            payload_size = frame_size - HEADER_SIZE['eth'] - HEADER_SIZE['ip']
-            flow1 = '[Ether(dst="%s")/Dot1Q(vlan=%s)/IP(src="%s",dst="%s")/("X"*%d)]' % (
-                self.virtio1_mac, vlan_id1, self.src1, self.dst1, payload_size)
-            flow2 = '[Ether(dst="%s")/Dot1Q(vlan=%s)/IP(src="%s",dst="%s")/("X"*%d)]' % (
-                self.virtio2_mac, vlan_id2, self.src2, self.dst2, payload_size)
-
-            self.tester.scapy_append('wrpcap("flow1.pcap", %s)' % flow1)
-            self.tester.scapy_append('wrpcap("flow2.pcap",%s)' % flow2)
-            self.tester.scapy_execute()
-
-        # Capture the performance
-            tgenInput = []
-            port = self.tester.get_local_port(self.pf)
-            tgenInput.append((port, port, "flow2.pcap"))
-
-            _, pps = self.tester.traffic_generator_throughput(
-                tgenInput, delay=15)
-            pps /= 1000000.0
-            linerate = self.wirespeed(self.nic, frame_size, 1)
-            pct = pps * 100 / linerate
-            scenario = self.running_case
-            mode = "vhost user"
-            data_row = [scenario, mode, frame_size, str(pps), str(pct)]
-            dts.results_table_add_row(data_row)
-
-        dts.results_table_print()
-
+        self.send_verify(self.running_case, self.frame_sizes, vlan_id1)
         # Stop the Vhost sample
         self.dut.send_expect("killall -s INT vhost-switch", "#", 20)
 
@@ -339,16 +357,18 @@ class TestVhostSample(TestCase, IxiaPacketGenerator):
         #
         # Test the performance of one vm with 2virtio devices in legacy fwd
         #
-
         # start testpmd on VM
         self.vm_testpmd_start()
         time.sleep(5)
         # Clean the output to ensure the commands cat ./vhost.out can be sent
         # and got answered correctly.
-        out_clean = self.dut.get_session_output(timeout=2)
-        out = self.dut.send_expect("cat ./vhost.out", "# ", 20)
+        self.dut.get_session_output(timeout=2)
+        self.dut.session.copy_file_from("/root/dpdk/vhost.out")
+        fp = open('./vhost.out', 'r')
+        out = fp.read()
+        fp.close()
         # Get the VLAN ID for virtio
-        print out, "\ncheck the vlan info: \n"
+        print "Check the vlan info: \n"
         l1 = re.findall(
             'MAC_ADDRESS.*?%s.*?and.*?VLAN_TAG.*?(\d+).*?registered' %
             (str(self.virtio1_mac)), out) @@ -361,43 +381,9 @@ class TestVhostSample(TestCase, IxiaPacketGenerator):
         if len(l2) > 0:
             vlan_id2 = l2[0]
             print vlan_id2
-
-        dts.results_table_add_header(self.header_row)
-
-        # Create pcap file and ixia traffic
-        for frame_size in self.frame_sizes:
-            info = "Running test %s, and %d frame size.\n" % (
-                self.running_case, frame_size)
-            self.logger.info(info)
-
-            payload_size = frame_size - HEADER_SIZE['eth'] - HEADER_SIZE['ip']
-            flow1 = '[Ether(dst="%s")/Dot1Q(vlan=%s)/IP(src="%s",dst="%s")/("X"*%d)]' % (
-                self.virtio1_mac, vlan_id1, self.src1, self.dst1, payload_size)
-            flow2 = '[Ether(dst="%s")/Dot1Q(vlan=%s)/IP(src="%s",dst="%s")/("X"*%d)]' % (
-                self.virtio2_mac, vlan_id2, self.src2, self.dst2, payload_size)
-            self.tester.scapy_append('wrpcap("flow1.pcap", %s)' % flow1)
-            self.tester.scapy_append('wrpcap("flow2.pcap",%s)' % flow2)
-            self.tester.scapy_execute()
-
-        # Capture the performance
-            tgenInput = []
-            port = self.tester.get_local_port(self.pf)
-            tgenInput.append((port, port, "flow1.pcap"))
-            tgenInput.append((port, port, "flow2.pcap"))
-
-            _, pps = self.tester.traffic_generator_throughput(
-                tgenInput, delay=15)
-            pps /= 1000000.0
-            linerate = self.wirespeed(self.nic, frame_size, 1)
-            pct = pps * 100 / linerate
-            scenario = self.running_case
-            mode = "vhost user"
-            data_row = [scenario, mode, frame_size, str(pps), str(pct)]
-            dts.results_table_add_row(data_row)
-
-        dts.results_table_print()
+        self.send_verify(self.running_case, self.frame_sizes, vlan_id1)
         # Stop testpmd
-        print self.vm_dut.send_expect("stop", "testpmd>")
+        self.vm_dut.send_expect("stop", "testpmd>")
         time.sleep(1)
         self.vm_dut.send_expect("quit", "# ")
 
--
2.1.0