test suite reviews and discussions
 help / color / Atom feed
* [dts] [PATCH V1 00/17] change virtio case mpps verification value
@ 2020-05-22  8:31 Xiao Qimai
  2020-05-22  8:31 ` [dts] [PATCH V1 01/17]loopback_multi_paths_port_restart: change verfiy mpps to largger than 0.5 Xiao Qimai
                   ` (17 more replies)
  0 siblings, 18 replies; 20+ messages in thread
From: Xiao Qimai @ 2020-05-22  8:31 UTC (permalink / raw)
  To: dts; +Cc: Xiao Qimai

*. change verify mpps from 0 to a certain value to avoid misc packet interfeer

Xiao Qimai (17):
  change verfiy mpps to largger than 0.5
  change verfiy mpps to largger than 5
  change verfiy mpps to largger than 0.5
  change verified mpps value according to linerate
  change verified mpps value according to linerate
  change verified mpps value according to linerate
  change verified mpps value according to linerate
  change verified mpps value according to linerate
  change verified mpps value according to linerate
  change verified mpps value according to linerate
  change verified mpps value according to linerate
  change verified mpps value according to linerate
  change verified mpps value according to linerate
  change verified mpps value according to linerate
  change verified mpps value according to linerate
  change verified mpps value according to linerate
  change verified mpps value to 0.5

 .../TestSuite_loopback_multi_paths_port_restart.py |  2 +-
 tests/TestSuite_loopback_multi_queues.py           |  2 +-
 .../TestSuite_loopback_virtio_user_server_mode.py  |  2 +-
 tests/TestSuite_pvp_diff_qemu_version.py           | 13 ++++-
 tests/TestSuite_pvp_multi_paths_performance.py     | 56 +++++++++++++---------
 ...vp_multi_paths_vhost_single_core_performance.py | 14 +++++-
 ...p_multi_paths_virtio_single_core_performance.py | 14 +++++-
 .../TestSuite_pvp_qemu_multi_paths_port_restart.py | 13 ++++-
 ...TestSuite_pvp_vhost_user_built_in_net_driver.py | 13 ++++-
 tests/TestSuite_pvp_virtio_user_2M_hugepages.py    | 13 ++++-
 tests/TestSuite_pvp_virtio_user_4k_pages.py        | 13 ++++-
 ...te_pvp_virtio_user_multi_queues_port_restart.py | 13 ++++-
 tests/TestSuite_vhost_dequeue_zero_copy.py         | 13 ++++-
 tests/TestSuite_vhost_multi_queue_qemu.py          | 13 ++++-
 tests/TestSuite_virtio_pvp_regression.py           | 13 ++++-
 ...stSuite_virtio_user_for_container_networking.py | 13 ++++-
 tests/TestSuite_vm2vm_virtio_pmd.py                |  3 +-
 17 files changed, 182 insertions(+), 41 deletions(-)

-- 
1.8.3.1


^ permalink raw reply	[flat|nested] 20+ messages in thread

* [dts] [PATCH V1 01/17]loopback_multi_paths_port_restart: change verfiy mpps to largger than 0.5
  2020-05-22  8:31 [dts] [PATCH V1 00/17] change virtio case mpps verification value Xiao Qimai
@ 2020-05-22  8:31 ` Xiao Qimai
  2020-05-22  8:31 ` [dts] [PATCH V1 02/17]loopback_multi_queues: change verfiy mpps to largger than 5 Xiao Qimai
                   ` (16 subsequent siblings)
  17 siblings, 0 replies; 20+ messages in thread
From: Xiao Qimai @ 2020-05-22  8:31 UTC (permalink / raw)
  To: dts; +Cc: Xiao Qimai

Signed-off-by: Xiao Qimai <qimaix.xiao@intel.com>
---
 tests/TestSuite_loopback_multi_paths_port_restart.py | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/tests/TestSuite_loopback_multi_paths_port_restart.py b/tests/TestSuite_loopback_multi_paths_port_restart.py
index 8d23196..526bc48 100644
--- a/tests/TestSuite_loopback_multi_paths_port_restart.py
+++ b/tests/TestSuite_loopback_multi_paths_port_restart.py
@@ -159,7 +159,7 @@ class TestLoopbackPortRestart(TestCase):
             result = lines.group(1)
             results += float(result)
         Mpps = results / (1000000 * 10)
-        self.verify(Mpps > 0, "%s can not receive packets" % self.running_case)
+        self.verify(Mpps > 0.5, "%s can not receive packets" % self.running_case)
         return Mpps
 
     def send_and_verify(self, case_info, frame_size, restart_times=1):
-- 
1.8.3.1


^ permalink raw reply	[flat|nested] 20+ messages in thread

* [dts] [PATCH V1 02/17]loopback_multi_queues: change verfiy mpps to largger than 5
  2020-05-22  8:31 [dts] [PATCH V1 00/17] change virtio case mpps verification value Xiao Qimai
  2020-05-22  8:31 ` [dts] [PATCH V1 01/17]loopback_multi_paths_port_restart: change verfiy mpps to largger than 0.5 Xiao Qimai
@ 2020-05-22  8:31 ` Xiao Qimai
  2020-05-22  8:31 ` [dts] [PATCH V1 03/17]loopback_virtio_user_server_mode: change verfiy mpps to largger than 0.5 Xiao Qimai
                   ` (15 subsequent siblings)
  17 siblings, 0 replies; 20+ messages in thread
From: Xiao Qimai @ 2020-05-22  8:31 UTC (permalink / raw)
  To: dts; +Cc: Xiao Qimai

Signed-off-by: Xiao Qimai <qimaix.xiao@intel.com>
---
 tests/TestSuite_loopback_multi_queues.py | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/tests/TestSuite_loopback_multi_queues.py b/tests/TestSuite_loopback_multi_queues.py
index 029c1f0..c08b252 100644
--- a/tests/TestSuite_loopback_multi_queues.py
+++ b/tests/TestSuite_loopback_multi_queues.py
@@ -116,7 +116,7 @@ class TestLoopbackMultiQueues(TestCase):
             result = lines.group(1)
             results += float(result)
         Mpps = results / (1000000 * 10)
-        self.verify(Mpps > 0, "port can not receive packets")
+        self.verify(Mpps > 5, "port can not receive packets")
         return Mpps
 
     def update_result_table(self, frame_size, case_info, Mpps):
-- 
1.8.3.1


^ permalink raw reply	[flat|nested] 20+ messages in thread

* [dts] [PATCH V1 03/17]loopback_virtio_user_server_mode: change verfiy mpps to largger than 0.5
  2020-05-22  8:31 [dts] [PATCH V1 00/17] change virtio case mpps verification value Xiao Qimai
  2020-05-22  8:31 ` [dts] [PATCH V1 01/17]loopback_multi_paths_port_restart: change verfiy mpps to largger than 0.5 Xiao Qimai
  2020-05-22  8:31 ` [dts] [PATCH V1 02/17]loopback_multi_queues: change verfiy mpps to largger than 5 Xiao Qimai
@ 2020-05-22  8:31 ` Xiao Qimai
  2020-06-01  3:27   ` Wang, Yinan
  2020-05-22  8:31 ` [dts] [PATCH V1 04/17]pvp_diff_qemu_version: change verified mpps value according to linerate Xiao Qimai
                   ` (14 subsequent siblings)
  17 siblings, 1 reply; 20+ messages in thread
From: Xiao Qimai @ 2020-05-22  8:31 UTC (permalink / raw)
  To: dts; +Cc: Xiao Qimai

Signed-off-by: Xiao Qimai <qimaix.xiao@intel.com>
---
 tests/TestSuite_loopback_virtio_user_server_mode.py | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/tests/TestSuite_loopback_virtio_user_server_mode.py b/tests/TestSuite_loopback_virtio_user_server_mode.py
index 32ee1c3..afc4108 100644
--- a/tests/TestSuite_loopback_virtio_user_server_mode.py
+++ b/tests/TestSuite_loopback_virtio_user_server_mode.py
@@ -183,7 +183,7 @@ class TestLoopbackVirtioUserServerMode(TestCase):
             result = lines.group(1)
             results += float(result)
         Mpps = results / (1000000 * 10)
-        self.verify(Mpps > 0, "port can not receive packets")
+        self.verify(Mpps > 0.5, "port can not receive packets")
 
         results_row.append(case_info)
         results_row.append('64')
-- 
1.8.3.1


^ permalink raw reply	[flat|nested] 20+ messages in thread

* [dts] [PATCH V1 04/17]pvp_diff_qemu_version: change verified mpps value according to linerate
  2020-05-22  8:31 [dts] [PATCH V1 00/17] change virtio case mpps verification value Xiao Qimai
                   ` (2 preceding siblings ...)
  2020-05-22  8:31 ` [dts] [PATCH V1 03/17]loopback_virtio_user_server_mode: change verfiy mpps to largger than 0.5 Xiao Qimai
@ 2020-05-22  8:31 ` Xiao Qimai
  2020-05-22  8:31 ` [dts] [PATCH V1 05/17]pvp_multi_paths_performance: " Xiao Qimai
                   ` (13 subsequent siblings)
  17 siblings, 0 replies; 20+ messages in thread
From: Xiao Qimai @ 2020-05-22  8:31 UTC (permalink / raw)
  To: dts; +Cc: Xiao Qimai

Signed-off-by: Xiao Qimai <qimaix.xiao@intel.com>
---
 tests/TestSuite_pvp_diff_qemu_version.py | 13 ++++++++++++-
 1 file changed, 12 insertions(+), 1 deletion(-)

diff --git a/tests/TestSuite_pvp_diff_qemu_version.py b/tests/TestSuite_pvp_diff_qemu_version.py
index b9ce822..de4e0fd 100644
--- a/tests/TestSuite_pvp_diff_qemu_version.py
+++ b/tests/TestSuite_pvp_diff_qemu_version.py
@@ -220,6 +220,15 @@ class TestVhostPVPDiffQemuVersion(TestCase):
             self.vm_dut.send_expect("set fwd mac", "testpmd> ", 20)
             self.vm_dut.send_expect("start", "testpmd> ")
 
+    @property
+    def check_value(self):
+        check_dict = dict.fromkeys(self.frame_sizes)
+        linerate = {64: 0.085, 128: 0.12, 256: 0.20, 512: 0.35, 1024: 0.50, 1280: 0.55, 1518: 0.60}
+        for size in self.frame_sizes:
+            speed = self.wirespeed(self.nic, size, self.number_of_ports)
+            check_dict[size] = round(speed * linerate[size], 2)
+        return check_dict
+
     def send_verify(self, qemu_version, vlan_id1=0, tag="Performance"):
         self.result_table_create(self.header_row)
         for frame_size in self.frame_sizes:
@@ -243,7 +252,9 @@ class TestVhostPVPDiffQemuVersion(TestCase):
             _, pps = self.tester.pktgen.measure_throughput(stream_ids=streams, options=traffic_opt)
             Mpps = pps / 1000000.0
             pct = Mpps * 100 / float(self.wirespeed(self.nic, frame_size, 1))
-            self.verify(Mpps != 0, "can not received data of frame size %d" % frame_size)
+            self.verify(Mpps > self.check_value[frame_size],
+                        "%s of frame size %d speed verify failed, expect %s, result %s" % (
+                        self.running_case, frame_size, self.check_value[frame_size], Mpps))
             # update print table info
             data_row = [qemu_version, frame_size, str(Mpps), str(pct), tag]
             self.result_table_add(data_row)
-- 
1.8.3.1


^ permalink raw reply	[flat|nested] 20+ messages in thread

* [dts] [PATCH V1 05/17]pvp_multi_paths_performance: change verified mpps value according to linerate
  2020-05-22  8:31 [dts] [PATCH V1 00/17] change virtio case mpps verification value Xiao Qimai
                   ` (3 preceding siblings ...)
  2020-05-22  8:31 ` [dts] [PATCH V1 04/17]pvp_diff_qemu_version: change verified mpps value according to linerate Xiao Qimai
@ 2020-05-22  8:31 ` " Xiao Qimai
  2020-05-22  8:31 ` [dts] [PATCH V1 06/17]pvp_multi_paths_vhost_single_core_performance: " Xiao Qimai
                   ` (12 subsequent siblings)
  17 siblings, 0 replies; 20+ messages in thread
From: Xiao Qimai @ 2020-05-22  8:31 UTC (permalink / raw)
  To: dts; +Cc: Xiao Qimai

Signed-off-by: Xiao Qimai <qimaix.xiao@intel.com>
---
 tests/TestSuite_pvp_multi_paths_performance.py | 56 +++++++++++++++-----------
 1 file changed, 33 insertions(+), 23 deletions(-)

diff --git a/tests/TestSuite_pvp_multi_paths_performance.py b/tests/TestSuite_pvp_multi_paths_performance.py
index 14a0fd7..3e6e593 100644
--- a/tests/TestSuite_pvp_multi_paths_performance.py
+++ b/tests/TestSuite_pvp_multi_paths_performance.py
@@ -99,6 +99,15 @@ class TestPVPMultiPathPerformance(TestCase):
         self.test_result = {}
         self.nb_desc = self.test_parameters[64][0]
 
+    @property
+    def check_value(self):
+        check_dict = dict.fromkeys(self.frame_sizes)
+        linerate = {64: 0.085, 128: 0.12, 256: 0.20, 512: 0.35, 1024: 0.50, 1280: 0.55, 1518: 0.60}
+        for size in self.frame_sizes:
+            speed = self.wirespeed(self.nic, size, self.number_of_ports)
+            check_dict[size] = round(speed * linerate[size], 2)
+        return check_dict
+
     def send_and_verify(self, case_info):
         """
         Send packet with packet generator and verify
@@ -108,7 +117,7 @@ class TestPVPMultiPathPerformance(TestCase):
             self.throughput[frame_size] = dict()
 
             self.logger.info("Test running at parameters: " +
-                "framesize: {}, rxd/txd: {}".format(frame_size, self.nb_desc))
+                             "framesize: {}, rxd/txd: {}".format(frame_size, self.nb_desc))
             rx_port = self.tester.get_local_port(
                 self.dut_ports[0])
             tx_port = self.tester.get_local_port(
@@ -124,11 +133,12 @@ class TestPVPMultiPathPerformance(TestCase):
             streams = self.pktgen_helper.prepare_stream_from_tginput(tgen_input, 100, None, self.tester.pktgen)
             _, pps = self.tester.pktgen.measure_throughput(stream_ids=streams)
             Mpps = pps / 1000000.0
-            self.verify(Mpps > 0.0, "%s can not receive packets of frame size %d" % (self.running_case, frame_size))
-
+            self.verify(Mpps > self.check_value[frame_size],
+                        "%s of frame size %d speed verify failed, expect %s, result %s" % (
+                        self.running_case, frame_size, self.check_value[frame_size], Mpps))
             self.throughput[frame_size][self.nb_desc] = Mpps
             linerate = Mpps * 100 / \
-                    float(self.wirespeed(self.nic, frame_size, self.number_of_ports))
+                       float(self.wirespeed(self.nic, frame_size, self.number_of_ports))
 
             results_row = [frame_size]
             results_row.append(case_info)
@@ -153,7 +163,7 @@ class TestPVPMultiPathPerformance(TestCase):
                                                    ports=[self.dut.ports_info[self.dut_ports[0]]['pci']],
                                                    vdevs=['net_vhost0,iface=vhost-net,queues=1,client=0'])
         command_line_client = "./%s/app/testpmd " % self.target + eal_param + \
-                    " -- -i --nb-cores=1 --txd=%d --rxd=%d" % (self.nb_desc, self.nb_desc)
+                              " -- -i --nb-cores=1 --txd=%d --rxd=%d" % (self.nb_desc, self.nb_desc)
         self.vhost.send_expect(command_line_client, "testpmd> ", 120)
         self.vhost.send_expect("set fwd mac", "testpmd> ", 120)
         self.vhost.send_expect("start", "testpmd> ", 120)
@@ -169,7 +179,7 @@ class TestPVPMultiPathPerformance(TestCase):
         if self.check_2M_env:
             eal_param += " --single-file-segments"
         command_line_user = "./%s/app/testpmd " % self.target + eal_param + \
-                    " -- -i %s --rss-ip --nb-cores=1 --txd=%d --rxd=%d" % \
+                            " -- -i %s --rss-ip --nb-cores=1 --txd=%d --rxd=%d" % \
                             (args["path"], self.nb_desc, self.nb_desc)
         self.vhost_user = self.dut.new_session(suite="user")
         self.vhost_user.send_expect(command_line_user, "testpmd> ", 120)
@@ -203,14 +213,14 @@ class TestPVPMultiPathPerformance(TestCase):
                 ret_data[header[0]] = frame_size
                 ret_data[header[1]] = nb_desc
                 ret_data[header[2]] = "{:.3f} Mpps".format(
-                        self.throughput[frame_size][nb_desc])
+                    self.throughput[frame_size][nb_desc])
                 ret_data[header[3]] = "{:.3f}%".format(
-                        self.throughput[frame_size][nb_desc] * 100 / wirespeed)
+                    self.throughput[frame_size][nb_desc] * 100 / wirespeed)
                 ret_data[header[4]] = "{:.3f} Mpps".format(
-                        self.expected_throughput[frame_size][nb_desc])
+                    self.expected_throughput[frame_size][nb_desc])
                 ret_data[header[5]] = "{:.3f} Mpps".format(
-                        self.throughput[frame_size][nb_desc] -
-                        self.expected_throughput[frame_size][nb_desc])
+                    self.throughput[frame_size][nb_desc] -
+                    self.expected_throughput[frame_size][nb_desc])
                 ret_datas[nb_desc] = deepcopy(ret_data)
             self.test_result[frame_size] = deepcopy(ret_datas)
         # Create test results table
@@ -266,8 +276,8 @@ class TestPVPMultiPathPerformance(TestCase):
                 self.json_obj[case_name].append(row_dict0)
                 status_result.append(row_dict0['status'])
         with open(os.path.join(rst.path2Result,
-                        '{0:s}_{1}.json'.format(
-                            self.nic, self.suite_name)), 'w') as fp:
+                               '{0:s}_{1}.json'.format(
+                                   self.nic, self.suite_name)), 'w') as fp:
             json.dump(self.json_obj, fp)
         self.verify("FAIL" not in status_result, "Exceeded Gap")
 
@@ -292,7 +302,7 @@ class TestPVPMultiPathPerformance(TestCase):
         self.test_target = self.running_case
         self.expected_throughput = self.get_suite_cfg()['expected_throughput'][self.test_target]
         virtio_pmd_arg = {"version": "in_order=0,packed_vq=1,mrg_rxbuf=1",
-                            "path": "--tx-offloads=0x0 --enable-hw-vlan-strip"}
+                          "path": "--tx-offloads=0x0 --enable-hw-vlan-strip"}
         self.start_vhost_testpmd()
         self.start_virtio_testpmd(virtio_pmd_arg)
         self.send_and_verify("virtio_1.1_mergeable on")
@@ -309,7 +319,7 @@ class TestPVPMultiPathPerformance(TestCase):
         self.test_target = self.running_case
         self.expected_throughput = self.get_suite_cfg()['expected_throughput'][self.test_target]
         virtio_pmd_arg = {"version": "in_order=0,packed_vq=1,mrg_rxbuf=0",
-                            "path": "--tx-offloads=0x0 --enable-hw-vlan-strip"}
+                          "path": "--tx-offloads=0x0 --enable-hw-vlan-strip"}
         self.start_vhost_testpmd()
         self.start_virtio_testpmd(virtio_pmd_arg)
         self.send_and_verify("virtio_1.1_normal")
@@ -326,7 +336,7 @@ class TestPVPMultiPathPerformance(TestCase):
         self.test_target = self.running_case
         self.expected_throughput = self.get_suite_cfg()['expected_throughput'][self.test_target]
         virtio_pmd_arg = {"version": "in_order=1,packed_vq=1,mrg_rxbuf=1",
-                            "path": "--tx-offloads=0x0 --enable-hw-vlan-strip"}
+                          "path": "--tx-offloads=0x0 --enable-hw-vlan-strip"}
         self.start_vhost_testpmd()
         self.start_virtio_testpmd(virtio_pmd_arg)
         self.send_and_verify("virtio_1.1_inorder_mergeable on")
@@ -343,7 +353,7 @@ class TestPVPMultiPathPerformance(TestCase):
         self.test_target = self.running_case
         self.expected_throughput = self.get_suite_cfg()['expected_throughput'][self.test_target]
         virtio_pmd_arg = {"version": "in_order=1,packed_vq=1,mrg_rxbuf=0,vectorized=1",
-                            "path": "--rx-offloads=0x10 --enable-hw-vlan-strip"}
+                          "path": "--rx-offloads=0x10 --enable-hw-vlan-strip"}
         self.start_vhost_testpmd()
         self.start_virtio_testpmd(virtio_pmd_arg)
         self.send_and_verify("virtio_1.1_inorder_normal")
@@ -360,7 +370,7 @@ class TestPVPMultiPathPerformance(TestCase):
         self.test_target = self.running_case
         self.expected_throughput = self.get_suite_cfg()['expected_throughput'][self.test_target]
         virtio_pmd_arg = {"version": "in_order=1,packed_vq=1,mrg_rxbuf=0,vectorized=1",
-                            "path": "--tx-offloads=0x0 --enable-hw-vlan-strip"}
+                          "path": "--tx-offloads=0x0 --enable-hw-vlan-strip"}
         self.start_vhost_testpmd()
         self.start_virtio_testpmd(virtio_pmd_arg)
         self.send_and_verify("virtio_1.1_inorder_normal")
@@ -377,7 +387,7 @@ class TestPVPMultiPathPerformance(TestCase):
         self.test_target = self.running_case
         self.expected_throughput = self.get_suite_cfg()['expected_throughput'][self.test_target]
         virtio_pmd_arg = {"version": "packed_vq=0,in_order=1,mrg_rxbuf=1",
-                            "path": "--tx-offloads=0x0 --enable-hw-vlan-strip"}
+                          "path": "--tx-offloads=0x0 --enable-hw-vlan-strip"}
         self.start_vhost_testpmd()
         self.start_virtio_testpmd(virtio_pmd_arg)
         self.send_and_verify("inoder mergeable on")
@@ -394,7 +404,7 @@ class TestPVPMultiPathPerformance(TestCase):
         self.test_target = self.running_case
         self.expected_throughput = self.get_suite_cfg()['expected_throughput'][self.test_target]
         virtio_pmd_arg = {"version": "packed_vq=0,in_order=1,mrg_rxbuf=0",
-                        "path": "--tx-offloads=0x0 --enable-hw-vlan-strip"}
+                          "path": "--tx-offloads=0x0 --enable-hw-vlan-strip"}
         self.start_vhost_testpmd()
         self.start_virtio_testpmd(virtio_pmd_arg)
         self.send_and_verify("inoder mergeable off")
@@ -411,7 +421,7 @@ class TestPVPMultiPathPerformance(TestCase):
         self.test_target = self.running_case
         self.expected_throughput = self.get_suite_cfg()['expected_throughput'][self.test_target]
         virtio_pmd_arg = {"version": "packed_vq=0,in_order=0,mrg_rxbuf=1",
-                            "path": "--tx-offloads=0x0 --enable-hw-vlan-strip"}
+                          "path": "--tx-offloads=0x0 --enable-hw-vlan-strip"}
         self.start_vhost_testpmd()
         self.start_virtio_testpmd(virtio_pmd_arg)
         self.send_and_verify("virito mergeable")
@@ -428,7 +438,7 @@ class TestPVPMultiPathPerformance(TestCase):
         self.test_target = self.running_case
         self.expected_throughput = self.get_suite_cfg()['expected_throughput'][self.test_target]
         virtio_pmd_arg = {"version": "packed_vq=0,in_order=0,mrg_rxbuf=0,vectorized=1",
-                            "path": "--tx-offloads=0x0 --enable-hw-vlan-strip"}
+                          "path": "--tx-offloads=0x0 --enable-hw-vlan-strip"}
         self.start_vhost_testpmd()
         self.start_virtio_testpmd(virtio_pmd_arg)
         self.send_and_verify("virito normal")
@@ -445,7 +455,7 @@ class TestPVPMultiPathPerformance(TestCase):
         self.test_target = self.running_case
         self.expected_throughput = self.get_suite_cfg()['expected_throughput'][self.test_target]
         virtio_pmd_arg = {"version": "packed_vq=0,in_order=0,mrg_rxbuf=0,vectorized=1",
-                            "path": "--tx-offloads=0x0 "}
+                          "path": "--tx-offloads=0x0 "}
         self.start_vhost_testpmd()
         self.start_virtio_testpmd(virtio_pmd_arg)
         self.send_and_verify("virito vector rx")
-- 
1.8.3.1


^ permalink raw reply	[flat|nested] 20+ messages in thread

* [dts] [PATCH V1 06/17]pvp_multi_paths_vhost_single_core_performance: change verified mpps value according to linerate
  2020-05-22  8:31 [dts] [PATCH V1 00/17] change virtio case mpps verification value Xiao Qimai
                   ` (4 preceding siblings ...)
  2020-05-22  8:31 ` [dts] [PATCH V1 05/17]pvp_multi_paths_performance: " Xiao Qimai
@ 2020-05-22  8:31 ` " Xiao Qimai
  2020-05-22  8:31 ` [dts] [PATCH V1 07/17]pvp_multi_paths_virtio_single_core_performance: " Xiao Qimai
                   ` (11 subsequent siblings)
  17 siblings, 0 replies; 20+ messages in thread
From: Xiao Qimai @ 2020-05-22  8:31 UTC (permalink / raw)
  To: dts; +Cc: Xiao Qimai

Signed-off-by: Xiao Qimai <qimaix.xiao@intel.com>
---
 ...tSuite_pvp_multi_paths_vhost_single_core_performance.py | 14 ++++++++++++--
 1 file changed, 12 insertions(+), 2 deletions(-)

diff --git a/tests/TestSuite_pvp_multi_paths_vhost_single_core_performance.py b/tests/TestSuite_pvp_multi_paths_vhost_single_core_performance.py
index cddd831..6f0f139 100644
--- a/tests/TestSuite_pvp_multi_paths_vhost_single_core_performance.py
+++ b/tests/TestSuite_pvp_multi_paths_vhost_single_core_performance.py
@@ -99,6 +99,15 @@ class TestPVPMultiPathVhostPerformance(TestCase):
         self.test_result = {}
         self.nb_desc = self.test_parameters[64][0]
 
+    @property
+    def check_value(self):
+        check_dict = dict.fromkeys(self.frame_sizes)
+        linerate = {64: 0.085, 128: 0.12, 256: 0.20, 512: 0.35, 1024: 0.50, 1280: 0.55, 1518: 0.60}
+        for size in self.frame_sizes:
+            speed = self.wirespeed(self.nic, size, self.number_of_ports)
+            check_dict[size] = round(speed * linerate[size], 2)
+        return check_dict
+
     def send_and_verify(self, case_info):
         """
         Send packet with packet generator and verify
@@ -123,8 +132,9 @@ class TestPVPMultiPathVhostPerformance(TestCase):
             streams = self.pktgen_helper.prepare_stream_from_tginput(tgen_input, 100, None, self.tester.pktgen)
             _, pps = self.tester.pktgen.measure_throughput(stream_ids=streams)
             Mpps = pps / 1000000.0
-            self.verify(Mpps > 0.0, "%s can not receive packets of frame size %d" % (self.running_case, frame_size))
-
+            self.verify(Mpps > self.check_value[frame_size],
+                        "%s of frame size %d speed verify failed, expect %s, result %s" % (
+                        self.running_case, frame_size, self.check_value[frame_size], Mpps))
             linerate = Mpps * 100 / \
                          float(self.wirespeed(self.nic, frame_size, self.number_of_ports))
             self.throughput[frame_size][self.nb_desc] = Mpps
-- 
1.8.3.1


^ permalink raw reply	[flat|nested] 20+ messages in thread

* [dts] [PATCH V1 07/17]pvp_multi_paths_virtio_single_core_performance: change verified mpps value according to linerate
  2020-05-22  8:31 [dts] [PATCH V1 00/17] change virtio case mpps verification value Xiao Qimai
                   ` (5 preceding siblings ...)
  2020-05-22  8:31 ` [dts] [PATCH V1 06/17]pvp_multi_paths_vhost_single_core_performance: " Xiao Qimai
@ 2020-05-22  8:31 ` " Xiao Qimai
  2020-05-22  8:31 ` [dts] [PATCH V1 08/17]pvp_qemu_multi_paths_port_restart: " Xiao Qimai
                   ` (10 subsequent siblings)
  17 siblings, 0 replies; 20+ messages in thread
From: Xiao Qimai @ 2020-05-22  8:31 UTC (permalink / raw)
  To: dts; +Cc: Xiao Qimai

Signed-off-by: Xiao Qimai <qimaix.xiao@intel.com>
---
 ...Suite_pvp_multi_paths_virtio_single_core_performance.py | 14 ++++++++++++--
 1 file changed, 12 insertions(+), 2 deletions(-)

diff --git a/tests/TestSuite_pvp_multi_paths_virtio_single_core_performance.py b/tests/TestSuite_pvp_multi_paths_virtio_single_core_performance.py
index b98048b..13ec493 100644
--- a/tests/TestSuite_pvp_multi_paths_virtio_single_core_performance.py
+++ b/tests/TestSuite_pvp_multi_paths_virtio_single_core_performance.py
@@ -100,6 +100,15 @@ class TestPVPMultiPathVirtioPerformance(TestCase):
         self.test_result = {}
         self.nb_desc = self.test_parameters[64][0]
 
+    @property
+    def check_value(self):
+        check_dict = dict.fromkeys(self.frame_sizes)
+        linerate = {64: 0.085, 128: 0.12, 256: 0.20, 512: 0.35, 1024: 0.50, 1280: 0.55, 1518: 0.60}
+        for size in self.frame_sizes:
+            speed = self.wirespeed(self.nic, size, self.number_of_ports)
+            check_dict[size] = round(speed * linerate[size], 2)
+        return check_dict
+
     def send_and_verify(self, case_info):
         """
         Send packet with packet generator and verify
@@ -126,8 +135,9 @@ class TestPVPMultiPathVirtioPerformance(TestCase):
             traffic_opt = {'delay': 5}
             _, pps = self.tester.pktgen.measure_throughput(stream_ids=streams, options=traffic_opt)
             Mpps = pps / 1000000.0
-            self.verify(Mpps > 0.0, "%s can not receive packets of frame size %d" % (self.running_case, frame_size))
-
+            self.verify(Mpps > self.check_value[frame_size],
+                        "%s of frame size %d speed verify failed, expect %s, result %s" % (
+                        self.running_case, frame_size, self.check_value[frame_size], Mpps))
             linerate = Mpps * 100 / \
                          float(self.wirespeed(self.nic, frame_size, self.number_of_ports))
             self.throughput[frame_size][self.nb_desc] = Mpps
-- 
1.8.3.1


^ permalink raw reply	[flat|nested] 20+ messages in thread

* [dts] [PATCH V1 08/17]pvp_qemu_multi_paths_port_restart: change verified mpps value according to linerate
  2020-05-22  8:31 [dts] [PATCH V1 00/17] change virtio case mpps verification value Xiao Qimai
                   ` (6 preceding siblings ...)
  2020-05-22  8:31 ` [dts] [PATCH V1 07/17]pvp_multi_paths_virtio_single_core_performance: " Xiao Qimai
@ 2020-05-22  8:31 ` " Xiao Qimai
  2020-05-22  8:32 ` [dts] [PATCH V1 09/17]pvp_vhost_user_built_in_net_driver: " Xiao Qimai
                   ` (9 subsequent siblings)
  17 siblings, 0 replies; 20+ messages in thread
From: Xiao Qimai @ 2020-05-22  8:31 UTC (permalink / raw)
  To: dts; +Cc: Xiao Qimai

Signed-off-by: Xiao Qimai <qimaix.xiao@intel.com>
---
 tests/TestSuite_pvp_qemu_multi_paths_port_restart.py | 13 ++++++++++++-
 1 file changed, 12 insertions(+), 1 deletion(-)

diff --git a/tests/TestSuite_pvp_qemu_multi_paths_port_restart.py b/tests/TestSuite_pvp_qemu_multi_paths_port_restart.py
index 601000e..ad15b2d 100644
--- a/tests/TestSuite_pvp_qemu_multi_paths_port_restart.py
+++ b/tests/TestSuite_pvp_qemu_multi_paths_port_restart.py
@@ -197,6 +197,15 @@ class TestPVPQemuMultiPathPortRestart(TestCase):
         results_row.append(Cycle)
         self.result_table_add(results_row)
 
+    @property
+    def check_value(self):
+        check_dict = dict.fromkeys(self.frame_sizes)
+        linerate = {64: 0.085, 128: 0.12, 256: 0.20, 512: 0.35, 1024: 0.50, 1280: 0.55, 1518: 0.60}
+        for size in self.frame_sizes:
+            speed = self.wirespeed(self.nic, size, self.number_of_ports)
+            check_dict[size] = round(speed * linerate[size], 2)
+        return check_dict
+
     def calculate_avg_throughput(self, frame_size):
         """
         start to send packet and get the throughput
@@ -214,7 +223,9 @@ class TestPVPQemuMultiPathPortRestart(TestCase):
         traffic_opt = {'delay': 5}
         _, pps = self.tester.pktgen.measure_throughput(stream_ids=streams, options=traffic_opt)
         Mpps = pps / 1000000.0
-        self.verify(Mpps > 0, "can not receive packets of frame size %d" % (frame_size))
+        self.verify(Mpps > self.check_value[frame_size],
+                    "%s of frame size %d speed verify failed, expect %s, result %s" % (
+                        self.running_case, frame_size, self.check_value[frame_size], Mpps))
         throughput = Mpps * 100 / \
                     float(self.wirespeed(self.nic, frame_size, 1))
         return Mpps, throughput
-- 
1.8.3.1


^ permalink raw reply	[flat|nested] 20+ messages in thread

* [dts] [PATCH V1 09/17]pvp_vhost_user_built_in_net_driver: change verified mpps value according to linerate
  2020-05-22  8:31 [dts] [PATCH V1 00/17] change virtio case mpps verification value Xiao Qimai
                   ` (7 preceding siblings ...)
  2020-05-22  8:31 ` [dts] [PATCH V1 08/17]pvp_qemu_multi_paths_port_restart: " Xiao Qimai
@ 2020-05-22  8:32 ` " Xiao Qimai
  2020-05-22  8:32 ` [dts] [PATCH V1 10/17]pvp_virtio_user_2M_hugepages: " Xiao Qimai
                   ` (8 subsequent siblings)
  17 siblings, 0 replies; 20+ messages in thread
From: Xiao Qimai @ 2020-05-22  8:32 UTC (permalink / raw)
  To: dts; +Cc: Xiao Qimai

Signed-off-by: Xiao Qimai <qimaix.xiao@intel.com>
---
 tests/TestSuite_pvp_vhost_user_built_in_net_driver.py | 13 ++++++++++++-
 1 file changed, 12 insertions(+), 1 deletion(-)

diff --git a/tests/TestSuite_pvp_vhost_user_built_in_net_driver.py b/tests/TestSuite_pvp_vhost_user_built_in_net_driver.py
index cfdd704..4a6b9f3 100644
--- a/tests/TestSuite_pvp_vhost_user_built_in_net_driver.py
+++ b/tests/TestSuite_pvp_vhost_user_built_in_net_driver.py
@@ -129,6 +129,15 @@ class TestPVPVhostUserBuiltInNetDriver(TestCase):
         self.verify("Error" not in out, "compilation l3fwd-power error")
         self.verify("No such" not in out, "Compilation error")
 
+    @property
+    def check_value(self):
+        check_dict = dict.fromkeys(self.frame_sizes)
+        linerate = {64: 0.085, 128: 0.12, 256: 0.20, 512: 0.35, 1024: 0.50, 1280: 0.55, 1518: 0.60}
+        for size in self.frame_sizes:
+            speed = self.wirespeed(self.nic, size, self.number_of_ports)
+            check_dict[size] = round(speed * linerate[size], 2)
+        return check_dict
+
     def send_and_verify(self):
         """
         Send packet with packet generator and verify
@@ -150,7 +159,9 @@ class TestPVPVhostUserBuiltInNetDriver(TestCase):
             trans_options={'delay':5, 'duration': 20}
             _, pps = self.tester.pktgen.measure_throughput(stream_ids=streams, options=trans_options)
             Mpps = pps / 1000000.0
-            self.verify(Mpps > 0, "%s can not receive packets of frame size %d" % (self.running_case, frame_size))
+            self.verify(Mpps > self.check_value[frame_size],
+                        "%s of frame size %d speed verify failed, expect %s, result %s" % (
+                            self.running_case, frame_size, self.check_value[frame_size], Mpps))
             throughput = Mpps * 100 / \
                      float(self.wirespeed(self.nic, frame_size, 1))
 
-- 
1.8.3.1


^ permalink raw reply	[flat|nested] 20+ messages in thread

* [dts] [PATCH V1 10/17]pvp_virtio_user_2M_hugepages: change verified mpps value according to linerate
  2020-05-22  8:31 [dts] [PATCH V1 00/17] change virtio case mpps verification value Xiao Qimai
                   ` (8 preceding siblings ...)
  2020-05-22  8:32 ` [dts] [PATCH V1 09/17]pvp_vhost_user_built_in_net_driver: " Xiao Qimai
@ 2020-05-22  8:32 ` " Xiao Qimai
  2020-05-22  8:32 ` [dts] [PATCH V1 11/17]pvp_virtio_user_4k_pages: " Xiao Qimai
                   ` (7 subsequent siblings)
  17 siblings, 0 replies; 20+ messages in thread
From: Xiao Qimai @ 2020-05-22  8:32 UTC (permalink / raw)
  To: dts; +Cc: Xiao Qimai

Signed-off-by: Xiao Qimai <qimaix.xiao@intel.com>
---
 tests/TestSuite_pvp_virtio_user_2M_hugepages.py | 13 ++++++++++++-
 1 file changed, 12 insertions(+), 1 deletion(-)

diff --git a/tests/TestSuite_pvp_virtio_user_2M_hugepages.py b/tests/TestSuite_pvp_virtio_user_2M_hugepages.py
index 71df7d5..0539948 100644
--- a/tests/TestSuite_pvp_virtio_user_2M_hugepages.py
+++ b/tests/TestSuite_pvp_virtio_user_2M_hugepages.py
@@ -92,6 +92,15 @@ class TestPVPVirtioWith2Mhuge(TestCase):
         self.table_header.append("% linerate")
         self.result_table_create(self.table_header)
 
+    @property
+    def check_value(self):
+        check_dict = dict.fromkeys(self.frame_sizes)
+        linerate = {64: 0.085, 128: 0.12, 256: 0.20, 512: 0.35, 1024: 0.50, 1280: 0.55, 1518: 0.60}
+        for size in self.frame_sizes:
+            speed = self.wirespeed(self.nic, size, self.number_of_ports)
+            check_dict[size] = round(speed * linerate[size], 2)
+        return check_dict
+
     def send_and_verify(self):
         """
         Send packet with packet generator and verify
@@ -112,7 +121,9 @@ class TestPVPVirtioWith2Mhuge(TestCase):
                             None, self.tester.pktgen)
             _, pps = self.tester.pktgen.measure_throughput(stream_ids=streams)
             Mpps = pps / 1000000.0
-            self.verify(Mpps > 0, "%s can not receive packets" % (self.running_case))
+            self.verify(Mpps > self.check_value[frame_size],
+                        "%s of frame size %d speed verify failed, expect %s, result %s" % (
+                            self.running_case, frame_size, self.check_value[frame_size], Mpps))
             throughput = Mpps * 100 / \
                      float(self.wirespeed(self.nic, 64, 1))
 
-- 
1.8.3.1


^ permalink raw reply	[flat|nested] 20+ messages in thread

* [dts] [PATCH V1 11/17]pvp_virtio_user_4k_pages: change verified mpps value according to linerate
  2020-05-22  8:31 [dts] [PATCH V1 00/17] change virtio case mpps verification value Xiao Qimai
                   ` (9 preceding siblings ...)
  2020-05-22  8:32 ` [dts] [PATCH V1 10/17]pvp_virtio_user_2M_hugepages: " Xiao Qimai
@ 2020-05-22  8:32 ` " Xiao Qimai
  2020-05-22  8:32 ` [dts] [PATCH V1 12/17]pvp_virtio_user_multi_queues_port_restart: " Xiao Qimai
                   ` (6 subsequent siblings)
  17 siblings, 0 replies; 20+ messages in thread
From: Xiao Qimai @ 2020-05-22  8:32 UTC (permalink / raw)
  To: dts; +Cc: Xiao Qimai

Signed-off-by: Xiao Qimai <qimaix.xiao@intel.com>
---
 tests/TestSuite_pvp_virtio_user_4k_pages.py | 13 ++++++++++++-
 1 file changed, 12 insertions(+), 1 deletion(-)

diff --git a/tests/TestSuite_pvp_virtio_user_4k_pages.py b/tests/TestSuite_pvp_virtio_user_4k_pages.py
index 532b6c1..392ed06 100644
--- a/tests/TestSuite_pvp_virtio_user_4k_pages.py
+++ b/tests/TestSuite_pvp_virtio_user_4k_pages.py
@@ -96,6 +96,15 @@ class TestPvpVirtioUser4kPages(TestCase):
         self.table_header.append("% linerate")
         self.result_table_create(self.table_header)
 
+    @property
+    def check_value(self):
+        check_dict = dict.fromkeys(self.frame_sizes)
+        linerate = {64: 0.085, 128: 0.12, 256: 0.20, 512: 0.35, 1024: 0.50, 1280: 0.55, 1518: 0.60}
+        for size in self.frame_sizes:
+            speed = self.wirespeed(self.nic, size, self.number_of_ports)
+            check_dict[size] = round(speed * linerate[size], 2)
+        return check_dict
+
     def send_and_verify(self):
         """
         Send packet with packet generator and verify
@@ -114,7 +123,9 @@ class TestPvpVirtioUser4kPages(TestCase):
                             None, self.tester.pktgen)
             _, pps = self.tester.pktgen.measure_throughput(stream_ids=streams)
             Mpps = pps / 1000000.0
-            self.verify(Mpps > 0, "%s can not receive packets" % (self.running_case))
+            self.verify(Mpps > self.check_value[frame_size],
+                        "%s of frame size %d speed verify failed, expect %s, result %s" % (
+                            self.running_case, frame_size, self.check_value[frame_size], Mpps))
             throughput = Mpps * 100 / \
                      float(self.wirespeed(self.nic, 64, 1))
 
-- 
1.8.3.1


^ permalink raw reply	[flat|nested] 20+ messages in thread

* [dts] [PATCH V1 12/17]pvp_virtio_user_multi_queues_port_restart: change verified mpps value according to linerate
  2020-05-22  8:31 [dts] [PATCH V1 00/17] change virtio case mpps verification value Xiao Qimai
                   ` (10 preceding siblings ...)
  2020-05-22  8:32 ` [dts] [PATCH V1 11/17]pvp_virtio_user_4k_pages: " Xiao Qimai
@ 2020-05-22  8:32 ` " Xiao Qimai
  2020-05-22  8:32 ` [dts] [PATCH V1 13/17]vhost_dequeue_zero_copy: " Xiao Qimai
                   ` (5 subsequent siblings)
  17 siblings, 0 replies; 20+ messages in thread
From: Xiao Qimai @ 2020-05-22  8:32 UTC (permalink / raw)
  To: dts; +Cc: Xiao Qimai

Signed-off-by: Xiao Qimai <qimaix.xiao@intel.com>
---
 .../TestSuite_pvp_virtio_user_multi_queues_port_restart.py  | 13 ++++++++++++-
 1 file changed, 12 insertions(+), 1 deletion(-)

diff --git a/tests/TestSuite_pvp_virtio_user_multi_queues_port_restart.py b/tests/TestSuite_pvp_virtio_user_multi_queues_port_restart.py
index d948e15..ad3d2be 100644
--- a/tests/TestSuite_pvp_virtio_user_multi_queues_port_restart.py
+++ b/tests/TestSuite_pvp_virtio_user_multi_queues_port_restart.py
@@ -166,11 +166,22 @@ class TestPVPVirtioUserMultiQueuesPortRestart(TestCase):
         traffic_opt = {'delay': 5}
         _, pps = self.tester.pktgen.measure_throughput(stream_ids=streams, options=traffic_opt)
         Mpps = pps / 1000000.0
-        self.verify(Mpps > 0, "can not receive packets of frame size %d" % (frame_size))
+        self.verify(Mpps > self.check_value[frame_size],
+                    "%s of frame size %d speed verify failed, expect %s, result %s" % (
+                        self.running_case, frame_size, self.check_value[frame_size], Mpps))
         throughput = Mpps * 100 / \
                     float(self.wirespeed(self.nic, frame_size, 1))
         return Mpps, throughput
 
+    @property
+    def check_value(self):
+        check_dict = dict.fromkeys(self.frame_sizes)
+        linerate = {64: 0.085, 128: 0.12, 256: 0.20, 512: 0.35, 1024: 0.50, 1280: 0.55, 1518: 0.60}
+        for size in self.frame_sizes:
+            speed = self.wirespeed(self.nic, size, self.number_of_ports)
+            check_dict[size] = round(speed * linerate[size], 2)
+        return check_dict
+
     def send_and_verify(self, case_info):
         """
         start to send packets and verify it
-- 
1.8.3.1


^ permalink raw reply	[flat|nested] 20+ messages in thread

* [dts] [PATCH V1 13/17]vhost_dequeue_zero_copy: change verified mpps value according to linerate
  2020-05-22  8:31 [dts] [PATCH V1 00/17] change virtio case mpps verification value Xiao Qimai
                   ` (11 preceding siblings ...)
  2020-05-22  8:32 ` [dts] [PATCH V1 12/17]pvp_virtio_user_multi_queues_port_restart: " Xiao Qimai
@ 2020-05-22  8:32 ` " Xiao Qimai
  2020-05-22  8:32 ` [dts] [PATCH V1 14/17]vhost_multi_queue_qemu: " Xiao Qimai
                   ` (4 subsequent siblings)
  17 siblings, 0 replies; 20+ messages in thread
From: Xiao Qimai @ 2020-05-22  8:32 UTC (permalink / raw)
  To: dts; +Cc: Xiao Qimai

Signed-off-by: Xiao Qimai <qimaix.xiao@intel.com>
---
 tests/TestSuite_vhost_dequeue_zero_copy.py | 13 ++++++++++++-
 1 file changed, 12 insertions(+), 1 deletion(-)

diff --git a/tests/TestSuite_vhost_dequeue_zero_copy.py b/tests/TestSuite_vhost_dequeue_zero_copy.py
index 8fca47a..8332b8a 100644
--- a/tests/TestSuite_vhost_dequeue_zero_copy.py
+++ b/tests/TestSuite_vhost_dequeue_zero_copy.py
@@ -303,7 +303,9 @@ class TestVhostDequeueZeroCopy(TestCase):
             Mpps = stats[1] / 1000000.0
         # when the fwd mode is rxonly, we can not receive data, so should not verify it
         if fwd_mode != "rxonly":
-            self.verify(Mpps > 0, "can not receive packets of frame size %d" % (frame_size))
+            self.verify(Mpps > self.check_value[frame_size],
+                        "%s of frame size %d speed verify failed, expect %s, result %s" % (
+                            self.running_case, frame_size, self.check_value[frame_size], Mpps))
         throughput = Mpps * 100 / \
                     float(self.wirespeed(self.nic, frame_size, 1))
         return Mpps, throughput
@@ -335,6 +337,15 @@ class TestVhostDequeueZeroCopy(TestCase):
 
         self.vhost_user.send_expect("start", "testpmd> ", 60)
 
+    @property
+    def check_value(self):
+        check_dict = dict.fromkeys(self.frame_sizes)
+        linerate = {64: 0.085, 128: 0.12, 256: 0.20, 512: 0.35, 1024: 0.50, 1280: 0.55, 1518: 0.60}
+        for size in self.frame_sizes:
+            speed = self.wirespeed(self.nic, size, self.number_of_ports)
+            check_dict[size] = round(speed * linerate[size], 2)
+        return check_dict
+
     def send_and_verify_throughput(self, cycle="", fwd_mode=""):
         """
         start to send packets and verify it
-- 
1.8.3.1


^ permalink raw reply	[flat|nested] 20+ messages in thread

* [dts] [PATCH V1 14/17]vhost_multi_queue_qemu: change verified mpps value according to linerate
  2020-05-22  8:31 [dts] [PATCH V1 00/17] change virtio case mpps verification value Xiao Qimai
                   ` (12 preceding siblings ...)
  2020-05-22  8:32 ` [dts] [PATCH V1 13/17]vhost_dequeue_zero_copy: " Xiao Qimai
@ 2020-05-22  8:32 ` " Xiao Qimai
  2020-05-22  8:32 ` [dts] [PATCH V1 15/17]virtio_pvp_regression: " Xiao Qimai
                   ` (3 subsequent siblings)
  17 siblings, 0 replies; 20+ messages in thread
From: Xiao Qimai @ 2020-05-22  8:32 UTC (permalink / raw)
  To: dts; +Cc: Xiao Qimai

Signed-off-by: Xiao Qimai <qimaix.xiao@intel.com>
---
 tests/TestSuite_vhost_multi_queue_qemu.py | 13 ++++++++++++-
 1 file changed, 12 insertions(+), 1 deletion(-)

diff --git a/tests/TestSuite_vhost_multi_queue_qemu.py b/tests/TestSuite_vhost_multi_queue_qemu.py
index c111846..b440b95 100644
--- a/tests/TestSuite_vhost_multi_queue_qemu.py
+++ b/tests/TestSuite_vhost_multi_queue_qemu.py
@@ -137,6 +137,15 @@ class TestVhostMultiQueueQemu(TestCase):
         self.verify(len(cores) >= 3, "Insufficient cores for speed testing, add the cpu number in cfg file.")
         self.vm_coremask = utils.create_mask(cores)
 
+    @property
+    def check_value(self):
+        check_dict = dict.fromkeys(self.frame_sizes)
+        linerate = {64: 0.085, 128: 0.12, 256: 0.20, 512: 0.35, 1024: 0.50, 1280: 0.55, 1518: 0.60}
+        for size in self.frame_sizes:
+            speed = self.wirespeed(self.nic, size, self.number_of_ports)
+            check_dict[size] = round(speed * linerate[size], 2)
+        return check_dict
+
     def vhost_performance(self):
         """
         Verify the testpmd can receive and forward the data
@@ -168,7 +177,9 @@ class TestVhostMultiQueueQemu(TestCase):
                                      self.number_of_ports))
             data_row = [frame_size, str(Mpps), str(pct), "Mergeable Multiqueue Performance"]
             self.result_table_add(data_row)
-            self.verify(Mpps != 0, "The receive data of frame-size: %d is 0" % frame_size)
+            self.verify(Mpps > self.check_value[frame_size],
+                        "%s of frame size %d speed verify failed, expect %s, result %s" % (
+                            self.running_case, frame_size, self.check_value[frame_size], Mpps))
         self.result_table_print()
 
     def send_and_verify(self, verify_type):
-- 
1.8.3.1


^ permalink raw reply	[flat|nested] 20+ messages in thread

* [dts] [PATCH V1 15/17]virtio_pvp_regression: change verified mpps value according to linerate
  2020-05-22  8:31 [dts] [PATCH V1 00/17] change virtio case mpps verification value Xiao Qimai
                   ` (13 preceding siblings ...)
  2020-05-22  8:32 ` [dts] [PATCH V1 14/17]vhost_multi_queue_qemu: " Xiao Qimai
@ 2020-05-22  8:32 ` " Xiao Qimai
  2020-05-22  8:32 ` [dts] [PATCH V1 16/17]virtio_user_for_container_networking: " Xiao Qimai
                   ` (2 subsequent siblings)
  17 siblings, 0 replies; 20+ messages in thread
From: Xiao Qimai @ 2020-05-22  8:32 UTC (permalink / raw)
  To: dts; +Cc: Xiao Qimai

Signed-off-by: Xiao Qimai <qimaix.xiao@intel.com>
---
 tests/TestSuite_virtio_pvp_regression.py | 13 ++++++++++++-
 1 file changed, 12 insertions(+), 1 deletion(-)

diff --git a/tests/TestSuite_virtio_pvp_regression.py b/tests/TestSuite_virtio_pvp_regression.py
index 6a67fed..46621fb 100644
--- a/tests/TestSuite_virtio_pvp_regression.py
+++ b/tests/TestSuite_virtio_pvp_regression.py
@@ -302,6 +302,15 @@ class TestVirtioPVPRegression(TestCase):
 
         self.vhost.send_expect("start", "testpmd> ", 60)
 
+    @property
+    def check_value(self):
+        check_dict = dict.fromkeys(self.frame_sizes)
+        linerate = {64: 0.085, 128: 0.12, 256: 0.20, 512: 0.35, 1024: 0.50, 1280: 0.55, 1518: 0.60}
+        for size in self.frame_sizes:
+            speed = self.wirespeed(self.nic, size, self.number_of_ports)
+            check_dict[size] = round(speed * linerate[size], 2)
+        return check_dict
+
     def send_verify(self, case_info, qemu_version, tag):
         for frame_size in self.frame_sizes:
             info = "Running test %s, and %d frame size." % (self.running_case, frame_size)
@@ -326,7 +335,9 @@ class TestVirtioPVPRegression(TestCase):
             _, pps = self.tester.pktgen.measure_throughput(stream_ids=streams, options=traffic_opt)
             Mpps = pps / 1000000.0
             pct = Mpps * 100 / float(self.wirespeed(self.nic, frame_size, 1))
-            self.verify(Mpps != 0, "can not received data of frame size %d" % frame_size)
+            self.verify(Mpps > self.check_value[frame_size],
+                        "%s of frame size %d speed verify failed, expect %s, result %s" % (
+                            self.running_case, frame_size, self.check_value[frame_size], Mpps))
             # check each queue has data
             self.check_packets_of_each_queue(frame_size)
             # update print table info
-- 
1.8.3.1


^ permalink raw reply	[flat|nested] 20+ messages in thread

* [dts] [PATCH V1 16/17]virtio_user_for_container_networking: change verified mpps value according to linerate
  2020-05-22  8:31 [dts] [PATCH V1 00/17] change virtio case mpps verification value Xiao Qimai
                   ` (14 preceding siblings ...)
  2020-05-22  8:32 ` [dts] [PATCH V1 15/17]virtio_pvp_regression: " Xiao Qimai
@ 2020-05-22  8:32 ` " Xiao Qimai
  2020-05-22  8:32 ` [dts] [PATCH V1 17/17]vm2vm_virtio_pmd: change verified mpps value to 0.5 Xiao Qimai
  2020-05-22  8:51 ` [dts] [PATCH V1 00/17] change virtio case mpps verification value Xiao, QimaiX
  17 siblings, 0 replies; 20+ messages in thread
From: Xiao Qimai @ 2020-05-22  8:32 UTC (permalink / raw)
  To: dts; +Cc: Xiao Qimai

Signed-off-by: Xiao Qimai <qimaix.xiao@intel.com>
---
 tests/TestSuite_virtio_user_for_container_networking.py | 13 ++++++++++++-
 1 file changed, 12 insertions(+), 1 deletion(-)

diff --git a/tests/TestSuite_virtio_user_for_container_networking.py b/tests/TestSuite_virtio_user_for_container_networking.py
index 1f9a666..65f60cb 100644
--- a/tests/TestSuite_virtio_user_for_container_networking.py
+++ b/tests/TestSuite_virtio_user_for_container_networking.py
@@ -101,6 +101,15 @@ class TestVirtioUserForContainer(TestCase):
         self.core_list_virtio_user = core_list[self.nb_cores+1:self.nb_cores*2+2]
         self.core_mask_virtio_user = utils.create_mask(core_list_virtio_user)
 
+    @property
+    def check_value(self):
+        check_dict = dict.fromkeys(self.frame_sizes)
+        linerate = {64: 0.085, 128: 0.12, 256: 0.20, 512: 0.35, 1024: 0.50, 1280: 0.55, 1518: 0.60}
+        for size in self.frame_sizes:
+            speed = self.wirespeed(self.nic, size, self.number_of_ports)
+            check_dict[size] = round(speed * linerate[size], 2)
+        return check_dict
+
     def send_and_verify(self):
         """
         Send packet with packet generator and verify
@@ -122,7 +131,9 @@ class TestVirtioUserForContainer(TestCase):
                         vm_config, self.tester.pktgen)
             _, pps = self.tester.pktgen.measure_throughput(stream_ids=streams)
             Mpps = pps / 1000000.0
-            self.verify(Mpps > 0, '%s can not receive packets of frame size %d' % (self.running_case, frame_size))
+            self.verify(Mpps > self.check_value[frame_size],
+                        "%s of frame size %d speed verify failed, expect %s, result %s" % (
+                            self.running_case, frame_size, self.check_value[frame_size], Mpps))
             throughput = Mpps * 100 / \
                      float(self.wirespeed(self.nic, frame_size, 1))
 
-- 
1.8.3.1


^ permalink raw reply	[flat|nested] 20+ messages in thread

* [dts] [PATCH V1 17/17]vm2vm_virtio_pmd: change verified mpps value to 0.5
  2020-05-22  8:31 [dts] [PATCH V1 00/17] change virtio case mpps verification value Xiao Qimai
                   ` (15 preceding siblings ...)
  2020-05-22  8:32 ` [dts] [PATCH V1 16/17]virtio_user_for_container_networking: " Xiao Qimai
@ 2020-05-22  8:32 ` Xiao Qimai
  2020-05-22  8:51 ` [dts] [PATCH V1 00/17] change virtio case mpps verification value Xiao, QimaiX
  17 siblings, 0 replies; 20+ messages in thread
From: Xiao Qimai @ 2020-05-22  8:32 UTC (permalink / raw)
  To: dts; +Cc: Xiao Qimai

Signed-off-by: Xiao Qimai <qimaix.xiao@intel.com>
---
 tests/TestSuite_vm2vm_virtio_pmd.py | 3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)

diff --git a/tests/TestSuite_vm2vm_virtio_pmd.py b/tests/TestSuite_vm2vm_virtio_pmd.py
index d4ffdc7..870b88d 100644
--- a/tests/TestSuite_vm2vm_virtio_pmd.py
+++ b/tests/TestSuite_vm2vm_virtio_pmd.py
@@ -1,3 +1,4 @@
+710703wyn
 # BSD LICENSE
 #
 # Copyright(c) <2019> Intel Corporation.
@@ -229,7 +230,7 @@ class TestVM2VMVirtioPMD(TestCase):
             result = lines.group(1)
             results += float(result)
         Mpps = results / (1000000 * 10)
-        self.verify(Mpps > 0, "%s can not receive packets" % self.running_case)
+        self.verify(Mpps > 0.5, "%s can not receive packets" % self.running_case)
         return Mpps
 
     def update_table_info(self, case_info, frame_size, Mpps, path):
-- 
1.8.3.1


^ permalink raw reply	[flat|nested] 20+ messages in thread

* Re: [dts] [PATCH V1 00/17] change virtio case mpps verification value
  2020-05-22  8:31 [dts] [PATCH V1 00/17] change virtio case mpps verification value Xiao Qimai
                   ` (16 preceding siblings ...)
  2020-05-22  8:32 ` [dts] [PATCH V1 17/17]vm2vm_virtio_pmd: change verified mpps value to 0.5 Xiao Qimai
@ 2020-05-22  8:51 ` Xiao, QimaiX
  17 siblings, 0 replies; 20+ messages in thread
From: Xiao, QimaiX @ 2020-05-22  8:51 UTC (permalink / raw)
  To: dts

Tested-by: Xiao, QimaiX <qimaix.xiao@intel.com>

Regards,
Xiao Qimai

> -----Original Message-----
> From: Xiao, QimaiX <qimaix.xiao@intel.com>
> Sent: Friday, May 22, 2020 4:32 PM
> To: dts@dpdk.org
> Cc: Xiao, QimaiX <qimaix.xiao@intel.com>
> Subject: [PATCH V1 00/17] change virtio case mpps verification value
> 
> *. change verify mpps from 0 to a certain value to avoid misc packet interfeer
> 
> Xiao Qimai (17):
>   change verfiy mpps to largger than 0.5
>   change verfiy mpps to largger than 5
>   change verfiy mpps to largger than 0.5
>   change verified mpps value according to linerate
>   change verified mpps value according to linerate
>   change verified mpps value according to linerate
>   change verified mpps value according to linerate
>   change verified mpps value according to linerate
>   change verified mpps value according to linerate
>   change verified mpps value according to linerate
>   change verified mpps value according to linerate
>   change verified mpps value according to linerate
>   change verified mpps value according to linerate
>   change verified mpps value according to linerate
>   change verified mpps value according to linerate
>   change verified mpps value according to linerate
>   change verified mpps value to 0.5
> 
>  .../TestSuite_loopback_multi_paths_port_restart.py |  2 +-
>  tests/TestSuite_loopback_multi_queues.py           |  2 +-
>  .../TestSuite_loopback_virtio_user_server_mode.py  |  2 +-
>  tests/TestSuite_pvp_diff_qemu_version.py           | 13 ++++-
>  tests/TestSuite_pvp_multi_paths_performance.py     | 56 +++++++++++++-
> --------
>  ...vp_multi_paths_vhost_single_core_performance.py | 14 +++++-
>   ...p_multi_paths_virtio_single_core_performance.py | 14 +++++-
>   .../TestSuite_pvp_qemu_multi_paths_port_restart.py | 13 ++++-
>   ...TestSuite_pvp_vhost_user_built_in_net_driver.py | 13 ++++-
>  tests/TestSuite_pvp_virtio_user_2M_hugepages.py    | 13 ++++-
>  tests/TestSuite_pvp_virtio_user_4k_pages.py        | 13 ++++-
>  ...te_pvp_virtio_user_multi_queues_port_restart.py | 13 ++++-
>  tests/TestSuite_vhost_dequeue_zero_copy.py         | 13 ++++-
>  tests/TestSuite_vhost_multi_queue_qemu.py          | 13 ++++-
>  tests/TestSuite_virtio_pvp_regression.py           | 13 ++++-
>  ...stSuite_virtio_user_for_container_networking.py | 13 ++++-
>  tests/TestSuite_vm2vm_virtio_pmd.py                |  3 +-
>  17 files changed, 182 insertions(+), 41 deletions(-)
> 
> --
> 1.8.3.1


^ permalink raw reply	[flat|nested] 20+ messages in thread

* Re: [dts] [PATCH V1 03/17]loopback_virtio_user_server_mode: change verfiy mpps to largger than 0.5
  2020-05-22  8:31 ` [dts] [PATCH V1 03/17]loopback_virtio_user_server_mode: change verfiy mpps to largger than 0.5 Xiao Qimai
@ 2020-06-01  3:27   ` Wang, Yinan
  0 siblings, 0 replies; 20+ messages in thread
From: Wang, Yinan @ 2020-06-01  3:27 UTC (permalink / raw)
  To: Xiao, QimaiX, dts; +Cc: Xiao, QimaiX

Hi Qimai,

This expected check value is lower than reality, may need change dts code and this expected value in V2.

BR,
Yinan
> -----Original Message-----
> From: dts <dts-bounces@dpdk.org> On Behalf Of Xiao Qimai
> Sent: 2020^[$BG/^[(B5^[$B7n^[(B22^[$BF|^[(B 16:32
> To: dts@dpdk.org
> Cc: Xiao, QimaiX <QimaiX.Xiao@intel.com>
> Subject: [dts] [PATCH V1 03/17]loopback_virtio_user_server_mode: change
> verfiy mpps to largger than 0.5
> 
> Signed-off-by: Xiao Qimai <qimaix.xiao@intel.com>
> ---
>  tests/TestSuite_loopback_virtio_user_server_mode.py | 2 +-
>  1 file changed, 1 insertion(+), 1 deletion(-)
> 
> diff --git a/tests/TestSuite_loopback_virtio_user_server_mode.py
> b/tests/TestSuite_loopback_virtio_user_server_mode.py
> index 32ee1c3..afc4108 100644
> --- a/tests/TestSuite_loopback_virtio_user_server_mode.py
> +++ b/tests/TestSuite_loopback_virtio_user_server_mode.py
> @@ -183,7 +183,7 @@ class TestLoopbackVirtioUserServerMode(TestCase):
>              result = lines.group(1)
>              results += float(result)
>          Mpps = results / (1000000 * 10)
> -        self.verify(Mpps > 0, "port can not receive packets")
> +        self.verify(Mpps > 0.5, "port can not receive packets")
> 
>          results_row.append(case_info)
>          results_row.append('64')
> --
> 1.8.3.1


^ permalink raw reply	[flat|nested] 20+ messages in thread

end of thread, back to index

Thread overview: 20+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2020-05-22  8:31 [dts] [PATCH V1 00/17] change virtio case mpps verification value Xiao Qimai
2020-05-22  8:31 ` [dts] [PATCH V1 01/17]loopback_multi_paths_port_restart: change verfiy mpps to largger than 0.5 Xiao Qimai
2020-05-22  8:31 ` [dts] [PATCH V1 02/17]loopback_multi_queues: change verfiy mpps to largger than 5 Xiao Qimai
2020-05-22  8:31 ` [dts] [PATCH V1 03/17]loopback_virtio_user_server_mode: change verfiy mpps to largger than 0.5 Xiao Qimai
2020-06-01  3:27   ` Wang, Yinan
2020-05-22  8:31 ` [dts] [PATCH V1 04/17]pvp_diff_qemu_version: change verified mpps value according to linerate Xiao Qimai
2020-05-22  8:31 ` [dts] [PATCH V1 05/17]pvp_multi_paths_performance: " Xiao Qimai
2020-05-22  8:31 ` [dts] [PATCH V1 06/17]pvp_multi_paths_vhost_single_core_performance: " Xiao Qimai
2020-05-22  8:31 ` [dts] [PATCH V1 07/17]pvp_multi_paths_virtio_single_core_performance: " Xiao Qimai
2020-05-22  8:31 ` [dts] [PATCH V1 08/17]pvp_qemu_multi_paths_port_restart: " Xiao Qimai
2020-05-22  8:32 ` [dts] [PATCH V1 09/17]pvp_vhost_user_built_in_net_driver: " Xiao Qimai
2020-05-22  8:32 ` [dts] [PATCH V1 10/17]pvp_virtio_user_2M_hugepages: " Xiao Qimai
2020-05-22  8:32 ` [dts] [PATCH V1 11/17]pvp_virtio_user_4k_pages: " Xiao Qimai
2020-05-22  8:32 ` [dts] [PATCH V1 12/17]pvp_virtio_user_multi_queues_port_restart: " Xiao Qimai
2020-05-22  8:32 ` [dts] [PATCH V1 13/17]vhost_dequeue_zero_copy: " Xiao Qimai
2020-05-22  8:32 ` [dts] [PATCH V1 14/17]vhost_multi_queue_qemu: " Xiao Qimai
2020-05-22  8:32 ` [dts] [PATCH V1 15/17]virtio_pvp_regression: " Xiao Qimai
2020-05-22  8:32 ` [dts] [PATCH V1 16/17]virtio_user_for_container_networking: " Xiao Qimai
2020-05-22  8:32 ` [dts] [PATCH V1 17/17]vm2vm_virtio_pmd: change verified mpps value to 0.5 Xiao Qimai
2020-05-22  8:51 ` [dts] [PATCH V1 00/17] change virtio case mpps verification value Xiao, QimaiX

test suite reviews and discussions

Archives are clonable:
	git clone --mirror http://inbox.dpdk.org/dts/0 dts/git/0.git

	# If you have public-inbox 1.1+ installed, you may
	# initialize and index your mirror using the following commands:
	public-inbox-init -V2 dts dts/ http://inbox.dpdk.org/dts \
		dts@dpdk.org
	public-inbox-index dts


Newsgroup available over NNTP:
	nntp://inbox.dpdk.org/inbox.dpdk.dts


AGPL code for this site: git clone https://public-inbox.org/ public-inbox