[dts] [DTS][PATCH 1/2] unit test: add memcpy performance unit test plan

[dts] [DTS][PATCH 1/2] unit test: add memcpy performance unit test plan

[Jingguo Fu]

From: Jingguo Fu <jingguox.fu@intel.comwq>

    case name: memcpy_perf_analysis
          it will show four ratio value for the performance

Signed-off-by: Jingguo Fu <jingguox.fu@intel.comwq>
---
 test_plans/unit_tests_memcpy_perf_test_plan.rst | 69 +++++++++++++++++++++++++
 1 file changed, 69 insertions(+)
 create mode 100644 test_plans/unit_tests_memcpy_perf_test_plan.rst

diff --git a/test_plans/unit_tests_memcpy_perf_test_plan.rst b/test_plans/unit_tests_memcpy_perf_test_plan.rst
new file mode 100644
index 0000000..35b646b
--- /dev/null
+++ b/test_plans/unit_tests_memcpy_perf_test_plan.rst
@@ -0,0 +1,69 @@
+.. Copyright (c) <2010, 2016>, Intel Corporation
+   All rights reserved.
+   
+   Redistribution and use in source and binary forms, with or without
+   modification, are permitted provided that the following conditions
+   are met:
+   
+   - Redistributions of source code must retain the above copyright
+     notice, this list of conditions and the following disclaimer.
+   
+   - Redistributions in binary form must reproduce the above copyright
+     notice, this list of conditions and the following disclaimer in
+     the documentation and/or other materials provided with the
+     distribution.
+   
+   - Neither the name of Intel Corporation nor the names of its
+     contributors may be used to endorse or promote products derived
+     from this software without specific prior written permission.
+   
+   THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
+   "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
+   LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS
+   FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE
+   COPYRIGHT OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT,
+   INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES
+   (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR
+   SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION)
+   HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT,
+   STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE)
+   ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED
+   OF THE POSSIBILITY OF SUCH DAMAGE.
+
+======================
+Test suite description
+======================
+
+This document is for dpdk unit test: memcpy performance auto test.
+For this suite, manily aims to analysis the cache to cache memcopy
+performance after run the memcpy performance unit test.
+
+The performance data is divided into four parts. Each part has two
+colums data for cache to cache performance. After extract the data,
+compute the geomatric mean for each colum, and then let the second
+geomatric divide the first goematric mean, then get the ratio of the
+two geomatric means.
+
+if the ratio greater than 1, then the test got a great performance.
+
+
+=============
+Prerequisites
+=============
+
+Test Case: memcpy_perf_analysis
+===============================
+
+1. run memcpy_perf unit test
+    ./x86_64-native-linuxapp-gcc/app/test -c f -n 1 
+    memcpy_perf_autotest
+    quit
+    performance data stored in  memcpy_perf_auto.log file
+
+2. copy log file to current server
+3. analysis the data
++--------------------+-----------------------+------------------+---------------------+
+| 32B Aligned(ticks) | 32B Aligned CT(ticks) | Unaligned(ticks) | Unaligned CT(ticks) |
++====================+=======================+==================+=====================+
+| 1.893              | 2.509                 | 1.507            | 1.500               |
++--------------------+-----------------------+------------------+---------------------+
-- 
1.9.3

[dts] [DTS][PATCH 2/2] unit test: add memcpy performance unit test case

[Jingguo Fu]

From: Jingguo Fu <jingguox.fu@intel.comwq>

    case name: memcpy_perf_analysis

Signed-off-by: Jingguo Fu <jingguox.fu@intel.comwq>
---
 tests/TestSuite_unit_tests_memcpy_perf.py | 165 ++++++++++++++++++++++++++++++
 1 file changed, 165 insertions(+)
 create mode 100644 tests/TestSuite_unit_tests_memcpy_perf.py

diff --git a/tests/TestSuite_unit_tests_memcpy_perf.py b/tests/TestSuite_unit_tests_memcpy_perf.py
new file mode 100644
index 0000000..668a684
--- /dev/null
+++ b/tests/TestSuite_unit_tests_memcpy_perf.py
@@ -0,0 +1,165 @@
+# BSD LICENSE
+#
+# Copyright(c) 2010-2014 Intel Corporation. All rights reserved.
+# All rights reserved.
+#
+# Redistribution and use in source and binary forms, with or without
+# modification, are permitted provided that the following conditions
+# are met:
+#
+#   * Redistributions of source code must retain the above copyright
+#     notice, this list of conditions and the following disclaimer.
+#   * Redistributions in binary form must reproduce the above copyright
+#     notice, this list of conditions and the following disclaimer in
+#     the documentation and/or other materials provided with the
+#     distribution.
+#   * Neither the name of Intel Corporation nor the names of its
+#     contributors may be used to endorse or promote products derived
+#     from this software without specific prior written permission.
+#
+# THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
+# "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
+# LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
+# A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
+# OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
+# SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
+# LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
+# DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
+# THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
+# (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
+# OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
+
+
+"""
+DPDK Test suite.
+Test purly memcpy test.
+"""
+import os
+import re
+import time
+import math
+
+import dts
+from test_case import TestCase
+from pmd_output import PmdOutput
+
+
+def geomatric_mean_list(list):
+    mult = 1.0
+    for data in list:
+        mult = mult * data
+    return math.pow(mult, 1.0 / len(list))
+
+class TestMemcpy(TestCase):
+
+    def get_cpu_model_name(self):
+        out = self.dut.send_expect("lscpu |grep 'Model name'", "#", 30)
+        module_name = out.split(':')[1].strip("^' '")
+        return module_name
+
+    def set_up_all(self):
+        """
+        Run at the start of each test suite.
+        Prerequisites:
+            platform support
+        """
+        self.cpu_model = self.get_cpu_model_name()
+
+    def set_up(self):
+        """
+        Run before each test case.
+        Nothing to do.
+        """
+        pass
+
+    def test_memcpy_perf_analysis(self):
+        log_name = "memcpy_perf_auto.log"
+        self.dut.send_expect('rm -rf %s' % log_name), "#", 10)
+        memcpy_perf_cmdline = "echo -e 'memcpy_perf_autotest\\nquit\\n' | ./x86_64-native-linuxapp-gcc/app/test -c f -n 1"
+        self.dut.send_expect('%s > %s' % (memcpy_perf_cmdline, log_name), "#", 400)
+        self.dut.session.copy_file_from("/root/dpdk/%s" % log_name)
+        fp = open(log_name, "r")
+        log = fp.readlines()
+        fp.close()
+        self.verify("Test OK" in log[-2], "Test memcpy perf unit test failed")
+
+        perf_flag = False
+        aligned_buf = []
+        for line in log:
+            if "32B aligned" in line:
+                perf_flag = True
+                continue
+            if perf_flag and "-------" in line:
+                perf_flag = False
+                break
+            if perf_flag:
+                aligned_buf.append(line)
+        aligned_perf_res = []
+        for line in aligned_buf:
+            res_data = line.split()
+            aligned_perf_res.append(float(res_data[3]) / float(res_data[1]))
+        aligned_perf_average = geomatric_mean_list(aligned_perf_res)
+        print "aligned perf average is %.3f " % aligned_perf_average
+
+        aligned_ctc_buf = []
+        for line in log:
+            if line.startswith("-------"):
+                perf_flag = True
+                continue
+            if line.startswith("======") and "Unaligned" in line:
+                perf_flag = False
+                break
+            if perf_flag and line.startswith('C'):
+                aligned_ctc_buf.append(line)
+        aligned_ctc_perf_res = []
+        for line in aligned_ctc_buf:
+            res_data = line.split()
+            aligned_ctc_perf_res.append(float(res_data[4])/float(res_data[2]))
+        aligned_ctc_perf_average = geomatric_mean_list(aligned_ctc_perf_res)
+        print "aligned ctc perf average is %.3f " % aligned_ctc_perf_average
+
+        unaligned_buf = []
+        for line in log:
+            if line.startswith("====") and "Unaligned" in line:
+                perf_flag = True
+                continue
+            if perf_flag and line.startswith("-------"):
+                perf_flag = False
+                break
+            if perf_flag:
+                unaligned_buf.append(line)
+        unaligned_perf_res = []
+        for line in unaligned_buf:
+            res_data = line.split()
+            unaligned_perf_res.append(float(res_data[3])/float(res_data[1]))
+        unaligned_perf_average = geomatric_mean_list(unaligned_perf_res)
+        print "unaligned perf average is %.3f " % unaligned_perf_average
+
+        unaligned_ctc_buf = []
+        for line in log:
+            if line.startswith("=====") and "Unaligned" in line:
+                perf_flag = True
+                continue
+            if perf_flag and line.startswith("====="):
+                perf_flag = False
+                break
+            if perf_flag and line.startswith('C'):
+                unaligned_ctc_buf.append(line)
+        unaligned_ctc_perf_res = []
+        for line in unaligned_ctc_buf:
+            res_data = line.split()
+            unaligned_ctc_perf_res.append(float(res_data[4]) / float(res_data[2]))
+        unaligned_ctc_perf_average =  geomatric_mean_list(unaligned_ctc_perf_res)
+        print "unaligned ctc perf average is %.3f " % unaligned_ctc_perf_average
+        table_header = ["32B Aligned(ticks)", "32B Aligned CT(ticks)", 
+                      "Unaligned(ticks)", "Unaligned CT(ticks)"]
+        dts.results_table_add_header(table_header)
+        dts.results_table_add_row([aligned_perf_average, aligned_ctc_perf_average,
+                                  unaligned_perf_average, unaligned_ctc_perf_average])
+        dts.results_table_print()
+
+    def tear_down(self):
+        pass
+
+    def tear_down_all(self):
+        pass
-- 
1.9.3

Re: [dts] [DTS][PATCH 1/2] unit test: add memcpy performance unit test plan

[Liu, Yong]

Jingguo, some comments below.

> -----Original Message-----
> From: Fu, JingguoX
> Sent: Tuesday, March 08, 2016 1:48 PM
> To: dts@dpdk.org
> Cc: Liu, Yong; Xu, Qian Q; Jingguo Fu
> Subject: [DTS][PATCH 1/2] unit test: add memcpy performance unit test
> plan
> 
> From: Jingguo Fu <jingguox.fu@intel.comwq>
> 
>     case name: memcpy_perf_analysis
>           it will show four ratio value for the performance
> 
> Signed-off-by: Jingguo Fu <jingguox.fu@intel.comwq>
> ---
>  test_plans/unit_tests_memcpy_perf_test_plan.rst | 69
> +++++++++++++++++++++++++
>  1 file changed, 69 insertions(+)
>  create mode 100644 test_plans/unit_tests_memcpy_perf_test_plan.rst
> 
> diff --git a/test_plans/unit_tests_memcpy_perf_test_plan.rst
> b/test_plans/unit_tests_memcpy_perf_test_plan.rst
> new file mode 100644
> index 0000000..35b646b
> --- /dev/null
> +++ b/test_plans/unit_tests_memcpy_perf_test_plan.rst
> @@ -0,0 +1,69 @@
> +.. Copyright (c) <2010, 2016>, Intel Corporation
> +   All rights reserved.
> +
> +   Redistribution and use in source and binary forms, with or without
> +   modification, are permitted provided that the following conditions
> +   are met:
> +
> +   - Redistributions of source code must retain the above copyright
> +     notice, this list of conditions and the following disclaimer.
> +
> +   - Redistributions in binary form must reproduce the above copyright
> +     notice, this list of conditions and the following disclaimer in
> +     the documentation and/or other materials provided with the
> +     distribution.
> +
> +   - Neither the name of Intel Corporation nor the names of its
> +     contributors may be used to endorse or promote products derived
> +     from this software without specific prior written permission.
> +
> +   THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
> +   "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
> +   LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS
> +   FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE
> +   COPYRIGHT OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT,
> +   INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES
> +   (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR
> +   SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION)
> +   HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT,
> +   STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE)
> +   ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED
> +   OF THE POSSIBILITY OF SUCH DAMAGE.
> +
> +======================
> +Test suite description
> +======================
> +
> +This document is for dpdk unit test: memcpy performance auto test.
> +For this suite, manily aims to analysis the cache to cache memcopy
Typo, should be "mainly" and  "memcpy".

> +performance after run the memcpy performance unit test.
> +
> +The performance data is divided into four parts. Each part has two
> +colums data for cache to cache performance. After extract the data,

Could you show one basic sample for performance data, it’s hard to understand from your description.

> +compute the geomatric mean for each colum, and then let the second
> +geomatric divide the first goematric mean, then get the ratio of the
> +two geomatric means.
> +
> +if the ratio greater than 1, then the test got a great performance.
> +
> +
> +=============
> +Prerequisites
> +=============
> +
> +Test Case: memcpy_perf_analysis
> +===============================
> +
> +1. run memcpy_perf unit test
> +    ./x86_64-native-linuxapp-gcc/app/test -c f -n 1
> +    memcpy_perf_autotest
> +    quit
> +    performance data stored in  memcpy_perf_auto.log file
> +
> +2. copy log file to current server
> +3. analysis the data
> ++--------------------+-----------------------+------------------+-------
> --------------+
> +| 32B Aligned(ticks) | 32B Aligned CT(ticks) | Unaligned(ticks) |
> Unaligned CT(ticks) |
> ++====================+=======================+==================+=======
> ==============+
> +| 1.893              | 2.509                 | 1.507            | 1.500
> |
> ++--------------------+-----------------------+------------------+-------
> --------------+
Could you give basic description of "CT"， it’s hard to understand the meaning of it.

> --
> 1.9.3

Re: [dts] [DTS][PATCH 1/2] unit test: add memcpy performance unit test plan

[Fu, JingguoX]

Hi Marvin,

Update in V2 patch, thanks!

Regards,
Jingguo Fu

-----Original Message-----
From: Liu, Yong 
Sent: Wednesday, March 09, 2016 9:05 AM
To: Fu, JingguoX; dts@dpdk.org
Cc: Xu, Qian Q; Jingguo Fu
Subject: RE: [DTS][PATCH 1/2] unit test: add memcpy performance unit test plan

Jingguo, some comments below.

> -----Original Message-----
> From: Fu, JingguoX
> Sent: Tuesday, March 08, 2016 1:48 PM
> To: dts@dpdk.org
> Cc: Liu, Yong; Xu, Qian Q; Jingguo Fu
> Subject: [DTS][PATCH 1/2] unit test: add memcpy performance unit test 
> plan
> 
> From: Jingguo Fu <jingguox.fu@intel.comwq>
> 
>     case name: memcpy_perf_analysis
>           it will show four ratio value for the performance
> 
> Signed-off-by: Jingguo Fu <jingguox.fu@intel.comwq>
> ---
>  test_plans/unit_tests_memcpy_perf_test_plan.rst | 69
> +++++++++++++++++++++++++
>  1 file changed, 69 insertions(+)
>  create mode 100644 test_plans/unit_tests_memcpy_perf_test_plan.rst
> 
> diff --git a/test_plans/unit_tests_memcpy_perf_test_plan.rst
> b/test_plans/unit_tests_memcpy_perf_test_plan.rst
> new file mode 100644
> index 0000000..35b646b
> --- /dev/null
> +++ b/test_plans/unit_tests_memcpy_perf_test_plan.rst
> @@ -0,0 +1,69 @@
> +.. Copyright (c) <2010, 2016>, Intel Corporation
> +   All rights reserved.
> +
> +   Redistribution and use in source and binary forms, with or without
> +   modification, are permitted provided that the following conditions
> +   are met:
> +
> +   - Redistributions of source code must retain the above copyright
> +     notice, this list of conditions and the following disclaimer.
> +
> +   - Redistributions in binary form must reproduce the above copyright
> +     notice, this list of conditions and the following disclaimer in
> +     the documentation and/or other materials provided with the
> +     distribution.
> +
> +   - Neither the name of Intel Corporation nor the names of its
> +     contributors may be used to endorse or promote products derived
> +     from this software without specific prior written permission.
> +
> +   THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
> +   "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
> +   LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS
> +   FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE
> +   COPYRIGHT OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT,
> +   INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES
> +   (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR
> +   SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION)
> +   HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT,
> +   STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE)
> +   ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED
> +   OF THE POSSIBILITY OF SUCH DAMAGE.
> +
> +======================
> +Test suite description
> +======================
> +
> +This document is for dpdk unit test: memcpy performance auto test.
> +For this suite, manily aims to analysis the cache to cache memcopy
Typo, should be "mainly" and  "memcpy".

> +performance after run the memcpy performance unit test.
> +
> +The performance data is divided into four parts. Each part has two 
> +colums data for cache to cache performance. After extract the data,

Could you show one basic sample for performance data, it’s hard to understand from your description.

> +compute the geomatric mean for each colum, and then let the second 
> +geomatric divide the first goematric mean, then get the ratio of the 
> +two geomatric means.
> +
> +if the ratio greater than 1, then the test got a great performance.
> +
> +
> +=============
> +Prerequisites
> +=============
> +
> +Test Case: memcpy_perf_analysis
> +===============================
> +
> +1. run memcpy_perf unit test
> +    ./x86_64-native-linuxapp-gcc/app/test -c f -n 1
> +    memcpy_perf_autotest
> +    quit
> +    performance data stored in  memcpy_perf_auto.log file
> +
> +2. copy log file to current server
> +3. analysis the data
> ++--------------------+-----------------------+------------------+----
> ++--------------------+-----------------------+------------------+---
> --------------+
> +| 32B Aligned(ticks) | 32B Aligned CT(ticks) | Unaligned(ticks) |
> Unaligned CT(ticks) |
> ++====================+=======================+==================+====
> ++===
> ==============+
> +| 1.893              | 2.509                 | 1.507            | 1.500
> |
> ++--------------------+-----------------------+------------------+----
> ++--------------------+-----------------------+------------------+---
> --------------+
Could you give basic description of "CT"， it’s hard to understand the meaning of it.

> --
> 1.9.3