[dts] [PATCH v3] Add Cryptodev Unit Tests Cases

[dts] [PATCH v3] Add Cryptodev Unit Tests Cases

[Chen, Zhaoyan]

v3:

- Fix pep8 issues

v2:

- Update copyright dates
- Update copyright icon


---
 test_plans/unit_tests_cryptodev_test_plan.rst |  85 +++++++++++++
 tests/TestSuite_unit_tests_cryptodev.py       | 169 ++++++++++++++++++++++++++
 2 files changed, 254 insertions(+)
 create mode 100644 test_plans/unit_tests_cryptodev_test_plan.rst
 create mode 100644 tests/TestSuite_unit_tests_cryptodev.py

diff --git a/test_plans/unit_tests_cryptodev_test_plan.rst b/test_plans/unit_tests_cryptodev_test_plan.rst
new file mode 100644
index 0000000..7a60ff3
--- /dev/null
+++ b/test_plans/unit_tests_cryptodev_test_plan.rst
@@ -0,0 +1,85 @@
+.. Copyright (c) 2016-2017 Intel Corporation
+   All rights reserved.
+   
+   Redistribution and use in source and binary forms, with or without
+   modification, are permitted provided that the following conditions
+   are met:
+   
+   - Redistributions of source code must retain the above copyright
+     notice, this list of conditions and the following disclaimer.
+   
+   - Redistributions in binary form must reproduce the above copyright
+     notice, this list of conditions and the following disclaimer in
+     the documentation and/or other materials provided with the
+     distribution.
+   
+   - Neither the name of Intel Corporation nor the names of its
+     contributors may be used to endorse or promote products derived
+     from this software without specific prior written permission.
+   
+   THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
+   "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
+   LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS
+   FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE
+   COPYRIGHT OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT,
+   INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES
+   (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR
+   SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION)
+   HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT,
+   STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE)
+   ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED
+   OF THE POSSIBILITY OF SUCH DAMAGE.
+
+==============================================
+Testing of Cryptodev Unit tests
+==============================================
+
+
+Description
+===========
+
+This document provides the plan for testing Cryptodev API via Cryptodev unit tests.
+Unit tests include supported Hardware and Software PMD(polling mode device) and supported algorithms.
+Cryptodev API provides ability to do encryption/decryption by integrating QAT(Intel@ QuickAssist
+Technology) into DPDK. The QAT provides poll mode crypto driver support for 
+Intel@ QuickAssist Adapter 8950 hardware accelerator.
+
+The testing of Crytpodev API should be tested under either Intel QuickAssist Technology DH895xxC hardware
+accelerator or AES-NI library.
+
+This test suite will run all cryptodev related unit test cases. Alternatively, you could execute
+the unit tests manually by app/test DPDK application.
+
+Unit Test List
+==============
+
+- cryptodev_qat_autotest
+- cryptodev_qat_perftest
+- cryptodev_aesni_mb_perftest
+- cryptodev_sw_snow3g_perftest
+- cryptodev_qat_snow3g_perftest
+- cryptodev_aesni_gcm_perftest
+- cryptodev_openssl_perftest
+- cryptodev_qat_continual_perftest
+- cryptodev_aesni_mb_autotest
+- cryptodev_openssl_autotest
+- cryptodev_aesni_gcm_autotest
+- cryptodev_null_autotest
+- cryptodev_sw_snow3g_autotest
+- cryptodev_sw_kasumi_autotest
+- cryptodev_sw_zuc_autotest
+
+
+Test Case Setup
+===============
+
+1. Build DPDK and app/test app
+2. Bind cryptodev devices to igb_uio driver
+3. Manually verify the app/test by this command, as example, in your build folder
+* ./app/test -c 1 -n 1
+* RTE>> cryptodev_qat_autotest
+
+All Unit Test Cases are listed above. 
+
+Expected all tests could pass in testing.
+
diff --git a/tests/TestSuite_unit_tests_cryptodev.py b/tests/TestSuite_unit_tests_cryptodev.py
new file mode 100644
index 0000000..1fd3827
--- /dev/null
+++ b/tests/TestSuite_unit_tests_cryptodev.py
@@ -0,0 +1,169 @@
+# BSD LICENSE
+#
+# Copyright(c) 2016-2017 Intel Corporation. All rights reserved.
+# All rights reserved.
+#
+# Redistribution and use in source and binary forms, with or without
+# modification, are permitted provided that the following conditions
+# are met:
+#
+#   * Redistributions of source code must retain the above copyright
+#     notice, this list of conditions and the following disclaimer.
+#   * Redistributions in binary form must reproduce the above copyright
+#     notice, this list of conditions and the following disclaimer in
+#     the documentation and/or other materials provided with the
+#     distribution.
+#   * Neither the name of Intel Corporation nor the names of its
+#     contributors may be used to endorse or promote products derived
+#     from this software without specific prior written permission.
+#
+# THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
+# "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
+# LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
+# A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
+# OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
+# SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
+# LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
+# DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
+# THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
+# (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
+# OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
+
+import utils
+from test_case import TestCase
+
+
+class UnitTestsCryptodev(TestCase):
+
+    def set_up_all(self):
+
+        self.core_config = "1S/2C/1T"
+        self.number_of_ports = 1
+        self.dut_ports = self.dut.get_ports(self.nic)
+        self.verify(len(self.dut_ports) >= self.number_of_ports,
+                    "Not enough ports for " + self.nic)
+        self.ports_socket = self.dut.get_numa_id(self.dut_ports[0])
+
+        self.logger.info("core config = " + self.core_config)
+        self.logger.info("number of ports = " + str(self.number_of_ports))
+        self.logger.info("dut ports = " + str(self.dut_ports))
+        self.logger.info("ports_socket = " + str(self.ports_socket))
+
+        self.core_mask = utils.create_mask(self.dut.get_core_list(
+            self.core_config,
+            socket=self.ports_socket))
+        self.port_mask = utils.create_mask([self.dut_ports[0]])
+
+        self.tx_port = self.tester.get_local_port(self.dut_ports[0])
+        self.rx_port = self.tester.get_local_port(self.dut_ports[0])
+
+        self.tx_interface = self.tester.get_interface(self.tx_port)
+        self.rx_interface = self.tester.get_interface(self.rx_port)
+
+        self.logger.info("core mask = " + self.core_mask)
+        self.logger.info("port mask = " + self.port_mask)
+        self.logger.info("tx interface = " + self.tx_interface)
+        self.logger.info("rx interface = " + self.rx_interface)
+
+        # Rebuild the dpdk with cryptodev pmds CONFIG_RTE_LIBRTE_PMD_ZUC=n
+        self.dut.send_expect("export AESNI_MULTI_BUFFER_LIB_PATH=/root/ipsec_043/code/", "#")
+        self.dut.send_expect("export LIBSSO_SNOW3G_PATH=/root/libsso_snow3g/snow3g/", "#")
+        self.dut.send_expect("export LIBSSO_ZUC_PATH=/root/libsso_zuc.1.0.1.1-8/zuc", "#")
+        self.dut.send_expect("export LIBSSO_KASUMI_PATH=/root/LibSSO_0_3_1/isg_cid-wireless_libs/ciphers/kasumi/", "#")
+        self.dut.send_expect("sed -i 's/CONFIG_RTE_LIBRTE_PMD_QAT=n$/CONFIG_RTE_LIBRTE_PMD_QAT=y/' config/common_base", "# ")
+        self.dut.send_expect("sed -i 's/CONFIG_RTE_LIBRTE_PMD_AESNI_GCM=n$/CONFIG_RTE_LIBRTE_PMD_AESNI_GCM=y/' config/common_base", "# ")
+        self.dut.send_expect("sed -i 's/CONFIG_RTE_LIBRTE_PMD_NULL_CRYPTO=n$/CONFIG_RTE_LIBRTE_PMD_NULL_CRYPTO=y/' config/common_base", "# ")
+        self.dut.send_expect("sed -i 's/CONFIG_RTE_LIBRTE_PMD_SNOW3G=n$/CONFIG_RTE_LIBRTE_PMD_SNOW3G=y/' config/common_base", "# ")
+        self.dut.send_expect("sed -i 's/CONFIG_RTE_LIBRTE_PMD_KASUMI=n$/CONFIG_RTE_LIBRTE_PMD_KASUMI=y/' config/common_base", "# ")
+        self.dut.send_expect("sed -i 's/CONFIG_RTE_LIBRTE_PMD_OPENSSL=n$/CONFIG_RTE_LIBRTE_PMD_OPENSSL=y/' config/common_base", "# ")
+        self.dut.send_expect("sed -i 's/CONFIG_RTE_LIBRTE_PMD_AESNI_MB=n$/CONFIG_RTE_LIBRTE_PMD_AESNI_MB=y/' config/common_base", "# ")
+        self.dut.send_expect("sed -i 's/CONFIG_RTE_LIBRTE_PMD_ZUC=n$/CONFIG_RTE_LIBRTE_PMD_ZUC=y/' config/common_base", "# ")
+        self.dut.skip_setup = False
+        self.dut.build_install_dpdk(self.dut.target)
+
+        # Bind QAT VF devices
+        out = self.dut.send_expect("lspci -d:443|awk '{print $1}'", "# ", 10)
+        self.dut.send_expect('echo "8086 0443" > /sys/bus/pci/drivers/igb_uio/new_id', "# ", 10)
+        for line in out.replace("\r", "\n").replace("\n\n", "\n").split("\n"):
+            cmd = "echo 0000:{} > /sys/bus/pci/devices/0000\:{}/driver/unbind".format(line, line.replace(":", "\:"))
+            self.dut.send_expect(cmd, "# ", 10)
+            cmd = "echo 0000:{} > /sys/bus/pci/drivers/igb_uio/bind".format(line)
+            self.dut.send_expect(cmd, "# ", 10)
+
+    def set_up(self):
+        pass
+
+    def tear_down(self):
+        pass
+
+    def tear_down_all(self):
+        self.dut.send_expect("sed -i 's/CONFIG_RTE_LIBRTE_PMD_QAT=y$/CONFIG_RTE_LIBRTE_PMD_QAT=n/' config/common_base", "# ")
+        self.dut.send_expect("sed -i 's/CONFIG_RTE_LIBRTE_PMD_AESNI_GCM=y$/CONFIG_RTE_LIBRTE_PMD_AESNI_GCM=n/' config/common_base", "# ")
+        self.dut.send_expect("sed -i 's/CONFIG_RTE_LIBRTE_PMD_NULL_CRYPTO=y$/CONFIG_RTE_LIBRTE_PMD_NULL_CRYPTO=n/' config/common_base", "# ")
+        self.dut.send_expect("sed -i 's/CONFIG_RTE_LIBRTE_PMD_SNOW3G=y$/CONFIG_RTE_LIBRTE_PMD_SNOW3G=n/' config/common_base", "# ")
+        self.dut.send_expect("sed -i 's/CONFIG_RTE_LIBRTE_PMD_KASUMI=y$/CONFIG_RTE_LIBRTE_PMD_KASUMI=n/' config/common_base", "# ")
+        self.dut.send_expect("sed -i 's/CONFIG_RTE_LIBRTE_PMD_OPENSSL=y$/CONFIG_RTE_LIBRTE_PMD_OPENSSL=n/' config/common_base", "# ")
+        self.dut.send_expect("sed -i 's/CONFIG_RTE_LIBRTE_PMD_AESNI_MB=y$/CONFIG_RTE_LIBRTE_PMD_AESNI_MB=n/' config/common_base", "# ")
+        self.dut.send_expect("sed -i 's/CONFIG_RTE_LIBRTE_PMD_ZUC=y$/CONFIG_RTE_LIBRTE_PMD_ZUC=n/' config/common_base", "# ")
+
+    def test_cryptodev_qat_autotest(self):
+        self.__run_unit_test("cryptodev_qat_autotest")
+
+    def test_cryptodev_qat_perftest(self):
+        self.__run_unit_test("cryptodev_qat_perftest")
+
+    def test_cryptodev_aesni_mb_perftest(self):
+        self.__run_unit_test("cryptodev_aesni_mb_perftest")
+
+    def test_cryptodev_sw_snow3g_perftest(self):
+        self.__run_unit_test("cryptodev_sw_snow3g_perftest")
+
+    def test_cryptodev_qat_snow3g_perftest(self):
+        self.__run_unit_test("cryptodev_qat_snow3g_perftest")
+
+    def test_cryptodev_aesni_gcm_perftest(self):
+        self.__run_unit_test("cryptodev_aesni_gcm_perftest")
+
+    def test_cryptodev_openssl_perftest(self):
+        self.__run_unit_test("cryptodev_openssl_perftest")
+
+    def test_cryptodev_qat_continual_perftest(self):
+        self.__run_unit_test("cryptodev_qat_continual_perftest")
+
+    def test_cryptodev_aesni_mb_autotest(self):
+        self.__run_unit_test("cryptodev_aesni_mb_autotest")
+
+    def test_cryptodev_openssl_autotest(self):
+        self.__run_unit_test("cryptodev_openssl_autotest")
+
+    def test_cryptodev_aesni_gcm_autotest(self):
+        self.__run_unit_test("cryptodev_aesni_gcm_autotest")
+
+    def test_cryptodev_null_autotest(self):
+        self.__run_unit_test("cryptodev_null_autotest")
+
+    def test_cryptodev_sw_snow3g_autotest(self):
+        self.__run_unit_test("cryptodev_sw_snow3g_autotest")
+
+    def test_cryptodev_sw_kasumi_autotest(self):
+        self.__run_unit_test("cryptodev_sw_kasumi_autotest")
+
+    def test_cryptodev_sw_zuc_autotest(self):
+        self.__run_unit_test("cryptodev_sw_zuc_autotest")
+
+    def __run_unit_test(self, testsuite, timeout=600):
+        print "STEP_TEST: " + testsuite
+        self.dut.send_expect("dmesg -C", "# ", 30)
+        self.dut.send_expect("./{target}/app/test -n 1 -c 0xf".format(target=self.dut.target), "RTE>>", 30)
+        out = ""
+        try:
+            out = self.dut.send_expect(testsuite, "RTE>>", timeout)
+            self.dut.send_expect("quit", "# ", 30)
+        except Exception, ex:
+            print ex
+            print "run cryptodev_unit_tests process coredump"
+            self.dut.alt_session.send_expect("killall test", "# ", 30)
+            dmesg = self.dut.alt_session.send_expect("dmesg", "# ", 30)
+            print dmesg
+        print out
+        self.verify("Test OK" in out, "Test Failed")
-- 
1.9.3

Re: [dts] [PATCH v3] Add Cryptodev Unit Tests Cases

[Liu, Yong]

Zhaoyan,
Why try to recover test application when there's no response? Is that 
application not stable after run crypto unit test?
If there's one need to do that, please do it in tear_down function.

Thanks,
Marvin

On 03/08/2017 01:01 PM, Chen, Zhaoyan wrote:
> +        try:
> +            out = self.dut.send_expect(testsuite, "RTE>>", timeout)
> +            self.dut.send_expect("quit", "# ", 30)
> +        except Exception, ex:
> +            print ex
> +            print "run cryptodev_unit_tests process coredump"
> +            self.dut.alt_session.send_expect("killall test", "# ", 30)
> +            dmesg = self.dut.alt_session.send_expect("dmesg", "# ", 30)
> +            print dmesg
> +        print out

Re: [dts] [PATCH v3] Add Cryptodev Unit Tests Cases

[Chen, Zhaoyan]

I do not recover the application, just error handling.  
Since QAT HW unstable sometime.

Since just function test, (not stable test), 
So, for each test case will launch a separated test application. 

But, there is another idea that run different unit tests by 
a defined scenario. Actually, there is 1 DPDK-AE bug find by this mode.

Anyway, for the unit test, I just want to de-couple the completed factors. 
So each test app launching  for each unit test suite.

Next step is that design some combined tests by requirements.

Thanks.

Regards,
Zhaoyan Chen

> -----Original Message-----
> From: Liu, Yong
> Sent: Wednesday, March 8, 2017 3:05 PM
> To: Chen, Zhaoyan <zhaoyan.chen@intel.com>; dts@dpdk.org
> Subject: Re: [dts] [PATCH v3] Add Cryptodev Unit Tests Cases
> 
> Zhaoyan,
> Why try to recover test application when there's no response? Is that
> application not stable after run crypto unit test?
> If there's one need to do that, please do it in tear_down function.
> 
> Thanks,
> Marvin
> 
> On 03/08/2017 01:01 PM, Chen, Zhaoyan wrote:
> > +        try:
> > +            out = self.dut.send_expect(testsuite, "RTE>>", timeout)
> > +            self.dut.send_expect("quit", "# ", 30)
> > +        except Exception, ex:
> > +            print ex
> > +            print "run cryptodev_unit_tests process coredump"
> > +            self.dut.alt_session.send_expect("killall test", "# ", 30)
> > +            dmesg = self.dut.alt_session.send_expect("dmesg", "# ", 30)
> > +            print dmesg
> > +        print out

Re: [dts] [PATCH v3] Add Cryptodev Unit Tests Cases

[Liu, Yong]

Zhaoyan,
I guess the reason why Kill this applcation is for next around. Am I right?
We used to handle this kind of action in tear_down function.
For exception handler, there's no need to dump exception for the only one reason is timeout.
In general, let framework handle those kind exceptions and keep suite clean. 

Thanks,
Marvin

> -----Original Message-----
> From: Chen, Zhaoyan
> Sent: Wednesday, March 08, 2017 3:32 PM
> To: Liu, Yong <yong.liu@intel.com>; dts@dpdk.org
> Subject: RE: [dts] [PATCH v3] Add Cryptodev Unit Tests Cases
> 
> I do not recover the application, just error handling.
> Since QAT HW unstable sometime.
> 
> Since just function test, (not stable test),
> So, for each test case will launch a separated test application.
> 
> But, there is another idea that run different unit tests by
> a defined scenario. Actually, there is 1 DPDK-AE bug find by this mode.
> 
> Anyway, for the unit test, I just want to de-couple the completed factors.
> So each test app launching  for each unit test suite.
> 
> Next step is that design some combined tests by requirements.
> 
> Thanks.
> 
> Regards,
> Zhaoyan Chen
> 
> > -----Original Message-----
> > From: Liu, Yong
> > Sent: Wednesday, March 8, 2017 3:05 PM
> > To: Chen, Zhaoyan <zhaoyan.chen@intel.com>; dts@dpdk.org
> > Subject: Re: [dts] [PATCH v3] Add Cryptodev Unit Tests Cases
> >
> > Zhaoyan,
> > Why try to recover test application when there's no response? Is that
> > application not stable after run crypto unit test?
> > If there's one need to do that, please do it in tear_down function.
> >
> > Thanks,
> > Marvin
> >
> > On 03/08/2017 01:01 PM, Chen, Zhaoyan wrote:
> > > +        try:
> > > +            out = self.dut.send_expect(testsuite, "RTE>>", timeout)
> > > +            self.dut.send_expect("quit", "# ", 30)
> > > +        except Exception, ex:
> > > +            print ex
> > > +            print "run cryptodev_unit_tests process coredump"
> > > +            self.dut.alt_session.send_expect("killall test", "# ", 30)
> > > +            dmesg = self.dut.alt_session.send_expect("dmesg", "# ",
> 30)
> > > +            print dmesg
> > > +        print out