test suite reviews and discussions
 help / color / mirror / Atom feed
* [dts] [PATCH V1 0/2]test-crypto-perf: Add crypto performance auto test code and test plain
@ 2017-03-10  2:41 Gang Yang
  2017-03-10  2:41 ` [dts] [PATCH V1 1/2]test-crypto-perf:Add crypto performance auto test code Gang Yang
  2017-03-10  2:41 ` [dts] [PATCH V1 1/2]test-crypto-perf:add crypto performance test plan Gang Yang
  0 siblings, 2 replies; 3+ messages in thread
From: Gang Yang @ 2017-03-10  2:41 UTC (permalink / raw)
  To: dts; +Cc: gangyanx

From: gangyanx <gangx.yang@intel.com>

* add QAT,AESNI_MB,AESNI_GCM,OPENSSL,NULL,KASUMI,SNOW3G and ZUC pmd's performace auto test code 
* and also add the test plan 

gangyanx (2):
  Add crypto performance auto test code
  Add crypto performance test plan

 test_plans/crypto_perf_test_plan.rst |  310 +++++++
 tests/TestSuite_crypto_perf.py       | 1492 ++++++++++++++++++++++++++++++++++
 2 files changed, 1802 insertions(+)
 create mode 100755 test_plans/crypto_perf_test_plan.rst
 create mode 100755 tests/TestSuite_crypto_perf.py

-- 
2.5.5

^ permalink raw reply	[flat|nested] 3+ messages in thread

* [dts] [PATCH V1 1/2]test-crypto-perf:Add crypto performance auto test code
  2017-03-10  2:41 [dts] [PATCH V1 0/2]test-crypto-perf: Add crypto performance auto test code and test plain Gang Yang
@ 2017-03-10  2:41 ` Gang Yang
  2017-03-10  2:41 ` [dts] [PATCH V1 1/2]test-crypto-perf:add crypto performance test plan Gang Yang
  1 sibling, 0 replies; 3+ messages in thread
From: Gang Yang @ 2017-03-10  2:41 UTC (permalink / raw)
  To: dts; +Cc: gangyanx

From: gangyanx <gangx.yang@intel.com>

* add new crypto performance auto test code , currently support range buffer size and list buffer size to test performace 

Signed-off-by: gangyanx <gangx.yang@intel.com>
---
 tests/TestSuite_crypto_perf.py | 1492 ++++++++++++++++++++++++++++++++++++++++
 1 file changed, 1492 insertions(+)
 create mode 100755 tests/TestSuite_crypto_perf.py

diff --git a/tests/TestSuite_crypto_perf.py b/tests/TestSuite_crypto_perf.py
new file mode 100755
index 0000000..aaf79fe
--- /dev/null
+++ b/tests/TestSuite_crypto_perf.py
@@ -0,0 +1,1492 @@
+# BSD LICENSE
+#
+# Copyright(c) 2010-2014 Intel Corporation. All rights reserved.
+# All rights reserved.
+#
+# Redistribution and use in source and binary forms, with or without
+# modification, are permitted provided that the following conditions
+# are met:
+#
+#   * Redistributions of source code must retain the above copyright
+#     notice, this list of conditions and the following disclaimer.
+#   * Redistributions in binary form must reproduce the above copyright
+#     notice, this list of conditions and the following disclaimer in
+#     the documentation and/or other materials provided with the
+#     distribution.
+#   * Neither the name of Intel Corporation nor the names of its
+#     contributors may be used to endorse or promote products derived
+#     from this software without specific prior written permission.
+#
+# THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
+# "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
+# LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
+# A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
+# OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
+# SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
+# LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
+# DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
+# THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
+# (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
+# OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
+
+import hmac
+import hashlib
+import binascii
+import time
+import os
+import sys
+import dts
+import utils
+import commands
+import copy
+import time
+import random
+from test_case import TestCase
+sys.path.append("/root")
+from CryptoMobile.CM import *
+
+class TestL2fwdCrypto(TestCase):
+
+    
+    def set_up_all(self):
+        
+        self.filename=""
+        self.__LTCY = ""  
+        self.__THPT = ""
+        self.qat_wilte_list = []
+        self.total_buffer_list = []
+        self.waiting_to_coll_buf = []
+        
+        self.core_config = "1S/2C/1T"
+        self.number_of_ports = 1
+        self.dut_ports = self.dut.get_ports(self.nic)
+        self.verify(len(self.dut_ports) >= self.number_of_ports,
+                    "Not enough ports for " + self.nic)
+        self.ports_socket = self.dut.get_numa_id(self.dut_ports[0])
+
+        self.logger.info("core config = " + self.core_config)
+        self.logger.info("number of ports = " + str(self.number_of_ports))
+        self.logger.info("dut ports = " + str(self.dut_ports))
+        self.logger.info("ports_socket = " + str(self.ports_socket))
+
+        self.core_mask = utils.create_mask(self.dut.get_core_list(
+                                         self.core_config,
+                                         socket=self.ports_socket))
+        self.port_mask = utils.create_mask([self.dut_ports[0]])
+
+        self.tx_port = self.tester.get_local_port(self.dut_ports[0])
+        self.rx_port = self.tester.get_local_port(self.dut_ports[0])
+
+        self.tx_interface = self.tester.get_interface(self.tx_port)
+        self.rx_interface = self.tester.get_interface(self.rx_port)
+
+        self.logger.info("core mask = " + self.core_mask)
+        self.logger.info("port mask = " + self.port_mask)
+        self.logger.info("tx interface = " + self.tx_interface)
+        self.logger.info("rx interface = " + self.rx_interface)
+
+        # Rebuild the dpdk with cryptodev pmds CONFIG_RTE_LIBRTE_PMD_ZUC=n
+        self.dut.send_expect("export AESNI_MULTI_BUFFER_LIB_PATH=/root/ipsec_044/code/", "#")
+        self.dut.send_expect("export LIBSSO_SNOW3G_PATH=/root/libsso_snow3g/snow3g/", "#")
+        self.dut.send_expect("export LIBSSO_ZUC_PATH=/root/libsso_zuc.1.0.1.1-8/zuc", "#")
+        self.dut.send_expect("export LIBSSO_KASUMI_PATH=/root/LibSSO_0_3_1/isg_cid-wireless_libs/ciphers/kasumi/", "#")
+        self.dut.send_expect("sed -i 's/CONFIG_RTE_LIBRTE_PMD_QAT=n$/CONFIG_RTE_LIBRTE_PMD_QAT=y/' config/common_base", "# ")
+        self.dut.send_expect("sed -i 's/CONFIG_RTE_LIBRTE_PMD_AESNI_GCM=n$/CONFIG_RTE_LIBRTE_PMD_AESNI_GCM=y/' config/common_base", "# ")
+        self.dut.send_expect("sed -i 's/CONFIG_RTE_LIBRTE_PMD_NULL_CRYPTO=n$/CONFIG_RTE_LIBRTE_PMD_NULL_CRYPTO=y/' config/common_base", "# ")
+        self.dut.send_expect("sed -i 's/CONFIG_RTE_LIBRTE_PMD_SNOW3G=n$/CONFIG_RTE_LIBRTE_PMD_SNOW3G=y/' config/common_base", "# ")
+        self.dut.send_expect("sed -i 's/CONFIG_RTE_LIBRTE_PMD_KASUMI=n$/CONFIG_RTE_LIBRTE_PMD_KASUMI=y/' config/common_base", "# ")
+        self.dut.send_expect("sed -i 's/CONFIG_RTE_LIBRTE_PMD_OPENSSL=n$/CONFIG_RTE_LIBRTE_PMD_OPENSSL=y/' config/common_base", "# ")
+        self.dut.send_expect("sed -i 's/CONFIG_RTE_LIBRTE_PMD_AESNI_MB=n$/CONFIG_RTE_LIBRTE_PMD_AESNI_MB=y/' config/common_base", "# ") 
+        self.dut.send_expect("sed -i 's/CONFIG_RTE_LIBRTE_PMD_ZUC=n$/CONFIG_RTE_LIBRTE_PMD_ZUC=y/' config/common_base", "# ")
+        self.dut.skip_setup = False
+        self.dut.build_install_dpdk(self.dut.target)
+
+        # test-crypto-perf compile
+        out = self.dut.build_dpdk_apps("./app/test-crypto-perf")
+        '''
+        self.verify("Error" not in out, "Compilation error")
+        self.verify("No such" not in out, "Compilation error")
+        '''
+        # Bind QAT VF devices
+        out = self.dut.send_expect("lspci -d:37c9|awk '{print $1}'", "# ", 10)
+        self.dut.send_expect('echo "8086 37c9" > /sys/bus/pci/drivers/igb_uio/new_id', "# ", 10)
+        for line in out.replace("\r", "\n").replace("\n\n", "\n").split("\n"):
+            cmd = "echo 0000:{} > /sys/bus/pci/devices/0000\:{}/driver/unbind".format(line, line.replace(":", "\:"))
+            qat_device = "0000:{}".format(line,line.replace(":","\:"))
+            self.qat_wilte_list.append(qat_device)
+            self.dut.send_expect(cmd, "# ", 10)
+            cmd = "echo 0000:{} > /sys/bus/pci/drivers/igb_uio/bind".format(line)
+            self.dut.send_expect(cmd, "# ", 10)       
+        self.__check_buffer_size_number()
+        
+    def set_up(self):
+        pass
+
+    def test_qat_AES_CBC_perf(self):
+        result = True
+        self.logger.info("Test qat_c_AES_CBC_Perf")
+        result = self.__execute_crypto_perf_test(
+                 test_vectors, "qat_c_AES_CBC_Perf")
+        self.verify(result,True)
+        
+    def test_qat_AES_CTR_perf(self):
+        result = True
+        self.logger.info("Test qat_c_AES_CTR_Perf")
+        result = self.__execute_crypto_perf_test(
+                 test_vectors, "qat_c_AES_CTR_Perf")
+        self.verify(result,True)
+    
+    def test_qat_AES_GCM_perf(self):
+        result = True
+        self.logger.info("Test qat_c_AES_GCM_Perf")
+        result = self.__execute_crypto_perf_test(
+                 test_vectors, "qat_c_AES_GCM_Perf")
+        self.verify(result,True)
+        
+    def test_qat_3DES_CBC_perf(self):
+        result = True
+        self.logger.info("Test qat_c_3DES_CTR_Perf")
+        result = self.__execute_crypto_perf_test(
+                 test_vectors, "qat_c_3DES_CTR_Perf")
+        self.verify(result,True)
+          
+    def test_qat_3DES_CTR_perf(self):
+        result = True
+        self.logger.info("Test qat_c_3DES_CBC_Perf")
+        result = self.__execute_crypto_perf_test(
+                 test_vectors, "qat_c_3DES_CBC_Perf")
+        self.verify(result,True)
+        
+    def test_qat_KASUMI_perf(self):
+        result = True
+        self.logger.info("Test qat_c_KASUMMI_Perf")
+        result = self.__execute_crypto_perf_test(
+                 test_vectors, "qat_c_KASUMMI_Perf")
+        self.verify(result,True)
+    
+        
+    def test_qat_SNOW3G_perf(self):
+        result = True
+        self.logger.info("Test qat_c_SNOW3G_Perf")
+        result = self.__execute_crypto_perf_test(
+                 test_vectors, "qat_c_SNOW3G_Perf")
+        self.verify(result,True)
+        
+    def test_qat_HASH_perf(self):
+        result = True
+        self.logger.info("Test qat_h_auto_Perf")
+        result = self.__execute_crypto_perf_test(
+                 test_vectors, "qat_h_auto_Perf")
+        self.verify(result,True)
+    
+    def test_qat_NULL_perf(self):
+        result = True
+        self.logger.info("Test qat_NULL_auto_Perf")
+        result = self.__execute_crypto_perf_test(
+                 test_vectors, "qat_NULL_auto_Perf")
+        self.verify(result,True)
+        
+    def test_aesni_mb_AES_CBC_perf(self):
+        result = True
+        self.logger.info("Test aesni_mb_c_AES_CBC_Perf")
+        result = self.__execute_crypto_perf_test(
+                 test_vectors, "aesni_mb_c_AES_CBC_Perf")
+        self.verify(result,True)
+        
+    def test_aesni_mb_AES_CTR_perf(self):
+        result = True
+        self.logger.info("Test aesni_mb_c_AES_CTR_Perf")
+        result = self.__execute_crypto_perf_test(
+                 test_vectors, "aesni_mb_c_AES_CTR_Perf")
+        self.verify(result,True)
+                  
+    def test_aesni_mb_HASH_perf(self):
+        result = True
+        self.logger.info("Test aesni_mb_h_auto_Perf")
+        result = self.__execute_crypto_perf_test(
+                 test_vectors, "aesni_mb_h_auto_Perf")
+        self.verify(result,True)
+    
+    def test_aesni_gcm_AES_GCM_perf(self):
+        result = True
+        self.logger.info("Test aes_gcm_c_AES_GCM_Perf")
+        result = self.__execute_crypto_perf_test(
+                 test_vectors, "aes_gcm_c_AES_GCM_Perf")
+        self.verify(result,True)
+        
+    def test_kasumi_KASUMI_perf(self):
+        result = True
+        self.logger.info("Test kasumi_KASUMI_auto_Perf")
+        result = self.__execute_crypto_perf_test(
+                 test_vectors, "kasumi_KASUMI_auto_Perf")
+        self.verify(result,True)
+        
+    def test_snow3g_SNOW3G_perf(self):
+        result = True
+        self.logger.info("Test snow3g_SNOW3G_auto_Perf")
+        result = self.__execute_crypto_perf_test(
+                 test_vectors, "snow3g_SNOW3G_auto_Perf")
+        self.verify(result,True)
+        
+    def test_zuc_ZUC_perf(self):
+        result = True
+        self.logger.info("Test zuc_ZUC_auto_Perf")
+        result = self.__execute_crypto_perf_test(
+                 test_vectors, "zuc_ZUC_auto_Perf")
+        self.verify(result,True)
+        
+    def test_null_NULL_perf(self):
+        result = True
+        self.logger.info("Test null_NULL_auto_Perf")
+        result = self.__execute_crypto_perf_test(
+                 test_vectors, "null_NULL_auto_Perf")
+        self.verify(result,True)
+    
+    def test_openssl_AES_CBC_perf(self):
+        result = True
+        self.logger.info("Test openssl_c_AES_CBC_Perf")
+        result = self.__execute_crypto_perf_test(
+                 test_vectors, "openssl_c_AES_CBC_Perf")
+        self.verify(result,True)
+        
+    def test_openssl_AES_CTR_perf(self):
+        result = True
+        self.logger.info("Test openssl_c_AES_CTR_Perf")
+        result = self.__execute_crypto_perf_test(
+                 test_vectors, "openssl_c_AES_CTR_Perf")
+        self.verify(result,True)
+        
+    def test_openssl_AES_GCM_perf(self):
+        result = True
+        self.logger.info("Test openssl_c_AES_GCM_Perf")
+        result = self.__execute_crypto_perf_test(
+                 test_vectors, "openssl_c_AES_GCM_Perf")
+        self.verify(result,True)
+        
+    def test_openssl_3DES_CBC_perf(self):
+        result = True
+        self.logger.info("Test openssl_c_3DES_CBC_Perf")
+        result = self.__execute_crypto_perf_test(
+                 test_vectors, "openssl_c_3DES_CBC_Perf")
+        self.verify(result,True) 
+    
+    def test_openssl_3DES_CTR_perf(self):
+        result = True
+        self.logger.info("Test openssl_c_3DES_CTR_Perf")
+        result = self.__execute_crypto_perf_test(
+                 test_vectors, "openssl_c_3DES_CTR_Perf")
+        self.verify(result,True) 
+        
+    def test_openssl_HASH_perf(self):
+        result = True
+        self.logger.info("Test openssl_h_HASH_auto_Perf")
+        result = self.__execute_crypto_perf_test(
+                 test_vectors, "openssl_h_HASH_auto_Perf")
+        self.verify(result,True) 
+            
+    def test_calculatr_case_number(self):
+        
+        self.__calculate_totall_cases_numb()
+        
+    def __execute_crypto_perf_test(self, test_vectors, test_vector_name):
+        failed_count = 0
+        result = True
+        
+        if test_vector_name not in test_vectors:
+            self.logger.warn("SKIP : " + test_vector_name)
+            return True
+
+        test_vector = test_vectors[test_vector_name]
+          
+        test_vector_list = self.__test_vector_to_vector_list(test_vector,
+                core_mask=self.core_mask,
+                port_mask=self.port_mask)
+
+        currently_time = time.strftime('%Y-%m-%d %H:%M:%S',time.localtime(time.time()))
+        case_line = "<<<[{time}][{name}] Total Generated: [{num}] Cases>>>\n".format( \
+                     time=currently_time,name=test_vector_name,num=len(test_vector_list))
+        self.logger.info(case_line)
+        
+        for test_vector in test_vector_list:
+            self.logger.debug(test_vector)
+            cmd_str = self.__test_vector_to_cmd(test_vector,
+                                                core_mask=self.core_mask,
+                                                port_mask=self.port_mask)
+            out = self.dut.send_expect(cmd_str, "]# ", 600)
+            self.logger.info("Test Result:\n*********************\n {ret}".format(ret=out))
+            ret = self.__result_collection(out, test_vector)
+            if ret == False :
+                failed_count = failed_count + 1
+            result = result and ret
+        self.logger.info(("[{name}] Total Failed cases: [{num}]\n".format( \
+                            name=test_vector_name, num=failed_count)))
+        return result
+
+    def tear_down(self):
+        pass
+
+    def tear_down_all(self):
+        f = open(self.filename,"a")
+        currently_time = time.strftime('%Y-%m-%d %H:%M:%S',time.localtime(time.time()))        
+        f.write("...Test Finished [{time}]...".format(time=currently_time))
+        self.dut.send_expect("sed -i 's/CONFIG_RTE_LIBRTE_PMD_QAT=y$/CONFIG_RTE_LIBRTE_PMD_QAT=n/' config/common_base", "# ")
+        self.dut.send_expect("sed -i 's/CONFIG_RTE_LIBRTE_PMD_AESNI_GCM=y$/CONFIG_RTE_LIBRTE_PMD_AESNI_GCM=n/' config/common_base", "# ")
+        self.dut.send_expect("sed -i 's/CONFIG_RTE_LIBRTE_PMD_NULL_CRYPTO=y$/CONFIG_RTE_LIBRTE_PMD_NULL_CRYPTO=n/' config/common_base", "# ")
+        self.dut.send_expect("sed -i 's/CONFIG_RTE_LIBRTE_PMD_SNOW3G=y$/CONFIG_RTE_LIBRTE_PMD_SNOW3G=n/' config/common_base", "# ")
+        self.dut.send_expect("sed -i 's/CONFIG_RTE_LIBRTE_PMD_KASUMI=y$/CONFIG_RTE_LIBRTE_PMD_KASUMI=n/' config/common_base", "# ")
+        self.dut.send_expect("sed -i 's/CONFIG_RTE_LIBRTE_PMD_OPENSSL=y$/CONFIG_RTE_LIBRTE_PMD_OPENSSL=n/' config/common_base", "# ")
+        self.dut.send_expect("sed -i 's/CONFIG_RTE_LIBRTE_PMD_AESNI_MB=y$/CONFIG_RTE_LIBRTE_PMD_AESNI_MB=n/' config/common_base", "# ") 
+        self.dut.send_expect("sed -i 's/CONFIG_RTE_LIBRTE_PMD_ZUC=y$/CONFIG_RTE_LIBRTE_PMD_ZUC=n/' config/common_base", "# ")
+        f.close()
+  
+    def __check_buffer_size_number(self):  
+        buf_numb = 0
+        buf_list = []
+                
+        for vector_name in test_vectors:
+            temp_numb = 0
+            temp_list = []
+            vector = test_vectors[vector_name]
+            if vector["buffer-sz"] != "":
+                if vector["buffer-sz"].find(",") != -1:
+                    buf = vector["buffer-sz"].split(",")
+                    temp_list=copy.deepcopy(buf)
+                    temp_numb = len(buf)
+                elif vector["buffer-sz"].find(":") != -1:
+                    buf = vector["buffer-sz"].split(":")
+                    if len(buf) != 3:
+                        self.logger.info("buffer-sz format is invalid")
+                    else:
+                        temp = buf[0]
+                        while temp <= buf[2]:
+                            temp_list.append(temp)
+                            temp_numb = temp_numb +1 
+                            temp = temp + buf[1]
+                else:
+                    temp_numb = 1
+            if temp_numb > buf_numb:
+                buf_numb = temp_numb
+                buf_list=copy.deepcopy(temp_list)
+                
+        head = "PMD,chain,ptest,cipher_algo,cipher_op,cipher_key,iv,auth_algo,auth_op,auth_key,aad,digest," 
+        self.__LTCY = head
+        self.__THPT = head
+        self.total_buffer_list = copy.deepcopy(buf_list)
+        for buf_sz in buf_list:
+            self.__LTCY = self.__LTCY + "buf{buffer} Ltcy(us),buf{buffer} Cyc,".format(buffer=buf_sz)
+            self.__THPT = self.__THPT + "buf{buffer} Thpt(Gbps),buf{buffer} Ops,".format(buffer=buf_sz)
+        self.__LTCY = self.__LTCY[:len(self.__LTCY) - 1] + "\n"
+        self.__THPT = self.__THPT[:len(self.__THPT) - 1] + "\n"
+            
+        str_pid = str(os.getpid())
+        self.filename = "/tmp/test_crypto_perf_{0}.csv".format(time.time())
+        f = open(self.filename,"a")
+        if f == None:
+            self.logger.info("open scv file failed.")
+            return
+        
+        current_time = time.strftime('%Y-%m-%d %H:%M:%S',time.localtime(time.time()))
+        f.write("...Test Start [{time}]...\n".format(time=current_time))
+        f.write(self.__THPT)
+        f.write(self.__LTCY)
+        f.close()
+        
+    def __calculate_totall_cases_numb(self):
+        alg_map = {}
+        pmd_map = {}
+        map_combine  = {}
+        count = 0
+        alg = ""
+        pmd = ""
+        alg_list = ["AES_CBC","AES_CTR","AES_GCM","3DES_CBC", \
+                   "3DES_CTR","SNOW3G","KASUMI","ZUC","NULL","MD_SHA"]
+        pmd_list = ["qat","aesni_mb","aesni_gcm","snow3g",\
+                   "kasumi","zuc","openssl","null"]
+        valid_map = {
+                    "qat": ["AES_CBC", "AES_CTR","AES_GCM","3DES_CBC", \
+                            "3DES_CTR","SNOW3G","KASUMI","NULL","MD_SHA"],
+                    "aesni_mb":["AES_CBC", "AES_CTR"],
+                    "aesni_gcm":["AES_GCM"],
+                    "snow3g":["SNOW3G"],
+                    "kasumi":["KASUMI"],
+                    "zuc":["ZUC"],
+                    "openssl":["AES_CBC", "AES_CTR","AES_GCM","3DES_CBC","3DES_CTR","MD_SHA"],
+                    "null":["NULL"]                    
+                    }
+        
+        for index,value in test_vectors.iteritems():
+            test_vector_list = self.__test_vector_to_vector_list(value,
+                core_mask="-1",port_mask=self.port_mask)
+            count = count + len(test_vector_list)
+            for i in alg_list:
+                if (index.upper()).find(i) != -1:
+                    alg = i
+                    if i in alg_map:
+                        alg_map[i] += len(test_vector_list)
+                    else:
+                        alg_map[i] = len(test_vector_list)
+            for j in pmd_list:
+                if (index).find(j) != -1:
+                    pmd = j 
+                    if j in pmd_map:
+                        pmd_map[j] += len(test_vector_list)
+                    else:
+                        pmd_map[j] = len(test_vector_list)
+            if (pmd !="") and (alg in valid_map[pmd]):
+                temp_str = pmd + "_" + alg
+                if temp_str in map_combine:
+                    map_combine[temp_str] += len(test_vector_list)
+                else:
+                    map_combine[temp_str] = len(test_vector_list)
+        for k,v in alg_map.iteritems():
+            
+            self.logger.info("Total {name} cases:\t\t\t{number}".format(name=k,number=v))
+        for k,v in pmd_map.iteritems():
+            self.logger.info("Total {name} cases:\t\t\t{number}".format(name=k,number=v))
+        for k,v in map_combine.iteritems():
+            self.logger.info("Total {name} cases:\t\t\t{number}".format(name=k,number=v))
+        self.logger.info("Total cases:\t\t\t {0}".format(count))
+            
+    def __parse_througuput_data(self,original_out):
+        throughPut_line = ""
+        count = 0 
+        data_finished = False
+        index = original_out.find("# lcore id")
+        if index == -1:
+            seg_index = original_out.find("Segmentation fault")
+            throughPut_line ="sgtf,sgtf" if seg_index != -1  else "otf,otf"
+        else:
+            value_data = (original_out[index:]).replace("\r","")
+            value_list = (value_data.replace("\n\n","\n")).split("\n")
+            temp_list = filter(lambda x: (x.find("lcore id") == -1) and (x != ""), value_list)
+            length = len(temp_list)
+            for data_list in temp_list:
+                count = count + 1 
+                value_data_list = (data_list).split(";")
+                if length == count :
+                    data_finished = True
+                throughPut_line =throughPut_line + self.__buffer_size_result_check( \
+                                           value_data_list[1],value_data_list[8], \
+                                           value_data_list[7],data_finished)
+        throughPut_line = throughPut_line + "\n"
+        return throughPut_line
+    
+    def __parse_latency_data(self,original_out):
+        latency = ""
+        count = 0 
+        data_finished = False
+        index = original_out.find("# lcore")
+        if index == -1:
+            seg_index = original_out.find("Segmentation fault")
+            latency = "sgtf,sgtf" if seg_index != -1 else "otf,otf"
+        else:
+            value_data = (original_out[index:]).replace("\r","")
+            value_list = (value_data.replace("\r","")).split("\n")
+            temp_list = filter(lambda x: (x.find("lcore") == -1) and (x != ""), value_list)
+            length = len(temp_list)
+            for data_list in temp_list:
+                count = count + 1 
+                value_data_list = data_list.split(";")
+                if length == count :
+                    data_finished = True
+                latency = latency + self.__buffer_size_result_check(value_data_list[1],value_data_list[5],value_data_list[4],data_finished)
+        latency = latency + "\n"
+        return latency
+    
+    def __buffer_size_result_check(self,buf_size,thpt_ltcy,ops_cycl,finished):
+        count = 0
+        result = ""
+        buffer_sz = copy.deepcopy(self.total_buffer_list)
+        
+        for buf  in self.waiting_to_coll_buf:
+            if int(buf_size) > int(buf):
+                count = count + 1 
+                result = result + "*,*,"
+            elif int(buf_size) == int(buf):
+                count = count + 1
+                self.waiting_to_coll_buf = self.waiting_to_coll_buf[count:]
+                result = result + "{throughput_latenct},{ops_cycle},".format( \
+                             buffer=buf_size,throughput_latenct=thpt_ltcy,ops_cycle=ops_cycl)
+            else:
+                if finished == True:
+                    result = result + "*,*,"
+                else:
+                    break
+        return result
+                
+    def __parse_dut_out_data(self,original_out,test_vector):
+        perf_data_collect=""
+        if test_vector["ptest"]=="latency":
+            perf_data_collect = self.__parse_latency_data(original_out)
+        else:
+            perf_data_collect = self.__parse_througuput_data(original_out)
+                    
+        return perf_data_collect
+    
+    def __valid_data_write_line(self,test_vector):
+        iv          =   "{IV}".format(IV=test_vector["cipher-iv-sz"]) if test_vector["cipher-iv-sz"] != "" else "/"
+        aad         =   "{Aad}".format(Aad=test_vector["auth-aad-sz"]) if test_vector["auth-aad-sz"] !="" else "/"
+        ptest       =   (test_vector["ptest"]).upper()
+        optype      =   test_vector["optype"]
+        digest      =   "{dgst}".format(dgst=test_vector["auth-digest-sz"]) if test_vector["auth-digest-sz"] != "" else "/"
+        auth_op     =   (test_vector["auth-op"]).capitalize() if test_vector["auth-op"] != "" else "/"
+        devtype     =   ((test_vector["devtype"]).replace("crypto_","")).upper()
+        auth_key    =   "{key}".format(key=test_vector["auth-key-sz"]) if test_vector["auth-key-sz"] != "" else "/"
+        cipher_op   =   (test_vector["cipher-op"]).capitalize() if test_vector["cipher-op"] != "" else "-"
+        auth_algo   =   ((test_vector["auth-algo"]).replace("-","_")).upper() if test_vector["auth-algo"] != "" else "/"
+        cipher_key  =   "{key}".format(key=test_vector["cipher-key-sz"]) if test_vector["cipher-key-sz"] != "" else "/"
+        cipher_algo =   ((test_vector["cipher-algo"]).replace("-","_")).upper() if test_vector["cipher-algo"] != "" else "/"
+        
+        if optype.find("only") != -1 :
+            chain = (optype.replace("-","_")).upper()
+        else:
+            chain = ((optype.replace("-then-","_")).replace("auth", "hash")).upper() 
+        
+        w_line = "{pmd},{Chain},{test_type},{cAlg},{cop},{cKey},{IV},{hAlg},{hop},{hkey},{Aad},{dgst},".format( \
+                pmd=devtype,Chain=chain,test_type=ptest,cAlg=cipher_algo,cop=cipher_op,cKey=cipher_key,IV=iv,\
+                hAlg=auth_algo,hop=auth_op,hkey=auth_key,Aad=aad,dgst=digest) 
+ 
+        return w_line 
+    
+    def __result_collection(self,original_out,test_vector):
+        result = True
+        perf_line = ""
+        f = open(self.filename,"a")
+        if f==None:
+            logger.info("open scv file failed.")
+            return                
+        self.waiting_to_coll_buf = copy.deepcopy(self.total_buffer_list)
+        perf_line = self.__valid_data_write_line(test_vector) +  \
+                    self.__parse_dut_out_data(original_out, test_vector) 
+        f.write(perf_line)
+
+        if (perf_line.find("sgtf") != -1) or (perf_line.find("otf") != -1) :
+            result = False
+        return result
+        
+    def __test_vector_to_cmd(self, test_vector, core_mask="", port_mask=""):
+        TEST_CRYPTO_PERF = "./app/test-crypto-perf/build/app/dpdk-test-crypto-perf"
+        EAL_CORE_MASK = " -cf" if core_mask == "" else " -c" + core_mask
+        EAL_SEP = " --"
+        QUEUE_NUM = " "
+        CSV_FRIENDLY = " --csv-friendly"
+        
+        #port info 
+        pci_bus = ""
+        for port in self.dut.ports_info :
+            pci_bus =" -w " + port['pci']
+        
+        qat_bus = ""
+        if len(self.qat_wilte_list) > 0 :
+            qat_bus = " -w " + self.qat_wilte_list[random.randint(0 , len(self.qat_wilte_list) - 1)]
+        
+        vdev = ""
+        if self.__check_field_in_vector(test_vector, "vdev"):
+            vdev = " --vdev " + test_vector["vdev"]
+        
+        ptest = ""
+        if self.__check_field_in_vector(test_vector, "ptest"):
+            ptest = " --ptest " + test_vector["ptest"]
+            
+        devtype = ""
+        if self.__check_field_in_vector(test_vector, "devtype"):
+            devtype = " --devtype " + test_vector["devtype"]
+            
+        chain = ""
+        if self.__check_field_in_vector(test_vector, "optype"):
+            chain = " --optype " + test_vector["optype"]
+
+        cipher_algo = ""
+        if self.__check_field_in_vector(test_vector, "cipher-algo"):
+            cipher_algo = " --cipher-algo " + test_vector["cipher-algo"]
+
+        cipher_op = ""
+        if self.__check_field_in_vector(test_vector, "cipher-op"):
+            cipher_op = " --cipher-op " + test_vector["cipher-op"]
+
+        cipher_key = ""
+        if self.__check_field_in_vector(test_vector, "cipher-key-sz"):
+            cipher_key = " --cipher-key-sz {size}".format(size=test_vector["cipher-key-sz"])
+
+        iv = ""
+        if self.__check_field_in_vector(test_vector, "cipher-iv-sz"):
+            iv = " --cipher-iv-sz {size}".format(size=test_vector["cipher-iv-sz"])
+
+        auth_algo = ""
+        if self.__check_field_in_vector(test_vector, "auth-algo"):
+            auth_algo = " --auth-algo " + test_vector["auth-algo"]
+
+        auth_op = ""
+        if self.__check_field_in_vector(test_vector, "auth-op"):
+            auth_op = " --auth-op " + test_vector["auth-op"]
+
+        auth_key = ""
+        if self.__check_field_in_vector(test_vector, "auth-key-sz"):
+            auth_key = " --auth-key-sz {size}".format(size=test_vector["auth-key-sz"])
+
+        aad = ""
+        if self.__check_field_in_vector(test_vector, "auth-aad-sz"):
+            aad = " --auth-aad-sz {size}".format(size=test_vector["auth-aad-sz"])
+
+        digest = ""
+        if self.__check_field_in_vector(test_vector, "auth-digest-sz"):
+            digest = " --auth-digest-sz {size}".format(size=test_vector["auth-digest-sz"])
+            
+        total_ops = ""
+        if self.__check_field_in_vector(test_vector, "total-ops"):
+            total_ops = " --total-ops {size}".format(size=test_vector["total-ops"])
+        
+        burst_size = ""
+        if self.__check_field_in_vector(test_vector, "burst-sz"):
+            burst_size = " --burst-sz {size}".format(size=test_vector["burst-sz"])
+        
+        buffer_size = ""
+        if self.__check_field_in_vector(test_vector, "buffer-sz"):
+            buffer_size = " --buffer-sz {size}".format(size=test_vector["buffer-sz"])
+
+        cmd_str = "".join([TEST_CRYPTO_PERF, EAL_CORE_MASK, vdev,pci_bus,qat_bus, EAL_SEP,QUEUE_NUM,
+                           ptest, devtype, chain, cipher_algo, cipher_op, cipher_key,iv, auth_algo, 
+                           auth_op, auth_key, aad, digest, total_ops, burst_size,buffer_size,CSV_FRIENDLY])
+        
+        return cmd_str
+
+    def __check_field_in_vector(self, test_vector, field_name):
+        if field_name in test_vector and test_vector[field_name]:
+            return True
+        return False
+
+    def __format_hex_to_param(self, hex_str, sep=":", prefix=""):
+        if not hex_str:
+            return ""
+        if len(hex_str) == 1:
+            return prefix + "0" + hex_str
+
+        result = prefix + hex_str[0:2]
+        for i in range(2, len(hex_str), 2):
+            if len(hex_str) < i + 2:
+                result = result + sep + "0" + hex_str[i:]
+            else:
+                result = result + sep + hex_str[i:i+2]
+
+        return result
+
+    def __var2list(self, var):
+        var_list = var if isinstance(var, list) else [var]
+        return var_list
+
+    def __is_valid_op(self, chain, op):
+        chain_op_map = {
+                "aead": ["encrypt", "generate"],
+                "cipher-only": ["encrypt", "decrypt"],
+                "auth-only": ["generate", "verify"],
+                "cipher-then-auth": ["encrypt", "generate"],
+                "auth-then-cipher": ["decrypt", "verify"],
+                }
+        if op in chain_op_map[chain]:
+            return True
+        return False
+    
+    def __is_valid_size(self, key_type, algo, size):
+        algo_size_map = {
+                "aes-cbc": {
+                    "cipher-key-sz": [16, 24, 32],
+                    "cipher-iv-sz": [16]
+                    },
+                "aes-ctr": {
+                    "cipher-key-sz": [16, 24, 32],
+                    "cipher-iv-sz": [16]
+                    },
+                "3des-cbc": {
+                    "cipher-key-sz": [16, 24],
+                    "cipher-iv-sz": [8]
+                    },
+                "3des-ctr": {
+                    "cipher-key-sz": [16, 24],
+                    "cipher-iv-sz": [8]
+                    },
+                "aes-gcm": {
+                    "cipher-key-sz": [16, 24, 32],
+                    "auth-key-sz": [16, 24, 32],
+                    "auth-aad-sz": [1,2,3,4,5,6,8,9,12,16,24,32,64,128,155,256,1024,65535],
+                    "cipher-iv-sz": [12,16],
+                    "auth-digest-sz":[16]
+                    },
+                "snow3g-uea2": {
+                    "cipher-key-sz": [16],
+                    "cipher-iv-sz": [16]
+                    },
+                "kasumi-f8": {
+                    "cipher-key-sz": [16],
+                    "cipher-iv-sz": [8]
+                    },
+                "zuc-eea3": {
+                    "cipher-key-sz": [16],
+                    "cipher-iv-sz": [16]
+                    },
+                "null": {
+                    "cipher-key-sz": [0],
+                    "auth-key-sz": [0],
+                    "auth-aad-sz": [0],
+                    "cipher-iv-sz": [0],
+                    "auth-digest-sz":[0]
+                    },                        
+                "md5-hmac": {
+                    "auth-key-sz": [64],
+                    "auth-aad-sz": [0],
+                    "auth-digest-sz":[16]                    
+                    },
+                "sha1-hmac": {
+                    "auth-key-sz": [64],
+                    "auth-aad-sz": [0],
+                    "auth-digest-sz":[20]
+                    },
+                "sha2-224-hmac": {
+                    "auth-key-sz": [64],
+                    "auth-aad-sz": [0],
+                    "auth-digest-sz":[28]
+                    },
+                "sha2-256-hmac": {
+                    "auth-key-sz": [64],
+                    "auth-aad-sz": [0],
+                    "auth-digest-sz":[32]
+                    },
+                "sha2-384-hmac": {
+                    "auth-key-sz": [128],
+                    "auth-aad-sz": [0],
+                    "auth-digest-sz":[48]
+                    },
+                "sha2-512-hmac": {
+                    "auth-key-sz": [128],
+                    "auth-aad-sz": [0],
+                    "auth-digest-sz":[64]
+                    },
+                "aes-gmac": {
+                    "auth-key-sz": [16, 24, 32],
+                    "auth-aad-sz": [1,2,3,4,5,6,8,9,12,16,24,32,64,128,155,256,1024,65535],
+                    "auth-digest-sz":[16]
+                    },
+                "aes-xcbc-hmac": {
+                    "auth-key-sz": [16],
+                    "auth-aad-sz": [0],
+                    "auth-digest-sz":[16]
+                    },
+                "snow3g-uia2": {
+                    "auth-key-sz": [16],
+                    "auth-aad-sz": [16],
+                    "auth-digest-sz":[4]
+                    },
+                "kasumi-f9": {
+                    "auth-key-sz": [16],
+                    "auth-aad-sz": [8],
+                    "auth-digest-sz":[4]
+                    },
+                "zuc-eia3": {
+                    "auth-key-sz": [16],
+                    "auth-aad-sz": [16],
+                    "auth-digest-sz":[4]
+                    },
+                "md5": {
+                    "auth-key-sz": [0],
+                    "auth-aad-sz": [0],
+                    "auth-digest-sz":[16]
+                    },
+                "sha1": {
+                    "auth-key-sz": [0],
+                    "auth-aad-sz": [0],
+                    "auth-digest-sz":[20]
+                    },
+                "sha2-224": {
+                    "auth-key-sz": [0],
+                    "auth-aad-sz": [0],
+                    "auth-digest-sz":[28]
+                    },
+                "sha2-256": {
+                    "auth-key-sz": [0],
+                    "auth-aad-sz": [0],
+                    "auth-digest-sz":[32]
+                    },
+                "sha2-384": {
+                    "auth-key-sz": [0],
+                    "auth-aad-sz": [0],
+                    "auth-digest-sz":[48]
+                    },
+                "sha2-512": {
+                    "auth-key-sz": [0],
+                    "auth-aad-sz": [0],
+                    "auth-digest-sz":[64]
+                    },
+                }
+        result = False
+        if algo in algo_size_map:
+            if key_type in algo_size_map[algo]:
+                if size in algo_size_map[algo][key_type]:
+                    result = True
+        return result
+    
+    def __actually_aesni_mb_digest(self,vector):
+
+        if ((vector["vdev"]).find("crypto_aesni_mb") == -1):
+            return
+        
+        auth_algo_dgst_map = {
+                "md5-hmac": 12,
+                "sha1-hamc": 12,
+                "sha2-224-hamc": 14,
+                "sha2-256-hamc": 16,
+                "sha2-384-hamc": 24,
+                "sha2-512-hamc": 32
+                }
+        if vector["auth-algo"] in auth_algo_dgst_map:
+            digest = auth_algo_dgst_map[vector["auth-algo"]]
+            vector["auth-digest-sz"] = digest
+    
+    def __iter_optype(self, vector, vector_list, core_mask="", port_mask=""):
+        test_vector = vector.copy()
+        chain_list = self.__var2list(test_vector["optype"])
+        for op_type in chain_list:
+            test_vector = vector.copy()
+            test_vector["optype"] = op_type
+            self.__iter_cipher_algo(test_vector, vector_list, core_mask, port_mask)       
+        
+    def __iter_cipher_algo(self, vector, vector_list, core_mask="", port_mask=""):
+        test_vector = vector.copy()
+        if test_vector["optype"] == "auth-only":
+            test_vector["cipher-algo"] = ""
+            self.__iter_cipher_op(test_vector, vector_list, core_mask, port_mask)
+        else:
+            cipher_algo_list = self.__var2list(test_vector["cipher-algo"])
+            for cipher_algo in cipher_algo_list:
+                test_vector = vector.copy()
+                test_vector["cipher-algo"] = cipher_algo
+                self.__iter_cipher_op(test_vector, vector_list, core_mask, port_mask)
+
+    def __iter_cipher_op(self, vector, vector_list, core_mask="", port_mask=""):
+        test_vector = vector.copy()
+        if test_vector["optype"] == "auth-only":
+            test_vector["cipher-op"] = ""
+            self.__iter_cipher_key(test_vector, vector_list, core_mask, port_mask)
+        else:
+            cipher_op_list = self.__var2list(test_vector["cipher-op"])
+            for cipher_op in cipher_op_list:
+                if self.__is_valid_op(test_vector["optype"], cipher_op):
+                    test_vector = vector.copy()
+                    test_vector["cipher-op"] = cipher_op
+                    self.__iter_cipher_key(test_vector, vector_list, \
+                            core_mask, port_mask)
+
+    def __iter_cipher_key(self, vector, vector_list, core_mask="", port_mask=""):
+        test_vector = vector.copy()
+        if test_vector["optype"] == "auth-only":
+            test_vector["cipher-key-sz"] = ""
+            self.__iter_iv(test_vector, vector_list, core_mask, port_mask)
+        else:
+            cipher_key_list = self.__var2list(test_vector["cipher-key-sz"])
+            for cipher_key in cipher_key_list:
+                test_vector = vector.copy()
+                if isinstance(cipher_key, int):
+                    if self.__is_valid_size("cipher-key-sz", \
+                            test_vector["cipher-algo"], \
+                            cipher_key):
+                        test_vector["cipher-key-sz"] = cipher_key
+                        self.__iter_iv(test_vector, vector_list, core_mask, port_mask)
+                    else:
+                        continue
+                else:
+                    test_vector["cipher-key-sz"] = 0
+                    self.__iter_iv(test_vector, vector_list, core_mask, port_mask)
+
+    def __iter_iv(self, vector, vector_list, core_mask="", port_mask=""):
+        test_vector = vector.copy()
+        if test_vector["optype"] == "auth-only":
+            test_vector["cipher-iv-sz"] = ""
+            self.__iter_auth_algo(test_vector, vector_list, core_mask, port_mask)
+        else:
+            iv_list = self.__var2list(test_vector["cipher-iv-sz"])
+            for iv in iv_list:
+                test_vector = vector.copy()
+                if isinstance(iv, int):
+                    if self.__is_valid_size("cipher-iv-sz", \
+                            test_vector["cipher-algo"], \
+                            iv):
+                        test_vector["cipher-iv-sz"] = iv
+                        self.__iter_auth_algo(test_vector, vector_list, core_mask, port_mask)
+                    else:
+                        continue
+                else:    
+                    test_vector["cipher-iv-sz"] = 0
+                    self.__iter_auth_algo(test_vector, vector_list, \
+                                          core_mask, port_mask)
+
+    def __iter_auth_algo(self, vector, vector_list, core_mask="", port_mask=""):
+        test_vector = vector.copy()
+        if test_vector["optype"] == "cipher-only":
+            test_vector["auth-algo"] = ""
+            self.__iter_auth_op(test_vector, vector_list, core_mask, port_mask)
+        else:
+            auth_algo_list = self.__var2list(test_vector["auth-algo"])
+            for auth_algo in auth_algo_list:
+                test_vector = vector.copy()
+                test_vector["auth-algo"] = auth_algo
+                self.__iter_auth_op(test_vector, vector_list, core_mask, port_mask)
+
+    def __iter_auth_op(self, vector, vector_list, core_mask="", port_mask=""):
+        test_vector = vector.copy()
+        if test_vector["optype"] == "cipher-only":
+            test_vector["auth-op"] = ""
+            self.__iter_auth_key(test_vector, vector_list, core_mask, port_mask)
+        else:
+            auth_op_list = self.__var2list(test_vector["auth-op"])
+            for auth_op in auth_op_list:
+                if self.__is_valid_op(test_vector["optype"], auth_op):
+                    test_vector = vector.copy()
+                    test_vector["auth-op"] = auth_op 
+                    self.__iter_auth_key(test_vector, vector_list, \
+                            core_mask, port_mask)
+
+    def __iter_auth_key(self, vector, vector_list, core_mask="", port_mask=""):
+        test_vector = vector.copy()
+        if test_vector["optype"] == "cipher-only":
+            test_vector["auth-key-sz"] = ""
+            self.__iter_aad(test_vector, vector_list, core_mask, port_mask)
+        else:
+            auth_key_list = self.__var2list(test_vector["auth-key-sz"])
+            for auth_key in auth_key_list:
+                test_vector = vector.copy()
+                if isinstance(auth_key, int):
+                    if self.__is_valid_size("auth-key-sz", \
+                            test_vector["auth-algo"], \
+                            auth_key):
+                        test_vector["auth-key-sz"] = auth_key
+                        self.__iter_aad(test_vector, vector_list, core_mask, port_mask)
+                    else:
+                        continue
+                else:
+                    test_vector["auth-key-sz"] = 0 
+                    self.__iter_aad(test_vector, vector_list, core_mask, port_mask)
+
+    def __iter_aad(self, vector, vector_list, core_mask="", port_mask=""):
+        test_vector = vector.copy()
+        if test_vector["optype"] == "cipher-only":
+            test_vector["auth-aad-sz"] = ""
+            self.__iter_digest(test_vector, vector_list, core_mask, port_mask)
+        else:
+            aad_list = self.__var2list(test_vector["auth-aad-sz"])
+            for aad in aad_list:
+                test_vector = vector.copy()
+                if isinstance(aad, int):
+                    if self.__is_valid_size("auth-aad-sz", \
+                            test_vector["auth-algo"], \
+                            aad):
+                        test_vector["auth-aad-sz"] = aad
+                        self.__iter_digest(test_vector, vector_list, \
+                                core_mask, port_mask)
+                    else:
+                        continue
+                else:
+                    test_vector["auth-aad-sz"] = 0
+                    self.__iter_digest(test_vector, vector_list, \
+                            core_mask, port_mask)
+
+    def __iter_digest(self, vector, vector_list, core_mask="", port_mask=""):
+        test_vector = vector.copy()
+        if test_vector["optype"] == "cipher-only":
+            test_vector["auth-digest-sz"] = ""
+            self.__iter_buffer_size(test_vector, vector_list, \
+                    core_mask, port_mask)
+        else:
+            digest_list = self.__var2list(vector["auth-digest-sz"])
+            for digest in digest_list:
+                test_vector = vector.copy()
+                if isinstance(digest, int):
+                    if self.__is_valid_size("auth-digest-sz", \
+                            test_vector["auth-algo"], \
+                            digest):
+                        test_vector["auth-digest-sz"] = digest                    
+                    else:
+                        continue
+                else:
+                    test_vector["auth-digest-sz"] = 0
+                self.__actually_aesni_mb_digest(test_vector)
+                self.__iter_buffer_size(test_vector, vector_list, \
+                        core_mask, port_mask)               
+                
+    def __iter_buffer_size(self,vector, vector_list, core_mask="", port_mask=""):
+        test_vector = vector.copy()
+        buffer_sz_list = vector["buffer-sz"]
+        if buffer_sz_list == "":
+            buffer_sz_list = "32,64,128,256, 512,768, 1024,1280,1536,1792,2048"
+        test_vector["buffer-sz"] = buffer_sz_list
+        vector_list.append(test_vector)
+        
+    def __test_vector_to_vector_list(self, test_vector,core_mask="", port_mask=""):
+        vector_list = []
+        ptest_list = self.__var2list(test_vector["ptest"])
+        for ptest in ptest_list :
+            test_vector["ptest"] = ptest
+            self.__iter_optype(test_vector, vector_list, core_mask, port_mask)
+            
+        return vector_list
+    
+test_vectors = {
+                
+    "qat_c_AES_CBC_Perf": {
+        "vdev": "",
+        "devtype":"crypto_qat",
+        "ptest": ["throughput","latency"],
+        "optype": ["cipher-only", "cipher-then-auth"],
+        "cipher-algo": ["aes-cbc"],
+        "cipher-op": ["encrypt", "decrypt"],
+        "cipher-key-sz": [16, 24, 32],
+        "cipher-iv-sz": [16],
+        "auth-algo": ["md5-hmac", "sha1-hmac", "sha2-224-hmac", "sha2-256-hmac",
+                      "sha2-384-hmac", "sha2-512-hmac"],
+        "auth-op": ["generate"],
+        "auth-key-sz": [8, 16, 24, 32, 40, 48, 56, 64, 128],
+        "auth-aad-sz": [0],
+        "auth-digest-sz": [16,20,28,32,48,64],
+        "buffer-sz":"32,64,128,256,512,768,1024,1280,1536,1792,2048",
+        "burst-sz":32,
+        "total-ops":10000000
+    },
+                
+    "qat_c_AES_CTR_Perf": {
+        "vdev": "",
+        "devtype":"crypto_qat",
+        "ptest": ["throughput","latency"],
+        "optype": ["cipher-only", "cipher-then-auth"],
+        "cipher-algo": ["aes-ctr"],
+        "cipher-op": ["encrypt", "decrypt"],
+        "cipher-key-sz": [16, 24, 32],
+        "cipher-iv-sz": [16],
+        "auth-algo": ["md5-hmac", "sha1-hmac", "sha2-224-hmac", "sha2-256-hmac",
+                      "sha2-384-hmac", "sha2-512-hmac"],
+        "auth-op": ["generate"],
+        "auth-key-sz": [16, 24, 32, 40, 48, 56, 64, 128],
+        "auth-aad-sz": [0],
+        "auth-digest-sz": [16,20,28,32,48,64],
+        "buffer-sz":"32,64,128,256,512,768,1024,1280,1536,1792,2048",
+        "burst-sz":32,
+        "total-ops":10000000
+    },
+    
+    "qat_c_AES_GCM_Perf": {
+        "vdev": "",
+        "devtype":"crypto_qat",
+        "ptest": ["throughput","latency"],
+        "optype": ["aead"],
+        "cipher-algo": ["aes-gcm"],
+        "cipher-op": ["aead"],
+        "cipher-key-sz": [16, 24, 32],
+        "cipher-iv-sz": [16],
+        "auth-algo": ["aes-gcm","aes-gmac","md5-hmac", "sha1-hmac", "sha2-224-hmac", "sha2-256-hmac",
+                      "sha2-384-hmac", "sha2-512-hmac"],
+        "auth-op": ["generate"],
+        "auth-key-sz": [8, 16, 24, 32, 40, 48, 56, 64, 128],
+        "auth-aad-sz": [0,8,12],
+        "auth-digest-sz": [8,12,16,20,28,32,48,64],
+        "buffer-sz":"32,64,128,256,512,768,1024,1280,1536,1792,2048",
+        "burst-sz":32,
+        "total-ops":10000000
+    },
+                
+    "qat_c_3DES_CBC_Perf": {
+        "vdev": "",
+        "devtype":"crypto_qat",
+        "ptest": ["throughput","latency"],
+        "optype": ["cipher-only", "cipher-then-auth"],
+        "cipher-algo": ["3des-cbc"],
+        "cipher-op": ["encrypt", "decrypt"],
+        "cipher-key-sz": [16, 24],
+        "cipher-iv-sz": [8],
+        "auth-algo": ["md5-hmac", "sha1-hmac", "sha2-224-hmac", "sha2-256-hmac",
+                      "sha2-384-hmac", "sha2-512-hmac"],
+        "auth-op": ["generate"],
+        "auth-key-sz": [16, 24, 32, 40, 48, 56, 64, 128],
+        "auth-aad-sz": [0],
+        "auth-digest-sz": [16,20,28,32,48,64],
+        "buffer-sz":"32,64,128,256,512,768,1024,1280,1536,1792,2048",
+        "burst-sz":32,
+        "total-ops":10000000
+    },
+                
+    "qat_c_3DES_CTR_Perf": {
+        "vdev": "",
+        "devtype":"crypto_qat",
+        "ptest": ["throughput","latency"],
+        "optype": ["cipher-only", "cipher-then-auth"],
+        "cipher-algo": ["3des-ctr"],
+        "cipher-op": ["encrypt", "decrypt"],
+        "cipher-key-sz": [16, 24],
+        "cipher-iv-sz": [8],
+        "auth-algo": ["md5-hmac", "sha1-hmac", "sha2-224-hmac", "sha2-256-hmac",
+                      "sha2-384-hmac", "sha2-512-hmac"],
+        "auth-op": ["generate"],
+        "auth-key-sz": [16, 24, 32, 40, 48, 56, 64, 128],
+        "auth-aad-sz": [0],
+        "auth-digest-sz": [16,20,28,32,48,64],
+        "buffer-sz":"32,64,128,256,512,768,1024,1280,1536,1792,2048",
+        "burst-sz":32,
+        "total-ops":10000000
+    },
+    
+    "qat_c_KASUMMI_Perf": {
+        "vdev": "",
+        "devtype":"crypto_qat",
+        "ptest": ["throughput","latency"],
+        "optype": ["cipher-only", "cipher-then-auth"],
+        "cipher-algo": ["kasumi-f8"],
+        "cipher-op": ["encrypt", "decrypt"],
+        "cipher-key-sz": [16],
+        "cipher-iv-sz": [8],
+        "auth-algo": ["kasumi-f9"],
+        "auth-op": ["generate"],
+        "auth-key-sz": [16],
+        "auth-aad-sz": [8],
+        "auth-digest-sz": [4],
+        "buffer-sz":"32,64,128,256,512,768,1024,1280,1536,1792,2048",
+        "burst-sz":32,
+        "total-ops":10000000
+    },
+                
+    "qat_c_SNOW3G_Perf": {
+        "vdev": "",
+        "devtype":"crypto_qat",
+        "ptest": ["throughput","latency"],
+        "optype": ["cipher-only", "cipher-then-auth"],
+        "cipher-algo": ["snow3g-uea2"],
+        "cipher-op": ["encrypt", "decrypt"],
+        "cipher-key-sz": [16],
+        "cipher-iv-sz": [16],
+        "auth-algo": ["snow3g-uia2"],
+        "auth-op": ["generate"],
+        "auth-key-sz": [16],
+        "auth-aad-sz": [16],
+        "auth-digest-sz": [4],
+        "buffer-sz":"32,64,128,256,512,768,1024,1280,1536,1792,2048",
+        "burst-sz":32,
+        "total-ops":10000000
+    },
+                
+    "qat_h_auto_Perf": {
+        "vdev": "",
+        "devtype":"crypto_qat",
+        "ptest": ["throughput","latency"],
+        "optype": ["auth-only"],
+        "cipher-algo": "",
+        "cipher-op": "",
+        "cipher-key-sz": "",
+        "cipher-iv-sz": "",
+        "auth-algo": ["aes-gcm","aes-gmac","kasumi-f9","snow3g-uia2","md5-hmac", 
+                      "sha1-hmac", "sha2-224-hmac", "sha2-256-hmac","sha2-384-hmac",
+                       "sha2-512-hmac"],
+        "auth-op": ["generate"],
+        "auth-key-sz": [8, 16, 24, 32, 40, 48, 56, 64, 128],
+        "auth-aad-sz": [0,8,12],
+        "auth-digest-sz": [4,8,12,16, 20,28, 32, 48, 64],
+        "buffer-sz":"32,64,128,256,512,768,1024,1280,1536,1792,2048",
+        "burst-sz":32,
+        "total-ops":10000000
+    },
+        
+    "qat_NULL_auto_Perf": {
+        "vdev": "",
+        "devtype":"crypto_qat",
+        "ptest": ["throughput","latency"],
+        "optype": ["cipher-only"],
+        "cipher-algo": ["null"],
+        "cipher-op": ["encrypt", "decrypt"],
+        "cipher-key-sz": [0],
+        "cipher-iv-sz": [0],
+        "auth-algo": ["null"],
+        "auth-op": ["generate"],
+        "auth-key-sz": [0],
+        "auth-aad-sz": [0],
+        "auth-digest-sz": [0],
+        "buffer-sz":"32,64,128,256,512,768,1024,1280,1536,1792,2048",
+        "burst-sz":32,
+        "total-ops":10000000
+    },
+                
+    "aesni_mb_c_AES_CBC_Perf": {
+        "vdev": "crypto_aesni_mb_pmd",
+        "devtype":"crypto_aesni_mb",
+        "ptest": ["throughput","latency"],
+        "optype": ["cipher-only", "cipher-then-auth"],
+        "cipher-algo": ["aes-cbc"],
+        "cipher-op": ["encrypt", "decrypt"],
+        "cipher-key-sz": [16, 24, 32],
+        "cipher-iv-sz": [16],
+        "auth-algo": ["md5-hmac", "sha1-hmac", "sha2-224-hmac", "sha2-256-hmac",
+                      "sha2-384-hmac", "sha2-512-hmac"],
+        "auth-op": ["generate"],
+        "auth-key-sz": [8, 16, 24, 32, 40, 48, 56, 64, 128],
+        "auth-aad-sz": [0],
+        "auth-digest-sz": [12, 14, 16, 24,32],
+        "buffer-sz":"32,64,128,256,512,768,1024,1280,1536,1792,2048",
+        "burst-sz":32,
+        "total-ops":10000000
+    },    
+                  
+    "aesni_mb_c_AES_CTR_Perf": {
+        "vdev": "crypto_aesni_mb_pmd",
+        "devtype":"crypto_aesni_mb",
+        "ptest": ["throughput","latency"],
+        "optype": ["cipher-only", "cipher-then-auth"],
+        "cipher-algo": ["aes-ctr"],
+        "cipher-op": ["encrypt", "decrypt"],
+        "cipher-key-sz": [16, 24, 32],
+        "cipher-iv-sz": [16],
+        "auth-algo": ["md5-hmac", "sha1-hmac", "sha2-224-hmac", "sha2-256-hmac",
+                      "sha2-384-hmac", "sha2-512-hmac"],
+        "auth-op": ["generate"],
+        "auth-key-sz": [8, 16, 24, 32, 40, 48, 56, 64, 128],
+        "auth-aad-sz": [0],
+        "auth-digest-sz": [12, 14, 16, 24,32],
+        "buffer-sz":"32,64,128,256,512,768,1024,1280,1536,1792,2048",
+        "burst-sz":32,
+        "total-ops":10000000
+    }, 
+                  
+    "aesni_mb_h_auto_Perf": {
+        "vdev": "crypto_aesni_mb_pmd",
+        "devtype":"crypto_aesni_mb",
+        "ptest": ["throughput","latency"],
+        "optype": ["auth-only", "cipher-then-auth"],
+        "cipher-algo": "",
+        "cipher-op": "",
+        "cipher-key-sz": "",
+        "cipher-iv-sz": "",
+        "auth-algo": ["md5-hmac","sha1-hmac", "sha2-224-hmac", "sha2-256-hmac",
+                      "sha2-384-hmac","sha2-512-hmac"],
+        "auth-op": ["generate"],
+        "auth-key-sz": [8, 16, 24, 32, 40, 48, 56, 64, 128],
+        "auth-aad-sz": [0],
+        "auth-digest-sz": [12, 14, 16, 24,32],
+        "buffer-sz":"32,64,128,256,512,768,1024,1280,1536,1792,2048",
+        "burst-sz":32,
+        "total-ops":10000000
+    },
+                
+    "aes_gcm_c_AES_GCM_Perf": {
+        "vdev": "crypto_aesni_gcm_pmd",
+        "devtype":"crypto_aesni_gcm",
+        "ptest": ["throughput","latency"],
+        "optype": ["aead"],
+        "cipher-algo": ["aes-gcm"],
+        "cipher-op": ["encrypt", "decrypt"],
+        "cipher-key-sz": [16,32],
+        "cipher-iv-sz": [12],
+        "auth-algo": ["aes-gcm","aes-gmac"],
+        "auth-op": ["generate"],
+        "auth-key-sz": [16,32],
+        "auth-aad-sz": [1,4,8,12,16,65535],
+        "auth-digest-sz": [8,12,16],
+        "buffer-sz":"32,64,128,256,512,768,1024,1280,1536,1792,2048",
+        "burst-sz":32,
+        "total-ops":10000000
+    },
+    
+    "kasumi_KASUMI_auto_Perf": {
+        "vdev": "crypto_kasumi_pmd",
+        "devtype":"crypto_kasumi",
+        "ptest": ["throughput","latency"],
+        "optype": ["cipher-only","auth-only", "cipher-then-auth"],
+        "cipher-algo": ["kasumi-f8"],
+        "cipher-op": ["encrypt", "decrypt"],
+        "cipher-key-sz": [16],
+        "cipher-iv-sz": [8],
+        "auth-algo": ["kasumi-f9"],
+        "auth-op": ["generate"],
+        "auth-key-sz": [16],
+        "auth-aad-sz": [8],
+        "auth-digest-sz": [4],
+        "buffer-sz":"32,64,128,256,512,768,1024,1280,1536,1792,2048",
+        "burst-sz":32,
+        "total-ops":10000000
+    },
+                
+    "snow3g_SNOW3G_auto_Perf": {
+        "vdev": "crypto_snow3g_pmd",
+        "devtype":"crypto_snow3g",
+        "ptest": ["throughput","latency"],
+        "optype": ["cipher-only", "auth-only","cipher-then-auth"],
+        "cipher-algo": ["snow3g-uea2"],
+        "cipher-op": ["encrypt", "decrypt"],
+        "cipher-key-sz": [16],
+        "cipher-iv-sz": [16],
+        "auth-algo": ["snow3g-uia2"],
+        "auth-op": ["generate"],
+        "auth-key-sz": [16],
+        "auth-aad-sz": [16],
+        "auth-digest-sz": [4],
+        "buffer-sz":"32,64,128,256,512,768,1024,1280,1536,1792,2048",
+        "burst-sz":32,
+        "total-ops":10000000
+    },
+                
+    "zuc_ZUC_auto_Perf": {
+        "vdev": "crypto_zuc_pmd",
+        "devtype":"crypto_zuc",
+        "ptest": ["throughput","latency"],
+        "optype": ["cipher-only", "auth-only","cipher-then-auth"],
+        "cipher-algo": ["zuc-eea3"],
+        "cipher-op": ["encrypt", "decrypt"],
+        "cipher-key-sz": [16],
+        "cipher-iv-sz": [16],
+        "auth-algo": ["zuc-eia3"],
+        "auth-op": ["generate"],
+        "auth-key-sz": [16],
+        "auth-aad-sz": [16],
+        "auth-digest-sz": [4],
+        "buffer-sz":"32,64,128,256,512,768,1024,1280,1536,1792,2048",
+        "burst-sz":32,
+        "total-ops":10000000
+    },
+                        
+    "null_NULL_auto_Perf": {
+        "vdev": "",
+        "devtype":"crypto_qat",
+        "ptest": ["throughput","latency"],
+        "optype": ["cipher-only"],
+        "cipher-algo": ["null"],
+        "cipher-op": ["encrypt", "decrypt"],
+        "cipher-key-sz": [0],
+        "cipher-iv-sz": [0],
+        "auth-algo": ["null"],
+        "auth-op": ["generate"],
+        "auth-key-sz": [0],
+        "auth-aad-sz": [0],
+        "auth-digest-sz": [0],
+        "buffer-sz":"32,64,128,256,512,768,1024,1280,1536,1792,2048",
+        "burst-sz":32,
+        "total-ops":10000000
+    },
+                
+    "openssl_c_AES_CBC_Perf": {
+        "vdev": "crypto_openssl_pmd",
+        "devtype":"crypto_openssl",
+        "ptest": ["throughput","latency"],
+        "optype": ["cipher-only", "cipher-then-auth"],
+        "cipher-algo": ["aes-cbc"],
+        "cipher-op": ["encrypt", "decrypt"],
+        "cipher-key-sz": [16, 24, 32],
+        "cipher-iv-sz": [16],
+        "auth-algo": ["md5", "sha1", "sha2-224", "sha2-256","sha2-384", "sha2-512",
+                      "md5-hmac", "sha1-hmac", "sha2-224-hmac", "sha2-256-hmac",
+                      "sha2-384-hmac", "sha2-512-hmac"],
+        "auth-op": ["generate"],
+        "auth-key-sz": [0,64, 128],
+        "auth-aad-sz": [0],
+        "auth-digest-sz": [16, 20,28, 32,48, 64],
+        "buffer-sz":"32,64,128,256",
+        "burst-sz":32,
+        "total-ops":10000000
+    },
+                
+    "openssl_c_AES_CTR_Perf": {
+        "vdev": "crypto_openssl_pmd",
+        "devtype":"crypto_openssl",
+        "ptest": ["throughput","latency"],
+        "optype": ["cipher-only", "cipher-then-auth"],
+        "cipher-algo": ["aes-ctr"],
+        "cipher-op": ["encrypt", "decrypt"],
+        "cipher-key-sz": [16, 24, 32],
+        "cipher-iv-sz": [16],
+        "auth-algo": ["md5", "sha1", "sha2-224", "sha2-256","sha2-384", "sha2-512",
+                      "md5-hmac", "sha1-hmac", "sha2-224-hmac", "sha2-256-hmac",
+                      "sha2-384-hmac", "sha2-512-hmac"],
+        "auth-op": ["generate"],
+        "auth-key-sz": [0, 64, 128],
+        "auth-aad-sz": [0],
+        "auth-digest-sz": [16, 20,28, 32,48, 64],
+        "buffer-sz":"32,64,128,256",
+        "burst-sz":32,
+        "total-ops":10000000
+    },
+    
+    "openssl_c_AES_GCM_Perf": {
+        "vdev": "crypto_openssl_pmd",
+        "devtype":"crypto_openssl",
+        "ptest": ["throughput","latency"],
+        "optype": ["aead"],
+        "cipher-algo": ["aes-gcm"],
+        "cipher-op": ["aead"],
+        "cipher-key-sz": [16],
+        "cipher-iv-sz": [12,16],
+        "auth-algo": ["aes-gcm","aes-gmac","md5", "sha1", "sha2-224", "sha2-256",
+                      "sha2-384", "sha2-512","md5-hmac", "sha1-hmac", "sha2-224-hmac", 
+                      "sha2-256-hmac","sha2-384-hmac", "sha2-512-hmac"],
+        "auth-op": ["generate"],
+        "auth-key-sz": [0, 16, 24, 32, 64, 128],
+        "auth-aad-sz": [0,8,12],
+        "auth-digest-sz": [8,12,16,20,28, 32,48, 64],
+        "buffer-sz":"32,64,128,256",
+        "burst-sz":32,
+        "total-ops":10000000
+    },
+                
+    "openssl_c_3DES_CBC_Perf": {
+        "vdev": "crypto_openssl_pmd",
+        "devtype":"crypto_openssl",
+        "ptest": ["throughput","latency"],
+        "optype": ["cipher-only", "cipher-then-auth"],
+        "cipher-algo": ["3des-cbc"],
+        "cipher-op": ["encrypt", "decrypt"],
+        "cipher-key-sz": [16, 24],
+        "cipher-iv-sz": [8],
+        "auth-algo": ["md5", "sha1", "sha2-224", "sha2-256","sha2-384", "sha2-512",
+                      "md5-hmac", "sha1-hmac", "sha2-224-hmac", "sha2-256-hmac",
+                      "sha2-384-hmac", "sha2-512-hmac"],
+        "auth-op": ["generate"],
+        "auth-key-sz": [0,64, 128],
+        "auth-aad-sz": [0],
+        "auth-digest-sz": [16, 20,28, 32,48, 64],
+        "buffer-sz":"32,64",
+        "burst-sz":32,
+        "total-ops":10000000
+    },
+                
+    "openssl_c_3DES_CTR_Perf": {
+        "vdev": "crypto_openssl_pmd",
+        "devtype":"crypto_openssl",
+        "ptest": ["throughput","latency"],
+        "optype": ["cipher-only", "cipher-then-auth"],
+        "cipher-algo": ["3des-ctr"],
+        "cipher-op": ["encrypt", "decrypt"],
+        "cipher-key-sz": [16, 24],
+        "cipher-iv-sz": [8],
+        "auth-algo": ["md5", "sha1", "sha2-224", "sha2-256","sha2-384", "sha2-512",
+                      "md5-hmac", "sha1-hmac", "sha2-224-hmac", "sha2-256-hmac",
+                      "sha2-384-hmac", "sha2-512-hmac"],
+        "auth-op": ["generate"],
+        "auth-key-sz": [0,64,128],
+        "auth-aad-sz": [0],
+        "auth-digest-sz": [16, 20,28, 32,48, 64],
+        "buffer-sz":"32,64",
+        "burst-sz":32,
+        "total-ops":10000000
+    },
+                
+    "openssl_h_HASH_auto_Perf": {
+        "vdev": "crypto_openssl_pmd",
+        "devtype":"crypto_openssl",
+        "ptest": ["throughput","latency"],
+        "optype": ["auth-only"],
+        "cipher-algo": "",
+        "cipher-op": "",
+        "cipher-key-sz": "",
+        "cipher-iv-sz": "",
+        "auth-algo": ["md5", "sha1", "sha2-224", "sha2-256","sha2-384", "sha2-512",
+                      "md5-hmac", "sha1-hmac", "sha2-224-hmac", "sha2-256-hmac",
+                      "sha2-384-hmac", "sha2-512-hmac"],
+        "auth-op": ["generate"],
+        "auth-key-sz": [0,64,128],
+        "auth-aad-sz": [0],
+        "auth-digest-sz": [16, 20,28, 32,48, 64],
+        "buffer-sz":"32,64",
+        "burst-sz":32,
+        "total-ops":10000000
+    },
+}
+
+
-- 
2.5.5

^ permalink raw reply	[flat|nested] 3+ messages in thread

* [dts] [PATCH V1 1/2]test-crypto-perf:add crypto performance test plan
  2017-03-10  2:41 [dts] [PATCH V1 0/2]test-crypto-perf: Add crypto performance auto test code and test plain Gang Yang
  2017-03-10  2:41 ` [dts] [PATCH V1 1/2]test-crypto-perf:Add crypto performance auto test code Gang Yang
@ 2017-03-10  2:41 ` Gang Yang
  1 sibling, 0 replies; 3+ messages in thread
From: Gang Yang @ 2017-03-10  2:41 UTC (permalink / raw)
  To: dts; +Cc: gangyanx

From: gangyanx <gangx.yang@intel.com>

* add crypto performance test plan , and give some example 

Signed-off-by: gangyanx <gangx.yang@intel.com>
---
 test_plans/crypto_perf_test_plan.rst | 310 +++++++++++++++++++++++++++++++++++
 1 file changed, 310 insertions(+)
 create mode 100755 test_plans/crypto_perf_test_plan.rst

diff --git a/test_plans/crypto_perf_test_plan.rst b/test_plans/crypto_perf_test_plan.rst
new file mode 100755
index 0000000..6c043a9
--- /dev/null
+++ b/test_plans/crypto_perf_test_plan.rst
@@ -0,0 +1,310 @@
+.. Copyright (c) 2010,2011 Intel Corporation
+   All rights reserved.
+   
+   Redistribution and use in source and binary forms, with or without
+   modification, are permitted provided that the following conditions
+   are met:
+   
+   - Redistributions of source code must retain the above copyright
+     notice, this list of conditions and the following disclaimer.
+   
+   - Redistributions in binary form must reproduce the above copyright
+     notice, this list of conditions and the following disclaimer in
+     the documentation and/or other materials provided with the
+     distribution.
+   
+   - Neither the name of Intel Corporation nor the names of its
+     contributors may be used to endorse or promote products derived
+     from this software without specific prior written permission.
+   
+   THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
+   "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
+   LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS
+   FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE
+   COPYRIGHT OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT,
+   INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES
+   (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR
+   SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION)
+   HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT,
+   STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE)
+   ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED
+   OF THE POSSIBILITY OF SUCH DAMAGE.
+
+==============================================
+Testing of Cryptodev performance in DPDK
+==============================================
+
+
+Description
+===========
+
+This document provides the test plan for testing Cryptodev performance by 
+crypto perf application. The crypto perf application is a DPDK example under 
+DPDK examples folder. 
+
+Crypto perf application supports most of Cryptodev PMDs(polling mode dirver)
+Intel QuickAssist Technology DH895xxC/DH_C62xx hardware 
+accelerator (QAT PMD), AESNI MB PMD, AESNI GCM PMD, NULL PMD, KASUMI PMD,
+SNOW3G PMD,ZUC PMD or OPENSSL library PMD.
+
+AESNI MB PMD algorithm table 
+The table below contains AESNI MB algorithms which supported in crypto perf. 
+Part of the algorithms are not supported currently.
+
++---------+---------+---------+---------+----------+----------+----------+----------+----------+----------+   
+|Algorithm|  Mode             | Detail                                                                    | 
+|         |                                                                                               |
++---------+---------+---------+---------+----------+----------+----------+----------+----------+----------+
+| AES     | CBC               |  Encrypt/Decrypt;Key size: 128, 192, 256 bits                             | 
++---------+---------+---------+---------+----------+----------+----------+----------+----------+----------+
+| AES     | CTR               |  Encrypt/Decrypt;Key size: 128, 192, 256 bits                             | 
++---------+---------+---------+---------+----------+----------+----------+----------+----------+----------+
+| MD      |                   |  MD5                                                                      | 
++---------+---------+---------+---------+----------+----------+----------+----------+----------+----------+
+| SHA     |                   |  SHA-1, SHA2-224, SHA2-384, SHA2-256, SHA2-512                            | 
++---------+---------+---------+---------+----------+----------+----------+----------+----------+----------+
+| HMAC    |                   |  Support MD5 and SHA implementations SHA-1, SHA2-224, SHA2-256,           |
+|         |                   |  SHA2-384, SHA2-512                                                       |
+|         |                   |  Key Size versus Block size support: Key Size must be <= block size;      |
+|         |                   |  Mac Len Supported SHA-1 10, 12, 16, 20 bytes;                            |
+|         |                   |  Mac Len Supported SHA2-256 16, 24, 32 bytes;                             |
+|         |                   |  Mac Len Supported SHA2-384 24,32, 40, 48 bytes;                          |
+|         |                   |  Mac Len Supported SHA2-512 32, 40, 48, 56, 64 bytes;                     |
++---------+---------+---------+---------+----------+----------+----------+----------+----------+----------+
+
+QAT algorithm table:
+The table below contains QAT Algorithms which supported in crypto perf. 
+Part of the algorithms are not supported currently.
+
++---------+---------+---------+---------+----------+----------+----------+----------+----------+----------+
+|Algorithm|  Mode             | Detail                                                                    | 
+|         |                                                                                               |
++---------+---------+---------+---------+----------+----------+----------+----------+----------+----------+
+| AES     | CBC               |  Encrypt/Decrypt;Key size: 128, 192, 256 bits                             | 
++---------+---------+---------+---------+----------+----------+----------+----------+----------+----------+
+| AES     | CTR               |  Encrypt/Decrypt;Key size: 128, 192, 256 bits                             | 
++---------+---------+---------+---------+----------+----------+----------+----------+----------+----------+
+| 3DES    | CBC               |  Encrypt/Decrypt;Key size: 128, 192 bits                                  | 
++---------+---------+---------+---------+----------+----------+----------+----------+----------+----------+
+| 3DES    | CTR               |  Encrypt/Decrypt;Key size: 128, 192 bits                                  | 
++---------+---------+---------+---------+----------+----------+----------+----------+----------+----------+
+| MD      |                   |  MD5                                                                      | 
++---------+---------+---------+---------+----------+----------+----------+----------+----------+----------+
+| SHA     |                   |  SHA-1, SHA2-224, SHA2-256, SHA2-384, SHA2-512                            |
++---------+---------+---------+---------+----------+----------+----------+----------+----------+----------+
+| HMAC    |                   |  Support MD5 and SHA implementations SHA-1, SHA2-224, SHA2-256,           |
+|         |                   |  SHA2-384, SHA2-512                                                       |
+|         |                   |  Key Size versus Block size support: Key Size must be <= block size;      |
+|         |                   |  Mac Len Supported SHA-1 10, 12, 16, 20 bytes;                            |
+|         |                   |  Mac Len Supported SHA2-256 16, 24, 32 bytes;                             |
+|         |                   |  Mac Len Supported SHA2-384 24,32, 40, 48 bytes;                          |
+|         |                   |  Mac Len Supported SHA2-512 32, 40, 48, 56, 64 bytes;                     |
++---------+---------+---------+---------+----------+----------+----------+----------+----------+----------+
+| AES     |  GCM              |  Key Sizes:128, 192, 256 bits;                                            |
+|         |                   |  Associated Data Length: 0 ~ 240 bytes;                                   |
+|         |                   |  Payload Length: 0 ~ (2^32 -1) bytes;                                     |
+|         |                   |  IV source: external;                                                     |
+|         |                   |  IV Lengths: 96 bits;                                                     |
+|         |                   |  Tag Lengths: 8, 12, 16 bytes;                                            |
++---------+---------+---------+---------+----------+----------+----------+----------+----------+----------+
+| Kasumi  |  F8               |  Encrypt/Decrypt; Key size: 128                                           |
++         +---------+---------+---------+----------+----------+----------+----------+----------+----------+
+|         |  F9               |  Generate/Verify; Key size: 128                                           |
++---------+---------+---------+---------+----------+----------+----------+----------+----------+----------+
+| Snow3G  |  UEA2             |  Encrypt/Decrypt; Key size: 128                                           |
++         +---------+---------+---------+----------+----------+----------+----------+----------+----------+
+|         |  UIA2             |  Generate/Verify; Key size: 128                                           |
++---------+---------+---------+---------+----------+----------+----------+----------+----------+----------+
+
+AESNI_GCM algorithm table 
+The table below contains AESNI GCM PMD algorithms which are supported 
+in crypto perf
+
++---------+---------+---------+---------+----------+----------+----------+----------+----------+----------+   
+|Algorithm|  Mode             | Detail                                                                    | 
+|         |                                                                                               |
++---------+---------+---------+---------+----------+----------+----------+----------+----------+----------+
+| AES     |  GCM              |  Encrypt/Decrypt;Key Sizes:128, 256 bits;                                 |
+|         |                   |  IV source: external;                                                     |
+|         |                   |  IV Lengths: 96 bits;                                                     |
+|         |                   |  Generate/Verify;Key Sizes:128,192,256 bits;                              |
+|         |                   |  Associated Data Length: 0 ~ 240 bytes;                                   |
+|         |                   |  Payload Length: 0 ~ (2^32 -1) bytes;                                     |
+|         |                   |  Tag Lengths: 8, 12, 16 bytes;                                            | 
++---------+---------+---------+---------+----------+----------+----------+----------+----------+----------+
+| AES     | GMAC              |  Generate/Verify;Key Sizes:128,192,256 bits;                              | 
+|         |                   |  Associated Data Length: 0 ~ 240 bytes;                                   |
+|         |                   |  Payload Length: 0 ~ (2^32 -1) bytes;                                     |
+|         |                   |  Tag Lengths: 8, 12, 16 bytes;                                            | 
++---------+---------+---------+---------+----------+----------+----------+----------+----------+----------+
+
+NULL algorithm table 
+The table below contains NULL algorithms which are supported in crypto perf. 
+Part of the algorithms are not supported currently.
+
++---------+---------+---------+---------+----------+----------+----------+----------+----------+----------+   
+|Algorithm|  Mode             | Detail                                                                    | 
+|         |                                                                                               |
++---------+---------+---------+---------+----------+----------+----------+----------+----------+----------+
+| NULL    |  NULL             |  Encrypt/Decrypt;Key Sizes:0 bits;                                        |
+|         |                   |  IV Lengths: 0 bits;                                                      |
+|         |                   |  Generate/Verify;Key Sizes:0 bits;                                        |
+|         |                   |  Associated Data Length: 1 bytes;                                         |
+|         |                   |  Payload Length: 0  bytes;                                                |
+|         |                   |  Tag Lengths: 0 bytes;                                                    | 
++---------+---------+---------+---------+----------+----------+----------+----------+----------+----------+
+
+KASUMI algorithm table 
+The table below contains KASUMI algorithms which are supported in crypto perf.
+
++---------+---------+---------+---------+----------+----------+----------+----------+----------+----------+   
+|Algorithm|  Mode             | Detail                                                                    | 
+|         |                                                                                               |
++---------+---------+---------+---------+----------+----------+----------+----------+----------+----------+
+| KASUMI  |  F8               |  Encrypt/Decrypt;Key Sizes:128 bits;                                      |
+|         |                   |  IV source: external;                                                     |
+|         |                   |  IV Lengths: 64 bits;                                                     |
++---------+---------+---------+---------+----------+----------+----------+----------+----------+----------+
+| KASUMI  |  F9               |  Generate/Verify;Key Sizes:128  bits;                                     | 
+|         |                   |  Payload Length: 64 bytes;                                                |
+|         |                   |  Tag Lengths: 4 bytes;                                                    | 
++---------+---------+---------+---------+----------+----------+----------+----------+----------+----------+
+
+SNOW3G algorithm table 
+The table below contains SNOW3G algorithms which are supported in crypto perf.
+
++---------+---------+---------+---------+----------+----------+----------+----------+----------+----------+   
+|Algorithm|  Mode             | Detail                                                                    | 
+|         |                                                                                               |
++---------+---------+---------+---------+----------+----------+----------+----------+----------+----------+
+| SNOW3G  |  UEA2             |  Encrypt/Decrypt;Key Sizes:128 bits;                                      |
+|         |                   |  IV source: external;                                                     |
+|         |                   |  IV Lengths: 128 bits;                                                    |
++---------+---------+---------+---------+----------+----------+----------+----------+----------+----------+
+| SNOW3G  |  UIA2             |  Generate/Verify;Key Sizes:128  bits;                                     | 
+|         |                   |  Payload Length: 128 bytes;                                               |
+|         |                   |  Tag Lengths: 4 bytes;                                                    | 
++---------+---------+---------+---------+----------+----------+----------+----------+----------+----------+
+
+ZUC algorithm table 
+The table below contains ZUC algorithms which are supported in crypto perf. 
+
++---------+---------+---------+---------+----------+----------+----------+----------+----------+----------+   
+|Algorithm|  Mode             | Detail                                                                    | 
+|         |                                                                                               |
++---------+---------+---------+---------+----------+----------+----------+----------+----------+----------+
+| ZUC     |  EEA3             |  Encrypt/Decrypt;Key Sizes:128 bits;                                      |
+|         |                   |  IV source: external;                                                     |
+|         |                   |  IV Lengths: 128 bits;                                                    |
++---------+---------+---------+---------+----------+----------+----------+----------+----------+----------+
+| ZUC     |  EIA2             |  Generate/Verify;Key Sizes:128  bits;                                     | 
+|         |                   |  Payload Length: 128 bytes;                                               |
+|         |                   |  Tag Lengths: 4 bytes;                                                    | 
++---------+---------+---------+---------+----------+----------+----------+----------+----------+----------+
+
+OPENSSL algorithm table:
+The table below contains OPENSSL algorithms which are supported in crypto perf. 
+
++---------+---------+---------+---------+----------+----------+----------+----------+----------+----------+
+|Algorithm|  Mode             | Detail                                                                    | 
+|         |                                                                                               |
++---------+---------+---------+---------+----------+----------+----------+----------+----------+----------+
+| AES     | CBC               |  Encrypt/Decrypt;Key size: 128, 192, 256 bits                             | 
++---------+---------+---------+---------+----------+----------+----------+----------+----------+----------+
+| AES     | CTR               |  Encrypt/Decrypt;Key size: 128, 192, 256 bits                             | 
++---------+---------+---------+---------+----------+----------+----------+----------+----------+----------+
+| 3DES    | CBC               |  Encrypt/Decrypt;Key size: 128, 192 bits                                  | 
++---------+---------+---------+---------+----------+----------+----------+----------+----------+----------+
+| 3DES    | CTR               |  Encrypt/Decrypt;Key size: 128, 192 bits                                  | 
++---------+---------+---------+---------+----------+----------+----------+----------+----------+----------+
+| MD      |                   |  MD5                                                                      | 
++---------+---------+---------+---------+----------+----------+----------+----------+----------+----------+
+| SHA     |                   |  SHA-1, SHA2-224, SHA2-256, SHA2-384, SHA2-512                            |
++---------+---------+---------+---------+----------+----------+----------+----------+----------+----------+
+| HMAC    |                   |  Support MD5 and SHA implementations SHA-1, SHA2-224, SHA2-256,           |
+|         |                   |  SHA2-384, SHA2-512                                                       |
+|         |                   |  Key Size versus Block size support: Key Size must be <= block size;      |
+|         |                   |  Mac Len Supported SHA-1 10, 12, 16, 20 bytes;                            |
+|         |                   |  Mac Len Supported SHA2-256 16, 24, 32 bytes;                             |
+|         |                   |  Mac Len Supported SHA2-384 24,32, 40, 48 bytes;                          |
+|         |                   |  Mac Len Supported SHA2-512 32, 40, 48, 56, 64 bytes;                     |
++---------+---------+---------+---------+----------+----------+----------+----------+----------+----------+
+| AES     |  GCM              |  Encrypt/Decrypt;Key Sizes:128 bits;                                      |
+|         |                   |  IV source: external;                                                     |
+|         |                   |  IV Lengths: 96 bits;                                                     |
+|         |                   |  Associated Data Length: 0 ~ 240 bytes;                                   |
+|         |                   |  Generate/Verify; 128, 192,256 bytes;                                     |
+|         |                   |  Payload Length: 64,128 bytes;                                            |
+|         |                   |  Tag Lengths: 16 bytes;                                                   |
++---------+---------+---------+---------+----------+----------+----------+----------+----------+----------+
+| AES     | GMAC              |  Generate/Verify;Key Sizes:128,192,256 bits;                              | 
+|         |                   |  Associated Data Length: 0 ~ 240 bytes;                                   |
+|         |                   |  Payload Length: 8 ~ (2^32 -4) bytes;                                     |
+|         |                   |  Tag Lengths:16 bytes;                                                    | 
++---------+---------+---------+---------+----------+----------+----------+----------+----------+----------+
+
+
+Prerequisites
+=============
+To test CryptoDev performance, an application 
+test_crypto_perf is added into DPDK.
+
+The test commands of test_crypto_perf is below::
+    
+    ./build/app/dpdk-test-crypto-perf -c COREMASK --vdev (AESNI_MB|QAT|AESNI_GCM|OPENSSL|SNOW3G|KASUMI|ZUC|NULL) -w (PCI:DEVICE:FUNCTION) -w (PCI:DEVICE:FUNCTION) -- --ptest (throughput|latency) --devtype (crypto_aesni_mb|crypto_qat|crypto_aes_gcm|crypto_openssl|crypto_snow3g|crypto_kasumi|crypto_zuc|crypto_null) --optype (aead|cipher-only|auth-only|cipher-then-auth|auth-then-cipher)  --cipher-algo (ALGO) --cipher-op (encrypt|decrypt) --cipher-key-sz (key_size) --cipher-iv-sz (iv_size) --auth-algo (ALGO) --auth-op (generate|verify) --auth-key-sz (key_size) --auth-aad-sz (aad_size) --auth-digest-sz (digest_size) --total-ops (ops_number) --burst-sz (burst_size) --buffer-sz (buffer_size) 
+
+
+
+Test case: CryptoDev performance test
+====================================================
++----------+                 +----------+
+|          |                 |          |
+|          | --------------> |          |
+|  Tester  |                 |   DUT    |
+|          |                 |          |
+|          | <-------------> |          |
++----------+                 +----------+
+
+common::
+        
+ --vdev (AESNI_MB|QAT|AESNI_GCM|OPENSSL|SNOW3G|KASUMI|ZUC|NULL) this value can be set as : crypto_aesni_mb_pmd, crypto_aes_gcm_pmd, crypto_openssl_pmd, crypto_snow3g_pmd, crypto_kasumi_pmd, crypto_zuc_pmd or  crypto_null_pmd . if pmd is QAT this parameter should not be set
+
+ -w (PCI:DEVICE:FUNCTION) this value is the port whitelist or QAT device whitelist . if vdev is  set and devtype is not crypto_qat , the QAT device whitelist is not needed , but you also can set it on the cmd line . 
+ 
+ --optype (aead|cipher-only|auth-only|cipher-then-auth|auth-then-cipher): if cipher-algo is aes-gcm or gmac this value must be set to aead . otherwise it will  be set to others. please notice , null algorithm only support cipher-only test.
+
+ other parameters please reference above table's parameter .
+
+QAT PMD Commmand line Eg::
+
+    ./build/app/dpdk-test-crypto-perf -c 0xf  -w 0000:01:00.0 -w 0000:03:3d.0 -- --ptest throughput --devtype crypto_qat --optype cipher-then-auth  --cipher-algo aes-cbc --cipher-op encrypt --cipher-key-sz 16 --cipher-iv-sz 16 --auth-algo sha1-hmac --auth-op generate --auth-key-sz 64 --auth-aad-sz 0 --auth-digest-sz 20 --total-ops 10000000 --burst-sz 32 --buffer-sz 1024
+
+AESNI_MB PMD Commmand line Eg::
+
+    ./build/app/dpdk-test-crypto-perf -c 0xf --vdev crypto_aesni_mb_pmd  -w 0000:01:00.0 -w 0000:03:3d.0 -- --ptest throughput --devtype crypto_aesni_mb --optype cipher-then-auth  --cipher-algo aes-cbc --cipher-op encrypt --cipher-key-sz 16 --cipher-iv-sz 16 --auth-algo sha1-hmac --auth-op generate --auth-key-sz 64 --auth-aad-sz 0 --auth-digest-sz 20 --total-ops 10000000 --burst-sz 32 --buffer-sz 1024         
+
+AESNI_GCM PMD Commmand line Eg::
+
+    ./build/app/dpdk-test-crypto-perf -c 0xf --vdev crypto_aesni_gcm_pmd  -w 0000:01:00.0 -w 0000:03:3d.0 -- --ptest throughput --devtype crypto_aesni_gcm  --optype aead  --cipher-algo aes-gcm --cipher-op encrypt --cipher-key-sz 16 --cipher-iv-sz 12 --auth-algo aes-gcm --auth-op generate --auth-key-sz 16 --auth-aad-sz 4 --auth-digest-sz 12 --total-ops 10000000 --burst-sz 32 --buffer-sz 1024
+
+OPENSSL PMD Commmand line Eg:: 
+
+    ./build/app/dpdk-test-crypto-perf -c 0xf --vdev crypto_openssl_pmd  -w 0000:01:00.0 -w 0000:03:3d.0 -- --ptest throughput --devtype crypto_openssl --optype cipher-then-auth  --cipher-algo aes-cbc --cipher-op encrypt --cipher-key-sz 16 --cipher-iv-sz 16 --auth-algo sha1-hmac --auth-op generate --auth-key-sz 64 --auth-aad-sz 0 --auth-digest-sz 20 --total-ops 10000000 --burst-sz 32 --buffer-sz 64
+
+NULL PMD Commmand line Eg::
+
+    ./build/app/dpdk-test-crypto-perf -c 0xf --vdev crypto_null_pmd  -w 0000:01:00.0 -w 0000:03:3d.0 -- --ptest throughput --devtype crypto_null  --optype cipher-only  --cipher-algo null --cipher-op encrypt --cipher-key-sz 0 --cipher-iv-sz 0  --total-ops 10000000 --burst-sz 32 --buffer-sz 1024
+
+KASUMI PMD Commmand line Eg::
+
+    ./build/app/dpdk-test-crypto-perf -c 0xf --vdev crypto_kasumi_pmd  -w 0000:01:00.0 -w 0000:03:3d.0 -- --ptest throughput --devtype crypto_kasumi --optype cipher-then-auth  --cipher-algo kasumi-f8 --cipher-op encrypt --cipher-key-sz 16 --cipher-iv-sz 8 --auth-algo kasumi-f9 --auth-op generate --auth-key-sz 16 --auth-aad-sz 8 --auth-digest-sz 4 --total-ops 10000000 --burst-sz 32 --buffer-sz 1024
+
+SNOW3G PMD Commmand line Eg::
+
+    ./build/app/dpdk-test-crypto-perf -c 0xf --vdev crypto_snow3g_pmd  -w 0000:01:00.0 -w 0000:03:3d.0 -- --ptest throughput --devtype crypto_snow3g --optype cipher-then-auth  --cipher-algo snow3g-uea2 --cipher-op encrypt --cipher-key-sz 16 --cipher-iv-sz 16 --auth-algo snow3g-uia2 --auth-op generate --auth-key-sz 16 --auth-aad-sz 16 --auth-digest-sz 4 --total-ops 10000000 --burst-sz 32 --buffer-sz 1024
+
+ZUC PMD Commmand line Eg::
+
+    ./build/app/dpdk-test-crypto-perf -c 0xf --vdev crypto_zuc_pmd  -w 0000:01:00.0 -w 0000:03:3d.0 -- --ptest throughput --devtype crypto_zuc_mb --optype cipher-then-auth  --cipher-algo zuc-eea3 --cipher-op encrypt --cipher-key-sz 16 --cipher-iv-sz 16 --auth-algo zuc-eia3  --auth-op generate --auth-key-sz 16 --auth-aad-sz 16 --auth-digest-sz 4 --total-ops 10000000 --burst-sz 32 --buffer-sz 1024
+
+
-- 
2.5.5

^ permalink raw reply	[flat|nested] 3+ messages in thread

end of thread, other threads:[~2017-03-10  2:46 UTC | newest]

Thread overview: 3+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2017-03-10  2:41 [dts] [PATCH V1 0/2]test-crypto-perf: Add crypto performance auto test code and test plain Gang Yang
2017-03-10  2:41 ` [dts] [PATCH V1 1/2]test-crypto-perf:Add crypto performance auto test code Gang Yang
2017-03-10  2:41 ` [dts] [PATCH V1 1/2]test-crypto-perf:add crypto performance test plan Gang Yang

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).