From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mail-wm0-f67.google.com (mail-wm0-f67.google.com [74.125.82.67]) by dpdk.org (Postfix) with ESMTP id 963A66CD3 for ; Wed, 22 Jun 2016 15:57:13 +0200 (CEST) Received: by mail-wm0-f67.google.com with SMTP id r201so1397461wme.0 for ; Wed, 22 Jun 2016 06:57:13 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:from:date:message-id:subject:to; bh=ROKm8AP5wLSDGKH445OQjDmUTOIZSsEdqlQIZdCrJ9w=; b=b1qTDNhpbNpLZ9OHTa8O8X+DFY7NVtORc4urpKaClextXAPTqQhiY2xsX0n/kSgq0V TzP5ByuNSf8HGUvS5CtPUpKxhg6Kqp8JY/A6MmM7grj3CYzfefV8oUXdrxFH1x2rvgfP e9Rc6DpXUBusJWFg0m48xkrVgItzBNI5t/Z310NZ4+MHT9EkX00pZ9tExgvT1e/KhZDD 3qE7w0SMKx3dgsglGpmvE1l8PSYhp7xxWRB64EB5FEbxZDOefkG+bYeBiuS5zhY8jMeh w6XGctumYXDnvMWul1HorXsHZKu9AOBBYi4xiArz52WfVocei/3olkAJYwp+qi8iH9TU o2VA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:mime-version:from:date:message-id:subject:to; bh=ROKm8AP5wLSDGKH445OQjDmUTOIZSsEdqlQIZdCrJ9w=; b=Q+uZCAwY/kPYEtFCBUxiRCKzwEoZnYU9K1syYlDNswD/Ps3E6efdIMM0fwFqbu41Sy GOcffKm3ICRp6Ciwki2zSmNE2ALiYJO653VQOw1xNlRqkvihfcGQaXMeb+CbiNMso/Y2 g08F3bP/v0a142OKCMy68qf2JCVFm5WZ6rRHjVkEIQJulgH2N9FWLxrV0s+nufhXaDeJ dnC1Y/mRakWo0F9dbyfFFBC1ZB9y/dSrtZwN0ufxJIxnSucuYBp3JbNh7AR5216NWPkP NMMjWsS8SeiAGEIBpyadcxhiNLJWWYvAMZ4epCG2uYfoaTWSC/cnU0JOt+XzvWYxI7W7 CZYg== X-Gm-Message-State: ALyK8tLOxjl6weRN859IGQCqnYY22pW6vZbpYhUF7wKWQSVhkr2cJj+YHj4UbSfgNjP1nZdp7lFTr2ADYsLbXQ== X-Received: by 10.194.56.134 with SMTP id a6mr15839785wjq.106.1466603832713; Wed, 22 Jun 2016 06:57:12 -0700 (PDT) MIME-Version: 1.0 Received: by 10.194.14.1 with HTTP; Wed, 22 Jun 2016 06:57:11 -0700 (PDT) From: Chinmaya Dwibedy Date: Wed, 22 Jun 2016 19:27:11 +0530 Message-ID: To: users@dpdk.org Content-Type: text/plain; charset=UTF-8 X-Content-Filtered-By: Mailman/MimeDel 2.1.15 Subject: [dpdk-users] Issue with cryptodev_qat_autotest on CentOS 7.2 host system using dpdk-2.2.0 X-BeenThere: users@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: usage discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 22 Jun 2016 13:57:13 -0000 Hi All, I do have CentOS Linux release 7.2.1511 (Core) host system, where two Intel QAT devices are installed. I have installed dpdk-2.2.0 and QAT driver as mentioned in section 6.4. (i.e., Installation using 01.org QAT driver) of http://dpdk.readthedocs.io/en/v16.04/cryptodevs/qat.html. I tested "cryptodev_qat_autotest" under app/test. I found this test case to be failed. It appears that, it the dequeue burst API fails to retrieve any processed operations available from the queue pair (on the QAT Crypto device), from the devices processed queue. I have captured and attached the console.log here. Can anyone please suggest, is it an issue with the setup/configuration or QAT PMD code? Kindly point me to right direction whenever you find me wrong.Thank you in advance for your time and support. Here goes my configuration. [root@localhost ~]# service qat_service status There is 2 acceleration device(s) in the system: icp_dev0 - type=dh895xcc, inst_id=0, node_id=1, bdf=83:00:0, #accel=6, #engines=12, state=up icp_dev1 - type=dh895xcc, inst_id=1, node_id=1, bdf=88:00:0, #accel=6, #engines=12, state=up [root@localhost ~]# lspci -nn | grep 0435 83:00.0 Co-processor [0b40]: Intel Corporation DH895XCC Series QAT [8086:0435] 88:00.0 Co-processor [0b40]: Intel Corporation DH895XCC Series QAT [8086:0435] [root@localhost ~]# [root@localhost ~]# lspci -nn|grep -e "QAT.*Virtual" 83:01.0 Co-processor [0b40]: Intel Corporation DH895XCC Series QAT Virtual Function [8086:0443] 83:01.1 Co-processor [0b40]: Intel Corporation DH895XCC Series QAT Virtual Function [8086:0443] 83:01.2 Co-processor [0b40]: Intel Corporation DH895XCC Series QAT Virtual Function [8086:0443] 83:01.3 Co-processor [0b40]: Intel Corporation DH895XCC Series QAT Virtual Function [8086:0443] 83:01.4 Co-processor [0b40]: Intel Corporation DH895XCC Series QAT Virtual Function [8086:0443] 83:01.5 Co-processor [0b40]: Intel Corporation DH895XCC Series QAT Virtual Function [8086:0443] ------------------------------------------------------------------------------------------------------------- ------------------------------------------------------------------------------------------------------------- 88:04.1 Co-processor [0b40]: Intel Corporation DH895XCC Series QAT Virtual Function [8086:0443] 88:04.2 Co-processor [0b40]: Intel Corporation DH895XCC Series QAT Virtual Function [8086:0443] 88:04.3 Co-processor [0b40]: Intel Corporation DH895XCC Series QAT Virtual Function [8086:0443] 88:04.4 Co-processor [0b40]: Intel Corporation DH895XCC Series QAT Virtual Function [8086:0443] 88:04.5 Co-processor [0b40]: Intel Corporation DH895XCC Series QAT Virtual Function [8086:0443] 88:04.6 Co-processor [0b40]: Intel Corporation DH895XCC Series QAT Virtual Function [8086:0443] 88:04.7 Co-processor [0b40]: Intel Corporation DH895XCC Series QAT Virtual Function [8086:0443] [root@localhost ~]# lspci -nn|grep -e "QAT.*Virtual" | wc -l 64 [root@localhost ~]# Note: This host is capable of PCI passthrough and SR-IOV. [root@localhost ~]# grep -i dmar /var/log/dmesg [ 0.000000] ACPI: DMAR 000000007b69a000 00130 (v01 INTEL S2600WT 00000001 INTL 20091013) [ 0.135336] dmar: Host address width 46 [ 0.135587] dmar: DRHD base: 0x000000fbffc000 flags: 0x0 [ 0.135855] dmar: IOMMU 0: reg_base_addr fbffc000 ver 1:0 cap d2078c106f0466 ecap f020de [ 0.136309] dmar: DRHD base: 0x000000c7ffc000 flags: 0x1 [ 0.136569] dmar: IOMMU 1: reg_base_addr c7ffc000 ver 1:0 cap d2078c106f0466 ecap f020de [ 0.137016] dmar: RMRR base: 0x0000007a3db000 end: 0x0000007a3ddfff [ 0.137274] dmar: ATSR flags: 0x0 [ 0.137522] dmar: ATSR flags: 0x0 [root@localhost ~]# grep -i "iommu.*enabled" /var/log/dmesg [ 0.000000] Intel-IOMMU: enabled [root@localhost test(keystone_admin)]# cat /proc/cmdline BOOT_IMAGE=/vmlinuz-3.10.0-327.18.2.el7.x86_64 root=/dev/mapper/centos-root ro rd.lvm.lv=centos/swap crashkernel=auto nomodeset rd.lvm.lv=centos/root rhgb iommu=pt intel_iommu=on [root@localhost test(keystone_admin)]# The below output confirms that, QAT driver is correctly installed. [root@localhost ~]# cat /proc/icp_dh895xcc_dev0/version +--------------------------------------------------+ | Hardware and Software versions for device 0 | +--------------------------------------------------+ Hardware Version: A0 SKU2 Firmware Version: 2.6.0 MMP Version: 1.0.0 Driver Version: 2.6.0 Lowest Compatible Driver: 2.3 QuickAssist API CY Version: 1.9 QuickAssist API DC Version: 1.6 +--------------------------------------------------+ [root@localhost ~]# cat /proc/icp_dh895xcc_dev1/version +--------------------------------------------------+ | Hardware and Software versions for device 1 | +--------------------------------------------------+ Hardware Version: A0 SKU2 Firmware Version: 2.6.0 MMP Version: 1.0.0 Driver Version: 2.6.0 Lowest Compatible Driver: 2.3 QuickAssist API CY Version: 1.9 QuickAssist API DC Version: 1.6 +--------------------------------------------------+ [root@localhost ~]# The # lspci -d:443 confirms the bdf of the 32 VF devices are available per QAT device. [root@localhost ~]# lspci -d:443 83:01.0 Co-processor: Intel Corporation DH895XCC Series QAT Virtual Function 83:01.1 Co-processor: Intel Corporation DH895XCC Series QAT Virtual Function 83:01.2 Co-processor: Intel Corporation DH895XCC Series QAT Virtual Function 83:01.3 Co-processor: Intel Corporation DH895XCC Series QAT Virtual Function ------------------------------------------------------------------------------------- ------------------------------------------------------------------------------------- 88:04.3 Co-processor: Intel Corporation DH895XCC Series QAT Virtual Function 88:04.4 Co-processor: Intel Corporation DH895XCC Series QAT Virtual Function 88:04.5 Co-processor: Intel Corporation DH895XCC Series QAT Virtual Function 88:04.6 Co-processor: Intel Corporation DH895XCC Series QAT Virtual Function 88:04.7 Co-processor: Intel Corporation DH895XCC Series QAT Virtual Function [root@localhost ~]# *DPDK configuration options in config/defconfig_x86_64-native-linuxapp-gcc* CONFIG_RTE_LIBRTE_PMD_AESNI_MB=n CONFIG_RTE_LIBRTE_CRYPTODEV_DEBUG=y CONFIG_RTE_LIBRTE_VIRTIO_PMD=n CONFIG_RTE_LIBRTE_VIRTIO_DEBUG_INIT=n CONFIG_RTE_LIBRTE_VIRTIO_DEBUG_DRIVER=n CONFIG_RTE_LIBRTE_PMD_QAT=y CONFIG_RTE_LIBRTE_PMD_QAT_DEBUG_INIT=y CONFIG_RTE_LIBRTE_PMD_QAT_DEBUG_TX=y CONFIG_RTE_LIBRTE_PMD_QAT_DEBUG_RX=y CONFIG_RTE_LIBRTE_PMD_QAT_DEBUG_DRIVER=y CONFIG_RTE_MAX_MEMSEG=4096 [root@localhost dpdk-2.2.0]# Here goes my setup.sh make config T=x86_64-native-linuxapp-gcc sed -ri 's,(PMD_PCAP=).*,\1y,' build/.config make install T=x86_64-native-linuxapp-gcc mkdir -p /mnt/huge mount -t hugetlbfs nodev /mnt/huge echo 8192 > /sys/devices/system/node/node0/hugepages/hugepages-2048kB/nr_hugepages echo 8192 > /sys/devices/system/node/node1/hugepages/hugepages-2048kB/nr_hugepages sudo modprobe uio_pci_generic sudo modprobe uio rmmod igb_uio insmod ./x86_64-native-linuxapp-gcc/build/lib/librte_eal/linuxapp/igb_uio/igb_uio.ko echo -n "0000:83:01.0" > /sys/bus/pci/devices/0000\:83\:01.0/driver/unbind echo "8086 0443" > /sys/bus/pci/drivers/igb_uio/new_id The below OP confirms that, the QAT VF is bound to the DPDK UIO driver. 83:01.0 Co-processor: Intel Corporation DH895XCC Series QAT Virtual Function Subsystem: Intel Corporation Device 0000 Control: I/O- Mem- BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR- FastB2B- DisINTx- Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort- SERR-