From: Chinmaya Dwibedy <ckdwibedy@gmail.com>
To: users@dpdk.org
Subject: [dpdk-users] Issue with cryptodev_qat_autotest on CentOS 7.2 host system using dpdk-2.2.0
Date: Wed, 22 Jun 2016 19:27:11 +0530 [thread overview]
Message-ID: <CAMHS8RFMRbJM_ad9TZ+j-1q2vr80Q__1xHcLLrE+Y3qX7MW0AQ@mail.gmail.com> (raw)
Hi All,
I do have CentOS Linux release 7.2.1511 (Core) host system, where two Intel
QAT devices are installed. I have installed dpdk-2.2.0 and QAT driver as
mentioned in section 6.4. (i.e., Installation using 01.org QAT driver) of
http://dpdk.readthedocs.io/en/v16.04/cryptodevs/qat.html. I tested
"cryptodev_qat_autotest" under app/test. I found this test case to be
failed. It appears that, it the dequeue burst API fails to retrieve any
processed operations available from the queue pair (on the QAT Crypto
device), from the devices processed queue. I have captured and attached
the console.log here.
Can anyone please suggest, is it an issue with the setup/configuration or
QAT PMD code? Kindly point me to right direction whenever you find me
wrong.Thank
you in advance for your time and support.
Here goes my configuration.
[root@localhost ~]# service qat_service status
There is 2 acceleration device(s) in the system:
icp_dev0 - type=dh895xcc, inst_id=0, node_id=1, bdf=83:00:0, #accel=6,
#engines=12, state=up
icp_dev1 - type=dh895xcc, inst_id=1, node_id=1, bdf=88:00:0, #accel=6,
#engines=12, state=up
[root@localhost ~]# lspci -nn | grep 0435
83:00.0 Co-processor [0b40]: Intel Corporation DH895XCC Series QAT
[8086:0435]
88:00.0 Co-processor [0b40]: Intel Corporation DH895XCC Series QAT
[8086:0435]
[root@localhost ~]#
[root@localhost ~]# lspci -nn|grep -e "QAT.*Virtual"
83:01.0 Co-processor [0b40]: Intel Corporation DH895XCC Series QAT Virtual
Function [8086:0443]
83:01.1 Co-processor [0b40]: Intel Corporation DH895XCC Series QAT Virtual
Function [8086:0443]
83:01.2 Co-processor [0b40]: Intel Corporation DH895XCC Series QAT Virtual
Function [8086:0443]
83:01.3 Co-processor [0b40]: Intel Corporation DH895XCC Series QAT Virtual
Function [8086:0443]
83:01.4 Co-processor [0b40]: Intel Corporation DH895XCC Series QAT Virtual
Function [8086:0443]
83:01.5 Co-processor [0b40]: Intel Corporation DH895XCC Series QAT Virtual
Function [8086:0443]
-------------------------------------------------------------------------------------------------------------
-------------------------------------------------------------------------------------------------------------
88:04.1 Co-processor [0b40]: Intel Corporation DH895XCC Series QAT Virtual
Function [8086:0443]
88:04.2 Co-processor [0b40]: Intel Corporation DH895XCC Series QAT Virtual
Function [8086:0443]
88:04.3 Co-processor [0b40]: Intel Corporation DH895XCC Series QAT Virtual
Function [8086:0443]
88:04.4 Co-processor [0b40]: Intel Corporation DH895XCC Series QAT Virtual
Function [8086:0443]
88:04.5 Co-processor [0b40]: Intel Corporation DH895XCC Series QAT Virtual
Function [8086:0443]
88:04.6 Co-processor [0b40]: Intel Corporation DH895XCC Series QAT Virtual
Function [8086:0443]
88:04.7 Co-processor [0b40]: Intel Corporation DH895XCC Series QAT Virtual
Function [8086:0443]
[root@localhost ~]# lspci -nn|grep -e "QAT.*Virtual" | wc -l
64
[root@localhost ~]#
Note: This host is capable of PCI passthrough and SR-IOV.
[root@localhost ~]# grep -i dmar /var/log/dmesg
[ 0.000000] ACPI: DMAR 000000007b69a000 00130 (v01 INTEL S2600WT
00000001 INTL 20091013)
[ 0.135336] dmar: Host address width 46
[ 0.135587] dmar: DRHD base: 0x000000fbffc000 flags: 0x0
[ 0.135855] dmar: IOMMU 0: reg_base_addr fbffc000 ver 1:0 cap
d2078c106f0466 ecap f020de
[ 0.136309] dmar: DRHD base: 0x000000c7ffc000 flags: 0x1
[ 0.136569] dmar: IOMMU 1: reg_base_addr c7ffc000 ver 1:0 cap
d2078c106f0466 ecap f020de
[ 0.137016] dmar: RMRR base: 0x0000007a3db000 end: 0x0000007a3ddfff
[ 0.137274] dmar: ATSR flags: 0x0
[ 0.137522] dmar: ATSR flags: 0x0
[root@localhost ~]# grep -i "iommu.*enabled" /var/log/dmesg
[ 0.000000] Intel-IOMMU: enabled
[root@localhost test(keystone_admin)]# cat /proc/cmdline
BOOT_IMAGE=/vmlinuz-3.10.0-327.18.2.el7.x86_64 root=/dev/mapper/centos-root
ro rd.lvm.lv=centos/swap crashkernel=auto nomodeset rd.lvm.lv=centos/root
rhgb iommu=pt intel_iommu=on
[root@localhost test(keystone_admin)]#
The below output confirms that, QAT driver is correctly installed.
[root@localhost ~]# cat /proc/icp_dh895xcc_dev0/version
+--------------------------------------------------+
| Hardware and Software versions for device 0 |
+--------------------------------------------------+
Hardware Version: A0 SKU2
Firmware Version: 2.6.0
MMP Version: 1.0.0
Driver Version: 2.6.0
Lowest Compatible Driver: 2.3
QuickAssist API CY Version: 1.9
QuickAssist API DC Version: 1.6
+--------------------------------------------------+
[root@localhost ~]# cat /proc/icp_dh895xcc_dev1/version
+--------------------------------------------------+
| Hardware and Software versions for device 1 |
+--------------------------------------------------+
Hardware Version: A0 SKU2
Firmware Version: 2.6.0
MMP Version: 1.0.0
Driver Version: 2.6.0
Lowest Compatible Driver: 2.3
QuickAssist API CY Version: 1.9
QuickAssist API DC Version: 1.6
+--------------------------------------------------+
[root@localhost ~]#
The # lspci -d:443 confirms the bdf of the 32 VF devices are available per
QAT device.
[root@localhost ~]# lspci -d:443
83:01.0 Co-processor: Intel Corporation DH895XCC Series QAT Virtual Function
83:01.1 Co-processor: Intel Corporation DH895XCC Series QAT Virtual Function
83:01.2 Co-processor: Intel Corporation DH895XCC Series QAT Virtual Function
83:01.3 Co-processor: Intel Corporation DH895XCC Series QAT Virtual Function
-------------------------------------------------------------------------------------
-------------------------------------------------------------------------------------
88:04.3 Co-processor: Intel Corporation DH895XCC Series QAT Virtual Function
88:04.4 Co-processor: Intel Corporation DH895XCC Series QAT Virtual Function
88:04.5 Co-processor: Intel Corporation DH895XCC Series QAT Virtual Function
88:04.6 Co-processor: Intel Corporation DH895XCC Series QAT Virtual Function
88:04.7 Co-processor: Intel Corporation DH895XCC Series QAT Virtual Function
[root@localhost ~]#
*DPDK configuration options in config/defconfig_x86_64-native-linuxapp-gcc*
CONFIG_RTE_LIBRTE_PMD_AESNI_MB=n
CONFIG_RTE_LIBRTE_CRYPTODEV_DEBUG=y
CONFIG_RTE_LIBRTE_VIRTIO_PMD=n
CONFIG_RTE_LIBRTE_VIRTIO_DEBUG_INIT=n
CONFIG_RTE_LIBRTE_VIRTIO_DEBUG_DRIVER=n
CONFIG_RTE_LIBRTE_PMD_QAT=y
CONFIG_RTE_LIBRTE_PMD_QAT_DEBUG_INIT=y
CONFIG_RTE_LIBRTE_PMD_QAT_DEBUG_TX=y
CONFIG_RTE_LIBRTE_PMD_QAT_DEBUG_RX=y
CONFIG_RTE_LIBRTE_PMD_QAT_DEBUG_DRIVER=y
CONFIG_RTE_MAX_MEMSEG=4096
[root@localhost dpdk-2.2.0]#
Here goes my setup.sh
make config T=x86_64-native-linuxapp-gcc
sed -ri 's,(PMD_PCAP=).*,\1y,' build/.config
make install T=x86_64-native-linuxapp-gcc
mkdir -p /mnt/huge
mount -t hugetlbfs nodev /mnt/huge
echo 8192 >
/sys/devices/system/node/node0/hugepages/hugepages-2048kB/nr_hugepages
echo 8192 >
/sys/devices/system/node/node1/hugepages/hugepages-2048kB/nr_hugepages
sudo modprobe uio_pci_generic
sudo modprobe uio
rmmod igb_uio
insmod
./x86_64-native-linuxapp-gcc/build/lib/librte_eal/linuxapp/igb_uio/igb_uio.ko
echo -n "0000:83:01.0" > /sys/bus/pci/devices/0000\:83\:01.0/driver/unbind
echo "8086 0443" > /sys/bus/pci/drivers/igb_uio/new_id
The below OP confirms that, the QAT VF is bound to the DPDK UIO driver.
83:01.0 Co-processor: Intel Corporation DH895XCC Series QAT Virtual Function
Subsystem: Intel Corporation Device 0000
Control: I/O- Mem- BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr-
Stepping- SERR- FastB2B- DisINTx-
Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort-
<TAbort- <MAbort- >SERR- <PERR- INTx-
Latency: 0
Region 0: [virtual] Memory at c8280000 (64-bit, non-prefetchable)
[size=4K]
Region 2: [virtual] Memory at c82a0000 (64-bit, non-prefetchable)
[size=4K]
Capabilities: [90] MSI: Enable- Count=1/1 Maskable+ 64bit+
Address: 0000000000000000 Data: 0000
Masking: 00000000 Pending: 00000000
Kernel driver in use: igb_uio
Regards,
Chinmaya
next reply other threads:[~2016-06-22 13:57 UTC|newest]
Thread overview: 3+ messages / expand[flat|nested] mbox.gz Atom feed top
2016-06-22 13:57 Chinmaya Dwibedy [this message]
2016-06-22 14:07 ` Chinmaya Dwibedy
2016-06-22 14:46 ` Chinmaya Dwibedy
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=CAMHS8RFMRbJM_ad9TZ+j-1q2vr80Q__1xHcLLrE+Y3qX7MW0AQ@mail.gmail.com \
--to=ckdwibedy@gmail.com \
--cc=users@dpdk.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).