From: Chinmaya Dwibedy <ckdwibedy@gmail.com>
To: Sergio Gonzalez Monroy <sergio.gonzalez.monroy@intel.com>
Cc: users@dpdk.org
Subject: Re: [dpdk-users] Issue with ipsec-secgw sample application on VM using Intel QAT device (pass-through mode)
Date: Wed, 22 Jun 2016 13:39:32 +0530 [thread overview]
Message-ID: <CAMHS8RG0FGTuUU_pViDe8bXcUR-+QoLS0yyjZFCsOp-hwjy+Fg@mail.gmail.com> (raw)
In-Reply-To: <b7d02aa4-332a-7b1f-1f5c-608d72061c67@intel.com>
Hi Sergio,
Thank your for clarification. I am using dpdk-2.2.0. Here goes our
configuration on VM. Can you please have a look into the same and let me
know if anything is wrong. Will try to run the autotest, ipsec-segw on host
as per your suggestion.
[root@vpn-s ~]# uname -r
3.12.9-301.fc20.x86_64
[root@vpn-s ~]#
[root@vpn-s ~]# cat /etc/redhat-release
Fedora release 20 (Heisenbug)
[root@vpn-s ~]#
[root@vpn-s ~]# cat /proc/cmdline
BOOT_IMAGE=/boot/vmlinuz-3.12.9-301.fc20.x86_64
root=UUID=ab47cbc9-68ee-403c-96a5-184e68238e65 ro console=ttyS0,115200n8
iommu=pt intel_iommu=on
[root@vpn-s ~]#
[root@vpn-s QAT]# grep -i "iommu.*enabled" /var/log/messages
Jun 22 07:31:17 vpn-s kernel: [ 0.000000] Intel-IOMMU: enabled
[root@vpn-s QAT]#
[root@vpn-s dpdk-2.2.0]# lspci -nn | grep 0443
00:06.0 Co-processor [0b40]: Intel Corporation Device [8086:0443]
[root@vpn-s dpdk-2.2.0]#
Installed Intel qat driver installation on VM (downloaded from
https://01.org/packet-processing/intel%C2%AE-quickassist-technology-drivers-and-patches)
and followed the
https://01.org/sites/default/files/page/330750-005_qat_gsg.pdf guide.
Note, used the following installation options i.e., ./installer.sh install
QAT1.6 guest. After this , it detects QAT VF.
[root@vpn-s QAT]# service qat_service status
There is 1 acceleration device(s) in the system:
icp_dev0 - type=dh895xcc, inst_id=0, node_id=0, bdf=00:06:0, #accel=1,
#engines=1, state=up
[root@vpn-s QAT]#
[root@vpn-s QAT]# lsmod | grep qa
icp_qa_al_vf 813442 1
[root@vpn-s QAT]#
The below are DPDK configuration options in
config/defconfig_x86_64-native-linuxapp-gcc.
include "common_linuxapp"
CONFIG_RTE_MACHINE="native"
CONFIG_RTE_ARCH="x86_64"
CONFIG_RTE_ARCH_X86_64=y
CONFIG_RTE_ARCH_64=y
CONFIG_RTE_TOOLCHAIN="gcc"
CONFIG_RTE_TOOLCHAIN_GCC=y
CONFIG_RTE_LIBRTE_PMD_AESNI_MB=y
CONFIG_RTE_LIBRTE_CRYPTODEV_DEBUG=y
CONFIG_RTE_LIBRTE_VIRTIO_PMD=y
CONFIG_RTE_LIBRTE_VIRTIO_DEBUG_INIT=y
CONFIG_RTE_LIBRTE_VIRTIO_DEBUG_DRIVER=y
CONFIG_RTE_LIBRTE_PMD_QAT=y
CONFIG_RTE_QAT_PMD_MAX_NB_SESSIONS=128
CONFIG_RTE_LIBRTE_PMD_QAT_DEBUG_INIT=y
CONFIG_RTE_LIBRTE_PMD_QAT_DEBUG_TX=y
CONFIG_RTE_LIBRTE_PMD_QAT_DEBUG_RX=y
CONFIG_RTE_LIBRTE_PMD_QAT_DEBUG_DRIVER=y
The below is my setup.sh
export RTE_SDK=/opt/dpdk-2.2.0
export RTE_TARGET=x86_64-native-linuxapp-gcc
export RTE_ARCH=x86_64
export AESNI_MULTI_BUFFER_LIB_PATH=/opt/code
make config T=x86_64-native-linuxapp-gcc
sed -ri 's,(PMD_PCAP=).*,\1y,' build/.config
make install T=x86_64-native-linuxapp-gcc
mkdir -p /mnt/huge
mount -t hugetlbfs nodev /mnt/huge
echo 2048 >
/sys/devices/system/node/node0/hugepages/hugepages-2048kB/nr_hugepages
sudo modprobe uio_pci_generic
sudo modprobe uio
rmmod igb_uio
insmod
./x86_64-native-linuxapp-gcc/build/lib/librte_eal/linuxapp/igb_uio/igb_uio.ko
echo -n "0000:00:06.0" > /sys/bus/pci/devices/0000\:00\:06.0/driver/unbind
echo "8086 0443" > /sys/bus/pci/drivers/igb_uio/new_id
lspci -vvd:443
./tools/dpdk_nic_bind.py --bind=uio_pci_generic eth1
./tools/dpdk_nic_bind.py --bind=uio_pci_generic 00:03.0
./tools/dpdk_nic_bind.py -s
Also modified the cryptodevs_init() ( examples/ipsec-secgw/ipsec-secgw.c)
to rte_cryptodev_start(cdev_id); after rte_cryptodev_queue_pair_setup().
The purpose of calling this to start the transmit and the receive units of
the QAT device. It returns success. But still getting the same issue i.e.
rte_cryptodev_dequeue_burst () does not return crypto packet. Note that,
when I call rte_cryptodev_queue_pair_setup () with 3rd argument as NULL (in
which case default configuration will be used) , it crashes. Is it a known
issue?
Regards,
Chinmaya
On Tue, Jun 21, 2016 at 9:10 PM, Sergio Gonzalez Monroy <
sergio.gonzalez.monroy@intel.com> wrote:
> On 21/06/2016 08:26, Chinmaya Dwibedy wrote:
>
>>
>> Hi Sergio,
>>
>>
>> As suggested by you, run ‘app/test' application and then run
>> 'cryptodev_qat_autotest' . Found the same issue i.e., It halts in
>> rte_cryptodev_dequeue_burst () (while loop) in process_crypto_request ()
>> function (app/test/test_cryptodev.c).
>>
>> Debugged the qat_crypto_pkt_rx_burst() (in
>> drivers/crypto/qat/qat_crypto.c) and found that , the value of resp_msg is
>> 0x7F7F7F7F (i.e., ADF_RING_EMPTY_SIG) . Thus it is not getting crypto
>> packet (i.e., rte_mbuf) from the RX queue. But I find that, in
>> qat_crypto_pkt_tx_burst(), the qat_alg_write_mbuf_entry() is being called.
>> The qat_alg_write_mbuf_entry () prints qat_req using rte_hexdump () and
>> returns zero.
>>
>>
>> One question, the base_addr + tail of tx_q (in qat_crypto_pkt_tx_burst())
>> should be same as base_addr + head (in qat_crypto_pkt_rx_burst()). Right ?
>> Please feel free to correct me if I am wrong.
>>
>>
>> Also can you please provide some pointers for further debugging? Would it
>> be configuration issue? Note that, I am using VF in VM.
>>
>>
> In theory you should be able to run ipsec-secgw sample app with QAT VF as
> long as the setup is right.
> That the autotest fails would likely point to some possible issues with
> the setup.
>
> Have you tried to run the autotest in the host?
>
> Sergio
>
>
>> Thank you once again for your prompt response and great support.
>>
>>
>> Regards,
>>
>> Chinmaya
>>
>>
>> On Mon, Jun 20, 2016 at 3:04 PM, Sergio Gonzalez Monroy <
>> sergio.gonzalez.monroy@intel.com <mailto:sergio.gonzalez.monroy@intel.com>>
>> wrote:
>>
>> On 20/06/2016 10:08, Chinmaya Dwibedy wrote:
>>
>>
>> Hi Sergio,
>>
>>
>> Agreed. We might not dequeue the same amount of crypto ops we
>> just previously enqueued, it's asynchronous. But in this case,
>> I have sent just one UDP packet. So there will be one crypto
>> ops. Right? Also I put a sleep (50) after the
>> rte_crypto_enqueue_burst() function in ipsec_processing()
>> (ipsec.c) , so as to allow more time ( for QAT device) for
>> processing. Still getting the same result i.e., the
>> rte_crypto_dequeue_burst () functionreturns zero.
>>
>>
>> In case of S/W crypto device (i.e., AESNI), the VM gets
>> inbound UDP packets on Port 1/eth1, encapsulates (after
>> consulting its SPD) in an IPsec ESP packet and sends to its
>> peer through Port 0/eth0 interface.
>>
>>
>> Yes, the security policy, security association and Routing
>> entries/configurations are exactly same. Please feel free to
>> let me know if you need additional information.
>>
>>
>> Could you try to run 'app/test' application then run
>> 'cryptodev_qat_autotest' ? That is a functional test for cryptodev
>> QAT PMD.
>>
>> Sergio
>>
>>
>>
>
next prev parent reply other threads:[~2016-06-22 8:09 UTC|newest]
Thread overview: 11+ messages / expand[flat|nested] mbox.gz Atom feed top
2016-06-15 12:01 Chinmaya Dwibedy
2016-06-15 12:35 ` Sergio Gonzalez Monroy
2016-06-20 7:33 ` Chinmaya Dwibedy
2016-06-20 8:06 ` Sergio Gonzalez Monroy
2016-06-20 9:08 ` Chinmaya Dwibedy
2016-06-20 9:34 ` Sergio Gonzalez Monroy
2016-06-21 7:26 ` Chinmaya Dwibedy
2016-06-21 15:40 ` Sergio Gonzalez Monroy
2016-06-22 8:09 ` Chinmaya Dwibedy [this message]
2016-06-23 11:48 ` Chinmaya Dwibedy
2016-06-24 9:34 ` Chinmaya Dwibedy
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=CAMHS8RG0FGTuUU_pViDe8bXcUR-+QoLS0yyjZFCsOp-hwjy+Fg@mail.gmail.com \
--to=ckdwibedy@gmail.com \
--cc=sergio.gonzalez.monroy@intel.com \
--cc=users@dpdk.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).