From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mail-lb0-f172.google.com (mail-lb0-f172.google.com [209.85.217.172]) by dpdk.org (Postfix) with ESMTP id A604A8E7F for ; Wed, 22 Jun 2016 10:09:33 +0200 (CEST) Received: by mail-lb0-f172.google.com with SMTP id o4so17528469lbp.2 for ; Wed, 22 Jun 2016 01:09:33 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:from:date:message-id:subject:to :cc; bh=9Kk81gpFL2WqbyK532jeAlGcXJ3TE4dR1OeJO8yREGE=; b=ggiQzKV2aqoLs9S7aFN1I2dfPbQRJj93s7YSXc2OZrcn7v+qOYsrDK26Z8/3/rDmXM DvU9HfhKHVqyEESTA7/rNINCw0PjcKDL9RdFIAWIOpuLe1bGBK9kvoHk6yPhB3jffzUn 0ceVvsNvHIEVC0kQnNDlJwKia+X+h+Q7S/IIyxMWY1k2KoCgIR9SjMG6eiaVDGOMHiY1 Jw9TCTfzdPg0Dyjnka3tIhODc4JRIC4+tKPTYTCdMCCSi9iAyuXfBk4DZxOFNAy1z7uc IlwvNBHiJP0liBeT9bTCjzTQYmr/WZdxavvGwOuGecgCk+ntB1Sl2VlcdHrz87ChcLXl /ZGQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:mime-version:in-reply-to:references:from:date :message-id:subject:to:cc; bh=9Kk81gpFL2WqbyK532jeAlGcXJ3TE4dR1OeJO8yREGE=; b=CIL+OnOhmmy035+pjcLdrP8VKZatFB6JWYnz0+KcsJPXSOGeFZ9ykwk0tv3HKthHfQ i9rNtww4lLpE80iupL4BhOMaTB99Moi1OI4WoclK6McxUiy2iS12yF47ZKC7rNdH2Hk7 3ESZFTvSWfypxueo3CGX4pNLzbSDJ3EwieFniGTg5pLzVgVck4450rLzXfW/T/l5wpoh Q8xObyJeRkv807FW6AQB2+JT9P8AfNAyFdfkqO1QeieNcUoOJdKD+oJqlzsL62p+bHlD uHxoypLPsStnk77/QCTap9m2k6m/fV4F+HzKbdni/RqPiWe81vHlB97SLafXPWaBe53K pvcg== X-Gm-Message-State: ALyK8tK3mvzeGYibx/uUAcROdjX0mLRxCE3f+qFYxVBxUmIL4S72xGz3AOe7q098t6GQlkyWl6pJSONrXlT3Ig== X-Received: by 10.194.133.98 with SMTP id pb2mr23094146wjb.144.1466582972941; Wed, 22 Jun 2016 01:09:32 -0700 (PDT) MIME-Version: 1.0 Received: by 10.194.14.1 with HTTP; Wed, 22 Jun 2016 01:09:32 -0700 (PDT) In-Reply-To: References: <6e0329b1-d67c-3b23-55d2-dc88b78f6f2a@intel.com> From: Chinmaya Dwibedy Date: Wed, 22 Jun 2016 13:39:32 +0530 Message-ID: To: Sergio Gonzalez Monroy Cc: users@dpdk.org Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable X-Content-Filtered-By: Mailman/MimeDel 2.1.15 Subject: Re: [dpdk-users] Issue with ipsec-secgw sample application on VM using Intel QAT device (pass-through mode) X-BeenThere: users@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: usage discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 22 Jun 2016 08:09:33 -0000 Hi Sergio, Thank your for clarification. I am using dpdk-2.2.0. Here goes our configuration on VM. Can you please have a look into the same and let me know if anything is wrong. Will try to run the autotest, ipsec-segw on host as per your suggestion. [root@vpn-s ~]# uname -r 3.12.9-301.fc20.x86_64 [root@vpn-s ~]# [root@vpn-s ~]# cat /etc/redhat-release Fedora release 20 (Heisenbug) [root@vpn-s ~]# [root@vpn-s ~]# cat /proc/cmdline BOOT_IMAGE=3D/boot/vmlinuz-3.12.9-301.fc20.x86_64 root=3DUUID=3Dab47cbc9-68ee-403c-96a5-184e68238e65 ro console=3DttyS0,11520= 0n8 iommu=3Dpt intel_iommu=3Don [root@vpn-s ~]# [root@vpn-s QAT]# grep -i "iommu.*enabled" /var/log/messages Jun 22 07:31:17 vpn-s kernel: [ 0.000000] Intel-IOMMU: enabled [root@vpn-s QAT]# [root@vpn-s dpdk-2.2.0]# lspci -nn | grep 0443 00:06.0 Co-processor [0b40]: Intel Corporation Device [8086:0443] [root@vpn-s dpdk-2.2.0]# Installed Intel qat driver installation on VM (downloaded from https://01.org/packet-processing/intel%C2%AE-quickassist-technology-drivers= -and-patches) and followed the https://01.org/sites/default/files/page/330750-005_qat_gsg.pdf guide. Note, used the following installation options i.e., ./installer.sh install QAT1.6 guest. After this , it detects QAT VF. [root@vpn-s QAT]# service qat_service status There is 1 acceleration device(s) in the system: icp_dev0 - type=3Ddh895xcc, inst_id=3D0, node_id=3D0, bdf=3D00:06:0, #acc= el=3D1, #engines=3D1, state=3Dup [root@vpn-s QAT]# [root@vpn-s QAT]# lsmod | grep qa icp_qa_al_vf 813442 1 [root@vpn-s QAT]# The below are DPDK configuration options in config/defconfig_x86_64-native-linuxapp-gcc. include "common_linuxapp" CONFIG_RTE_MACHINE=3D"native" CONFIG_RTE_ARCH=3D"x86_64" CONFIG_RTE_ARCH_X86_64=3Dy CONFIG_RTE_ARCH_64=3Dy CONFIG_RTE_TOOLCHAIN=3D"gcc" CONFIG_RTE_TOOLCHAIN_GCC=3Dy CONFIG_RTE_LIBRTE_PMD_AESNI_MB=3Dy CONFIG_RTE_LIBRTE_CRYPTODEV_DEBUG=3Dy CONFIG_RTE_LIBRTE_VIRTIO_PMD=3Dy CONFIG_RTE_LIBRTE_VIRTIO_DEBUG_INIT=3Dy CONFIG_RTE_LIBRTE_VIRTIO_DEBUG_DRIVER=3Dy CONFIG_RTE_LIBRTE_PMD_QAT=3Dy CONFIG_RTE_QAT_PMD_MAX_NB_SESSIONS=3D128 CONFIG_RTE_LIBRTE_PMD_QAT_DEBUG_INIT=3Dy CONFIG_RTE_LIBRTE_PMD_QAT_DEBUG_TX=3Dy CONFIG_RTE_LIBRTE_PMD_QAT_DEBUG_RX=3Dy CONFIG_RTE_LIBRTE_PMD_QAT_DEBUG_DRIVER=3Dy The below is my setup.sh export RTE_SDK=3D/opt/dpdk-2.2.0 export RTE_TARGET=3Dx86_64-native-linuxapp-gcc export RTE_ARCH=3Dx86_64 export AESNI_MULTI_BUFFER_LIB_PATH=3D/opt/code make config T=3Dx86_64-native-linuxapp-gcc sed -ri 's,(PMD_PCAP=3D).*,\1y,' build/.config make install T=3Dx86_64-native-linuxapp-gcc mkdir -p /mnt/huge mount -t hugetlbfs nodev /mnt/huge echo 2048 > /sys/devices/system/node/node0/hugepages/hugepages-2048kB/nr_hugepages sudo modprobe uio_pci_generic sudo modprobe uio rmmod igb_uio insmod ./x86_64-native-linuxapp-gcc/build/lib/librte_eal/linuxapp/igb_uio/igb_uio.= ko echo -n "0000:00:06.0" > /sys/bus/pci/devices/0000\:00\:06.0/driver/unbind echo "8086 0443" > /sys/bus/pci/drivers/igb_uio/new_id lspci -vvd:443 ./tools/dpdk_nic_bind.py --bind=3Duio_pci_generic eth1 ./tools/dpdk_nic_bind.py --bind=3Duio_pci_generic 00:03.0 ./tools/dpdk_nic_bind.py -s Also modified the cryptodevs_init() ( examples/ipsec-secgw/ipsec-secgw.c) to rte_cryptodev_start(cdev_id); after rte_cryptodev_queue_pair_setup(). The purpose of calling this to start the transmit and the receive units of the QAT device. It returns success. But still getting the same issue i.e. rte_cryptodev_dequeue_burst () does not return crypto packet. Note that, when I call rte_cryptodev_queue_pair_setup () with 3rd argument as NULL (in which case default configuration will be used) , it crashes. Is it a known issue? Regards, Chinmaya On Tue, Jun 21, 2016 at 9:10 PM, Sergio Gonzalez Monroy < sergio.gonzalez.monroy@intel.com> wrote: > On 21/06/2016 08:26, Chinmaya Dwibedy wrote: > >> >> Hi Sergio, >> >> >> As suggested by you, run =E2=80=98app/test' application and then run >> 'cryptodev_qat_autotest' . Found the same issue i.e., It halts in >> rte_cryptodev_dequeue_burst () (while loop) in process_crypto_request () >> function (app/test/test_cryptodev.c). >> >> Debugged the qat_crypto_pkt_rx_burst() (in >> drivers/crypto/qat/qat_crypto.c) and found that , the value of resp_msg = is >> 0x7F7F7F7F (i.e., ADF_RING_EMPTY_SIG) . Thus it is not getting crypto >> packet (i.e., rte_mbuf) from the RX queue. But I find that, in >> qat_crypto_pkt_tx_burst(), the qat_alg_write_mbuf_entry() is being call= ed. >> The qat_alg_write_mbuf_entry () prints qat_req using rte_hexdump () and >> returns zero. >> >> >> One question, the base_addr + tail of tx_q (in qat_crypto_pkt_tx_burst()= ) >> should be same as base_addr + head (in qat_crypto_pkt_rx_burst()). Right= ? >> Please feel free to correct me if I am wrong. >> >> >> Also can you please provide some pointers for further debugging? Would i= t >> be configuration issue? Note that, I am using VF in VM. >> >> > In theory you should be able to run ipsec-secgw sample app with QAT VF a= s > long as the setup is right. > That the autotest fails would likely point to some possible issues with > the setup. > > Have you tried to run the autotest in the host? > > Sergio > > >> Thank you once again for your prompt response and great support. >> >> >> Regards, >> >> Chinmaya >> >> >> On Mon, Jun 20, 2016 at 3:04 PM, Sergio Gonzalez Monroy < >> sergio.gonzalez.monroy@intel.com > >> wrote: >> >> On 20/06/2016 10:08, Chinmaya Dwibedy wrote: >> >> >> Hi Sergio, >> >> >> Agreed. We might not dequeue the same amount of crypto ops we >> just previously enqueued, it's asynchronous. But in this case, >> I have sent just one UDP packet. So there will be one crypto >> ops. Right? Also I put a sleep (50) after the >> rte_crypto_enqueue_burst() function in ipsec_processing() >> (ipsec.c) , so as to allow more time ( for QAT device) for >> processing. Still getting the same result i.e., the >> rte_crypto_dequeue_burst () functionreturns zero. >> >> >> In case of S/W crypto device (i.e., AESNI), the VM gets >> inbound UDP packets on Port 1/eth1, encapsulates (after >> consulting its SPD) in an IPsec ESP packet and sends to its >> peer through Port 0/eth0 interface. >> >> >> Yes, the security policy, security association and Routing >> entries/configurations are exactly same. Please feel free to >> let me know if you need additional information. >> >> >> Could you try to run 'app/test' application then run >> 'cryptodev_qat_autotest' ? That is a functional test for cryptodev >> QAT PMD. >> >> Sergio >> >> >> >