From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mail-wm0-f47.google.com (mail-wm0-f47.google.com [74.125.82.47]) by dpdk.org (Postfix) with ESMTP id 86BFCC462 for ; Wed, 15 Jun 2016 14:01:51 +0200 (CEST) Received: by mail-wm0-f47.google.com with SMTP id a66so21201096wme.0 for ; Wed, 15 Jun 2016 05:01:51 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:from:date:message-id:subject:to; bh=VIB4pCZpIQUX4CLLjbgyYbV4Oq92TYDlVvwbpqCPbLQ=; b=d+1lVwpiouLWMJykNb/k7q15b7j9qSSgeqqSHnXwcFwx+4dW1NP1yqaIC1Mhcf7Hqi ODH2TrzAsq2/jDfmzrYCOhFa6cJ7L0RzIVf+s7aHGgIgHH5Nmp5wNv6Nn7bi9Hnmn7ow Uos3X+pHiqQcYauYInSDKAUOH9iuUnEtpiyqzqHApfmY5l7LPUJ9mz/028WWaP8SDpsM 5v5IF7NHwS62hmLXKWScdHaohOo7Nyrh+WeA992XwzhsK5wBjiINv4/HUfhnQ3cY/sh4 47gG06ZMuUwlzU7xhJAEvQF+Bcdl3b8kJrIkT9lYphL/0ULJGIJLlMq3hBSh4sLxwJ1N VOKg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:mime-version:from:date:message-id:subject:to; bh=VIB4pCZpIQUX4CLLjbgyYbV4Oq92TYDlVvwbpqCPbLQ=; b=NHgqIAQ9zlVjSxVkALAX1To6gLqPsG5YWtdGzOOIdY//LOWBjQKB6sXXod7tphw4lY DDkw2q3dgR3aXKRrZQ2OOBpjpaNxnm2KJO4T5fIeTvfdMDdekpUkIPXWsBsWia79Z8ca sFlA5vJ3DazeRYKytbLUxuacMfgJHbY1WVO7XmNuAPFnHuiGQKa1ZkU7pNBHAgXaDx2W M9M09GfGFigw3MpC+LT84dGg5WSBN24SVYQ+0nceNvzLxBeTcODLnr1gwZyOe2JIVzYB Jv5IaeoSEln152WeFl27608XAAdWATl+5OgfWaw8Cojb9LanfJKk7Pnt0oWvLa2xsZUk RXXw== X-Gm-Message-State: ALyK8tJa/Cby2J3tlHlRTDlV7/YNMBmX5LaelHp3McvZIvD+xxTHyV0kDllKbsxiuqpk+HO4A0bRJocAj7b5dw== X-Received: by 10.194.133.98 with SMTP id pb2mr10857786wjb.144.1465992110967; Wed, 15 Jun 2016 05:01:50 -0700 (PDT) MIME-Version: 1.0 Received: by 10.194.14.1 with HTTP; Wed, 15 Jun 2016 05:01:50 -0700 (PDT) From: Chinmaya Dwibedy Date: Wed, 15 Jun 2016 17:31:50 +0530 Message-ID: To: users@dpdk.org Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable X-Content-Filtered-By: Mailman/MimeDel 2.1.15 Subject: [dpdk-users] Issue with ipsec-secgw sample application on VM using Intel QAT device (pass-through mode) X-BeenThere: users@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: usage discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 15 Jun 2016 12:01:51 -0000 Hi All, I have created two VM instances (Fedora release 20 (Heisenbug) using openstack. The kernel version of the VM : 3.12.9-301.fc20.x86_64. The compute node (real system has CentOS 7.2, kernel version: 3.10.0-327.18.2.el7.x86_64) has two Intel QAT devices. [root@localhost ~(keystone_admin)]# lspci -vd:0435 83:00.0 Co-processor: Intel Corporation DH895XCC Series QAT Subsystem: Intel Corporation Device 35c5 Physical Slot: 0-1 Flags: bus master, fast devsel, latency 0, IRQ 35 Memory at 387fffd00000 (64-bit, prefetchable) [size=3D512K] Memory at c8200000 (64-bit, non-prefetchable) [size=3D256K] Memory at c8240000 (64-bit, non-prefetchable) [size=3D256K] Capabilities: [b0] MSI: Enable- Count=3D1/1 Maskable+ 64bit+ Capabilities: [60] MSI-X: Enable+ Count=3D33 Masked- Capabilities: [6c] Power Management version 3 Capabilities: [74] Express Endpoint, MSI 00 Capabilities: [100] Advanced Error Reporting Capabilities: [138] Alternative Routing-ID Interpretation (ARI) Capabilities: [140] Single Root I/O Virtualization (SR-IOV) Kernel driver in use: vfio-pci 88:00.0 Co-processor: Intel Corporation DH895XCC Series QAT Subsystem: Intel Corporation Device 35c5 Physical Slot: 0-2 Flags: bus master, fast devsel, latency 0, IRQ 39 Memory at 387fffe00000 (64-bit, prefetchable) [size=3D512K] Memory at c8000000 (64-bit, non-prefetchable) [size=3D256K] Memory at c8040000 (64-bit, non-prefetchable) [size=3D256K] Capabilities: [b0] MSI: Enable- Count=3D1/1 Maskable+ 64bit+ Capabilities: [60] MSI-X: Enable+ Count=3D33 Masked- Capabilities: [6c] Power Management version 3 Capabilities: [74] Express Endpoint, MSI 00 Capabilities: [100] Advanced Error Reporting Capabilities: [138] Alternative Routing-ID Interpretation (ARI) Capabilities: [140] Single Root I/O Virtualization (SR-IOV) Kernel driver in use: vfio-pci [root@localhost ~(keystone_admin)]# I have configured both the QAT devices to be pass-through devices. So that, each VM will have exclusive access to an Intel QAT card. On VMs, I configured, build and installed latest Intel driver (qatmux.l.2.6.0-60) (downloaded from https://01.org/packet-processing/intel%C2%AE-quickassist-technology-drivers= -and-patches). Started the driver and checked via #service qat_service status and found that, it detects 1 acceleration device(s) in the system. [root@vpn-c openssl-async]# service qat_service status There is 1 acceleration device(s) in the system: icp_dev0 - type=3Ddh895xcc, inst_id=3D0, node_id=3D0, bdf=3D00:05:0, #acc= el=3D6, #engines=3D12, state=3Dup [root@vpn-c openssl-async]# Also the output lspci on VM as follows [root@vpn-c openssl-async]# lspci -nn | grep 0435 00:05.0 Co-processor [0b40]: Intel Corporation Coleto Creek PCIe Endpoint [8086:0435] [root@vpn-c openssl-async]# I have configured =E2=80=9Cintel_iommu=3Doff=E2=80=9D in boot arguments (VM= s). [root@vpn-c dpdk-2.2.0]# cat /proc/cmdline BOOT_IMAGE=3D/boot/vmlinuz-3.12.9-301.fc20.x86_64 root=3DUUID=3Dab47cbc9-68ee-403c-96a5-184e68238e65 ro console=3DttyS0,11520= 0n8 intel_iommu=3Doff [root@vpn-c dpdk-2.2.0]# I run the ipsec-secgw sample application ( http://dpdk.org/browse/dpdk/tree/examples/ipsec-secgw) on VM with -cdev QAT (e.g., #./examples/ipsec-secgw/build/ipsec-secgw -l 0,1 -n 4 -- -p 0x3 -P --config=3D"(0,0,0), (1,0,1)" --cdev QAT --ep0) . I found that, rte_cryptodev_count_devtype() returns zero (i.e., no QAT crypto device found). Can anyone please suggest what might be the issue and the way to resolve this ? Also kindly let me know using QAT device (passed through mode) by DPDK application is possible or not. Thank you in advance for your time and help. please feel free to let me know if additional information is needed. Here go the console logs. EAL: Master lcore 0 is ready (tid=3De110a940;cpuset=3D[0]) EAL: lcore 1 is ready (tid=3D2f0b7700;cpuset=3D[1]) EAL: PCI device 0000:00:03.0 on NUMA socket -1 EAL: probe driver: 1af4:1000 rte_virtio_pmd PMD: parse_sysfs_value(): parse_sysfs_value(): cannot open sysfs value /sys/bus/pci/devices/0000:00:03.0/uio/uio1/portio/port0/size PMD: virtio_resource_init_by_uio(): virtio_resource_init_by_uio(): cannot parse size PMD: virtio_resource_init_by_ioports(): PCI Port IO found start=3D0xc060 wi= th size=3D0x20 PMD: virtio_negotiate_features(): guest_features before negotiate =3D cf802= 0 PMD: virtio_negotiate_features(): host_features before negotiate =3D 799fff= e3 PMD: virtio_negotiate_features(): features after negotiate =3D 8f8020 PMD: eth_virtio_dev_init(): PORT MAC: FA:16:3E:BB:8A:DC PMD: eth_virtio_dev_init(): VIRTIO_NET_F_MQ is not supported PMD: virtio_dev_cq_queue_setup(): >> PMD: virtio_dev_queue_setup(): selecting queue: 2 PMD: virtio_dev_queue_setup(): vq_size: 64 nb_desc:0 PMD: virtio_dev_queue_setup(): vring_size: 4612, rounded_vring_size: 8192 PMD: virtio_dev_queue_setup(): vq->vq_ring_mem: 0x219143000 PMD: virtio_dev_queue_setup(): vq->vq_ring_virt_mem: 0x7fcf30f43000 PMD: eth_virtio_dev_init(): config->max_virtqueue_pairs=3D1 PMD: eth_virtio_dev_init(): config->status=3D1 PMD: eth_virtio_dev_init(): PORT MAC: FA:16:3E:BB:8A:DC PMD: eth_virtio_dev_init(): hw->max_rx_queues=3D1 hw->max_tx_queues=3D1 PMD: eth_virtio_dev_init(): port 0 vendorID=3D0x1af4 deviceID=3D0x1000 PMD: virtio_dev_vring_start(): >> EAL: PCI device 0000:00:06.0 on NUMA socket -1 EAL: probe driver: 8086:10fb rte_ixgbe_pmd EAL: PCI memory mapped at 0x7fd0dee00000 EAL: PCI memory mapped at 0x7fd0def00000 PMD: eth_ixgbe_dev_init(): MAC: 2, PHY: 15, SFP+: 5 PMD: eth_ixgbe_dev_init(): port 1 vendorID=3D0x8086 deviceID=3D0x10fb Promiscuous mode selected endpoint 0 Found 0 QAT devices PANIC in cryptodevs_init(): Not enough QAT devices detected, need 2 (1 per core), found 0 5: [./examples/ipsec-secgw/build/ipsec-secgw() [0x42ebf5]] 4: [/lib64/libc.so.6(__libc_start_main+0xf5) [0x7fd0dfe49d65]] 3: [./examples/ipsec-secgw/build/ipsec-secgw(main+0xd75) [0x42d695]] 2: [./examples/ipsec-secgw/build/ipsec-secgw(__rte_panic+0xc9) [0x42733e]] 1: [./examples/ipsec-secgw/build/ipsec-secgw(rte_dump_stack+0x1a) [0x49561a]] ./run.sh: line 5: 8202 Aborted (core dumped) ./examples/ipsec-secgw/build/ipsec-secgw -l 0,1 -n 4 -- -p 0x3 -P --config=3D"(0,0,0), (1,0,1)" --cdev QAT --ep0 Regards, Chinmaya