From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mail-wm0-f44.google.com (mail-wm0-f44.google.com [74.125.82.44]) by dpdk.org (Postfix) with ESMTP id DCC3A9AB8 for ; Mon, 20 Jun 2016 09:33:26 +0200 (CEST) Received: by mail-wm0-f44.google.com with SMTP id v199so56894472wmv.0 for ; Mon, 20 Jun 2016 00:33:26 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:from:date:message-id:subject:to :cc; bh=ntElV4fFuo43pkH83pEhHFipyAFQnOaZlyP3hrSVduk=; b=A/zDgrDMMaiIH+Z3R06q2L6IWNmfR+NZhvyd5CBye85Nn3qo2zrtuC8oVZdfjm1492 JnS1i//a4jk+IIFsSYJpq19zWZzSF0UBBlJWuQDmsPA9wCGb8wTS61JWsdkjVJ4JAoS+ pBfiyssh/xtxkhMMhG0HFCAwJvvF36DSJjH5X3oVBKS4jg6EA2nl72XiveEVuKCNcjH+ mmBT/Loca4dATYFQiEVvsaqYXf+YCxXl3iZLci5d4Dd8ua4a4H6KsjesM/MeEYQnuUD8 pjZh0Xrg5J375E4B1mikel71fT4o5E7mBEnORaYPGEQzBFVnLhx6dx3R+GY2jSIB1hPp LW0w== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:mime-version:in-reply-to:references:from:date :message-id:subject:to:cc; bh=ntElV4fFuo43pkH83pEhHFipyAFQnOaZlyP3hrSVduk=; b=DVFNP90f84T7qUUk6EGZovAN7OwzZkuMu7Z7zF1Yj+lwmRsvDVUASpGiab00sIjkE2 8MWvhuSwDRwItS2AgqMxH/1tfvw4H8XMGJJfglexGm99fyS3FU3Ot8uEs0NNJQfQyu+1 g/g8aCwsRH3vc3Zj+deMHBGdJiOR75XUEjFdHSgDOXoo/CSbBV9Vpj3Y/HK8JUBqFxmy LvHPSLwTnxUg61aVE/qoxoeY/69tlGym1RC1GYAeldq28CmxWonjL8rU5HnHa0koT/49 y+P8QUIgQE4bSWyJwKWnXWKcI2Qynq88Xkw4aGnbJ4ujqDZe/PoF9qNbL8zRKMxVIFmk fIqA== X-Gm-Message-State: ALyK8tJv9Vfq6nEWljzd58sfbNPaSUuUSvRd/A0r49CDfpu5ExDdycS+1iyYwrqcRGanb44qO+HMDxFBfOgOGg== X-Received: by 10.194.56.134 with SMTP id a6mr3984179wjq.106.1466408006492; Mon, 20 Jun 2016 00:33:26 -0700 (PDT) MIME-Version: 1.0 Received: by 10.194.14.1 with HTTP; Mon, 20 Jun 2016 00:33:25 -0700 (PDT) In-Reply-To: References: From: Chinmaya Dwibedy Date: Mon, 20 Jun 2016 13:03:25 +0530 Message-ID: To: Sergio Gonzalez Monroy Cc: users@dpdk.org Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable X-Content-Filtered-By: Mailman/MimeDel 2.1.15 Subject: Re: [dpdk-users] Issue with ipsec-secgw sample application on VM using Intel QAT device (pass-through mode) X-BeenThere: users@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: usage discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 20 Jun 2016 07:33:27 -0000 Hi All, @Sergio: Thank you for your valuable suggestion. I passed through one of VFs and it detected the QAT device in VM. I just sent one UDP datagram which has to be encapsulated using H/W crypto device (i.e., Intel QAT). I run the ipsec-segw application with following arguments. ./build/ipsec-secgw -l 0 -n 4 -- -p 0x3 -P --config=3D"(0,0,0),(1,0,0)" --cdev QAT --ep0. Found that, the rte_crypto_enqueue_burst() function returns one. It means it could able to suceesfully place packet on the queue =E2=80=9Cqueue_id=E2=80=9D of the QAT= device designated by its =E2=80=9Cdev_id=E2=80=9D*. But The rte_crypto_dequeue_bu= rst() function returns zero. It means it cannot dequeue the packet and that packet might not have been processed by QAT device. Please note that, I have tested the same application with AESNI device and did not encounter any such issue. Furthermore the rte_crypto_dequeue_burst() API does not provide any error notification. Can anyone please suggest what might be the issue and is there any way to debug further? Here are the configuration options and console logs . [root@vpn-s ipsec-secgw]# lspci -nn | grep 0443 00:06.0 Co-processor [0b40]: Intel Corporation Device [8086:0443] [root@vpn-s ipsec-secgw]# I have configured the followings in config file CONFIG_RTE_LIBRTE_PMD_QAT=3Dy CONFIG_RTE_LIBRTE_PMD_QAT_DEBUG_INIT=3Dy CONFIG_RTE_LIBRTE_PMD_QAT_DEBUG_TX=3Dy CONFIG_RTE_LIBRTE_PMD_QAT_DEBUG_RX=3Dy CONFIG_RTE_LIBRTE_PMD_QAT_DEBUG_DRIVER=3Dy Console log while running ipsec-segw sample application PMD: crypto_qat_dev_init(): >> PMD: crypto_qat_dev_init(): Found crypto device at 00:06.0 Promiscuous mode selected endpoint 0 Found 1 QAT devices PMD: qat_dev_info_get(): >> Initialising QAT device 0 CRYPTODEV: rte_cryptodev_queue_pairs_config() line 467: Setup 1 queues pairs on device 0 PMD: qat_dev_info_get(): >> PMD: qat_crypto_sym_session_init(): >> PMD: qat_crypto_sym_session_init(): >> PMD: qat_crypto_sym_session_init(): >> PMD: qat_crypto_sym_session_init(): >> PMD: qat_crypto_sym_session_init(): >> PMD: qat_crypto_sym_session_init(): >> PMD: qat_crypto_sym_session_init(): >> PMD: qat_crypto_sym_session_init(): >> PMD: qat_crypto_sym_session_init(): >> PMD: qat_crypto_sym_session_init(): >> PMD: qat_crypto_sym_session_init(): >> PMD: qat_crypto_sym_session_init(): >> PMD: qat_crypto_sym_session_init(): >> PMD: qat_crypto_sym_session_init(): >> CRYPTODEV: rte_crypto_session_pool_create() line 1031: cdev_0_sess_mp mempool created! PMD: qat_crypto_sym_qp_setup(): >> PMD: qat_tx_queue_create(): >> PMD: qat_tx_queue_create(): TX ring for 2048 msgs: qp_id 0, bundle 0, ring = 2 PMD: qat_queue_create(): >> PMD: queue_dma_zone_reserve(): >> PMD: queue_dma_zone_reserve(): Allocate memzone for rte_qat_pmd_qp_mem_0_0_2, size 262144 on socket 0 PMD: qat_qp_check_queue_alignment(): >> PMD: adf_verify_queue_size(): >> PMD: qat_queue_create(): RING size in CSR: 12, in bytes 262144, nb msgs 2048, msg_size 128, max_inflights 2047 modulo 18 PMD: qat_rx_queue_create(): >> PMD: qat_rx_queue_create(): RX ring for 2048 msgs: qp id 0, bundle 0, ring 10 PMD: qat_queue_create(): >> PMD: queue_dma_zone_reserve(): >> PMD: queue_dma_zone_reserve(): Allocate memzone for rte_qat_pmd_qp_mem_0_0_10, size 65536 on socket 0 PMD: qat_qp_check_queue_alignment(): >> PMD: adf_verify_queue_size(): >> PMD: qat_queue_create(): RING size in CSR: 10, in bytes 65536, nb msgs 2048, msg_size 32, max_inflights 2047 modulo 16 PMD: adf_configure_queues(): >> PMD: adf_queue_arb_enable(): >> PMD: qat_dev_info_get(): >> Creating SA context with 64 maximum entries Creating SA context with 64 maximum entries PMD: qat_crypto_sym_configure_session(): >> PMD: qat_alg_aead_session_create_content_desc(): >> PMD: qat_alg_do_precomputes(): >> PMD: partial_hash_compute(): >> PMD: partial_hash_compute(): >> PMD: qat_alg_init_common_hdr(): >> PMD: qat_crypto_sym_configure_session(): >> PMD: qat_alg_aead_session_create_content_desc(): >> PMD: qat_alg_do_precomputes(): >> PMD: partial_hash_compute(): >> PMD: partial_hash_compute(): >> PMD: qat_alg_init_common_hdr(): >> PMD: qat_crypto_sym_configure_session(): >> PMD: qat_alg_aead_session_create_content_desc(): >> PMD: partial_hash_compute(): >> PMD: qat_alg_init_common_hdr(): >> Creating SP context with 1000 max rules IPv4 sp_ipv4_in_0 entries [4]: 1:0.0.0.0/0 30.1.1.0/24 0 : 65535 0 : 65535 0x0/0x0 0x1-0x1-0x5 2:0.0.0.0/0 192.168.116.0/24 0 : 65535 0 : 65535 0x0/0x0 0x1-0x2-0x= 6 3:0.0.0.0/0 192.168.117.0/24 0 : 65535 0 : 65535 0x0/0x0 0x1-0x3-0x= 7 4:0.0.0.0/0 192.168.118.0/24 0 : 65535 0 : 65535 0x0/0x0 0x1-0x4-0x= 8 ACL: Gen phase for ACL "sp_ipv4_in_0": runtime memory footprint on socket 0: single nodes/bytes used: 4/32 quad nodes/vectors/bytes used: 5/19/152 DFA nodes/group64/bytes used: 1/4/4104 match nodes/bytes used: 4/512 total: 7008 bytes max limit: 18446744073709551615 bytes ACL: Build phase for ACL "sp_ipv4_in_0": node limit for tree split: 2048 nodes created: 14 memory consumed: 8388615 ACL: trie 0: number of rules: 4, indexes: 2 acl context @0x7faefcb88500 socket_id=3D0 alg=3D3 max_rules=3D1000 rule_size=3D96 num_rules=3D4 num_categories=3D1 num_tries=3D1 Creating SP context with 1000 max rules IPv4 sp_ipv4_out_0 entries [4]: 1:0.0.0.0/0 30.1.1.0/24 0 : 65535 0 : 65535 0x0/0x0 0x1-0x1-0x5 2:0.0.0.0/0 192.168.4.0/24 0 : 65535 0 : 65535 0x0/0x0 0x1-0x2-0x6 3:0.0.0.0/0 192.168.5.0/24 0 : 65535 0 : 65535 0x0/0x0 0x1-0x3-0x7 4:0.0.0.0/0 192.168.6.0/24 0 : 65535 0 : 65535 0x0/0x0 0x1-0x4-0x8 ACL: Gen phase for ACL "sp_ipv4_out_0": runtime memory footprint on socket 0: single nodes/bytes used: 4/32 quad nodes/vectors/bytes used: 5/19/152 DFA nodes/group64/bytes used: 1/4/4104 match nodes/bytes used: 4/512 total: 7008 bytes max limit: 18446744073709551615 bytes ACL: Build phase for ACL "sp_ipv4_out_0": node limit for tree split: 2048 ------------------------------------- Port 0 Link Up - speed 10000 Mbps - full-duplex Port 1 Link Up - speed 10000 Mbps - full-duplex IPSEC: entering main loop on lcore 0 IPSEC: -- lcoreid=3D0 portid=3D0 rxqueueid=3D0 IPSEC: -- lcoreid=3D0 portid=3D1 rxqueueid=3D0 IPSEC_ESP: pktlen 110, esp_len 24, digest_len 12, payload_len 110, pad_payload_len 112, block_sz 16, pad_len 2 IPSEC_ESP: pktlen 168 qat_req: at [0x7faefcb40000], len=3D128 00000000: 00 02 04 80 24 00 00 00 50 B4 42 B1 00 00 00 00 | ....$...P.B....= . 00000010: 00 00 28 00 00 00 00 00 80 FA A5 75 AF 7F 00 00 | ..(........u...= . 00000020: 12 FC 25 AA 00 00 00 00 12 FC 25 AA 00 00 00 00 | ..%.......%....= . 00000030: A8 00 00 00 A8 00 00 00 2C 00 00 00 70 00 00 00 | ........,...p..= . 00000040: 85 CD 08 06 74 43 34 53 29 A0 2E 16 B5 0D FC 6A | ....tC4S)......= j 00000050: 14 00 00 00 88 00 00 00 00 00 00 00 00 00 00 00 | ...............= . 00000060: AE FC 25 AA 00 00 00 00 00 00 00 0C 02 02 00 21 | ..%............= ! 00000070: 00 00 10 42 00 00 0C 0C 00 18 15 18 00 00 00 00 | ...B...........= . IPSEC: Cryptodev 0 queue 0: enqueued 1 packets (out of 1) IPSEC: Cryptodev 0 queue 0: dequeued 0 packets (out of 32) Regards, Chinmaya On Wed, Jun 15, 2016 at 6:05 PM, Sergio Gonzalez Monroy < sergio.gonzalez.monroy@intel.com> wrote: > On 15/06/2016 13:01, Chinmaya Dwibedy wrote: > >> Also the output lspci on VM as follows >> >> [root@vpn-c openssl-async]# lspci -nn | grep 0435 >> >> 00:05.0 Co-processor [0b40]: Intel Corporation Coleto Creek PCIe Endpoin= t >> [8086:0435] >> >> > This is the PF, not the VF. > > I run the ipsec-secgw sample application ( >> http://dpdk.org/browse/dpdk/tree/examples/ipsec-secgw) on VM with -cdev >> QAT >> (e.g., #./examples/ipsec-secgw/build/ipsec-secgw -l 0,1 -n 4 -- -p 0x3 >> -P >> --config=3D"(0,0,0), (1,0,1)" --cdev QAT --ep0) . I found that, >> rte_cryptodev_count_devtype() returns zero (i.e., no QAT crypto device >> found). >> >> >> Can anyone please suggest what might be the issue and the way to resolve >> this ? Also kindly let me know using QAT device (passed through mode) = by >> DPDK application is possible or not. Thank you in advance for your time >> and >> help. please feel free to let me know if additional information is neede= d. >> >> > I reckon that your issue is that you are trying to use the PF (not > supported by DPDK) > instead of the VF. > > From > http://dpdk.readthedocs.io/en/v2.2.0/cryptodevs/qat.html#installation-usi= ng-01-org-qat-driver > : > > "You can use|cat/proc/icp_dh895xcc_dev0/version|to confirm the driver is > correctly installed. > You can use|lspci-d:443|to confirm the bdf of the 32 VF devices are > available per|DH895xCC|device." > > You should pass-through one of the VFs. > > Sergio >