* [dpdk-users] Issue with Physical Function passthrough on VM Guest (Fedora 20) using qatmux.l.2.6.0-60 @ 2016-06-10 7:16 Chinmaya Dwibedy 2016-06-10 7:33 ` SAKTHIVEL ANAND S 0 siblings, 1 reply; 3+ messages in thread From: Chinmaya Dwibedy @ 2016-06-10 7:16 UTC (permalink / raw) To: users Hi, I have assigned Intel QAT device to VMs as a full device (PF - physical function, PCI passthrough). Below is the output of lspci in host (Cent OS 7.2). [root@localhost nova(keystone_admin)]# lspci -nn | grep 0435 83:00.0 Co-processor [0b40]: Intel Corporation DH895XCC Series QAT [ 8086:0435] 88:00.0 Co-processor [0b40]: Intel Corporation DH895XCC Series QAT [ 8086:0435] [root@localhost nova(keystone_admin)]# [root@localhost ~]# lspci -vnd 8086:0435 83:00.0 0b40: 8086:0435 Subsystem: 8086:35c5 Physical Slot: 0-1 Flags: bus master, fast devsel, latency 0, IRQ 35 Memory at 387fffd00000 (64-bit, prefetchable) [size=512K] Memory at c8200000 (64-bit, non-prefetchable) [size=256K] Memory at c8240000 (64-bit, non-prefetchable) [size=256K] Capabilities: [b0] MSI: Enable- Count=1/1 Maskable+ 64bit+ Capabilities: [60] MSI-X: Enable- Count=33 Masked- Capabilities: [6c] Power Management version 3 Capabilities: [74] Express Endpoint, MSI 00 Capabilities: [100] Advanced Error Reporting Capabilities: [138] Alternative Routing-ID Interpretation (ARI) Capabilities: [140] Single Root I/O Virtualization (SR-IOV) Kernel driver in use: vfio-pci 88:00.0 0b40: 8086:0435 Subsystem: 8086:35c5 Physical Slot: 0-2 Flags: fast devsel, IRQ 39 Memory at 387fffe00000 (64-bit, prefetchable) [size=512K] Memory at c8000000 (64-bit, non-prefetchable) [size=256K] Memory at c8040000 (64-bit, non-prefetchable) [size=256K] Capabilities: [b0] MSI: Enable- Count=1/1 Maskable+ 64bit+ Capabilities: [60] MSI-X: Enable- Count=33 Masked- Capabilities: [6c] Power Management version 3 Capabilities: [74] Express Endpoint, MSI 00 Capabilities: [100] Advanced Error Reporting Capabilities: [138] Alternative Routing-ID Interpretation (ARI) Capabilities: [140] Single Root I/O Virtualization (SR-IOV) Kernel driver in use: vfio-pci [root@localhost ~]# In VM, the Intel QAT device shows up in lspci as "Intel Corporation Coleto Creek PCIe Endpoint". [root@test1 QAT]# lspci -nn | grep 0435 00:05.0 Co-processor [0b40]: Intel Corporation Coleto Creek PCIe Endpoint [8086:0435] [root@test1 QAT]# Afterward I installed Intel® QuickAssist Technology Software i.e., qatmux.l.2.6.0-60.tgz ( downloaded at https://01.org/packet-processing/intel%C2%AE-quickassist-technology-drivers-and-patches) on the Guest VM . Here are the details of VM. 1) Fedora release 20 (Heisenbug). 2) Linux Kernel version: 3.12.9-301.fc20.x86_64. 3) BOOT_IMAGE=/boot/vmlinuz-3.12.9-301.fc20.x86_64 root=UUID=ab47cbc9-68ee-403c-96a5-184e68238e65 ro console=ttyS0,115200n8 intel_iommu=off *Notes: * 1) We are not using SR-IOV and are instead passing through a PF for acceleration services on guest only. Hence we have not installed the Intel® QuickAssist Technology Software package on the host. 2) Found an Intel QAT driver with this kernel . Upon doing #lsmod | grep qa, showed icp_qa_al_vf. Thus did rmmod icp_qa_al_vf and service qat_service shutdown. This was done before building Intel QuickAssist Technology Software package. For passthrough of PF, opted the below Install Acceleration option a) export ICP_SRIOV=0 b) export ICP_WITHOUT_IOMMU=1 c) ./installer.sh build_accel QAT1.6 d) ./installer.sh install QAT1.6 guest e) insmod /root/QAT /QAT1.6/build/icp_qa_al.ko When I issue the below command to start the acceleration software, it fails to start the driver. [root@test1 ~]# service qat_service start Processing file: /etc/dh895xcc_qa_dev0.conf ADF_CONFIG_CTL err: adf_set_bank_core_affinity: Invalid core affinity settings for accelerator 0 bank 8. Max cpu number is 3. Defaulting to 0 ADF_CONFIG_CTL err: adf_set_bank_core_affinity: Invalid core affinity settings for accelerator 0 bank 9. Max cpu number is 3. Defaulting to 0 Parity err reporting is disabled. Error: failed to start device 0 There is 1 acceleration device(s) in the system: icp_dev0 - type=dh895xcc, inst_id=0, node_id=0, bdf=00:05:0, #accel=6, #engines=12, state=down [root@test1 ~]# Here are the kernel logs [root@test1 ~]# dmesg -c [ 3104.979666] Reading config file. [ 3104.980250] Starting acceleration device icp_dev0. [ 3104.980731] Resetting device icp_dev0 [ 3105.126920] qat_1_6_adf 0000:00:05.0: irq 45 for MSI/MSI-X [ 3105.126939] qat_1_6_adf 0000:00:05.0: irq 46 for MSI/MSI-X [ 3105.126956] qat_1_6_adf 0000:00:05.0: irq 47 for MSI/MSI-X [ 3105.126973] qat_1_6_adf 0000:00:05.0: irq 48 for MSI/MSI-X [ 3105.126989] qat_1_6_adf 0000:00:05.0: irq 49 for MSI/MSI-X [ 3105.127029] qat_1_6_adf 0000:00:05.0: irq 50 for MSI/MSI-X [ 3105.127048] qat_1_6_adf 0000:00:05.0: irq 51 for MSI/MSI-X [ 3105.127065] qat_1_6_adf 0000:00:05.0: irq 52 for MSI/MSI-X [ 3105.127082] qat_1_6_adf 0000:00:05.0: irq 53 for MSI/MSI-X [ 3105.127098] qat_1_6_adf 0000:00:05.0: irq 54 for MSI/MSI-X [ 3105.127115] qat_1_6_adf 0000:00:05.0: irq 55 for MSI/MSI-X [ 3105.127131] qat_1_6_adf 0000:00:05.0: irq 56 for MSI/MSI-X [ 3105.127148] qat_1_6_adf 0000:00:05.0: irq 57 for MSI/MSI-X [ 3105.127164] qat_1_6_adf 0000:00:05.0: irq 58 for MSI/MSI-X [ 3105.127181] qat_1_6_adf 0000:00:05.0: irq 59 for MSI/MSI-X [ 3105.127198] qat_1_6_adf 0000:00:05.0: irq 60 for MSI/MSI-X [ 3105.127215] qat_1_6_adf 0000:00:05.0: irq 61 for MSI/MSI-X [ 3105.127231] qat_1_6_adf 0000:00:05.0: irq 62 for MSI/MSI-X [ 3105.127248] qat_1_6_adf 0000:00:05.0: irq 63 for MSI/MSI-X [ 3105.127265] qat_1_6_adf 0000:00:05.0: irq 64 for MSI/MSI-X [ 3105.127282] qat_1_6_adf 0000:00:05.0: irq 65 for MSI/MSI-X [ 3105.127298] qat_1_6_adf 0000:00:05.0: irq 66 for MSI/MSI-X [ 3105.127315] qat_1_6_adf 0000:00:05.0: irq 67 for MSI/MSI-X [ 3105.127331] qat_1_6_adf 0000:00:05.0: irq 68 for MSI/MSI-X [ 3105.127348] qat_1_6_adf 0000:00:05.0: irq 69 for MSI/MSI-X [ 3105.127364] qat_1_6_adf 0000:00:05.0: irq 70 for MSI/MSI-X [ 3105.127380] qat_1_6_adf 0000:00:05.0: irq 71 for MSI/MSI-X [ 3105.127397] qat_1_6_adf 0000:00:05.0: irq 72 for MSI/MSI-X [ 3105.127413] qat_1_6_adf 0000:00:05.0: irq 73 for MSI/MSI-X [ 3105.127429] qat_1_6_adf 0000:00:05.0: irq 74 for MSI/MSI-X [ 3105.127446] qat_1_6_adf 0000:00:05.0: irq 75 for MSI/MSI-X [ 3105.127462] qat_1_6_adf 0000:00:05.0: irq 76 for MSI/MSI-X [ 3105.127487] qat_1_6_adf 0000:00:05.0: irq 77 for MSI/MSI-X [ 3105.157973] icp_qa_al err: adf_subsystemInit: Invalid SRIOV_Enabled value, driver is built with SRIOV. [ 3105.158891] icp_qa_al err: adf_do_init: adf_subsystemInit error, stopping and shutting down [ 3106.162962] icp_qa_al err: adf_init_all: failed to initialize accelerator [ 3106.163670] icp_qa_al err: adf_init_devices: failed init accel 0 [ 3106.164256] icp_qa_al err: adf_ctl_ioctl_dev_start: failed call to adf_init_devices [root@test1 ~]# Can anyone please let me know what might be the issue and its remedy ? please feel free to correct me if I am wrong. If it is not the right forum , please let me know the correct one. Thank you in advance for time and support. Regards, Chinmaya ^ permalink raw reply [flat|nested] 3+ messages in thread
* Re: [dpdk-users] Issue with Physical Function passthrough on VM Guest (Fedora 20) using qatmux.l.2.6.0-60 2016-06-10 7:16 [dpdk-users] Issue with Physical Function passthrough on VM Guest (Fedora 20) using qatmux.l.2.6.0-60 Chinmaya Dwibedy @ 2016-06-10 7:33 ` SAKTHIVEL ANAND S 2016-06-10 8:57 ` Chinmaya Dwibedy 0 siblings, 1 reply; 3+ messages in thread From: SAKTHIVEL ANAND S @ 2016-06-10 7:33 UTC (permalink / raw) To: Chinmaya Dwibedy; +Cc: users If a failed to start device error is shown during the Acceleration installation, ensure that the kernel option intel_iommu=off has been configured as specified in Updating Grub Configuration File on page 13. If this kernel parameter is not specified, acceleration services are only available in a guest operating system. The above lines are from QAT official document.. so if u r using QAT in VM, you have to make intel_iommu=on in the grub config. On Fri, Jun 10, 2016 at 12:46 PM, Chinmaya Dwibedy <ckdwibedy@gmail.com> wrote: > Hi, > > > I have assigned Intel QAT device to VMs as a full device (PF - physical > function, PCI passthrough). Below is the output of lspci in host (Cent OS > 7.2). > > > [root@localhost nova(keystone_admin)]# lspci -nn | grep 0435 > > 83:00.0 Co-processor [0b40]: Intel Corporation DH895XCC Series QAT [ > 8086:0435] > > 88:00.0 Co-processor [0b40]: Intel Corporation DH895XCC Series QAT [ > 8086:0435] > > [root@localhost nova(keystone_admin)]# > > [root@localhost ~]# lspci -vnd 8086:0435 > > 83:00.0 0b40: 8086:0435 > > Subsystem: 8086:35c5 > > Physical Slot: 0-1 > > Flags: bus master, fast devsel, latency 0, IRQ 35 > > Memory at 387fffd00000 (64-bit, prefetchable) [size=512K] > > Memory at c8200000 (64-bit, non-prefetchable) [size=256K] > > Memory at c8240000 (64-bit, non-prefetchable) [size=256K] > > Capabilities: [b0] MSI: Enable- Count=1/1 Maskable+ 64bit+ > > Capabilities: [60] MSI-X: Enable- Count=33 Masked- > > Capabilities: [6c] Power Management version 3 > > Capabilities: [74] Express Endpoint, MSI 00 > > Capabilities: [100] Advanced Error Reporting > > Capabilities: [138] Alternative Routing-ID Interpretation (ARI) > > Capabilities: [140] Single Root I/O Virtualization (SR-IOV) > > Kernel driver in use: vfio-pci > > > > 88:00.0 0b40: 8086:0435 > > Subsystem: 8086:35c5 > > Physical Slot: 0-2 > > Flags: fast devsel, IRQ 39 > > Memory at 387fffe00000 (64-bit, prefetchable) [size=512K] > > Memory at c8000000 (64-bit, non-prefetchable) [size=256K] > > Memory at c8040000 (64-bit, non-prefetchable) [size=256K] > > Capabilities: [b0] MSI: Enable- Count=1/1 Maskable+ 64bit+ > > Capabilities: [60] MSI-X: Enable- Count=33 Masked- > > Capabilities: [6c] Power Management version 3 > > Capabilities: [74] Express Endpoint, MSI 00 > > Capabilities: [100] Advanced Error Reporting > > Capabilities: [138] Alternative Routing-ID Interpretation (ARI) > > Capabilities: [140] Single Root I/O Virtualization (SR-IOV) > > Kernel driver in use: vfio-pci > > > > [root@localhost ~]# > > > In VM, the Intel QAT device shows up in lspci as "Intel Corporation Coleto > Creek PCIe Endpoint". > > > [root@test1 QAT]# lspci -nn | grep 0435 > > 00:05.0 Co-processor [0b40]: Intel Corporation Coleto Creek PCIe Endpoint > [8086:0435] > > [root@test1 QAT]# > > > Afterward I installed Intel® QuickAssist Technology Software i.e., > qatmux.l.2.6.0-60.tgz ( downloaded at > > https://01.org/packet-processing/intel%C2%AE-quickassist-technology-drivers-and-patches > ) > on the Guest VM . Here are the details of VM. > > 1) Fedora release 20 (Heisenbug). > > 2) Linux Kernel version: 3.12.9-301.fc20.x86_64. > > 3) BOOT_IMAGE=/boot/vmlinuz-3.12.9-301.fc20.x86_64 > root=UUID=ab47cbc9-68ee-403c-96a5-184e68238e65 ro console=ttyS0,115200n8 > intel_iommu=off > > > > *Notes: * > > 1) We are not using SR-IOV and are instead passing through a PF for > acceleration services on guest only. Hence we have not installed the > Intel® QuickAssist Technology Software package on the host. > > 2) Found an Intel QAT driver with this kernel . Upon doing #lsmod | > grep qa, showed icp_qa_al_vf. Thus did rmmod icp_qa_al_vf and service > qat_service shutdown. This was done before building Intel QuickAssist > Technology Software package. > > > > For passthrough of PF, opted the below Install Acceleration option > > a) export ICP_SRIOV=0 > > b) export ICP_WITHOUT_IOMMU=1 > > c) ./installer.sh build_accel QAT1.6 > > d) ./installer.sh install QAT1.6 guest > > e) insmod /root/QAT /QAT1.6/build/icp_qa_al.ko > > > > When I issue the below command to start the acceleration software, it fails > to start the driver. > > > [root@test1 ~]# service qat_service start > > Processing file: /etc/dh895xcc_qa_dev0.conf > > ADF_CONFIG_CTL err: adf_set_bank_core_affinity: Invalid core affinity > settings for accelerator 0 bank 8. Max cpu number is 3. Defaulting to 0 > > ADF_CONFIG_CTL err: adf_set_bank_core_affinity: Invalid core affinity > settings for accelerator 0 bank 9. Max cpu number is 3. Defaulting to 0 > > Parity err reporting is disabled. > > Error: failed to start device 0 > > There is 1 acceleration device(s) in the system: > > icp_dev0 - type=dh895xcc, inst_id=0, node_id=0, bdf=00:05:0, #accel=6, > #engines=12, state=down > > [root@test1 ~]# > > > > Here are the kernel logs > > > > [root@test1 ~]# dmesg -c > > [ 3104.979666] Reading config file. > > [ 3104.980250] Starting acceleration device icp_dev0. > > [ 3104.980731] Resetting device icp_dev0 > > [ 3105.126920] qat_1_6_adf 0000:00:05.0: irq 45 for MSI/MSI-X > > [ 3105.126939] qat_1_6_adf 0000:00:05.0: irq 46 for MSI/MSI-X > > [ 3105.126956] qat_1_6_adf 0000:00:05.0: irq 47 for MSI/MSI-X > > [ 3105.126973] qat_1_6_adf 0000:00:05.0: irq 48 for MSI/MSI-X > > [ 3105.126989] qat_1_6_adf 0000:00:05.0: irq 49 for MSI/MSI-X > > [ 3105.127029] qat_1_6_adf 0000:00:05.0: irq 50 for MSI/MSI-X > > [ 3105.127048] qat_1_6_adf 0000:00:05.0: irq 51 for MSI/MSI-X > > [ 3105.127065] qat_1_6_adf 0000:00:05.0: irq 52 for MSI/MSI-X > > [ 3105.127082] qat_1_6_adf 0000:00:05.0: irq 53 for MSI/MSI-X > > [ 3105.127098] qat_1_6_adf 0000:00:05.0: irq 54 for MSI/MSI-X > > [ 3105.127115] qat_1_6_adf 0000:00:05.0: irq 55 for MSI/MSI-X > > [ 3105.127131] qat_1_6_adf 0000:00:05.0: irq 56 for MSI/MSI-X > > [ 3105.127148] qat_1_6_adf 0000:00:05.0: irq 57 for MSI/MSI-X > > [ 3105.127164] qat_1_6_adf 0000:00:05.0: irq 58 for MSI/MSI-X > > [ 3105.127181] qat_1_6_adf 0000:00:05.0: irq 59 for MSI/MSI-X > > [ 3105.127198] qat_1_6_adf 0000:00:05.0: irq 60 for MSI/MSI-X > > [ 3105.127215] qat_1_6_adf 0000:00:05.0: irq 61 for MSI/MSI-X > > [ 3105.127231] qat_1_6_adf 0000:00:05.0: irq 62 for MSI/MSI-X > > [ 3105.127248] qat_1_6_adf 0000:00:05.0: irq 63 for MSI/MSI-X > > [ 3105.127265] qat_1_6_adf 0000:00:05.0: irq 64 for MSI/MSI-X > > [ 3105.127282] qat_1_6_adf 0000:00:05.0: irq 65 for MSI/MSI-X > > [ 3105.127298] qat_1_6_adf 0000:00:05.0: irq 66 for MSI/MSI-X > > [ 3105.127315] qat_1_6_adf 0000:00:05.0: irq 67 for MSI/MSI-X > > [ 3105.127331] qat_1_6_adf 0000:00:05.0: irq 68 for MSI/MSI-X > > [ 3105.127348] qat_1_6_adf 0000:00:05.0: irq 69 for MSI/MSI-X > > [ 3105.127364] qat_1_6_adf 0000:00:05.0: irq 70 for MSI/MSI-X > > [ 3105.127380] qat_1_6_adf 0000:00:05.0: irq 71 for MSI/MSI-X > > [ 3105.127397] qat_1_6_adf 0000:00:05.0: irq 72 for MSI/MSI-X > > [ 3105.127413] qat_1_6_adf 0000:00:05.0: irq 73 for MSI/MSI-X > > [ 3105.127429] qat_1_6_adf 0000:00:05.0: irq 74 for MSI/MSI-X > > [ 3105.127446] qat_1_6_adf 0000:00:05.0: irq 75 for MSI/MSI-X > > [ 3105.127462] qat_1_6_adf 0000:00:05.0: irq 76 for MSI/MSI-X > > [ 3105.127487] qat_1_6_adf 0000:00:05.0: irq 77 for MSI/MSI-X > > [ 3105.157973] icp_qa_al err: adf_subsystemInit: Invalid SRIOV_Enabled > value, driver is built with SRIOV. > > [ 3105.158891] icp_qa_al err: adf_do_init: adf_subsystemInit error, > stopping and shutting down > > [ 3106.162962] icp_qa_al err: adf_init_all: failed to initialize > accelerator > > [ 3106.163670] icp_qa_al err: adf_init_devices: failed init accel 0 > > [ 3106.164256] icp_qa_al err: adf_ctl_ioctl_dev_start: failed call to > adf_init_devices > > [root@test1 ~]# > > > > Can anyone please let me know what might be the issue and its remedy ? > please feel free to correct me if I am wrong. If it is not the right forum > , please let me know the correct one. Thank you in advance for time and > support. > > > > Regards, > > Chinmaya > -- Thanks Sakthivel S OM ^ permalink raw reply [flat|nested] 3+ messages in thread
* Re: [dpdk-users] Issue with Physical Function passthrough on VM Guest (Fedora 20) using qatmux.l.2.6.0-60 2016-06-10 7:33 ` SAKTHIVEL ANAND S @ 2016-06-10 8:57 ` Chinmaya Dwibedy 0 siblings, 0 replies; 3+ messages in thread From: Chinmaya Dwibedy @ 2016-06-10 8:57 UTC (permalink / raw) To: SAKTHIVEL ANAND S; +Cc: users Hi SAKTHIVEL, Thank your for your response . On VM, added “intel_iommu=on” in /boot/grub2/grub.cfg and rebooted the VM. Still getting the same issue i.e., failed to start device. Here are the logs. I have done “export ICP_SRIOV=0” to build driver without SRIOV. Please let me know if I am missing anything. [root@test1 QAT]# service qat_service start Processing file: /etc/dh895xcc_qa_dev0.conf ADF_CONFIG_CTL err: adf_set_bank_core_affinity: Invalid core affinity settings for accelerator 0 bank 8. Max cpu number is 3. Defaulting to 0 ADF_CONFIG_CTL err: adf_set_bank_core_affinity: Invalid core affinity settings for accelerator 0 bank 9. Max cpu number is 3. Defaulting to 0 Parity err reporting is disabled. Error: failed to start device 0 There is 1 acceleration device(s) in the system: icp_dev0 - type=dh895xcc, inst_id=0, node_id=0, bdf=00:05:0, #accel=6, #engines=12, state=down [root@test1 QAT]# [root@test1 QAT]# cat /proc/cmdline BOOT_IMAGE=/boot/vmlinuz-3.12.9-301.fc20.x86_64 root=UUID=ab47cbc9-68ee-403c-96a5-184e68238e65 ro console=ttyS0,115200n8 intel_iommu=on [root@test1 QAT]# *Kernel logs (#dmesg )* [ 693.776133] icp_qa_al err: adf_subsystemInit: Invalid SRIOV_Enabled value, driver is built with SRIOV. [ 693.777043] icp_qa_al err: adf_do_init: adf_subsystemInit error, stopping and shutting down [ 694.780830] icp_qa_al err: adf_init_all: failed to initialize accelerator [ 694.781554] icp_qa_al err: adf_init_devices: failed init accel 0 [ 694.782153] icp_qa_al err: adf_ctl_ioctl_dev_start: failed call to adf_init_devices [root@test1 QAT]# Regards, Chinmaya On Fri, Jun 10, 2016 at 1:03 PM, SAKTHIVEL ANAND S <anand.sa88@gmail.com> wrote: > If a failed to start device > error is shown during the Acceleration > installation, ensure that the kernel option > intel_iommu=off > has been > configured as specified in > Updating Grub Configuration File > on page 13. If > this kernel parameter is not specified, acceleration services are only > available in a guest operating system. > > The above lines are from QAT official document.. so if u r using QAT in > VM, you have to make intel_iommu=on in the grub config. > > > > On Fri, Jun 10, 2016 at 12:46 PM, Chinmaya Dwibedy <ckdwibedy@gmail.com> > wrote: > >> Hi, >> >> >> I have assigned Intel QAT device to VMs as a full device (PF - physical >> function, PCI passthrough). Below is the output of lspci in host (Cent OS >> 7.2). >> >> >> [root@localhost nova(keystone_admin)]# lspci -nn | grep 0435 >> >> 83:00.0 Co-processor [0b40]: Intel Corporation DH895XCC Series QAT [ >> 8086:0435] >> >> 88:00.0 Co-processor [0b40]: Intel Corporation DH895XCC Series QAT [ >> 8086:0435] >> >> [root@localhost nova(keystone_admin)]# >> >> [root@localhost ~]# lspci -vnd 8086:0435 >> >> 83:00.0 0b40: 8086:0435 >> >> Subsystem: 8086:35c5 >> >> Physical Slot: 0-1 >> >> Flags: bus master, fast devsel, latency 0, IRQ 35 >> >> Memory at 387fffd00000 (64-bit, prefetchable) [size=512K] >> >> Memory at c8200000 (64-bit, non-prefetchable) [size=256K] >> >> Memory at c8240000 (64-bit, non-prefetchable) [size=256K] >> >> Capabilities: [b0] MSI: Enable- Count=1/1 Maskable+ 64bit+ >> >> Capabilities: [60] MSI-X: Enable- Count=33 Masked- >> >> Capabilities: [6c] Power Management version 3 >> >> Capabilities: [74] Express Endpoint, MSI 00 >> >> Capabilities: [100] Advanced Error Reporting >> >> Capabilities: [138] Alternative Routing-ID Interpretation (ARI) >> >> Capabilities: [140] Single Root I/O Virtualization (SR-IOV) >> >> Kernel driver in use: vfio-pci >> >> >> >> 88:00.0 0b40: 8086:0435 >> >> Subsystem: 8086:35c5 >> >> Physical Slot: 0-2 >> >> Flags: fast devsel, IRQ 39 >> >> Memory at 387fffe00000 (64-bit, prefetchable) [size=512K] >> >> Memory at c8000000 (64-bit, non-prefetchable) [size=256K] >> >> Memory at c8040000 (64-bit, non-prefetchable) [size=256K] >> >> Capabilities: [b0] MSI: Enable- Count=1/1 Maskable+ 64bit+ >> >> Capabilities: [60] MSI-X: Enable- Count=33 Masked- >> >> Capabilities: [6c] Power Management version 3 >> >> Capabilities: [74] Express Endpoint, MSI 00 >> >> Capabilities: [100] Advanced Error Reporting >> >> Capabilities: [138] Alternative Routing-ID Interpretation (ARI) >> >> Capabilities: [140] Single Root I/O Virtualization (SR-IOV) >> >> Kernel driver in use: vfio-pci >> >> >> >> [root@localhost ~]# >> >> >> In VM, the Intel QAT device shows up in lspci as "Intel Corporation >> Coleto >> Creek PCIe Endpoint". >> >> >> [root@test1 QAT]# lspci -nn | grep 0435 >> >> 00:05.0 Co-processor [0b40]: Intel Corporation Coleto Creek PCIe Endpoint >> [8086:0435] >> >> [root@test1 QAT]# >> >> >> Afterward I installed Intel® QuickAssist Technology Software i.e., >> qatmux.l.2.6.0-60.tgz ( downloaded at >> >> https://01.org/packet-processing/intel%C2%AE-quickassist-technology-drivers-and-patches >> ) >> on the Guest VM . Here are the details of VM. >> >> 1) Fedora release 20 (Heisenbug). >> >> 2) Linux Kernel version: 3.12.9-301.fc20.x86_64. >> >> 3) BOOT_IMAGE=/boot/vmlinuz-3.12.9-301.fc20.x86_64 >> root=UUID=ab47cbc9-68ee-403c-96a5-184e68238e65 ro console=ttyS0,115200n8 >> intel_iommu=off >> >> >> >> *Notes: * >> >> >> 1) We are not using SR-IOV and are instead passing through a PF for >> acceleration services on guest only. Hence we have not installed the >> Intel® QuickAssist Technology Software package on the host. >> >> 2) Found an Intel QAT driver with this kernel . Upon doing #lsmod | >> grep qa, showed icp_qa_al_vf. Thus did rmmod icp_qa_al_vf and service >> qat_service shutdown. This was done before building Intel QuickAssist >> Technology Software package. >> >> >> >> For passthrough of PF, opted the below Install Acceleration option >> >> a) export ICP_SRIOV=0 >> >> b) export ICP_WITHOUT_IOMMU=1 >> >> c) ./installer.sh build_accel QAT1.6 >> >> d) ./installer.sh install QAT1.6 guest >> >> e) insmod /root/QAT /QAT1.6/build/icp_qa_al.ko >> >> >> >> When I issue the below command to start the acceleration software, it >> fails >> to start the driver. >> >> >> [root@test1 ~]# service qat_service start >> >> Processing file: /etc/dh895xcc_qa_dev0.conf >> >> ADF_CONFIG_CTL err: adf_set_bank_core_affinity: Invalid core affinity >> settings for accelerator 0 bank 8. Max cpu number is 3. Defaulting to 0 >> >> ADF_CONFIG_CTL err: adf_set_bank_core_affinity: Invalid core affinity >> settings for accelerator 0 bank 9. Max cpu number is 3. Defaulting to 0 >> >> Parity err reporting is disabled. >> >> Error: failed to start device 0 >> >> There is 1 acceleration device(s) in the system: >> >> icp_dev0 - type=dh895xcc, inst_id=0, node_id=0, bdf=00:05:0, #accel=6, >> #engines=12, state=down >> >> [root@test1 ~]# >> >> >> >> Here are the kernel logs >> >> >> >> [root@test1 ~]# dmesg -c >> >> [ 3104.979666] Reading config file. >> >> [ 3104.980250] Starting acceleration device icp_dev0. >> >> [ 3104.980731] Resetting device icp_dev0 >> >> [ 3105.126920] qat_1_6_adf 0000:00:05.0: irq 45 for MSI/MSI-X >> >> [ 3105.126939] qat_1_6_adf 0000:00:05.0: irq 46 for MSI/MSI-X >> >> [ 3105.126956] qat_1_6_adf 0000:00:05.0: irq 47 for MSI/MSI-X >> >> [ 3105.126973] qat_1_6_adf 0000:00:05.0: irq 48 for MSI/MSI-X >> >> [ 3105.126989] qat_1_6_adf 0000:00:05.0: irq 49 for MSI/MSI-X >> >> [ 3105.127029] qat_1_6_adf 0000:00:05.0: irq 50 for MSI/MSI-X >> >> [ 3105.127048] qat_1_6_adf 0000:00:05.0: irq 51 for MSI/MSI-X >> >> [ 3105.127065] qat_1_6_adf 0000:00:05.0: irq 52 for MSI/MSI-X >> >> [ 3105.127082] qat_1_6_adf 0000:00:05.0: irq 53 for MSI/MSI-X >> >> [ 3105.127098] qat_1_6_adf 0000:00:05.0: irq 54 for MSI/MSI-X >> >> [ 3105.127115] qat_1_6_adf 0000:00:05.0: irq 55 for MSI/MSI-X >> >> [ 3105.127131] qat_1_6_adf 0000:00:05.0: irq 56 for MSI/MSI-X >> >> [ 3105.127148] qat_1_6_adf 0000:00:05.0: irq 57 for MSI/MSI-X >> >> [ 3105.127164] qat_1_6_adf 0000:00:05.0: irq 58 for MSI/MSI-X >> >> [ 3105.127181] qat_1_6_adf 0000:00:05.0: irq 59 for MSI/MSI-X >> >> [ 3105.127198] qat_1_6_adf 0000:00:05.0: irq 60 for MSI/MSI-X >> >> [ 3105.127215] qat_1_6_adf 0000:00:05.0: irq 61 for MSI/MSI-X >> >> [ 3105.127231] qat_1_6_adf 0000:00:05.0: irq 62 for MSI/MSI-X >> >> [ 3105.127248] qat_1_6_adf 0000:00:05.0: irq 63 for MSI/MSI-X >> >> [ 3105.127265] qat_1_6_adf 0000:00:05.0: irq 64 for MSI/MSI-X >> >> [ 3105.127282] qat_1_6_adf 0000:00:05.0: irq 65 for MSI/MSI-X >> >> [ 3105.127298] qat_1_6_adf 0000:00:05.0: irq 66 for MSI/MSI-X >> >> [ 3105.127315] qat_1_6_adf 0000:00:05.0: irq 67 for MSI/MSI-X >> >> [ 3105.127331] qat_1_6_adf 0000:00:05.0: irq 68 for MSI/MSI-X >> >> [ 3105.127348] qat_1_6_adf 0000:00:05.0: irq 69 for MSI/MSI-X >> >> [ 3105.127364] qat_1_6_adf 0000:00:05.0: irq 70 for MSI/MSI-X >> >> [ 3105.127380] qat_1_6_adf 0000:00:05.0: irq 71 for MSI/MSI-X >> >> [ 3105.127397] qat_1_6_adf 0000:00:05.0: irq 72 for MSI/MSI-X >> >> [ 3105.127413] qat_1_6_adf 0000:00:05.0: irq 73 for MSI/MSI-X >> >> [ 3105.127429] qat_1_6_adf 0000:00:05.0: irq 74 for MSI/MSI-X >> >> [ 3105.127446] qat_1_6_adf 0000:00:05.0: irq 75 for MSI/MSI-X >> >> [ 3105.127462] qat_1_6_adf 0000:00:05.0: irq 76 for MSI/MSI-X >> >> [ 3105.127487] qat_1_6_adf 0000:00:05.0: irq 77 for MSI/MSI-X >> >> [ 3105.157973] icp_qa_al err: adf_subsystemInit: Invalid SRIOV_Enabled >> value, driver is built with SRIOV. >> >> [ 3105.158891] icp_qa_al err: adf_do_init: adf_subsystemInit error, >> stopping and shutting down >> >> [ 3106.162962] icp_qa_al err: adf_init_all: failed to initialize >> accelerator >> >> [ 3106.163670] icp_qa_al err: adf_init_devices: failed init accel 0 >> >> [ 3106.164256] icp_qa_al err: adf_ctl_ioctl_dev_start: failed call to >> adf_init_devices >> >> [root@test1 ~]# >> >> >> >> Can anyone please let me know what might be the issue and its remedy ? >> please feel free to correct me if I am wrong. If it is not the right >> forum >> , please let me know the correct one. Thank you in advance for time and >> support. >> >> >> >> Regards, >> >> Chinmaya >> > > > > -- > Thanks > Sakthivel S OM > > ^ permalink raw reply [flat|nested] 3+ messages in thread
end of thread, other threads:[~2016-06-10 8:57 UTC | newest] Thread overview: 3+ messages (download: mbox.gz / follow: Atom feed) -- links below jump to the message on this page -- 2016-06-10 7:16 [dpdk-users] Issue with Physical Function passthrough on VM Guest (Fedora 20) using qatmux.l.2.6.0-60 Chinmaya Dwibedy 2016-06-10 7:33 ` SAKTHIVEL ANAND S 2016-06-10 8:57 ` Chinmaya Dwibedy
This is a public inbox, see mirroring instructions for how to clone and mirror all data and code used for this inbox; as well as URLs for NNTP newsgroup(s).