Re: Intel QAT 8970 accel card on ARM Ampere Server

[David Marchand <david.marchand@redhat.com>]

Hello,

On Tue, Oct 10, 2023 at 5:56 AM Dharmik Jayesh Thakkar
<DharmikJayesh.Thakkar@arm.com> wrote:
>
> Hi Patrick,
>
> Can you provide the grub settings? Is iommu.passthrough=1 included?
>
>
>
> Also, is qat_c62xvf loaded as well?
>
>
>
> Finally, a few guidelines on the vfio driver:
>
> At times, we need to configure the vfio driver.
>
> On kernel vers. 5.9+ we need to load the vfio-pci driver with the additional parameter disable_denylist=1

o_O
I did not know this option, but it scares me a bit, reading its description.
Could you please elaborate why this is needed?


>
> Unload the vfio-pci driver if it is already loaded so that we can reload it with the correct parameters :
> sudo modprobe -r vfio_iommu_type1; sudo modprobe -r vfio_pci; sudo modprobe -r vfio_virqfd; sudo modprobe -r vfio
>
> If you can't unload the vfio driver because it's been built into the kernel, you'll have to find another way to change VFIO parameters, or to rebuild your kernel with VFIO_PCI set as a module. Failing to do that, you might encounter issues later on when you try to bind the VFs to VFIO.
>
> Load the vfio-pci driver and bind it to QAT VFs device ids:
> sudo modprobe vfio-pci disable_denylist=1 enable_sriov=1 vfio-pci.ids=8086:37c9
>
> Enable no-iommu-mode:
> echo "1" | sudo tee /sys/module/vfio/parameters/enable_unsafe_noiommu_mode
>
>  /sys/module/vfio/parameter is missing ?
>
> If /sys/module/vfio/parameters does not exist, you might be missing the kernel module VFIO_NOIOMMU
>
>
>
> Automatically set VFIO params on boot
>
> It's possible to set these parameters automatically on boot by creating a /etc/modprobe.d/vfio-pci.conf file with the parameters :
> cat /etc/modprobe.d/vfio-pci.conf
> options vfio enable_unsafe_noiommu_mode=1
> options vfio-pci disable_denylist=1 enable_sriov=1 vfio-pci.ids=8086:37c9
>
>
>
> We haven’t encountered this issue in the past, so just making sure the configuration is correct. I don’t think having the driver static/loadable should make a difference, I will try with building statically on my setup.


-- 
David Marchand