Re: Intel QAT 8970 accel card on ARM Ampere Server

[Patrick Robb <probb@iol.unh.edu>]

[-- Attachment #1: Type: text/plain, Size: 2312 bytes --]

On Wed, Oct 11, 2023 at 4:14 AM Juraj Linkeš <juraj.linkes@pantheon.tech>
wrote:

>
>
> On Tue, Oct 10, 2023 at 5:59 PM Patrick Robb <probb@iol.unh.edu> wrote:
>
>>
>>
>> On Mon, Oct 9, 2023 at 11:56 PM Dharmik Jayesh Thakkar <
>> DharmikJayesh.Thakkar@arm.com> wrote:
>>
>>> Hi Patrick,
>>>
>>>
>>>
>>> Can you provide the grub settings? Is iommu.passthrough=1 included?
>>>
>>
>> Sure. I'm not sure if you just wanted the kernel cmdline options or the
>> whole grub config, but I assume you just meant kernel cmdline. Let me know
>> if you meant more.
>>
>> GRUB_CMDLINE_LINUX_DEFAULT="default_hugepagesz=1G hugepagesz=1G
>> hugepages=32 iommu=pt intel_iommu=on isolcpus=39-79 nohz_full=39-79
>> rcu_nocbs=39-79 processor.max_cstate=1 intel_pstate=disable
>> console=ttyS0,115200 console=tty0"
>>
>> But, iommu.passthrough=1 is not included, so I can add that if we need
>> to. Do you know that this won't have any bad implications for the (intel,
>> nvidia, broadcom) NICs which we test on this server?
>>
>>
>
> Just a note here, Patrick. The iommu kernel and intel_pstate parameters
> aren't supported on arm, so you can remove those. And when
> iommu.passthrouh=1, IOMMU is bypassed and intel_iommu doesn't do anything
> (and maybe isn't supported on arm, but that's not clear from the docs
> <https://www.kernel.org/doc/Documentation/admin-guide/kernel-parameters.txt>),
> so that can be removed as well.
>

Thanks Dharmik and Juraj. Updated kernel cmdline args:
BOOT_IMAGE=/vmlinuz-5.15.82+ root=/dev/mapper/ubuntu--vg--1-ubuntu--lv ro
default_hugepagesz=1G hugepagesz=1G hugepages=32 isolcpus=39-79
nohz_full=39-79 rcu_nocbs=39-79 processor.max_cstate=1 iommu.passthrough=1
console=ttyS0,115200 console=tty0

I added the iommu.passthrough option and tried again, to no avail. FYI I am
still using the guidance here:
https://doc.dpdk.org/guides/cryptodevs/qat.html along with your added steps.

root@arm-ampere-dut:~# echo 16 >
/sys/bus/pci/drivers/c6xx/0000:03:00.0/sriov_numvfs
Segmentation fault (core dumped)

As you know the above setting of the 48 VFs is a prerequisite to binding
the VFs to vfio-pci. But, I did run through loading the custom vfio and
there were no issues, so once we clear this initial hurdle we should be
fine.

[-- Attachment #2: Type: text/html, Size: 3569 bytes --]