DPDK CI discussions
 help / color / mirror / Atom feed
From: Patrick Robb <probb@iol.unh.edu>
To: "Ruifeng Wang" <Ruifeng.Wang@arm.com>,
	"Honnappa Nagarahalli" <honnappa.nagarahalli@arm.com>,
	"Juraj Linkeš" <juraj.linkes@pantheon.tech>
Cc: dharmikjayesh.thakkar@arm.com, ci@dpdk.org
Subject: Intel QAT 8970 accel card on ARM Ampere Server
Date: Mon, 31 Jul 2023 13:13:49 -0400	[thread overview]
Message-ID: <CAJvnSUCeHvf7dLo4KiV0GVrgMVaGT9zWbYDmUP-WMpj=5Bv3mw@mail.gmail.com> (raw)

[-- Attachment #1: Type: text/plain, Size: 3323 bytes --]

Hi Ruifeng, Honnappa, Juraj,

The Intel QAT 8970 accelerator card has arrived to the Community Lab, and
we've installed it on the Ampere server. Presumably, we should test both
crypto and compress operations (and their respective performance metrics).
To that end, there are also DTS testsuites for testing QAT crypto/compress
functions. These testsuites make use of the crypto perf dpdk app and the
compress perf dpdk app. If you want, you can setup the DTS stuff yourself,
both on the system side, and the Jenkins side (you are allowed to submit
PRs on our gitlab now), but we can also do that on the lab side as we
probably have more experience. I do, however, have a question about the QAT
kernel driver and corresponding PMDs.

compress suite:
https://git.dpdk.org/tools/dts/tree/test_plans/compressdev_qat_pmd_test_plan.rst
crypto suite:
https://git.dpdk.org/tools/dts/tree/test_plans/crypto_perf_cryptodev_perf_test_plan.rst

For reference, the DPDK docs page explaining QAT driver capabilities and
building the QAT PMDs (crypto sym, crypto asym, and compress) is here:
https://doc.dpdk.org/guides/cryptodevs/qat.html#building-qat

Some notes before I get to my main question:
-The 8970 is a C62x device
-OpenSSL (arm requires it for QAT) is installed
-3 PFs are visible from lspci (expected)
-SRIOV is enabled

However, although the system is on a valid kernel version for the QAT
driver, the kernel module for QAT is not loaded, so in trying to set up
testing, I am unable to create the 16 VFs for the 3 PFs respectively, like
the example below:

echo 16 > /sys/bus/pci/drivers/c6xx/(pci address)/sriov_numvfs
echo 16 > /sys/bus/pci/drivers/c6xx/(pci address)/sriov_numvfs
echo 16 > /sys/bus/pci/drivers/c6xx/(pci address)/sriov_numvfs

There is also an option to download the firmware from the kernel firmware
repo and copy the qat binaries to /lib/firmware and start the qat modules
from there. I wasn't able to resolve the situation with this method, but it
also could have been user error on my part.

There is an option to install using the IDZ QAT Driver
<https://www.intel.com/content/www/us/en/developer/topic-technology/open/quick-assist-technology/overview.html>,
but it should not be required given the kernel version the Ampere server is
on, and I don't want to go down the road of relying on this "fall back"
method without consulting you first. Is it possible that there is anything
specific to running a QAT device on ARM specifically which I am missing
here? The DTS testsuite testplans actually seem to recommend going down
this road in general, but the DPDK docs say to use the kernel driver, so I
don't know.

In any case, one of you should be able to login to the Ampere server in
situations like this, or just in general. Ruifeng/Juraj I see you both have
accounts on our IdM system, so you should have access. Please let me know
if you need renewed vpn cert configs and I will send you one. If you do
login, know this system could be running CI testing at any time. I can
always schedule time for it to be offline and available for maintenance if
you want to do anything which could be disruptive to testing.

I also CC'd Dharmik on this as I see he sent an email regarding QAT support
on aarch64 in June.

Let me know if you have any thoughts on the QAT kernel driver part.

Thanks,
Patrick

[-- Attachment #2: Type: text/html, Size: 4040 bytes --]

             reply	other threads:[~2023-07-31 17:14 UTC|newest]

Thread overview: 31+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2023-07-31 17:13 Patrick Robb [this message]
2023-08-04  9:48 ` Ruifeng Wang
2023-08-08  7:07   ` Juraj Linkeš
2023-08-08  7:11     ` Ruifeng Wang
2023-08-11 21:18       ` Patrick Robb
2023-08-21  8:45         ` Juraj Linkeš
2023-08-30  0:05           ` Patrick Robb
2023-09-01 21:30           ` Patrick Robb
2023-09-11  8:13             ` Juraj Linkeš
2023-09-20 18:28               ` Patrick Robb
2023-09-25 15:19                 ` Ruifeng Wang
2023-10-09 16:34                   ` Patrick Robb
2023-10-10  2:28                     ` Patrick Robb
2023-10-10  3:55                       ` Dharmik Jayesh Thakkar
2023-10-10  7:25                         ` David Marchand
2023-10-10 15:03                           ` Dharmik Jayesh Thakkar
2023-10-10 15:12                             ` David Marchand
2023-10-10 15:59                         ` Patrick Robb
2023-10-10 21:50                           ` Dharmik Jayesh Thakkar
2023-10-11  8:14                           ` Juraj Linkeš
2023-10-11 20:13                             ` Patrick Robb
2023-11-02 22:00                               ` Patrick Robb
2023-11-14  7:34                                 ` Ruifeng Wang
2023-11-14 14:36                                   ` Patrick Robb
2024-02-27  6:58                                   ` Patrick Robb
2024-02-27 13:50                                     ` Honnappa Nagarahalli
2024-02-28 20:00                                       ` Patrick Robb
2024-02-28 20:40                                         ` Honnappa Nagarahalli
2024-03-07  5:27                                           ` Patrick Robb
2024-03-07  7:56                                             ` David Marchand
2023-10-11 11:51                           ` David Marchand

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to='CAJvnSUCeHvf7dLo4KiV0GVrgMVaGT9zWbYDmUP-WMpj=5Bv3mw@mail.gmail.com' \
    --to=probb@iol.unh.edu \
    --cc=Ruifeng.Wang@arm.com \
    --cc=ci@dpdk.org \
    --cc=dharmikjayesh.thakkar@arm.com \
    --cc=honnappa.nagarahalli@arm.com \
    --cc=juraj.linkes@pantheon.tech \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).