From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 3F565425FC; Wed, 20 Sep 2023 20:28:35 +0200 (CEST) Received: from mails.dpdk.org (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 1CD9340DFB; Wed, 20 Sep 2023 20:28:35 +0200 (CEST) Received: from mail-oo1-f47.google.com (mail-oo1-f47.google.com [209.85.161.47]) by mails.dpdk.org (Postfix) with ESMTP id 3000A402BA for ; Wed, 20 Sep 2023 20:28:34 +0200 (CEST) Received: by mail-oo1-f47.google.com with SMTP id 006d021491bc7-57129417cecso31679eaf.1 for ; Wed, 20 Sep 2023 11:28:34 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=iol.unh.edu; s=unh-iol; t=1695234513; x=1695839313; darn=dpdk.org; h=cc:to:subject:message-id:date:from:in-reply-to:references :mime-version:from:to:cc:subject:date:message-id:reply-to; bh=Ao76meLuptYi0wAxqSIpxys067L1TQnvueXI7F5meu4=; b=hbe5dFsjEMXV4W/6tvgdzfNc50XHLOkjCSLnKmHPT7bExBbPNP2KS6idNRU0Q2Kz8O tpeDTDekm+RLAoztvi+1DXYPufrLS/Bhci2kmyr1P54OkDVDqv2IP+b4yFA8ISgYGU/J IRYZ6YzgIHLF/BDM4xQ0LNMC7THTk9VCEefGM= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1695234513; x=1695839313; h=cc:to:subject:message-id:date:from:in-reply-to:references :mime-version:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=Ao76meLuptYi0wAxqSIpxys067L1TQnvueXI7F5meu4=; b=KaZga321w4EWCDJKJjKIpx2a5IS55jm5mCLNNmRD61QFbTHp6acR9Wdm4CdPn/FvFx wryRkpN2aJPrHXDCMpmryd93oF37Q/VbkvU4v/agFDirRQKB9QFJjtmUcDFWhe6htDnH mJGxtqGv75N+cv3Pxr7iSum3yPhDTR8N2M9dKccM3obS65+k34PpfzVe7ieEHhIMyie0 kXmDQwKALcQZ8S7DRnhJp/CkuIEtNYSkMXnfudRuJO0O+1T6mIaMQwPPcaKMIdLKaYTS XoY7WvVoupK8BgXhOUMvMFWUVGutk5VYyYWHYkouvVI9mvM6daxA9qjYfw2ZYI/BeOoc AsDA== X-Gm-Message-State: AOJu0Yy0ozYttQHeaXkykJqtO1r2oScztdN9nDevFDWeTGnsUQfvZT6M PaorKhoT8KkzQOyfiXHydy+BIMrxCFZJ48iulf+0Cw== X-Google-Smtp-Source: AGHT+IHuPupUTSqAFE3XFjQp+fYgqKzqCw4ThScWW5zTb/+YSY89uNz5hGgfgNTDMBsX9XzdQAZtTE0i/bNJihCbqUg= X-Received: by 2002:a05:6820:2288:b0:576:b301:a9b1 with SMTP id ck8-20020a056820228800b00576b301a9b1mr4210374oob.0.1695234513178; Wed, 20 Sep 2023 11:28:33 -0700 (PDT) MIME-Version: 1.0 References: In-Reply-To: From: Patrick Robb Date: Wed, 20 Sep 2023 14:28:22 -0400 Message-ID: Subject: Re: Intel QAT 8970 accel card on ARM Ampere Server To: =?UTF-8?Q?Juraj_Linke=C5=A1?= Cc: Ruifeng Wang , Honnappa Nagarahalli , "ci@dpdk.org" , nd Content-Type: multipart/alternative; boundary="000000000000f827b10605ce8a0a" X-BeenThere: ci@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK CI discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: ci-bounces@dpdk.org --000000000000f827b10605ce8a0a Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable Hi Juraj, Sorry for the late reply. So, yes I applied those diffs and set the QAT modules to =3Dy in the .config file when building the custom kernel. It appears to have worked correctly. The qat module is now built into the new kernel running on the ampere server (called 5.15.82+). You can see it listed on modules.builtin and from modinfo. probb@arm-ampere-dut:~$ modinfo qat_c62x name: qat_c62x filename: (builtin) version: 0.6.0 description: Intel(R) QuickAssist Technology firmware: qat_c62x_mmp.bin firmware: qat_c62x.bin author: Intel license: Dual BSD/GPL file: drivers/crypto/qat/qat_c62x/qat_c62x And there is a /sys/bus/pci/drivers/c6xx/(pci address)/sriov_numvfs path now so that's good. However, when trying to create the VFs for the 3 PFs on the card, a segmentation fault was returned the first time, and on subsequent tries it hangs now. So like: root@arm-ampere-dut:~# lspci -d:37c8 0000:03:00.0 Co-processor: Intel Corporation C62x Chipset QuickAssist Technology (rev 04) 0000:04:00.0 Co-processor: Intel Corporation C62x Chipset QuickAssist Technology (rev 04) 0000:05:00.0 Co-processor: Intel Corporation C62x Chipset QuickAssist Technology (rev 04) root@arm-ampere-dut:~# echo 16 > /sys/bus/pci/drivers/c6xx/0000:03:00.0/sriov_numvfs The sriov_numvfs file should be writable from root so I'm a bit perplexed. I am wondering whether it is relevant to statically build in the qat_c62x module with the kernel, vs having it be a loadable driver? What do you do? On Mon, Sep 11, 2023 at 4:13=E2=80=AFAM Juraj Linke=C5=A1 wrote: > Hi Patrick, > > This is good news. How does the server fare after the restart? > > Juraj > > On Fri, Sep 1, 2023 at 11:30=E2=80=AFPM Patrick Robb = wrote: > >> Thanks Juraj, >> >> I did bring the system to 22.04 based on our conversation from yesterday= . >> From there and from checking out to the new current kernel >> (5.15.0-82-generic) yes the diffs cleanly apply, removing the x86 >> dependency on the QAT kernel drivers, and then you can make the kernel, >> enabling the QAT driver. It looks like that all worked fine. >> >> I didn't actually install and reboot with the custom kernel today becaus= e >> I don't want to do that with a production server right before the weeken= d, >> particularly with USA having a holiday on Monday. I will reboot with the >> custom kernel on Tuesday morning though, and then hopefully the >> compress/crypto testsuites on QAT will be unblocked. Thanks, the guidanc= e >> is greatly appreciated. >> >> Best, >> Patrick >> >> --=20 Patrick Robb Technical Service Manager UNH InterOperability Laboratory 21 Madbury Rd, Suite 100, Durham, NH 03824 www.iol.unh.edu --000000000000f827b10605ce8a0a Content-Type: text/html; charset="UTF-8" Content-Transfer-Encoding: quoted-printable
Hi=C2=A0Juraj,

Sorry for the late reply= . So, yes I applied those diffs and set the QAT modules to =3Dy in the .con= fig file when building the custom kernel. It appears to have worked correct= ly. The qat module is now built into the new kernel running on the ampere s= erver (called=C2=A05.15.82+). You can see it listed on modules.builtin and = from modinfo.

probb@arm-ampere-dut:~$ modinfo qat_= c62x
name: =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 qat_c62x
filename: =C2= =A0 =C2=A0 =C2=A0 (builtin)
version: =C2=A0 =C2=A0 =C2=A0 =C2=A00.6.0description: =C2=A0 =C2=A0Intel(R) QuickAssist Technology
firmware: =C2= =A0 =C2=A0 =C2=A0 qat_c62x_mmp.bin
firmware: =C2=A0 =C2=A0 =C2=A0 qat_c6= 2x.bin
author: =C2=A0 =C2=A0 =C2=A0 =C2=A0 Intel
license: =C2=A0 =C2= =A0 =C2=A0 =C2=A0Dual BSD/GPL
file: =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 d= rivers/crypto/qat/qat_c62x/qat_c62x

And there = is a /sys/bus/pci/drivers/c6xx/(pci address)/sriov_numvfs path now so that&= #39;s good. However, when trying to create the VFs for the 3 PFs on the car= d, a segmentation fault was returned the first time, and on subsequent trie= s it hangs now. So like:=C2=A0

root@arm-ampere-dut= :~# lspci -d:37c8
0000:03:00.0 Co-processor: Intel Corporation C62x Chip= set QuickAssist Technology (rev 04)
0000:04:00.0 Co-processor: Intel Cor= poration C62x Chipset QuickAssist Technology (rev 04)
0000:05:00.0 Co-pr= ocessor: Intel Corporation C62x Chipset QuickAssist Technology (rev 04)
= root@arm-ampere-dut:~# echo 16 > /sys/bus/pci/drivers/c6xx/0000:03:00.0/= sriov_numvfs

The sriov_numvfs file should be w= ritable=C2=A0from root so I'm a bit perplexed. I am wondering whether i= t is relevant to statically build in the qat_c62x module with the kernel, v= s having it be a loadable driver? What do you do?=C2=A0

On Mon, Sep 11= , 2023 at 4:13=E2=80=AFAM Juraj Linke=C5=A1 <juraj.linkes@pantheon.tech&= gt; wrote:
Hi Patrick,

This is good news. How does the = server fare after the restart?

Juraj
On Fri, = Sep 1, 2023 at 11:30=E2=80=AFPM Patrick Robb <probb@iol.unh.edu> wrote:
Thanks Juraj,

I did bring the system to 22.04 based= on our conversation from yesterday. From there and from checking out to th= e new current kernel (5.15.0-82-generic) yes the diffs cleanly apply, remov= ing the x86 dependency on the QAT kernel drivers, and then you can make the= kernel, enabling the QAT driver. It looks like that all worked fine.=C2=A0=

I didn't actually install and reboot with the= custom kernel today because I don't want to do that with a production = server right before the weekend, particularly with USA having a holiday on = Monday. I will reboot with the custom kernel on Tuesday morning though, and= then hopefully the compress/crypto testsuites on QAT will be unblocked. Th= anks, the guidance is greatly appreciated.=C2=A0

B= est,
Patrick=C2=A0



--

Patrick Robb

<= span style=3D"font-size:10pt;font-family:Arial;color:rgb(0,0,0);background-= color:transparent;vertical-align:baseline;white-space:pre-wrap">Technical S= ervice Manager

UNH InterOperability Laboratory

21 Madbury Rd, Suite 100, Durham, NH 03824

www.iol.unh.edu


--000000000000f827b10605ce8a0a--