From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id A786742348; Tue, 10 Oct 2023 09:26:05 +0200 (CEST) Received: from mails.dpdk.org (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id A0764402CB; Tue, 10 Oct 2023 09:26:05 +0200 (CEST) Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) by mails.dpdk.org (Postfix) with ESMTP id 78EFB4029F for ; Tue, 10 Oct 2023 09:26:04 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1696922764; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=4sSH8vjWbOi5pfoSrTh/ewbp/kL7yhWr3TAb1VhWnMY=; b=XfgofvmJSpxL/jmX4SeotvLIHTMqyg+b6Ifc7OptgIqgewmIGJdJzGISQFnHVl3baxaXzc PnxKn8M7w7UAXs0G4Y3T42QpvADHNoE9OUtmzRIt4TkJYbtu4ubjs9oiloMrIcAzKsUa6A P9m0/6T63HGSls3dVdz4H4F1r86ilbQ= Received: from mail-lf1-f72.google.com (mail-lf1-f72.google.com [209.85.167.72]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-132-MKsaxr8BP3SghIgVlkpx_w-1; Tue, 10 Oct 2023 03:26:01 -0400 X-MC-Unique: MKsaxr8BP3SghIgVlkpx_w-1 Received: by mail-lf1-f72.google.com with SMTP id 2adb3069b0e04-5047e8f812bso5022807e87.3 for ; Tue, 10 Oct 2023 00:26:00 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1696922759; x=1697527559; h=content-transfer-encoding:cc:to:subject:message-id:date:from :in-reply-to:references:mime-version:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=4sSH8vjWbOi5pfoSrTh/ewbp/kL7yhWr3TAb1VhWnMY=; b=WLqp8cbjpsxtvQsdDVhr1PSMIkn5kJDYcWOq6+5AXRqyPwrnc4/Vmr0XaTBP2JcfHQ 9CW2SiZHvyXaqGhfnzOxtj2aYomc1iNCRJUDdVEaj08hb/YB6szcqltP+nKx5k35AR8N ovJP97SQHGKHQae+Y9bk5XSdIJ8THNMfC+vUhwOMFX7A2X1MUtz4i/w1uUvBHOGBia0g wx41+T8y8023G1B6lfid5lFtiIbUOyft5qHMbHOYg13n+tWhz8Q/mSLbV938hi3Xwfwc RIAmh3fuqglF1eRslSDv7dkhfu2+p6+igRxxCmawWAwanKPvrFfsHdlhVl/W+SesvVvr 8ngw== X-Gm-Message-State: AOJu0YyvhjIRIPqRXxj7FWH4lchWIrmYBaTo6uXj9GLmMHYb+ggjYlph i0fotMdQBFdmn4wzxgxckwW8AXp/vtq74zPg39cDPuTRdpRUTeQCxhL0W62+6YM2KZm29OpQIYe 347TkNx8HEtEpzEOo+Q== X-Received: by 2002:a05:6512:e96:b0:500:b3fe:916e with SMTP id bi22-20020a0565120e9600b00500b3fe916emr19671899lfb.2.1696922759591; Tue, 10 Oct 2023 00:25:59 -0700 (PDT) X-Google-Smtp-Source: AGHT+IGEU2R0oGJoVukOwDxKI8gnlltmMV90UO98C1dExYPCyn7uIsRQ7qLw5jljKUW4QebV2xO2SurKECjP+wjInss= X-Received: by 2002:a05:6512:e96:b0:500:b3fe:916e with SMTP id bi22-20020a0565120e9600b00500b3fe916emr19671885lfb.2.1696922759187; Tue, 10 Oct 2023 00:25:59 -0700 (PDT) MIME-Version: 1.0 References: In-Reply-To: From: David Marchand Date: Tue, 10 Oct 2023 09:25:47 +0200 Message-ID: Subject: Re: Intel QAT 8970 accel card on ARM Ampere Server To: Dharmik Jayesh Thakkar Cc: Patrick Robb , Ruifeng Wang , =?UTF-8?Q?Juraj_Linke=C5=A1?= , Honnappa Nagarahalli , "ci@dpdk.org" , nd , Thomas Monjalon , Maxime Coquelin X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-BeenThere: ci@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK CI discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: ci-bounces@dpdk.org Hello, On Tue, Oct 10, 2023 at 5:56=E2=80=AFAM Dharmik Jayesh Thakkar wrote: > > Hi Patrick, > > Can you provide the grub settings? Is iommu.passthrough=3D1 included? > > > > Also, is qat_c62xvf loaded as well? > > > > Finally, a few guidelines on the vfio driver: > > At times, we need to configure the vfio driver. > > On kernel vers. 5.9+ we need to load the vfio-pci driver with the additio= nal parameter disable_denylist=3D1 o_O I did not know this option, but it scares me a bit, reading its description= . Could you please elaborate why this is needed? > > Unload the vfio-pci driver if it is already loaded so that we can reload = it with the correct parameters : > sudo modprobe -r vfio_iommu_type1; sudo modprobe -r vfio_pci; sudo modpro= be -r vfio_virqfd; sudo modprobe -r vfio > > If you can't unload the vfio driver because it's been built into the kern= el, you'll have to find another way to change VFIO parameters, or to rebuil= d your kernel with VFIO_PCI set as a module. Failing to do that, you might = encounter issues later on when you try to bind the VFs to VFIO. > > Load the vfio-pci driver and bind it to QAT VFs device ids: > sudo modprobe vfio-pci disable_denylist=3D1 enable_sriov=3D1 vfio-pci.ids= =3D8086:37c9 > > Enable no-iommu-mode: > echo "1" | sudo tee /sys/module/vfio/parameters/enable_unsafe_noiommu_mod= e > > /sys/module/vfio/parameter is missing ? > > If /sys/module/vfio/parameters does not exist, you might be missing the k= ernel module VFIO_NOIOMMU > > > > Automatically set VFIO params on boot > > It's possible to set these parameters automatically on boot by creating a= /etc/modprobe.d/vfio-pci.conf file with the parameters : > cat /etc/modprobe.d/vfio-pci.conf > options vfio enable_unsafe_noiommu_mode=3D1 > options vfio-pci disable_denylist=3D1 enable_sriov=3D1 vfio-pci.ids=3D808= 6:37c9 > > > > We haven=E2=80=99t encountered this issue in the past, so just making sur= e the configuration is correct. I don=E2=80=99t think having the driver sta= tic/loadable should make a difference, I will try with building statically = on my setup. --=20 David Marchand