From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 87DFB43004 for ; Tue, 8 Aug 2023 02:26:51 +0200 (CEST) Received: from mails.dpdk.org (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id E656240DFD; Tue, 8 Aug 2023 02:26:50 +0200 (CEST) Received: from mail-pl1-f170.google.com (mail-pl1-f170.google.com [209.85.214.170]) by mails.dpdk.org (Postfix) with ESMTP id 004B440DDC for ; Tue, 8 Aug 2023 02:26:48 +0200 (CEST) Received: by mail-pl1-f170.google.com with SMTP id d9443c01a7336-1bb893e6365so33024835ad.2 for ; Mon, 07 Aug 2023 17:26:48 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=networkplumber-org.20221208.gappssmtp.com; s=20221208; t=1691454408; x=1692059208; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:subject:cc:to:from:date:from:to:cc:subject:date :message-id:reply-to; bh=4RbBMdfR7YzvsC3b1OoNyRQAdZ7cb2g19axOpWQnSyQ=; b=u67OFZz7U1UbrnA11XGTN2nsQ8bz19NWAQv/m3TrLrWjQchgi5vQdGo1QA9OKho4bJ N5U8rbyFcS3Q/VHNKR0WKK8q6N7u6+j8h9Z1AmMPor+S0IdgDgaQzG5xdPSc3jfxO4Pu gTILYig+yryqWJrTFr1n7DMaIChdSSGfetVOaysFUdBRYdH9hjLaN+m9rDThNmOT+/WB 3G7K0nWBfaYX2FOCvGu6X4gmn1DWy1e39jbq3jEi4ib0xjNnPkxR62iueM9tb11e3uOj mu3TjaMMAsFCuA2GArt9329EEA9IOn4df1LOOOw/Q24//fU74Cdj7et2OzL0aARL/FeM SHCA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1691454408; x=1692059208; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:subject:cc:to:from:date:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=4RbBMdfR7YzvsC3b1OoNyRQAdZ7cb2g19axOpWQnSyQ=; b=iI7lhLLCFzenVSIRkTLOfrYTWLPjl0LO18D1Rrsx5C5FRbFj+XRAJl7Ac6JCAIUlsV KYdDSLiUBPRzFTTpS+/3vKABsNQjeJi5kiC5t0K2rEQN/IhjspH1gJghLmjXlQ7StoFa ElLA77pStG3eJ0wAfYSJ05u5MN7HafG/31VwatCWPN4TdFfVAmJTGB88Wa3w9474KeNn 0cFT+LtfmTWBYwcD6rgXgR/JaU5I+wpseSREr8aN/l3CWw0zLBzQHQZ+Gd7FlfpFJvFr fLvfFQnDsFMK9P6D3w9PobPcpcptH5Acsd4AnGOiUpgFYug3glb5mvIXv+IoyKw6EmQ5 cu5w== X-Gm-Message-State: AOJu0YzoDEiXu8ne55a4iQ+yda3ghhjtWbRL7hScKBTECBg4U9ntIlkY JpSFmtQtpEkZei1xahil5bLFv1+/TaKVXMEldauiI3H/ X-Google-Smtp-Source: AGHT+IFT4MNphVGrEPYtULbdtdZF7+G/wKYbqLqVl4AopE9FshJRhEUZOUoUl20/uN2gna+F5zv5OQ== X-Received: by 2002:a17:902:b193:b0:1bb:91f2:bb3e with SMTP id s19-20020a170902b19300b001bb91f2bb3emr8915238plr.49.1691454407930; Mon, 07 Aug 2023 17:26:47 -0700 (PDT) Received: from hermes.local (204-195-127-207.wavecable.com. [204.195.127.207]) by smtp.gmail.com with ESMTPSA id b4-20020a170902b60400b001b830d8bc40sm7514049pls.74.2023.08.07.17.26.47 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 07 Aug 2023 17:26:47 -0700 (PDT) Date: Mon, 7 Aug 2023 17:26:45 -0700 From: Stephen Hemminger To: Alan Beadle Cc: users@dpdk.org Subject: Re: Help Running Example Message-ID: <20230807172645.3288f7e4@hermes.local> In-Reply-To: <20230807124021.325b6d68@hermes.local> References: <20230807124021.325b6d68@hermes.local> MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit X-BeenThere: users@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK usage discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: users-bounces@dpdk.org On Mon, 7 Aug 2023 12:40:21 -0700 Stephen Hemminger wrote: > On Sun, 6 Aug 2023 11:33:43 -0400 > Alan Beadle wrote: > > > Hi, > > > > I need some help getting DPDK working. I am running Ubuntu 20.04 with > > a modified Linux 5.4 kernel, but I have also tried the stock Ubuntu > > 5.15 kernel with the same results. > > > > Here is my NIC info from lspci: > > 00:1f.6 Ethernet controller: Intel Corporation Ethernet Connection (3) > > I219-LM (rev 09) > > > > I built and installed DPDK from source, and applied the following boot > > flags: "intel_iommu=on iommu=pt" > > > > After booting I did the following as root: > > echo 1024 > /sys/kernel/mm/hugepages/hugepages-2048kB/nr_hugepages > > ifconfig enp0s31f6 down > > dpdk-devbind.py --bind=vfio-pci 0000:00:1f.6 > > > > All of this appeared to work. > > > > I tried running the "skeleton" example program and got the following output: > > sudo ./build/basicfwd > > EAL: Detected CPU lcores: 16 > > EAL: Detected NUMA nodes: 1 > > EAL: Detected shared linkage of DPDK > > EAL: Multi-process socket /var/run/dpdk/rte/mp_socket > > EAL: Selected IOVA mode 'VA' > > EAL: VFIO support initialized > > EAL: 0000:00:1f.6 VFIO group is not viable! Not all devices in IOMMU > > group bound to VFIO or unbound > > EAL: Requested device 0000:00:1f.6 cannot be used > > TELEMETRY: No legacy callbacks, legacy socket not created > > EAL: Error - exiting with code: 1 > > Cause: Error: number of ports must be even > > > > I'm not at all familiar with DPDK or VFIO. What might the problem be? > > > > -Alan > > IOMMU groups are when multiple PCI devices share the same channel > in the IOMMU. The group is used to determine what mapping to use when > device does DMA. Since this is a security thing, devices in same IOMMU > group can not be shared between kernel and non-kernel usage. > > The IOMMU group is determined by wiring on the motherboard. > Usually it is things like multiple Ethernet ports sharing the same group. > But can be much more confused. > > The only option is to unbind all devices in the group before using > one with DPDK. More info on IOMMU groups is in kernel documentation: https://www.kernel.org/doc/html/latest/driver-api/vfio.html and in this article https://access.redhat.com/documentation/en-us/red_hat_enterprise_linux/7/html/virtualization_deployment_and_administration_guide/sect-iommu-deep-dive