From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 849C042EC3 for ; Thu, 20 Jul 2023 06:08:51 +0200 (CEST) Received: from mails.dpdk.org (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id E5F7B40E2D; Thu, 20 Jul 2023 06:08:50 +0200 (CEST) Received: from wout1-smtp.messagingengine.com (wout1-smtp.messagingengine.com [64.147.123.24]) by mails.dpdk.org (Postfix) with ESMTP id 3AFFD40DF5 for ; Thu, 20 Jul 2023 06:08:50 +0200 (CEST) Received: from compute5.internal (compute5.nyi.internal [10.202.2.45]) by mailout.west.internal (Postfix) with ESMTP id 2A9F73202372; Thu, 20 Jul 2023 00:08:49 -0400 (EDT) Received: from mailfrontend1 ([10.202.2.162]) by compute5.internal (MEProxy); Thu, 20 Jul 2023 00:08:49 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=monjalon.net; h= cc:cc:content-transfer-encoding:content-type:content-type:date :date:from:from:in-reply-to:in-reply-to:message-id:mime-version :references:reply-to:sender:subject:subject:to:to; s=fm3; t= 1689826128; x=1689912528; bh=RInrMPwgLBkynmDb9v0QRPkEVoBSIxy73zi Pz+GOEEY=; b=LQ9u//Uu2djGHuUFwwUdtC9Ki8dhPE1R08kf9xNksOoH8Mph61D q+gzS2Mzx1ikGLsfo5bFt2wRUCCeUH9TCnpjfayow1VoMu5Kkw2oYppBlPMjjm8W RGkA2gEK8VW20yqLwGMr/0aIktqRmQY9kpVLGaueAFLJyuF3LtccY1nDFi4BMoWX IxLEonMoyLX3OQkoi1+XChMuNWOetSE5EYRCEwsOn7mLdMWTfsD0X69f5hMlk7q7 AxpFgNaVkksATQpOZjWsD9hdzZw4yTxO6BQ9ZO7fame4kBe0Uni/uZtlYaQ1nZmV 0zPYzJv2JxJ6Itl/CCvRGH4HYpBDv7Txjfg== DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d= messagingengine.com; h=cc:cc:content-transfer-encoding :content-type:content-type:date:date:feedback-id:feedback-id :from:from:in-reply-to:in-reply-to:message-id:mime-version :references:reply-to:sender:subject:subject:to:to:x-me-proxy :x-me-proxy:x-me-sender:x-me-sender:x-sasl-enc; s=fm3; t= 1689826128; x=1689912528; bh=RInrMPwgLBkynmDb9v0QRPkEVoBSIxy73zi Pz+GOEEY=; b=V3shNIllz8M50Pf48065sfIwQu0kNYNGzxfTjzNE0znvFaXAdHI a5hiqdpoJ0A6UHoqbVZN0nuw6yzJ3SEdte/tuos3Yp+rKlM37jqKZvCMMRfkEQMo U/8oCq2Y6rj1ZwpbkEPnJSiXGwrFHxIz0CCdmAPEsViLrRp//xQ4qcB9ruG+g0Pa kp8l+HEeUKNPvI6fKWoKp02KQFe6b+wWJ7dMwfX/Olw9Q0HP2jVJ6gZZluUmmo+L NiZGX94EB2hka3xr2S+LjlNCXsQ0gJ8X5mNr3ndqXc1Au/WgF56X0CC3LSBO/CTJ 3GsyTN5cQf7hoHXSuSEVmmn8mVbFoplWWFg== X-ME-Sender: X-ME-Received: X-ME-Proxy-Cause: gggruggvucftvghtrhhoucdtuddrgedviedrgeelgdejlecutefuodetggdotefrodftvf curfhrohhfihhlvgemucfhrghsthforghilhdpqfgfvfdpuffrtefokffrpgfnqfghnecu uegrihhlohhuthemuceftddtnecunecujfgurhephffvvefufffkjghfggfgtgesthfure dttddtvdenucfhrhhomhepvfhhohhmrghsucfoohhnjhgrlhhonhcuoehthhhomhgrshes mhhonhhjrghlohhnrdhnvghtqeenucggtffrrghtthgvrhhnpeejjefffffgffekfeffle fgkeelteejffelledugefhheelffetheevudffudfgvdenucffohhmrghinhepughpughk rdhorhhgnecuvehluhhsthgvrhfuihiivgeptdenucfrrghrrghmpehmrghilhhfrhhomh epthhhohhmrghssehmohhnjhgrlhhonhdrnhgvth X-ME-Proxy: Feedback-ID: i47234305:Fastmail Received: by mail.messagingengine.com (Postfix) with ESMTPA; Thu, 20 Jul 2023 00:08:47 -0400 (EDT) From: Thomas Monjalon To: Igor de Paula Cc: users@dpdk.org Subject: Re: help Date: Thu, 20 Jul 2023 06:08:46 +0200 Message-ID: <10081895.2WqB4rESCP@thomas> In-Reply-To: References: MIME-Version: 1.0 Content-Transfer-Encoding: 7Bit Content-Type: text/plain; charset="us-ascii" X-BeenThere: users@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK usage discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: users-bounces@dpdk.org Hello, The first error is "Cause: Error: number of ports must be even" 03/05/2023 18:13, Igor de Paula: > I am running a VM inside a VMWARE server (vSphere). > My goal it to set up DPDK with two HW ports, and set up a virtio_user to > interact with the kernel stack. > In another app I have it working but instead of virtio_user I am running > KNI, it works in IOVA-PA mode. > I am looking to replace the KNI. > > When I try to set up virtio_user port as in the doc: > https://doc.dpdk.org/guides/howto/virtio_user_as_exception_path.html#virtio-user-as-exception-path > I get a error it can't run in PA mode. > > > When I try to run as VA mode from a parameter, I get the following errors: > EAL: lib.eal log level changed from info to debug > EAL: Detected lcore 0 as core 0 on socket 0 > EAL: Detected lcore 1 as core 0 on socket 0 > EAL: Support maximum 128 logical core(s) by configuration. > EAL: Detected 2 lcore(s) > EAL: Detected 1 NUMA nodes > EAL: Checking presence of .so 'librte_eal.so.21.3' > EAL: Checking presence of .so 'librte_eal.so.21' > EAL: Checking presence of .so 'librte_eal.so' > EAL: Detected static linkage of DPDK > EAL: Ask a virtual area of 0x7000 bytes > EAL: Virtual area found at 0x100000000 (size = 0x7000) > EAL: Multi-process socket /var/run/dpdk/rte/mp_socket > EAL: DPAA Bus not present. Skipping. > EAL: VFIO PCI modules not loaded > EAL: Selected IOVA mode 'VA' > EAL: Probing VFIO support... > EAL: IOMMU type 1 (Type 1) is supported > EAL: IOMMU type 7 (sPAPR) is not supported > EAL: IOMMU type 8 (No-IOMMU) is supported > EAL: VFIO support initialized > EAL: Ask a virtual area of 0x5b000 bytes > EAL: Virtual area found at 0x100007000 (size = 0x5b000) > EAL: Setting up physically contiguous memory... > EAL: Setting maximum number of open files to 1048576 > EAL: Detected memory type: socket_id:0 hugepage_sz:1073741824 > EAL: Creating 2 segment lists: n_segs:128 socket_id:0 hugepage_sz:1073741824 > EAL: Ask a virtual area of 0x2000 bytes > EAL: Virtual area found at 0x100062000 (size = 0x2000) > EAL: Memseg list allocated at socket 0, page size 0x100000kB > EAL: Ask a virtual area of 0x2000000000 bytes > EAL: Virtual area found at 0x140000000 (size = 0x2000000000) > EAL: VA reserved for memseg list at 0x140000000, size 2000000000 > EAL: Ask a virtual area of 0x2000 bytes > EAL: Virtual area found at 0x2140000000 (size = 0x2000) > EAL: Memseg list allocated at socket 0, page size 0x100000kB > EAL: Ask a virtual area of 0x2000000000 bytes > EAL: Virtual area found at 0x2180000000 (size = 0x2000000000) > EAL: VA reserved for memseg list at 0x2180000000, size 2000000000 > EAL: TSC frequency is ~2350000 KHz > EAL: Main lcore 0 is ready (tid=7f8ad790ec00;cpuset=[0]) > EAL: lcore 1 is ready (tid=7f8ad6907400;cpuset=[1]) > EAL: Trying to obtain current memory policy. > EAL: Setting policy MPOL_PREFERRED for socket 0 > EAL: Restoring previous memory policy: 0 > EAL: request: mp_malloc_sync > EAL: Heap on socket 0 was expanded by 1024MB > EAL: PCI device 0000:0b:00.0 on NUMA socket -1 > EAL: probe driver: 15ad:7b0 net_vmxnet3 > EAL: Expecting 'PA' IOVA mode but current mode is 'VA', not initializing > EAL: Requested device 0000:0b:00.0 cannot be used > EAL: PCI device 0000:13:00.0 on NUMA socket -1 > EAL: probe driver: 15ad:7b0 net_vmxnet3 > EAL: Expecting 'PA' IOVA mode but current mode is 'VA', not initializing > EAL: Requested device 0000:13:00.0 cannot be used > EAL: Bus (pci) probe failed. > EAL: lib.telemetry log level changed from disabled to warning > EAL: Error - exiting with code: 1 > Cause: Error: number of ports must be even > EAL: request: mp_malloc_sync > EAL: Heap on socket 0 was shrunk by 1024MB > > > > For some reason the HW ports won't setup. From what I understand > net_vmxnet3 should work with VA mode. > I enabled I/OMUU for the VM. > The weird thing even when enabled, I still have the > enable_unsafe_noiommu_mode flag on. > And because it's on the this: > > dev_iova_mode = pci_device_iova_mode(dr, dev); > > return PA mode, and it fails. > > When I disable it by modifying > /sys/module/vfio/parameters/enable_unsafe_noiommu_mode, I get another error. > The error is that it doesn't find a suitable IOMMU type: > Just putting the relevant message: > > > EAL: Heap on socket 0 was expanded by 1024MB > EAL: PCI device 0000:0b:00.0 on NUMA socket -1 > EAL: probe driver: 15ad:7b0 net_vmxnet3 > EAL: Set IOMMU type 1 (Type 1) failed, error 19 (No such device) > EAL: Set IOMMU type 7 (sPAPR) failed, error 19 (No such device) > EAL: Set IOMMU type 8 (No-IOMMU) failed, error 19 (No such device) > EAL: 0000:0b:00.0 failed to select IOMMU type > EAL: Requested device 0000:0b:00.0 cannot be used > EAL: PCI device 0000:13:00.0 on NUMA socket -1 > EAL: probe driver: 15ad:7b0 net_vmxnet3 > EAL: Set IOMMU type 1 (Type 1) failed, error 19 (No such device) > EAL: Set IOMMU type 7 (sPAPR) failed, error 19 (No such device) > EAL: Set IOMMU type 8 (No-IOMMU) failed, error 19 (No such device) > EAL: 0000:13:00.0 failed to select IOMMU type > EAL: Requested device 0000:13:00.0 cannot be used > > > I can see where it fails: > > > const struct vfio_iommu_type * > vfio_set_iommu_type(int vfio_container_fd) > { > unsigned idx; > for (idx = 0; idx < RTE_DIM(iommu_types); idx++) { > const struct vfio_iommu_type *t = &iommu_types[idx]; > > int ret = ioctl(vfio_container_fd, VFIO_SET_IOMMU, > t->type_id); > if (!ret) { > RTE_LOG(INFO, EAL, "Using IOMMU type %d (%s)\n", > t->type_id, t->name); > return t; > } > /* not an error, there may be more supported IOMMU types */ > RTE_LOG(DEBUG, EAL, "Set IOMMU type %d (%s) failed, error " > "%i (%s)\n", t->type_id, t->name, errno, > strerror(errno)); > } > /* if we didn't find a suitable IOMMU type, fail */ > return NULL; > } > > > The ioctl returns -1. > > > > I tried many things. > Enabling/disabling IOMMU. > Modifying the enable_unsafe_noiommu_mode flag. > Running in PA mode. > Any help will be much appreciated. >