From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 16CBE42ED9 for ; Fri, 21 Jul 2023 13:22:00 +0200 (CEST) Received: from mails.dpdk.org (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 8C0C140DDC; Fri, 21 Jul 2023 13:21:59 +0200 (CEST) Received: from NAM11-BN8-obe.outbound.protection.outlook.com (mail-bn8nam11on2041.outbound.protection.outlook.com [40.107.236.41]) by mails.dpdk.org (Postfix) with ESMTP id 5021640DD8 for ; Fri, 21 Jul 2023 13:21:57 +0200 (CEST) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=atxqJMXFqM8tfTM8PUxJSB+MZ4PrhzzaFbTexxWpTI+JaXpEOUpLGC3T285m67+VKixPDJwcAv0p5Eb/XMJe1zbyC+7XqP7txJH9BrpYtHX7pyoKk7v1wtp8kA2wo3u/QGCj9OJvhpwoZuzgHISp6lopJ2n+y7PFBoGPxsRjXng0eousqvRfuWUorwvRnnUm5YtagR1BdKGfZMndBQvMOLu8rlwUD5wm+qG28VlEuw/JZe6FzB5kYii53UkC0LzzmhWFwoqIs+zMYXcT/sTbyvxNfu5eR5ZG+V1ri4eCbedyIMSJtu3ktbpsLX6Ykpz7846vfOvtj6th6ELwbA0qww== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=pcjh/zgicAZE8btEQT0hidjhBYXe5O6D9N+QyXMiAyg=; b=ZZY5E5JK+3eA1KXoUw0g8xHUYxeRcQuAl09v2XluwfATnqT6F0+Pdh3x0zO4ASHgU/yFKHzAeuWBzCLDEaVDsZ8Vgu3N5OwQcJ/1rPcdvQni1Tfs0g7q85iiRZOwAY35U46ASgUxsNG0TH2cABnTIGOEXj491xxgE14Tl61ifnFGm70fazoK70FlHimYeUZD3SvHj9pPpo/yRXHFvr7qwZPkslVziQ1m0TKW7bM5E51N4iP4t5zfgGzl/AJ+ZNIjGkGhSGFMudEHlKGqQTLM9qluzSe3IMvIjmxeXclyPIR7bpURi7EzMXQCjSFl60xhPTbTzhfRN/HvGZ1/X/UhsA== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=amd.com; dmarc=pass action=none header.from=amd.com; dkim=pass header.d=amd.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amd.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=pcjh/zgicAZE8btEQT0hidjhBYXe5O6D9N+QyXMiAyg=; b=Nnqcv+TEresQXgmPx5DhHLbUVUi/QeVfdqINk//3OA01w5zY1JZ4Gbo+2YCEuotFtsU44SijBmJd0VyVjH/gHHeESVX/F+FxTykx5OaHQvp7AzVEHVlJYTY8+utqho4YEt+bm+9n8DV358v8TJua/kfmzv+eMRw2tdwP2eSlNPk= Authentication-Results: dkim=none (message not signed) header.d=none;dmarc=none action=none header.from=amd.com; Received: from CH2PR12MB4294.namprd12.prod.outlook.com (2603:10b6:610:a9::11) by CY8PR12MB8244.namprd12.prod.outlook.com (2603:10b6:930:72::7) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.6588.31; Fri, 21 Jul 2023 11:21:54 +0000 Received: from CH2PR12MB4294.namprd12.prod.outlook.com ([fe80::15a9:4e83:4217:8b49]) by CH2PR12MB4294.namprd12.prod.outlook.com ([fe80::15a9:4e83:4217:8b49%7]) with mapi id 15.20.6609.026; Fri, 21 Jul 2023 11:21:54 +0000 Message-ID: <32a6b0b0-c76d-b535-e727-e62e796ca429@amd.com> Date: Fri, 21 Jul 2023 12:21:48 +0100 User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:102.0) Gecko/20100101 Thunderbird/102.13.0 Content-Language: en-US To: Jochen Behrens , Thomas Monjalon , Igor de Paula Cc: "users@dpdk.org" , Nipun Gupta , Nikhil Agarwal , Ronak Doshi , venkat.immanni@amd.com, "Varghese, Vipin" References: <2663502.2IRrRt1zHL@thomas> <7070795.j6PcuT4dK6@thomas> From: Ferruh Yigit Subject: Re: help In-Reply-To: Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-ClientProxiedBy: LO4P123CA0173.GBRP123.PROD.OUTLOOK.COM (2603:10a6:600:18a::16) To CH2PR12MB4294.namprd12.prod.outlook.com (2603:10b6:610:a9::11) MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: CH2PR12MB4294:EE_|CY8PR12MB8244:EE_ X-MS-Office365-Filtering-Correlation-Id: 61931915-7cae-4dd6-dbdb-08db89dcaff5 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: IhaqRG+u8BaZHg41zOCHB557K9GXWbXPaKSxG5zL+Dc2SF5KsC+oCcSU57PuTHNDmpo8Q/BLmYFzbuGYuTm3v58cEwP+sHUD97Rc9YBCqfpgVwdFSojWKdnLGGQ/bAzg7bOGcsTwc0+5jxR+Y+xWK5YowTf/eZhCzzgKPTjjZCZuP2zvetfjDWz2YziUF2TAD4fIth76y9HGGUYaEwWplQNTF9UuSjpQ0S0rH3eY6UiA38xZoEUjbLuJc3aY68jF/kyMZqC64dYiihKq4tDBW9JDPkP17nxCN3xFHPz6kEWmN20PHj61sNKNVr+VmPKfOIuyBM/mTnIBeh1BphiSg6ikqPEJ3UW9jW+JJ76JA9ZbSjFX0uAtUYfcLOXaHTNp1+l8g/1gYXIl8Z636kVKrKSZOmjVbX5smXn9WNjZbJygC2b4JDL6iiF8B1f8NYqSwWQ4jxLk/3KhKsANOXWbvI5mpgfTXPuAwqM/Jx/D/7WtFK+Gi+BSvge7V1SZ32YUwo65+aQsR4ojH8TEZRUpM6aMZ0H9agKrRGUoFR/cfq49q8qdpdwAbF+u1pOLO6JFcHfGwBMuPCry9RtefsdNjalYAPmTjgPKJZup+gVjaMnLse7ZY83nIaDItXKdmAYH7QkUJLIN44e9/DpPIhxzMQ== X-Forefront-Antispam-Report: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:CH2PR12MB4294.namprd12.prod.outlook.com; PTR:; CAT:NONE; SFS:(13230028)(4636009)(376002)(136003)(366004)(39860400002)(396003)(346002)(451199021)(31686004)(6666004)(478600001)(6486002)(45080400002)(110136005)(66556008)(54906003)(83380400001)(31696002)(3480700007)(53546011)(86362001)(30864003)(6506007)(2906002)(36756003)(2616005)(966005)(316002)(6512007)(5660300002)(186003)(26005)(8936002)(66476007)(66946007)(44832011)(41300700001)(38100700002)(7116003)(8676002)(4326008)(45980500001)(43740500002); DIR:OUT; SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: =?utf-8?B?RUdBdVgwY3JoNXUyMFJORCtXN1Q5djZKWVZYRWxteUV6Z1RvSFMwcVp2dzJi?= =?utf-8?B?NlpNd09XZnFlSjA3WnVaR2ZvQ00vYVkwMWNxS0hXa0tOSFJETEgzRXVSQVdx?= =?utf-8?B?dmx6VlE0aUZueU5ITS9Pcng4ekpuK2NNa2ZyeE9tUXJmVUZoWUhOek1ZWG5i?= =?utf-8?B?ckNZUkhtN2VacVlTR0QrdkdYTTRoVHlRek5rM3RFb1FxMkQyOTdlSy9VOHA2?= =?utf-8?B?TnQ0Y2p4TFJwc01aNmd3NVNyMVJHR1JTdU93RTNQV2RNTE5jcHphdytwUUE3?= =?utf-8?B?aGtvdnNUTkNLeUJqMDFqZXYxRm8vQ2hPVEppWURmZVQ4ZGRLMXAwb1R4d3cx?= =?utf-8?B?MlhCRG8wc2hjSUJsb1IvRWIzSExGR1JGUHBrclpabjhCS3EvV2NMcHVXQXdX?= =?utf-8?B?TldVcDM2ZlF3azF4MFozdTNOUFJydi9oSS8yNnY1a1NiNnV0bEtsSk9UQW5a?= =?utf-8?B?cjRxZnFnNG9FOWxab2lEVmZlRENyVUFVM1dzdWd3YTh1M3dFNkVyNEFaeER3?= =?utf-8?B?VWVVNXE1d0xOdytzd3J0RVhWRFVqeVU2WUFQV0xZYUltVUF2VTdRTVNidnVw?= =?utf-8?B?S3EzK1NVUUNtMGlQa2p4MUNzNnlGTnFKSlBpdGgrZ055V1Roam5YUFlQTW9T?= =?utf-8?B?VTdjQmtPT0hBQkFJWG1iLzc2VVJHdEUyWFJVaitMaE1VdU91dE5rSFYwMHZI?= =?utf-8?B?QUVHMUJ2Z2gzaE9IbkVYUmZFZFRZZE1rck1rd2tDcTFXRTZrVnQrdkFJV2N1?= =?utf-8?B?TnRwMkJMNG84MmhId1N5TGdDb0hSUXZOQXlCa2ZPT2JjTmpZRmNJUWM3M090?= =?utf-8?B?MnBIeTkzRGtqM3hUUFQ3WFk0R1J2OUIzTVN3Vnl6UHUvSVY1QzBHK21ZREpX?= =?utf-8?B?RlZqN3hrMzc1YzFrUTQ3c2lUQktSeEx2OU5BVW1HZkFkcHA4TW1ja0VzZ0Zk?= =?utf-8?B?U1k0MWljdlpXWXI3N2xZS0w1TUF6T0loWUJMYkJzNklFTWVrTDdXQ1FiaUxw?= =?utf-8?B?OVhsOTFrRVhUcDBPM1BDTlRRVm5MY1dsQUhTSzkzWTg5MmVuYnRqckhER3Q4?= =?utf-8?B?ZjJUSThoNFhhSUcva2JqWGRrN1hWQlM4T01UdXhJM3dDY3F0Q3BRR29CTk9K?= =?utf-8?B?cVM1TFRPVWpMZHNIbjVrbUJ4TlhIQ0x4bnVsYUdpVzBYVUprM0treHdXOUFi?= =?utf-8?B?azErUW9pWUZuV1RXSFMyQmFIVWM3OVIrVDJUWTFIUkhlcVJEenVuWWxsRC9l?= =?utf-8?B?RHJaQUhteUJQTU5pZ05KS00zSnE3cEVXeW5PdUxDYzFqQUwwckRuTUgxdktN?= =?utf-8?B?dmwzaVhUU0FNMWxsWlB2VlRFTWc3V1o5RWxQekZWZ2RaVk4rZzRsZjZJcEhB?= =?utf-8?B?bWl6cFI5VFR2S2U5ZmdrN2pMQlVWc2pWaVp1empTdDExK1d0LzY5K3BOMzYr?= =?utf-8?B?Qk4rMGc4VVVaOGF0RTBWU2FkSzhJa0dPRUlrTFBEbU9nQ1Y1NnhGcWVtMnlI?= =?utf-8?B?anM2bld1VTE2MmtrQ3FEOWpUOGJONjNBUWJKMVJqWlMyV2xneTVsYkR6bXVN?= =?utf-8?B?S1JYWk52WjdSYVNBYUVkc2NGMFI1OW9ZUlB2bUtpcjN3TThVVkNhVjFjdE11?= =?utf-8?B?MkdvenFRVDJZeXhGcGsvZ1VLaG8vS0g4NE5hOURHNDdwZFptekxrT2sxR3Bs?= =?utf-8?B?bW40ZEpiY1kyQ2FoaUdrS2pTV0RjdFhXcHBkOFZ6Z3l2Q3l6RVJzbEZZRTRm?= =?utf-8?B?M2VwYlhhREJuNWsrUVdXTDhoV2lqa2FMTGQyVmp4dnRmbm9weFFvc2haZGMx?= =?utf-8?B?bmpOYWZoV3dsTGlIMWdKSFg3YmFZNlRDOUExcG1NRDYvUU9MRHF1TG9IL0tU?= =?utf-8?B?QjZZbW5hTXM2Ym5iMkdpc0lnZzYrWXdJZHlWMWNNVWxyVmhmRWtpdWcrb0ZI?= =?utf-8?B?ckEvRVlrQStucHI1bUhZSmpxalJYNnJPREFTN1UwWEk1bGN3a21VTnlGME5M?= =?utf-8?B?U1RGallOdHFaZ3p2UFNmVkxOZXZiZVA0TzJzRHJDQWF4OGZZNm11eVRzcGo2?= =?utf-8?B?TjFVeUg1aHZ1bDBWMEgzOGxnNk5YdmFQVkFhdnV5aXk5YzEybUZUYXB2TkZO?= =?utf-8?Q?kSxrEG8Wy6aNv/TSSsTVIzk26?= X-OriginatorOrg: amd.com X-MS-Exchange-CrossTenant-Network-Message-Id: 61931915-7cae-4dd6-dbdb-08db89dcaff5 X-MS-Exchange-CrossTenant-AuthSource: CH2PR12MB4294.namprd12.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 21 Jul 2023 11:21:54.3458 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: 0VEtzpzk44CAW2QPHrbQ4gC4uWeuN2DIK+tTc10uQ2+qzjX6WQmrdOha1d2BUCIc X-MS-Exchange-Transport-CrossTenantHeadersStamped: CY8PR12MB8244 X-BeenThere: users@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK usage discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: users-bounces@dpdk.org Hi Igor, VM doesn't have IOMMU, and vmxnet3 requires PA mode, for this can you please try with: - enable 'enable_unsafe_noiommu_mode' flag - Force PA mode via '--iova-mode=pa' eal parameter Also to be able to figure out AMD IOMMU support level, can you please provide: - AMD part number - ESXi/hypervisor version - ESXi & VM kernel logs Thanks, Ferruh On 7/20/2023 5:21 PM, Jochen Behrens wrote: > +Ronak from the ESX team > >   > >   > > In our usage, we do set amd_iommu=off in the boot command line from > grub. (Or intel_iommu=off for Intel processors.) > >   > >                 Jochen > >   > > *From: *Thomas Monjalon > *Date: *Thursday, July 20, 2023 at 6:00 AM > *To: *Igor de Paula > *Cc: *users@dpdk.org , Jochen Behrens > , Nipun Gupta , Nikhil Agarwal > , Ferruh Yigit > *Subject: *Re: help > > !! External Email > > +Cc some AMD maintainers, they can have an idea about IOMMU settings. > > > 20/07/2023 14:44, Igor de Paula: >> I have enabled it in the host and in the BIOS for AMD... >> In the Bios I changed to amd_iommu=on and in the host it's the same for >> either. >> >> On Thu, Jul 20, 2023 at 1:31 PM Thomas Monjalon wrote: >> >> > 20/07/2023 11:35, Igor de Paula: >> > > The weird thing is that it only happens when I am using a host with an >> > AMD >> > > processor. It doesn't happen when I use a host with an Intel processor. >> > >> > So it's probably a matter of BIOS settings for the IOMMU? >> > >> > >> > > On Thu, Jul 20, 2023 at 10:32 AM Thomas Monjalon >> > > wrote: >> > > >> > > > +Cc the vmxnet3 maintainer. >> > > > >> > > > Please Jochen, do you have an idea what's wrong below? >> > > > >> > > > >> > > > 20/07/2023 11:25, Igor de Paula: >> > > > > This is because it can't negotiate the IOMMU type with any port. >> > > > > >> > > > > On Thu, Jul 20, 2023 at 5:08 AM Thomas Monjalon > > > >> > > > wrote: >> > > > > >> > > > > > Hello, >> > > > > > >> > > > > > The first error is "Cause: Error: number of ports must be even" >> > > > > > >> > > > > > >> > > > > > 03/05/2023 18:13, Igor de Paula: >> > > > > > > I am running a VM inside a VMWARE server (vSphere). >> > > > > > > My goal it to set up DPDK with two HW ports, and set up a >> > > > virtio_user to >> > > > > > > interact with the kernel stack. >> > > > > > > In another app I have it working but instead of virtio_user I am >> > > > running >> > > > > > > KNI, it works in IOVA-PA mode. >> > > > > > > I am looking to replace the KNI. >> > > > > > > >> > > > > > > When I try to set up virtio_user port as in the doc: >> > > > > > > >> > > > > > >> > > > >> > https://doc.dpdk.org/guides/howto/virtio_user_as_exception_path.html#virtio-user-as-exception-path >> > > > > > > I get a error it can't run in PA mode. >> > > > > > > >> > > > > > > >> > > > > > > When I try to run as VA mode from a parameter, I get the >> > following >> > > > > > errors: >> > > > > > > EAL: lib.eal log level changed from info to debug >> > > > > > > EAL: Detected lcore 0 as core 0 on socket 0 >> > > > > > > EAL: Detected lcore 1 as core 0 on socket 0 >> > > > > > > EAL: Support maximum 128 logical core(s) by configuration. >> > > > > > > EAL: Detected 2 lcore(s) >> > > > > > > EAL: Detected 1 NUMA nodes >> > > > > > > EAL: Checking presence of .so 'librte_eal.so.21.3' >> > > > > > > EAL: Checking presence of .so 'librte_eal.so.21' >> > > > > > > EAL: Checking presence of .so 'librte_eal.so' >> > > > > > > EAL: Detected static linkage of DPDK >> > > > > > > EAL: Ask a virtual area of 0x7000 bytes >> > > > > > > EAL: Virtual area found at 0x100000000 (size = 0x7000) >> > > > > > > EAL: Multi-process socket /var/run/dpdk/rte/mp_socket >> > > > > > > EAL: DPAA Bus not present. Skipping. >> > > > > > > EAL: VFIO PCI modules not loaded >> > > > > > > EAL: Selected IOVA mode 'VA' >> > > > > > > EAL: Probing VFIO support... >> > > > > > > EAL: IOMMU type 1 (Type 1) is supported >> > > > > > > EAL: IOMMU type 7 (sPAPR) is not supported >> > > > > > > EAL: IOMMU type 8 (No-IOMMU) is supported >> > > > > > > EAL: VFIO support initialized >> > > > > > > EAL: Ask a virtual area of 0x5b000 bytes >> > > > > > > EAL: Virtual area found at 0x100007000 (size = 0x5b000) >> > > > > > > EAL: Setting up physically contiguous memory... >> > > > > > > EAL: Setting maximum number of open files to 1048576 >> > > > > > > EAL: Detected memory type: socket_id:0 hugepage_sz:1073741824 >> > > > > > > EAL: Creating 2 segment lists: n_segs:128 socket_id:0 >> > > > > > hugepage_sz:1073741824 >> > > > > > > EAL: Ask a virtual area of 0x2000 bytes >> > > > > > > EAL: Virtual area found at 0x100062000 (size = 0x2000) >> > > > > > > EAL: Memseg list allocated at socket 0, page size 0x100000kB >> > > > > > > EAL: Ask a virtual area of 0x2000000000 bytes >> > > > > > > EAL: Virtual area found at 0x140000000 (size = 0x2000000000) >> > > > > > > EAL: VA reserved for memseg list at 0x140000000, size 2000000000 >> > > > > > > EAL: Ask a virtual area of 0x2000 bytes >> > > > > > > EAL: Virtual area found at 0x2140000000 (size = 0x2000) >> > > > > > > EAL: Memseg list allocated at socket 0, page size 0x100000kB >> > > > > > > EAL: Ask a virtual area of 0x2000000000 bytes >> > > > > > > EAL: Virtual area found at 0x2180000000 (size = 0x2000000000) >> > > > > > > EAL: VA reserved for memseg list at 0x2180000000, size 2000000000 >> > > > > > > EAL: TSC frequency is ~2350000 KHz >> > > > > > > EAL: Main lcore 0 is ready (tid=7f8ad790ec00;cpuset=[0]) >> > > > > > > EAL: lcore 1 is ready (tid=7f8ad6907400;cpuset=[1]) >> > > > > > > EAL: Trying to obtain current memory policy. >> > > > > > > EAL: Setting policy MPOL_PREFERRED for socket 0 >> > > > > > > EAL: Restoring previous memory policy: 0 >> > > > > > > EAL: request: mp_malloc_sync >> > > > > > > EAL: Heap on socket 0 was expanded by 1024MB >> > > > > > > EAL: PCI device 0000:0b:00.0 on NUMA socket -1 >> > > > > > > EAL:   probe driver: 15ad:7b0 net_vmxnet3 >> > > > > > > EAL:   Expecting 'PA' IOVA mode but current mode is 'VA', not >> > > > > > initializing >> > > > > > > EAL: Requested device 0000:0b:00.0 cannot be used >> > > > > > > EAL: PCI device 0000:13:00.0 on NUMA socket -1 >> > > > > > > EAL:   probe driver: 15ad:7b0 net_vmxnet3 >> > > > > > > EAL:   Expecting 'PA' IOVA mode but current mode is 'VA', not >> > > > > > initializing >> > > > > > > EAL: Requested device 0000:13:00.0 cannot be used >> > > > > > > EAL: Bus (pci) probe failed. >> > > > > > > EAL: lib.telemetry log level changed from disabled to warning >> > > > > > > EAL: Error - exiting with code: 1 >> > > > > > >   Cause: Error: number of ports must be even >> > > > > > > EAL: request: mp_malloc_sync >> > > > > > > EAL: Heap on socket 0 was shrunk by 1024MB >> > > > > > > >> > > > > > > >> > > > > > > >> > > > > > > For some reason the HW ports won't setup. From what I understand >> > > > > > > net_vmxnet3 should work with VA mode. >> > > > > > > I enabled I/OMUU for the VM. >> > > > > > > The weird thing even when enabled, I still have the >> > > > > > > enable_unsafe_noiommu_mode flag on. >> > > > > > > And because it's on the this: >> > > > > > > >> > > > > > > dev_iova_mode = pci_device_iova_mode(dr, dev); >> > > > > > > >> > > > > > > return PA mode, and it fails. >> > > > > > > >> > > > > > > When I disable it by modifying >> > > > > > > /sys/module/vfio/parameters/enable_unsafe_noiommu_mode, I get >> > another >> > > > > > error. >> > > > > > > The error is that it doesn't find a suitable IOMMU type: >> > > > > > > Just putting the relevant message: >> > > > > > > >> > > > > > > >> > > > > > > EAL: Heap on socket 0 was expanded by 1024MB >> > > > > > > EAL: PCI device 0000:0b:00.0 on NUMA socket -1 >> > > > > > > EAL:   probe driver: 15ad:7b0 net_vmxnet3 >> > > > > > > EAL: Set IOMMU type 1 (Type 1) failed, error 19 (No such device) >> > > > > > > EAL: Set IOMMU type 7 (sPAPR) failed, error 19 (No such device) >> > > > > > > EAL: Set IOMMU type 8 (No-IOMMU) failed, error 19 (No such >> > device) >> > > > > > > EAL: 0000:0b:00.0 failed to select IOMMU type >> > > > > > > EAL: Requested device 0000:0b:00.0 cannot be used >> > > > > > > EAL: PCI device 0000:13:00.0 on NUMA socket -1 >> > > > > > > EAL:   probe driver: 15ad:7b0 net_vmxnet3 >> > > > > > > EAL: Set IOMMU type 1 (Type 1) failed, error 19 (No such device) >> > > > > > > EAL: Set IOMMU type 7 (sPAPR) failed, error 19 (No such device) >> > > > > > > EAL: Set IOMMU type 8 (No-IOMMU) failed, error 19 (No such >> > device) >> > > > > > > EAL: 0000:13:00.0 failed to select IOMMU type >> > > > > > > EAL: Requested device 0000:13:00.0 cannot be used >> > > > > > > >> > > > > > > >> > > > > > > I can see where it fails: >> > > > > > > >> > > > > > > >> > > > > > > const struct vfio_iommu_type * >> > > > > > > vfio_set_iommu_type(int vfio_container_fd) >> > > > > > > { >> > > > > > > unsigned idx; >> > > > > > > for (idx = 0; idx < RTE_DIM(iommu_types); idx++) { >> > > > > > > const struct vfio_iommu_type *t = &iommu_types[idx]; >> > > > > > > >> > > > > > > int ret = ioctl(vfio_container_fd, VFIO_SET_IOMMU, >> > > > > > > t->type_id); >> > > > > > > if (!ret) { >> > > > > > > RTE_LOG(INFO, EAL, "Using IOMMU type %d (%s)\n", >> > > > > > > t->type_id, t->name); >> > > > > > > return t; >> > > > > > > } >> > > > > > > /* not an error, there may be more supported IOMMU types */ >> > > > > > > RTE_LOG(DEBUG, EAL, "Set IOMMU type %d (%s) failed, error " >> > > > > > > "%i (%s)\n", t->type_id, t->name, errno, >> > > > > > > strerror(errno)); >> > > > > > > } >> > > > > > > /* if we didn't find a suitable IOMMU type, fail */ >> > > > > > > return NULL; >> > > > > > > } >> > > > > > > >> > > > > > > >> > > > > > > The ioctl returns -1. >> > > > > > > >> > > > > > > >> > > > > > > >> > > > > > > I tried many things. >> > > > > > > Enabling/disabling IOMMU. >> > > > > > > Modifying the enable_unsafe_noiommu_mode flag. >> > > > > > > Running in PA mode. >> > > > > > > Any help will be much appreciated. >> > >> > >> > >> > >> > >> > >> > > > > > > > !! External Email: This email originated from outside of the > organization. Do not click links or open attachments unless you > recognize the sender. >