From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 649C7460E0; Wed, 22 Jan 2025 09:15:12 +0100 (CET) Received: from mails.dpdk.org (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 384FF402C9; Wed, 22 Jan 2025 09:15:12 +0100 (CET) Received: from EUR02-DB5-obe.outbound.protection.outlook.com (mail-db5eur02on2049.outbound.protection.outlook.com [40.107.249.49]) by mails.dpdk.org (Postfix) with ESMTP id 24135402A9 for ; Wed, 22 Jan 2025 09:15:11 +0100 (CET) ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=bYGPaViRSzN7y6GuZnJKaBogIRr0U7H1S47FjrkMQhEgG9oXq0yj79tKuQldtT4ZfOvwftW81HKjv0C1L7j5kKaatKiWDRzVMUSeo3KewfHmOzJo9kDm6Ak/JA+VvYnXqtY43sD3RZ7ry4+y+bnppq+bg/yO/zMi13aVjZUOBjatx3P5hLzXr3Ix1MQvY1j4CT1eyiJ2ZXE3mXEd29ANpfTuF2k9sll6bK+uvFDRzGGbLZKmpnNSASS41j4IbR9WjB6+3Qq+HE6outa1czuwZ1YaRIGwmyX8p/oWU3Pvba0oGr5wCp0jN8DY+gbcKLWfIDEPxdtixb2F+Q1gJmuFyA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=04nZrpQAdbTm5vQplU4DefDayQ1j0lOu50zboDMCIu0=; b=RJzND2jL1xBMWpDIs2bsLZPMOPyhRaLQVjAPl36MM/Ow0VA6L8PA0djyZ651q/Ej0EIISMGfpVCpzR2dGTpvV4ujyip1vD2OW4cG5E0k8mvxIJ78/1zDME6RhWgbR+XIZibxta0YXyzaOEVT0OwreJKSMtLsGH5pAHtDo0lxOJFW8ASwUUq5KiFDecQ4nwKuv07UF2Lq8KlNqdhBHEV1tZNfwVPWuNzTxE2EuYNiwZN3dcqdEzWpbP4alFx7fHxIq1yeqVSLW2EhHNbTlXDq4Xt5Exvbp/Isoyuec1CpLExRFBZK1XNqzPqpr15b6L72nTFKzjEVKI7GMZSK5AqrhQ== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=nokia-sbell.com; dmarc=pass action=none header.from=nokia-sbell.com; dkim=pass header.d=nokia-sbell.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=nokia-sbell.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=04nZrpQAdbTm5vQplU4DefDayQ1j0lOu50zboDMCIu0=; b=oA4s16zzqyTEePRXXksWTz7OpP9gfTXA77DKOYI1xHfh/I39KS1g4I/8Nvml/gXH90MTGgFKuPBfR0E5qHytSKJQczJKRX1HYvR/3JkDKhLgPtorHVli+bys2kTyss+NBwhvfXr3IUHQcZGxRK5XjtghAJc/Le3RhPr3U/kmA7Dpveml4rHhm1du7ovkda5wDC6lClcjHrqwDoA8urmKXx7nfideEwQrwMSofEVUm3gDNGeB7ru3b3/dVTDW/GKuRWNaOwIUT45xyVdLxfSzcXg6tj4FDno3HO3lEfW0VDD4rT4uzwH3FEMfRwx8EeO4ErlyP0un+VuZrET6KQUziw== Authentication-Results: dkim=none (message not signed) header.d=none;dmarc=none action=none header.from=nokia-sbell.com; Received: from VI1PR07MB9898.eurprd07.prod.outlook.com (2603:10a6:800:119::6) by AM9PR07MB7826.eurprd07.prod.outlook.com (2603:10a6:20b:301::21) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.8356.22; Wed, 22 Jan 2025 08:15:09 +0000 Received: from VI1PR07MB9898.eurprd07.prod.outlook.com ([fe80::1928:4afb:4eee:1f21]) by VI1PR07MB9898.eurprd07.prod.outlook.com ([fe80::1928:4afb:4eee:1f21%6]) with mapi id 15.20.8356.020; Wed, 22 Jan 2025 08:15:09 +0000 Message-ID: Date: Wed, 22 Jan 2025 16:15:03 +0800 User-Agent: Mozilla Thunderbird Subject: Re: [PATCH] Skip vfio in the scenario of non-privileged mode To: Stephen Hemminger Cc: Anatoly Burakov , dev@dpdk.org References: <20250117072847.2741-1-ming.1.yang@nokia-sbell.com> <20250117084750.3259cfa3@hermes.local> Content-Language: en-US From: Yang Ming In-Reply-To: <20250117084750.3259cfa3@hermes.local> Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 8bit X-ClientProxiedBy: SI2PR01CA0010.apcprd01.prod.exchangelabs.com (2603:1096:4:191::12) To VI1PR07MB9898.eurprd07.prod.outlook.com (2603:10a6:800:119::6) MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: VI1PR07MB9898:EE_|AM9PR07MB7826:EE_ X-MS-Office365-Filtering-Correlation-Id: 05db0d00-7e12-4dc3-d405-08dd3abce2e2 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0;ARA:13230040|376014|1800799024|366016; X-Microsoft-Antispam-Message-Info: =?utf-8?B?ckVIVStkN3p4RkRoOHU3WEkwNjJUMmdiNXkraHNiSzF2TUEvSUJYRUVibnV2?= =?utf-8?B?WFFXMm00Z0I0NUJ2MVpFZjRReG40VXhhVWpuU2dWRUxRWDZsOXZRMHJqUTBy?= =?utf-8?B?TG1FMlJpcGQ2VGJlS0lTaVMzUndEYzA3Z25IeEpleEFHVXE2NkQ3UVZmWTI3?= =?utf-8?B?d3k0MUgzS1RacS9pMGc2VCtwUzE0a3B5R0xMS0x4SVlXR2NCeWZVL0VMcGpn?= =?utf-8?B?dW5YYWd6V1pCRzcxK04wbFJDb25qYWtmbGpSdzVSeUlFSDFOekl3aHAyZHNn?= =?utf-8?B?VTlncEtVNVkwMW0zRlZwUWFzcUVuSCtNRWFiNWhNblovRlJ1eFJqWDdidFQw?= =?utf-8?B?UmFFWi9yOEJDY3JVRS9OaGQ0azNUZHdVbkJTSFdzWkcybW42VGhZQzE3clEx?= =?utf-8?B?RjdmdUIrM2M0WkdWUUVUcW16UGh4RFpYMDIxQWFoU2R1ZVVWZEk4NkNRWEJZ?= =?utf-8?B?OXlDVS96T1NINktaWS9WanhOaVB6ZTZEY08vdjR1dlFhenBDNk1HYk5HdVZY?= =?utf-8?B?NDZDY2JVZEc1NTNqTEpqTmYyczhyQTZpUDYrSmVWeGRPQ2luMlhKZy9vWmZz?= =?utf-8?B?Tm1YbXlmWC9lZE5OdGJPZmMvenZtaEtLWjVtellPZi9BZW13RXdXOXhITVVz?= =?utf-8?B?SnVYUnFGZktaNEVxTWIrV0xkWFJBSmVWMkZZQlV1YmFsaVVDeFRoUHR4WlVX?= =?utf-8?B?QUJYc3VQUnpYb3VSRnR1WnJ2NVpRdFNYTDlPU3c3bi9Tei9GbTZubzBUUG5M?= =?utf-8?B?a08rNytRekNkc3JXZG16eGVIK0t2dnZHek11UFJZdmJKcHVQWGtqVkk0NVJL?= =?utf-8?B?UG5neHNaMWlmbXo0RmRlS1NJUm5zSlN2dU16L0V5Ym5XY0RuWFVoNVdpUTV3?= =?utf-8?B?QWtWdnlVNEdiWGlCNmxnLzNtWXAybjladGh5dFpqbHo1VU5IL0FYQzBCOHc5?= =?utf-8?B?S0xwU0Z2RjFSMlRJWDdFN1R2dWQ2cC9RNmxBTE1NSzZ6NTFjYnUrVlcvVkdl?= =?utf-8?B?UmlVWm5iNlI0a2R6Y1Q5WnA2cFI1dHV2NmdtN3JKSjdRZTJwNWZvcnducFNx?= =?utf-8?B?NkgzdGVVNlVqamY0VWZQNDg1MEZRK2lXcjVUT0hGaS9iemJ0Zkdwamo4bTNS?= =?utf-8?B?THZWWW1oQStETDd4ZUJFa3dVUzFuM2ZGYUNsT0R0THNIeTdnWmw2VkkxbUg4?= =?utf-8?B?b3lhVWVRa2FjWHF3Q1h2d1hncDNEZHU4TjR1cExteUpEQzFvYVoybnI1b1Nt?= =?utf-8?B?SXpvakZZeWRoaGhxZGdTUStoSFB0ZHNoTldFVXVFZGhhZHEwcXZhMFR1THRk?= =?utf-8?B?Vm9mb1BpQ3R4cVNrYUZzM3ZOUWdOdGJHMW9wbGdDUlB1K3A2RXdMekdRR0xy?= =?utf-8?B?YklGVlRhY3FSdVhtN1JRdkNvZzFVcDVGZGMweUhIbTVLYktNbFdjNkxDOGdl?= =?utf-8?B?K2JCazFWQUw2TW9YN2N0TEk4aHI1VHVYbDJYbSttQm1yRWJtVkxlS2FNeCtJ?= =?utf-8?B?SElCUUxPa0ZuakxKMGJnOWZqMXpXaE9jV3JPeUh4bEs2Y1dseWdjQ2hEaUQ5?= =?utf-8?B?WVVOY3ZPYjB5ZE1QcGZHS3Q3SmdYQnRxditvYitJbkJUQUNKU3JYS1VpT1BX?= =?utf-8?B?ZDRLRHlyWWZTMnBUM2tFb25jUUFLZU5HcWtSK09pbDNPNXpYNFVKTGpOelhJ?= =?utf-8?B?Z0k4NGlqa3FLaDVmTXlZMWRLV2R0YjBRb2ZRU1BJNjRnS0JSY21zQkNDU09t?= =?utf-8?B?ZVA0WWVvZ2tuMkNkSWNTN1J3ZWk3bkdWYU5Nc3ZJOXQ2KzdHWG12RWlFakx0?= =?utf-8?B?Vi9UTVhtTDVnM1Y1T3o4dz09?= X-Forefront-Antispam-Report: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:VI1PR07MB9898.eurprd07.prod.outlook.com; PTR:; CAT:NONE; SFS:(13230040)(376014)(1800799024)(366016); DIR:OUT; SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: =?utf-8?B?VWFGdmZmQlBRTkFoM3ZOd3l3b29DcUwzZEZUbisxQ2FJWUx5NVR4UG10T2o4?= =?utf-8?B?ZDJHT0x3aVhqUTZEYkFmUkhzSjhMeVRrWEdTY2x2RzFOTTlrQ2ZRdTkrU1RZ?= =?utf-8?B?NmRqWEx5bk5ZbitSaE5BdGkxTG1lMzBRUVIvWXNTdXJoMHRteldISHMvbkk3?= =?utf-8?B?dzdIOEcvV0tuWTlIcHVPN2E2M1lrSC9HMHdSWk5oeWxBMEM5THhUanVpcW5l?= =?utf-8?B?RDFnR1RNby9mTkxKMDVGVXI1elZ1dUNJWWR0L2Zoc0V2UE50ZkwxdTVZQTFq?= =?utf-8?B?WXBhejNzWUJ1SVlkN0NzRktUb1N6eVlwelZ4YlQ5bFF0MjFCenpzL0dUbllm?= =?utf-8?B?RC9QNTQwdmJMMkRVaCt3SGRyVWMyS0dEck9NSXJBaVc4ZXdZeGRDK2VVVjBn?= =?utf-8?B?OWhXdDVKRFh1cXluSnM2bllDOGU1NWdhZ2huejdrdnZjYUNXbEZCb2pYSkhY?= =?utf-8?B?Znc4a3FoTXA1bzdHTFl6TkRGcU9RdHNxeVd0RDR1dWFpazhCd0xiSXlKTkF2?= =?utf-8?B?YlcySW5jOS9xOGdIc1k2SUt5ZTZyQ0JXLy9uOVpobmNVTWNTQVVvaDBSZVpp?= =?utf-8?B?SzZoSWZJd2w4Y0w3dWkwMnlIOGpUQ2JVWWpjQXhVY1NQUVRzVElFenMzUThJ?= =?utf-8?B?QlJ2RTh2N1p0OWpIZzRZb21DWVF6MDRiZFJYUDJzcXJvTmhwMGlCMHZ1NUFW?= =?utf-8?B?dlVwOVB2UWs3R2o3T1pHMGxyR2pOVWk4ZTlNNzBHUysyRklZYThDbmRhbFpL?= =?utf-8?B?dHJsZTFJVFBVOXp3R251M01LeWROblp2UUtSTXprTHZYOTNTbE01aERVWTJl?= =?utf-8?B?eHhPUXdCem1wZUgwWElFVk5VMUV4aXpFM0hZbkF1VUcybGFjenVqcUlzdUY2?= =?utf-8?B?UFI4U1R1cWpPNTR5SVpYUTVtUEhLbzFzWW9vSmFDc0xQbk4wTWZyMjhTenEy?= =?utf-8?B?Qkx5d01nOW5rOEFhVlNHbUdTbFZQUWQ4aVZBQkpIc3k3SVZDdEJVL3YyL3lE?= =?utf-8?B?eGk5Zm9QL3FQSzV1TEJoMkE5M3hkN21JTFp4L1FGb2N1ZThXMlF2UkpBYlNh?= =?utf-8?B?dWpuVnFuNHJaL2cyNzhkU3ZZMXpqVzRQWEUrcHUvMUkyS21kMUVJalVmblVX?= =?utf-8?B?NjdQVTBCTmR0dXgwR3AxRzNqUzdCc0U2dXAvTUQxR2I2OXJNcjl5czRQNG1w?= =?utf-8?B?bkoraGUwVVRIZFEycDNmbFhtUG9XSUsyQ2xyWjdYN2U3b3BCakhCYmwrVGNZ?= =?utf-8?B?Mi9TYnZ1a1ExRDlOd2lwYkM0ZDRRTDhnSCtpWTdOR0w2L3FzZnp0UnM3bHBz?= =?utf-8?B?UHhXbTJFLzE0WXBlWVg4d2lOQk5xS1dVVm9KVm14NzUzSzhoWC9Hc04zTEkv?= =?utf-8?B?cEVJZWlMS3dia1NPY0h4bDB5UnM5eHN2V2RDaDdMMHdSL0xueVdSS2VvTkpx?= =?utf-8?B?bE1CcEpPRmE5c3NuOU5HSVp1eWpXRDd1bmFjS0orSFd0K1RBM001aUVBa01L?= =?utf-8?B?VDhvZXhVOXNTSVFBUnRUOTV1Rnc4ZVhQYm13NjgrZDJhZ1NIQzFWQzhieGFE?= =?utf-8?B?MjdkeStQa0tMMS8zQlIrVjZpblR2T3pWVjFqa0QvUllFaDZCZ1lUbWg5VVJF?= =?utf-8?B?MWNqbDk3cTBUSFl6SFQvTThuVDNuNmZkTVlkR2NHSko3cjVxTXdsZ0ZwMUtI?= =?utf-8?B?ZVNxQjdmcGgvZU42VmVCdDQrTVptUVZnd0JDbUxiRXFTbzVGaWQ1MGk3NVZK?= =?utf-8?B?em0wWFp3VXhNbmRRY1h2MGNhVHQ2ekVheU1MMGZNYmhsNWJNQWQ1TXQzV3Iz?= =?utf-8?B?ZktjbFJVM2d5SENRYVZ6eUNVYzlGemQxbURmRDVTYXF1U3h2ajBRbUc5cC9S?= =?utf-8?B?YnVXYVhRZzFCT3VWZzh4Uk5WakgvZkZIUUdiTHdXYzJwOXAvdnhoZ2gySjZ5?= =?utf-8?B?NFpoN1dMY3VCTnc1TmdaQXBJUHNZVjFXdm95WTdUMlIveENvOUsyUytncWlB?= =?utf-8?B?eWJ5cExXamYrVCtlK29GTGg0ZlV0L0lrbzZaRzg0SFdoSTJLQm9HRFFYV29U?= =?utf-8?B?RndBZE5nRlRBd3ZjVzE3VjhLSXFMU2psVzNoMDg0QTMraExnK3FxalZ3NVMr?= =?utf-8?B?emRaRGtlL0xhUFp1ejlpT1dkd0t2azAvS0pyQmpKcnFieG1ZcHo5c0lCNTlo?= =?utf-8?B?QVE9PQ==?= X-OriginatorOrg: nokia-sbell.com X-MS-Exchange-CrossTenant-Network-Message-Id: 05db0d00-7e12-4dc3-d405-08dd3abce2e2 X-MS-Exchange-CrossTenant-AuthSource: VI1PR07MB9898.eurprd07.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 22 Jan 2025 08:15:09.4002 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 5d471751-9675-428d-917b-70f44f9630b0 X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: 5an8JPSr8LCYmRdjQj4JIehD08jBHGrj9FBHTYWt0yV+JHQZUHH6RDHqiorguqUzDkda5eafNQ1i4/eqkzlW5YP+spT+7HhhJRvQHEFjeo8= X-MS-Exchange-Transport-CrossTenantHeadersStamped: AM9PR07MB7826 X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org On 2025/1/18 00:47, Stephen Hemminger wrote: > Caution: This is an external email. Please be very careful when clicking links or opening attachments. See http://nok.it/nsb for additional information. > > On Fri, 17 Jan 2025 15:28:47 +0800 > Yang Ming wrote: > >> DPDK detect vfio container according the existence of vfio >> module. But for container with non-privileged mode, there is >> possibility that no VFIO_DIR(/dev/vfio) mapping from host to >> container when host have both Intel NIC and Mellanox NIC but >> this conntainer only allocate VFs from Mellanox NIC. >> In this case, vfio kernel module has already been loaded from >> the host. >> This scenario will cause the error log occurs in DPDK primary >> process as below: >> 'EAL: cannot open VFIO container, error 2 (No such file or >> directory)' >> 'EAL: VFIO support could not be initialized' >> Because `rte_vfio_enable()` call `rte_vfio_get_container_fd()` >> to execute `vfio_container_fd = open(VFIO_CONTAINER_PATH, >> O_RDWR);` but VFIO_CONTAINER_PATH(/dev/vfio/vfio) doesn't exist >> in this container. >> This scenario will also lead to the delay of DPDK secondary >> process because `default_vfio_cfg->vfio_enabled = 0` and >> `default_vfio_cfg->vfio_container_fd = -1`, socket error will >> be set in DPDK primary process when it sync this info to >> the secondary process. >> This patch use to skip this kind of useless detection for this >> scenario. >> >> Signed-off-by: Yang Ming >> --- >> lib/eal/linux/eal_vfio.c | 11 +++++++++++ >> 1 file changed, 11 insertions(+) >> >> diff --git a/lib/eal/linux/eal_vfio.c b/lib/eal/linux/eal_vfio.c >> index 7132e24cba..1679d29263 100644 >> --- a/lib/eal/linux/eal_vfio.c >> +++ b/lib/eal/linux/eal_vfio.c >> @@ -7,6 +7,7 @@ >> #include >> #include >> #include >> +#include >> >> #include >> #include >> @@ -1083,6 +1084,7 @@ rte_vfio_enable(const char *modname) >> /* initialize group list */ >> int i, j; >> int vfio_available; >> + DIR *dir; >> const struct internal_config *internal_conf = >> eal_get_internal_configuration(); >> >> @@ -1119,6 +1121,15 @@ rte_vfio_enable(const char *modname) >> return 0; >> } >> >> + /* return 0 if VFIO directory not exist for container with non-privileged mode */ >> + dir = opendir(VFIO_DIR); >> + if (dir == NULL) { >> + EAL_LOG(DEBUG, >> + "VFIO directory not exist, skipping VFIO support..."); >> + return 0; >> + } >> + closedir(dir); > You need to test the non-container cases. > If vfio is loaded /dev/vfio is a character device (not a directory) > > Also looks suspicious that VFIO_DIR is defined but never used currently. > Hi Stephen, For non-container test, /dev/vfio/vfio will be character device, not /dev/vfio. Here is the command result on my testing environment with Intel NIC. [root@computer-1 testuser]# ls -l /dev/vfio total 0 crw-rw-rw-. 1 root root 10, 196 Jan 22 01:50 vfio [root@computer-1 testuser]# dpdk-devbind.py -b vfio-pci 0000:04:10.2 [root@computer-1 testuser]# ls -l /dev/vfio total 0 crw-------. 1 root root 239,   0 Jan 22 01:52 59 crw-rw-rw-. 1 root root  10, 196 Jan 22 01:50 vfio [root@computer-1 testuser]# dpdk-devbind.py -b ixgbevf 0000:04:10.2 [root@computer-1 testuser]# ls -l /dev/vfio total 0 crw-rw-rw-. 1 root root 10, 196 Jan 22 01:50 vfio Can you confirm your test scenario?