[PATCH] Skip vfio in the scenario of non-privileged mode

[PATCH] Skip vfio in the scenario of non-privileged mode

[Yang Ming]

DPDK detect vfio container according the existence of vfio
module. But for container with non-privileged mode, there is
possibility that no VFIO_DIR(/dev/vfio) mapping from host to
container when host have both Intel NIC and Mellanox NIC but
this conntainer only allocate VFs from Mellanox NIC.
In this case, vfio kernel module has already been loaded from
the host.
This scenario will cause the error log occurs in DPDK primary
process as below:
'EAL:   cannot open VFIO container, error 2 (No such file or
directory)'
'EAL: VFIO support could not be initialized'
Because `rte_vfio_enable()` call `rte_vfio_get_container_fd()`
to execute `vfio_container_fd = open(VFIO_CONTAINER_PATH,
O_RDWR);` but VFIO_CONTAINER_PATH(/dev/vfio/vfio) doesn't exist
in this container.
This scenario will also lead to the delay of DPDK secondary
process because `default_vfio_cfg->vfio_enabled = 0` and
`default_vfio_cfg->vfio_container_fd = -1`, socket error will
be set in DPDK primary process when it sync this info to
the secondary process.
This patch use to skip this kind of useless detection for this
scenario.

Signed-off-by: Yang Ming <ming.1.yang@nokia-sbell.com>
---
 lib/eal/linux/eal_vfio.c | 11 +++++++++++
 1 file changed, 11 insertions(+)

diff --git a/lib/eal/linux/eal_vfio.c b/lib/eal/linux/eal_vfio.c
index 7132e24cba..1679d29263 100644
--- a/lib/eal/linux/eal_vfio.c
+++ b/lib/eal/linux/eal_vfio.c
@@ -7,6 +7,7 @@
 #include <fcntl.h>
 #include <unistd.h>
 #include <sys/ioctl.h>
+#include <dirent.h>
 
 #include <rte_errno.h>
 #include <rte_log.h>
@@ -1083,6 +1084,7 @@ rte_vfio_enable(const char *modname)
 	/* initialize group list */
 	int i, j;
 	int vfio_available;
+	DIR *dir;
 	const struct internal_config *internal_conf =
 		eal_get_internal_configuration();
 
@@ -1119,6 +1121,15 @@ rte_vfio_enable(const char *modname)
 		return 0;
 	}
 
+	/* return 0 if VFIO directory not exist for container with non-privileged mode */
+	dir = opendir(VFIO_DIR);
+	if (dir == NULL) {
+		EAL_LOG(DEBUG,
+			"VFIO directory not exist, skipping VFIO support...");
+		return 0;
+	}
+	closedir(dir);
+
 	if (internal_conf->process_type == RTE_PROC_PRIMARY) {
 		if (vfio_mp_sync_setup() == -1) {
 			default_vfio_cfg->vfio_container_fd = -1;
-- 
2.34.1

Re: [PATCH] Skip vfio in the scenario of non-privileged mode

[Stephen Hemminger]

On Fri, 17 Jan 2025 15:28:47 +0800
Yang Ming <ming.1.yang@nokia-sbell.com> wrote:

> DPDK detect vfio container according the existence of vfio
> module. But for container with non-privileged mode, there is
> possibility that no VFIO_DIR(/dev/vfio) mapping from host to
> container when host have both Intel NIC and Mellanox NIC but
> this conntainer only allocate VFs from Mellanox NIC.
> In this case, vfio kernel module has already been loaded from
> the host.
> This scenario will cause the error log occurs in DPDK primary
> process as below:
> 'EAL:   cannot open VFIO container, error 2 (No such file or
> directory)'
> 'EAL: VFIO support could not be initialized'
> Because `rte_vfio_enable()` call `rte_vfio_get_container_fd()`
> to execute `vfio_container_fd = open(VFIO_CONTAINER_PATH,
> O_RDWR);` but VFIO_CONTAINER_PATH(/dev/vfio/vfio) doesn't exist
> in this container.
> This scenario will also lead to the delay of DPDK secondary
> process because `default_vfio_cfg->vfio_enabled = 0` and
> `default_vfio_cfg->vfio_container_fd = -1`, socket error will
> be set in DPDK primary process when it sync this info to
> the secondary process.
> This patch use to skip this kind of useless detection for this
> scenario.
> 
> Signed-off-by: Yang Ming <ming.1.yang@nokia-sbell.com>
> ---
>  lib/eal/linux/eal_vfio.c | 11 +++++++++++
>  1 file changed, 11 insertions(+)
> 
> diff --git a/lib/eal/linux/eal_vfio.c b/lib/eal/linux/eal_vfio.c
> index 7132e24cba..1679d29263 100644
> --- a/lib/eal/linux/eal_vfio.c
> +++ b/lib/eal/linux/eal_vfio.c
> @@ -7,6 +7,7 @@
>  #include <fcntl.h>
>  #include <unistd.h>
>  #include <sys/ioctl.h>
> +#include <dirent.h>
>  
>  #include <rte_errno.h>
>  #include <rte_log.h>
> @@ -1083,6 +1084,7 @@ rte_vfio_enable(const char *modname)
>  	/* initialize group list */
>  	int i, j;
>  	int vfio_available;
> +	DIR *dir;
>  	const struct internal_config *internal_conf =
>  		eal_get_internal_configuration();
>  
> @@ -1119,6 +1121,15 @@ rte_vfio_enable(const char *modname)
>  		return 0;
>  	}
>  
> +	/* return 0 if VFIO directory not exist for container with non-privileged mode */
> +	dir = opendir(VFIO_DIR);
> +	if (dir == NULL) {
> +		EAL_LOG(DEBUG,
> +			"VFIO directory not exist, skipping VFIO support...");
> +		return 0;
> +	}
> +	closedir(dir);

You need to test the non-container cases.
If vfio is loaded /dev/vfio is a character device (not a directory)

Also looks suspicious that VFIO_DIR is defined but never used currently.

Re: [PATCH] Skip vfio in the scenario of non-privileged mode

[Yang Ming]

On 2025/1/18 00:47, Stephen Hemminger wrote:
> Caution: This is an external email. Please be very careful when clicking links or opening attachments. See http://nok.it/nsb for additional information.
>
> On Fri, 17 Jan 2025 15:28:47 +0800
> Yang Ming <ming.1.yang@nokia-sbell.com> wrote:
>
>> DPDK detect vfio container according the existence of vfio
>> module. But for container with non-privileged mode, there is
>> possibility that no VFIO_DIR(/dev/vfio) mapping from host to
>> container when host have both Intel NIC and Mellanox NIC but
>> this conntainer only allocate VFs from Mellanox NIC.
>> In this case, vfio kernel module has already been loaded from
>> the host.
>> This scenario will cause the error log occurs in DPDK primary
>> process as below:
>> 'EAL:   cannot open VFIO container, error 2 (No such file or
>> directory)'
>> 'EAL: VFIO support could not be initialized'
>> Because `rte_vfio_enable()` call `rte_vfio_get_container_fd()`
>> to execute `vfio_container_fd = open(VFIO_CONTAINER_PATH,
>> O_RDWR);` but VFIO_CONTAINER_PATH(/dev/vfio/vfio) doesn't exist
>> in this container.
>> This scenario will also lead to the delay of DPDK secondary
>> process because `default_vfio_cfg->vfio_enabled = 0` and
>> `default_vfio_cfg->vfio_container_fd = -1`, socket error will
>> be set in DPDK primary process when it sync this info to
>> the secondary process.
>> This patch use to skip this kind of useless detection for this
>> scenario.
>>
>> Signed-off-by: Yang Ming <ming.1.yang@nokia-sbell.com>
>> ---
>>   lib/eal/linux/eal_vfio.c | 11 +++++++++++
>>   1 file changed, 11 insertions(+)
>>
>> diff --git a/lib/eal/linux/eal_vfio.c b/lib/eal/linux/eal_vfio.c
>> index 7132e24cba..1679d29263 100644
>> --- a/lib/eal/linux/eal_vfio.c
>> +++ b/lib/eal/linux/eal_vfio.c
>> @@ -7,6 +7,7 @@
>>   #include <fcntl.h>
>>   #include <unistd.h>
>>   #include <sys/ioctl.h>
>> +#include <dirent.h>
>>   
>>   #include <rte_errno.h>
>>   #include <rte_log.h>
>> @@ -1083,6 +1084,7 @@ rte_vfio_enable(const char *modname)
>>   	/* initialize group list */
>>   	int i, j;
>>   	int vfio_available;
>> +	DIR *dir;
>>   	const struct internal_config *internal_conf =
>>   		eal_get_internal_configuration();
>>   
>> @@ -1119,6 +1121,15 @@ rte_vfio_enable(const char *modname)
>>   		return 0;
>>   	}
>>   
>> +	/* return 0 if VFIO directory not exist for container with non-privileged mode */
>> +	dir = opendir(VFIO_DIR);
>> +	if (dir == NULL) {
>> +		EAL_LOG(DEBUG,
>> +			"VFIO directory not exist, skipping VFIO support...");
>> +		return 0;
>> +	}
>> +	closedir(dir);
> You need to test the non-container cases.
> If vfio is loaded /dev/vfio is a character device (not a directory)
>
> Also looks suspicious that VFIO_DIR is defined but never used currently.
>
Hi Stephen,
For non-container test, /dev/vfio/vfio will be character device, not 
/dev/vfio.
Here is the command result on my testing environment with Intel NIC.

[root@computer-1 testuser]# ls -l /dev/vfio
total 0
crw-rw-rw-. 1 root root 10, 196 Jan 22 01:50 vfio
[root@computer-1 testuser]# dpdk-devbind.py -b vfio-pci 0000:04:10.2
[root@computer-1 testuser]# ls -l /dev/vfio
total 0
crw-------. 1 root root 239,   0 Jan 22 01:52 59
crw-rw-rw-. 1 root root  10, 196 Jan 22 01:50 vfio
[root@computer-1 testuser]# dpdk-devbind.py -b ixgbevf 0000:04:10.2
[root@computer-1 testuser]# ls -l /dev/vfio
total 0
crw-rw-rw-. 1 root root 10, 196 Jan 22 01:50 vfio

Can you confirm your test scenario?

Re: [PATCH] Skip vfio in the scenario of non-privileged mode

[Yang Ming]

On 2025/1/22 16:15, Yang Ming wrote:
> Hi Stephen

Hi Stephen,

Could you please confirm the comment above?

Brs,

Yang Ming

Re: [PATCH] Skip vfio in the scenario of non-privileged mode

[Stephen Hemminger]

On Wed, 22 Jan 2025 16:15:03 +0800
Yang Ming <ming.1.yang@nokia-sbell.com> wrote:

> On 2025/1/18 00:47, Stephen Hemminger wrote:
> > Caution: This is an external email. Please be very careful when clicking links or opening attachments. See http://nok.it/nsb for additional information.
> >
> > On Fri, 17 Jan 2025 15:28:47 +0800
> > Yang Ming <ming.1.yang@nokia-sbell.com> wrote:
> >  
> >> DPDK detect vfio container according the existence of vfio
> >> module. But for container with non-privileged mode, there is
> >> possibility that no VFIO_DIR(/dev/vfio) mapping from host to
> >> container when host have both Intel NIC and Mellanox NIC but
> >> this conntainer only allocate VFs from Mellanox NIC.
> >> In this case, vfio kernel module has already been loaded from
> >> the host.
> >> This scenario will cause the error log occurs in DPDK primary
> >> process as below:
> >> 'EAL:   cannot open VFIO container, error 2 (No such file or
> >> directory)'
> >> 'EAL: VFIO support could not be initialized'
> >> Because `rte_vfio_enable()` call `rte_vfio_get_container_fd()`
> >> to execute `vfio_container_fd = open(VFIO_CONTAINER_PATH,
> >> O_RDWR);` but VFIO_CONTAINER_PATH(/dev/vfio/vfio) doesn't exist
> >> in this container.
> >> This scenario will also lead to the delay of DPDK secondary
> >> process because `default_vfio_cfg->vfio_enabled = 0` and
> >> `default_vfio_cfg->vfio_container_fd = -1`, socket error will
> >> be set in DPDK primary process when it sync this info to
> >> the secondary process.
> >> This patch use to skip this kind of useless detection for this
> >> scenario.
> >>
> >> Signed-off-by: Yang Ming <ming.1.yang@nokia-sbell.com>
> >> ---
> >>   lib/eal/linux/eal_vfio.c | 11 +++++++++++
> >>   1 file changed, 11 insertions(+)
> >>
> >> diff --git a/lib/eal/linux/eal_vfio.c b/lib/eal/linux/eal_vfio.c
> >> index 7132e24cba..1679d29263 100644
> >> --- a/lib/eal/linux/eal_vfio.c
> >> +++ b/lib/eal/linux/eal_vfio.c
> >> @@ -7,6 +7,7 @@
> >>   #include <fcntl.h>
> >>   #include <unistd.h>
> >>   #include <sys/ioctl.h>
> >> +#include <dirent.h>
> >>   
> >>   #include <rte_errno.h>
> >>   #include <rte_log.h>
> >> @@ -1083,6 +1084,7 @@ rte_vfio_enable(const char *modname)
> >>   	/* initialize group list */
> >>   	int i, j;
> >>   	int vfio_available;
> >> +	DIR *dir;
> >>   	const struct internal_config *internal_conf =
> >>   		eal_get_internal_configuration();
> >>   
> >> @@ -1119,6 +1121,15 @@ rte_vfio_enable(const char *modname)
> >>   		return 0;
> >>   	}
> >>   
> >> +	/* return 0 if VFIO directory not exist for container with non-privileged mode */
> >> +	dir = opendir(VFIO_DIR);
> >> +	if (dir == NULL) {
> >> +		EAL_LOG(DEBUG,
> >> +			"VFIO directory not exist, skipping VFIO support...");
> >> +		return 0;
> >> +	}
> >> +	closedir(dir);  
> > You need to test the non-container cases.
> > If vfio is loaded /dev/vfio is a character device (not a directory)
> >
> > Also looks suspicious that VFIO_DIR is defined but never used currently.
> >  
> Hi Stephen,
> For non-container test, /dev/vfio/vfio will be character device, not 
> /dev/vfio.
> Here is the command result on my testing environment with Intel NIC.
> 
> [root@computer-1 testuser]# ls -l /dev/vfio
> total 0
> crw-rw-rw-. 1 root root 10, 196 Jan 22 01:50 vfio
> [root@computer-1 testuser]# dpdk-devbind.py -b vfio-pci 0000:04:10.2
> [root@computer-1 testuser]# ls -l /dev/vfio
> total 0
> crw-------. 1 root root 239,   0 Jan 22 01:52 59
> crw-rw-rw-. 1 root root  10, 196 Jan 22 01:50 vfio
> [root@computer-1 testuser]# dpdk-devbind.py -b ixgbevf 0000:04:10.2
> [root@computer-1 testuser]# ls -l /dev/vfio
> total 0
> crw-rw-rw-. 1 root root 10, 196 Jan 22 01:50 vfio
> 
> Can you confirm your test scenario?
> 
> 

When vfio-pci is loaded but no device bound:
$ ls -l /dev/vfio
total 0
crw-rw-rw- 1 root root 10, 196 Feb 26 05:39 vfio

After binding device
$ ls -l /dev/vfio
total 0
crw------- 1 root root 511,   0 Feb 26 05:42 15
crw-rw-rw- 1 root root  10, 196 Feb 26 05:39 vfio

So testing for /dev/vfio is good indication that module is loaded.
Not sure what I was thinking earlier.

Re: [PATCH] Skip vfio in the scenario of non-privileged mode

[Yang Ming]

On 2025/2/26 21:45, Stephen Hemminger wrote:
> Caution: This is an external email. Please be very careful when clicking links or opening attachments. See http://nok.it/nsb for additional information.
>
> On Wed, 22 Jan 2025 16:15:03 +0800
> Yang Ming <ming.1.yang@nokia-sbell.com> wrote:
>
>> On 2025/1/18 00:47, Stephen Hemminger wrote:
>>> Caution: This is an external email. Please be very careful when clicking links or opening attachments. See http://nok.it/nsb for additional information.
>>>
>>> On Fri, 17 Jan 2025 15:28:47 +0800
>>> Yang Ming <ming.1.yang@nokia-sbell.com> wrote:
>>>   
>>>> DPDK detect vfio container according the existence of vfio
>>>> module. But for container with non-privileged mode, there is
>>>> possibility that no VFIO_DIR(/dev/vfio) mapping from host to
>>>> container when host have both Intel NIC and Mellanox NIC but
>>>> this conntainer only allocate VFs from Mellanox NIC.
>>>> In this case, vfio kernel module has already been loaded from
>>>> the host.
>>>> This scenario will cause the error log occurs in DPDK primary
>>>> process as below:
>>>> 'EAL:   cannot open VFIO container, error 2 (No such file or
>>>> directory)'
>>>> 'EAL: VFIO support could not be initialized'
>>>> Because `rte_vfio_enable()` call `rte_vfio_get_container_fd()`
>>>> to execute `vfio_container_fd = open(VFIO_CONTAINER_PATH,
>>>> O_RDWR);` but VFIO_CONTAINER_PATH(/dev/vfio/vfio) doesn't exist
>>>> in this container.
>>>> This scenario will also lead to the delay of DPDK secondary
>>>> process because `default_vfio_cfg->vfio_enabled = 0` and
>>>> `default_vfio_cfg->vfio_container_fd = -1`, socket error will
>>>> be set in DPDK primary process when it sync this info to
>>>> the secondary process.
>>>> This patch use to skip this kind of useless detection for this
>>>> scenario.
>>>>
>>>> Signed-off-by: Yang Ming <ming.1.yang@nokia-sbell.com>
>>>> ---
>>>>    lib/eal/linux/eal_vfio.c | 11 +++++++++++
>>>>    1 file changed, 11 insertions(+)
>>>>
>>>> diff --git a/lib/eal/linux/eal_vfio.c b/lib/eal/linux/eal_vfio.c
>>>> index 7132e24cba..1679d29263 100644
>>>> --- a/lib/eal/linux/eal_vfio.c
>>>> +++ b/lib/eal/linux/eal_vfio.c
>>>> @@ -7,6 +7,7 @@
>>>>    #include <fcntl.h>
>>>>    #include <unistd.h>
>>>>    #include <sys/ioctl.h>
>>>> +#include <dirent.h>
>>>>    
>>>>    #include <rte_errno.h>
>>>>    #include <rte_log.h>
>>>> @@ -1083,6 +1084,7 @@ rte_vfio_enable(const char *modname)
>>>>    	/* initialize group list */
>>>>    	int i, j;
>>>>    	int vfio_available;
>>>> +	DIR *dir;
>>>>    	const struct internal_config *internal_conf =
>>>>    		eal_get_internal_configuration();
>>>>    
>>>> @@ -1119,6 +1121,15 @@ rte_vfio_enable(const char *modname)
>>>>    		return 0;
>>>>    	}
>>>>    
>>>> +	/* return 0 if VFIO directory not exist for container with non-privileged mode */
>>>> +	dir = opendir(VFIO_DIR);
>>>> +	if (dir == NULL) {
>>>> +		EAL_LOG(DEBUG,
>>>> +			"VFIO directory not exist, skipping VFIO support...");
>>>> +		return 0;
>>>> +	}
>>>> +	closedir(dir);
>>> You need to test the non-container cases.
>>> If vfio is loaded /dev/vfio is a character device (not a directory)
>>>
>>> Also looks suspicious that VFIO_DIR is defined but never used currently.
>>>   
>> Hi Stephen,
>> For non-container test, /dev/vfio/vfio will be character device, not
>> /dev/vfio.
>> Here is the command result on my testing environment with Intel NIC.
>>
>> [root@computer-1 testuser]# ls -l /dev/vfio
>> total 0
>> crw-rw-rw-. 1 root root 10, 196 Jan 22 01:50 vfio
>> [root@computer-1 testuser]# dpdk-devbind.py -b vfio-pci 0000:04:10.2
>> [root@computer-1 testuser]# ls -l /dev/vfio
>> total 0
>> crw-------. 1 root root 239,   0 Jan 22 01:52 59
>> crw-rw-rw-. 1 root root  10, 196 Jan 22 01:50 vfio
>> [root@computer-1 testuser]# dpdk-devbind.py -b ixgbevf 0000:04:10.2
>> [root@computer-1 testuser]# ls -l /dev/vfio
>> total 0
>> crw-rw-rw-. 1 root root 10, 196 Jan 22 01:50 vfio
>>
>> Can you confirm your test scenario?
>>
>>
> When vfio-pci is loaded but no device bound:
> $ ls -l /dev/vfio
> total 0
> crw-rw-rw- 1 root root 10, 196 Feb 26 05:39 vfio
>
> After binding device
> $ ls -l /dev/vfio
> total 0
> crw------- 1 root root 511,   0 Feb 26 05:42 15
> crw-rw-rw- 1 root root  10, 196 Feb 26 05:39 vfio
>
> So testing for /dev/vfio is good indication that module is loaded.
> Not sure what I was thinking earlier.
>
>
>
>
Hi Stephen,

Thank you very much for your explanation. It's very clear.
Can you help to accept this patch, or we need more comments?

Brs,
Yang Ming

[PATCH v2] eal/linux: skip vfio for non-privileged container

[Yang Ming]

DPDK detect vfio container according the existence of vfio
module. But for container with non-privileged mode, there is
possibility that no VFIO_DIR(/dev/vfio) mapping from host to
container when host have both Intel NIC and Mellanox NIC but
this conntainer only allocate VFs from Mellanox NIC.
In this case, vfio kernel module has already been loaded from
the host.
This scenario will cause the error log occurs in DPDK primary
process as below:
'EAL:   cannot open VFIO container, error 2 (No such file or
directory)'
'EAL: VFIO support could not be initialized'
Because `rte_vfio_enable()` call `rte_vfio_get_container_fd()`
to execute `vfio_container_fd = open(VFIO_CONTAINER_PATH,
O_RDWR);` but VFIO_CONTAINER_PATH(/dev/vfio/vfio) doesn't exist
in this container.
This scenario will also lead to the delay of DPDK secondary
process because `default_vfio_cfg->vfio_enabled = 0` and
`default_vfio_cfg->vfio_container_fd = -1`, socket error will
be set in DPDK primary process when it sync this info to
the secondary process.
This patch use to skip this kind of useless detection for this
scenario.

Signed-off-by: Yang Ming <ming.1.yang@nokia-sbell.com>
---
 lib/eal/linux/eal_vfio.c | 11 +++++++++++
 1 file changed, 11 insertions(+)

diff --git a/lib/eal/linux/eal_vfio.c b/lib/eal/linux/eal_vfio.c
index 7132e24cba..1679d29263 100644
--- a/lib/eal/linux/eal_vfio.c
+++ b/lib/eal/linux/eal_vfio.c
@@ -7,6 +7,7 @@
 #include <fcntl.h>
 #include <unistd.h>
 #include <sys/ioctl.h>
+#include <dirent.h>
 
 #include <rte_errno.h>
 #include <rte_log.h>
@@ -1083,6 +1084,7 @@ rte_vfio_enable(const char *modname)
 	/* initialize group list */
 	int i, j;
 	int vfio_available;
+	DIR *dir;
 	const struct internal_config *internal_conf =
 		eal_get_internal_configuration();
 
@@ -1119,6 +1121,15 @@ rte_vfio_enable(const char *modname)
 		return 0;
 	}
 
+	/* return 0 if VFIO directory not exist for container with non-privileged mode */
+	dir = opendir(VFIO_DIR);
+	if (dir == NULL) {
+		EAL_LOG(DEBUG,
+			"VFIO directory not exist, skipping VFIO support...");
+		return 0;
+	}
+	closedir(dir);
+
 	if (internal_conf->process_type == RTE_PROC_PRIMARY) {
 		if (vfio_mp_sync_setup() == -1) {
 			default_vfio_cfg->vfio_container_fd = -1;
-- 
2.34.1

Re: [PATCH v2] eal/linux: skip vfio for non-privileged container

[David Marchand]

On Thu, Mar 27, 2025 at 8:57 AM Yang Ming <ming.1.yang@nokia-sbell.com> wrote:
>
> DPDK detect vfio container according the existence of vfio
> module. But for container with non-privileged mode, there is
> possibility that no VFIO_DIR(/dev/vfio) mapping from host to
> container when host have both Intel NIC and Mellanox NIC but
> this conntainer only allocate VFs from Mellanox NIC.
> In this case, vfio kernel module has already been loaded from
> the host.
> This scenario will cause the error log occurs in DPDK primary
> process as below:
> 'EAL:   cannot open VFIO container, error 2 (No such file or
> directory)'
> 'EAL: VFIO support could not be initialized'
> Because `rte_vfio_enable()` call `rte_vfio_get_container_fd()`
> to execute `vfio_container_fd = open(VFIO_CONTAINER_PATH,
> O_RDWR);` but VFIO_CONTAINER_PATH(/dev/vfio/vfio) doesn't exist
> in this container.
> This scenario will also lead to the delay of DPDK secondary
> process because `default_vfio_cfg->vfio_enabled = 0` and
> `default_vfio_cfg->vfio_container_fd = -1`, socket error will
> be set in DPDK primary process when it sync this info to
> the secondary process.
> This patch use to skip this kind of useless detection for this
> scenario.
>
> Signed-off-by: Yang Ming <ming.1.yang@nokia-sbell.com>

With such a change, is the check on the passed kernel module still needed?


-- 
David Marchand

Re: [PATCH v2] eal/linux: skip vfio for non-privileged container

[Moses Young]

[-- Attachment #1: Type: text/plain, Size: 1842 bytes --]

On 5/16/2025 2:46 PM, David Marchand wrote:
> On Thu, Mar 27, 2025 at 8:57 AM Yang Ming<ming.1.yang@nokia-sbell.com> wrote:
>> DPDK detect vfio container according the existence of vfio
>> module. But for container with non-privileged mode, there is
>> possibility that no VFIO_DIR(/dev/vfio) mapping from host to
>> container when host have both Intel NIC and Mellanox NIC but
>> this conntainer only allocate VFs from Mellanox NIC.
>> In this case, vfio kernel module has already been loaded from
>> the host.
>> This scenario will cause the error log occurs in DPDK primary
>> process as below:
>> 'EAL:   cannot open VFIO container, error 2 (No such file or
>> directory)'
>> 'EAL: VFIO support could not be initialized'
>> Because `rte_vfio_enable()` call `rte_vfio_get_container_fd()`
>> to execute `vfio_container_fd = open(VFIO_CONTAINER_PATH,
>> O_RDWR);` but VFIO_CONTAINER_PATH(/dev/vfio/vfio) doesn't exist
>> in this container.
>> This scenario will also lead to the delay of DPDK secondary
>> process because `default_vfio_cfg->vfio_enabled = 0` and
>> `default_vfio_cfg->vfio_container_fd = -1`, socket error will
>> be set in DPDK primary process when it sync this info to
>> the secondary process.
>> This patch use to skip this kind of useless detection for this
>> scenario.
>>
>> Signed-off-by: Yang Ming<ming.1.yang@nokia-sbell.com>
> With such a change, is the check on the passed kernel module still needed?

Hi David,

Thanks for your comment.

Yes. We still need this checks:
1. Module check (rte_eal_check_module(modname)) ensures the host has the 
VFIO driver loaded.
2. Directory check (opendir("/dev/vfio")) skips the open call in 
unprivileged containers without /dev/vfio, avoiding a noisy error.
This patch adds the second check. Please let me know if you'd like any 
more details!

Best regards,
Yang Ming

[-- Attachment #2: Type: text/html, Size: 3373 bytes --]

Re: [PATCH v2] eal/linux: skip vfio for non-privileged container

[Burakov, Anatoly]

On 3/27/2025 8:57 AM, Yang Ming wrote:
> DPDK detect vfio container according the existence of vfio
> module. But for container with non-privileged mode, there is
> possibility that no VFIO_DIR(/dev/vfio) mapping from host to
> container when host have both Intel NIC and Mellanox NIC but
> this conntainer only allocate VFs from Mellanox NIC.
> In this case, vfio kernel module has already been loaded from
> the host.
> This scenario will cause the error log occurs in DPDK primary
> process as below:
> 'EAL:   cannot open VFIO container, error 2 (No such file or
> directory)'
> 'EAL: VFIO support could not be initialized'
> Because `rte_vfio_enable()` call `rte_vfio_get_container_fd()`
> to execute `vfio_container_fd = open(VFIO_CONTAINER_PATH,
> O_RDWR);` but VFIO_CONTAINER_PATH(/dev/vfio/vfio) doesn't exist
> in this container.
> This scenario will also lead to the delay of DPDK secondary
> process because `default_vfio_cfg->vfio_enabled = 0` and
> `default_vfio_cfg->vfio_container_fd = -1`, socket error will
> be set in DPDK primary process when it sync this info to
> the secondary process.
> This patch use to skip this kind of useless detection for this
> scenario.
> 
> Signed-off-by: Yang Ming <ming.1.yang@nokia-sbell.com>

With a few code grammar fixes below,

Acked-by: Anatoly Burakov <anatoly.burakov@intel.com>

> ---
>   lib/eal/linux/eal_vfio.c | 11 +++++++++++
>   1 file changed, 11 insertions(+)
> 
> diff --git a/lib/eal/linux/eal_vfio.c b/lib/eal/linux/eal_vfio.c
> index 7132e24cba..1679d29263 100644
> --- a/lib/eal/linux/eal_vfio.c
> +++ b/lib/eal/linux/eal_vfio.c
> @@ -7,6 +7,7 @@
>   #include <fcntl.h>
>   #include <unistd.h>
>   #include <sys/ioctl.h>
> +#include <dirent.h>
>   
>   #include <rte_errno.h>
>   #include <rte_log.h>
> @@ -1083,6 +1084,7 @@ rte_vfio_enable(const char *modname)
>   	/* initialize group list */
>   	int i, j;
>   	int vfio_available;
> +	DIR *dir;
>   	const struct internal_config *internal_conf =
>   		eal_get_internal_configuration();
>   
> @@ -1119,6 +1121,15 @@ rte_vfio_enable(const char *modname)
>   		return 0;
>   	}
>   
> +	/* return 0 if VFIO directory not exist for container with non-privileged mode */

/* VFIO directory might not exist (e.g. unprivileged containers) */

> +	dir = opendir(VFIO_DIR);
> +	if (dir == NULL) {
> +		EAL_LOG(DEBUG,

"VFIO directory does not exist, skipping VFIO support..."

> +			"VFIO directory not exist, skipping VFIO support...");
> +		return 0;
> +	}
> +	closedir(dir);
> +
>   	if (internal_conf->process_type == RTE_PROC_PRIMARY) {
>   		if (vfio_mp_sync_setup() == -1) {
>   			default_vfio_cfg->vfio_container_fd = -1;


-- 
Thanks,
Anatoly