[PATCH v1 1/1] usertools/devbind: fix NUMA node display

[PATCH v1 1/1] usertools/devbind: fix NUMA node display

[Anatoly Burakov]

On some systems, even though NUMA nodes may be present in sysfs, the lspci
command will not have NUMANode keys in them, which will cause an exception.
Fix to check if NUMANode keys are available in lspci output before enabling
NUMA node output.

Fixes: a7d69cef8f20 ("usertools/devbind: print device NUMA node")

Signed-off-by: Anatoly Burakov <anatoly.burakov@intel.com>
---
 usertools/dpdk-devbind.py | 11 ++++++++++-
 1 file changed, 10 insertions(+), 1 deletion(-)

diff --git a/usertools/dpdk-devbind.py b/usertools/dpdk-devbind.py
index 34f8f3ed3b..8a4aac371c 100755
--- a/usertools/dpdk-devbind.py
+++ b/usertools/dpdk-devbind.py
@@ -113,7 +113,16 @@
 
 # check if this system has NUMA support
 def is_numa():
-    return os.path.exists('/sys/devices/system/node')
+    if not os.path.exists("/sys/devices/system/node"):
+        return False
+    # occasionally, system may report NUMA support but lspci will not, so we
+    # want to go through all devices and see if any of them do not have NUMANode
+    # property - this will mean it is not safe to try to access it
+    for device_dict in devices.values():
+        if "NUMANode" not in device_dict:
+            return False
+    # all checks passed
+    return True
 
 
 # check if a specific kernel module is loaded
-- 
2.43.5

Re: [PATCH v1 1/1] usertools/devbind: fix NUMA node display

[Stephen Hemminger]

On Thu, 28 Nov 2024 16:08:55 +0000
Anatoly Burakov <anatoly.burakov@intel.com> wrote:

> +    # occasionally, system may report NUMA support but lspci will not, so we
> +    # want to go through all devices and see if any of them do not have NUMANode
> +    # property - this will mean it is not safe to try to access it
> +    for device_dict in devices.values():
> +        if "NUMANode" not in device_dict:
> +            return False

Any indication as to why this happens, what kernel, what device?

Re: [PATCH v1 1/1] usertools/devbind: fix NUMA node display

[Burakov, Anatoly]

On 11/28/2024 6:20 PM, Stephen Hemminger wrote:
> On Thu, 28 Nov 2024 16:08:55 +0000
> Anatoly Burakov <anatoly.burakov@intel.com> wrote:
> 
>> +    # occasionally, system may report NUMA support but lspci will not, so we
>> +    # want to go through all devices and see if any of them do not have NUMANode
>> +    # property - this will mean it is not safe to try to access it
>> +    for device_dict in devices.values():
>> +        if "NUMANode" not in device_dict:
>> +            return False
> 
> Any indication as to why this happens, what kernel, what device?

We've had internal validation team report this happenning on VM's, 
particularly ESXi. I did not dig deeper as to what particular 
configurations cause this to happen, I didn't have a chance to reproduce 
this myself.

-- 
Thanks,
Anatoly

Re: [PATCH v1 1/1] usertools/devbind: fix NUMA node display

[Burakov, Anatoly]

On 11/29/2024 10:12 AM, Burakov, Anatoly wrote:
> On 11/28/2024 6:20 PM, Stephen Hemminger wrote:
>> On Thu, 28 Nov 2024 16:08:55 +0000
>> Anatoly Burakov <anatoly.burakov@intel.com> wrote:
>>
>>> +    # occasionally, system may report NUMA support but lspci will 
>>> not, so we
>>> +    # want to go through all devices and see if any of them do not 
>>> have NUMANode
>>> +    # property - this will mean it is not safe to try to access it
>>> +    for device_dict in devices.values():
>>> +        if "NUMANode" not in device_dict:
>>> +            return False
>>
>> Any indication as to why this happens, what kernel, what device?
> 
> We've had internal validation team report this happenning on VM's, 
> particularly ESXi. I did not dig deeper as to what particular 
> configurations cause this to happen, I didn't have a chance to reproduce 
> this myself.
> 

I just reproduced this on my machine by disabling NUMA support in BIOS. 
I can confirm that /sys/devices/system/node exists in sysfs but lspci 
will not report NUMA node in that case. So, it probably applies to all 
kernels.

-- 
Thanks,
Anatoly

Re: [PATCH v1 1/1] usertools/devbind: fix NUMA node display

[Robin Jarry]

Anatoly Burakov, Nov 28, 2024 at 17:08:
> On some systems, even though NUMA nodes may be present in sysfs, the lspci
> command will not have NUMANode keys in them, which will cause an exception.
> Fix to check if NUMANode keys are available in lspci output before enabling
> NUMA node output.
>
> Fixes: a7d69cef8f20 ("usertools/devbind: print device NUMA node")
>
> Signed-off-by: Anatoly Burakov <anatoly.burakov@intel.com>

Acked-by: Robin Jarry <rjarry@redhat.com>

[PATCH v1 1/1] usertools/devbind: fix NUMA node display

[Anatoly Burakov]

On some systems (particularly ones with NUMA disabled in BIOS), even
though NUMA nodes may be present in sysfs, the lspci command will not
have NUMANode keys in them, which will cause an exception. Fix to check
if NUMANode keys are available in lspci output before enabling NUMA node
output.

Fixes: a7d69cef8f20 ("usertools/devbind: print device NUMA node")

Signed-off-by: Anatoly Burakov <anatoly.burakov@intel.com>
---

Notes:
    This is an alternative fix to patch 34067 [1]
    
    The difference between this and the original patch is that we drop the
    check for sysfs node, because it does not give us any useful information,
    and instead rely just on lspci dictionary to let us know if it's safe to
    enable NUMA node display.
    
    [1] https://patches.dpdk.org/project/dpdk/patch/9af1231398c4ba116d3b89164690feace37293a9.1732810125.git.anatoly.burakov@intel.com/

 usertools/dpdk-devbind.py | 10 +++-------
 1 file changed, 3 insertions(+), 7 deletions(-)

diff --git a/usertools/dpdk-devbind.py b/usertools/dpdk-devbind.py
index 34f8f3ed3b..283707fc16 100755
--- a/usertools/dpdk-devbind.py
+++ b/usertools/dpdk-devbind.py
@@ -111,11 +111,6 @@
 args = []
 
 
-# check if this system has NUMA support
-def is_numa():
-    return os.path.exists('/sys/devices/system/node')
-
-
 # check if a specific kernel module is loaded
 def module_is_loaded(module):
     global loaded_modules
@@ -595,9 +590,12 @@ def show_device_status(devices_type, device_name, if_field=False):
     dpdk_drv = []
     no_drv = []
 
+    print_numa = True  # by default, assume we can print NUMA information
+
     # split our list of network devices into the three categories above
     for d in devices.keys():
         if device_type_match(devices[d], devices_type):
+            print_numa &= "NUMANode" in devices[d]
             if not has_driver(d):
                 no_drv.append(devices[d])
                 continue
@@ -616,8 +614,6 @@ def show_device_status(devices_type, device_name, if_field=False):
         print("".join('=' * len(msg)))
         return
 
-    print_numa = is_numa()
-
     # print each category separately, so we can clearly see what's used by DPDK
     if dpdk_drv:
         extra_param = "drv=%(Driver_str)s unused=%(Module_str)s"
-- 
2.43.5