From mboxrd@z Thu Jan  1 00:00:00 1970
Return-Path: <dev-bounces@dpdk.org>
Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124])
	by inbox.dpdk.org (Postfix) with ESMTP id 21A9845823;
	Tue, 20 Aug 2024 17:35:31 +0200 (CEST)
Received: from mails.dpdk.org (localhost [127.0.0.1])
	by mails.dpdk.org (Postfix) with ESMTP id 33D8840BA5;
	Tue, 20 Aug 2024 17:35:26 +0200 (CEST)
Received: from mgamail.intel.com (mgamail.intel.com [192.198.163.7])
 by mails.dpdk.org (Postfix) with ESMTP id 57BE8400D6
 for <dev@dpdk.org>; Tue, 20 Aug 2024 17:35:23 +0200 (CEST)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple;
 d=intel.com; i=@intel.com; q=dns/txt; s=Intel;
 t=1724168124; x=1755704124;
 h=from:to:cc:subject:date:message-id:in-reply-to:
 references:mime-version:content-transfer-encoding;
 bh=jQWQgJfDJsVTC8gNOPu9uxLUZ5aTVeiUewDqIDtLrM4=;
 b=bJRmCU4xYwnMk8rozVxHNG7fXITZ0DakagxI8ugwMBYj+ZsLAuP3m97h
 RukE6Fk4sRnPhuC89ePzU48DGj4hKAA/xXkiuFWbYy8cluPHSLffWN7DM
 rDltiAVa07HGiZfBkpg7EKFadb2JeMDAl3J1K6T0iLCGYJjYWsKIS0Ssb
 OOmrxq4QPNR+030RpPKJK++9wY3u/uGSMsMcHAt4leGL72el3ctDseNGY
 HdNRA2tDgeegGUo/B+jfr+Xrhhvcojlfes2QdJfZjdkmAzowUBuOefVR3
 zn7b2IVY//5JsXdgjUOMWL0Y5S4QxVmehfzkdnVyCtbSOv/x7Pa3FIJmA w==;
X-CSE-ConnectionGUID: YNtqsB3TR6WhovyJXMhXiw==
X-CSE-MsgGUID: o+UZadSaT3ieQyXYCjY+EA==
X-IronPort-AV: E=McAfee;i="6700,10204,11170"; a="47872778"
X-IronPort-AV: E=Sophos;i="6.10,162,1719903600"; d="scan'208";a="47872778"
Received: from orviesa006.jf.intel.com ([10.64.159.146])
 by fmvoesa101.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384;
 20 Aug 2024 08:35:23 -0700
X-CSE-ConnectionGUID: g7ZPpkkVQta3Wh1FEZr9dw==
X-CSE-MsgGUID: ZLoH1+g4SUOMuNmPg5e8+w==
X-ExtLoop1: 1
X-IronPort-AV: E=Sophos;i="6.10,162,1719903600"; d="scan'208";a="61074478"
Received: from silpixa00401119.ir.intel.com ([10.55.129.167])
 by orviesa006.jf.intel.com with ESMTP; 20 Aug 2024 08:35:22 -0700
From: Anatoly Burakov <anatoly.burakov@intel.com>
To: dev@dpdk.org,
	Robin Jarry <rjarry@redhat.com>
Cc: bruce.richardson@intel.com
Subject: [PATCH v3 2/4] usertools/cpu_layout: print out NUMA nodes
Date: Tue, 20 Aug 2024 16:35:15 +0100
Message-ID: <2e995fbdd90061cdee290262043b469213566bc4.1724168117.git.anatoly.burakov@intel.com>
X-Mailer: git-send-email 2.43.5
In-Reply-To: <e55217b1dccdb22c87dc7dd32c05b527ec527591.1724168117.git.anatoly.burakov@intel.com>
References: <e51d00fea7c535404dda4d63e483639b13c48b2d.1723634354.git.anatoly.burakov@intel.com>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit
X-BeenThere: dev@dpdk.org
X-Mailman-Version: 2.1.29
Precedence: list
List-Id: DPDK patches and discussions <dev.dpdk.org>
List-Unsubscribe: <https://mails.dpdk.org/options/dev>,
 <mailto:dev-request@dpdk.org?subject=unsubscribe>
List-Archive: <http://mails.dpdk.org/archives/dev/>
List-Post: <mailto:dev@dpdk.org>
List-Help: <mailto:dev-request@dpdk.org?subject=help>
List-Subscribe: <https://mails.dpdk.org/listinfo/dev>,
 <mailto:dev-request@dpdk.org?subject=subscribe>
Errors-To: dev-bounces@dpdk.org

In traditional NUMA case, NUMA nodes and physical sockets were used
interchangeably, but there are cases where there can be multiple NUMA
nodes per socket, as well as all CPU's being assigned NUMA node 0 even in
cases of multiple sockets. Use sysfs to print out NUMA information.

Signed-off-by: Anatoly Burakov <anatoly.burakov@intel.com>
---

Notes:
    v2 -> v3:
    - Sort imports alphabetically

 usertools/cpu_layout.py | 36 +++++++++++++++++++++++++++++++-----
 1 file changed, 31 insertions(+), 5 deletions(-)

diff --git a/usertools/cpu_layout.py b/usertools/cpu_layout.py
index 1c255ff1a1..78b119d729 100755
--- a/usertools/cpu_layout.py
+++ b/usertools/cpu_layout.py
@@ -5,6 +5,7 @@
 
 """Display CPU topology information."""
 
+import glob
 import typing as T
 
 
@@ -29,12 +30,21 @@ def read_sysfs(path: str) -> str:
         return fd.read().strip()
 
 
+def read_numa_node(base: str) -> int:
+    """Read the NUMA node of a CPU."""
+    node_glob = f"{base}/node*"
+    node_dirs = glob.glob(node_glob)
+    if not node_dirs:
+        return 0  # default to node 0
+    return int(node_dirs[0].split("node")[1])
+
+
 def print_row(row: T.Tuple[str, ...], col_widths: T.List[int]) -> None:
     """Print a row of a table with the given column widths."""
     first, *rest = row
     w_first, *w_rest = col_widths
     first_end = " " * 4
-    rest_end = " " * 10
+    rest_end = " " * 4
 
     print(first.ljust(w_first), end=first_end)
     for cell, width in zip(rest, w_rest):
@@ -56,6 +66,7 @@ def main() -> None:
     sockets_s: T.Set[int] = set()
     cores_s: T.Set[int] = set()
     core_map: T.Dict[T.Tuple[int, int], T.List[int]] = {}
+    numa_map: T.Dict[int, int] = {}
     base_path = "/sys/devices/system/cpu"
 
     cpus = range_expand(read_sysfs(f"{base_path}/online"))
@@ -64,12 +75,14 @@ def main() -> None:
         lcore_base = f"{base_path}/cpu{cpu}"
         core = int(read_sysfs(f"{lcore_base}/topology/core_id"))
         socket = int(read_sysfs(f"{lcore_base}/topology/physical_package_id"))
+        node = read_numa_node(lcore_base)
 
         cores_s.add(core)
         sockets_s.add(socket)
         key = (socket, core)
         core_map.setdefault(key, [])
         core_map[key].append(cpu)
+        numa_map[cpu] = node
 
     cores = sorted(cores_s)
     sockets = sorted(sockets_s)
@@ -79,24 +92,37 @@ def main() -> None:
 
     print("cores = ", cores)
     print("sockets = ", sockets)
+    print("numa = ", sorted(set(numa_map.values())))
     print()
 
-    # Core, [Socket, Socket, ...]
-    heading_strs = "", *[f"Socket {s}" for s in sockets]
+    # Core, [NUMA, Socket, NUMA, Socket, ...]
+    heading_strs = "", *[v for s in sockets for v in ("", f"Socket {s}")]
     sep_strs = tuple("-" * len(hstr) for hstr in heading_strs)
     rows: T.List[T.Tuple[str, ...]] = []
 
+    prev_numa = None
     for c in cores:
         # Core,
         row: T.Tuple[str, ...] = (f"Core {c}",)
 
-        # [lcores, lcores, ...]
+        # assume NUMA changes symmetrically
+        first_lcore = core_map[(0, c)][0]
+        cur_numa = numa_map[first_lcore]
+        numa_changed = prev_numa != cur_numa
+        prev_numa = cur_numa
+
+        # [NUMA, lcores, NUMA, lcores, ...]
         for s in sockets:
             try:
                 lcores = core_map[(s, c)]
+                numa = numa_map[lcores[0]]
+                if numa_changed:
+                    row += (f"NUMA {numa}",)
+                else:
+                    row += ("",)
                 row += (str(lcores),)
             except KeyError:
-                row += ("",)
+                row += ("", "")
         rows += [row]
 
     # find max widths for each column, including header and rows
-- 
2.43.5