test suite reviews and discussions
 help / color / mirror / Atom feed
From: Thinh Tran <thinhtr@linux.vnet.ibm.com>
To: dts@dpdk.org
Cc: Thinh Tran <thinhtr@linux.vnet.ibm.com>
Subject: [dts] [DTS] [PATCH] Framwork: fix the first numa node may not be '0'
Date: Thu, 30 Sep 2021 14:41:40 +0000	[thread overview]
Message-ID: <20210930144140.673576-1-thinhtr@linux.vnet.ibm.com> (raw)

On certain Virtual Machine system  such as Power Virtual Machine, PowerVM, 
assigns the CPUs of its virtual machines (or Logical Partitions LPARs) to
NUMA nodes, and so the VM (or LPARs) 's CPUs may be on different ID
other than '0'. 
This patch should fix that.


Signed-off-by: Thinh Tran <thinhtr@linux.vnet.ibm.com>
---
 framework/crb.py | 13 ++++++-------
 framework/dut.py | 32 +++++++++++++++++++-------------
 2 files changed, 25 insertions(+), 20 deletions(-)

diff --git a/framework/crb.py b/framework/crb.py
index 40fe4fd1..9b0742ba 100644
--- a/framework/crb.py
+++ b/framework/crb.py
@@ -204,24 +204,23 @@ class Crb(object):
         else:
             return ''
 
-    def set_huge_pages(self, huge_pages, numa=-1):
+    def set_huge_pages(self, huge_pages, numa=""):
         """
         Set numbers of huge pages
         """
         page_size = self.send_expect("awk '/Hugepagesize/ {print $2}' /proc/meminfo", "# ")
 
-        if numa == -1:
+        if not numa:
             self.send_expect('echo %d > /sys/kernel/mm/hugepages/hugepages-%skB/nr_hugepages' % (huge_pages, page_size), '# ', 5)
         else:
-            # sometimes we set hugepage on kernel cmdline, so we clear all nodes' default hugepages at the first time.
-            if numa == 0:
-                self.send_expect('echo 0 > /sys/kernel/mm/hugepages/hugepages-%skB/nr_hugepages' % (page_size), '# ', 5)
+            # sometimes we set hugepage on kernel cmdline, so we clear it
+            self.send_expect('echo 0 > /sys/kernel/mm/hugepages/hugepages-%skB/nr_hugepages' % (page_size), '# ', 5)
 
             # some platform not support numa, example vm dut
             try:
-                self.send_expect('echo %d > /sys/devices/system/node/node%d/hugepages/hugepages-%skB/nr_hugepages' % (huge_pages, numa, page_size), '# ', 5)
+                self.send_expect('echo %d > /sys/devices/system/node/%s/hugepages/hugepages-%skB/nr_hugepages' % (huge_pages, numa, page_size), '# ', 5)
             except:
-                self.logger.warning("set %d hugepage on socket %d error" % (huge_pages, numa))
+                self.logger.warning("set %d hugepage on %s error" % (huge_pages, numa))
                 self.send_expect('echo %d > /sys/kernel/mm/hugepages/hugepages-%skB/nr_hugepages' % (huge_pages. page_size), '# ', 5)
 
     def set_speedup_options(self, read_cache, skip_setup):
diff --git a/framework/dut.py b/framework/dut.py
index f5481d06..1b85cb1c 100644
--- a/framework/dut.py
+++ b/framework/dut.py
@@ -547,16 +547,15 @@ class Dut(Crb):
             return
         hugepages_size = self.send_expect("awk '/Hugepagesize/ {print $2}' /proc/meminfo", "# ")
         total_huge_pages = self.get_total_huge_pages()
-        total_numa_nodes = self.send_expect("ls /sys/devices/system/node | grep node* | wc -l", "# ")
-        numa_service_num = self.get_def_rte_config('CONFIG_RTE_MAX_NUMA_NODES')
-        try:
-            int(total_numa_nodes)
-        except ValueError:
+        numa_nodes = self.send_expect("ls /sys/devices/system/node | grep node*", "# ")
+        if not numa_nodes:
             total_numa_nodes = -1
-        if numa_service_num is not None:
-            numa = min(int(total_numa_nodes), int(numa_service_num))
         else:
-            numa = total_numa_nodes
+            numa_nodes = numa_nodes.splitlines()
+            total_numa_nodes = len(numa_nodes)
+            self.logger.info(numa_nodes)
+
+
         force_socket = False
 
         if int(hugepages_size) < (1024 * 1024):
@@ -580,14 +579,21 @@ class Dut(Crb):
             if total_huge_pages != arch_huge_pages:
                 # before all hugepage average distribution  by all socket,
                 # but sometimes create mbuf pool on socket 0 failed when setup testpmd,
-                # so set all huge page on socket 0
+                # so set all huge page on first socket 
                 if force_socket:
-                    self.set_huge_pages(arch_huge_pages, 0)
+                    self.set_huge_pages(arch_huge_pages, numa_nodes[0])
+                    self.logger.info("force_socket on %s" % numa_nodes[0])
                 else:
-                    for numa_id in range(0, int(numa)):
-                        self.set_huge_pages(arch_huge_pages, numa_id)
-                    if numa == -1:
+                    if total_numa_nodes == -1:
                         self.set_huge_pages(arch_huge_pages)
+                    else:
+                        numa_service_num = self.get_def_rte_config('CONFIG_RTE_MAX_NUMA_NODES')
+                        if numa_service_num is not None:
+                            total_numa_nodes = min(total_numa_nodes, int(numa_service_num))
+
+                        # set huge pages to configured total_numa_nodes
+                        for numa_node in numa_nodes[:total_numa_nodes]:
+                            self.set_huge_pages(arch_huge_pages, numa_node)
 
         self.mount_huge_pages()
         self.hugepage_path = self.strip_hugepage_path()
-- 
2.25.1


                 reply	other threads:[~2021-09-30 14:41 UTC|newest]

Thread overview: [no followups] expand[flat|nested]  mbox.gz  Atom feed

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=20210930144140.673576-1-thinhtr@linux.vnet.ibm.com \
    --to=thinhtr@linux.vnet.ibm.com \
    --cc=dts@dpdk.org \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).