[dpdk-dev] [PATCH] app/test: fix autotest_runner crash

[dpdk-dev] [PATCH] app/test: fix autotest_runner crash

[Herakliusz Lipiec]

On some systems when dpdk test is executed with make test command
autotest_runner crashes in first_cpu_on_node. This happens when list
of available cpus contains something that is not a cpu as first element.
Fixed by removing all non-cpu values from list of available cpus.

Bugzilla ID: 253
Fixes: 22dcd9a4d90f ("test: parallelize unit tests")
Cc: anatoly.burakov@intel.com
Cc: stable@dpdk.org
Signed-off-by: Herakliusz Lipiec <herakliusz.lipiec@intel.com>
---
 app/test/autotest_runner.py | 9 ++++-----
 1 file changed, 4 insertions(+), 5 deletions(-)

diff --git a/app/test/autotest_runner.py b/app/test/autotest_runner.py
index b72716e1e..7aece8905 100644
--- a/app/test/autotest_runner.py
+++ b/app/test/autotest_runner.py
@@ -43,11 +43,10 @@ def get_numa_nodes():
 # find first (or any, really) CPU on a particular node, will be used to spread
 # processes around NUMA nodes to avoid exhausting memory on particular node
 def first_cpu_on_node(node_nr):
-    cpu_path = glob.glob("/sys/devices/system/node/node%d/cpu*" % node_nr)[0]
-    cpu_name = os.path.basename(cpu_path)
-    m = re.match(r"cpu(\d+)", cpu_name)
-    return int(m.group(1))
-
+    cpu_path = glob.glob("/sys/devices/system/node/node%d/cpu*" % node_nr)
+    r = re.compile(r"cpu(\d+)")
+    cpu_name = filter(None ,map(r.match,  map(os.path.basename, cpu_path)))
+    return int(next(iter(cpu_name)).group(1))
 
 pool_child = None  # per-process child
 
-- 
2.17.2

Re: [dpdk-dev] [PATCH] app/test: fix autotest_runner crash

[Burakov, Anatoly]

On 12-Jun-19 2:33 PM, Herakliusz Lipiec wrote:
> On some systems when dpdk test is executed with make test command
> autotest_runner crashes in first_cpu_on_node. This happens when list
> of available cpus contains something that is not a cpu as first element.
> Fixed by removing all non-cpu values from list of available cpus.
> 
> Bugzilla ID: 253
> Fixes: 22dcd9a4d90f ("test: parallelize unit tests")
> Cc: anatoly.burakov@intel.com
> Cc: stable@dpdk.org
> Signed-off-by: Herakliusz Lipiec <herakliusz.lipiec@intel.com>
> ---
>   app/test/autotest_runner.py | 9 ++++-----
>   1 file changed, 4 insertions(+), 5 deletions(-)
> 
> diff --git a/app/test/autotest_runner.py b/app/test/autotest_runner.py
> index b72716e1e..7aece8905 100644
> --- a/app/test/autotest_runner.py
> +++ b/app/test/autotest_runner.py
> @@ -43,11 +43,10 @@ def get_numa_nodes():
>   # find first (or any, really) CPU on a particular node, will be used to spread
>   # processes around NUMA nodes to avoid exhausting memory on particular node
>   def first_cpu_on_node(node_nr):
> -    cpu_path = glob.glob("/sys/devices/system/node/node%d/cpu*" % node_nr)[0]
> -    cpu_name = os.path.basename(cpu_path)
> -    m = re.match(r"cpu(\d+)", cpu_name)
> -    return int(m.group(1))
> -

This is an unnecessary whitespace change.

> +    cpu_path = glob.glob("/sys/devices/system/node/node%d/cpu*" % node_nr)
> +    r = re.compile(r"cpu(\d+)")
> +    cpu_name = filter(None ,map(r.match,  map(os.path.basename, cpu_path)))

Typo, should be "None, map" rather than "None ,map". Also, perhaps 
splitting this on multiple lines would benefit readability, like this:

cpu_name = filter(None,
         map(r.match,
             map(os.path.basename, cpu_path)
         )
)

No preference though, can be left as is.

> +    return int(next(iter(cpu_name)).group(1))

IMO too much going on on one line, maybe leave the return as it was and 
just fix the match? As in,

m = next(iter(cpu_name))
return int(m.group(1))

Also probably needs a comment explaining why iter() is there - namely, 
that return value from a filter is a list in Python 2, but a generator 
in Python 3, and next() only works with generators.

Otherwise,

Reviewed-by: Anatoly Burakov <anatoly.burakov@intel.com>

>   
>   pool_child = None  # per-process child
>   
> 


-- 
Thanks,
Anatoly

[dpdk-dev] [PATCH v2] app/test: fix autotest_runner crash

[Herakliusz Lipiec]

On some systems when dpdk test is executed with make test command
autotest_runner crashes in first_cpu_on_node. This happens when list
of available cpus contains something that is not a cpu as first element.
Fixed by removing all non-cpu values from list of available cpus.

Bugzilla ID: 253
Fixes: 22dcd9a4d90f ("test: parallelize unit tests")
Cc: anatoly.burakov@intel.com
Cc: stable@dpdk.org
Signed-off-by: Herakliusz Lipiec <herakliusz.lipiec@intel.com>
---
 app/test/autotest_runner.py | 13 ++++++++++---
 1 file changed, 10 insertions(+), 3 deletions(-)

diff --git a/app/test/autotest_runner.py b/app/test/autotest_runner.py
index b72716e1e..95e74c760 100644
--- a/app/test/autotest_runner.py
+++ b/app/test/autotest_runner.py
@@ -43,9 +43,16 @@ def get_numa_nodes():
 # find first (or any, really) CPU on a particular node, will be used to spread
 # processes around NUMA nodes to avoid exhausting memory on particular node
 def first_cpu_on_node(node_nr):
-    cpu_path = glob.glob("/sys/devices/system/node/node%d/cpu*" % node_nr)[0]
-    cpu_name = os.path.basename(cpu_path)
-    m = re.match(r"cpu(\d+)", cpu_name)
+    cpu_path = glob.glob("/sys/devices/system/node/node%d/cpu*" % node_nr)
+    r = re.compile(r"cpu(\d+)")
+    cpu_name = filter(None,
+            map(r.match,
+                map(os.path.basename, cpu_path)
+            )
+    )
+    # for compatibility between python 3 and 2 we need to make interable out
+    # of filter return as it returns list in python 2 and a generator in 3
+    m = next(iter(cpu_name))
     return int(m.group(1))
 
 
-- 
2.17.2

Re: [dpdk-dev] [PATCH v2] app/test: fix autotest_runner crash

[Burakov, Anatoly]

On 12-Jun-19 3:46 PM, Herakliusz Lipiec wrote:
> On some systems when dpdk test is executed with make test command
> autotest_runner crashes in first_cpu_on_node. This happens when list
> of available cpus contains something that is not a cpu as first element.
> Fixed by removing all non-cpu values from list of available cpus.
> 
> Bugzilla ID: 253
> Fixes: 22dcd9a4d90f ("test: parallelize unit tests")
> Cc: anatoly.burakov@intel.com
> Cc: stable@dpdk.org
> Signed-off-by: Herakliusz Lipiec <herakliusz.lipiec@intel.com>
> ---

Missed reviewed tag from previous revision.

Reviewed-by: Anatoly Burakov <anatoly.burakov@intel.com>

-- 
Thanks,
Anatoly

Re: [dpdk-dev] [PATCH v2] app/test: fix autotest_runner crash

[Thomas Monjalon]

12/06/2019 16:48, Burakov, Anatoly:
> On 12-Jun-19 3:46 PM, Herakliusz Lipiec wrote:
> > On some systems when dpdk test is executed with make test command
> > autotest_runner crashes in first_cpu_on_node. This happens when list
> > of available cpus contains something that is not a cpu as first element.
> > Fixed by removing all non-cpu values from list of available cpus.
> > 
> > Bugzilla ID: 253
> > Fixes: 22dcd9a4d90f ("test: parallelize unit tests")
> > Cc: anatoly.burakov@intel.com
> > Cc: stable@dpdk.org
> > Signed-off-by: Herakliusz Lipiec <herakliusz.lipiec@intel.com>
> 
> Missed reviewed tag from previous revision.
> 
> Reviewed-by: Anatoly Burakov <anatoly.burakov@intel.com>

Applied, thanks