[dpdk-dev] [PATCH] usertools: show valid hugepage sizes if user requests an invalid hugepage size

[dpdk-dev] [PATCH] usertools: show valid hugepage sizes if user requests an invalid hugepage size

[Sarosh Arif]

If user requests a hugepage size which is not supported by the system,
currently user gets an error message saying that the requested size
is not a valid system huge page size. In addition to this if we display
the valid hugepage sizes it will be convenient for the user to request
the right size next time. 

Signed-off-by: Sarosh Arif <sarosh.arif@emumba.com>
---
 usertools/dpdk-hugepages.py | 13 +++++++++++--
 1 file changed, 11 insertions(+), 2 deletions(-)

diff --git a/usertools/dpdk-hugepages.py b/usertools/dpdk-hugepages.py
index 1be100ca3..f503ad377 100755
--- a/usertools/dpdk-hugepages.py
+++ b/usertools/dpdk-hugepages.py
@@ -49,6 +49,14 @@ def get_hugepages(path):
         return int(nr_hugepages.read())
     return 0
 
+def show_valid_page_sizes(path):
+    valid_page_sizes = ""
+    hugepage_dir_path = os.path.split(path)[0]
+    hugepage_dirs = os.listdir(hugepage_dir_path)
+    for each_dir in hugepage_dirs:
+        hugepage_size = each_dir.split("-")[1]
+        valid_page_sizes = valid_page_sizes + " " + hugepage_size
+    print("valid page sizes: {}".format(valid_page_sizes))
 
 def set_hugepages(path, pages):
     '''Write the number of reserved huge pages'''
@@ -61,8 +69,9 @@ def set_hugepages(path, pages):
     except FileNotFoundError:
         filename = os.path.basename(path)
         size = filename[10:]
-        sys.exit('{} is not a valid system huge page size'.format(size))
-
+        print('{} is not a valid system huge page size'.format(size))
+        show_valid_page_sizes(path)
+        sys.exit()
 
 def show_numa_pages():
     '''Show huge page reservations on Numa system'''
-- 
2.25.1

Re: [dpdk-dev] [PATCH] usertools: show valid hugepage sizes if user requests an invalid hugepage size

[Stephen Hemminger]

On Mon, 30 Nov 2020 17:47:19 +0500
Sarosh Arif <sarosh.arif@emumba.com> wrote:

> If user requests a hugepage size which is not supported by the system,
> currently user gets an error message saying that the requested size
> is not a valid system huge page size. In addition to this if we display
> the valid hugepage sizes it will be convenient for the user to request
> the right size next time. 
> 
> Signed-off-by: Sarosh Arif <sarosh.arif@emumba.com>
> ---
>  usertools/dpdk-hugepages.py | 13 +++++++++++--
>  1 file changed, 11 insertions(+), 2 deletions(-)

If the two patches depend on each other, they should be sent in one
email thread with 1/2 2/2 in header.

This patch introduces pylint warning

usertools/dpdk-hugepages.py:52:0: C0116: Missing function or method docstring (missing-function-docstring)

Re: [dpdk-dev] [PATCH] usertools: show valid hugepage sizes if user requests an invalid hugepage size

[Stephen Hemminger]

On Mon, 30 Nov 2020 17:47:19 +0500
Sarosh Arif <sarosh.arif@emumba.com> wrote:

> -        sys.exit('{} is not a valid system huge page size'.format(size))
> -
> +        print('{} is not a valid system huge page size'.format(size))
> +        show_valid_page_sizes(path)
> +        sys.exit()

This is not the same. You should change the helper function to return
a string and do:

	sys.exit('Invalid page size. Valid sizes are:', valid_page_sizes())

[dpdk-dev] [v2 PATCH] usertools: show valid hugepage sizes if user requests an invalid hugepage size

[Sarosh Arif]

If user requests a hugepage size which is not supported by the system,
currently user gets an error message saying that the requested size
is not a valid system huge page size. In addition to this if we display
the valid hugepage sizes it will be convenient for the user to request
the right size next time.

Signed-off-by: Sarosh Arif <sarosh.arif@emumba.com>
---
v2:
pass string in sys.exit() to remove pylint warning 
---
 usertools/dpdk-hugepages.py | 14 ++++++++++----
 1 file changed, 10 insertions(+), 4 deletions(-)

diff --git a/usertools/dpdk-hugepages.py b/usertools/dpdk-hugepages.py
index 1be100ca3..09d35bf55 100755
--- a/usertools/dpdk-hugepages.py
+++ b/usertools/dpdk-hugepages.py
@@ -49,6 +49,14 @@ def get_hugepages(path):
         return int(nr_hugepages.read())
     return 0
 
+def get_valid_page_sizes(path):
+    valid_page_sizes = ""
+    hugepage_dir_path = os.path.split(path)[0]
+    hugepage_dirs = os.listdir(hugepage_dir_path)
+    for each_dir in hugepage_dirs:
+        hugepage_size = each_dir.split("-")[1]
+        valid_page_sizes = valid_page_sizes + " " + hugepage_size
+    return valid_page_sizes
 
 def set_hugepages(path, pages):
     '''Write the number of reserved huge pages'''
@@ -59,10 +67,8 @@ def set_hugepages(path, pages):
     except PermissionError:
         sys.exit('Permission denied: need to be root!')
     except FileNotFoundError:
-        filename = os.path.basename(path)
-        size = filename[10:]
-        sys.exit('{} is not a valid system huge page size'.format(size))
-
+        sys.exit("Invalid page size. Valid page sizes: {}".format(
+                                        get_valid_page_sizes(path)))
 
 def show_numa_pages():
     '''Show huge page reservations on Numa system'''
-- 
2.25.1

Re: [dpdk-dev] [v2 PATCH] usertools: show valid hugepage sizes if user requests an invalid hugepage size

[Thomas Monjalon]

02/12/2020 12:06, Sarosh Arif:
> If user requests a hugepage size which is not supported by the system,
> currently user gets an error message saying that the requested size
> is not a valid system huge page size. In addition to this if we display
> the valid hugepage sizes it will be convenient for the user to request
> the right size next time.
> 
> Signed-off-by: Sarosh Arif <sarosh.arif@emumba.com>
> ---
> v2:
> pass string in sys.exit() to remove pylint warning

Stephen, is this version OK?

> +def get_valid_page_sizes(path):
> +    valid_page_sizes = ""
> +    hugepage_dir_path = os.path.split(path)[0]

Sorry for not following. Why this split?
Maybe add a comment?

> +    hugepage_dirs = os.listdir(hugepage_dir_path)
> +    for each_dir in hugepage_dirs:
> +        hugepage_size = each_dir.split("-")[1]
> +        valid_page_sizes = valid_page_sizes + " " + hugepage_size

Why not using += here?

> +    return valid_page_sizes
>  
>  def set_hugepages(path, pages):
>      '''Write the number of reserved huge pages'''
> @@ -59,10 +67,8 @@ def set_hugepages(path, pages):
>      except PermissionError:
>          sys.exit('Permission denied: need to be root!')
>      except FileNotFoundError:
> -        filename = os.path.basename(path)
> -        size = filename[10:]
> -        sys.exit('{} is not a valid system huge page size'.format(size))
> -
> +        sys.exit("Invalid page size. Valid page sizes: {}".format(
> +                                        get_valid_page_sizes(path)))

Re: [dpdk-dev] [v2 PATCH] usertools: show valid hugepage sizes if user requests an invalid hugepage size

[Stephen Hemminger]

On Fri, 05 Feb 2021 19:17:52 +0100
Thomas Monjalon <thomas@monjalon.net> wrote:

> 02/12/2020 12:06, Sarosh Arif:
> > If user requests a hugepage size which is not supported by the system,
> > currently user gets an error message saying that the requested size
> > is not a valid system huge page size. In addition to this if we display
> > the valid hugepage sizes it will be convenient for the user to request
> > the right size next time.
> > 
> > Signed-off-by: Sarosh Arif <sarosh.arif@emumba.com>
> > ---
> > v2:
> > pass string in sys.exit() to remove pylint warning  
> 
> Stephen, is this version OK?

This looks good.

> 
> > +def get_valid_page_sizes(path):
> > +    valid_page_sizes = ""
> > +    hugepage_dir_path = os.path.split(path)[0]  
> 
> Sorry for not following. Why this split?
> Maybe add a comment?

Guess the code here is using split ad equivalent of dirname() on
the path.  The path is the string which holds the full filename of
the attempted write to sysfs.  So this is to get the directory
part which could be numa or non-numa.

> 
> > +    hugepage_dirs = os.listdir(hugepage_dir_path)
> > +    for each_dir in hugepage_dirs:
> > +        hugepage_size = each_dir.split("-")[1]
> > +        valid_page_sizes = valid_page_sizes + " " + hugepage_size  
> 
> Why not using += here?

Should work. Overall, this reads a bit like C code here.
A python wizard would use list comprehensions here
https://docs.python.org/2/tutorial/datastructures.html#list-comprehensions


> 
> > +    return valid_page_sizes
> >  
> >  def set_hugepages(path, pages):
> >      '''Write the number of reserved huge pages'''
> > @@ -59,10 +67,8 @@ def set_hugepages(path, pages):
> >      except PermissionError:
> >          sys.exit('Permission denied: need to be root!')
> >      except FileNotFoundError:
> > -        filename = os.path.basename(path)
> > -        size = filename[10:]
> > -        sys.exit('{} is not a valid system huge page size'.format(size))
> > -
> > +        sys.exit("Invalid page size. Valid page sizes: {}".format(
> > +                                        get_valid_page_sizes(path)))  
> 
> 
> 

Re: [dpdk-dev] [v2 PATCH] usertools: show valid hugepage sizes if user requests an invalid hugepage size

[Burakov, Anatoly]

On 02-Dec-20 11:06 AM, Sarosh Arif wrote:
> If user requests a hugepage size which is not supported by the system,
> currently user gets an error message saying that the requested size
> is not a valid system huge page size. In addition to this if we display
> the valid hugepage sizes it will be convenient for the user to request
> the right size next time.
> 
> Signed-off-by: Sarosh Arif <sarosh.arif@emumba.com>
> ---
> v2:
> pass string in sys.exit() to remove pylint warning
> ---
>   usertools/dpdk-hugepages.py | 14 ++++++++++----
>   1 file changed, 10 insertions(+), 4 deletions(-)
> 
> diff --git a/usertools/dpdk-hugepages.py b/usertools/dpdk-hugepages.py
> index 1be100ca3..09d35bf55 100755
> --- a/usertools/dpdk-hugepages.py
> +++ b/usertools/dpdk-hugepages.py
> @@ -49,6 +49,14 @@ def get_hugepages(path):
>           return int(nr_hugepages.read())
>       return 0
>   
> +def get_valid_page_sizes(path):
> +    valid_page_sizes = ""
> +    hugepage_dir_path = os.path.split(path)[0]

os.path.dirname() ?

> +    hugepage_dirs = os.listdir(hugepage_dir_path)
> +    for each_dir in hugepage_dirs:
> +        hugepage_size = each_dir.split("-")[1]
> +        valid_page_sizes = valid_page_sizes + " " + hugepage_size

You could rewrite this to be clearer and more concise:

def get_valid_page_sizes(path):
     dir = os.path.dirname(path)
     pg_sizes = (d.split("-")[1] for d in os.listdir(dir))
     return " ".join(pg_sizes)

> +    return valid_page_sizes
>   
>   def set_hugepages(path, pages):
>       '''Write the number of reserved huge pages'''
> @@ -59,10 +67,8 @@ def set_hugepages(path, pages):
>       except PermissionError:
>           sys.exit('Permission denied: need to be root!')
>       except FileNotFoundError:
> -        filename = os.path.basename(path)
> -        size = filename[10:]
> -        sys.exit('{} is not a valid system huge page size'.format(size))
> -
> +        sys.exit("Invalid page size. Valid page sizes: {}".format(
> +                                        get_valid_page_sizes(path)))
>   
>   def show_numa_pages():
>       '''Show huge page reservations on Numa system'''
> 


-- 
Thanks,
Anatoly

[dpdk-dev] [v3 PATCH] usertools: show valid hugepage sizes if user requests an invalid hugepage size

[Sarosh Arif]

If user requests a hugepage size which is not supported by the system,
currently user gets an error message saying that the requested size
is not a valid system huge page size. In addition to this if we display
the valid hugepage sizes it will be convenient for the user to request
the right size next time.

Signed-off-by: Sarosh Arif <sarosh.arif@emumba.com>
---
v2:
pass string in sys.exit() to remove pylint warning
v3:
modify get_valid_page_sizes() 
---

 usertools/dpdk-hugepages.py | 11 ++++++++---
 1 file changed, 8 insertions(+), 3 deletions(-)

diff --git a/usertools/dpdk-hugepages.py b/usertools/dpdk-hugepages.py
index 70432f9cd..57ae9f391 100755
--- a/usertools/dpdk-hugepages.py
+++ b/usertools/dpdk-hugepages.py
@@ -43,6 +43,12 @@ def is_numa():
     return os.path.exists('/sys/devices/system/node')
 
 
+def get_valid_page_sizes(path):
+    '''Extract valid hugepage sizes'''
+    dir = os.path.dirname(path)
+    pg_sizes = (d.split("-")[1] for d in os.listdir(dir))
+    return " ".join(pg_sizes)
+
 def get_hugepages(path):
     '''Read number of reserved pages'''
     with open(path + '/nr_hugepages') as nr_hugepages:
@@ -59,9 +65,8 @@ def set_hugepages(path, pages):
     except PermissionError:
         sys.exit('Permission denied: need to be root!')
     except FileNotFoundError:
-        filename = os.path.basename(path)
-        size = filename[10:]
-        sys.exit('{} is not a valid system huge page size'.format(size))
+        sys.exit("Invalid page size. Valid page sizes: {}".format(
+                                get_valid_page_sizes(path)))
     if get_hugepages(path) != pages:
         sys.exit('Unable to reserve required pages.')
 
-- 
2.25.1

Re: [dpdk-dev] [v3 PATCH] usertools: show valid hugepage sizes if user requests an invalid hugepage size

[Burakov, Anatoly]

On 10-Feb-21 6:16 AM, Sarosh Arif wrote:
> If user requests a hugepage size which is not supported by the system,
> currently user gets an error message saying that the requested size
> is not a valid system huge page size. In addition to this if we display
> the valid hugepage sizes it will be convenient for the user to request
> the right size next time.
> 
> Signed-off-by: Sarosh Arif <sarosh.arif@emumba.com>
> ---
> v2:
> pass string in sys.exit() to remove pylint warning
> v3:
> modify get_valid_page_sizes()
> ---
> 
>   usertools/dpdk-hugepages.py | 11 ++++++++---
>   1 file changed, 8 insertions(+), 3 deletions(-)
> 
> diff --git a/usertools/dpdk-hugepages.py b/usertools/dpdk-hugepages.py
> index 70432f9cd..57ae9f391 100755
> --- a/usertools/dpdk-hugepages.py
> +++ b/usertools/dpdk-hugepages.py
> @@ -43,6 +43,12 @@ def is_numa():
>       return os.path.exists('/sys/devices/system/node')
>   
>   
> +def get_valid_page_sizes(path):
> +    '''Extract valid hugepage sizes'''
> +    dir = os.path.dirname(path)
> +    pg_sizes = (d.split("-")[1] for d in os.listdir(dir))
> +    return " ".join(pg_sizes)
> +
>   def get_hugepages(path):
>       '''Read number of reserved pages'''
>       with open(path + '/nr_hugepages') as nr_hugepages:
> @@ -59,9 +65,8 @@ def set_hugepages(path, pages):
>       except PermissionError:
>           sys.exit('Permission denied: need to be root!')
>       except FileNotFoundError:
> -        filename = os.path.basename(path)
> -        size = filename[10:]
> -        sys.exit('{} is not a valid system huge page size'.format(size))
> +        sys.exit("Invalid page size. Valid page sizes: {}".format(
> +                                get_valid_page_sizes(path)))

Nitpicking, but i think this indentation wouldn't be OK with a PEP-8 
style checker. This should be better:

sys.exit("Invalid page size. Valid page sizes: {}"
          .format(get_validpage_sizes(path))

(note the alignment of .format with the quote)

Otherwise, LGTM

Acked-by: Anatoly Burakov <anatoly.burakov@intel.com>

>       if get_hugepages(path) != pages:
>           sys.exit('Unable to reserve required pages.')
>   
> 


-- 
Thanks,
Anatoly

[dpdk-dev] [v4 PATCH] usertools: show valid hugepage sizes if user requests an invalid hugepage size

[Sarosh Arif]

If user requests a hugepage size which is not supported by the system,
currently user gets an error message saying that the requested size
is not a valid system huge page size. In addition to this if we display
the valid hugepage sizes it will be convenient for the user to request
the right size next time.

Signed-off-by: Sarosh Arif <sarosh.arif@emumba.com>
---
v2:
pass string in sys.exit() to remove pylint warning
v3:
modify get_valid_page_sizes()
v4:
fix indentation
---
 usertools/dpdk-hugepages.py | 11 ++++++++---
 1 file changed, 8 insertions(+), 3 deletions(-)

diff --git a/usertools/dpdk-hugepages.py b/usertools/dpdk-hugepages.py
index 70432f9cd..e68bae5a4 100755
--- a/usertools/dpdk-hugepages.py
+++ b/usertools/dpdk-hugepages.py
@@ -43,6 +43,12 @@ def is_numa():
     return os.path.exists('/sys/devices/system/node')
 
 
+def get_valid_page_sizes(path):
+    '''Extract valid hugepage sizes'''
+    dir = os.path.dirname(path)
+    pg_sizes = (d.split("-")[1] for d in os.listdir(dir))
+    return " ".join(pg_sizes)
+
 def get_hugepages(path):
     '''Read number of reserved pages'''
     with open(path + '/nr_hugepages') as nr_hugepages:
@@ -59,9 +65,8 @@ def set_hugepages(path, pages):
     except PermissionError:
         sys.exit('Permission denied: need to be root!')
     except FileNotFoundError:
-        filename = os.path.basename(path)
-        size = filename[10:]
-        sys.exit('{} is not a valid system huge page size'.format(size))
+        sys.exit("Invalid page size. Valid page sizes: {}".format(
+                get_valid_page_sizes(path)))
     if get_hugepages(path) != pages:
         sys.exit('Unable to reserve required pages.')
 
-- 
2.25.1

Re: [dpdk-dev] [v4 PATCH] usertools: show valid hugepage sizes if user requests an invalid hugepage size

[Stephen Hemminger]

On Thu, 11 Feb 2021 13:20:04 +0500
Sarosh Arif <sarosh.arif@emumba.com> wrote:

> If user requests a hugepage size which is not supported by the system,
> currently user gets an error message saying that the requested size
> is not a valid system huge page size. In addition to this if we display
> the valid hugepage sizes it will be convenient for the user to request
> the right size next time.
> 
> Signed-off-by: Sarosh Arif <sarosh.arif@emumba.com>

Looks good, thanks for following up.

Acked-by: Stephen Hemminger <stephen@networkplumber.org>

Re: [dpdk-dev] [v4 PATCH] usertools: show valid hugepage sizes if user requests an invalid hugepage size

[Thomas Monjalon]

11/02/2021 09:20, Sarosh Arif:
> If user requests a hugepage size which is not supported by the system,
> currently user gets an error message saying that the requested size
> is not a valid system huge page size. In addition to this if we display
> the valid hugepage sizes it will be convenient for the user to request
> the right size next time.
> 
> Signed-off-by: Sarosh Arif <sarosh.arif@emumba.com>
> ---
> --- a/usertools/dpdk-hugepages.py
> +++ b/usertools/dpdk-hugepages.py
> @@ -43,6 +43,12 @@ def is_numa():
>      return os.path.exists('/sys/devices/system/node')
>  
>  
> +def get_valid_page_sizes(path):
> +    '''Extract valid hugepage sizes'''
> +    dir = os.path.dirname(path)
> +    pg_sizes = (d.split("-")[1] for d in os.listdir(dir))
> +    return " ".join(pg_sizes)
> +

A second blank line is required here.

>  def get_hugepages(path):
>      '''Read number of reserved pages'''
>      with open(path + '/nr_hugepages') as nr_hugepages:
> @@ -59,9 +65,8 @@ def set_hugepages(path, pages):
>      except PermissionError:
>          sys.exit('Permission denied: need to be root!')
>      except FileNotFoundError:
> -        filename = os.path.basename(path)
> -        size = filename[10:]
> -        sys.exit('{} is not a valid system huge page size'.format(size))
> +        sys.exit("Invalid page size. Valid page sizes: {}".format(
> +                get_valid_page_sizes(path)))

Alignment needs a small fixup.

After these small changes, flake8 is OK.

Applied, thanks.

Re: [dpdk-dev] [v4 PATCH] usertools: show valid hugepage sizes if user requests an invalid hugepage size

[Stephen Hemminger]

On Thu, 11 Feb 2021 22:44:03 +0100
Thomas Monjalon <thomas@monjalon.net> wrote:

> 11/02/2021 09:20, Sarosh Arif:
> > If user requests a hugepage size which is not supported by the system,
> > currently user gets an error message saying that the requested size
> > is not a valid system huge page size. In addition to this if we display
> > the valid hugepage sizes it will be convenient for the user to request
> > the right size next time.
> > 
> > Signed-off-by: Sarosh Arif <sarosh.arif@emumba.com>
> > ---
> > --- a/usertools/dpdk-hugepages.py
> > +++ b/usertools/dpdk-hugepages.py
> > @@ -43,6 +43,12 @@ def is_numa():
> >      return os.path.exists('/sys/devices/system/node')
> >  
> >  
> > +def get_valid_page_sizes(path):
> > +    '''Extract valid hugepage sizes'''
> > +    dir = os.path.dirname(path)
> > +    pg_sizes = (d.split("-")[1] for d in os.listdir(dir))
> > +    return " ".join(pg_sizes)
> > +  
> 
> A second blank line is required here.
> 
> >  def get_hugepages(path):
> >      '''Read number of reserved pages'''
> >      with open(path + '/nr_hugepages') as nr_hugepages:
> > @@ -59,9 +65,8 @@ def set_hugepages(path, pages):
> >      except PermissionError:
> >          sys.exit('Permission denied: need to be root!')
> >      except FileNotFoundError:
> > -        filename = os.path.basename(path)
> > -        size = filename[10:]
> > -        sys.exit('{} is not a valid system huge page size'.format(size))
> > +        sys.exit("Invalid page size. Valid page sizes: {}".format(
> > +                get_valid_page_sizes(path)))  
> 
> Alignment needs a small fixup.
> 
> After these small changes, flake8 is OK.
> 
> Applied, thanks.

I used Yet-Another-Python-Formmatter originally (yapf).
Its preferred style for that would be:
diff --git a/usertools/dpdk-hugepages.py b/usertools/dpdk-hugepages.py
index fb368b693399..28eba86641f7 100755
--- a/usertools/dpdk-hugepages.py
+++ b/usertools/dpdk-hugepages.py
@@ -67,7 +67,7 @@ def set_hugepages(path, pages):
         sys.exit('Permission denied: need to be root!')
     except FileNotFoundError:
         sys.exit("Invalid page size. Valid page sizes: {}".format(
-                 get_valid_page_sizes(path)))
+            get_valid_page_sizes(path)))
     if get_hugepages(path) != pages:
         sys.exit('Unable to reserve required pages.')