[PATCH] cmdline-gen: fix error when command list has empty lines

[PATCH] cmdline-gen: fix error when command list has empty lines

[Robin Jarry]

Fix the following error when a command list file contains empty lines:

Traceback (most recent call last):
  File "buildtools/dpdk-cmdline-gen.py", line 202, in <module>
    main()
  File "buildtools/dpdk-cmdline-gen.py", line 184, in main
    process_commands(args.infile, sys.stdout, None, args.context_name)
  File "buildtools/dpdk-cmdline-gen.py", line 141, in process_commands
    cmd_inst, h_out, c_out = process_command(lineno, tokens.strip().spl…
                             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^…
  File "buildtools/dpdk-cmdline-gen.py", line 36, in process_command
    if tokens[0].startswith("<"):
       ~~~~~~^^^
IndexError: list index out of range

Use shlex.split() to properly split each line arguments into tokens and
strip comments.

If there are no tokens, ignore the line.

Fixes: 37666691e9ed ("buildtools: add a tool to generate cmdline boilerplate")

Cc: Bruce Richardson <bruce.richardson@intel.com>
Signed-off-by: Robin Jarry <rjarry@redhat.com>
---
 buildtools/dpdk-cmdline-gen.py | 13 ++++++++-----
 1 file changed, 8 insertions(+), 5 deletions(-)

diff --git a/buildtools/dpdk-cmdline-gen.py b/buildtools/dpdk-cmdline-gen.py
index 8922bb5fc38e..49b03bee4a26 100755
--- a/buildtools/dpdk-cmdline-gen.py
+++ b/buildtools/dpdk-cmdline-gen.py
@@ -7,6 +7,7 @@
 """
 
 import argparse
+import shlex
 import sys
 
 PARSE_FN_PARAMS = "void *parsed_result, struct cmdline *cl, void *data"
@@ -133,12 +134,14 @@ def process_commands(infile, hfile, cfile, ctxname):
     )
 
     for lineno, line in enumerate(infile.readlines()):
-        if line.lstrip().startswith("#"):
+        tokens = shlex.split(line, comments=True)
+        if not tokens:
             continue
-        if "#" not in line:
-            line = line + "#"  # ensure split always works, even if no help text
-        tokens, comment = line.split("#", 1)
-        cmd_inst, h_out, c_out = process_command(lineno, tokens.strip().split(), comment.strip())
+        if "#" in line:
+            comment = line.split("#", 1)[-1].strip()
+        else:
+            comment = ""
+        cmd_inst, h_out, c_out = process_command(lineno, tokens, comment)
         hfile.write("\n".join(h_out))
         if cfile:
             cfile.write("\n".join(c_out))
-- 
2.41.0

Re: [PATCH] cmdline-gen: fix error when command list has empty lines

[Bruce Richardson]

On Thu, Nov 16, 2023 at 12:18:13PM +0100, Robin Jarry wrote:
> Fix the following error when a command list file contains empty lines:
> 
> Traceback (most recent call last):
>   File "buildtools/dpdk-cmdline-gen.py", line 202, in <module>
>     main()
>   File "buildtools/dpdk-cmdline-gen.py", line 184, in main
>     process_commands(args.infile, sys.stdout, None, args.context_name)
>   File "buildtools/dpdk-cmdline-gen.py", line 141, in process_commands
>     cmd_inst, h_out, c_out = process_command(lineno, tokens.strip().spl…
>                              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^…
>   File "buildtools/dpdk-cmdline-gen.py", line 36, in process_command
>     if tokens[0].startswith("<"):
>        ~~~~~~^^^
> IndexError: list index out of range
> 
> Use shlex.split() to properly split each line arguments into tokens and
> strip comments.
> 
> If there are no tokens, ignore the line.
> 
> Fixes: 37666691e9ed ("buildtools: add a tool to generate cmdline boilerplate")
> 
> Cc: Bruce Richardson <bruce.richardson@intel.com>
> Signed-off-by: Robin Jarry <rjarry@redhat.com>
> ---

LGTM, thanks.

Acked-by: Bruce Richardson <bruce.richardson@intel.com>

Re: [PATCH] cmdline-gen: fix error when command list has empty lines

[Thomas Monjalon]

17/11/2023 12:22, Bruce Richardson:
> On Thu, Nov 16, 2023 at 12:18:13PM +0100, Robin Jarry wrote:
> > Fix the following error when a command list file contains empty lines:
> > 
> > Traceback (most recent call last):
> >   File "buildtools/dpdk-cmdline-gen.py", line 202, in <module>
> >     main()
> >   File "buildtools/dpdk-cmdline-gen.py", line 184, in main
> >     process_commands(args.infile, sys.stdout, None, args.context_name)
> >   File "buildtools/dpdk-cmdline-gen.py", line 141, in process_commands
> >     cmd_inst, h_out, c_out = process_command(lineno, tokens.strip().spl…
> >                              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^…
> >   File "buildtools/dpdk-cmdline-gen.py", line 36, in process_command
> >     if tokens[0].startswith("<"):
> >        ~~~~~~^^^
> > IndexError: list index out of range
> > 
> > Use shlex.split() to properly split each line arguments into tokens and
> > strip comments.
> > 
> > If there are no tokens, ignore the line.
> > 
> > Fixes: 37666691e9ed ("buildtools: add a tool to generate cmdline boilerplate")
> > 
> > Cc: Bruce Richardson <bruce.richardson@intel.com>
> > Signed-off-by: Robin Jarry <rjarry@redhat.com>
> > ---
> 
> LGTM, thanks.
> 
> Acked-by: Bruce Richardson <bruce.richardson@intel.com>

Applied, thanks.