[PATCH 0/2] buildtools, devtools, dts: clear out Python syntax warnings

[PATCH 0/2] buildtools, devtools, dts: clear out Python syntax warnings

[Ariel Otilibili]

Hello,

This series clears out Python syntax warnings; it follows from a discussion with
Robin Jarry. Being one-liners, they are backported to stable.

Below are all occurences of the re module in the code base. Where it was due,
the string have been escaped.

Thank you,

```
$ git grep -Pn '[^\w_]re\.[a-z]+' | perl -pE 's/:(\d+):\s*/|$1|/' | sed -e '1iFILE_NAME|LINE|PATTERN'
FILE_NAME|LINE|PATTERN
buildtools/get-numa-count.py|14|numa_nodes.sort(key=lambda l: int(re.findall(r'\d+', l)[0]))
buildtools/get-test-suites.py|9|test_def_regex = re.compile("REGISTER_([A-Z]+)_TEST\s*\(\s*([a-z0-9_]+)")
buildtools/get-test-suites.py|12|non_suite_regex = re.compile("REGISTER_TEST_COMMAND\s*\(\s*([a-z0-9_]+)")
buildtools/pmdinfogen.py|74|if re.match(pattern, symbol.name):
buildtools/pmdinfogen.py|103|if re.match(pattern, symbol.name):
devtools/check-meson.py|54|if re.match(r'^ *\t', code):
devtools/update_version_map_abi.py|20|func_line_regex = re.compile(r"\s*"
devtools/update_version_map_abi.py|32|section_begin_regex = re.compile(r"\s*"
devtools/update_version_map_abi.py|41|section_end_regex = re.compile(r"\s*"
devtools/update_version_map_abi.py|183|if not re.match(r"\d{1,2}\.\d{1,2}", parsed.abi_version):
doc/api/generate_doxygen.py|8|pattern = re.compile('^Preprocessing (.*)...$')
doc/api/generate_doxygen.py|17|match = re.match(pattern, line)
dts/framework/parser.py|142|pattern = re.compile(pattern, flags)
dts/framework/parser.py|186|pattern = re.compile(pattern, flags)
dts/framework/remote_session/testpmd_shell.py|1669|iter = re.finditer(r"\*{21}.*?[\r\n]{4}", output + "\r\n", re.S)
dts/framework/remote_session/testpmd_shell.py|1777|iter = re.finditer(r"(^  #*.+#*$[^#]+)^  #*\r$", output, re.MULTILINE)
dts/framework/remote_session/testpmd_shell.py|1898|iter = re.finditer(
dts/framework/testbed_model/posix_session.py|339|match = re.match(pid_regex, out_line)
dts/framework/testbed_model/traffic_generator/scapy.py|236|list_of_packets_base64 = re.findall(
dts/tests/TestSuite_smoke_tests.py|138|devbind_info_for_nic = re.search(
usertools/dpdk-hugepages.py|105|match = re.match(rf"(\d+)([{BINARY_PREFIX}]?)$", arg.upper())
usertools/dpdk-hugepages.py|225|m = re.match(r"node(\d+)", os.path.basename(path))
usertools/dpdk-pmdinfo.py|229|match = re.match(r"^(\d+)\.(\d+).*$", str(version))
usertools/dpdk-pmdinfo.py|273|LDD_LIB_RE = re.compile(
usertools/dpdk-telemetry-exporter.py|322|fullname = re.sub(r"\W", "_", f"dpdk_{e.__name__}_{name}")
usertools/dpdk-telemetry-exporter.py|383|fullname = re.sub(r"\W", ".", f"dpdk.{e.__name__}.{name}")
```

Link: https://inbox.dpdk.org/dev/D6AMQXRSG8E7.33BAORRHRHV9A@redhat.com/
Link: https://inbox.dpdk.org/dev/CAJFAV8yEaWSX2kdiANWuKx7zsvTEnrVoVjnZAOQ_ocdzM8ZTAw@mail.gmail.com/

Ariel Otilibili (2):
  buildtools,devtools: clear out Python syntax warnings
  dts: clear out Python syntax warnings

 buildtools/get-numa-count.py                           | 2 +-
 devtools/check-meson.py                                | 2 +-
 dts/framework/testbed_model/traffic_generator/scapy.py | 2 +-
 dts/tests/TestSuite_smoke_tests.py                     | 2 +-
 4 files changed, 4 insertions(+), 4 deletions(-)

-- 
2.47.1

[PATCH 1/2] buildtools,devtools: clear out Python syntax warnings

[Ariel Otilibili]

* follow up of a discussion with Robin Jarry
* since 3.12 invalid escape sequences generate SyntaxWarning
* in the future, these invalid sequences will generate SyntaxError
* therefore changed syntax to raw string notation.

Link: https://inbox.dpdk.org/dev/D6AMQXRSG8E7.33BAORRHRHV9A@redhat.com/
Link: https://docs.python.org/3/whatsnew/3.12.html#other-language-changes
Fixes: 9d4efc5cc6f ("buildtools: fix NUMA nodes count")
Fixes: f88b0b89220 ("devtools: forbid indent with tabs in Meson")

Cc: Bruce Richardson <bruce.richardson@intel.com>
Cc: stable@dpdk.org
Suggested-by: Robin Jarry <rjarry@redhat.com>
Signed-off-by: Ariel Otilibili <otilibil@eurecom.fr>
---
 buildtools/get-numa-count.py | 2 +-
 devtools/check-meson.py      | 2 +-
 2 files changed, 2 insertions(+), 2 deletions(-)

diff --git a/buildtools/get-numa-count.py b/buildtools/get-numa-count.py
index 2f243886cd..f8bea5b58e 100644
--- a/buildtools/get-numa-count.py
+++ b/buildtools/get-numa-count.py
@@ -11,7 +11,7 @@
 if os.name == 'posix':
     if os.path.isdir('/sys/devices/system/node'):
         numa_nodes = glob.glob('/sys/devices/system/node/node*')
-        numa_nodes.sort(key=lambda l: int(re.findall('\d+', l)[0]))
+        numa_nodes.sort(key=lambda l: int(re.findall(r'\d+', l)[0]))
         print(int(os.path.basename(numa_nodes[-1])[4:]) + 1)
     else:
         subprocess.run(['sysctl', '-n', 'vm.ndomains'], check=False)
diff --git a/devtools/check-meson.py b/devtools/check-meson.py
index 4b2338828d..9a33d8b036 100755
--- a/devtools/check-meson.py
+++ b/devtools/check-meson.py
@@ -51,7 +51,7 @@ def check_indentation(filename, contents):
         code, comments = split_code_comments(line)
         if not code.strip():
             continue
-        if re.match('^ *\t', code):
+        if re.match(r'^ *\t', code):
             print(f'Error parsing {filename}:{lineno}, got some tabulation')
         if code.endswith('files('):
             if infiles:
-- 
2.47.1

[PATCH 2/2] dts: clear out Python syntax warnings

[Ariel Otilibili]

* follow up of a discussion with Robin Jarry
* since 3.12 invalid escape sequences generate SyntaxWarning
* in the future, these invalid sequences will generate SyntaxError
* therefore changed syntax to raw string notation.

Link: https://inbox.dpdk.org/dev/D6AMQXRSG8E7.33BAORRHRHV9A@redhat.com/
Link: https://docs.python.org/3/whatsnew/3.12.html#other-language-changes
Fixes: 99740300890 ("dts: use Python shell for Scapy instead of XML-RPC")
Fixes: 409359adce4 ("dts: fix smoke tests driver regex")

Cc: Paul Szczepanek <paul.szczepanek@arm.com>
Cc: Patrick Robb <probb@iol.unh.edu>
Cc: stable@dpdk.org
Suggested-by: Robin Jarry <rjarry@redhat.com>
Signed-off-by: Ariel Otilibili <otilibil@eurecom.fr>
---
 dts/framework/testbed_model/traffic_generator/scapy.py | 2 +-
 dts/tests/TestSuite_smoke_tests.py                     | 2 +-
 2 files changed, 2 insertions(+), 2 deletions(-)

diff --git a/dts/framework/testbed_model/traffic_generator/scapy.py b/dts/framework/testbed_model/traffic_generator/scapy.py
index 1251ca65a0..f42f663554 100644
--- a/dts/framework/testbed_model/traffic_generator/scapy.py
+++ b/dts/framework/testbed_model/traffic_generator/scapy.py
@@ -234,6 +234,6 @@ def _shell_start_and_stop_sniffing(self, duration: float) -> list[Packet]:
         )
         # In the string of bytes "b'XXXX'", we only want the contents ("XXXX")
         list_of_packets_base64 = re.findall(
-            f"^b'({REGEX_FOR_BASE64_ENCODING})'", packet_strs, re.MULTILINE
+            fr"^b'({REGEX_FOR_BASE64_ENCODING})'", packet_strs, re.MULTILINE
         )
         return [Ether(base64_bytes(pakt)) for pakt in list_of_packets_base64]
diff --git a/dts/tests/TestSuite_smoke_tests.py b/dts/tests/TestSuite_smoke_tests.py
index bc3a2a6bf9..a029a564eb 100644
--- a/dts/tests/TestSuite_smoke_tests.py
+++ b/dts/tests/TestSuite_smoke_tests.py
@@ -136,7 +136,7 @@ def test_device_bound_to_driver(self) -> None:
             # with the address for the nic we are on in the loop and then captures the
             # name of the driver in a group
             devbind_info_for_nic = re.search(
-                f"{nic.pci}[^\\n]*drv=([\\d\\w-]*) [^\\n]*",
+                fr"{nic.pci}[^\\n]*drv=([\\d\\w-]*) [^\\n]*",
                 all_nics_in_dpdk_devbind,
             )
             self.verify(
-- 
2.47.1

Re: [PATCH 2/2] dts: clear out Python syntax warnings

[Patrick Robb]

[-- Attachment #1: Type: text/plain, Size: 2759 bytes --]

Reviewed-by: Patrick Robb

A question for main branch maintainers - should this series go directly to
the main branch? I don't think I should apply this to next-dts due to the
1/2 patch.

On Sat, Dec 14, 2024 at 5:43 AM Ariel Otilibili <otilibil@eurecom.fr> wrote:

> * follow up of a discussion with Robin Jarry
> * since 3.12 invalid escape sequences generate SyntaxWarning
> * in the future, these invalid sequences will generate SyntaxError
> * therefore changed syntax to raw string notation.
>
> Link: https://inbox.dpdk.org/dev/D6AMQXRSG8E7.33BAORRHRHV9A@redhat.com/
> Link: https://docs.python.org/3/whatsnew/3.12.html#other-language-changes
> Fixes: 99740300890 ("dts: use Python shell for Scapy instead of XML-RPC")
> Fixes: 409359adce4 ("dts: fix smoke tests driver regex")
>
> Cc: Paul Szczepanek <paul.szczepanek@arm.com>
> Cc: Patrick Robb <probb@iol.unh.edu>
> Cc: stable@dpdk.org
> Suggested-by: Robin Jarry <rjarry@redhat.com>
> Signed-off-by: Ariel Otilibili <otilibil@eurecom.fr>
> ---
>  dts/framework/testbed_model/traffic_generator/scapy.py | 2 +-
>  dts/tests/TestSuite_smoke_tests.py                     | 2 +-
>  2 files changed, 2 insertions(+), 2 deletions(-)
>
> diff --git a/dts/framework/testbed_model/traffic_generator/scapy.py
> b/dts/framework/testbed_model/traffic_generator/scapy.py
> index 1251ca65a0..f42f663554 100644
> --- a/dts/framework/testbed_model/traffic_generator/scapy.py
> +++ b/dts/framework/testbed_model/traffic_generator/scapy.py
> @@ -234,6 +234,6 @@ def _shell_start_and_stop_sniffing(self, duration:
> float) -> list[Packet]:
>          )
>          # In the string of bytes "b'XXXX'", we only want the contents
> ("XXXX")
>          list_of_packets_base64 = re.findall(
> -            f"^b'({REGEX_FOR_BASE64_ENCODING})'", packet_strs,
> re.MULTILINE
> +            fr"^b'({REGEX_FOR_BASE64_ENCODING})'", packet_strs,
> re.MULTILINE
>          )
>          return [Ether(base64_bytes(pakt)) for pakt in
> list_of_packets_base64]
> diff --git a/dts/tests/TestSuite_smoke_tests.py
> b/dts/tests/TestSuite_smoke_tests.py
> index bc3a2a6bf9..a029a564eb 100644
> --- a/dts/tests/TestSuite_smoke_tests.py
> +++ b/dts/tests/TestSuite_smoke_tests.py
> @@ -136,7 +136,7 @@ def test_device_bound_to_driver(self) -> None:
>              # with the address for the nic we are on in the loop and then
> captures the
>              # name of the driver in a group
>              devbind_info_for_nic = re.search(
> -                f"{nic.pci}[^\\n]*drv=([\\d\\w-]*) [^\\n]*",
> +                fr"{nic.pci}[^\\n]*drv=([\\d\\w-]*) [^\\n]*",
>                  all_nics_in_dpdk_devbind,
>              )
>              self.verify(
> --
> 2.47.1
>
>

[-- Attachment #2: Type: text/html, Size: 3902 bytes --]