[dpdk-dev] [PATCH 0/9] python2 deprecation notice

[dpdk-dev] [PATCH 0/9] python2 deprecation notice

[Louise Kilheeney]

This patchset adds deprecation notices to python scripts,
warning of the removal of python2 support from the DPDK 20.11 release.

Louise Kilheeney (9):
  usertools/cpu_layout: add python2 deprecation notice
  usertools/dpdk-telemetry-client: python2 deprecation notice
  usertools/dpdk-devbind: add python2 deprecation notice
  devtools/update_version_map: add python2 deprecation notice
  app/test-cmdline: add python2 deprecation notice
  app/test: add python2 deprecation notice
  usertools/dpdk-pmdinfo: add python2 deprecation notice
  app/test-bbdev: python3 compatibility changes
  app/test-bbdev: add python2 deprecation notice

 app/test-bbdev/test-bbdev.py       | 9 +++++++--
 app/test-cmdline/cmdline_test.py   | 3 +++
 app/test/autotest.py               | 4 ++++
 devtools/update_version_map_abi.py | 4 ++++
 usertools/cpu_layout.py            | 4 ++++
 usertools/dpdk-devbind.py          | 4 ++++
 usertools/dpdk-pmdinfo.py          | 4 +++-
 usertools/dpdk-telemetry-client.py | 4 ++++
 8 files changed, 33 insertions(+), 3 deletions(-)

-- 
2.17.1

[dpdk-dev] [PATCH 1/9] usertools/cpu_layout: add python2 deprecation notice

[Louise Kilheeney]

add python2 deprecation notice

Signed-off-by: Louise Kilheeney <louise.kilheeney@intel.com>
---
 usertools/cpu_layout.py | 4 ++++
 1 file changed, 4 insertions(+)

diff --git a/usertools/cpu_layout.py b/usertools/cpu_layout.py
index 6f129b1db..5423c7965 100755
--- a/usertools/cpu_layout.py
+++ b/usertools/cpu_layout.py
@@ -10,6 +10,10 @@
 except NameError:
     xrange = range # Python 3
 
+if sys.version_info.major < 3:
+    print("WARNING: Python 2 is deprecated for use in DPDK, and will not work in future releases.", file=sys.stderr)
+    print("Please use Python 3 instead", file=sys.stderr)
+
 sockets = []
 cores = []
 core_map = {}
-- 
2.17.1

[dpdk-dev] [PATCH 2/9] usertools/dpdk-telemetry-client: python2 deprecation notice

[Louise Kilheeney]

add python2 depecation notice

Cc: Kevin Laatz <kevin.laatz@intel.com>

Signed-off-by: Louise Kilheeney <louise.kilheeney@intel.com>
---
 usertools/dpdk-telemetry-client.py | 4 ++++
 1 file changed, 4 insertions(+)

diff --git a/usertools/dpdk-telemetry-client.py b/usertools/dpdk-telemetry-client.py
index 35edb7cd2..98d28fa89 100755
--- a/usertools/dpdk-telemetry-client.py
+++ b/usertools/dpdk-telemetry-client.py
@@ -23,6 +23,10 @@
 except NameError:
     raw_input = input  # Python 3
 
+if sys.version_info.major < 3:
+    print("WARNING: Python 2 is deprecated for use in DPDK, and will not work in future releases.", file=sys.stderr)
+    print("Please use Python 3 instead", file=sys.stderr)
+
 class Socket:
 
     def __init__(self):
-- 
2.17.1

[dpdk-dev] [PATCH 3/9] usertools/dpdk-devbind: add python2 deprecation notice

[Louise Kilheeney]

add python2 deprecation notice

Signed-off-by: Louise Kilheeney <louise.kilheeney@intel.com>
---
 usertools/dpdk-devbind.py | 4 ++++
 1 file changed, 4 insertions(+)

diff --git a/usertools/dpdk-devbind.py b/usertools/dpdk-devbind.py
index dc008823f..86b6b53c4 100755
--- a/usertools/dpdk-devbind.py
+++ b/usertools/dpdk-devbind.py
@@ -10,6 +10,10 @@
 import subprocess
 from os.path import exists, abspath, dirname, basename
 
+if sys.version_info.major < 3:
+    print("WARNING: Python 2 is deprecated for use in DPDK, and will not work in future releases.", file=sys.stderr)
+    print("Please use Python 3 instead", file=sys.stderr)
+
 # The PCI base class for all devices
 network_class = {'Class': '02', 'Vendor': None, 'Device': None,
                     'SVendor': None, 'SDevice': None}
-- 
2.17.1

[dpdk-dev] [PATCH 4/9] devtools/update_version_map: add python2 deprecation notice

[Louise Kilheeney]

Cc: Neil Horman <nhorman@tuxdriver.com>
Cc: Ray Kinsella <mdr@ashroe.eu>

Signed-off-by: Louise Kilheeney <louise.kilheeney@intel.com>
---
 devtools/update_version_map_abi.py | 4 ++++
 1 file changed, 4 insertions(+)

diff --git a/devtools/update_version_map_abi.py b/devtools/update_version_map_abi.py
index e2104e61e..80a61641e 100755
--- a/devtools/update_version_map_abi.py
+++ b/devtools/update_version_map_abi.py
@@ -160,6 +160,10 @@ def __generate_internal_abi(f_out, lines):
     print("};", file=f_out)
 
 def __main():
+    if sys.version_info.major < 3:
+        print("WARNING: Python 2 is deprecated for use in DPDK, and will not work in future releases.", file=sys.stderr)
+        print("Please use Python 3 instead", file=sys.stderr)
+
     arg_parser = argparse.ArgumentParser(
         description='Merge versions in linker version script.')
 
-- 
2.17.1

Re: [dpdk-dev] [PATCH 4/9] devtools/update_version_map: add python2 deprecation notice

[Neil Horman]

On Fri, Jul 10, 2020 at 11:10:50AM +0100, Louise Kilheeney wrote:
> Cc: Neil Horman <nhorman@tuxdriver.com>
> Cc: Ray Kinsella <mdr@ashroe.eu>
> 
> Signed-off-by: Louise Kilheeney <louise.kilheeney@intel.com>
> ---
>  devtools/update_version_map_abi.py | 4 ++++
>  1 file changed, 4 insertions(+)
> 
> diff --git a/devtools/update_version_map_abi.py b/devtools/update_version_map_abi.py
> index e2104e61e..80a61641e 100755
> --- a/devtools/update_version_map_abi.py
> +++ b/devtools/update_version_map_abi.py
> @@ -160,6 +160,10 @@ def __generate_internal_abi(f_out, lines):
>      print("};", file=f_out)
>  
>  def __main():
> +    if sys.version_info.major < 3:
> +        print("WARNING: Python 2 is deprecated for use in DPDK, and will not work in future releases.", file=sys.stderr)
> +        print("Please use Python 3 instead", file=sys.stderr)
> +
>      arg_parser = argparse.ArgumentParser(
>          description='Merge versions in linker version script.')
>  
> -- 
> 2.17.1
> 
> 
Acked-by: Neil Horman <nhorman@tuxdriver.com><

Re: [dpdk-dev] [PATCH 4/9] devtools/update_version_map: add python2 deprecation notice

[Kinsella, Ray]

On 10/07/2020 11:10, Louise Kilheeney wrote:
> Cc: Neil Horman <nhorman@tuxdriver.com>
> Cc: Ray Kinsella <mdr@ashroe.eu>
> 
> Signed-off-by: Louise Kilheeney <louise.kilheeney@intel.com>
> ---
>  devtools/update_version_map_abi.py | 4 ++++
>  1 file changed, 4 insertions(+)
> 
> diff --git a/devtools/update_version_map_abi.py b/devtools/update_version_map_abi.py
> index e2104e61e..80a61641e 100755
> --- a/devtools/update_version_map_abi.py
> +++ b/devtools/update_version_map_abi.py
> @@ -160,6 +160,10 @@ def __generate_internal_abi(f_out, lines):
>      print("};", file=f_out)
>  
>  def __main():
> +    if sys.version_info.major < 3:
> +        print("WARNING: Python 2 is deprecated for use in DPDK, and will not work in future releases.", file=sys.stderr)
> +        print("Please use Python 3 instead", file=sys.stderr)
> +
>      arg_parser = argparse.ArgumentParser(
>          description='Merge versions in linker version script.')
>  
> 

Is the simpler way to resolve this, for the script be explicit about its requirements.

Something like 
#!/usr/bin/env python3

I also noted recently that rpm packaging requirements, seem to demands this from scripts now?

Thanks,

Ray K

[dpdk-dev] [PATCH 5/9] app/test-cmdline: add python2 deprecation notice

[Louise Kilheeney]

Cc: Olivier Matz <olivier.matz@6wind.com>

Signed-off-by: Louise Kilheeney <louise.kilheeney@intel.com>
---
 app/test-cmdline/cmdline_test.py | 3 +++
 1 file changed, 3 insertions(+)

diff --git a/app/test-cmdline/cmdline_test.py b/app/test-cmdline/cmdline_test.py
index 3a8fac426..954428e2b 100755
--- a/app/test-cmdline/cmdline_test.py
+++ b/app/test-cmdline/cmdline_test.py
@@ -19,6 +19,9 @@ def runTest(child, test):
         return 0
     child.expect(test["Result"], 1)
 
+if sys.version_info.major < 3:
+    print("WARNING: Python 2 is deprecated for use in DPDK, and will not work in future releases.", file=sys.stderr)
+    print("Please use Python 3 instead", file=sys.stderr)
 
 #
 # history test is a special case
-- 
2.17.1

[dpdk-dev] [PATCH 6/9] app/test: add python2 deprecation notice

[Louise Kilheeney]

add python2 deprecation notice

Signed-off-by: Louise Kilheeney <louise.kilheeney@intel.com>
---
 app/test/autotest.py | 4 ++++
 1 file changed, 4 insertions(+)

diff --git a/app/test/autotest.py b/app/test/autotest.py
index b42f48879..cf7584ccd 100644
--- a/app/test/autotest.py
+++ b/app/test/autotest.py
@@ -17,6 +17,10 @@ def usage():
     usage()
     sys.exit(1)
 
+if sys.version_info.major < 3:
+    print("WARNING: Python 2 is deprecated for use in DPDK, and will not work in future releases.", file=sys.stderr)
+    print("Please use Python 3 instead", file=sys.stderr)
+
 target = sys.argv[2]
 
 test_whitelist = None
-- 
2.17.1

[dpdk-dev] [PATCH 7/9] usertools/dpdk-pmdinfo: add python2 deprecation notice

[Louise Kilheeney]

Cc: Neil Horman <nhorman@tuxdriver.com>

Signed-off-by: Louise Kilheeney <louise.kilheeney@intel.com>
---
 usertools/dpdk-pmdinfo.py | 4 +++-
 1 file changed, 3 insertions(+), 1 deletion(-)

diff --git a/usertools/dpdk-pmdinfo.py b/usertools/dpdk-pmdinfo.py
index 12f20735e..f9ed75517 100755
--- a/usertools/dpdk-pmdinfo.py
+++ b/usertools/dpdk-pmdinfo.py
@@ -28,7 +28,9 @@
 pcidb = None
 
 # ===========================================
-
+if sys.version_info.major < 3:
+        print("WARNING: Python 2 is deprecated for use in DPDK, and will not work in future releases.", file=sys.stderr)
+        print("Please use Python 3 instead", file=sys.stderr)
 
 class Vendor:
     """
-- 
2.17.1

Re: [dpdk-dev] [PATCH 7/9] usertools/dpdk-pmdinfo: add python2 deprecation notice

[Neil Horman]

On Fri, Jul 10, 2020 at 11:10:53AM +0100, Louise Kilheeney wrote:
> Cc: Neil Horman <nhorman@tuxdriver.com>
> 
> Signed-off-by: Louise Kilheeney <louise.kilheeney@intel.com>
> ---
>  usertools/dpdk-pmdinfo.py | 4 +++-
>  1 file changed, 3 insertions(+), 1 deletion(-)
> 
> diff --git a/usertools/dpdk-pmdinfo.py b/usertools/dpdk-pmdinfo.py
> index 12f20735e..f9ed75517 100755
> --- a/usertools/dpdk-pmdinfo.py
> +++ b/usertools/dpdk-pmdinfo.py
> @@ -28,7 +28,9 @@
>  pcidb = None
>  
>  # ===========================================
> -
> +if sys.version_info.major < 3:
> +        print("WARNING: Python 2 is deprecated for use in DPDK, and will not work in future releases.", file=sys.stderr)
> +        print("Please use Python 3 instead", file=sys.stderr)
>  
>  class Vendor:
>      """
> -- 
> 2.17.1
> 
> 
Acked-by: Neil Horman <nhorman@tuxdriver.com>

[dpdk-dev] [PATCH 8/9] app/test-bbdev: python3 compatibility changes

[Louise Kilheeney]

use of the print function required for python3
compatibility.

Cc: Nicolas Chautru <nicolas.chautru@intel.com>

Signed-off-by: Louise Kilheeney <louise.kilheeney@intel.com>
---
 app/test-bbdev/test-bbdev.py | 5 +++--
 1 file changed, 3 insertions(+), 2 deletions(-)

diff --git a/app/test-bbdev/test-bbdev.py b/app/test-bbdev/test-bbdev.py
index 0194be046..e127fb2eb 100755
--- a/app/test-bbdev/test-bbdev.py
+++ b/app/test-bbdev/test-bbdev.py
@@ -3,6 +3,7 @@
 # SPDX-License-Identifier: BSD-3-Clause
 # Copyright(c) 2017 Intel Corporation
 
+from __future__ import print_function
 import sys
 import os
 import argparse
@@ -12,7 +13,7 @@
 from threading import Timer
 
 def kill(process):
-    print "ERROR: Test app timed out"
+    print("ERROR: Test app timed out")
     process.kill()
 
 if "RTE_SDK" in os.environ:
@@ -66,7 +67,7 @@ def kill(process):
 args = parser.parse_args()
 
 if not os.path.exists(args.testapp_path):
-    print "No such file: " + args.testapp_path
+    print("No such file: " + args.testapp_path)
     sys.exit(1)
 
 params = [args.testapp_path]
-- 
2.17.1

[dpdk-dev] [PATCH 9/9] app/test-bbdev: add python2 deprecation notice

[Louise Kilheeney]

Cc: Nicolas Chautru <nicolas.chautru@intel.com>

Signed-off-by: Louise Kilheeney <louise.kilheeney@intel.com>
---
 app/test-bbdev/test-bbdev.py | 4 ++++
 1 file changed, 4 insertions(+)

diff --git a/app/test-bbdev/test-bbdev.py b/app/test-bbdev/test-bbdev.py
index e127fb2eb..5ae2dc6c4 100755
--- a/app/test-bbdev/test-bbdev.py
+++ b/app/test-bbdev/test-bbdev.py
@@ -16,6 +16,10 @@ def kill(process):
     print("ERROR: Test app timed out")
     process.kill()
 
+if sys.version_info.major < 3:
+    print("WARNING: Python 2 is deprecated for use in DPDK, and will not work in future releases.", file=sys.stderr)
+    print("Please use Python 3 instead", file=sys.stderr)
+
 if "RTE_SDK" in os.environ:
     dpdk_path = os.environ["RTE_SDK"]
 else:
-- 
2.17.1

Re: [dpdk-dev] [PATCH 9/9] app/test-bbdev: add python2 deprecation notice

[Chautru, Nicolas]

> From: Kilheeney, Louise <louise.kilheeney@intel.com>
> 
> Cc: Nicolas Chautru <nicolas.chautru@intel.com>
> 
> Signed-off-by: Louise Kilheeney <louise.kilheeney@intel.com>
> ---
>  app/test-bbdev/test-bbdev.py | 4 ++++
>  1 file changed, 4 insertions(+)
> 
> diff --git a/app/test-bbdev/test-bbdev.py b/app/test-bbdev/test-bbdev.py
> index e127fb2eb..5ae2dc6c4 100755
> --- a/app/test-bbdev/test-bbdev.py
> +++ b/app/test-bbdev/test-bbdev.py
> @@ -16,6 +16,10 @@ def kill(process):
>      print("ERROR: Test app timed out")
>      process.kill()
> 
> +if sys.version_info.major < 3:
> +    print("WARNING: Python 2 is deprecated for use in DPDK, and will not
> work in future releases.", file=sys.stderr)
> +    print("Please use Python 3 instead", file=sys.stderr)
> +
>  if "RTE_SDK" in os.environ:
>      dpdk_path = os.environ["RTE_SDK"]
>  else:
> --

Tested the serie on my end to double check. 
Thanks

Acked-by: Nicolas Chautru <nicolas.chautru@intel.com>


> 2.17.1

Re: [dpdk-dev] [PATCH 0/9] python2 deprecation notice

[Bruce Richardson]

On Fri, Jul 10, 2020 at 11:10:46AM +0100, Louise Kilheeney wrote:
> This patchset adds deprecation notices to python scripts,
> warning of the removal of python2 support from the DPDK 20.11 release.
> 
> Louise Kilheeney (9):
>   usertools/cpu_layout: add python2 deprecation notice
>   usertools/dpdk-telemetry-client: python2 deprecation notice
>   usertools/dpdk-devbind: add python2 deprecation notice
>   devtools/update_version_map: add python2 deprecation notice
>   app/test-cmdline: add python2 deprecation notice
>   app/test: add python2 deprecation notice
>   usertools/dpdk-pmdinfo: add python2 deprecation notice
>   app/test-bbdev: python3 compatibility changes
>   app/test-bbdev: add python2 deprecation notice
> 
>  app/test-bbdev/test-bbdev.py       | 9 +++++++--
>  app/test-cmdline/cmdline_test.py   | 3 +++
>  app/test/autotest.py               | 4 ++++
>  devtools/update_version_map_abi.py | 4 ++++
>  usertools/cpu_layout.py            | 4 ++++
>  usertools/dpdk-devbind.py          | 4 ++++
>  usertools/dpdk-pmdinfo.py          | 4 +++-
>  usertools/dpdk-telemetry-client.py | 4 ++++
>  8 files changed, 33 insertions(+), 3 deletions(-)
> 
> -- 
Thanks for setting us up for Python 2 support removal.

Series-acked-by: Bruce Richardson <bruce.richardson@intel.com>

Re: [dpdk-dev] [PATCH 0/9] python2 deprecation notice

[Robin Jarry]

Hi Louise,

2020-07-10, Louise Kilheeney:
> This patchset adds deprecation notices to python scripts,
> warning of the removal of python2 support from the DPDK 20.11 release.

While showing warnings to users about Python 2 support drop in 20.11 is
good, it seems like the shebangs in a lot of these scripts still refer
to "python".

dpdk$ git describe 
v20.05-623-geff30b59cc2e
dpdk$ git grep '#.*!.*python\>'
app/test-bbdev/test-bbdev.py:1:#!/usr/bin/env python
app/test-cmdline/cmdline_test.py:1:#!/usr/bin/env python
app/test/autotest.py:1:#!/usr/bin/env python
buildtools/map_to_win.py:1:#!/usr/bin/env python
config/arm/armv8_machine.py:1:#!/usr/bin/python
devtools/update_version_map_abi.py:1:#!/usr/bin/env python
usertools/cpu_layout.py:1:#!/usr/bin/env python
usertools/dpdk-devbind.py:1:#! /usr/bin/env python
usertools/dpdk-pmdinfo.py:1:#!/usr/bin/env python
usertools/dpdk-telemetry-client.py:1:#! /usr/bin/env python

On many distros, "python" still points (as of today) to python2. You
series will cause warnings that cannot be avoided.

Also, on some distros, "python" does not exist at all (RHEL 8 and CentOS
8 for example). And only "python2" or "python3" are available.

I wonder if it would not be better to find a way to make these shebangs
"dynamic" somehow. It is not trivial and I don't see any other solution
than plain modification of the shebangs at build time.

However, there is no way (to my knowledge) to specify which version of
python is "selected" during the build.

Does anyone have a proper solution?

-- 
Robin

Re: [dpdk-dev] [PATCH 0/9] python2 deprecation notice

[Bruce Richardson]

On Fri, Jul 10, 2020 at 03:33:58PM +0200, Robin Jarry wrote:
> Hi Louise,
> 
> 2020-07-10, Louise Kilheeney:
> > This patchset adds deprecation notices to python scripts,
> > warning of the removal of python2 support from the DPDK 20.11 release.
> 
> While showing warnings to users about Python 2 support drop in 20.11 is
> good, it seems like the shebangs in a lot of these scripts still refer
> to "python".
> 
> dpdk$ git describe 
> v20.05-623-geff30b59cc2e
> dpdk$ git grep '#.*!.*python\>'
> app/test-bbdev/test-bbdev.py:1:#!/usr/bin/env python
> app/test-cmdline/cmdline_test.py:1:#!/usr/bin/env python
> app/test/autotest.py:1:#!/usr/bin/env python
> buildtools/map_to_win.py:1:#!/usr/bin/env python
> config/arm/armv8_machine.py:1:#!/usr/bin/python
> devtools/update_version_map_abi.py:1:#!/usr/bin/env python
> usertools/cpu_layout.py:1:#!/usr/bin/env python
> usertools/dpdk-devbind.py:1:#! /usr/bin/env python
> usertools/dpdk-pmdinfo.py:1:#!/usr/bin/env python
> usertools/dpdk-telemetry-client.py:1:#! /usr/bin/env python
> 
> On many distros, "python" still points (as of today) to python2. You
> series will cause warnings that cannot be avoided.
> 
> Also, on some distros, "python" does not exist at all (RHEL 8 and CentOS
> 8 for example). And only "python2" or "python3" are available.
> 
> I wonder if it would not be better to find a way to make these shebangs
> "dynamic" somehow. It is not trivial and I don't see any other solution
> than plain modification of the shebangs at build time.
> 
> However, there is no way (to my knowledge) to specify which version of
> python is "selected" during the build.
> 
> Does anyone have a proper solution?
> 
That's a good point and I'm not sure there is a clean solution to it -
especially if there is no "python" without a 2 or 3 at the end?

Theoretically we could put something like this at the start of each script
to ensure it runs with python3 if available and "python" otherwise:

from __future__ import print_function
import sys
import os
import distutils.spawn

if sys.version_info.major < 3:
    py3 = distutils.spawn.find_executable("python3")
    if py3:
        print("Running", ' '.join([py3] + sys.argv))
        sys.exit(os.system(' '.join([py3] + sys.argv)))

print("Running", str(sys.argv), "with python version",\
    str(sys.version_info.major) + "." + str(sys.version_info.minor))

It doesn't solve the problem of those systems where /usr/bin/env python
returns nothing, but in those cases the script won't run at all, so the
user needs to explicitly call via python2/3 directly and the deprecation
notice is correct if they pick the wrong one. For most distros where
"python" alone does exist, the above code will work to prevent the
deprecation notice appearing except where the user does not have python3
available.

If we want to go with a build-time selection approach, we should just
change the defaults in all scripts to python3 and only change them back in
the Makefile build if python3 is not found. [Meson requires python3, so we
can ignore any checks there.]

Regards,
/Bruce

Re: [dpdk-dev] [PATCH 0/9] python2 deprecation notice

[Bruce Richardson]

On Fri, Jul 10, 2020 at 03:33:58PM +0200, Robin Jarry wrote:
> Hi Louise,
> 
> 2020-07-10, Louise Kilheeney:
> > This patchset adds deprecation notices to python scripts,
> > warning of the removal of python2 support from the DPDK 20.11 release.
> 
> While showing warnings to users about Python 2 support drop in 20.11 is
> good, it seems like the shebangs in a lot of these scripts still refer
> to "python".
> 
> dpdk$ git describe 
> v20.05-623-geff30b59cc2e
> dpdk$ git grep '#.*!.*python\>'
> app/test-bbdev/test-bbdev.py:1:#!/usr/bin/env python
> app/test-cmdline/cmdline_test.py:1:#!/usr/bin/env python
> app/test/autotest.py:1:#!/usr/bin/env python
> buildtools/map_to_win.py:1:#!/usr/bin/env python
> config/arm/armv8_machine.py:1:#!/usr/bin/python
> devtools/update_version_map_abi.py:1:#!/usr/bin/env python
> usertools/cpu_layout.py:1:#!/usr/bin/env python
> usertools/dpdk-devbind.py:1:#! /usr/bin/env python
> usertools/dpdk-pmdinfo.py:1:#!/usr/bin/env python
> usertools/dpdk-telemetry-client.py:1:#! /usr/bin/env python
> 
> On many distros, "python" still points (as of today) to python2. You
> series will cause warnings that cannot be avoided.
> 
> Also, on some distros, "python" does not exist at all (RHEL 8 and CentOS
> 8 for example). And only "python2" or "python3" are available.
> 
> I wonder if it would not be better to find a way to make these shebangs
> "dynamic" somehow. It is not trivial and I don't see any other solution
> than plain modification of the shebangs at build time.
> 
> However, there is no way (to my knowledge) to specify which version of
> python is "selected" during the build.
> 
> Does anyone have a proper solution?
>

In the absense of a "proper" solution, is the simplest option to change the
shebangs to all be python3, but leave the python2 compatibility in place,
and add the warnings for anyone running it explicitly using python2? 

Re: [dpdk-dev] [PATCH 0/9] python2 deprecation notice

[Stephen Hemminger]

On Mon, 13 Jul 2020 10:21:49 +0100
Bruce Richardson <bruce.richardson@intel.com> wrote:

> On Fri, Jul 10, 2020 at 03:33:58PM +0200, Robin Jarry wrote:
> > Hi Louise,
> > 
> > 2020-07-10, Louise Kilheeney:  
> > > This patchset adds deprecation notices to python scripts,
> > > warning of the removal of python2 support from the DPDK 20.11 release.  
> > 
> > While showing warnings to users about Python 2 support drop in 20.11 is
> > good, it seems like the shebangs in a lot of these scripts still refer
> > to "python".
> > 
> > dpdk$ git describe 
> > v20.05-623-geff30b59cc2e
> > dpdk$ git grep '#.*!.*python\>'
> > app/test-bbdev/test-bbdev.py:1:#!/usr/bin/env python
> > app/test-cmdline/cmdline_test.py:1:#!/usr/bin/env python
> > app/test/autotest.py:1:#!/usr/bin/env python
> > buildtools/map_to_win.py:1:#!/usr/bin/env python
> > config/arm/armv8_machine.py:1:#!/usr/bin/python
> > devtools/update_version_map_abi.py:1:#!/usr/bin/env python
> > usertools/cpu_layout.py:1:#!/usr/bin/env python
> > usertools/dpdk-devbind.py:1:#! /usr/bin/env python
> > usertools/dpdk-pmdinfo.py:1:#!/usr/bin/env python
> > usertools/dpdk-telemetry-client.py:1:#! /usr/bin/env python
> > 
> > On many distros, "python" still points (as of today) to python2. You
> > series will cause warnings that cannot be avoided.
> > 
> > Also, on some distros, "python" does not exist at all (RHEL 8 and CentOS
> > 8 for example). And only "python2" or "python3" are available.
> > 
> > I wonder if it would not be better to find a way to make these shebangs
> > "dynamic" somehow. It is not trivial and I don't see any other solution
> > than plain modification of the shebangs at build time.
> > 
> > However, there is no way (to my knowledge) to specify which version of
> > python is "selected" during the build.
> > 
> > Does anyone have a proper solution?
> >  
> 
> In the absense of a "proper" solution, is the simplest option to change the
> shebangs to all be python3, but leave the python2 compatibility in place,
> and add the warnings for anyone running it explicitly using python2? 

The standard way to use python3 that I have seen is:

#! /usr/bin/env python3

Re: [dpdk-dev] [PATCH 0/9] python2 deprecation notice

[Robin Jarry]

2020-07-13, Bruce Richardson:
> In the absense of a "proper" solution, is the simplest option to
> change the shebangs to all be python3, but leave the python2
> compatibility in place, and add the warnings for anyone running it
> explicitly using python2?

Personally, I am OK with this solution. However, this is quite radical.

I guess we would need to make it obvious to everyone using 20.08 that
all Python scripts will now use python3 unless they explicitly replace
the shebangs when installing them.

I wonder if that would not be better to add a build option to force
these shebangs.

-- 
Robin

Re: [dpdk-dev] [PATCH 0/9] python2 deprecation notice

[Robin Jarry]

2020-07-13, Bruce Richardson:
> In the absense of a "proper" solution, is the simplest option to
> change the shebangs to all be python3, but leave the python2
> compatibility in place, and add the warnings for anyone running it
> explicitly using python2? 

I have found a hacky[1] but somewhat not too ugly way to dynamically use
any available python interpreter.

Basically, the scripts shebangs become:

 #!/bin/sh
 # shell script hack to dynamically use any available python interpreter
 ''':'
 if command -pv python3 >/dev/null 2>&1; then
   exec python3 "$0" "$@"
 else
   exec python2 "$0" "$@"
 fi
 '''
 # real python code starts here

This hack obviously would remain only for the time where we still have
python 2 support and can be removed after 20.08 is released.

What do you think?

[1] https://unix.stackexchange.com/a/66242

-- 
Robin

Re: [dpdk-dev] [PATCH 0/9] python2 deprecation notice

[Stephen Hemminger]

On Thu, 16 Jul 2020 16:44:29 +0200
Robin Jarry <robin.jarry@6wind.com> wrote:

> 2020-07-13, Bruce Richardson:
> > In the absense of a "proper" solution, is the simplest option to
> > change the shebangs to all be python3, but leave the python2
> > compatibility in place, and add the warnings for anyone running it
> > explicitly using python2?   
> 
> I have found a hacky[1] but somewhat not too ugly way to dynamically use
> any available python interpreter.
> 
> Basically, the scripts shebangs become:
> 
>  #!/bin/sh
>  # shell script hack to dynamically use any available python interpreter
>  ''':'
>  if command -pv python3 >/dev/null 2>&1; then
>    exec python3 "$0" "$@"
>  else
>    exec python2 "$0" "$@"
>  fi
>  '''
>  # real python code starts here
> 
> This hack obviously would remain only for the time where we still have
> python 2 support and can be removed after 20.08 is released.
> 
> What do you think?
> 
> [1] https://unix.stackexchange.com/a/66242
> 

No.

Re: [dpdk-dev] [PATCH 0/9] python2 deprecation notice

[Bruce Richardson]

On Thu, Jul 16, 2020 at 02:01:21PM -0700, Stephen Hemminger wrote:
> On Thu, 16 Jul 2020 16:44:29 +0200
> Robin Jarry <robin.jarry@6wind.com> wrote:
> 
> > 2020-07-13, Bruce Richardson:
> > > In the absense of a "proper" solution, is the simplest option to
> > > change the shebangs to all be python3, but leave the python2
> > > compatibility in place, and add the warnings for anyone running it
> > > explicitly using python2?   
> > 
> > I have found a hacky[1] but somewhat not too ugly way to dynamically use
> > any available python interpreter.
> > 
> > Basically, the scripts shebangs become:
> > 
> >  #!/bin/sh
> >  # shell script hack to dynamically use any available python interpreter
> >  ''':'
> >  if command -pv python3 >/dev/null 2>&1; then
> >    exec python3 "$0" "$@"
> >  else
> >    exec python2 "$0" "$@"
> >  fi
> >  '''
> >  # real python code starts here
> > 
> > This hack obviously would remain only for the time where we still have
> > python 2 support and can be removed after 20.08 is released.
> > 
> > What do you think?
> > 
> > [1] https://unix.stackexchange.com/a/66242
> > 
> 
> No.

I could live with this, but I think that python3 is widespread enough now
to just go with the simpler solution of python3 by default while still
allowing others to run with python2 explicitly if they so desire.

I think we have covered a number of options, but I'm not sure we have
converged on an agreed solution. Here are the options I would consider:

0. Do nothing to the code, and just print deprecation notice in docs. This
  will violate the previously documented plan, but won't break anything for
  anyone.
1. Just add deprecation notice but leave shebang lines unchanged, meaning
  lots of people see the notice but don't need to care since they just call
  the script.
2. As #1, but check if python3 is available and only print deprecation
  notice if not
3. Add deprecation notice, and change shebang lines to use python3. This
  means that anyone who does not have python3 installed will see a breaking
  change
4. Change shebangs to use python3 and detect at "make" build time if
  python3 is available. If py3 is unavailable, adjust shebang and add
  deprecation warning on "make install".
5. Use shell-hack above.

Any other options I'm missing?

For me, I think I can live with all options bar #1. Probably my top
preference would be #4 or #2, with #0 and #4 next best thereafter.

Regards,
/Bruce

Re: [dpdk-dev] [PATCH 0/9] python2 deprecation notice

[Stephen Hemminger]

On Fri, 17 Jul 2020 13:42:09 +0100
Bruce Richardson <bruce.richardson@intel.com> wrote:

> On Thu, Jul 16, 2020 at 02:01:21PM -0700, Stephen Hemminger wrote:
> > On Thu, 16 Jul 2020 16:44:29 +0200
> > Robin Jarry <robin.jarry@6wind.com> wrote:
> >   
> > > 2020-07-13, Bruce Richardson:  
> > > > In the absense of a "proper" solution, is the simplest option to
> > > > change the shebangs to all be python3, but leave the python2
> > > > compatibility in place, and add the warnings for anyone running it
> > > > explicitly using python2?     
> > > 
> > > I have found a hacky[1] but somewhat not too ugly way to dynamically use
> > > any available python interpreter.
> > > 
> > > Basically, the scripts shebangs become:
> > > 
> > >  #!/bin/sh
> > >  # shell script hack to dynamically use any available python interpreter
> > >  ''':'
> > >  if command -pv python3 >/dev/null 2>&1; then
> > >    exec python3 "$0" "$@"
> > >  else
> > >    exec python2 "$0" "$@"
> > >  fi
> > >  '''
> > >  # real python code starts here
> > > 
> > > This hack obviously would remain only for the time where we still have
> > > python 2 support and can be removed after 20.08 is released.
> > > 
> > > What do you think?
> > > 
> > > [1] https://unix.stackexchange.com/a/66242
> > >   
> > 
> > No.  
> 
> I could live with this, but I think that python3 is widespread enough now
> to just go with the simpler solution of python3 by default while still
> allowing others to run with python2 explicitly if they so desire.
> 
> I think we have covered a number of options, but I'm not sure we have
> converged on an agreed solution. Here are the options I would consider:
> 
> 0. Do nothing to the code, and just print deprecation notice in docs. This
>   will violate the previously documented plan, but won't break anything for
>   anyone.
> 1. Just add deprecation notice but leave shebang lines unchanged, meaning
>   lots of people see the notice but don't need to care since they just call
>   the script.
> 2. As #1, but check if python3 is available and only print deprecation
>   notice if not
> 3. Add deprecation notice, and change shebang lines to use python3. This
>   means that anyone who does not have python3 installed will see a breaking
>   change
> 4. Change shebangs to use python3 and detect at "make" build time if
>   python3 is available. If py3 is unavailable, adjust shebang and add
>   deprecation warning on "make install".
> 5. Use shell-hack above.
> 
> Any other options I'm missing?
> 
> For me, I think I can live with all options bar #1. Probably my top
> preference would be #4 or #2, with #0 and #4 next best thereafter.
> 

6. Change shebangs to use python3 and don't bother detecting.
   Since distributions will have python3.

7. Change shebangs and keep the python2 compat imports

8. Other alternative I have seen in code is to use
   try block in the import section:

try:
    # for Python2
    from Tkinter import *
except ImportError:
    # for Python3
    from tkinter import *

But supporting python2 for a longer time makes no sense.

I would go for #0 for 20.08 and full python3 in 20.11 (#3)

Re: [dpdk-dev] [PATCH 0/9] python2 deprecation notice

[Bruce Richardson]

On Fri, Jul 17, 2020 at 08:06:23AM -0700, Stephen Hemminger wrote:
> On Fri, 17 Jul 2020 13:42:09 +0100
> Bruce Richardson <bruce.richardson@intel.com> wrote:
> 
> > On Thu, Jul 16, 2020 at 02:01:21PM -0700, Stephen Hemminger wrote:
> > > On Thu, 16 Jul 2020 16:44:29 +0200
> > > Robin Jarry <robin.jarry@6wind.com> wrote:
> > >   
> > > > 2020-07-13, Bruce Richardson:  
> > > > > In the absense of a "proper" solution, is the simplest option to
> > > > > change the shebangs to all be python3, but leave the python2
> > > > > compatibility in place, and add the warnings for anyone running it
> > > > > explicitly using python2?     
> > > > 
> > > > I have found a hacky[1] but somewhat not too ugly way to dynamically use
> > > > any available python interpreter.
> > > > 
> > > > Basically, the scripts shebangs become:
> > > > 
> > > >  #!/bin/sh
> > > >  # shell script hack to dynamically use any available python interpreter
> > > >  ''':'
> > > >  if command -pv python3 >/dev/null 2>&1; then
> > > >    exec python3 "$0" "$@"
> > > >  else
> > > >    exec python2 "$0" "$@"
> > > >  fi
> > > >  '''
> > > >  # real python code starts here
> > > > 
> > > > This hack obviously would remain only for the time where we still have
> > > > python 2 support and can be removed after 20.08 is released.
> > > > 
> > > > What do you think?
> > > > 
> > > > [1] https://unix.stackexchange.com/a/66242
> > > >   
> > > 
> > > No.  
> > 
> > I could live with this, but I think that python3 is widespread enough now
> > to just go with the simpler solution of python3 by default while still
> > allowing others to run with python2 explicitly if they so desire.
> > 
> > I think we have covered a number of options, but I'm not sure we have
> > converged on an agreed solution. Here are the options I would consider:
> > 
> > 0. Do nothing to the code, and just print deprecation notice in docs. This
> >   will violate the previously documented plan, but won't break anything for
> >   anyone.
> > 1. Just add deprecation notice but leave shebang lines unchanged, meaning
> >   lots of people see the notice but don't need to care since they just call
> >   the script.
> > 2. As #1, but check if python3 is available and only print deprecation
> >   notice if not
> > 3. Add deprecation notice, and change shebang lines to use python3. This
> >   means that anyone who does not have python3 installed will see a breaking
> >   change
> > 4. Change shebangs to use python3 and detect at "make" build time if
> >   python3 is available. If py3 is unavailable, adjust shebang and add
> >   deprecation warning on "make install".
> > 5. Use shell-hack above.
> > 
> > Any other options I'm missing?
> > 
> > For me, I think I can live with all options bar #1. Probably my top
> > preference would be #4 or #2, with #0 and #4 next best thereafter.
> > 
> 
> 6. Change shebangs to use python3 and don't bother detecting.
>    Since distributions will have python3.
> 

This was what our originally patch submission was to do, but it was pointed
out that we said we would have deprecation notices in the scripts in 20.08
and only remove fully in 20.11

> 7. Change shebangs and keep the python2 compat imports

This is pretty much the same as #3. The python2 compat imports etc. will be
removed in 20.11 anyway, so we can ignore those I think.

> 
> 8. Other alternative I have seen in code is to use
>    try block in the import section:
> 
> try:
>     # for Python2
>     from Tkinter import *
> except ImportError:
>     # for Python3
>     from tkinter import *
> 
> But supporting python2 for a longer time makes no sense.a

This is already done in DPDK. All scripts already support both python2 and
python3 AFAIK. If any don't, it's a bug.

> 
> I would go for #0 for 20.08 and full python3 in 20.11 (#3)
> 

Full python3 in 20.11 is indeed the already agreed plan. All python2
compatibility will be removed then.

/Bruce

Re: [dpdk-dev] [PATCH 0/9] python2 deprecation notice

[Thomas Monjalon]

I vote for applying this patch series,
because it does not change what we have,
except printing a warning in case python3
is neither explicitly called, nor the default.

I consider it is a good step,
and I don't see the extra warning as a severe annoyance.


17/07/2020 18:24, Bruce Richardson:
> On Fri, Jul 17, 2020 at 08:06:23AM -0700, Stephen Hemminger wrote:
> > On Fri, 17 Jul 2020 13:42:09 +0100
> > Bruce Richardson <bruce.richardson@intel.com> wrote:
> > > I think we have covered a number of options, but I'm not sure we have
> > > converged on an agreed solution. Here are the options I would consider:
> > > 
> > > 0. Do nothing to the code, and just print deprecation notice in docs. This
> > >   will violate the previously documented plan, but won't break anything for
> > >   anyone.
> > > 1. Just add deprecation notice but leave shebang lines unchanged, meaning
> > >   lots of people see the notice but don't need to care since they just call
> > >   the script.
> > > 2. As #1, but check if python3 is available and only print deprecation
> > >   notice if not
> > > 3. Add deprecation notice, and change shebang lines to use python3. This
> > >   means that anyone who does not have python3 installed will see a breaking
> > >   change
> > > 4. Change shebangs to use python3 and detect at "make" build time if
> > >   python3 is available. If py3 is unavailable, adjust shebang and add
> > >   deprecation warning on "make install".
> > > 5. Use shell-hack above.
> > > 
> > > Any other options I'm missing?
> > > 
> > > For me, I think I can live with all options bar #1. Probably my top
> > > preference would be #4 or #2, with #0 and #4 next best thereafter.
> > > 
> > 
> > 6. Change shebangs to use python3 and don't bother detecting.
> >    Since distributions will have python3.
> > 
> 
> This was what our originally patch submission was to do, but it was pointed
> out that we said we would have deprecation notices in the scripts in 20.08
> and only remove fully in 20.11
> 
> > 7. Change shebangs and keep the python2 compat imports
> 
> This is pretty much the same as #3. The python2 compat imports etc. will be
> removed in 20.11 anyway, so we can ignore those I think.
> 
> > 
> > 8. Other alternative I have seen in code is to use
> >    try block in the import section:
> > 
> > try:
> >     # for Python2
> >     from Tkinter import *
> > except ImportError:
> >     # for Python3
> >     from tkinter import *
> > 
> > But supporting python2 for a longer time makes no sense.a
> 
> This is already done in DPDK. All scripts already support both python2 and
> python3 AFAIK. If any don't, it's a bug.
> 
> > 
> > I would go for #0 for 20.08 and full python3 in 20.11 (#3)
> > 
> 
> Full python3 in 20.11 is indeed the already agreed plan. All python2
> compatibility will be removed then.

Re: [dpdk-dev] [PATCH 0/9] python2 deprecation notice

[David Marchand]

On Tue, Jul 21, 2020 at 10:25 PM Thomas Monjalon <thomas@monjalon.net> wrote:
>
> I vote for applying this patch series,
> because it does not change what we have,
> except printing a warning in case python3
> is neither explicitly called, nor the default.
>
> I consider it is a good step,
> and I don't see the extra warning as a severe annoyance.

Users can call the scripts with the right interpreter if the warning
is too much for them and they don't want to switch their whole system
to python3.
+1 for this series.


-- 
David Marchand

Re: [dpdk-dev] [PATCH 0/9] python2 deprecation notice

[David Marchand]

On Fri, Jul 10, 2020 at 12:11 PM Louise Kilheeney
<louise.kilheeney@intel.com> wrote:
>
> This patchset adds deprecation notices to python scripts,
> warning of the removal of python2 support from the DPDK 20.11 release.
>
> Louise Kilheeney (9):
>   usertools/cpu_layout: add python2 deprecation notice
>   usertools/dpdk-telemetry-client: python2 deprecation notice
>   usertools/dpdk-devbind: add python2 deprecation notice
>   devtools/update_version_map: add python2 deprecation notice
>   app/test-cmdline: add python2 deprecation notice
>   app/test: add python2 deprecation notice
>   usertools/dpdk-pmdinfo: add python2 deprecation notice
>   app/test-bbdev: python3 compatibility changes
>   app/test-bbdev: add python2 deprecation notice

Moved patch 8 as first patch then squashed the rest as a single patch.
Applied, thanks Louise.


-- 
David Marchand