[dpdk-dev] [PATCH] usertools: add telemetry python3 compatibility

[dpdk-dev] [PATCH] usertools: add telemetry python3 compatibility

[Ciara Power]

The client script for use with the telemetry library did not support
python3, as the data being sent over the socket was in string format.
Python3 requires the data be explicitly converted to bytes before being
sent.  Similarily, the received bytes need to be decoded into string
format.

Cc: stable@dpdk.org

Signed-off-by: Ciara Power <ciara.power@intel.com>
---
 usertools/dpdk-telemetry-client.py | 17 +++++++++--------
 1 file changed, 9 insertions(+), 8 deletions(-)

diff --git a/usertools/dpdk-telemetry-client.py b/usertools/dpdk-telemetry-client.py
index 290345dcc..71a8a8852 100755
--- a/usertools/dpdk-telemetry-client.py
+++ b/usertools/dpdk-telemetry-client.py
@@ -65,18 +65,19 @@ def register(self): # Connects a client to DPDK-instance
         self.socket.recv_fd.settimeout(2)
         self.socket.send_fd.connect("/var/run/dpdk/rte/telemetry")
         JSON = (API_REG + self.file_path + "\"}}")
-        self.socket.send_fd.sendall(JSON)
+        self.socket.send_fd.sendall(JSON.encode())
+
         self.socket.recv_fd.listen(1)
         self.socket.client_fd = self.socket.recv_fd.accept()[0]
 
     def unregister(self): # Unregister a given client
-        self.socket.client_fd.send(API_UNREG + self.file_path + "\"}}")
+        self.socket.client_fd.send((API_UNREG + self.file_path + "\"}}").encode())
         self.socket.client_fd.close()
 
     def requestMetrics(self): # Requests metrics for given client
-        self.socket.client_fd.send(METRICS_REQ)
-        data = self.socket.client_fd.recv(BUFFER_SIZE)
-        print("\nResponse: \n", str(data))
+        self.socket.client_fd.send(METRICS_REQ.encode())
+        data = self.socket.client_fd.recv(BUFFER_SIZE).decode()
+        print("\nResponse: \n", data)
 
     def repeatedlyRequestMetrics(self, sleep_time): # Recursively requests metrics for given client
         print("\nPlease enter the number of times you'd like to continuously request Metrics:")
@@ -88,9 +89,9 @@ def repeatedlyRequestMetrics(self, sleep_time): # Recursively requests metrics f
             time.sleep(sleep_time)
 
     def requestGlobalMetrics(self): #Requests global metrics for given client
-        self.socket.client_fd.send(GLOBAL_METRICS_REQ)
-        data = self.socket.client_fd.recv(BUFFER_SIZE)
-        print("\nResponse: \n", str(data))
+        self.socket.client_fd.send(GLOBAL_METRICS_REQ.encode())
+        data = self.socket.client_fd.recv(BUFFER_SIZE).decode()
+        print("\nResponse: \n", data)
 
     def interactiveMenu(self, sleep_time): # Creates Interactive menu within the script
         while self.choice != 4:
-- 
2.17.1

Re: [dpdk-dev] [dpdk-stable] [PATCH] usertools: add telemetry python3 compatibility

[Thomas Monjalon]

+Cc Robin, known Python expert :)

16/01/2020 18:24, Ciara Power:
> The client script for use with the telemetry library did not support
> python3, as the data being sent over the socket was in string format.
> Python3 requires the data be explicitly converted to bytes before being
> sent.  Similarily, the received bytes need to be decoded into string
> format.
> 
> Cc: stable@dpdk.org
> 
> Signed-off-by: Ciara Power <ciara.power@intel.com>
> ---
>  usertools/dpdk-telemetry-client.py | 17 +++++++++--------
>  1 file changed, 9 insertions(+), 8 deletions(-)
> 
> diff --git a/usertools/dpdk-telemetry-client.py b/usertools/dpdk-telemetry-client.py
> index 290345dcc..71a8a8852 100755
> --- a/usertools/dpdk-telemetry-client.py
> +++ b/usertools/dpdk-telemetry-client.py
> @@ -65,18 +65,19 @@ def register(self): # Connects a client to DPDK-instance
>          self.socket.recv_fd.settimeout(2)
>          self.socket.send_fd.connect("/var/run/dpdk/rte/telemetry")
>          JSON = (API_REG + self.file_path + "\"}}")
> -        self.socket.send_fd.sendall(JSON)
> +        self.socket.send_fd.sendall(JSON.encode())
> +
>          self.socket.recv_fd.listen(1)
>          self.socket.client_fd = self.socket.recv_fd.accept()[0]
>  
>      def unregister(self): # Unregister a given client
> -        self.socket.client_fd.send(API_UNREG + self.file_path + "\"}}")
> +        self.socket.client_fd.send((API_UNREG + self.file_path + "\"}}").encode())
>          self.socket.client_fd.close()
>  
>      def requestMetrics(self): # Requests metrics for given client
> -        self.socket.client_fd.send(METRICS_REQ)
> -        data = self.socket.client_fd.recv(BUFFER_SIZE)
> -        print("\nResponse: \n", str(data))
> +        self.socket.client_fd.send(METRICS_REQ.encode())
> +        data = self.socket.client_fd.recv(BUFFER_SIZE).decode()
> +        print("\nResponse: \n", data)
>  
>      def repeatedlyRequestMetrics(self, sleep_time): # Recursively requests metrics for given client
>          print("\nPlease enter the number of times you'd like to continuously request Metrics:")
> @@ -88,9 +89,9 @@ def repeatedlyRequestMetrics(self, sleep_time): # Recursively requests metrics f
>              time.sleep(sleep_time)
>  
>      def requestGlobalMetrics(self): #Requests global metrics for given client
> -        self.socket.client_fd.send(GLOBAL_METRICS_REQ)
> -        data = self.socket.client_fd.recv(BUFFER_SIZE)
> -        print("\nResponse: \n", str(data))
> +        self.socket.client_fd.send(GLOBAL_METRICS_REQ.encode())
> +        data = self.socket.client_fd.recv(BUFFER_SIZE).decode()
> +        print("\nResponse: \n", data)
>  
>      def interactiveMenu(self, sleep_time): # Creates Interactive menu within the script
>          while self.choice != 4:
> 

Re: [dpdk-dev] [dpdk-stable] [PATCH] usertools: add telemetry python3 compatibility

[Robin Jarry]

16/01/2020 18:24, Ciara Power:
> The client script for use with the telemetry library did not support
> python3, as the data being sent over the socket was in string format.
> Python3 requires the data be explicitly converted to bytes before being
> sent.  Similarily, the received bytes need to be decoded into string
> format.
> 
> Cc: stable@dpdk.org
> 
> Signed-off-by: Ciara Power <ciara.power@intel.com>

Overall, it looks good to me. One minor grudge:

Mind that when using this script with python2, the literal strings
actually are bytes. This means that .encode() does not make any sense on
them. As it turns out, the str objects of python2 do have an .encode()
method that does not do anything (it returns the byte string object
unchanged), so calling it does not cause any problem.

Long story short, for consistency you should consider adding a future
import at the top:

    from __future__ import unicode_literals

So that all literal strings are unicode with python2 as with python3.
See related commit 4da069194ef4 ("usertools: fix pmdinfo with python
3 and pyelftools>=0.24").

Maybe the patch title should contain the word "fix" and some Fixes:
lines as there already were some attemps to make this script python3
compatible.

Reviewed-by: Robin Jarry <robin.jarry@6wind.com>

-- 
Robin

[dpdk-dev] [PATCH v2] usertools: fix telemetry python3 compatibility

[Ciara Power]

The client script for use with the telemetry library did not support
Python3, as the data being sent over the socket was in string format.
Python3 requires the data be explicitly converted to bytes before being
sent. Similarily, the received bytes need to be decoded into string
format.

Fixes: 53f293c9a783 ("usertools: replace unsafe input function")
Fixes: fe35622659ed ("usertools: fix telemetry client with python 3")
Fixes: d1b94da4a4e0 ("usertools: add client script for telemetry")
Fixes: 4080e46c8078 ("telemetry: support global metrics")
Cc: andrius.sirvys@intel.com
Cc: robin.jarry@6wind.com
Cc: reshma.pattan@intel.com
Cc: stable@dpdk.org

Signed-off-by: Ciara Power <ciara.power@intel.com>
Reviewed-by: Robin Jarry <robin.jarry@6wind.com>

---

V2: added future import for unicode literals to ensure consistency of
data types for Python2 and Python3.
---
 usertools/dpdk-telemetry-client.py | 18 ++++++++++--------
 1 file changed, 10 insertions(+), 8 deletions(-)

diff --git a/usertools/dpdk-telemetry-client.py b/usertools/dpdk-telemetry-client.py
index 290345dcc..35edb7cd2 100755
--- a/usertools/dpdk-telemetry-client.py
+++ b/usertools/dpdk-telemetry-client.py
@@ -3,6 +3,7 @@
 # Copyright(c) 2018 Intel Corporation
 
 from __future__ import print_function
+from __future__ import unicode_literals
 
 import socket
 import os
@@ -65,18 +66,19 @@ def register(self): # Connects a client to DPDK-instance
         self.socket.recv_fd.settimeout(2)
         self.socket.send_fd.connect("/var/run/dpdk/rte/telemetry")
         JSON = (API_REG + self.file_path + "\"}}")
-        self.socket.send_fd.sendall(JSON)
+        self.socket.send_fd.sendall(JSON.encode())
+
         self.socket.recv_fd.listen(1)
         self.socket.client_fd = self.socket.recv_fd.accept()[0]
 
     def unregister(self): # Unregister a given client
-        self.socket.client_fd.send(API_UNREG + self.file_path + "\"}}")
+        self.socket.client_fd.send((API_UNREG + self.file_path + "\"}}").encode())
         self.socket.client_fd.close()
 
     def requestMetrics(self): # Requests metrics for given client
-        self.socket.client_fd.send(METRICS_REQ)
-        data = self.socket.client_fd.recv(BUFFER_SIZE)
-        print("\nResponse: \n", str(data))
+        self.socket.client_fd.send(METRICS_REQ.encode())
+        data = self.socket.client_fd.recv(BUFFER_SIZE).decode()
+        print("\nResponse: \n", data)
 
     def repeatedlyRequestMetrics(self, sleep_time): # Recursively requests metrics for given client
         print("\nPlease enter the number of times you'd like to continuously request Metrics:")
@@ -88,9 +90,9 @@ def repeatedlyRequestMetrics(self, sleep_time): # Recursively requests metrics f
             time.sleep(sleep_time)
 
     def requestGlobalMetrics(self): #Requests global metrics for given client
-        self.socket.client_fd.send(GLOBAL_METRICS_REQ)
-        data = self.socket.client_fd.recv(BUFFER_SIZE)
-        print("\nResponse: \n", str(data))
+        self.socket.client_fd.send(GLOBAL_METRICS_REQ.encode())
+        data = self.socket.client_fd.recv(BUFFER_SIZE).decode()
+        print("\nResponse: \n", data)
 
     def interactiveMenu(self, sleep_time): # Creates Interactive menu within the script
         while self.choice != 4:
-- 
2.17.1

Re: [dpdk-dev] [PATCH v2] usertools: fix telemetry python3 compatibility

[Laatz, Kevin]

> The client script for use with the telemetry library did not support
> Python3, as the data being sent over the socket was in string format.
> Python3 requires the data be explicitly converted to bytes before being
> sent. Similarily, the received bytes need to be decoded into string
> format.
> 
> Fixes: 53f293c9a783 ("usertools: replace unsafe input function")
> Fixes: fe35622659ed ("usertools: fix telemetry client with python 3")
> Fixes: d1b94da4a4e0 ("usertools: add client script for telemetry")
> Fixes: 4080e46c8078 ("telemetry: support global metrics")
> Cc: andrius.sirvys@intel.com
> Cc: robin.jarry@6wind.com
> Cc: reshma.pattan@intel.com
> Cc: stable@dpdk.org
> 
> Signed-off-by: Ciara Power <ciara.power@intel.com>
> Reviewed-by: Robin Jarry <robin.jarry@6wind.com>
> 
<snip>

Acked-by: Kevin Laatz <kevin.laatz@intel.com>

Re: [dpdk-dev] [PATCH v2] usertools: fix telemetry python3 compatibility

[Thomas Monjalon]

24/01/2020 11:11, Laatz, Kevin:
> > The client script for use with the telemetry library did not support
> > Python3, as the data being sent over the socket was in string format.
> > Python3 requires the data be explicitly converted to bytes before being
> > sent. Similarily, the received bytes need to be decoded into string
> > format.
> > 
> > Fixes: 53f293c9a783 ("usertools: replace unsafe input function")
> > Fixes: fe35622659ed ("usertools: fix telemetry client with python 3")
> > Fixes: d1b94da4a4e0 ("usertools: add client script for telemetry")
> > Fixes: 4080e46c8078 ("telemetry: support global metrics")
> > Cc: andrius.sirvys@intel.com
> > Cc: robin.jarry@6wind.com
> > Cc: reshma.pattan@intel.com
> > Cc: stable@dpdk.org
> > 
> > Signed-off-by: Ciara Power <ciara.power@intel.com>
> > Reviewed-by: Robin Jarry <robin.jarry@6wind.com>
> > 
> <snip>
> 
> Acked-by: Kevin Laatz <kevin.laatz@intel.com>

Applied, thanks