[dpdk-dev] [RFC] A REST API based daemon of dpdk-devbind.py

[dpdk-dev] [RFC] A REST API based daemon of dpdk-devbind.py

[Muhammad Ahmad]

Hi,
I have observed that the dpdk-devbind.py is an interactive tool and
not quite useful for automated testing in CI/CD. We are planning to
create “dpdk-usertools-server” for making it streamlined.
The idea is to create a separate application, which provides REST interface to:
    - Query the current status of devices and drivers
    - Bind and unbind devices to different drivers

Future extensions will be to make dpdk-pmdinfo.py, cpu_layout.py and
dpdk-setup.sh more automation friendly.
=================Details of API====================
-Query Status of devices:
    Fetch the status of all devices or of a given group, supported
groups are: network, crypto, event and mempool devices.
        URL: /devices
        Method:GET
        URL Params: type=[all|net|baseband|crypto|event|mempool|compress]
        Sample call: curl http://127.0.0.1:5000/devices?type=crypto&type=net
        Success Response:
            HTTP response code: 200
            Content:
            [
             {
             "pci":"0000:b7:00.3",
             "type":"net",
             "infoString":"Ethernet Connection X722 for 10GbE SFP+ 37d0",
             "interface":"eno8",
             "driver":"i40e",
             "unusedDriver":"igb_uio",
             "active":false
             }
           ]
       Error response:
           HTTP Code: 400 BAD REQUEST
           Content:
           {
            "code": 1,
            "message": "invalid type"
           }


-Bind/Unbind device with a driver:
    Bind the PCI devices to the given driver or “none” to unbind
        URL:/devices/<pci>
        Method: PUT
        URL Prams: PCI=[PCI address]
        Sample call: curl http://127.0.0.1:5000/devices/0000:b7:00.3
--header "Content-Type: application/json" --request PUT --data
'{driver:"igb_uio", force: false}'
        Success Response:
            HTTP response code: 200
            Content:
            {
             "pci":"0000:b7:00.3",
             "type":"net",
             "infoString":"Ethernet Connection X722 for 10GbE SFP+ 37d0",
             "interface":"",
             "driver":"igb_uio",
             "unusedDriver":"i40e",
             "active":false
             }
       Error response:
           Code: 400 BAD REQUEST
           Content:
          {
           "code": 1,
           "message": "invalid id"
          }

                 OR
          Code: 400 BAD REQUEST
          Content:
          {
          "code": 2,
          "message": "invalid driver"
         }

                 OR
         Code: 405 METHOD NOT ALLOWED
         Content:
         {
          "code": 3,
           "message": "PCI Device active run with force=TRUE"
          }

=================++++++++====================

Signed-off-by: Muhammad Ahmad <muhammad.ahmad@emumba.com>

Re: [dpdk-dev] [RFC] A REST API based daemon of dpdk-devbind.py

[Bruce Richardson]

On Wed, Mar 04, 2020 at 12:11:05PM +0500, Muhammad Ahmad wrote:
> Hi,
> I have observed that the dpdk-devbind.py is an interactive tool and
> not quite useful for automated testing in CI/CD. We are planning to
> create “dpdk-usertools-server” for making it streamlined.
> The idea is to create a separate application, which provides REST interface to:
>     - Query the current status of devices and drivers
>     - Bind and unbind devices to different drivers
> 
> Future extensions will be to make dpdk-pmdinfo.py, cpu_layout.py and
> dpdk-setup.sh more automation friendly.

Hi,

I'm not sure what you mean by an "interactive tool", since devbind contains
no menus or interactive prompts. It's a commandline tool driven by
parameters passed - in the same way as ethtool, ip, and various other
standard Linux tools. I personally use it regularly in scripts, e.g. the
rc.local script on my test system to automatically adjust bindings on boot.

That being said, I can see why someone might want a rest tool to query and
adjust these settings, I'm just not sure if DPDK is the best place to host
such scripts. Would that functionality not be better built into other
existing tools for configuring and managing systems?

Regards,
/Bruce

Re: [dpdk-dev] [RFC] A REST API based daemon of dpdk-devbind.py

[Stephen Hemminger]

On Wed, 4 Mar 2020 12:11:05 +0500
Muhammad Ahmad <muhammad.ahmad@emumba.com> wrote:

> Hi,
> I have observed that the dpdk-devbind.py is an interactive tool and
> not quite useful for automated testing in CI/CD. We are planning to
> create “dpdk-usertools-server” for making it streamlined.
> The idea is to create a separate application, which provides REST interface to:
>     - Query the current status of devices and drivers
>     - Bind and unbind devices to different drivers
> 
> Future extensions will be to make dpdk-pmdinfo.py, cpu_layout.py and
> dpdk-setup.sh more automation friendly.
> =================Details of API====================
> -Query Status of devices:
>     Fetch the status of all devices or of a given group, supported
> groups are: network, crypto, event and mempool devices.
>         URL: /devices
>         Method:GET
>         URL Params: type=[all|net|baseband|crypto|event|mempool|compress]
>         Sample call: curl http://127.0.0.1:5000/devices?type=crypto&type=net
>         Success Response:
>             HTTP response code: 200
>             Content:
>             [
>              {
>              "pci":"0000:b7:00.3",
>              "type":"net",
>              "infoString":"Ethernet Connection X722 for 10GbE SFP+ 37d0",
>              "interface":"eno8",
>              "driver":"i40e",
>              "unusedDriver":"igb_uio",
>              "active":false
>              }
>            ]
>        Error response:
>            HTTP Code: 400 BAD REQUEST
>            Content:
>            {
>             "code": 1,
>             "message": "invalid type"
>            }
> 
> 
> -Bind/Unbind device with a driver:
>     Bind the PCI devices to the given driver or “none” to unbind
>         URL:/devices/<pci>
>         Method: PUT
>         URL Prams: PCI=[PCI address]
>         Sample call: curl http://127.0.0.1:5000/devices/0000:b7:00.3
> --header "Content-Type: application/json" --request PUT --data
> '{driver:"igb_uio", force: false}'
>         Success Response:
>             HTTP response code: 200
>             Content:
>             {
>              "pci":"0000:b7:00.3",
>              "type":"net",
>              "infoString":"Ethernet Connection X722 for 10GbE SFP+ 37d0",
>              "interface":"",
>              "driver":"igb_uio",
>              "unusedDriver":"i40e",
>              "active":false
>              }
>        Error response:
>            Code: 400 BAD REQUEST
>            Content:
>           {
>            "code": 1,
>            "message": "invalid id"
>           }
> 
>                  OR
>           Code: 400 BAD REQUEST
>           Content:
>           {
>           "code": 2,
>           "message": "invalid driver"
>          }
> 
>                  OR
>          Code: 405 METHOD NOT ALLOWED
>          Content:
>          {
>           "code": 3,
>            "message": "PCI Device active run with force=TRUE"
>           }
> 
> =================++++++++====================
> 
> Signed-off-by: Muhammad Ahmad <muhammad.ahmad@emumba.com>

There already is a better generic tool driverctl https://gitlab.com/driverctl/driverctl
It is already in many distributions

I would recommend using that instead of extending the limited DPDK tool.

Re: [dpdk-dev] [RFC] A REST API based daemon of dpdk-devbind.py

[Muhammad Ahmad]

Thanks for your comments.

"Better" may be subjective as the tool we are working on and using
within out infrastructure provides a RESTful API interface whereas
"driverctl" as you said seems more generic.

"driverctl" is an external tool and isn't part of DPDK toolset, what
we are providing here is more suitable for DPDK specific automated
testing, this is more of an argument of suitability of CLI interface
tools vs REST API tools which is what the industry preferred interface
"is/maybe".

We believed this is more suitable for "Automated Testing and
Integration" use-cases, and integrating various scripts in one main as
modules made more sense to us. I understand the tools functionality
does not directly affect DPDK as they aren't a part of core codebase,
but they certainly impact how DPDK is used.

Please let me know if this isn't something to be accepted in DPDK
upstream, we can then perhaps push it to upstream pip repo as a python
module or a CLI tool as a third-party DPDK tool.

Thanks
-Ahmad


On Wed, Mar 4, 2020 at 9:09 PM Stephen Hemminger
<stephen@networkplumber.org> wrote:
>
> On Wed, 4 Mar 2020 12:11:05 +0500
> Muhammad Ahmad <muhammad.ahmad@emumba.com> wrote:
>
> > Hi,
> > I have observed that the dpdk-devbind.py is an interactive tool and
> > not quite useful for automated testing in CI/CD. We are planning to
> > create “dpdk-usertools-server” for making it streamlined.
> > The idea is to create a separate application, which provides REST interface to:
> >     - Query the current status of devices and drivers
> >     - Bind and unbind devices to different drivers
> >
> > Future extensions will be to make dpdk-pmdinfo.py, cpu_layout.py and
> > dpdk-setup.sh more automation friendly.
> > =================Details of API====================
> > -Query Status of devices:
> >     Fetch the status of all devices or of a given group, supported
> > groups are: network, crypto, event and mempool devices.
> >         URL: /devices
> >         Method:GET
> >         URL Params: type=[all|net|baseband|crypto|event|mempool|compress]
> >         Sample call: curl http://127.0.0.1:5000/devices?type=crypto&type=net
> >         Success Response:
> >             HTTP response code: 200
> >             Content:
> >             [
> >              {
> >              "pci":"0000:b7:00.3",
> >              "type":"net",
> >              "infoString":"Ethernet Connection X722 for 10GbE SFP+ 37d0",
> >              "interface":"eno8",
> >              "driver":"i40e",
> >              "unusedDriver":"igb_uio",
> >              "active":false
> >              }
> >            ]
> >        Error response:
> >            HTTP Code: 400 BAD REQUEST
> >            Content:
> >            {
> >             "code": 1,
> >             "message": "invalid type"
> >            }
> >
> >
> > -Bind/Unbind device with a driver:
> >     Bind the PCI devices to the given driver or “none” to unbind
> >         URL:/devices/<pci>
> >         Method: PUT
> >         URL Prams: PCI=[PCI address]
> >         Sample call: curl http://127.0.0.1:5000/devices/0000:b7:00.3
> > --header "Content-Type: application/json" --request PUT --data
> > '{driver:"igb_uio", force: false}'
> >         Success Response:
> >             HTTP response code: 200
> >             Content:
> >             {
> >              "pci":"0000:b7:00.3",
> >              "type":"net",
> >              "infoString":"Ethernet Connection X722 for 10GbE SFP+ 37d0",
> >              "interface":"",
> >              "driver":"igb_uio",
> >              "unusedDriver":"i40e",
> >              "active":false
> >              }
> >        Error response:
> >            Code: 400 BAD REQUEST
> >            Content:
> >           {
> >            "code": 1,
> >            "message": "invalid id"
> >           }
> >
> >                  OR
> >           Code: 400 BAD REQUEST
> >           Content:
> >           {
> >           "code": 2,
> >           "message": "invalid driver"
> >          }
> >
> >                  OR
> >          Code: 405 METHOD NOT ALLOWED
> >          Content:
> >          {
> >           "code": 3,
> >            "message": "PCI Device active run with force=TRUE"
> >           }
> >
> > =================++++++++====================
> >
> > Signed-off-by: Muhammad Ahmad <muhammad.ahmad@emumba.com>
>
> There already is a better generic tool driverctl https://gitlab.com/driverctl/driverctl
> It is already in many distributions
>
> I would recommend using that instead of extending the limited DPDK tool.