[dpdk-dev] Instability of port ids

[dpdk-dev] Instability of port ids

[Stephen Hemminger]

Several customers have reported similar issues with how the owned/stack device model
works in DPDK. With failsafe/tap and VF or netvsc and VF there are DPDK ports which
are marked as owned and therefore not visible.

The problem is the application has to guess and workaround these port values in
the port mask that gets passed in on command line. This means a working application
has to modify its startup script to run on Azure. Worse the actual port values
change based on the number of NIC's configured.

Overall this is a nuisance for users. The whole DPDK port index concept is a bad
design. In Linux/BSD there is ifindex, but few applications care, they all use names
which is better. Very very few application care that eth1 is ifindex 4.

The whole assignment of ports is a mess as well since it is based on probe order
and that is based on PCI order, and not anything dependable. It gets worse with
command line arguments, vdev, owned devices etc.

All I can think of is that:
  * DPDK network devices need to have human readable names. current PCI is not good.
  * The names need to be repeatable/persistent. udev names are probably better than anything so far.
    Or bsd style names but they end up being device dependent.
  * The API to get from name to port needs to easy to use and the preferred method.
  * All examples and documentation should avoid using port index directly.
    You need port for fast rx/tx but setup should be by name.

Re: [dpdk-dev] Instability of port ids

[Wang, Haiyue]

> -----Original Message-----
> From: dev [mailto:dev-bounces@dpdk.org] On Behalf Of Stephen Hemminger
> Sent: Saturday, May 18, 2019 02:47
> To: dev@dpdk.org
> Subject: [dpdk-dev] Instability of port ids
> 
> Several customers have reported similar issues with how the owned/stack device model
> works in DPDK. With failsafe/tap and VF or netvsc and VF there are DPDK ports which
> are marked as owned and therefore not visible.
> 
> The problem is the application has to guess and workaround these port values in
> the port mask that gets passed in on command line. This means a working application
> has to modify its startup script to run on Azure. Worse the actual port values
> change based on the number of NIC's configured.
> 
> Overall this is a nuisance for users. The whole DPDK port index concept is a bad
> design. In Linux/BSD there is ifindex, but few applications care, they all use names
> which is better. Very very few application care that eth1 is ifindex 4.
> 
> The whole assignment of ports is a mess as well since it is based on probe order
> and that is based on PCI order, and not anything dependable. It gets worse with
> command line arguments, vdev, owned devices etc.
> 
> All I can think of is that:
>   * DPDK network devices need to have human readable names. current PCI is not good.
>   * The names need to be repeatable/persistent. udev names are probably better than anything so far.
>     Or bsd style names but they end up being device dependent.
>   * The API to get from name to port needs to easy to use and the preferred method.
>   * All examples and documentation should avoid using port index directly.
>     You need port for fast rx/tx but setup should be by name.

idea from system like enp24s0f0 ?
https://github.com/systemd/systemd/blob/master/docs/PREDICTABLE_INTERFACE_NAMES.md
https://github.com/systemd/systemd/blob/master/src/udev/udev-builtin-net_id.c

Re: [dpdk-dev] Instability of port ids

[Stephen Hemminger]

On Sat, 18 May 2019 06:03:22 +0000
"Wang, Haiyue" <haiyue.wang@intel.com> wrote:

> > -----Original Message-----
> > From: dev [mailto:dev-bounces@dpdk.org] On Behalf Of Stephen Hemminger
> > Sent: Saturday, May 18, 2019 02:47
> > To: dev@dpdk.org
> > Subject: [dpdk-dev] Instability of port ids
> > 
> > Several customers have reported similar issues with how the owned/stack device model
> > works in DPDK. With failsafe/tap and VF or netvsc and VF there are DPDK ports which
> > are marked as owned and therefore not visible.
> > 
> > The problem is the application has to guess and workaround these port values in
> > the port mask that gets passed in on command line. This means a working application
> > has to modify its startup script to run on Azure. Worse the actual port values
> > change based on the number of NIC's configured.
> > 
> > Overall this is a nuisance for users. The whole DPDK port index concept is a bad
> > design. In Linux/BSD there is ifindex, but few applications care, they all use names
> > which is better. Very very few application care that eth1 is ifindex 4.
> > 
> > The whole assignment of ports is a mess as well since it is based on probe order
> > and that is based on PCI order, and not anything dependable. It gets worse with
> > command line arguments, vdev, owned devices etc.
> > 
> > All I can think of is that:
> >   * DPDK network devices need to have human readable names. current PCI is not good.
> >   * The names need to be repeatable/persistent. udev names are probably better than anything so far.
> >     Or bsd style names but they end up being device dependent.
> >   * The API to get from name to port needs to easy to use and the preferred method.
> >   * All examples and documentation should avoid using port index directly.
> >     You need port for fast rx/tx but setup should be by name.  
> 
> idea from system like enp24s0f0 ?
> https://github.com/systemd/systemd/blob/master/docs/PREDICTABLE_INTERFACE_NAMES.md
> https://github.com/systemd/systemd/blob/master/src/udev/udev-builtin-net_id.c
> 

The other suggestion is to use an algorithm like VPP which generates
  TenGigabitEthernet0/2/2

Re: [dpdk-dev] Instability of port ids

[Jay Rolette]

On Fri, May 17, 2019 at 1:46 PM Stephen Hemminger <
stephen@networkplumber.org> wrote:

> Several customers have reported similar issues with how the owned/stack
> device model
> works in DPDK. With failsafe/tap and VF or netvsc and VF there are DPDK
> ports which
> are marked as owned and therefore not visible.
>
> The problem is the application has to guess and workaround these port
> values in
> the port mask that gets passed in on command line. This means a working
> application
> has to modify its startup script to run on Azure. Worse the actual port
> values
> change based on the number of NIC's configured.
>
> Overall this is a nuisance for users. The whole DPDK port index concept is
> a bad
> design. In Linux/BSD there is ifindex, but few applications care, they all
> use names
> which is better. Very very few application care that eth1 is ifindex 4.
>
> The whole assignment of ports is a mess as well since it is based on probe
> order
> and that is based on PCI order, and not anything dependable. It gets worse
> with
> command line arguments, vdev, owned devices etc.
>
> All I can think of is that:
>   * DPDK network devices need to have human readable names. current PCI is
> not good.
>   * The names need to be repeatable/persistent. udev names are probably
> better than anything so far.
>     Or bsd style names but they end up being device dependent.
>   * The API to get from name to port needs to easy to use and the
> preferred method.
>   * All examples and documentation should avoid using port index directly.
>     You need port for fast rx/tx but setup should be by name.
>

+1 agree completely

Re: [dpdk-dev] Instability of port ids

[Ray Kinsella]

On 18/05/2019 14:33, Stephen Hemminger wrote:
> On Sat, 18 May 2019 06:03:22 +0000
> "Wang, Haiyue" <haiyue.wang@intel.com> wrote:
> 
>>> -----Original Message-----
>>> From: dev [mailto:dev-bounces@dpdk.org] On Behalf Of Stephen Hemminger
>>> Sent: Saturday, May 18, 2019 02:47
>>> To: dev@dpdk.org
>>> Subject: [dpdk-dev] Instability of port ids
>>>
>>> Several customers have reported similar issues with how the owned/stack device model
>>> works in DPDK. With failsafe/tap and VF or netvsc and VF there are DPDK ports which
>>> are marked as owned and therefore not visible.
>>>
>>> The problem is the application has to guess and workaround these port values in
>>> the port mask that gets passed in on command line. This means a working application
>>> has to modify its startup script to run on Azure. Worse the actual port values
>>> change based on the number of NIC's configured.
>>>
>>> Overall this is a nuisance for users. The whole DPDK port index concept is a bad
>>> design. In Linux/BSD there is ifindex, but few applications care, they all use names
>>> which is better. Very very few application care that eth1 is ifindex 4.
>>>
>>> The whole assignment of ports is a mess as well since it is based on probe order
>>> and that is based on PCI order, and not anything dependable. It gets worse with
>>> command line arguments, vdev, owned devices etc.
>>>
>>> All I can think of is that:
>>>   * DPDK network devices need to have human readable names. current PCI is not good.
>>>   * The names need to be repeatable/persistent. udev names are probably better than anything so far.
>>>     Or bsd style names but they end up being device dependent.
>>>   * The API to get from name to port needs to easy to use and the preferred method.
>>>   * All examples and documentation should avoid using port index directly.
>>>     You need port for fast rx/tx but setup should be by name.  
>>
>> idea from system like enp24s0f0 ?
>> https://github.com/systemd/systemd/blob/master/docs/PREDICTABLE_INTERFACE_NAMES.md
>> https://github.com/systemd/systemd/blob/master/src/udev/udev-builtin-net_id.c
>>
> 
> The other suggestion is to use an algorithm like VPP which generates
>   TenGigabitEthernet0/2/2
> 

That was going to be my suggestion also ... :-)
Either/or I would be fine with, as you say indexes are unreliable, and
that only get's worse as you look at ephemeral interfaces like veth.

One comment though.
What I really do like about uDev/SystemD is that it is highly user
configurable, that the end user can set a udev to influence the naming
scheme.

Thanks,

Ray K

Ray K