Getting network port ID by ethdev port ID

Getting network port ID by ethdev port ID

[Ivan Malov]

Dear community,

Is there any means in DPDK to discover relationship between
network/physical ports of the given adapter/board and
etdevs deployed in DPDK application on top of it?

For example, in Linux, there are facilities like

> /sys/class/net/<iface>/phys_port_name
> /sys/class/net/<iface>/dev_port

and

> devlink port show

Do we have something similar in DPDK?

If no, would the feature be worthwhile implementing?

Interested to hear your input on this.
Thank you.

Re: Getting network port ID by ethdev port ID

[Thomas Monjalon]

Hello,

05/06/2023 15:09, Ivan Malov:
> Dear community,
> 
> Is there any means in DPDK to discover relationship between
> network/physical ports of the given adapter/board and
> etdevs deployed in DPDK application on top of it?
> 
> For example, in Linux, there are facilities like
> 
> > /sys/class/net/<iface>/phys_port_name
> > /sys/class/net/<iface>/dev_port
> 
> and
> 
> > devlink port show
> 
> Do we have something similar in DPDK?

We can get the device name of a port:
	rte_eth_dev_get_name_by_port()

> If no, would the feature be worthwhile implementing?

We may have discrepancies in different device classes.
Feel free to make a global status of device names.

Re: Getting network port ID by ethdev port ID

[Ivan Malov]

Hi Thomas,

Thanks for responding. Please see below.

On Mon, 5 Jun 2023, Thomas Monjalon wrote:

> Hello,
>
> 05/06/2023 15:09, Ivan Malov:
>> Dear community,
>>
>> Is there any means in DPDK to discover relationship between
>> network/physical ports of the given adapter/board and
>> etdevs deployed in DPDK application on top of it?
>>
>> For example, in Linux, there are facilities like
>>
>>> /sys/class/net/<iface>/phys_port_name
>>> /sys/class/net/<iface>/dev_port
>>
>> and
>>
>>> devlink port show
>>
>> Do we have something similar in DPDK?
>
> We can get the device name of a port:
> 	rte_eth_dev_get_name_by_port()

I'm afraid this won't do. Consider the following example.
Say, there's a NIC with two network ports and two PFs,
0000:01:00.0 and 0000:01:00.1. The user plugs these
PFs to DPDK application. The resulting ethdev IDs
are 0 and 1. If the user invokes the said API,
they will get 0000:01:00.0 and 0000:01:00.1.
But that's not what is really needed.

We seek a means to get the network port ID by
ethdev ID. For example, something like this:
- get_netport_by_ethdev(0) => 0
- get_netport_by_ethdev(1) => 1

If two different PCI functions are associated with the
same network port (0, for instance), this should be
- get_netport_by_ethdev(0) => 0
- get_netport_by_ethdev(1) => 0

Do we have something like that in DPDK?

>
>> If no, would the feature be worthwhile implementing?
>
> We may have discrepancies in different device classes.

I mean precisely "ethdev"s. I do realise, though, that
an ethdev may be backed by a vdev (af_xdp, etc.) = in
such cases the assumed "get_netport" method could
just return (-ENOTSUP). What do you think?

> Feel free to make a global status of device names.
>
Could you please elaborate on this?

Thank you.

Re: Getting network port ID by ethdev port ID

[Thomas Monjalon]

05/06/2023 16:03, Ivan Malov:
> Hi Thomas,
> 
> Thanks for responding. Please see below.
> 
> On Mon, 5 Jun 2023, Thomas Monjalon wrote:
> 
> > Hello,
> >
> > 05/06/2023 15:09, Ivan Malov:
> >> Dear community,
> >>
> >> Is there any means in DPDK to discover relationship between
> >> network/physical ports of the given adapter/board and
> >> etdevs deployed in DPDK application on top of it?
> >>
> >> For example, in Linux, there are facilities like
> >>
> >>> /sys/class/net/<iface>/phys_port_name
> >>> /sys/class/net/<iface>/dev_port
> >>
> >> and
> >>
> >>> devlink port show
> >>
> >> Do we have something similar in DPDK?
> >
> > We can get the device name of a port:
> > 	rte_eth_dev_get_name_by_port()
> 
> I'm afraid this won't do. Consider the following example.
> Say, there's a NIC with two network ports and two PFs,
> 0000:01:00.0 and 0000:01:00.1. The user plugs these
> PFs to DPDK application. The resulting ethdev IDs
> are 0 and 1. If the user invokes the said API,
> they will get 0000:01:00.0 and 0000:01:00.1.
> But that's not what is really needed.
> 
> We seek a means to get the network port ID by
> ethdev ID. For example, something like this:
> - get_netport_by_ethdev(0) => 0
> - get_netport_by_ethdev(1) => 1
> 
> If two different PCI functions are associated with the
> same network port (0, for instance), this should be
> - get_netport_by_ethdev(0) => 0
> - get_netport_by_ethdev(1) => 0
> 
> Do we have something like that in DPDK?

No we don't have such underlying index.
I don't understand why it is needed.
To me the name is more informative than a number.


> >> If no, would the feature be worthwhile implementing?
> >
> > We may have discrepancies in different device classes.
> 
> I mean precisely "ethdev"s. I do realise, though, that
> an ethdev may be backed by a vdev (af_xdp, etc.) = in
> such cases the assumed "get_netport" method could
> just return (-ENOTSUP). What do you think?

Are you interested only in PCI devices? Looks limited.

> > Feel free to make a global status of device names.
> >
> Could you please elaborate on this?

I thought you wanted to have device name in all device classes.

Re: Getting network port ID by ethdev port ID

[Ivan Malov]

On Mon, 5 Jun 2023, Thomas Monjalon wrote:

> 05/06/2023 16:03, Ivan Malov:
>> Hi Thomas,
>>
>> Thanks for responding. Please see below.
>>
>> On Mon, 5 Jun 2023, Thomas Monjalon wrote:
>>
>>> Hello,
>>>
>>> 05/06/2023 15:09, Ivan Malov:
>>>> Dear community,
>>>>
>>>> Is there any means in DPDK to discover relationship between
>>>> network/physical ports of the given adapter/board and
>>>> etdevs deployed in DPDK application on top of it?
>>>>
>>>> For example, in Linux, there are facilities like
>>>>
>>>>> /sys/class/net/<iface>/phys_port_name
>>>>> /sys/class/net/<iface>/dev_port
>>>>
>>>> and
>>>>
>>>>> devlink port show
>>>>
>>>> Do we have something similar in DPDK?
>>>
>>> We can get the device name of a port:
>>> 	rte_eth_dev_get_name_by_port()
>>
>> I'm afraid this won't do. Consider the following example.
>> Say, there's a NIC with two network ports and two PFs,
>> 0000:01:00.0 and 0000:01:00.1. The user plugs these
>> PFs to DPDK application. The resulting ethdev IDs
>> are 0 and 1. If the user invokes the said API,
>> they will get 0000:01:00.0 and 0000:01:00.1.
>> But that's not what is really needed.
>>
>> We seek a means to get the network port ID by
>> ethdev ID. For example, something like this:
>> - get_netport_by_ethdev(0) => 0
>> - get_netport_by_ethdev(1) => 1
>>
>> If two different PCI functions are associated with the
>> same network port (0, for instance), this should be
>> - get_netport_by_ethdev(0) => 0
>> - get_netport_by_ethdev(1) => 0
>>
>> Do we have something like that in DPDK?
>
> No we don't have such underlying index.
> I don't understand why it is needed.
> To me the name is more informative than a number.
>
>
>>>> If no, would the feature be worthwhile implementing?
>>>
>>> We may have discrepancies in different device classes.
>>
>> I mean precisely "ethdev"s. I do realise, though, that
>> an ethdev may be backed by a vdev (af_xdp, etc.) = in
>> such cases the assumed "get_netport" method could
>> just return (-ENOTSUP). What do you think?
>
> Are you interested only in PCI devices? Looks limited.
>
>>> Feel free to make a global status of device names.
>>>
>> Could you please elaborate on this?
>
> I thought you wanted to have device name in all device classes.
>

No, Thomas, I meant obtaining the physical port ID, in other
words, ID of the port where the cable/wire is attached.
Not the "device name". As I said, it might be possible
to have multiple PFs/VFs associated with a single
physical/"uplink" port of the network card. In
this case, the "ethdev name" is just "PF name"
and not the "physical port name".

We need the latter. Physical port name/index.

Thank you.

Re: Getting network port ID by ethdev port ID

[Ivan Malov]

Sorry, I missed your question. See below.

On Mon, 5 Jun 2023, Thomas Monjalon wrote:

> 05/06/2023 16:03, Ivan Malov:
>> Hi Thomas,
>>
>> Thanks for responding. Please see below.
>>
>> On Mon, 5 Jun 2023, Thomas Monjalon wrote:
>>
>>> Hello,
>>>
>>> 05/06/2023 15:09, Ivan Malov:
>>>> Dear community,
>>>>
>>>> Is there any means in DPDK to discover relationship between
>>>> network/physical ports of the given adapter/board and
>>>> etdevs deployed in DPDK application on top of it?
>>>>
>>>> For example, in Linux, there are facilities like
>>>>
>>>>> /sys/class/net/<iface>/phys_port_name
>>>>> /sys/class/net/<iface>/dev_port
>>>>
>>>> and
>>>>
>>>>> devlink port show
>>>>
>>>> Do we have something similar in DPDK?
>>>
>>> We can get the device name of a port:
>>> 	rte_eth_dev_get_name_by_port()
>>
>> I'm afraid this won't do. Consider the following example.
>> Say, there's a NIC with two network ports and two PFs,
>> 0000:01:00.0 and 0000:01:00.1. The user plugs these
>> PFs to DPDK application. The resulting ethdev IDs
>> are 0 and 1. If the user invokes the said API,
>> they will get 0000:01:00.0 and 0000:01:00.1.
>> But that's not what is really needed.
>>
>> We seek a means to get the network port ID by
>> ethdev ID. For example, something like this:
>> - get_netport_by_ethdev(0) => 0
>> - get_netport_by_ethdev(1) => 1
>>
>> If two different PCI functions are associated with the
>> same network port (0, for instance), this should be
>> - get_netport_by_ethdev(0) => 0
>> - get_netport_by_ethdev(1) => 0
>>
>> Do we have something like that in DPDK?
>
> No we don't have such underlying index.
> I don't understand why it is needed.
> To me the name is more informative than a number.
>
>
>>>> If no, would the feature be worthwhile implementing?
>>>
>>> We may have discrepancies in different device classes.
>>
>> I mean precisely "ethdev"s. I do realise, though, that
>> an ethdev may be backed by a vdev (af_xdp, etc.) = in
>> such cases the assumed "get_netport" method could
>> just return (-ENOTSUP). What do you think?
>
> Are you interested only in PCI devices? Looks limited.

Theoretically, even a vdev may handle this request
appropriately. For example, a failsafe device may
ask its current underlying PCI device abot the
physical port ID in use. For af_xdp and the
likes, it's also possible. The PMD may
query sysfs to provide the value.

Strictly speaking, it's not limited, but the primary
use case is querying the phys. port ID for PFs, yes.

This information may be needed by some applications
that not only operate the higher-level ethdevs but
also take the real physical/wire interconnects
into account. It might be complex to explain
in a single email thread, though.

Previously, DPDK even used to have a flow action PHY_PORT.
Yes, it has been deprecated, but that's not a problem.
The information can be useful anyway.

>
>>> Feel free to make a global status of device names.
>>>
>> Could you please elaborate on this?
>
> I thought you wanted to have device name in all device classes.
>
>
>

Re: Getting network port ID by ethdev port ID

[Thomas Monjalon]

05/06/2023 16:29, Ivan Malov:
> Sorry, I missed your question. See below.
> 
> On Mon, 5 Jun 2023, Thomas Monjalon wrote:
> 
> > 05/06/2023 16:03, Ivan Malov:
> >> Hi Thomas,
> >>
> >> Thanks for responding. Please see below.
> >>
> >> On Mon, 5 Jun 2023, Thomas Monjalon wrote:
> >>
> >>> Hello,
> >>>
> >>> 05/06/2023 15:09, Ivan Malov:
> >>>> Dear community,
> >>>>
> >>>> Is there any means in DPDK to discover relationship between
> >>>> network/physical ports of the given adapter/board and
> >>>> etdevs deployed in DPDK application on top of it?
> >>>>
> >>>> For example, in Linux, there are facilities like
> >>>>
> >>>>> /sys/class/net/<iface>/phys_port_name
> >>>>> /sys/class/net/<iface>/dev_port
> >>>>
> >>>> and
> >>>>
> >>>>> devlink port show
> >>>>
> >>>> Do we have something similar in DPDK?
> >>>
> >>> We can get the device name of a port:
> >>> 	rte_eth_dev_get_name_by_port()
> >>
> >> I'm afraid this won't do. Consider the following example.
> >> Say, there's a NIC with two network ports and two PFs,
> >> 0000:01:00.0 and 0000:01:00.1. The user plugs these
> >> PFs to DPDK application. The resulting ethdev IDs
> >> are 0 and 1. If the user invokes the said API,
> >> they will get 0000:01:00.0 and 0000:01:00.1.
> >> But that's not what is really needed.
> >>
> >> We seek a means to get the network port ID by
> >> ethdev ID. For example, something like this:
> >> - get_netport_by_ethdev(0) => 0
> >> - get_netport_by_ethdev(1) => 1
> >>
> >> If two different PCI functions are associated with the
> >> same network port (0, for instance), this should be
> >> - get_netport_by_ethdev(0) => 0
> >> - get_netport_by_ethdev(1) => 0
> >>
> >> Do we have something like that in DPDK?
> >
> > No we don't have such underlying index.
> > I don't understand why it is needed.
> > To me the name is more informative than a number.
> >
> >
> >>>> If no, would the feature be worthwhile implementing?
> >>>
> >>> We may have discrepancies in different device classes.
> >>
> >> I mean precisely "ethdev"s. I do realise, though, that
> >> an ethdev may be backed by a vdev (af_xdp, etc.) = in
> >> such cases the assumed "get_netport" method could
> >> just return (-ENOTSUP). What do you think?
> >
> > Are you interested only in PCI devices? Looks limited.
> 
> Theoretically, even a vdev may handle this request
> appropriately. For example, a failsafe device may
> ask its current underlying PCI device abot the
> physical port ID in use. For af_xdp and the
> likes, it's also possible. The PMD may
> query sysfs to provide the value.
> 
> Strictly speaking, it's not limited, but the primary
> use case is querying the phys. port ID for PFs, yes.
> 
> This information may be needed by some applications
> that not only operate the higher-level ethdevs but
> also take the real physical/wire interconnects
> into account. It might be complex to explain
> in a single email thread, though.
> 
> Previously, DPDK even used to have a flow action PHY_PORT.
> Yes, it has been deprecated, but that's not a problem.
> The information can be useful anyway.

In this case, this is something the driver should fill in rte_eth_dev_info.
Note that we already have rte_eth_dev_info::if_index but it looks different.

Who would be responsible of the numbering of the physical port?
Should we report kernel numbering or do we need yet another numbering scheme?

Re: Getting network port ID by ethdev port ID

[Stephen Hemminger]

On Mon, 05 Jun 2023 18:03:14 +0200
Thomas Monjalon <thomas@monjalon.net> wrote:

> 05/06/2023 16:29, Ivan Malov:
> > Sorry, I missed your question. See below.
> > 
> > On Mon, 5 Jun 2023, Thomas Monjalon wrote:
> >   
> > > 05/06/2023 16:03, Ivan Malov:  
> > >> Hi Thomas,
> > >>
> > >> Thanks for responding. Please see below.
> > >>
> > >> On Mon, 5 Jun 2023, Thomas Monjalon wrote:
> > >>  
> > >>> Hello,
> > >>>
> > >>> 05/06/2023 15:09, Ivan Malov:  
> > >>>> Dear community,
> > >>>>
> > >>>> Is there any means in DPDK to discover relationship between
> > >>>> network/physical ports of the given adapter/board and
> > >>>> etdevs deployed in DPDK application on top of it?
> > >>>>
> > >>>> For example, in Linux, there are facilities like
> > >>>>  
> > >>>>> /sys/class/net/<iface>/phys_port_name
> > >>>>> /sys/class/net/<iface>/dev_port  
> > >>>>
> > >>>> and
> > >>>>  
> > >>>>> devlink port show  
> > >>>>
> > >>>> Do we have something similar in DPDK?  
> > >>>
> > >>> We can get the device name of a port:
> > >>> 	rte_eth_dev_get_name_by_port()  
> > >>
> > >> I'm afraid this won't do. Consider the following example.
> > >> Say, there's a NIC with two network ports and two PFs,
> > >> 0000:01:00.0 and 0000:01:00.1. The user plugs these
> > >> PFs to DPDK application. The resulting ethdev IDs
> > >> are 0 and 1. If the user invokes the said API,
> > >> they will get 0000:01:00.0 and 0000:01:00.1.
> > >> But that's not what is really needed.
> > >>
> > >> We seek a means to get the network port ID by
> > >> ethdev ID. For example, something like this:
> > >> - get_netport_by_ethdev(0) => 0
> > >> - get_netport_by_ethdev(1) => 1
> > >>
> > >> If two different PCI functions are associated with the
> > >> same network port (0, for instance), this should be
> > >> - get_netport_by_ethdev(0) => 0
> > >> - get_netport_by_ethdev(1) => 0
> > >>
> > >> Do we have something like that in DPDK?  
> > >
> > > No we don't have such underlying index.
> > > I don't understand why it is needed.
> > > To me the name is more informative than a number.
> > >
> > >  
> > >>>> If no, would the feature be worthwhile implementing?  
> > >>>
> > >>> We may have discrepancies in different device classes.  
> > >>
> > >> I mean precisely "ethdev"s. I do realise, though, that
> > >> an ethdev may be backed by a vdev (af_xdp, etc.) = in
> > >> such cases the assumed "get_netport" method could
> > >> just return (-ENOTSUP). What do you think?  
> > >
> > > Are you interested only in PCI devices? Looks limited.  
> > 
> > Theoretically, even a vdev may handle this request
> > appropriately. For example, a failsafe device may
> > ask its current underlying PCI device abot the
> > physical port ID in use. For af_xdp and the
> > likes, it's also possible. The PMD may
> > query sysfs to provide the value.
> > 
> > Strictly speaking, it's not limited, but the primary
> > use case is querying the phys. port ID for PFs, yes.
> > 
> > This information may be needed by some applications
> > that not only operate the higher-level ethdevs but
> > also take the real physical/wire interconnects
> > into account. It might be complex to explain
> > in a single email thread, though.
> > 
> > Previously, DPDK even used to have a flow action PHY_PORT.
> > Yes, it has been deprecated, but that's not a problem.
> > The information can be useful anyway.  
> 
> In this case, this is something the driver should fill in rte_eth_dev_info.
> Note that we already have rte_eth_dev_info::if_index but it looks different.
> 
> Who would be responsible of the numbering of the physical port?
> Should we report kernel numbering or do we need yet another numbering scheme?

Very few DPDK hardware devices support multiple ports on same card.
And only a couple of devices (like Mellanox/Nvidia) use a kernel driver component.

Re: Getting network port ID by ethdev port ID

[Ivan Malov]

Hi Stephen, Thomas,

Thanks for responding. PSB.

On Mon, 5 Jun 2023, Stephen Hemminger wrote:

> On Mon, 05 Jun 2023 18:03:14 +0200
> Thomas Monjalon <thomas@monjalon.net> wrote:
>
>> 05/06/2023 16:29, Ivan Malov:
>>> Sorry, I missed your question. See below.
>>>
>>> On Mon, 5 Jun 2023, Thomas Monjalon wrote:
>>>
>>>> 05/06/2023 16:03, Ivan Malov:
>>>>> Hi Thomas,
>>>>>
>>>>> Thanks for responding. Please see below.
>>>>>
>>>>> On Mon, 5 Jun 2023, Thomas Monjalon wrote:
>>>>>
>>>>>> Hello,
>>>>>>
>>>>>> 05/06/2023 15:09, Ivan Malov:
>>>>>>> Dear community,
>>>>>>>
>>>>>>> Is there any means in DPDK to discover relationship between
>>>>>>> network/physical ports of the given adapter/board and
>>>>>>> etdevs deployed in DPDK application on top of it?
>>>>>>>
>>>>>>> For example, in Linux, there are facilities like
>>>>>>>
>>>>>>>> /sys/class/net/<iface>/phys_port_name
>>>>>>>> /sys/class/net/<iface>/dev_port
>>>>>>>
>>>>>>> and
>>>>>>>
>>>>>>>> devlink port show
>>>>>>>
>>>>>>> Do we have something similar in DPDK?
>>>>>>
>>>>>> We can get the device name of a port:
>>>>>> 	rte_eth_dev_get_name_by_port()
>>>>>
>>>>> I'm afraid this won't do. Consider the following example.
>>>>> Say, there's a NIC with two network ports and two PFs,
>>>>> 0000:01:00.0 and 0000:01:00.1. The user plugs these
>>>>> PFs to DPDK application. The resulting ethdev IDs
>>>>> are 0 and 1. If the user invokes the said API,
>>>>> they will get 0000:01:00.0 and 0000:01:00.1.
>>>>> But that's not what is really needed.
>>>>>
>>>>> We seek a means to get the network port ID by
>>>>> ethdev ID. For example, something like this:
>>>>> - get_netport_by_ethdev(0) => 0
>>>>> - get_netport_by_ethdev(1) => 1
>>>>>
>>>>> If two different PCI functions are associated with the
>>>>> same network port (0, for instance), this should be
>>>>> - get_netport_by_ethdev(0) => 0
>>>>> - get_netport_by_ethdev(1) => 0
>>>>>
>>>>> Do we have something like that in DPDK?
>>>>
>>>> No we don't have such underlying index.
>>>> I don't understand why it is needed.
>>>> To me the name is more informative than a number.
>>>>
>>>>
>>>>>>> If no, would the feature be worthwhile implementing?
>>>>>>
>>>>>> We may have discrepancies in different device classes.
>>>>>
>>>>> I mean precisely "ethdev"s. I do realise, though, that
>>>>> an ethdev may be backed by a vdev (af_xdp, etc.) = in
>>>>> such cases the assumed "get_netport" method could
>>>>> just return (-ENOTSUP). What do you think?
>>>>
>>>> Are you interested only in PCI devices? Looks limited.
>>>
>>> Theoretically, even a vdev may handle this request
>>> appropriately. For example, a failsafe device may
>>> ask its current underlying PCI device abot the
>>> physical port ID in use. For af_xdp and the
>>> likes, it's also possible. The PMD may
>>> query sysfs to provide the value.
>>>
>>> Strictly speaking, it's not limited, but the primary
>>> use case is querying the phys. port ID for PFs, yes.
>>>
>>> This information may be needed by some applications
>>> that not only operate the higher-level ethdevs but
>>> also take the real physical/wire interconnects
>>> into account. It might be complex to explain
>>> in a single email thread, though.
>>>
>>> Previously, DPDK even used to have a flow action PHY_PORT.
>>> Yes, it has been deprecated, but that's not a problem.
>>> The information can be useful anyway.
>>
>> In this case, this is something the driver should fill in rte_eth_dev_info.
>> Note that we already have rte_eth_dev_info::if_index but it looks different.
>>
>> Who would be responsible of the numbering of the physical port?
>> Should we report kernel numbering or do we need yet another numbering scheme?
>
> Very few DPDK hardware devices support multiple ports on same card.
> And only a couple of devices (like Mellanox/Nvidia) use a kernel driver component.
>

So.. by the sound of it, it would be nice to introduce
something like "int  phys_port_id" to rte_eth_dev_info,
correct? That would indicate either -1 (for example,
in the case of VFs connected with representors)
or some sensible value, as per internal mapping.

That would help certain applications to have
physical port IDs mapped to ethdev IDs.
Right now they have no way of knowing.

Thank you.

Re: Getting network port ID by ethdev port ID

[Stephen Hemminger]

On Tue, 6 Jun 2023 00:30:18 +0400 (+04)
Ivan Malov <ivan.malov@arknetworks.am> wrote:

> Hi Stephen, Thomas,
> 
> Thanks for responding. PSB.
> 
> On Mon, 5 Jun 2023, Stephen Hemminger wrote:
> 
> > On Mon, 05 Jun 2023 18:03:14 +0200
> > Thomas Monjalon <thomas@monjalon.net> wrote:
> >  
> >> 05/06/2023 16:29, Ivan Malov:  
> >>> Sorry, I missed your question. See below.
> >>>
> >>> On Mon, 5 Jun 2023, Thomas Monjalon wrote:
> >>>  
> >>>> 05/06/2023 16:03, Ivan Malov:  
> >>>>> Hi Thomas,
> >>>>>
> >>>>> Thanks for responding. Please see below.
> >>>>>
> >>>>> On Mon, 5 Jun 2023, Thomas Monjalon wrote:
> >>>>>  
> >>>>>> Hello,
> >>>>>>
> >>>>>> 05/06/2023 15:09, Ivan Malov:  
> >>>>>>> Dear community,
> >>>>>>>
> >>>>>>> Is there any means in DPDK to discover relationship between
> >>>>>>> network/physical ports of the given adapter/board and
> >>>>>>> etdevs deployed in DPDK application on top of it?
> >>>>>>>
> >>>>>>> For example, in Linux, there are facilities like
> >>>>>>>  
> >>>>>>>> /sys/class/net/<iface>/phys_port_name
> >>>>>>>> /sys/class/net/<iface>/dev_port  
> >>>>>>>
> >>>>>>> and
> >>>>>>>  
> >>>>>>>> devlink port show  
> >>>>>>>
> >>>>>>> Do we have something similar in DPDK?  
> >>>>>>
> >>>>>> We can get the device name of a port:
> >>>>>> 	rte_eth_dev_get_name_by_port()  
> >>>>>
> >>>>> I'm afraid this won't do. Consider the following example.
> >>>>> Say, there's a NIC with two network ports and two PFs,
> >>>>> 0000:01:00.0 and 0000:01:00.1. The user plugs these
> >>>>> PFs to DPDK application. The resulting ethdev IDs
> >>>>> are 0 and 1. If the user invokes the said API,
> >>>>> they will get 0000:01:00.0 and 0000:01:00.1.
> >>>>> But that's not what is really needed.
> >>>>>
> >>>>> We seek a means to get the network port ID by
> >>>>> ethdev ID. For example, something like this:
> >>>>> - get_netport_by_ethdev(0) => 0
> >>>>> - get_netport_by_ethdev(1) => 1
> >>>>>
> >>>>> If two different PCI functions are associated with the
> >>>>> same network port (0, for instance), this should be
> >>>>> - get_netport_by_ethdev(0) => 0
> >>>>> - get_netport_by_ethdev(1) => 0
> >>>>>
> >>>>> Do we have something like that in DPDK?  
> >>>>
> >>>> No we don't have such underlying index.
> >>>> I don't understand why it is needed.
> >>>> To me the name is more informative than a number.
> >>>>
> >>>>  
> >>>>>>> If no, would the feature be worthwhile implementing?  
> >>>>>>
> >>>>>> We may have discrepancies in different device classes.  
> >>>>>
> >>>>> I mean precisely "ethdev"s. I do realise, though, that
> >>>>> an ethdev may be backed by a vdev (af_xdp, etc.) = in
> >>>>> such cases the assumed "get_netport" method could
> >>>>> just return (-ENOTSUP). What do you think?  
> >>>>
> >>>> Are you interested only in PCI devices? Looks limited.  
> >>>
> >>> Theoretically, even a vdev may handle this request
> >>> appropriately. For example, a failsafe device may
> >>> ask its current underlying PCI device abot the
> >>> physical port ID in use. For af_xdp and the
> >>> likes, it's also possible. The PMD may
> >>> query sysfs to provide the value.
> >>>
> >>> Strictly speaking, it's not limited, but the primary
> >>> use case is querying the phys. port ID for PFs, yes.
> >>>
> >>> This information may be needed by some applications
> >>> that not only operate the higher-level ethdevs but
> >>> also take the real physical/wire interconnects
> >>> into account. It might be complex to explain
> >>> in a single email thread, though.
> >>>
> >>> Previously, DPDK even used to have a flow action PHY_PORT.
> >>> Yes, it has been deprecated, but that's not a problem.
> >>> The information can be useful anyway.  
> >>
> >> In this case, this is something the driver should fill in rte_eth_dev_info.
> >> Note that we already have rte_eth_dev_info::if_index but it looks different.
> >>
> >> Who would be responsible of the numbering of the physical port?
> >> Should we report kernel numbering or do we need yet another numbering scheme?  
> >
> > Very few DPDK hardware devices support multiple ports on same card.
> > And only a couple of devices (like Mellanox/Nvidia) use a kernel driver component.
> >  
> 
> So.. by the sound of it, it would be nice to introduce
> something like "int  phys_port_id" to rte_eth_dev_info,
> correct? That would indicate either -1 (for example,
> in the case of VFs connected with representors)
> or some sensible value, as per internal mapping.
> 
> That would help certain applications to have
> physical port IDs mapped to ethdev IDs.
> Right now they have no way of knowing.
> 
> Thank you.

You are better off using PCI information.  That is what systemd does in general.
If you really need it use the PCI go look in sysfs in application.
The multiport bifuricated nvidia driver is unique. IMHO not worth adding general
support in DPDK until/unless we have three vendors doing it.

Re: Getting network port ID by ethdev port ID

[Ivan Malov]

Hi Stephen,

On Mon, 5 Jun 2023, Stephen Hemminger wrote:

> On Tue, 6 Jun 2023 00:30:18 +0400 (+04)
> Ivan Malov <ivan.malov@arknetworks.am> wrote:
>
>> Hi Stephen, Thomas,
>>
>> Thanks for responding. PSB.
>>
>> On Mon, 5 Jun 2023, Stephen Hemminger wrote:
>>
>>> On Mon, 05 Jun 2023 18:03:14 +0200
>>> Thomas Monjalon <thomas@monjalon.net> wrote:
>>>
>>>> 05/06/2023 16:29, Ivan Malov:
>>>>> Sorry, I missed your question. See below.
>>>>>
>>>>> On Mon, 5 Jun 2023, Thomas Monjalon wrote:
>>>>>
>>>>>> 05/06/2023 16:03, Ivan Malov:
>>>>>>> Hi Thomas,
>>>>>>>
>>>>>>> Thanks for responding. Please see below.
>>>>>>>
>>>>>>> On Mon, 5 Jun 2023, Thomas Monjalon wrote:
>>>>>>>
>>>>>>>> Hello,
>>>>>>>>
>>>>>>>> 05/06/2023 15:09, Ivan Malov:
>>>>>>>>> Dear community,
>>>>>>>>>
>>>>>>>>> Is there any means in DPDK to discover relationship between
>>>>>>>>> network/physical ports of the given adapter/board and
>>>>>>>>> etdevs deployed in DPDK application on top of it?
>>>>>>>>>
>>>>>>>>> For example, in Linux, there are facilities like
>>>>>>>>>
>>>>>>>>>> /sys/class/net/<iface>/phys_port_name
>>>>>>>>>> /sys/class/net/<iface>/dev_port
>>>>>>>>>
>>>>>>>>> and
>>>>>>>>>
>>>>>>>>>> devlink port show
>>>>>>>>>
>>>>>>>>> Do we have something similar in DPDK?
>>>>>>>>
>>>>>>>> We can get the device name of a port:
>>>>>>>> 	rte_eth_dev_get_name_by_port()
>>>>>>>
>>>>>>> I'm afraid this won't do. Consider the following example.
>>>>>>> Say, there's a NIC with two network ports and two PFs,
>>>>>>> 0000:01:00.0 and 0000:01:00.1. The user plugs these
>>>>>>> PFs to DPDK application. The resulting ethdev IDs
>>>>>>> are 0 and 1. If the user invokes the said API,
>>>>>>> they will get 0000:01:00.0 and 0000:01:00.1.
>>>>>>> But that's not what is really needed.
>>>>>>>
>>>>>>> We seek a means to get the network port ID by
>>>>>>> ethdev ID. For example, something like this:
>>>>>>> - get_netport_by_ethdev(0) => 0
>>>>>>> - get_netport_by_ethdev(1) => 1
>>>>>>>
>>>>>>> If two different PCI functions are associated with the
>>>>>>> same network port (0, for instance), this should be
>>>>>>> - get_netport_by_ethdev(0) => 0
>>>>>>> - get_netport_by_ethdev(1) => 0
>>>>>>>
>>>>>>> Do we have something like that in DPDK?
>>>>>>
>>>>>> No we don't have such underlying index.
>>>>>> I don't understand why it is needed.
>>>>>> To me the name is more informative than a number.
>>>>>>
>>>>>>
>>>>>>>>> If no, would the feature be worthwhile implementing?
>>>>>>>>
>>>>>>>> We may have discrepancies in different device classes.
>>>>>>>
>>>>>>> I mean precisely "ethdev"s. I do realise, though, that
>>>>>>> an ethdev may be backed by a vdev (af_xdp, etc.) = in
>>>>>>> such cases the assumed "get_netport" method could
>>>>>>> just return (-ENOTSUP). What do you think?
>>>>>>
>>>>>> Are you interested only in PCI devices? Looks limited.
>>>>>
>>>>> Theoretically, even a vdev may handle this request
>>>>> appropriately. For example, a failsafe device may
>>>>> ask its current underlying PCI device abot the
>>>>> physical port ID in use. For af_xdp and the
>>>>> likes, it's also possible. The PMD may
>>>>> query sysfs to provide the value.
>>>>>
>>>>> Strictly speaking, it's not limited, but the primary
>>>>> use case is querying the phys. port ID for PFs, yes.
>>>>>
>>>>> This information may be needed by some applications
>>>>> that not only operate the higher-level ethdevs but
>>>>> also take the real physical/wire interconnects
>>>>> into account. It might be complex to explain
>>>>> in a single email thread, though.
>>>>>
>>>>> Previously, DPDK even used to have a flow action PHY_PORT.
>>>>> Yes, it has been deprecated, but that's not a problem.
>>>>> The information can be useful anyway.
>>>>
>>>> In this case, this is something the driver should fill in rte_eth_dev_info.
>>>> Note that we already have rte_eth_dev_info::if_index but it looks different.
>>>>
>>>> Who would be responsible of the numbering of the physical port?
>>>> Should we report kernel numbering or do we need yet another numbering scheme?
>>>
>>> Very few DPDK hardware devices support multiple ports on same card.
>>> And only a couple of devices (like Mellanox/Nvidia) use a kernel driver component.
>>>
>>
>> So.. by the sound of it, it would be nice to introduce
>> something like "int  phys_port_id" to rte_eth_dev_info,
>> correct? That would indicate either -1 (for example,
>> in the case of VFs connected with representors)
>> or some sensible value, as per internal mapping.
>>
>> That would help certain applications to have
>> physical port IDs mapped to ethdev IDs.
>> Right now they have no way of knowing.
>>
>> Thank you.
>
> You are better off using PCI information.  That is what systemd does in general.
> If you really need it use the PCI go look in sysfs in application.
> The multiport bifuricated nvidia driver is unique. IMHO not worth adding general
> support in DPDK until/unless we have three vendors doing it.
>
In general, I agree that there might not be too many vendors
that provide multi-port adapters. But in what comes to
bifurcated model = I'm not sure that I understand why
we confine our discussion to it. What I mean is not
Linux interface IDs. I mean enumerating physical
ports on the network card and providing mappings
to the application, like "physical port 0 maps
to PF 0". My hunch is that this information
can be available in vendors that do not use
the bifurcated model; they might be able to
retrieve it from their internals just like
any other aspect of card configuration.

When you suggest that I stick with using PCI information, do
you mean precisely "/sys/class/net/<iface>/dev_port" et al?
If yes, unfortunately, it seems like these fields are not
filled the same way for different vendors, sometimes they
aren't supported at all. So, I'm not pushing to add such
means to DPDK, but it might be useful to applications.

Thank you.

Re: Getting network port ID by ethdev port ID

[Stephen Hemminger]

On Tue, 6 Jun 2023 11:16:21 +0400 (+04)
Ivan Malov <ivan.malov@arknetworks.am> wrote:

> In general, I agree that there might not be too many vendors
> that provide multi-port adapters. But in what comes to
> bifurcated model = I'm not sure that I understand why
> we confine our discussion to it. What I mean is not
> Linux interface IDs. I mean enumerating physical
> ports on the network card and providing mappings
> to the application, like "physical port 0 maps
> to PF 0". My hunch is that this information
> can be available in vendors that do not use
> the bifurcated model; they might be able to
> retrieve it from their internals just like
> any other aspect of card configuration.
> 
> When you suggest that I stick with using PCI information, do
> you mean precisely "/sys/class/net/<iface>/dev_port" et al?
> If yes, unfortunately, it seems like these fields are not
> filled the same way for different vendors, sometimes they
> aren't supported at all. So, I'm not pushing to add such
> means to DPDK, but it might be useful to applications.

I meant look in /sys/devices/pci0000:00/0000:00:01.1/ etc.

But after looking deeper, it gets messy, so probably does need to be in
EAL to handle FreeBSD and Windows.

There is a multitude of different values possible.
 - acpi_index - which comes from BIOS
 - dev_port - only available if device has multiple ports
 - pci_slot - comes from hotplug
 - dev_id - old kernels with ipoib

These all do not depend on a bifurcated driver. All devices even
dedicated ones will have this. Probably more likely to see a dual
port dedicated NIC.

If some one makes a new API (rte_ethdev_slot_info_get)? it could
provide both slot and port in slot information.

Re: Getting network port ID by ethdev port ID

[Ferruh Yigit]

On 6/5/2023 9:30 PM, Ivan Malov wrote:
> Hi Stephen, Thomas,
> 
> Thanks for responding. PSB.
> 
> On Mon, 5 Jun 2023, Stephen Hemminger wrote:
> 
>> On Mon, 05 Jun 2023 18:03:14 +0200
>> Thomas Monjalon <thomas@monjalon.net> wrote:
>>
>>> 05/06/2023 16:29, Ivan Malov:
>>>> Sorry, I missed your question. See below.
>>>>
>>>> On Mon, 5 Jun 2023, Thomas Monjalon wrote:
>>>>
>>>>> 05/06/2023 16:03, Ivan Malov:
>>>>>> Hi Thomas,
>>>>>>
>>>>>> Thanks for responding. Please see below.
>>>>>>
>>>>>> On Mon, 5 Jun 2023, Thomas Monjalon wrote:
>>>>>>
>>>>>>> Hello,
>>>>>>>
>>>>>>> 05/06/2023 15:09, Ivan Malov:
>>>>>>>> Dear community,
>>>>>>>>
>>>>>>>> Is there any means in DPDK to discover relationship between
>>>>>>>> network/physical ports of the given adapter/board and
>>>>>>>> etdevs deployed in DPDK application on top of it?
>>>>>>>>
>>>>>>>> For example, in Linux, there are facilities like
>>>>>>>>
>>>>>>>>> /sys/class/net/<iface>/phys_port_name
>>>>>>>>> /sys/class/net/<iface>/dev_port
>>>>>>>>
>>>>>>>> and
>>>>>>>>
>>>>>>>>> devlink port show
>>>>>>>>
>>>>>>>> Do we have something similar in DPDK?
>>>>>>>
>>>>>>> We can get the device name of a port:
>>>>>>>     rte_eth_dev_get_name_by_port()
>>>>>>
>>>>>> I'm afraid this won't do. Consider the following example.
>>>>>> Say, there's a NIC with two network ports and two PFs,
>>>>>> 0000:01:00.0 and 0000:01:00.1. The user plugs these
>>>>>> PFs to DPDK application. The resulting ethdev IDs
>>>>>> are 0 and 1. If the user invokes the said API,
>>>>>> they will get 0000:01:00.0 and 0000:01:00.1.
>>>>>> But that's not what is really needed.
>>>>>>
>>>>>> We seek a means to get the network port ID by
>>>>>> ethdev ID. For example, something like this:
>>>>>> - get_netport_by_ethdev(0) => 0
>>>>>> - get_netport_by_ethdev(1) => 1
>>>>>>
>>>>>> If two different PCI functions are associated with the
>>>>>> same network port (0, for instance), this should be
>>>>>> - get_netport_by_ethdev(0) => 0
>>>>>> - get_netport_by_ethdev(1) => 0
>>>>>>
>>>>>> Do we have something like that in DPDK?
>>>>>
>>>>> No we don't have such underlying index.
>>>>> I don't understand why it is needed.
>>>>> To me the name is more informative than a number.
>>>>>
>>>>>
>>>>>>>> If no, would the feature be worthwhile implementing?
>>>>>>>
>>>>>>> We may have discrepancies in different device classes.
>>>>>>
>>>>>> I mean precisely "ethdev"s. I do realise, though, that
>>>>>> an ethdev may be backed by a vdev (af_xdp, etc.) = in
>>>>>> such cases the assumed "get_netport" method could
>>>>>> just return (-ENOTSUP). What do you think?
>>>>>
>>>>> Are you interested only in PCI devices? Looks limited.
>>>>
>>>> Theoretically, even a vdev may handle this request
>>>> appropriately. For example, a failsafe device may
>>>> ask its current underlying PCI device abot the
>>>> physical port ID in use. For af_xdp and the
>>>> likes, it's also possible. The PMD may
>>>> query sysfs to provide the value.
>>>>
>>>> Strictly speaking, it's not limited, but the primary
>>>> use case is querying the phys. port ID for PFs, yes.
>>>>
>>>> This information may be needed by some applications
>>>> that not only operate the higher-level ethdevs but
>>>> also take the real physical/wire interconnects
>>>> into account. It might be complex to explain
>>>> in a single email thread, though.
>>>>
>>>> Previously, DPDK even used to have a flow action PHY_PORT.
>>>> Yes, it has been deprecated, but that's not a problem.
>>>> The information can be useful anyway.
>>>
>>> In this case, this is something the driver should fill in
>>> rte_eth_dev_info.
>>> Note that we already have rte_eth_dev_info::if_index but it looks
>>> different.
>>>
>>> Who would be responsible of the numbering of the physical port?
>>> Should we report kernel numbering or do we need yet another numbering
>>> scheme?
>>
>> Very few DPDK hardware devices support multiple ports on same card.
>> And only a couple of devices (like Mellanox/Nvidia) use a kernel
>> driver component.
>>
> 
> So.. by the sound of it, it would be nice to introduce
> something like "int  phys_port_id" to rte_eth_dev_info,
> correct? That would indicate either -1 (for example,
> in the case of VFs connected with representors)
> or some sensible value, as per internal mapping.
> 

I am for having specific API for it (if we will have it), instead of
overloading the 'rte_eth_dev_info_get()', so purpose of new information
(and new API) can be more focused and clear.

> That would help certain applications to have
> physical port IDs mapped to ethdev IDs.
> Right now they have no way of knowing.
> 
> Thank you.