Re: Getting network port ID by ethdev port ID

[Ivan Malov <ivan.malov@arknetworks.am>]

Hi Stephen,

On Mon, 5 Jun 2023, Stephen Hemminger wrote:

> On Tue, 6 Jun 2023 00:30:18 +0400 (+04)
> Ivan Malov <ivan.malov@arknetworks.am> wrote:
>
>> Hi Stephen, Thomas,
>>
>> Thanks for responding. PSB.
>>
>> On Mon, 5 Jun 2023, Stephen Hemminger wrote:
>>
>>> On Mon, 05 Jun 2023 18:03:14 +0200
>>> Thomas Monjalon <thomas@monjalon.net> wrote:
>>>
>>>> 05/06/2023 16:29, Ivan Malov:
>>>>> Sorry, I missed your question. See below.
>>>>>
>>>>> On Mon, 5 Jun 2023, Thomas Monjalon wrote:
>>>>>
>>>>>> 05/06/2023 16:03, Ivan Malov:
>>>>>>> Hi Thomas,
>>>>>>>
>>>>>>> Thanks for responding. Please see below.
>>>>>>>
>>>>>>> On Mon, 5 Jun 2023, Thomas Monjalon wrote:
>>>>>>>
>>>>>>>> Hello,
>>>>>>>>
>>>>>>>> 05/06/2023 15:09, Ivan Malov:
>>>>>>>>> Dear community,
>>>>>>>>>
>>>>>>>>> Is there any means in DPDK to discover relationship between
>>>>>>>>> network/physical ports of the given adapter/board and
>>>>>>>>> etdevs deployed in DPDK application on top of it?
>>>>>>>>>
>>>>>>>>> For example, in Linux, there are facilities like
>>>>>>>>>
>>>>>>>>>> /sys/class/net/<iface>/phys_port_name
>>>>>>>>>> /sys/class/net/<iface>/dev_port
>>>>>>>>>
>>>>>>>>> and
>>>>>>>>>
>>>>>>>>>> devlink port show
>>>>>>>>>
>>>>>>>>> Do we have something similar in DPDK?
>>>>>>>>
>>>>>>>> We can get the device name of a port:
>>>>>>>> 	rte_eth_dev_get_name_by_port()
>>>>>>>
>>>>>>> I'm afraid this won't do. Consider the following example.
>>>>>>> Say, there's a NIC with two network ports and two PFs,
>>>>>>> 0000:01:00.0 and 0000:01:00.1. The user plugs these
>>>>>>> PFs to DPDK application. The resulting ethdev IDs
>>>>>>> are 0 and 1. If the user invokes the said API,
>>>>>>> they will get 0000:01:00.0 and 0000:01:00.1.
>>>>>>> But that's not what is really needed.
>>>>>>>
>>>>>>> We seek a means to get the network port ID by
>>>>>>> ethdev ID. For example, something like this:
>>>>>>> - get_netport_by_ethdev(0) => 0
>>>>>>> - get_netport_by_ethdev(1) => 1
>>>>>>>
>>>>>>> If two different PCI functions are associated with the
>>>>>>> same network port (0, for instance), this should be
>>>>>>> - get_netport_by_ethdev(0) => 0
>>>>>>> - get_netport_by_ethdev(1) => 0
>>>>>>>
>>>>>>> Do we have something like that in DPDK?
>>>>>>
>>>>>> No we don't have such underlying index.
>>>>>> I don't understand why it is needed.
>>>>>> To me the name is more informative than a number.
>>>>>>
>>>>>>
>>>>>>>>> If no, would the feature be worthwhile implementing?
>>>>>>>>
>>>>>>>> We may have discrepancies in different device classes.
>>>>>>>
>>>>>>> I mean precisely "ethdev"s. I do realise, though, that
>>>>>>> an ethdev may be backed by a vdev (af_xdp, etc.) = in
>>>>>>> such cases the assumed "get_netport" method could
>>>>>>> just return (-ENOTSUP). What do you think?
>>>>>>
>>>>>> Are you interested only in PCI devices? Looks limited.
>>>>>
>>>>> Theoretically, even a vdev may handle this request
>>>>> appropriately. For example, a failsafe device may
>>>>> ask its current underlying PCI device abot the
>>>>> physical port ID in use. For af_xdp and the
>>>>> likes, it's also possible. The PMD may
>>>>> query sysfs to provide the value.
>>>>>
>>>>> Strictly speaking, it's not limited, but the primary
>>>>> use case is querying the phys. port ID for PFs, yes.
>>>>>
>>>>> This information may be needed by some applications
>>>>> that not only operate the higher-level ethdevs but
>>>>> also take the real physical/wire interconnects
>>>>> into account. It might be complex to explain
>>>>> in a single email thread, though.
>>>>>
>>>>> Previously, DPDK even used to have a flow action PHY_PORT.
>>>>> Yes, it has been deprecated, but that's not a problem.
>>>>> The information can be useful anyway.
>>>>
>>>> In this case, this is something the driver should fill in rte_eth_dev_info.
>>>> Note that we already have rte_eth_dev_info::if_index but it looks different.
>>>>
>>>> Who would be responsible of the numbering of the physical port?
>>>> Should we report kernel numbering or do we need yet another numbering scheme?
>>>
>>> Very few DPDK hardware devices support multiple ports on same card.
>>> And only a couple of devices (like Mellanox/Nvidia) use a kernel driver component.
>>>
>>
>> So.. by the sound of it, it would be nice to introduce
>> something like "int  phys_port_id" to rte_eth_dev_info,
>> correct? That would indicate either -1 (for example,
>> in the case of VFs connected with representors)
>> or some sensible value, as per internal mapping.
>>
>> That would help certain applications to have
>> physical port IDs mapped to ethdev IDs.
>> Right now they have no way of knowing.
>>
>> Thank you.
>
> You are better off using PCI information.  That is what systemd does in general.
> If you really need it use the PCI go look in sysfs in application.
> The multiport bifuricated nvidia driver is unique. IMHO not worth adding general
> support in DPDK until/unless we have three vendors doing it.
>
In general, I agree that there might not be too many vendors
that provide multi-port adapters. But in what comes to
bifurcated model = I'm not sure that I understand why
we confine our discussion to it. What I mean is not
Linux interface IDs. I mean enumerating physical
ports on the network card and providing mappings
to the application, like "physical port 0 maps
to PF 0". My hunch is that this information
can be available in vendors that do not use
the bifurcated model; they might be able to
retrieve it from their internals just like
any other aspect of card configuration.

When you suggest that I stick with using PCI information, do
you mean precisely "/sys/class/net/<iface>/dev_port" et al?
If yes, unfortunately, it seems like these fields are not
filled the same way for different vendors, sometimes they
aren't supported at all. So, I'm not pushing to add such
means to DPDK, but it might be useful to applications.

Thank you.