Re: [PATCH 0/3] support setting lanes

[Thomas Monjalon <thomas@monjalon.net>]

22/03/2024 06:51, Jerin Jacob:
> On Fri, Mar 22, 2024 at 10:56 AM Ajit Khaparde
> <ajit.khaparde@broadcom.com> wrote:
> >
> > On Thu, Mar 21, 2024 at 9:39 PM Jerin Jacob <jerinjacobk@gmail.com> wrote:
> > >
> > > On Fri, Mar 22, 2024 at 7:58 AM huangdengdui <huangdengdui@huawei.com> wrote:
> > > >
> > > >
> > > >
> > > > On 2024/3/21 16:28, Thomas Monjalon wrote:
> > > > > 21/03/2024 03:02, huangdengdui:
> > > > >>
> > > > >> On 2024/3/20 20:31, Ferruh Yigit wrote:
> > > > >>> On 3/18/2024 9:26 PM, Damodharam Ammepalli wrote:
> > > > >>>> On Mon, Mar 18, 2024 at 7:56 AM Thomas Monjalon <thomas@monjalon.net> wrote:
> > > > >>>>>
> > > > >>>>> 12/03/2024 08:52, Dengdui Huang:
> > > > >>>>>> Some speeds can be achieved with different number of lanes. For example,
> > > > >>>>>> 100Gbps can be achieved using two lanes of 50Gbps or four lanes of 25Gbps.
> > > > >>>>>> When use different lanes, the port cannot be up.
> > > > >>>>>
> > > > >>>>> I'm not sure what you are referring to.
> > > > >>>>> I suppose it is not PCI lanes.
> > > > >>>>> Please could you link to an explanation of how a port is split in lanes?
> > > > >>>>> Which hardware does this?
> > > > >>>>>
> > > > >>>> This is a snapshot of 100Gb that the latest BCM576xx supports.
> > > > >>>> 100Gb (NRZ: 25G per lane, 4 lanes) link speed
> > > > >>>> 100Gb (PAM4-56: 50G per lane, 2 lanes) link speed
> > > > >>>> 100Gb (PAM4-112: 100G per lane, 1 lane) link speed
> > > > >>>>
> > > > >>>> Let the user feed in lanes=< integer value> and the NIC driver decides
> > > > >>>> the matching combination speed x lanes that works. In future if a new speed
> > > > >>>> is implemented with more than 8 lanes, there wouldn't be a need
> > > > >>>> to touch this speed command. Using separate lane command would
> > > > >>>> be a better alternative to support already shipped products and only new
> > > > >>>> drivers would consider this lanes configuration, if applicable.
> > > > >>>>
> > > > >>>
> > > > >>> As far as I understand, lane is related to the physical layer of the
> > > > >>> NIC, there are multiple copies of transmitter, receiver, modulator HW
> > > > >>> block and each set called as a 'lane' and multiple lanes work together
> > > > >>> to achieve desired speed. (please correct me if this is wrong).
> > > > >>>
> > > > >>> Why not just configuring the speed is not enough? Why user needs to know
> > > > >>> the detail and configuration of the lanes?
> > > > >>> Will it work if driver/device configure the "speed x lane" internally
> > > > >>> for the requested speed?
> > > > >>>
> > > > >>> Is there a benefit to force specific lane count for a specific speed
> > > > >>> (like power optimization, just a wild guess)?
> > > > >>>
> > > > >>>
> > > > >>> And +1 for auto-negotiation if possible.
> > > > >>
> > > > >> As you said above,，multiple lanes work together to achieve desired speed.
> > > > >> For example, the following solutions can be used to implement 100G:
> > > > >> 1、Combines four 25G lanes
> > > > >> 2、Combines two 50G lanes
> > > > >> 3、A single 100G lane
> > > > >>
> > > > >> It is assumed that two ports are interconnected and the two ports support
> > > > >> the foregoing three solutions. But, we just configured the speed to 100G and
> > > > >> one port uses four 25G lanes by default and the other port uses two 50G lanes
> > > > >> by default, the port cannot be up. In this case, we need to configure the
> > > > >> two ports to use the same solutions (for example, uses two 50G lanes)
> > > > >> so that the ports can be up.
> > > > >
> > > > > Why this config is not OK? How do we know?
> > > > > Really I have a very bad feeling about this feature.
> > > > >
> > > > >
> > > > Sorry, I don't quite understand your question.
> > > > Are you asking why cannot be up when one port uses four 25G lanes and the other port uses two 50G lanes?
> > > >
> > > > 100GBASE-SR2 (two 50G lanes) and 100GBASE-SR4 (four 25G lanes) have different standards at the physical layer.[1]
> > > > So it's not possible to communicate. Configuring lanes can help the driver choose the same standard.
> > >
> > > Typically, low-level drivers like FW configure this.
> > >
> > > For example, If FW configures, 100G port as 100GBASE-SR2 then two
> > > ethdev(port 0 and port1) will show up.
> > > Now, assume if we expose this API and Can end user configure port 1 as
> > > 25G lines if so,
> > > a) What happens to port0 and it states?
> > There should be no impact to port0.
> >
> > > b) Will port2, port3 will show up after issuing this API(As end user
> > > configured 25Gx4 for 100G)? Will application needs to hotplug to get
> > > use ports.
> > No. The port count does not change. Nor does the number of PCI
> > functions seen by the host. Unless designed otherwise.
> >
> > Changing the lane count does not change anything in physical terms.
> > What changes is the modulation or the signaling scheme.
> > The number of lanes which can be supported is determined by
> > the PHY itself and the cables used and needs to be negotiated appropriately
> > with the remote partner - which is just like using forced Ethernet Speed
> > instead of auto-negotiated speeds.

Thanks for the explanation Ajit.

> OK. It looks like platform independent then. At least cnxk driver, End
> user cannot simplify change the line config parameters
> while traffic is active also, it looks like other drivers need to have
> SerDes training with remote partner while reconfiguring it.
> 
> At least on cnxk platform, 25Gx4 on 100G will show as 4 ethdev devices.

That's a strange behaviour.
Why showing 4 ports which are not independent?

> Having said that, If other NICs support this feature without
> disturbing current port states, I don't have an objection to this API.