Re: [dpdk-dev] [PATCH v1 02/12] ethdev: add eswitch port item to flow API

[Ori Kam <orika@nvidia.com>]

Hi Ivan,

> -----Original Message-----
> From: Ivan Malov <Ivan.Malov@oktetlabs.ru>
> Cc: dev@dpdk.org
> Subject: Re: [PATCH v1 02/12] ethdev: add eswitch port item to flow API
> 
> Hi Ori,
> 
> On 04/10/2021 14:37, Ori Kam wrote:
> > Hi Ivan,
> >
> >> -----Original Message-----
> >> From: Ivan Malov <Ivan.Malov@oktetlabs.ru>
> >> Sent: Monday, October 4, 2021 2:06 PM
> >> Cc: dev@dpdk.org
> >> Subject: Re: [PATCH v1 02/12] ethdev: add eswitch port item to flow
> >> API
> >>
> >> Hi Ori,
> >>
> >> On 04/10/2021 08:45, Ori Kam wrote:
> >>> Hi Ivan,
> >>>
> >>>> -----Original Message-----
> >>>> From: Ivan Malov <Ivan.Malov@oktetlabs.ru>
> >>>> Sent: Sunday, October 3, 2021 9:11 PM
> >>>> Subject: Re: [PATCH v1 02/12] ethdev: add eswitch port item to flow
> >>>> API
> >>>>
> >>>>
> >>>>
> >>>> On 03/10/2021 15:40, Ori Kam wrote:
> >>>>> Hi Andrew and Ivan,
> >>>>>
> >>>>>> -----Original Message-----
> >>>>>> From: Andrew Rybchenko <andrew.rybchenko@oktetlabs.ru>
> >>>>>> Sent: Friday, October 1, 2021 4:47 PM
> >>>>>> Subject: [PATCH v1 02/12] ethdev: add eswitch port item to flow
> >>>>>> API
> >>>>>>
> >>>>>> From: Ivan Malov <ivan.malov@oktetlabs.ru>
> >>>>>>
> >>>>>> For use with "transfer" flows. Supposed to match traffic entering
> >>>>>> the e-switch from the external world (network, guests) via the
> >>>>>> port which is logically connected with the given ethdev.
> >>>>>>
> >>>>>> Must not be combined with attributes "ingress" / "egress".
> >>>>>>
> >>>>>> This item is meant to use the same structure as ethdev item.
> >>>>>>
> >>>>>
> >>>>> In case the app is not working with representors, meaning each
> >>>>> switch port is mapped to ethdev.
> >>>>> both items (ethdev and eswitch port ) have the same meaning?
> >>>>
> >>>> No. Ethdev means ethdev, and e-switch port is the point where this
> >>>> ethdev is plugged to. For example, "transfer + ESWITCH_PORT" for a
> >>>> regular PF ethdev typically means the network port (maybe you can
> >>>> recall the idea that a PF ethdev "represents" the network port it's
> >> associated with).
> >>>>
> >>>> I believe, that diagrams which these patches add to
> >>>> "doc/guides/prog_guide/rte_flow.rst" may come in handy to
> >>>> understand the meaning. Also, you can take a look at our larger
> >>>> diagram from the Sep 14 gathering.
> >>>>
> >>>
> >>> Lets look at the following system:
> >>> E-Switch has 3 ports - PF, VF1, VF2
> >>> The ports are distributed as follows:
> >>> DPDK application:
> >>> ethdev(0) pf,
> >>> ethdev(1) representor to VF1
> >>> ethdev(2) representor to VF2
> >>> ethdev(3) VF1
> >>>
> >>> VM:
> >>> VF2
> >>>
> >>> As we know all representors are realy connected to the PF(at least
> >>> in this example)
> >>
> >> This example tries to say that the e-switch has 3 ports in total,
> >> and, given your explanation, one may indeed agree that *in this
> >> example* representors re-use e-switch port of ethdev=0 (with some
> >> metadata to distinguish packets, etc.). But one can hardly assume
> >> that *all* representors with any vendor's NIC are connected to the
> >> e-switch the same way. It's vendor specific. Well, at least,
> >> applications don't have this knowledge and don't need to.
> >>
> >>>
> >>> So matching on ethdev(3)  means matching on traffic sent from DPDK
> >>> port
> >> 3 right?
> >>
> >> Item ETHDEV (ethdev_id=3) matches traffic sent by DPDK port 3. Looks
> >> like we're on the same page here.
> >>
> >
> > Good.
> >
> >>> And matching on eswitch_port(3) means matching in traffic that goes
> >>> into VF1 which is the same traffic as ethdev(3) right?
> >>
> >> I didn't catch the thought about "the same traffic". Direction is not the
> same.
> >> Item ESWITCH_PORT (ethdev_id=3) matches traffic sent by DPDK port 1.
> >>
> > This is the critical part for my understanding.
> > Matching on ethdev_id(3) means matching on traffic that is coming from
> DPDK port3.
> 
> Right.
> 
> > So from E-Switch view point it is traffic that goes into VF1?
> 
> No. Above you clearly say "coming from DPDK port3". That is, from the VF1.
> *Not* going into it. Port 3 (ethdev_id=3) *is* VF1.
> 
But taffic that goes from DPDK port 3 goes into VF1,
what am I missing?

> > While matching on E-Switch_port(3) means matching on traffic coming
> from VF1?
> 
> No. It means matching on traffic coming from ethdev 1. From the VF1's
> representor.
> 
> >
> > And by the same logic matching on ethdev_id(1) means matching on
> > taffic that was sent from DPDK port 1 and matching on E-Switch_port(1)
> > means matching on traffic coming from
> > VF1
> 
> In this case, you've got this right. But please see my above notes. By the
> looks of it, you might have run into confusion over there.
> 
That is the issue I'm not sure I understand, and I think that 
if I don't underdand this, this miss underdanding will be shared by others.

> >
> > So in this case eswitch_port(3) is equal ot eswitch_port(1) right?
> > While ethdev(1) is not equal to ethdev(3)
> 
> No.
> 
> Item ETHDEV (ethdev_id=1) equals item ESWITCH_PORT (ethdev_id=3).
> Item ETHDEV (ethdev_id=3) equals item ESWITCH_PORT (ethdev_id=1).
> 
I think this was my first undestaning, lets see if I can explain it using my words.
ETHDEV - means that the source port that we are matching on is the closest one
the dpdk application. Example like above ETHDEV(ethdev_id=1) means matching
on traffic coming from the PF with some metadata that marks this DPDK port.
ETHDEV(ethdev_id=3) means matching on traffic coming from VF1.

ESWITCH_PORT meaning matching on the port that is connected to the DPDK port
(the other side of the wire). Example ESWITCH_PORT(ethdev_id=1) means matching
on traffic coming from VF1
While matching on ESWITCH_PORT(ethdev_id=3) means matching on PF with some
metadata.

Everything assume that representors are on the PF and use some metadata.

Did I get it right?

> >
> > And just to complete the picture, matching on ethdev(2) will result in
> > traffic coming from the dpdk port and matching on eswitch_port(2) will
> > match on traffic coming from VF2
> 
> Exactly.
> 
> 
> But, Ori, let me draw your attention to the following issue. In order to
> simplify understanding, I suggest that we refrain from saying "traffic that
> GOES TO". Where it goes depends on default rules that are supposed to be
> maintained by the PMD when ports get plugged / unplugged.
> 
> The flow items ETHDEV and ESWITH_PORT define the SOURCE of traffic.
> That's it. They define where the traffic "goes FROM".
> 
> Say, the DPDK application sends a packet from ethdev 0. This packet enters
> the e-switch. Match engine sits in the e-switch and intercepts the packet. It
> doesn't care where the packet *would go* if it wasn't intercepted. It cares
> about where the packet comes from. And it comes from ethdev 0. So, in the
> focus, we have the SOURCE of the packet.
> 

Agree with you we should only look at coming from,
but something in the patch made me thing otherwise (not sure what part)

> 
> >
> >> Yes, in this case neither of the ports (1, 3) is truly "external"
> >> (they both interface the DPDK application), but, the thing is,
> >> they're "external" *to each
> >> other* in the sense that they sit at the opposite ends of the wire.
> >>
> >>>
> >>> Matching on ethdev(1) means matching on the PF port in the E-Switch
> >>> but
> >> with some
> >>> metadata that marks the traffic as coming from DPDK port 1 and not
> >>> from
> >> VF1 E-Switch
> >>> port right?
> >>
> >> That's vendor specific. The application doesn't have to know how
> >> exactly this particular ethdev is connected to the e-switch - whether
> >> it re-uses the PF's e-switch port or has its own. The e-switch port
> >> that connects the ethdev with the e-switch is just assumed to exist
> logically.
> >>
> >>>
> >>> While matching on eswitch_port(2) means matching on traffic coming
> >>> from
> >> the VM right?
> >>
> >> Right.
> >>
> >
> > I think the my above question will clear everything for me.
> >

[Snip]
> >>>>>
> >>>>> Best,
> >>>>> Ori
> >>>>>
> >>>>
> >>>> --
> >>>> Ivan M
> >>
> >> --
> >> Ivan M
> >
> > Thanks,
> > Ori
> >
> 
> --
> Ivan M