Re: [dpdk-dev] [PATCH v1 02/12] ethdev: add eswitch port item to flow API

[Ivan Malov <Ivan.Malov@oktetlabs.ru>]

BTW, one more alternative to "MIRROR_PORT" is "REMOTE_PORT".

On 06/10/2021 18:30, Ivan Malov wrote:
> Hi Ori,
> 
> By the looks of it, we are starting to run into slight misunderstanding.
> 
> As I see it, the main consequence of our Sep 14 gathering in Jitsi was
> understanding of the fact that the concept of item / action PORT_ID is
> vague and needs a replacement. As a bare minimum, separate items
> should be used: one for an ethdev port and another one for
> the "represented entity".
> 
> This "represented entity" can be network (via network port), or a guest
> machine (via a PF / VF plugged to it), or another ethdev (in the case
> when the ethdev we are looking at is a PF/VF representor and this
> PF/VF is also plugged to the DPDK application).
> 
> So, if I get this right, you don't object this summary. Very well.
> 
> But, in the current approach, we stick with term "ESWITCH_PORT" for
> that "represented entity", and, as I see it, this term is not quite
> descriptive when someone tries to understand which exact port of
> the embedded switch is implied. Attempts to clarify it by virtue
> of terms "external" or "the most remote" are not-so-successful.
> 
> I fully understand that.
> 
> But the good news is that the original diagram of the new concept can
> be improved in a way that can allow to dispose of misleading words.
> 
> 
>       [ A ]       <-- ethdev
>         |
>       [ B ]       <-- embedded switch (logical) port
>         |
>         |
>         |
> ===============  <-- plane of symmetry
>         |
>         |
>         |
>       [ C ]       <-- embedded switch (logical) port
>         |
>       [ D ]       <-- represented entity
> 
> 
> 1. The application sees the ethdev (A) and can assume that this
>     ethdev is served by some logical port (B) in the embedded
>     switch. Precise nature of this port is vendor-specific.
> 
>     For example, for a regular (non-representor) ethdev,
>     this port can be a PF, or a VF. This is obvious to
>     DPDK developers, but the application doesn't need
>     to have this knowledge. It only sees the ethdev.
> 
>     If this ethdev is a representor, port (B) can be a truly
>     separate logical port or, alternatively, some vendors
>     may use "PF + metadata" approach. This port (B) is
>     just assumed to exist. The rest is vendor-specific.
> 
> 2. The application fully understands that the "wire" plugged to
>     the ethdev it sees has an opposite end. Over there, there's
>     some "represented entity" (D). Once again, the application
>     does not know the nature of that "represented entity".
>     To it, this entity is just some traffic endpoint.
> 
>     And the application understands that this "represented entity"
>     is connected to the NIC by means of another logical port (C).
>     The nature of this port is unknown. The application does not
>     need to have this knowledge. To it, this port just exists.
> 
>     Examples of precise nature of (C) / (D):
>     -- (D) = network;        (C) = network port
>     -- (D) = guest machine;  (C) = VF / PF passed to the VM
>     -- (D) = another ethdev; (C) = the ethdev's logical port
> 
>     For "the ethdev's logical port" - see the explanation in (1).
> 
> 3. The pair of points (A), (B) is SYMMETRICAL to the pair
>     of points (C), (D). With respect to any given ethdev,
>     the embedded switch is split by a plane of symmetry.
> 
> 4. The property of symmetry in this drawing allows to perceive
>     point (D) as a reflection of point (A). A mirrored image.
> 
> 
> So, the short of it, looking at the problem the way this picture
> looks at it, one can easily understand that the "represented
> entity" is just a reflection of the ethdev (and vice versa).
> 
> No more need for wordy descriptions like "the most remote port".
> The symmetry says it all. But the problem is the name for the
> new item and action. Instead of ESWITCH_PORT, something else
> should be suggested to make readers think of this symmetry.
> 
> To me, the most clear term here would be MIRROR_PORT. As per the rules
> of English language, "MIRROR" here is like an adjective. It describes
> the property of the "PORT" being a reflection of the corresponding
> ethdev. And I'd call the latter an ETHDEV_PORT.
> 
> So, ETHDEV_PORT and its MIRROR_PORT.
> 
> Yes-yes, I do understand that "mirror" may at some point turn
> misleading to some readers who may perceive this word as a
> verb and confuse the item / action meaning with the concept
> of port mirroring. Alright: I'm aware of that. So let me
> then suggest another option: REFLEX_PORT.
> 
> REFLEX_PORT doesn't seem to sound as good as MIRROR_PORT,
> but it also indicates the property of this entity being
> a reflection of the ethdev port (and vice versa).
> 
> What do you think?  Does it make sense to make another version of
> our patch series with the new term in use and with the new
> explanation? Should you wish to suggest your own naming
> variants, you're welcome to do so.
> 
> On 05/10/2021 15:12, Ivan Malov wrote:
>  > Hi Ori, Andrew,
>  >
>  > On 05/10/2021 12:19, Andrew Rybchenko wrote:
>  > > Hi Ori,
>  > >
>  > > On 10/5/21 9:20 AM, Ori Kam wrote:
>  > >> Hi Ivan,
>  > >>
>  > >>> -----Original Message-----
>  > >>> From: Ivan Malov <Ivan.Malov at oktetlabs.ru>
>  > >>> Cc: dev at dpdk.org
>  > >>> Subject: Re: [PATCH v1 02/12] ethdev: add eswitch port item to 
> flow API
>  > >>>
>  > >>> Hi Ori,
>  > >>>
>  > >>> On 04/10/2021 14:37, Ori Kam wrote:
>  > >>>> Hi Ivan,
>  > >>>>
>  > >>>>> -----Original Message-----
>  > >>>>> From: Ivan Malov <Ivan.Malov at oktetlabs.ru>
>  > >>>>> Sent: Monday, October 4, 2021 2:06 PM
>  > >>>>> Cc: dev at dpdk.org
>  > >>>>> Subject: Re: [PATCH v1 02/12] ethdev: add eswitch port item to 
> flow
>  > >>>>> API
>  > >>>>>
>  > >>>>> Hi Ori,
>  > >>>>>
>  > >>>>> On 04/10/2021 08:45, Ori Kam wrote:
>  > >>>>>> Hi Ivan,
>  > >>>>>>
>  > >>>>>>> -----Original Message-----
>  > >>>>>>> From: Ivan Malov <Ivan.Malov at oktetlabs.ru>
>  > >>>>>>> Sent: Sunday, October 3, 2021 9:11 PM
>  > >>>>>>> Subject: Re: [PATCH v1 02/12] ethdev: add eswitch port item 
> to flow
>  > >>>>>>> API
>  > >>>>>>>
>  > >>>>>>>
>  > >>>>>>>
>  > >>>>>>> On 03/10/2021 15:40, Ori Kam wrote:
>  > >>>>>>>> Hi Andrew and Ivan,
>  > >>>>>>>>
>  > >>>>>>>>> -----Original Message-----
>  > >>>>>>>>> From: Andrew Rybchenko <andrew.rybchenko at oktetlabs.ru>
>  > >>>>>>>>> Sent: Friday, October 1, 2021 4:47 PM
>  > >>>>>>>>> Subject: [PATCH v1 02/12] ethdev: add eswitch port item to 
> flow
>  > >>>>>>>>> API
>  > >>>>>>>>>
>  > >>>>>>>>> From: Ivan Malov <ivan.malov at oktetlabs.ru>
>  > >>>>>>>>>
>  > >>>>>>>>> For use with "transfer" flows. Supposed to match traffic 
> entering
>  > >>>>>>>>> the e-switch from the external world (network, guests) via the
>  > >>>>>>>>> port which is logically connected with the given ethdev.
>  > >>>>>>>>>
>  > >>>>>>>>> Must not be combined with attributes "ingress" / "egress".
>  > >>>>>>>>>
>  > >>>>>>>>> This item is meant to use the same structure as ethdev item.
>  > >>>>>>>>>
>  > >>>>>>>>
>  > >>>>>>>> In case the app is not working with representors, meaning each
>  > >>>>>>>> switch port is mapped to ethdev.
>  > >>>>>>>> both items (ethdev and eswitch port ) have the same meaning?
>  > >>>>>>>
>  > >>>>>>> No. Ethdev means ethdev, and e-switch port is the point where 
> this
>  > >>>>>>> ethdev is plugged to. For example, "transfer + ESWITCH_PORT" 
> for a
>  > >>>>>>> regular PF ethdev typically means the network port (maybe you 
> can
>  > >>>>>>> recall the idea that a PF ethdev "represents" the network 
> port it's
>  > >>>>> associated with).
>  > >>>>>>>
>  > >>>>>>> I believe, that diagrams which these patches add to
>  > >>>>>>> "doc/guides/prog_guide/rte_flow.rst" may come in handy to
>  > >>>>>>> understand the meaning. Also, you can take a look at our larger
>  > >>>>>>> diagram from the Sep 14 gathering.
>  > >>>>>>>
>  > >>>>>>
>  > >>>>>> Lets look at the following system:
>  > >>>>>> E-Switch has 3 ports - PF, VF1, VF2
>  > >>>>>> The ports are distributed as follows:
>  > >>>>>> DPDK application:
>  > >>>>>> ethdev(0) pf,
>  > >>>>>> ethdev(1) representor to VF1
>  > >>>>>> ethdev(2) representor to VF2
>  > >>>>>> ethdev(3) VF1
>  > >>>>>>
>  > >>>>>> VM:
>  > >>>>>> VF2
>  > >>>>>>
>  > >>>>>> As we know all representors are realy connected to the PF(at 
> least
>  > >>>>>> in this example)
>  > >>>>>
>  > >>>>> This example tries to say that the e-switch has 3 ports in total,
>  > >>>>> and, given your explanation, one may indeed agree that *in this
>  > >>>>> example* representors re-use e-switch port of ethdev=0 (with some
>  > >>>>> metadata to distinguish packets, etc.). But one can hardly assume
>  > >>>>> that *all* representors with any vendor's NIC are connected to the
>  > >>>>> e-switch the same way. It's vendor specific. Well, at least,
>  > >>>>> applications don't have this knowledge and don't need to.
>  > >>>>>
>  > >>>>>>
>  > >>>>>> So matching on ethdev(3)  means matching on traffic sent from 
> DPDK
>  > >>>>>> port
>  > >>>>> 3 right?
>  > >>>>>
>  > >>>>> Item ETHDEV (ethdev_id=3) matches traffic sent by DPDK port 3. 
> Looks
>  > >>>>> like we're on the same page here.
>  > >>>>>
>  > >>>>
>  > >>>> Good.
>  > >>>>
>  > >>>>>> And matching on eswitch_port(3) means matching in traffic that 
> goes
>  > >>>>>> into VF1 which is the same traffic as ethdev(3) right?
>  > >>>>>
>  > >>>>> I didn't catch the thought about "the same traffic". Direction 
> is not the
>  > >>> same.
>  > >>>>> Item ESWITCH_PORT (ethdev_id=3) matches traffic sent by DPDK 
> port 1.
>  > >>>>>
>  > >>>> This is the critical part for my understanding.
>  > >>>> Matching on ethdev_id(3) means matching on traffic that is 
> coming from
>  > >>> DPDK port3.
>  > >>>
>  > >>> Right.
>  > >>>
>  > >>>> So from E-Switch view point it is traffic that goes into VF1?
>  > >>>
>  > >>> No. Above you clearly say "coming from DPDK port3". That is, from 
> the VF1.
>  > >>> *Not* going into it. Port 3 (ethdev_id=3) *is* VF1.
>  > >>>
>  > >> But taffic that goes from DPDK port 3 goes into VF1,
>  > >> what am I missing?
>  >
>  > Terms like "PF", "VF", "PHY_PORT" describe the nature of the e-switch
>  > ports. In your example, DPDK port 3 is just based on VF1. The
>  > application doesn't have such knowledge. To it, this is a regular 
> ethdev.
>  >
>  > In order to gain correct understanding, you should imagine an observer
>  > standing inside the e-switch. When that observer sees packets entering
>  > the e-switch FROM the VF1, we effectively talk about packets sent by the
>  > ethdev sitting on top of that VF1, that is, packets coming FROM DPDK
>  > port 3. And vice versa: if you say "traffic that goes into VF1", you
>  > mean traffic that LEAVES the e-switch via VF1, that is, traffic, going
>  > TO DPDK port 3.
>  >
>  > "VF1" is just a name of a logical "window" between the e-switch and its
>  > surroundings. If this VF is passed to a guest machine, then this VF is a
>  > "window" between the e-switch and the guest machine. If you plug this VF
>  > to the DPDK application (create an ethdev on top of this VF), then this
>  > VF is a "window" between the e-switch and the PMD. That's it.
>  >
>  > >
>  > > DPDK port 3 is a VF1 itself. So, is it loopback? I think no.
>  > >
>  > > Let me repeat your example:
>  > >
>  > > DPDK application:
>  > > ethdev(0) pf,
>  > > ethdev(1) representor to VF1
>  > > ethdev(2) representor to VF2
>  > > ethdev(3) VF1
>  > >
>  > > VM:
>  > > VF2
>  > >
>  > > Traffic that goes from DPDK port 3 (which is VF1 bound to DPDK)
>  > > goes to its representor by default, i.e. ethdev(1).
>  >
>  > +1
>  >
>  > >
>  > >>
>  > >>>> While matching on E-Switch_port(3) means matching on traffic coming
>  > >>> from VF1?
>  > >>>
>  > >>> No. It means matching on traffic coming from ethdev 1. From the 
> VF1's
>  > >>> representor.
>  > >>>
>  > >>>>
>  > >>>> And by the same logic matching on ethdev_id(1) means matching on
>  > >>>> taffic that was sent from DPDK port 1 and matching on 
> E-Switch_port(1)
>  > >>>> means matching on traffic coming from
>  > >>>> VF1
>  > >>>
>  > >>> In this case, you've got this right. But please see my above 
> notes. By the
>  > >>> looks of it, you might have run into confusion over there.
>  > >>>
>  > >> That is the issue I'm not sure I understand, and I think that
>  > >> if I don't underdand this, this miss underdanding will be shared 
> by others.
>  >
>  > Please see my diagram below.
>  >
>  > >
>  > > In order to address the confusion and misunderstanding we
>  > > should find out the source of it. We always match on source
>  > > (inbound port) and the result is a destination (outgoing port).
>  > > So, if item is ETHDEV, the source is DPDK application via
>  > > corresponding ethdev port.  If ETHDEV is an action, the
>  > > destination is the DPDK application via specified ethdev port.
>  > > Above is regardless of the ethdev port type (representor or
>  > > not). Always, no exceptions.
>  > >
>  > >>>>
>  > >>>> So in this case eswitch_port(3) is equal ot eswitch_port(1) right?
>  > >>>> While ethdev(1) is not equal to ethdev(3)
>  > >>>
>  > >>> No.
>  > >>>
>  > >>> Item ETHDEV (ethdev_id=1) equals item ESWITCH_PORT (ethdev_id=3).
>  > >>> Item ETHDEV (ethdev_id=3) equals item ESWITCH_PORT (ethdev_id=1).
>  > >>>
>  > >> I think this was my first undestaning, lets see if I can explain 
> it using my words.
>  > >> ETHDEV - means that the source port that we are matching on is the 
> closest one
>  > >> the dpdk application.
>  > >
>  > > Yes, it sounds right.
>  > >
>  > >> Example like above ETHDEV(ethdev_id=1) means matching
>  > >> on traffic coming from the PF with some metadata that marks this 
> DPDK port.
>  >
>  > Some vendors support allocation of separate logical e-switch ports for
>  > representors. Other vendors can't support that, and, in this case, they
>  > indeed have to devise "PF + metadata" solution. But DPDK framework (I
>  > mean, vendor-agnostic code) can't assume any of these options as a
>  > "standard". Neither can applications do that. It's truly 
> vendor-specific.
>  >
>  > >
>  > > No, no, no. It is the problem of too much knowledge and
>  > > diving too deep. Neither you nor API user should not
>  > > think about it. Representor ethdev(1) is a DPDK ethdev
>  > > port that's it. An application can send traffic via the
>  > > ethdev port and receive traffic from it. How the traffic
>  > > goes inside is vendor-specific implementation detail.
>  > > Application just know that if it sends traffic to VF1
>  > > representor ethdev(1), the traffic will be received
>  > > from VF1 by default. If it sends traffic from VF1,
>  > > the traffic will be received from VF1 representor
>  > > by default. It is the definition of the representor.
>  >
>  > +1
>  >
>  > >
>  > >> ETHDEV(ethdev_id=3) means matching on traffic coming from VF1.
>  > >
>  > > Yes.
>  > >
>  > >>
>  > >> ESWITCH_PORT meaning matching on the port that is connected to the 
> DPDK port
>  > >> (the other side of the wire). Example ESWITCH_PORT(ethdev_id=1) 
> means matching
>  > >> on traffic coming from VF1
>  > >
>  > > Yes, since ethdev(1) is VF1 representor.
>  > >
>  > >> While matching on ESWITCH_PORT(ethdev_id=3) means matching on PF 
> with some
>  > >> metadata.
>  > >
>  > > Again, yes and no. No, since you should not think about it this 
> way. It
>  > > is just traffic sent via VF1 representor (since ethdev(3) is VF1 
> and its
>  > > other side of the logical wire is VF1
>  > > representor). No vendor-specific implementation details.
>  > >
>  > >>
>  > >> Everything assume that representors are on the PF and use some 
> metadata.
>  > >
>  > > No, it is a vendor-specific implementation details.
>  > > Flow API definitions should not mention it.
>  >
>  > +1
>  >
>  > >
>  > >>
>  > >> Did I get it right?
>  >
>  > Let's try to look at the overall idea one more time. Just to get us on
>  > the same page.
>  >
>  > In general, for any given ethdev you have:
>  >
>  >
>  > ETHDEV (A)
>  > |
>  > |    (PMD area)
>  > |
>  > CLOSEST ESWITCH_PORT (B)
>  > |
>  > |    (e-switch area)
>  > |
>  > REMOTE ESWITCH_PORT (C)
>  > |
>  > X
>  >
>  >
>  > The point "B" is located closest to the ethdev (A). Not to the
>  > application in general, but to the given ethdev.
>  >
>  > The point "C" is the most remote point from the ethdev's perspective.
>  > Again, it's not the most remote from the application. It's the most
>  > remote from this specific ethdev.
>  >
>  >  From the application perspective, the nature of (B) and (C) is unknown.
>  > But the application knows for sure that these points just exist. To it,
>  > these points are some *logical* e-switch ports.
>  >
>  > Now.
>  >
>  > When you say "item ETHDEV", you tell the PMD that you want to match
>  > packets entering the *PMD area* FROM point (A). But you also add
>  > "transfer" attribute. This attribute makes the PMD translate your
>  > request to the e-switch viewpoint by "transferring" point "A" one level
>  > below. What was "A" becomes "B". This way, an imaginary observer
>  > standing inside the *e-switch area* intercepts packets which enter this
>  > area FROM (B).
>  > Summary: this way, you match packets going *DOWN* from (A).
>  >
>  > When you say "item ESWITCH_PORT", you tell the PMD that you want to
>  > match packets entering the *PMD area* FROM point (B). Once again,
>  > "transfer" translates this to the e-switch viewpoint, that is, (B)
>  > becomes (C). So, the e-switch knows that it should intercept packets
>  > entering its area FROM point (C).
>  > Summary: this way, you match packets going *UP* from (X).
>  >
>  >
>  > Some use case examples:
>  >
>  > - The ethdev (A) is a DPDK port based on a PF associated with a network
>  > port. In this case, (B) == PF and (C) == (X) == network port.
>  >
>  > - The ethdev (A) is a representor of VF1 which is passed to a VM. In
>  > this case, (B) is just an e-switch logical port. It's nature is vendor
>  > specific. It can be a PF, it can be not. It depends. (C) == VF, (X) 
> == VM.
>  >
>  > - The ethdev (A) is a representor of VF1 which is plugged to the same
>  > DPDK application. In this case, (B) is, once again, some logical
>  > e-switch port. (C) == VF. But (X) here is another ethdev. So, this way,
>  > ethdev (A) is a representor's ethdev and ethdev (X) is a VF's ethdev.
>  >
>  > In all cases the application doesn't care about the nature of the
>  > e-switch ports (B) and (C). It just knows that they exist.
>  >
>  > >
>  > > I think you get overall idea right, but explanations
>  > > are too complicated. It is simpler in fact.
>  > >
>  > >>>>
>  > >>>> And just to complete the picture, matching on ethdev(2) will 
> result in
>  > >>>> traffic coming from the dpdk port and matching on 
> eswitch_port(2) will
>  > >>>> match on traffic coming from VF2
>  > >>>
>  > >>> Exactly.
>  > >>>
>  > >>>
>  > >>> But, Ori, let me draw your attention to the following issue. In 
> order to
>  > >>> simplify understanding, I suggest that we refrain from saying 
> "traffic that
>  > >>> GOES TO". Where it goes depends on default rules that are 
> supposed to be
>  > >>> maintained by the PMD when ports get plugged / unplugged.
>  > >>>
>  > >>> The flow items ETHDEV and ESWITH_PORT define the SOURCE of traffic.
>  > >>> That's it. They define where the traffic "goes FROM".
>  > >>>
>  > >>> Say, the DPDK application sends a packet from ethdev 0. This 
> packet enters
>  > >>> the e-switch. Match engine sits in the e-switch and intercepts 
> the packet. It
>  > >>> doesn't care where the packet *would go* if it wasn't 
> intercepted. It cares
>  > >>> about where the packet comes from. And it comes from ethdev 0. 
> So, in the
>  > >>> focus, we have the SOURCE of the packet.
>  > >>>
>  > >>
>  > >> Agree with you we should only look at coming from,
>  > >> but something in the patch made me thing otherwise (not sure what 
> part)
>  > >
>  > > I've reread the documentation and failed to find,
>  > > but it will be hard for me to do it, since I read
>  > > it too many times already.
>  > >
>  > > Thanks,
>  > > Andrew.
>  > >
> 

-- 
Ivan M