From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 21B72A0C47; Tue, 5 Oct 2021 14:13:00 +0200 (CEST) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id F097B4135F; Tue, 5 Oct 2021 14:12:59 +0200 (CEST) Received: from shelob.oktetlabs.ru (shelob.oktetlabs.ru [91.220.146.113]) by mails.dpdk.org (Postfix) with ESMTP id 3F24F41353 for ; Tue, 5 Oct 2021 14:12:58 +0200 (CEST) Received: from [100.65.5.102] (unknown [5.144.121.98]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by shelob.oktetlabs.ru (Postfix) with ESMTPSA id D84A87F4FD; Tue, 5 Oct 2021 15:12:57 +0300 (MSK) DKIM-Filter: OpenDKIM Filter v2.11.0 shelob.oktetlabs.ru D84A87F4FD DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple; d=oktetlabs.ru; s=default; t=1633435977; bh=k2wZb2+Ra+okODLZ6eW3Ueo/PHyMV2tSLy3qyeD7Mr4=; h=Subject:To:Cc:References:From:Date:In-Reply-To; b=gxDH7DKA+EYNn+YQqDZ3QPIvIuHsYvhT+uVNt/6YZqp6984AxgSQEwYBjtnwGzZd8 x8Cl8FQQ7v7WEmyrd+mNpbvpRkhxMN1eawXhfLgrnm+qAyw8mKHAI5dySUTaTsRGlE VyltxnCnRuR34XRBKIw0ouZfFF19BYUJ61xSPYb4= To: Andrew Rybchenko , Ori Kam Cc: "dev@dpdk.org" , Xiaoyun Li , NBU-Contact-Thomas Monjalon , Ferruh Yigit References: <20210907125157.3843-1-ivan.malov@oktetlabs.ru> <20211001134716.1608857-1-andrew.rybchenko@oktetlabs.ru> <20211001134716.1608857-3-andrew.rybchenko@oktetlabs.ru> <98369ad5-f6d8-dadb-194a-fe4f87a34a11@oktetlabs.ru> From: Ivan Malov Message-ID: Date: Tue, 5 Oct 2021 15:12:46 +0300 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:78.0) Gecko/20100101 Thunderbird/78.12.0 MIME-Version: 1.0 In-Reply-To: Content-Type: text/plain; charset=utf-8; format=flowed Content-Language: en-US Content-Transfer-Encoding: 7bit Subject: Re: [dpdk-dev] [PATCH v1 02/12] ethdev: add eswitch port item to flow API X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" Hi Ori, Andrew, On 05/10/2021 12:19, Andrew Rybchenko wrote: > Hi Ori, > > On 10/5/21 9:20 AM, Ori Kam wrote: >> Hi Ivan, >> >>> -----Original Message----- >>> From: Ivan Malov >>> Cc: dev@dpdk.org >>> Subject: Re: [PATCH v1 02/12] ethdev: add eswitch port item to flow API >>> >>> Hi Ori, >>> >>> On 04/10/2021 14:37, Ori Kam wrote: >>>> Hi Ivan, >>>> >>>>> -----Original Message----- >>>>> From: Ivan Malov >>>>> Sent: Monday, October 4, 2021 2:06 PM >>>>> Cc: dev@dpdk.org >>>>> Subject: Re: [PATCH v1 02/12] ethdev: add eswitch port item to flow >>>>> API >>>>> >>>>> Hi Ori, >>>>> >>>>> On 04/10/2021 08:45, Ori Kam wrote: >>>>>> Hi Ivan, >>>>>> >>>>>>> -----Original Message----- >>>>>>> From: Ivan Malov >>>>>>> Sent: Sunday, October 3, 2021 9:11 PM >>>>>>> Subject: Re: [PATCH v1 02/12] ethdev: add eswitch port item to flow >>>>>>> API >>>>>>> >>>>>>> >>>>>>> >>>>>>> On 03/10/2021 15:40, Ori Kam wrote: >>>>>>>> Hi Andrew and Ivan, >>>>>>>> >>>>>>>>> -----Original Message----- >>>>>>>>> From: Andrew Rybchenko >>>>>>>>> Sent: Friday, October 1, 2021 4:47 PM >>>>>>>>> Subject: [PATCH v1 02/12] ethdev: add eswitch port item to flow >>>>>>>>> API >>>>>>>>> >>>>>>>>> From: Ivan Malov >>>>>>>>> >>>>>>>>> For use with "transfer" flows. Supposed to match traffic entering >>>>>>>>> the e-switch from the external world (network, guests) via the >>>>>>>>> port which is logically connected with the given ethdev. >>>>>>>>> >>>>>>>>> Must not be combined with attributes "ingress" / "egress". >>>>>>>>> >>>>>>>>> This item is meant to use the same structure as ethdev item. >>>>>>>>> >>>>>>>> >>>>>>>> In case the app is not working with representors, meaning each >>>>>>>> switch port is mapped to ethdev. >>>>>>>> both items (ethdev and eswitch port ) have the same meaning? >>>>>>> >>>>>>> No. Ethdev means ethdev, and e-switch port is the point where this >>>>>>> ethdev is plugged to. For example, "transfer + ESWITCH_PORT" for a >>>>>>> regular PF ethdev typically means the network port (maybe you can >>>>>>> recall the idea that a PF ethdev "represents" the network port it's >>>>> associated with). >>>>>>> >>>>>>> I believe, that diagrams which these patches add to >>>>>>> "doc/guides/prog_guide/rte_flow.rst" may come in handy to >>>>>>> understand the meaning. Also, you can take a look at our larger >>>>>>> diagram from the Sep 14 gathering. >>>>>>> >>>>>> >>>>>> Lets look at the following system: >>>>>> E-Switch has 3 ports - PF, VF1, VF2 >>>>>> The ports are distributed as follows: >>>>>> DPDK application: >>>>>> ethdev(0) pf, >>>>>> ethdev(1) representor to VF1 >>>>>> ethdev(2) representor to VF2 >>>>>> ethdev(3) VF1 >>>>>> >>>>>> VM: >>>>>> VF2 >>>>>> >>>>>> As we know all representors are realy connected to the PF(at least >>>>>> in this example) >>>>> >>>>> This example tries to say that the e-switch has 3 ports in total, >>>>> and, given your explanation, one may indeed agree that *in this >>>>> example* representors re-use e-switch port of ethdev=0 (with some >>>>> metadata to distinguish packets, etc.). But one can hardly assume >>>>> that *all* representors with any vendor's NIC are connected to the >>>>> e-switch the same way. It's vendor specific. Well, at least, >>>>> applications don't have this knowledge and don't need to. >>>>> >>>>>> >>>>>> So matching on ethdev(3) means matching on traffic sent from DPDK >>>>>> port >>>>> 3 right? >>>>> >>>>> Item ETHDEV (ethdev_id=3) matches traffic sent by DPDK port 3. Looks >>>>> like we're on the same page here. >>>>> >>>> >>>> Good. >>>> >>>>>> And matching on eswitch_port(3) means matching in traffic that goes >>>>>> into VF1 which is the same traffic as ethdev(3) right? >>>>> >>>>> I didn't catch the thought about "the same traffic". Direction is not the >>> same. >>>>> Item ESWITCH_PORT (ethdev_id=3) matches traffic sent by DPDK port 1. >>>>> >>>> This is the critical part for my understanding. >>>> Matching on ethdev_id(3) means matching on traffic that is coming from >>> DPDK port3. >>> >>> Right. >>> >>>> So from E-Switch view point it is traffic that goes into VF1? >>> >>> No. Above you clearly say "coming from DPDK port3". That is, from the VF1. >>> *Not* going into it. Port 3 (ethdev_id=3) *is* VF1. >>> >> But taffic that goes from DPDK port 3 goes into VF1, >> what am I missing? Terms like "PF", "VF", "PHY_PORT" describe the nature of the e-switch ports. In your example, DPDK port 3 is just based on VF1. The application doesn't have such knowledge. To it, this is a regular ethdev. In order to gain correct understanding, you should imagine an observer standing inside the e-switch. When that observer sees packets entering the e-switch FROM the VF1, we effectively talk about packets sent by the ethdev sitting on top of that VF1, that is, packets coming FROM DPDK port 3. And vice versa: if you say "traffic that goes into VF1", you mean traffic that LEAVES the e-switch via VF1, that is, traffic, going TO DPDK port 3. "VF1" is just a name of a logical "window" between the e-switch and its surroundings. If this VF is passed to a guest machine, then this VF is a "window" between the e-switch and the guest machine. If you plug this VF to the DPDK application (create an ethdev on top of this VF), then this VF is a "window" between the e-switch and the PMD. That's it. > > DPDK port 3 is a VF1 itself. So, is it loopback? I think no. > > Let me repeat your example: > > DPDK application: > ethdev(0) pf, > ethdev(1) representor to VF1 > ethdev(2) representor to VF2 > ethdev(3) VF1 > > VM: > VF2 > > Traffic that goes from DPDK port 3 (which is VF1 bound to DPDK) > goes to its representor by default, i.e. ethdev(1). +1 > >> >>>> While matching on E-Switch_port(3) means matching on traffic coming >>> from VF1? >>> >>> No. It means matching on traffic coming from ethdev 1. From the VF1's >>> representor. >>> >>>> >>>> And by the same logic matching on ethdev_id(1) means matching on >>>> taffic that was sent from DPDK port 1 and matching on E-Switch_port(1) >>>> means matching on traffic coming from >>>> VF1 >>> >>> In this case, you've got this right. But please see my above notes. By the >>> looks of it, you might have run into confusion over there. >>> >> That is the issue I'm not sure I understand, and I think that >> if I don't underdand this, this miss underdanding will be shared by others. Please see my diagram below. > > In order to address the confusion and misunderstanding we > should find out the source of it. We always match on source > (inbound port) and the result is a destination (outgoing port). > So, if item is ETHDEV, the source is DPDK application via > corresponding ethdev port. If ETHDEV is an action, the > destination is the DPDK application via specified ethdev port. > Above is regardless of the ethdev port type (representor or > not). Always, no exceptions. > >>>> >>>> So in this case eswitch_port(3) is equal ot eswitch_port(1) right? >>>> While ethdev(1) is not equal to ethdev(3) >>> >>> No. >>> >>> Item ETHDEV (ethdev_id=1) equals item ESWITCH_PORT (ethdev_id=3). >>> Item ETHDEV (ethdev_id=3) equals item ESWITCH_PORT (ethdev_id=1). >>> >> I think this was my first undestaning, lets see if I can explain it using my words. >> ETHDEV - means that the source port that we are matching on is the closest one >> the dpdk application. > > Yes, it sounds right. > >> Example like above ETHDEV(ethdev_id=1) means matching >> on traffic coming from the PF with some metadata that marks this DPDK port. Some vendors support allocation of separate logical e-switch ports for representors. Other vendors can't support that, and, in this case, they indeed have to devise "PF + metadata" solution. But DPDK framework (I mean, vendor-agnostic code) can't assume any of these options as a "standard". Neither can applications do that. It's truly vendor-specific. > > No, no, no. It is the problem of too much knowledge and > diving too deep. Neither you nor API user should not > think about it. Representor ethdev(1) is a DPDK ethdev > port that's it. An application can send traffic via the > ethdev port and receive traffic from it. How the traffic > goes inside is vendor-specific implementation detail. > Application just know that if it sends traffic to VF1 > representor ethdev(1), the traffic will be received > from VF1 by default. If it sends traffic from VF1, > the traffic will be received from VF1 representor > by default. It is the definition of the representor. +1 > >> ETHDEV(ethdev_id=3) means matching on traffic coming from VF1. > > Yes. > >> >> ESWITCH_PORT meaning matching on the port that is connected to the DPDK port >> (the other side of the wire). Example ESWITCH_PORT(ethdev_id=1) means matching >> on traffic coming from VF1 > > Yes, since ethdev(1) is VF1 representor. > >> While matching on ESWITCH_PORT(ethdev_id=3) means matching on PF with some >> metadata. > > Again, yes and no. No, since you should not think about it this way. It > is just traffic sent via VF1 representor (since ethdev(3) is VF1 and its > other side of the logical wire is VF1 > representor). No vendor-specific implementation details. > >> >> Everything assume that representors are on the PF and use some metadata. > > No, it is a vendor-specific implementation details. > Flow API definitions should not mention it. +1 > >> >> Did I get it right? Let's try to look at the overall idea one more time. Just to get us on the same page. In general, for any given ethdev you have: ETHDEV (A) | | (PMD area) | CLOSEST ESWITCH_PORT (B) | | (e-switch area) | REMOTE ESWITCH_PORT (C) | X The point "B" is located closest to the ethdev (A). Not to the application in general, but to the given ethdev. The point "C" is the most remote point from the ethdev's perspective. Again, it's not the most remote from the application. It's the most remote from this specific ethdev. From the application perspective, the nature of (B) and (C) is unknown. But the application knows for sure that these points just exist. To it, these points are some *logical* e-switch ports. Now. When you say "item ETHDEV", you tell the PMD that you want to match packets entering the *PMD area* FROM point (A). But you also add "transfer" attribute. This attribute makes the PMD translate your request to the e-switch viewpoint by "transferring" point "A" one level below. What was "A" becomes "B". This way, an imaginary observer standing inside the *e-switch area* intercepts packets which enter this area FROM (B). Summary: this way, you match packets going *DOWN* from (A). When you say "item ESWITCH_PORT", you tell the PMD that you want to match packets entering the *PMD area* FROM point (B). Once again, "transfer" translates this to the e-switch viewpoint, that is, (B) becomes (C). So, the e-switch knows that it should intercept packets entering its area FROM point (C). Summary: this way, you match packets going *UP* from (X). Some use case examples: - The ethdev (A) is a DPDK port based on a PF associated with a network port. In this case, (B) == PF and (C) == (X) == network port. - The ethdev (A) is a representor of VF1 which is passed to a VM. In this case, (B) is just an e-switch logical port. It's nature is vendor specific. It can be a PF, it can be not. It depends. (C) == VF, (X) == VM. - The ethdev (A) is a representor of VF1 which is plugged to the same DPDK application. In this case, (B) is, once again, some logical e-switch port. (C) == VF. But (X) here is another ethdev. So, this way, ethdev (A) is a representor's ethdev and ethdev (X) is a VF's ethdev. In all cases the application doesn't care about the nature of the e-switch ports (B) and (C). It just knows that they exist. > > I think you get overall idea right, but explanations > are too complicated. It is simpler in fact. > >>>> >>>> And just to complete the picture, matching on ethdev(2) will result in >>>> traffic coming from the dpdk port and matching on eswitch_port(2) will >>>> match on traffic coming from VF2 >>> >>> Exactly. >>> >>> >>> But, Ori, let me draw your attention to the following issue. In order to >>> simplify understanding, I suggest that we refrain from saying "traffic that >>> GOES TO". Where it goes depends on default rules that are supposed to be >>> maintained by the PMD when ports get plugged / unplugged. >>> >>> The flow items ETHDEV and ESWITH_PORT define the SOURCE of traffic. >>> That's it. They define where the traffic "goes FROM". >>> >>> Say, the DPDK application sends a packet from ethdev 0. This packet enters >>> the e-switch. Match engine sits in the e-switch and intercepts the packet. It >>> doesn't care where the packet *would go* if it wasn't intercepted. It cares >>> about where the packet comes from. And it comes from ethdev 0. So, in the >>> focus, we have the SOURCE of the packet. >>> >> >> Agree with you we should only look at coming from, >> but something in the patch made me thing otherwise (not sure what part) > > I've reread the documentation and failed to find, > but it will be hard for me to do it, since I read > it too many times already. > > Thanks, > Andrew. > -- Ivan M