From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from dpdk.org (dpdk.org [92.243.14.124]) by inbox.dpdk.org (Postfix) with ESMTP id 52A06A04A2; Mon, 4 Nov 2019 11:28:15 +0100 (CET) Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id C5E0237B4; Mon, 4 Nov 2019 11:28:14 +0100 (CET) Received: from relay9-d.mail.gandi.net (relay9-d.mail.gandi.net [217.70.183.199]) by dpdk.org (Postfix) with ESMTP id B856A37B0 for ; Mon, 4 Nov 2019 11:28:12 +0100 (CET) X-Originating-IP: 90.177.210.238 Received: from [192.168.1.110] (238.210.broadband10.iol.cz [90.177.210.238]) (Authenticated sender: i.maximets@ovn.org) by relay9-d.mail.gandi.net (Postfix) with ESMTPSA id 87F37FF805; Mon, 4 Nov 2019 10:28:08 +0000 (UTC) To: Shahaf Shuler , Ilya Maximets , Thomas Monjalon Cc: "dev@dpdk.org" , Jerin Jacob , Andrew Rybchenko , Ferruh Yigit , Stephen Hemminger , Roni Bar Yanai , Rony Efraim , "declan.doherty@intel.com" , "bernard.iremonger@intel.com" , "ajit.khaparde@broadcom.com" , "Ananyev, Konstantin" References: <4165509.5enYigmRGf@xps> <5190422.ormd5srm06@xps> <1968866.mbH2BcW0Fd@xps> From: Ilya Maximets Message-ID: <0fce7109-39da-1256-ba64-6b1291a8acbd@ovn.org> Date: Mon, 4 Nov 2019 11:28:08 +0100 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:60.0) Gecko/20100101 Thunderbird/60.9.0 MIME-Version: 1.0 In-Reply-To: Content-Type: text/plain; charset=utf-8; format=flowed Content-Language: en-US Content-Transfer-Encoding: 7bit Subject: Re: [dpdk-dev] [PATCH v2 0/3] ethdev: configure SR-IOV VF from host X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" On 03.11.2019 7:48, Shahaf Shuler wrote: > Friday, November 1, 2019 11:33 AM, Ilya Maximets: >> Subject: Re: [dpdk-dev] [PATCH v2 0/3] ethdev: configure SR-IOV VF from >> host >> >> On 30.10.2019 22:42, Thomas Monjalon wrote: >>> 30/10/2019 17:09, Ilya Maximets: >>>> On 30.10.2019 16:49, Thomas Monjalon wrote: >>>>> 30/10/2019 16:07, Ilya Maximets: >>>>>> On 29.10.2019 19:50, Thomas Monjalon wrote: >>>>>>> In a virtual environment, the network controller may have to >>>>>>> configure some SR-IOV VF parameters for security reasons. >>>>>>> > [...] > >>> If we consider what Intel did, i.e. configure VF in place of >>> representor for some operations, there are two drawbacks: >>> - confusing that some ops apply to representor, others apply to VF >>> - some ops are not possible on representor (because targetted to VF) >>> >>> I still feel that the addition of one single bit in the port ID is an >>> elegant solution to target either the VF or its representor. >> >> Since we already have a confusion about what is configured when operations >> are performed on a representor port we have 2 options: > > I don't agree we have. I don't think there is any design note or API doc that says the ethdev configuration on representor should be applied on VF (please share if I missed it). > The fact that there are some drivers that implemented it doesn't mean it is correct. > >> 1. Have this proposed API to configure representor itself while >> setting config to representor and configuring VF if special >> bit enabled. >> 2. Reverse the logic of current proposal, i.e. always apply >> configuration to VF while working with representor and apply >> configuration to representor itself if special bit is set. >> >> I'd probably prefer option #2, because: >> - From the OVS and OpenStack point of view, I think, we don't >> really need to configure representor itself in most cases. >> And OVS really should not know if it works with representor >> or some real port. > > I don't thinks OVS can be really agnostic to the fact it runs on top of representors: > 1. probing of representor has different command line -w ,representor=XXX OVS doesn't care about content of devargs. It just passes them to hotplug engine without any parsing (except a single case that must be eliminated with a proper device iterators, not an OVS issue). > 2. the whole acceleration framework based on insertion of flow rules for direct forward from the VF to target entity. Rules are applied on the representor and would not work if port is not such. OVS tries to offload rules to the netdev from which packet was received. That's it. If it succeeds - OK. If not, OVS doesn't care. > 3. some multi-port devices cannot do direct fwd between its different port. This is why rep has switch_id and application should query it and act upon. This is part of offloading engine that doesn't affect the generic code. If needed, OVS could request switch_id for netdev it tries to offload rules on. OVS should not know if it representor port or not. If this operation will not succeed for non-representors, OVS should not care because we can't offload anything for non-representors anyway. > 4. representor carry the VF port id. This is how application know to which VF (or vport) they associated with on their other side. This is just part of devargs, i.e. part of device unique identifier. Once again, OVS doesn't parse devargs and should not do that. > >> - It seems that most of the existing code in DPDK already works >> like this, i.e. applying configs to VF itself. Intel drivers >> works like this and Mellanox drivers, as Thomas said, doesn't >> have this functionality at all. > > As I said above, I don't think we need to refer to specific driver behavior, rather the API guidelines. > To me, it is a bit strange and not natural that ethdev configuration is applied to different port w/o any explicit request from the application. > This is why I would prefer #1 above. IMHO, the whole concept of representors is that representor is a way of attaching same port both to VM and vSwitch/hypervisor. If you're looking at representors as a separate ports on a switch, well.. In this case, for me VF configuration looks like something that vSwitch should not do at all, because it should not configure ports that doesn't attached to it. It's like configuring the other side of veth pair, which is nonsense. BTW, I don't know a way to find out if port is a representor of something or not in Linux kernel. > >> >>> >>> >>>>>> The this is that this new API will produce conceptual fragmentation >>>>>> between DPDK and the Linux kernel, because to do the same thing >>>>>> you'll have to use different ways. I mean, to change mac of VF in >>>>>> kernel you need to set mac to the representor, but in DPDK changing >>>>>> setting mac to representor will lead to changing the mac of the >>>>>> representor itself, not the VF. This will be really confusing for users. >>>>> >>>>> I am not responsible of the choices in Linux. >>>>> But I agree it would be interesting to check the reason of such decision. >>>>> Rony, please could you explain? >>> >>> I looked at few Linux drivers: >>> >>> bnxt_vf_rep_netdev_ops has no op to set MAC >>> bnxt_netdev_ops.ndo_set_vf_mac = set VF MAC from PF >>> >>> lio_vf_rep_ndev_ops has no op to set MAC >>> lionetdevops.ndo_set_vf_mac = set VF MAC from PF >>> >>> mlx5e_netdev_ops_rep has no op to set MAC >>> mlx5e_netdev_ops.ndo_set_vf_mac = set VF MAC from PF >>> mlx5e_netdev_ops_uplink_rep.ndo_set_vf_mac = set VF MAC from >> PF >>> >>> nfp_repr_netdev_ops.ndo_set_mac_address = set representor >> MAC >>> nfp_repr_netdev_ops.ndo_set_vf_mac = set VF MAC from >> representor >>> nfp_net_netdev_ops.ndo_set_vf_mac = set VF MAC from PF >>> >>> There is a big chance that the behaviour is not standardized in Linux >>> (as usual). So it is already confusing for users of Linux. >>> >>>