From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from dpdk.org (dpdk.org [92.243.14.124]) by inbox.dpdk.org (Postfix) with ESMTP id 4B609A00BE; Wed, 30 Oct 2019 22:42:24 +0100 (CET) Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id 462E01BF71; Wed, 30 Oct 2019 22:42:23 +0100 (CET) Received: from new3-smtp.messagingengine.com (new3-smtp.messagingengine.com [66.111.4.229]) by dpdk.org (Postfix) with ESMTP id 5B5CA1BEB7 for ; Wed, 30 Oct 2019 22:42:20 +0100 (CET) Received: from compute1.internal (compute1.nyi.internal [10.202.2.41]) by mailnew.nyi.internal (Postfix) with ESMTP id A40E15CF8; Wed, 30 Oct 2019 17:42:19 -0400 (EDT) Received: from mailfrontend2 ([10.202.2.163]) by compute1.internal (MEProxy); Wed, 30 Oct 2019 17:42:19 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=monjalon.net; h= from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding:content-type; s=mesmtp; bh=5f2GpVdfSlL4aLV/2BxCJlC0JJR+8YfvjFQqavElM+o=; b=YPlQAnkw54U2 bAbhjx3NsYAW+/weVjnA/9quDaFG1S7Md/a0G6HE0Q1Jkg/5C2vIYGYV/Rg87Jwj rbp5bKAChozJoO6MwG/H7eyplRqimKFHM15VDTsXWSIKf+AjVCfzqF6xfdIRyf75 +K5cY7zOKBpukV2nTgZlwwHml+HnFfA= DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d= messagingengine.com; h=cc:content-transfer-encoding:content-type :date:from:in-reply-to:message-id:mime-version:references :subject:to:x-me-proxy:x-me-proxy:x-me-sender:x-me-sender :x-sasl-enc; s=fm1; bh=5f2GpVdfSlL4aLV/2BxCJlC0JJR+8YfvjFQqavElM +o=; b=X9GuDD9OqJTHk41v0A9yMycyUbC4jDO4Sz4jw48eemogUfigcsf9nCCFh NIiQoBl6foj7R1js15yFvv3LIvqZScmtZiRLLv7Nlgeg0Iow2fnP5SKcwpOdHg+A rKcxReKrJqIl0OkX0LB3AqSiBJ2ypDVsREzbUHhMb+YJzPkoYw9peZ1sigZltAPT vv86SubWjVMxCzSMT92u+qELqJaQDbQ5V3SiDZhh7BeWqJREkg1uxWd7zGY5PzB+ 6IC7qceNRrs00nSnCSyfJaYhkLZVsAs4iWpvRwH1bNtTcqrCBDzpAboiOkb3WTJQ dRmcMlEp5VXGo4q0JaAuDLeSLAvJg== X-ME-Sender: X-ME-Proxy-Cause: gggruggvucftvghtrhhoucdtuddrgedufedruddtfedgudehjecutefuodetggdotefrod ftvfcurfhrohhfihhlvgemucfhrghsthforghilhdpqfgfvfdpuffrtefokffrpgfnqfgh necuuegrihhlohhuthemuceftddtnecusecvtfgvtghiphhivghnthhsucdlqddutddtmd enucfjughrpefhvffufffkjghfggfgtgesthfuredttddtvdenucfhrhhomhepvfhhohhm rghsucfoohhnjhgrlhhonhcuoehthhhomhgrshesmhhonhhjrghlohhnrdhnvghtqeenuc fkphepjeejrddufeegrddvtdefrddukeegnecurfgrrhgrmhepmhgrihhlfhhrohhmpeht hhhomhgrshesmhhonhhjrghlohhnrdhnvghtnecuvehluhhsthgvrhfuihiivgeptd X-ME-Proxy: Received: from xps.localnet (184.203.134.77.rev.sfr.net [77.134.203.184]) by mail.messagingengine.com (Postfix) with ESMTPA id 44DA2306005B; Wed, 30 Oct 2019 17:42:17 -0400 (EDT) From: Thomas Monjalon To: Ilya Maximets Cc: dev@dpdk.org, Shahaf Shuler , Jerin Jacob , Andrew Rybchenko , Ferruh Yigit , Stephen Hemminger , Roni Bar Yanai , Rony Efraim , declan.doherty@intel.com, bernard.iremonger@intel.com, ajit.khaparde@broadcom.com Date: Wed, 30 Oct 2019 22:42:16 +0100 Message-ID: <1968866.mbH2BcW0Fd@xps> In-Reply-To: References: <4165509.5enYigmRGf@xps> <5190422.ormd5srm06@xps> MIME-Version: 1.0 Content-Transfer-Encoding: 7Bit Content-Type: text/plain; charset="us-ascii" Subject: Re: [dpdk-dev] [PATCH v2 0/3] ethdev: configure SR-IOV VF from host X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" 30/10/2019 17:09, Ilya Maximets: > On 30.10.2019 16:49, Thomas Monjalon wrote: > > 30/10/2019 16:07, Ilya Maximets: > >> On 29.10.2019 19:50, Thomas Monjalon wrote: > >>> In a virtual environment, the network controller may have to configure > >>> some SR-IOV VF parameters for security reasons. > >>> > >>> When the PF (host port) is driven by DPDK (OVS-DPDK case), > >>> we face two different cases: > >>> - driver is bifurcated (Mellanox case), > >>> so the VF can be configured via the kernel. > >>> - driver is on top of UIO or VFIO, so DPDK API is required, > >>> and PMD-specific APIs were used. > >> > >> So, what is wrong with setting VF mac via the representor port as > >> it done now? From our previous discussion, I thought that we concluded > >> that is enough to have current API, i.e. just call set_default_mac() > >> for a representor port and this will lead to setting mac for VF. > >> This is how it's implemented in Linux kernel and this is how it's > >> implemented in current DPDK drivers that supports setting mac for > >> the representor. > > > > I don't know what is done in the Linux kernel. > > In DPDK, setting the MAC of the representor is really setting > > the MAC of the representor. Is it crazy? > > Kind of. And no, it doesn't work this way in DPDK. > Just look at the i40e driver: > > 325 static int > 326 i40e_vf_representor_mac_addr_set(struct rte_eth_dev *ethdev, > 327 struct ether_addr *mac_addr) > 328 { > 329 struct i40e_vf_representor *representor = ethdev->data->dev_private; > 330 > 331 return rte_pmd_i40e_set_vf_mac_addr( > 332 representor->adapter->eth_dev->data->port_id, > 333 representor->vf_id, mac_addr); > 334 } > .... > 423 static const struct eth_dev_ops i40e_representor_dev_ops = { > <...> > 445 .mac_addr_set = i40e_vf_representor_mac_addr_set, > > > It really calls the function to set VF mac address. Indeed, I missed that i40e_vf_representor_mac_addr_set() is calling rte_pmd_i40e_set_vf_mac_addr(). > And for MLX drivers it's the same. > MLX driver call netlink to set representor MAC, but netlink is in > kernel and this call inside the kernel translates to the setting > of mac address of the VF. > > Am I missing something? I am not sure about this kernel translation. At least not in mlx5. Setting MAC address of a VF representor seems not supported on mlx5. But there is a specific netlink request to set a VF MAC address: ip link set vf mac > >> The only use case for this new API is to be able to control mac of > >> the representor itself, which doesn't make much sense. At least there > >> are only hypothetical use cases. And once again, Linux kernel doesn't > >> support this behavior. > > > > I think it is sane to be able to set different MAC addresses > > for the representor and the VF. Let me explain better my thoughts. In the switchdev design, VF and uplink ports are all connected together via a switch. The representors are mirrors of those switch ports. So a VF representor port is supposed to mirror the switch port where a VF is connected to. It is different of the VF itself. This is a drawing of my understanding of switchdev design: VF1 ------ VF1 rep /--------\ | switch | uplink rep ------ uplink ------ wire VF2 ------ VF2 rep \--------/ (PF) Of course, there can be more VF and uplink ports. With some switch-aware protocols, it might be interesting to have different MAC addresses on a VF and its representor. And more generally, I imagine that the config of a VF representor could be different of the config of a VF. > >>> This new generic API will avoid vendors fragmentation. > >> > >> I don't see the fragmentation. Right now you can set VF mac from DPDK > >> by calling set_default_mac() for representor. This API exists for all > >> vendors. Not implemented for Intel, but new API is not implemented too. > > > > No, the current situation is the following: > > - for mlx5, VF MAC can be configured with iproute2 or netlink > > - for ixgbe, rte_pmd_ixgbe_set_vf_mac_addr() > > - for i40e, rte_pmd_i40e_set_vf_mac_addr() > > - for bnxt, rte_pmd_bnxt_set_vf_mac_addr() Thanks to Ilya's comment, this is an update of the DPDK situation: - for mlx5, VF MAC can be configured with iproute2 or netlink and rte_eth_dev_default_mac_addr_set(rep) is not supported - for ixgbe, rte_pmd_ixgbe_set_vf_mac_addr() and rte_eth_dev_default_mac_addr_set(rep) does the same - for i40e, rte_pmd_i40e_set_vf_mac_addr() and rte_eth_dev_default_mac_addr_set(rep) does the same - for bnxt, rte_pmd_bnxt_set_vf_mac_addr() and no representor If we consider what Intel did, i.e. configure VF in place of representor for some operations, there are two drawbacks: - confusing that some ops apply to representor, others apply to VF - some ops are not possible on representor (because targetted to VF) I still feel that the addition of one single bit in the port ID is an elegant solution to target either the VF or its representor. > >> The this is that this new API will produce conceptual fragmentation > >> between DPDK and the Linux kernel, because to do the same thing you'll > >> have to use different ways. I mean, to change mac of VF in kernel you > >> need to set mac to the representor, but in DPDK changing setting mac to > >> representor will lead to changing the mac of the representor itself, > >> not the VF. This will be really confusing for users. > > > > I am not responsible of the choices in Linux. > > But I agree it would be interesting to check the reason of such decision. > > Rony, please could you explain? I looked at few Linux drivers: bnxt_vf_rep_netdev_ops has no op to set MAC bnxt_netdev_ops.ndo_set_vf_mac = set VF MAC from PF lio_vf_rep_ndev_ops has no op to set MAC lionetdevops.ndo_set_vf_mac = set VF MAC from PF mlx5e_netdev_ops_rep has no op to set MAC mlx5e_netdev_ops.ndo_set_vf_mac = set VF MAC from PF mlx5e_netdev_ops_uplink_rep.ndo_set_vf_mac = set VF MAC from PF nfp_repr_netdev_ops.ndo_set_mac_address = set representor MAC nfp_repr_netdev_ops.ndo_set_vf_mac = set VF MAC from representor nfp_net_netdev_ops.ndo_set_vf_mac = set VF MAC from PF There is a big chance that the behaviour is not standardized in Linux (as usual). So it is already confusing for users of Linux.