From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from dpdk.org (dpdk.org [92.243.14.124]) by dpdk.space (Postfix) with ESMTP id 5E2F2A00E6 for ; Wed, 20 Mar 2019 12:47:02 +0100 (CET) Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id 7A90C5699; Wed, 20 Mar 2019 12:47:01 +0100 (CET) Received: from out5-smtp.messagingengine.com (out5-smtp.messagingengine.com [66.111.4.29]) by dpdk.org (Postfix) with ESMTP id 77681568A for ; Wed, 20 Mar 2019 12:47:00 +0100 (CET) Received: from compute1.internal (compute1.nyi.internal [10.202.2.41]) by mailout.nyi.internal (Postfix) with ESMTP id 12FD322200; Wed, 20 Mar 2019 07:47:00 -0400 (EDT) Received: from mailfrontend1 ([10.202.2.162]) by compute1.internal (MEProxy); Wed, 20 Mar 2019 07:47:00 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=monjalon.net; h= from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding:content-type; s=mesmtp; bh=gFHDQUwbn0QzuMPvJK1IUAvCqIrvqUSpMdWh+fSbdCs=; b=r9LMsq4ArCvc MgVjUBZajjDkZVSRq3Z9vV+x0+YwlRwtvPKIDAZwRnm6KcIkNEQyGTGUQm5YnUKw 08EUnlXlD6jZsPa6Asg4Vfs78DuVLq6LPM+CrzCZril0UNkzJDeUkcUoUTkRx1ZK ohWuA6q62koM0KQArRU7r+J5S7sWsLY= DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d= messagingengine.com; h=cc:content-transfer-encoding:content-type :date:from:in-reply-to:message-id:mime-version:references :subject:to:x-me-proxy:x-me-proxy:x-me-sender:x-me-sender :x-sasl-enc; s=fm2; bh=gFHDQUwbn0QzuMPvJK1IUAvCqIrvqUSpMdWh+fSbd Cs=; b=1aLQtuIX7FtFjYFKAFyuwNjMauCol2p0YHERpmwXOFuPjAu5z3i//QpnO Ts5k6GWAC48ZEJGZ9Y8QNr2+NMdTUxD9iGeLoSe9bzWj4FmgkBtBui+OyA5JHIhj 4sBTfMtSZcAiK9zK3XSlCBbVASwcWh2hccA6hWxYMRVHUaUuZIKQGBBCltu5ijO3 v1UYt0f46PK/26+ohsmOejB8zQe3Nu6RxmqaoCi484YL9nsjEtBzm4gpC+U41kKm QCinDkC/VmG0tLB8ryi9yDZ24P1QDaZiTLcUwEhRlTuwNKp4wHj9tnZfwNNYp5+p mGrt5lJ5i5M0NnUSZOzzYId40pRRw== X-ME-Sender: X-ME-Proxy-Cause: gggruggvucftvghtrhhoucdtuddrgedutddrieeigdeffecutefuodetggdotefrodftvf curfhrohhfihhlvgemucfhrghsthforghilhdpqfgfvfdpuffrtefokffrpgfnqfghnecu uegrihhlohhuthemuceftddtnecusecvtfgvtghiphhivghnthhsucdlqddutddtmdenuc fjughrpefhvffufffkjghfggfgtgesthfuredttddtvdenucfhrhhomhepvfhhohhmrghs ucfoohhnjhgrlhhonhcuoehthhhomhgrshesmhhonhhjrghlohhnrdhnvghtqeenucfkph epjeejrddufeegrddvtdefrddukeegnecurfgrrhgrmhepmhgrihhlfhhrohhmpehthhho mhgrshesmhhonhhjrghlohhnrdhnvghtnecuvehluhhsthgvrhfuihiivgeptd X-ME-Proxy: Received: from xps.localnet (184.203.134.77.rev.sfr.net [77.134.203.184]) by mail.messagingengine.com (Postfix) with ESMTPA id E4DBEE4176; Wed, 20 Mar 2019 07:46:56 -0400 (EDT) From: Thomas Monjalon To: Ferruh Yigit Cc: Hyong Youb Kim , "John Daley (johndale)" , Andrew Rybchenko , Qi Zhang , "dev@dpdk.org" , Shahaf Shuler , Jerin Jacob , David Marchand , Maxime Coquelin , Konstantin Ananyev , Hemant Agrawal , Stephen Hemminger , gaetan.rivet@6wind.com Date: Wed, 20 Mar 2019 12:46:55 +0100 Message-ID: <3306757.Lg3ZuM36Da@xps> In-Reply-To: <09e2672e-c94f-5f7d-84e3-343a7be7452e@intel.com> References: <20190305055659.3095-1-hyonkim@cisco.com> <3429576.gmczXWLcQl@xps> <09e2672e-c94f-5f7d-84e3-343a7be7452e@intel.com> MIME-Version: 1.0 Content-Transfer-Encoding: 7Bit Content-Type: text/plain; charset="UTF-8" Subject: Re: [dpdk-dev] [PATCH v2] net/enic: add private API to set ingress VLAN rewrite mode X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" Message-ID: <20190320114655.Ss2Ib3UMKALf1K_nwVLZTXAKAwDfZfqb_y4KAJMGdmA@z> 20/03/2019 11:45, Ferruh Yigit: > On 3/19/2019 8:30 PM, Thomas Monjalon wrote: > > 19/03/2019 19:00, Ferruh Yigit: > >> On 3/19/2019 5:36 PM, Thomas Monjalon wrote: > >>> 19/03/2019 18:29, Ferruh Yigit: > >>>> On 3/14/2019 10:04 PM, Thomas Monjalon wrote: > >>>>> 14/03/2019 03:58, Hyong Youb Kim: > >>>>>> On Wed, Mar 13, 2019 at 10:29:53PM +0100, Thomas Monjalon wrote: > >>>>>>> 13/03/2019 22:11, John Daley (johndale): > >>>>>>>> From: Thomas Monjalon > >>>>>>>>> 13/03/2019 19:32, Ferruh Yigit: > >>>>>>>>>> On 3/5/2019 7:11 AM, Hyong Youb Kim wrote: > >>>>>>>>>>> The driver currently has a devarg to set the rewrite mode during > >>>>>>>>>>> init. Some apps want to programatically set it after running > >>>>>>>>>>> rte_eal_init() and finding that ports are VIC. Add a private > >>>>>>>>>>> function to support such applications. > >>>>>>>>>> > >>>>>>>>>> It is not good idea to have PMD specific APIs (although we already have > >>>>>>>>> some). > >>>>>>>>>> > >>>>>>>>>> Specific to this case, as far as I can see it is to pass a config > >>>>>>>>>> value and do the action related to it, what would you think having a > >>>>>>>>>> generic key/value set/get API in ethdev for this? Similar to rawdev > >>>>>>>>> get_attr/set_attr [1]? > >>>>>>>>>> > >>>>>>>>>> My concern is it may turn into something like ioctl with many things > >>>>>>>>>> pushed to it, and cause possible duplication ... > >>>>>>>>> > >>>>>>>>> Yes, it is clearly ioctl style. > >>>>>>>>> > >>>>>>>>> Please could you explain more what is the rewrite mode? > >>>>>>>>> Does it apply to the port or the queue? > >>>>>>>>> > >>>>>>>> It applies to a port. By default the Cisco VIC VLAN tags every packet on ingress even if they were untagged coming in on the wire. They are tagged with VLAN 0 or a VLAN id programmed into the NIC depending on the configuration. Its part of the original design, to maintain priority bits, ancient history. > >>>>>>>> > >>>>>>>> Some apps don't like this (VPP) or take a slower path (OVS). Hyong added a ig-vlan-rewrite=untag devarg to disable this (leave untagged/default vlan packets untagged) during rte_eal_init and this is helpful for OVS, but VPP likes to set the rewrite mode after rte_eal_init() and finding the ports are VIC ports. So that is the reasoning behind the private API call. > >>>>>>> > >>>>>>> It looks like an application will always set this flag or never. > >>>>>>> So I don't see the need for an API function. > >>>>>>> Why VPP prefers set this flag later? > >>>>>>> Would it be better to have some driver-specific flags, no matter the ports? > >>>>>> > >>>>>> As is, there seem to be two ways apps deal with NIC-specific tweaks/quirks. > >>>>>> > >>>>>> 1. Leave everything to the user. > >>>>>> > >>>>>> Let the human user specify NIC-specific settings (e.g. devarg, > >>>>>> not-so-standard MTU/MRU, offloads with not-so-uniform behavior). The > >>>>>> app simply passes these to DPDK and does nothing else. Devargs are > >>>>>> passed to rte_eal_init. Other settings are applied during the > >>>>>> configure phase after rte_eal_init. > >>>>>> > >>>>>> For example, OVS seems to go for a smallest common denominator that > >>>>>> works with most NICs out of the box. Otherwiese, it kinda falls into > >>>>>> this camp. > >>>>>> > >>>>>> For a problematic NIC that needs user intervention, troubleshooting > >>>>>> goes like this :-) > >>>>>> - Install app. > >>>>>> - Run with settings that worked on a previous machine. > >>>>>> - Some features suddenly do not work. > >>>>>> - Google search this and that ("why this does not work on this server?"). > >>>>>> - Contact vendors. > >>>>>> - Find out this NIC has unexpected behavior. > >>>>>> - Set devarg, tweak MTU/MRU, ... ("Oh need to set this for .."). > >>>>>> - Now the features work. > >>>>>> > >>>>>> 2. Hide ugly tweaks from the user. > >>>>>> > >>>>>> VPP falls into this camp. The user specifies BDFs in the config (no > >>>>>> devargs). The app calls rte_eal_init(BDFs), iterates through the > >>>>>> discovered ports, applies whatever NIC-specific settings necessary > >>>>>> during the configure phase (i.e. do this for vendor A NIC, do that for > >>>>>> vendor B NIC), and then start the ports. > >>>>>> > >>>>>> The ingress vlan rewrite mode is devarg now, so is not usable in this > >>>>>> model. One way around it is a private API. Driver specific flags > >>>>>> during the configure phase would also work fine. Though, enic might be > >>>>>> the only user of those flags. > >>>>> > >>>>> I think DPDK needs some driver configuration. > >>>>> Currently the config is done per device with devargs. > >>>>> The next devargs format allow this: > >>>>> driver=enic,rewrite=on > >>>>> and it can be passed to rte_eal_init(). > >>>>> > >>>>> We did not progress on the implementation of this format in recent months, > >>>>> but you are welcome to help! > >>>>> Instead of passing devargs in the whitelist/blacklist options, > >>>>> we should introduce a new option, like --dev. > >>>> > >>>> But it will be still devarg in new implementation. > >>> > >>> With the new syntax, no need to specify a device. > >>> We can match a driver or multiple drivers sharing the same property. > >>> > >>>> I guess for this use case, there is a need to pass this information from an API. > >>>> Options can be: > >>>> 1- PMD specific API > >>>> 2- Extend ethdev dev_ops for each usecase > >>>> 3- Have a generic API, as suggested above > >>>> 4- Extend configure to accept flags > >>>> > >>>> I don't see a winner in above list, each has some problems. Any comment on how > >>>> to proceed? > >>> > >>> I don't see a problem with the devargs approach. > >> > >> Devargs either passed via command line to DPDK application, or parameter to > >> hotplug APIs. > > > > The application can pass whatever it wants to EAL. > > This means changing current device probe logic completely, right. > Instead of probing everything on start, probe nothing and application add > devices via eal (hotplug) API calls with providing devargs. > I have no issue with this picture, only it doesn't look soon. No, I mean probe everything at startup automatically as usual. Just need to pass an option to the driver during its initialization. > > In the case described above, the application wants to enable > > a mode of the driver for all its devices. > > That's why I think the right solution is a driver option, > > which can be achieved with the new devargs syntax. > > > >> If someone wants to use regular probe without any command line argument, and > >> later configure the device via an API, can devargs be used? > > > > This is a scenario different of what is asked above. > > In the case of a specific configuration of one device, > > we have three choices. > > These are your suggestions, with my comments: > > 1- PMD specific API > > 2- Extend ethdev dev_ops for each usecase > > (3- Have a generic API) = choice 2 > > (4- Extend configure to accept flags) = choice 1 > > This is a choice 3: > > - no support of exotic features > > Not sure if this is a real option for a vendor, HWs has exotic features and they > want to enable it, I believe SW should not be the blocker for this. > > Also I definitely agree that exotic features should not break main/common usage, > make it hard to use or make it confusing/complex etc. > > Until we have a better solution I guess we need to continue with private APIs. I think the driver option would work, but it seems I fail to correctly explain it :)