From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from out2-smtp.messagingengine.com (out2-smtp.messagingengine.com [66.111.4.26]) by dpdk.org (Postfix) with ESMTP id E90BD2C30 for ; Tue, 19 Mar 2019 18:37:04 +0100 (CET) Received: from compute1.internal (compute1.nyi.internal [10.202.2.41]) by mailout.nyi.internal (Postfix) with ESMTP id 6CBBB21B90; Tue, 19 Mar 2019 13:37:04 -0400 (EDT) Received: from mailfrontend2 ([10.202.2.163]) by compute1.internal (MEProxy); Tue, 19 Mar 2019 13:37:04 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=monjalon.net; h= from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding:content-type; s=mesmtp; bh=Gt2ppucIyT1ghxfNtdWteL3fBM8Fbifa8gkv7t4kqCs=; b=MnHkISd2tPPD 41kWn1yY8D86DVwbtJGUayYkSF7+UJAG+tH53rADci86fyj7iOIGgjjcvoW9za6j DzJbgBhO3jqTTpt8qzhbrTHZYwHb9r0nCl3p848YYEGUK96PMbTINg7+YDamlOcU gCtnpVPStzy8g5UYFzr2bHj0wTVpNyc= DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d= messagingengine.com; h=cc:content-transfer-encoding:content-type :date:from:in-reply-to:message-id:mime-version:references :subject:to:x-me-proxy:x-me-proxy:x-me-sender:x-me-sender :x-sasl-enc; s=fm2; bh=Gt2ppucIyT1ghxfNtdWteL3fBM8Fbifa8gkv7t4kq Cs=; b=S//8Z14Sypu4ojBvZZO5wp5yPWye6KgIMlNVxckYTDm2HoOSB4cJNTglq MKuoUCa1nSqcrZN+TRQmwstr5xWfMsYpsNblF6dGGQL3+5qa1Ed94gi6IU4VxhXy EjSpkzk8O1OrTGgiben8MsUKQJxriwXjx4NaAmTQnqgUxoy1B1EzukHCl67Bzh82 t/oeX69yYE7IOZUE3CjajaKMYOuf5F4b5BdYE8e8CbhNjU/4DMwwcmnyBHXaU/H4 co0c42j0UCc0WvNkbV8v4YN3/WFMwvqDkAoN2IKBhtRBPsnKF4A+a4lW5X/iFYKS MBCSOfh/1tpjn4xFScbgF03kSa+cA== X-ME-Sender: X-ME-Proxy-Cause: gggruggvucftvghtrhhoucdtuddrgedutddrieeggddutdehucetufdoteggodetrfdotf fvucfrrhhofhhilhgvmecuhfgrshhtofgrihhlpdfqfgfvpdfurfetoffkrfgpnffqhgen uceurghilhhouhhtmecufedttdenucesvcftvggtihhpihgvnhhtshculddquddttddmne cujfgurhephffvufffkfgjfhgggfgtsehtufertddttddvnecuhfhrohhmpefvhhhomhgr shcuofhonhhjrghlohhnuceothhhohhmrghssehmohhnjhgrlhhonhdrnhgvtheqnecukf hppeejjedrudefgedrvddtfedrudekgeenucfrrghrrghmpehmrghilhhfrhhomhepthhh ohhmrghssehmohhnjhgrlhhonhdrnhgvthenucevlhhushhtvghrufhiiigvpedt X-ME-Proxy: Received: from xps.localnet (184.203.134.77.rev.sfr.net [77.134.203.184]) by mail.messagingengine.com (Postfix) with ESMTPA id 2962C10288; Tue, 19 Mar 2019 13:37:01 -0400 (EDT) From: Thomas Monjalon To: Ferruh Yigit Cc: Hyong Youb Kim , "John Daley (johndale)" , Andrew Rybchenko , Qi Zhang , "dev@dpdk.org" , Shahaf Shuler , Jerin Jacob , David Marchand , Maxime Coquelin , Konstantin Ananyev , Hemant Agrawal , Stephen Hemminger , gaetan.rivet@6wind.com Date: Tue, 19 Mar 2019 18:36:58 +0100 Message-ID: <1715472.XOf6rlv5YZ@xps> In-Reply-To: <378fbf05-d8c5-8a25-a1e6-8e21e3083ae3@intel.com> References: <20190305055659.3095-1-hyonkim@cisco.com> <50100234.Q8zoU6jCWW@xps> <378fbf05-d8c5-8a25-a1e6-8e21e3083ae3@intel.com> MIME-Version: 1.0 Content-Transfer-Encoding: 7Bit Content-Type: text/plain; charset="us-ascii" Subject: Re: [dpdk-dev] [PATCH v2] net/enic: add private API to set ingress VLAN rewrite mode X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 19 Mar 2019 17:37:05 -0000 19/03/2019 18:29, Ferruh Yigit: > On 3/14/2019 10:04 PM, Thomas Monjalon wrote: > > 14/03/2019 03:58, Hyong Youb Kim: > >> On Wed, Mar 13, 2019 at 10:29:53PM +0100, Thomas Monjalon wrote: > >>> 13/03/2019 22:11, John Daley (johndale): > >>>> From: Thomas Monjalon > >>>>> 13/03/2019 19:32, Ferruh Yigit: > >>>>>> On 3/5/2019 7:11 AM, Hyong Youb Kim wrote: > >>>>>>> The driver currently has a devarg to set the rewrite mode during > >>>>>>> init. Some apps want to programatically set it after running > >>>>>>> rte_eal_init() and finding that ports are VIC. Add a private > >>>>>>> function to support such applications. > >>>>>> > >>>>>> It is not good idea to have PMD specific APIs (although we already have > >>>>> some). > >>>>>> > >>>>>> Specific to this case, as far as I can see it is to pass a config > >>>>>> value and do the action related to it, what would you think having a > >>>>>> generic key/value set/get API in ethdev for this? Similar to rawdev > >>>>> get_attr/set_attr [1]? > >>>>>> > >>>>>> My concern is it may turn into something like ioctl with many things > >>>>>> pushed to it, and cause possible duplication ... > >>>>> > >>>>> Yes, it is clearly ioctl style. > >>>>> > >>>>> Please could you explain more what is the rewrite mode? > >>>>> Does it apply to the port or the queue? > >>>>> > >>>> It applies to a port. By default the Cisco VIC VLAN tags every packet on ingress even if they were untagged coming in on the wire. They are tagged with VLAN 0 or a VLAN id programmed into the NIC depending on the configuration. Its part of the original design, to maintain priority bits, ancient history. > >>>> > >>>> Some apps don't like this (VPP) or take a slower path (OVS). Hyong added a ig-vlan-rewrite=untag devarg to disable this (leave untagged/default vlan packets untagged) during rte_eal_init and this is helpful for OVS, but VPP likes to set the rewrite mode after rte_eal_init() and finding the ports are VIC ports. So that is the reasoning behind the private API call. > >>> > >>> It looks like an application will always set this flag or never. > >>> So I don't see the need for an API function. > >>> Why VPP prefers set this flag later? > >>> Would it be better to have some driver-specific flags, no matter the ports? > >> > >> As is, there seem to be two ways apps deal with NIC-specific tweaks/quirks. > >> > >> 1. Leave everything to the user. > >> > >> Let the human user specify NIC-specific settings (e.g. devarg, > >> not-so-standard MTU/MRU, offloads with not-so-uniform behavior). The > >> app simply passes these to DPDK and does nothing else. Devargs are > >> passed to rte_eal_init. Other settings are applied during the > >> configure phase after rte_eal_init. > >> > >> For example, OVS seems to go for a smallest common denominator that > >> works with most NICs out of the box. Otherwiese, it kinda falls into > >> this camp. > >> > >> For a problematic NIC that needs user intervention, troubleshooting > >> goes like this :-) > >> - Install app. > >> - Run with settings that worked on a previous machine. > >> - Some features suddenly do not work. > >> - Google search this and that ("why this does not work on this server?"). > >> - Contact vendors. > >> - Find out this NIC has unexpected behavior. > >> - Set devarg, tweak MTU/MRU, ... ("Oh need to set this for .."). > >> - Now the features work. > >> > >> 2. Hide ugly tweaks from the user. > >> > >> VPP falls into this camp. The user specifies BDFs in the config (no > >> devargs). The app calls rte_eal_init(BDFs), iterates through the > >> discovered ports, applies whatever NIC-specific settings necessary > >> during the configure phase (i.e. do this for vendor A NIC, do that for > >> vendor B NIC), and then start the ports. > >> > >> The ingress vlan rewrite mode is devarg now, so is not usable in this > >> model. One way around it is a private API. Driver specific flags > >> during the configure phase would also work fine. Though, enic might be > >> the only user of those flags. > > > > I think DPDK needs some driver configuration. > > Currently the config is done per device with devargs. > > The next devargs format allow this: > > driver=enic,rewrite=on > > and it can be passed to rte_eal_init(). > > > > We did not progress on the implementation of this format in recent months, > > but you are welcome to help! > > Instead of passing devargs in the whitelist/blacklist options, > > we should introduce a new option, like --dev. > > But it will be still devarg in new implementation. With the new syntax, no need to specify a device. We can match a driver or multiple drivers sharing the same property. > I guess for this use case, there is a need to pass this information from an API. > Options can be: > 1- PMD specific API > 2- Extend ethdev dev_ops for each usecase > 3- Have a generic API, as suggested above > 4- Extend configure to accept flags > > I don't see a winner in above list, each has some problems. Any comment on how > to proceed? I don't see a problem with the devargs approach. From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from dpdk.org (dpdk.org [92.243.14.124]) by dpdk.space (Postfix) with ESMTP id 57EBFA00E6 for ; Tue, 19 Mar 2019 18:37:06 +0100 (CET) Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id 31CFB2C54; Tue, 19 Mar 2019 18:37:06 +0100 (CET) Received: from out2-smtp.messagingengine.com (out2-smtp.messagingengine.com [66.111.4.26]) by dpdk.org (Postfix) with ESMTP id E90BD2C30 for ; Tue, 19 Mar 2019 18:37:04 +0100 (CET) Received: from compute1.internal (compute1.nyi.internal [10.202.2.41]) by mailout.nyi.internal (Postfix) with ESMTP id 6CBBB21B90; Tue, 19 Mar 2019 13:37:04 -0400 (EDT) Received: from mailfrontend2 ([10.202.2.163]) by compute1.internal (MEProxy); Tue, 19 Mar 2019 13:37:04 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=monjalon.net; h= from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding:content-type; s=mesmtp; bh=Gt2ppucIyT1ghxfNtdWteL3fBM8Fbifa8gkv7t4kqCs=; b=MnHkISd2tPPD 41kWn1yY8D86DVwbtJGUayYkSF7+UJAG+tH53rADci86fyj7iOIGgjjcvoW9za6j DzJbgBhO3jqTTpt8qzhbrTHZYwHb9r0nCl3p848YYEGUK96PMbTINg7+YDamlOcU gCtnpVPStzy8g5UYFzr2bHj0wTVpNyc= DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d= messagingengine.com; h=cc:content-transfer-encoding:content-type :date:from:in-reply-to:message-id:mime-version:references :subject:to:x-me-proxy:x-me-proxy:x-me-sender:x-me-sender :x-sasl-enc; s=fm2; bh=Gt2ppucIyT1ghxfNtdWteL3fBM8Fbifa8gkv7t4kq Cs=; b=S//8Z14Sypu4ojBvZZO5wp5yPWye6KgIMlNVxckYTDm2HoOSB4cJNTglq MKuoUCa1nSqcrZN+TRQmwstr5xWfMsYpsNblF6dGGQL3+5qa1Ed94gi6IU4VxhXy EjSpkzk8O1OrTGgiben8MsUKQJxriwXjx4NaAmTQnqgUxoy1B1EzukHCl67Bzh82 t/oeX69yYE7IOZUE3CjajaKMYOuf5F4b5BdYE8e8CbhNjU/4DMwwcmnyBHXaU/H4 co0c42j0UCc0WvNkbV8v4YN3/WFMwvqDkAoN2IKBhtRBPsnKF4A+a4lW5X/iFYKS MBCSOfh/1tpjn4xFScbgF03kSa+cA== X-ME-Sender: X-ME-Proxy-Cause: gggruggvucftvghtrhhoucdtuddrgedutddrieeggddutdehucetufdoteggodetrfdotf fvucfrrhhofhhilhgvmecuhfgrshhtofgrihhlpdfqfgfvpdfurfetoffkrfgpnffqhgen uceurghilhhouhhtmecufedttdenucesvcftvggtihhpihgvnhhtshculddquddttddmne cujfgurhephffvufffkfgjfhgggfgtsehtufertddttddvnecuhfhrohhmpefvhhhomhgr shcuofhonhhjrghlohhnuceothhhohhmrghssehmohhnjhgrlhhonhdrnhgvtheqnecukf hppeejjedrudefgedrvddtfedrudekgeenucfrrghrrghmpehmrghilhhfrhhomhepthhh ohhmrghssehmohhnjhgrlhhonhdrnhgvthenucevlhhushhtvghrufhiiigvpedt X-ME-Proxy: Received: from xps.localnet (184.203.134.77.rev.sfr.net [77.134.203.184]) by mail.messagingengine.com (Postfix) with ESMTPA id 2962C10288; Tue, 19 Mar 2019 13:37:01 -0400 (EDT) From: Thomas Monjalon To: Ferruh Yigit Cc: Hyong Youb Kim , "John Daley (johndale)" , Andrew Rybchenko , Qi Zhang , "dev@dpdk.org" , Shahaf Shuler , Jerin Jacob , David Marchand , Maxime Coquelin , Konstantin Ananyev , Hemant Agrawal , Stephen Hemminger , gaetan.rivet@6wind.com Date: Tue, 19 Mar 2019 18:36:58 +0100 Message-ID: <1715472.XOf6rlv5YZ@xps> In-Reply-To: <378fbf05-d8c5-8a25-a1e6-8e21e3083ae3@intel.com> References: <20190305055659.3095-1-hyonkim@cisco.com> <50100234.Q8zoU6jCWW@xps> <378fbf05-d8c5-8a25-a1e6-8e21e3083ae3@intel.com> MIME-Version: 1.0 Content-Transfer-Encoding: 7Bit Content-Type: text/plain; charset="UTF-8" Subject: Re: [dpdk-dev] [PATCH v2] net/enic: add private API to set ingress VLAN rewrite mode X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" Message-ID: <20190319173658.pr8LjWzcNzTXY9JIXphjLWzzR_tLZDgXF1MkWlFAQuU@z> 19/03/2019 18:29, Ferruh Yigit: > On 3/14/2019 10:04 PM, Thomas Monjalon wrote: > > 14/03/2019 03:58, Hyong Youb Kim: > >> On Wed, Mar 13, 2019 at 10:29:53PM +0100, Thomas Monjalon wrote: > >>> 13/03/2019 22:11, John Daley (johndale): > >>>> From: Thomas Monjalon > >>>>> 13/03/2019 19:32, Ferruh Yigit: > >>>>>> On 3/5/2019 7:11 AM, Hyong Youb Kim wrote: > >>>>>>> The driver currently has a devarg to set the rewrite mode during > >>>>>>> init. Some apps want to programatically set it after running > >>>>>>> rte_eal_init() and finding that ports are VIC. Add a private > >>>>>>> function to support such applications. > >>>>>> > >>>>>> It is not good idea to have PMD specific APIs (although we already have > >>>>> some). > >>>>>> > >>>>>> Specific to this case, as far as I can see it is to pass a config > >>>>>> value and do the action related to it, what would you think having a > >>>>>> generic key/value set/get API in ethdev for this? Similar to rawdev > >>>>> get_attr/set_attr [1]? > >>>>>> > >>>>>> My concern is it may turn into something like ioctl with many things > >>>>>> pushed to it, and cause possible duplication ... > >>>>> > >>>>> Yes, it is clearly ioctl style. > >>>>> > >>>>> Please could you explain more what is the rewrite mode? > >>>>> Does it apply to the port or the queue? > >>>>> > >>>> It applies to a port. By default the Cisco VIC VLAN tags every packet on ingress even if they were untagged coming in on the wire. They are tagged with VLAN 0 or a VLAN id programmed into the NIC depending on the configuration. Its part of the original design, to maintain priority bits, ancient history. > >>>> > >>>> Some apps don't like this (VPP) or take a slower path (OVS). Hyong added a ig-vlan-rewrite=untag devarg to disable this (leave untagged/default vlan packets untagged) during rte_eal_init and this is helpful for OVS, but VPP likes to set the rewrite mode after rte_eal_init() and finding the ports are VIC ports. So that is the reasoning behind the private API call. > >>> > >>> It looks like an application will always set this flag or never. > >>> So I don't see the need for an API function. > >>> Why VPP prefers set this flag later? > >>> Would it be better to have some driver-specific flags, no matter the ports? > >> > >> As is, there seem to be two ways apps deal with NIC-specific tweaks/quirks. > >> > >> 1. Leave everything to the user. > >> > >> Let the human user specify NIC-specific settings (e.g. devarg, > >> not-so-standard MTU/MRU, offloads with not-so-uniform behavior). The > >> app simply passes these to DPDK and does nothing else. Devargs are > >> passed to rte_eal_init. Other settings are applied during the > >> configure phase after rte_eal_init. > >> > >> For example, OVS seems to go for a smallest common denominator that > >> works with most NICs out of the box. Otherwiese, it kinda falls into > >> this camp. > >> > >> For a problematic NIC that needs user intervention, troubleshooting > >> goes like this :-) > >> - Install app. > >> - Run with settings that worked on a previous machine. > >> - Some features suddenly do not work. > >> - Google search this and that ("why this does not work on this server?"). > >> - Contact vendors. > >> - Find out this NIC has unexpected behavior. > >> - Set devarg, tweak MTU/MRU, ... ("Oh need to set this for .."). > >> - Now the features work. > >> > >> 2. Hide ugly tweaks from the user. > >> > >> VPP falls into this camp. The user specifies BDFs in the config (no > >> devargs). The app calls rte_eal_init(BDFs), iterates through the > >> discovered ports, applies whatever NIC-specific settings necessary > >> during the configure phase (i.e. do this for vendor A NIC, do that for > >> vendor B NIC), and then start the ports. > >> > >> The ingress vlan rewrite mode is devarg now, so is not usable in this > >> model. One way around it is a private API. Driver specific flags > >> during the configure phase would also work fine. Though, enic might be > >> the only user of those flags. > > > > I think DPDK needs some driver configuration. > > Currently the config is done per device with devargs. > > The next devargs format allow this: > > driver=enic,rewrite=on > > and it can be passed to rte_eal_init(). > > > > We did not progress on the implementation of this format in recent months, > > but you are welcome to help! > > Instead of passing devargs in the whitelist/blacklist options, > > we should introduce a new option, like --dev. > > But it will be still devarg in new implementation. With the new syntax, no need to specify a device. We can match a driver or multiple drivers sharing the same property. > I guess for this use case, there is a need to pass this information from an API. > Options can be: > 1- PMD specific API > 2- Extend ethdev dev_ops for each usecase > 3- Have a generic API, as suggested above > 4- Extend configure to accept flags > > I don't see a winner in above list, each has some problems. Any comment on how > to proceed? I don't see a problem with the devargs approach.