From mboxrd@z Thu Jan 1 00:00:00 1970
Return-Path: <hyonkim@cisco.com>
Received: from alln-iport-2.cisco.com (alln-iport-2.cisco.com [173.37.142.89])
by dpdk.org (Postfix) with ESMTP id 13F46324D
for <dev@dpdk.org>; Thu, 14 Mar 2019 03:58:08 +0100 (CET)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple;
d=cisco.com; i=@cisco.com; l=3770; q=dns/txt; s=iport;
t=1552532289; x=1553741889;
h=date:from:to:cc:subject:message-id:references:
mime-version:in-reply-to;
bh=8ncFlA/tRvtWLi2xnZT9d2xBrDsGFhwKXP0Sxdw0r+k=;
b=OmydZIy3Dlw6gYLTWcqy24GLu88Ulls33cU2aZ/85ATKhiAHsRyscRct
pylNOVCBZPSyd9IHY/8CsolWxKNK0DcA7I/LtLf6whEPoHEa1LijlG29s
iMQtZ9C1qCWlBAzZEyO4/KgdEjLE7XZQvWbBzBAFICOOqWnSamY4nevL6 U=;
X-IronPort-AV: E=Sophos;i="5.58,476,1544486400"; d="scan'208";a="245299313"
Received: from rcdn-core-5.cisco.com ([173.37.93.156])
by alln-iport-2.cisco.com with ESMTP/TLS/DHE-RSA-SEED-SHA;
14 Mar 2019 02:58:08 +0000
Received: from HYONKIM-7R0DR.cisco.com ([10.24.122.195])
by rcdn-core-5.cisco.com (8.15.2/8.15.2) with ESMTPS id x2E2w2eR006843
(version=TLSv1.2 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=NO);
Thu, 14 Mar 2019 02:58:05 GMT
Date: Thu, 14 Mar 2019 11:58:01 +0900
From: Hyong Youb Kim <hyonkim@cisco.com>
To: Thomas Monjalon <thomas@monjalon.net>
Cc: "John Daley (johndale)" <johndale@cisco.com>,
Ferruh Yigit <ferruh.yigit@intel.com>,
Andrew Rybchenko <arybchenko@solarflare.com>,
Qi Zhang <qi.z.zhang@intel.com>, "dev@dpdk.org" <dev@dpdk.org>,
Shahaf Shuler <shahafs@mellanox.com>,
Jerin Jacob <jerin.jacob@caviumnetworks.com>,
David Marchand <david.marchand@6wind.com>,
Maxime Coquelin <maxime.coquelin@redhat.com>,
Konstantin Ananyev <konstantin.ananyev@intel.com>,
Hemant Agrawal <hemant.agrawal@nxp.com>,
Stephen Hemminger <stephen@networkplumber.org>
Message-ID: <20190314025800.GA24620@HYONKIM-7R0DR.cisco.com>
References: <20190305055659.3095-1-hyonkim@cisco.com>
<155267265.tCld3OmuaL@xps>
<CY4PR1101MB221537E6A8EF8295F4BDBCDACF4A0@CY4PR1101MB2215.namprd11.prod.outlook.com>
<119440374.MxkLxCl2uq@xps>
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
In-Reply-To: <119440374.MxkLxCl2uq@xps>
User-Agent: Mutt/1.10.0 (2018-05-17)
X-Outbound-SMTP-Client: 10.24.122.195, [10.24.122.195]
X-Outbound-Node: rcdn-core-5.cisco.com
Subject: Re: [dpdk-dev] [PATCH v2] net/enic: add private API to set ingress
VLAN rewrite mode
X-BeenThere: dev@dpdk.org
X-Mailman-Version: 2.1.15
Precedence: list
List-Id: DPDK patches and discussions <dev.dpdk.org>
List-Unsubscribe: <https://mails.dpdk.org/options/dev>,
<mailto:dev-request@dpdk.org?subject=unsubscribe>
List-Archive: <http://mails.dpdk.org/archives/dev/>
List-Post: <mailto:dev@dpdk.org>
List-Help: <mailto:dev-request@dpdk.org?subject=help>
List-Subscribe: <https://mails.dpdk.org/listinfo/dev>,
<mailto:dev-request@dpdk.org?subject=subscribe>
X-List-Received-Date: Thu, 14 Mar 2019 02:58:09 -0000
On Wed, Mar 13, 2019 at 10:29:53PM +0100, Thomas Monjalon wrote:
> 13/03/2019 22:11, John Daley (johndale):
> > From: Thomas Monjalon <thomas@monjalon.net>
> > > 13/03/2019 19:32, Ferruh Yigit:
> > > > On 3/5/2019 7:11 AM, Hyong Youb Kim wrote:
> > > > > The driver currently has a devarg to set the rewrite mode during
> > > > > init. Some apps want to programatically set it after running
> > > > > rte_eal_init() and finding that ports are VIC. Add a private
> > > > > function to support such applications.
> > > >
> > > > It is not good idea to have PMD specific APIs (although we already have
> > > some).
> > > >
> > > > Specific to this case, as far as I can see it is to pass a config
> > > > value and do the action related to it, what would you think having a
> > > > generic key/value set/get API in ethdev for this? Similar to rawdev
> > > get_attr/set_attr [1]?
> > > >
> > > > My concern is it may turn into something like ioctl with many things
> > > > pushed to it, and cause possible duplication ...
> > >
> > > Yes, it is clearly ioctl style.
> > >
> > > Please could you explain more what is the rewrite mode?
> > > Does it apply to the port or the queue?
> > >
> > It applies to a port. By default the Cisco VIC VLAN tags every packet on ingress even if they were untagged coming in on the wire. They are tagged with VLAN 0 or a VLAN id programmed into the NIC depending on the configuration. Its part of the original design, to maintain priority bits, ancient history.
> >
> > Some apps don't like this (VPP) or take a slower path (OVS). Hyong added a ig-vlan-rewrite=untag devarg to disable this (leave untagged/default vlan packets untagged) during rte_eal_init and this is helpful for OVS, but VPP likes to set the rewrite mode after rte_eal_init() and finding the ports are VIC ports. So that is the reasoning behind the private API call.
>
> It looks like an application will always set this flag or never.
> So I don't see the need for an API function.
> Why VPP prefers set this flag later?
> Would it be better to have some driver-specific flags, no matter the ports?
>
>
As is, there seem to be two ways apps deal with NIC-specific tweaks/quirks.
1. Leave everything to the user.
Let the human user specify NIC-specific settings (e.g. devarg,
not-so-standard MTU/MRU, offloads with not-so-uniform behavior). The
app simply passes these to DPDK and does nothing else. Devargs are
passed to rte_eal_init. Other settings are applied during the
configure phase after rte_eal_init.
For example, OVS seems to go for a smallest common denominator that
works with most NICs out of the box. Otherwiese, it kinda falls into
this camp.
For a problematic NIC that needs user intervention, troubleshooting
goes like this :-)
- Install app.
- Run with settings that worked on a previous machine.
- Some features suddenly do not work.
- Google search this and that ("why this does not work on this server?").
- Contact vendors.
- Find out this NIC has unexpected behavior.
- Set devarg, tweak MTU/MRU, ... ("Oh need to set this for ..").
- Now the features work.
2. Hide ugly tweaks from the user.
VPP falls into this camp. The user specifies BDFs in the config (no
devargs). The app calls rte_eal_init(BDFs), iterates through the
discovered ports, applies whatever NIC-specific settings necessary
during the configure phase (i.e. do this for vendor A NIC, do that for
vendor B NIC), and then start the ports.
The ingress vlan rewrite mode is devarg now, so is not usable in this
model. One way around it is a private API. Driver specific flags
during the configure phase would also work fine. Though, enic might be
the only user of those flags.
Thanks.
-Hyong
From mboxrd@z Thu Jan 1 00:00:00 1970
Return-Path: <dev-bounces@dpdk.org>
Received: from dpdk.org (dpdk.org [92.243.14.124])
by dpdk.space (Postfix) with ESMTP id 630E2A0096
for <public@inbox.dpdk.org>; Thu, 14 Mar 2019 03:58:11 +0100 (CET)
Received: from [92.243.14.124] (localhost [127.0.0.1])
by dpdk.org (Postfix) with ESMTP id 542343572;
Thu, 14 Mar 2019 03:58:10 +0100 (CET)
Received: from alln-iport-2.cisco.com (alln-iport-2.cisco.com [173.37.142.89])
by dpdk.org (Postfix) with ESMTP id 13F46324D
for <dev@dpdk.org>; Thu, 14 Mar 2019 03:58:08 +0100 (CET)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple;
d=cisco.com; i=@cisco.com; l=3770; q=dns/txt; s=iport;
t=1552532289; x=1553741889;
h=date:from:to:cc:subject:message-id:references:
mime-version:in-reply-to;
bh=8ncFlA/tRvtWLi2xnZT9d2xBrDsGFhwKXP0Sxdw0r+k=;
b=OmydZIy3Dlw6gYLTWcqy24GLu88Ulls33cU2aZ/85ATKhiAHsRyscRct
pylNOVCBZPSyd9IHY/8CsolWxKNK0DcA7I/LtLf6whEPoHEa1LijlG29s
iMQtZ9C1qCWlBAzZEyO4/KgdEjLE7XZQvWbBzBAFICOOqWnSamY4nevL6 U=;
X-IronPort-AV: E=Sophos;i="5.58,476,1544486400"; d="scan'208";a="245299313"
Received: from rcdn-core-5.cisco.com ([173.37.93.156])
by alln-iport-2.cisco.com with ESMTP/TLS/DHE-RSA-SEED-SHA;
14 Mar 2019 02:58:08 +0000
Received: from HYONKIM-7R0DR.cisco.com ([10.24.122.195])
by rcdn-core-5.cisco.com (8.15.2/8.15.2) with ESMTPS id x2E2w2eR006843
(version=TLSv1.2 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=NO);
Thu, 14 Mar 2019 02:58:05 GMT
Date: Thu, 14 Mar 2019 11:58:01 +0900
From: Hyong Youb Kim <hyonkim@cisco.com>
To: Thomas Monjalon <thomas@monjalon.net>
Cc: "John Daley (johndale)" <johndale@cisco.com>,
Ferruh Yigit <ferruh.yigit@intel.com>,
Andrew Rybchenko <arybchenko@solarflare.com>,
Qi Zhang <qi.z.zhang@intel.com>, "dev@dpdk.org" <dev@dpdk.org>,
Shahaf Shuler <shahafs@mellanox.com>,
Jerin Jacob <jerin.jacob@caviumnetworks.com>,
David Marchand <david.marchand@6wind.com>,
Maxime Coquelin <maxime.coquelin@redhat.com>,
Konstantin Ananyev <konstantin.ananyev@intel.com>,
Hemant Agrawal <hemant.agrawal@nxp.com>,
Stephen Hemminger <stephen@networkplumber.org>
Message-ID: <20190314025800.GA24620@HYONKIM-7R0DR.cisco.com>
References: <20190305055659.3095-1-hyonkim@cisco.com>
<155267265.tCld3OmuaL@xps>
<CY4PR1101MB221537E6A8EF8295F4BDBCDACF4A0@CY4PR1101MB2215.namprd11.prod.outlook.com>
<119440374.MxkLxCl2uq@xps>
MIME-Version: 1.0
Content-Type: text/plain; charset="UTF-8"
Content-Disposition: inline
In-Reply-To: <119440374.MxkLxCl2uq@xps>
User-Agent: Mutt/1.10.0 (2018-05-17)
X-Outbound-SMTP-Client: 10.24.122.195, [10.24.122.195]
X-Outbound-Node: rcdn-core-5.cisco.com
Subject: Re: [dpdk-dev] [PATCH v2] net/enic: add private API to set ingress
VLAN rewrite mode
X-BeenThere: dev@dpdk.org
X-Mailman-Version: 2.1.15
Precedence: list
List-Id: DPDK patches and discussions <dev.dpdk.org>
List-Unsubscribe: <https://mails.dpdk.org/options/dev>,
<mailto:dev-request@dpdk.org?subject=unsubscribe>
List-Archive: <http://mails.dpdk.org/archives/dev/>
List-Post: <mailto:dev@dpdk.org>
List-Help: <mailto:dev-request@dpdk.org?subject=help>
List-Subscribe: <https://mails.dpdk.org/listinfo/dev>,
<mailto:dev-request@dpdk.org?subject=subscribe>
Errors-To: dev-bounces@dpdk.org
Sender: "dev" <dev-bounces@dpdk.org>
Message-ID: <20190314025801.95KAcz1mIwpgsNEZJRapPHB1vbN4vVubxkr4D4OSVkQ@z>
On Wed, Mar 13, 2019 at 10:29:53PM +0100, Thomas Monjalon wrote:
> 13/03/2019 22:11, John Daley (johndale):
> > From: Thomas Monjalon <thomas@monjalon.net>
> > > 13/03/2019 19:32, Ferruh Yigit:
> > > > On 3/5/2019 7:11 AM, Hyong Youb Kim wrote:
> > > > > The driver currently has a devarg to set the rewrite mode during
> > > > > init. Some apps want to programatically set it after running
> > > > > rte_eal_init() and finding that ports are VIC. Add a private
> > > > > function to support such applications.
> > > >
> > > > It is not good idea to have PMD specific APIs (although we already have
> > > some).
> > > >
> > > > Specific to this case, as far as I can see it is to pass a config
> > > > value and do the action related to it, what would you think having a
> > > > generic key/value set/get API in ethdev for this? Similar to rawdev
> > > get_attr/set_attr [1]?
> > > >
> > > > My concern is it may turn into something like ioctl with many things
> > > > pushed to it, and cause possible duplication ...
> > >
> > > Yes, it is clearly ioctl style.
> > >
> > > Please could you explain more what is the rewrite mode?
> > > Does it apply to the port or the queue?
> > >
> > It applies to a port. By default the Cisco VIC VLAN tags every packet on ingress even if they were untagged coming in on the wire. They are tagged with VLAN 0 or a VLAN id programmed into the NIC depending on the configuration. Its part of the original design, to maintain priority bits, ancient history.
> >
> > Some apps don't like this (VPP) or take a slower path (OVS). Hyong added a ig-vlan-rewrite=untag devarg to disable this (leave untagged/default vlan packets untagged) during rte_eal_init and this is helpful for OVS, but VPP likes to set the rewrite mode after rte_eal_init() and finding the ports are VIC ports. So that is the reasoning behind the private API call.
>
> It looks like an application will always set this flag or never.
> So I don't see the need for an API function.
> Why VPP prefers set this flag later?
> Would it be better to have some driver-specific flags, no matter the ports?
>
>
As is, there seem to be two ways apps deal with NIC-specific tweaks/quirks.
1. Leave everything to the user.
Let the human user specify NIC-specific settings (e.g. devarg,
not-so-standard MTU/MRU, offloads with not-so-uniform behavior). The
app simply passes these to DPDK and does nothing else. Devargs are
passed to rte_eal_init. Other settings are applied during the
configure phase after rte_eal_init.
For example, OVS seems to go for a smallest common denominator that
works with most NICs out of the box. Otherwiese, it kinda falls into
this camp.
For a problematic NIC that needs user intervention, troubleshooting
goes like this :-)
- Install app.
- Run with settings that worked on a previous machine.
- Some features suddenly do not work.
- Google search this and that ("why this does not work on this server?").
- Contact vendors.
- Find out this NIC has unexpected behavior.
- Set devarg, tweak MTU/MRU, ... ("Oh need to set this for ..").
- Now the features work.
2. Hide ugly tweaks from the user.
VPP falls into this camp. The user specifies BDFs in the config (no
devargs). The app calls rte_eal_init(BDFs), iterates through the
discovered ports, applies whatever NIC-specific settings necessary
during the configure phase (i.e. do this for vendor A NIC, do that for
vendor B NIC), and then start the ports.
The ingress vlan rewrite mode is devarg now, so is not usable in this
model. One way around it is a private API. Driver specific flags
during the configure phase would also work fine. Though, enic might be
the only user of those flags.
Thanks.
-Hyong