[dpdk-dev] KNI broken again with 4.9 kernel

[dpdk-dev] KNI broken again with 4.9 kernel

[Stephen Hemminger]

/build/lib/librte_eal/linuxapp/kni/igb_main.c:2317:21: error: initialization from incompatible pointer type [-Werror=incompatible-pointer-types]
  .ndo_set_vf_vlan = igb_ndo_set_vf_vlan,
                     ^~~~~~~~~~~~~~~~~~~

I am sure Ferruh Yigit will fix it.

Which raises a couple of questions:
 1. Why is DPDK still keeping KNI support for Intel specific ethtool functionality.
    This always breaks, is code bloat, and means a 3rd copy of base code (Linux, DPDK PMD, + KNI)

 2. Why is KNI not upstream?
    If not acceptable due to security or supportablity then why does it still exist?

 3. If not upstream, then maintainer should track upstream kernel changes and fix DPDK before
    kernel is released.  The ABI is normally set early in the rc cycle weeks before release.

Re: [dpdk-dev] KNI broken again with 4.9 kernel

[Ferruh Yigit]

Hi Stephen,

I can't reproduce the error, I think it is not broken, that issue should
be already fixed with commit [1].



On 12/14/2016 11:40 PM, Stephen Hemminger wrote:
> /build/lib/librte_eal/linuxapp/kni/igb_main.c:2317:21: error: initialization from incompatible pointer type [-Werror=incompatible-pointer-types]
>   .ndo_set_vf_vlan = igb_ndo_set_vf_vlan,
>                      ^~~~~~~~~~~~~~~~~~~

Thanks for the report, not everyone is working with latest kernel, so
these reports are useful.

> 
> I am sure Ferruh Yigit will fix it.

I think I already did J, please check the commit [1], it is merged in
v16.11, can you please double check the DPDK version.


[1]
 commit 6445198f802d993c73f4b246353b2ceb2dfafc32
 Refs: v16.11-rc1-2-g6445198
 Author:     Ferruh Yigit <ferruh.yigit@intel.com>
 AuthorDate: Mon Oct 17 11:23:14 2016 +0100
 Commit:     Thomas Monjalon <thomas.monjalon@6wind.com>
 CommitDate: Tue Oct 25 16:20:43 2016 +0200

     kni: fix build with kernel 4.9

     compile error:
       CC [M]  .../lib/librte_eal/linuxapp/kni/igb_main.o
     .../lib/librte_eal/linuxapp/kni/igb_main.c:2317:21:
     error: initialization from incompatible pointer type
             [-Werror=incompatible-pointer-types]
       .ndo_set_vf_vlan = igb_ndo_set_vf_vlan,
                          ^~~~~~~~~~~~~~~~~~~

     Linux kernel 4.9 updates API for ndo_set_vf_vlan:
     Linux: 79aab093a0b5 ("net: Update API for VF vlan protocol 802.1ad
support")

     Use new API for Linux kernels >= 4.9

     Signed-off-by: Ferruh Yigit <ferruh.yigit@intel.com>
     Tested-by: Pablo de Lara <pablo.de.lara.guarch@intel.com>


<...>
I will reply questions with a different subject...

Re: [dpdk-dev] KNI Questions

[Ferruh Yigit]

Hi Stephen,

<...>

> 
> Which raises a couple of questions:
>  1. Why is DPDK still keeping KNI support for Intel specific ethtool functionality.
>     This always breaks, is code bloat, and means a 3rd copy of base code (Linux, DPDK PMD, + KNI)

I agree on you comments related to the ethtool functionality,
but right now that is a functionality that people may be using, I think
we should not remove it without providing an alternative to it.

> 
>  2. Why is KNI not upstream?
>     If not acceptable due to security or supportablity then why does it still exist?

I believe you are one of the most knowledgeable person in the mail list
on upstreaming, any support is welcome.

> 
>  3. If not upstream, then maintainer should track upstream kernel changes and fix DPDK before
>     kernel is released.  The ABI is normally set early in the rc cycle weeks before release.

I am trying to track as much as possible, any help appreciated.

> 

Re: [dpdk-dev] KNI Questions

[Stephen Hemminger]

On Thu, 15 Dec 2016 11:53:59 +0000
Ferruh Yigit <ferruh.yigit@intel.com> wrote:

> Hi Stephen,
> 
> <...>
> 
> > 
> > Which raises a couple of questions:
> >  1. Why is DPDK still keeping KNI support for Intel specific ethtool functionality.
> >     This always breaks, is code bloat, and means a 3rd copy of base code (Linux, DPDK PMD, + KNI)  
> 
> I agree on you comments related to the ethtool functionality,
> but right now that is a functionality that people may be using, I think
> we should not remove it without providing an alternative to it.
> 
> > 
> >  2. Why is KNI not upstream?
> >     If not acceptable due to security or supportablity then why does it still exist?  
> 
> I believe you are one of the most knowledgeable person in the mail list
> on upstreaming, any support is welcome.

It should be upstreamable but I doubt it would make it past the maintainer.
Mostly because it supports DPDK which he is not in favor of but also since
it is a specialized interface only usable by DPDK, ie. not a general infrastructure.

Re: [dpdk-dev] KNI Questions

[John Fastabend]

On 16-12-15 09:16 AM, Stephen Hemminger wrote:
> On Thu, 15 Dec 2016 11:53:59 +0000
> Ferruh Yigit <ferruh.yigit@intel.com> wrote:
> 
>> Hi Stephen,
>>
>> <...>
>>
>>>
>>> Which raises a couple of questions:
>>>  1. Why is DPDK still keeping KNI support for Intel specific ethtool functionality.
>>>     This always breaks, is code bloat, and means a 3rd copy of base code (Linux, DPDK PMD, + KNI)  
>>
>> I agree on you comments related to the ethtool functionality,
>> but right now that is a functionality that people may be using, I think
>> we should not remove it without providing an alternative to it.
>>
>>>
>>>  2. Why is KNI not upstream?
>>>     If not acceptable due to security or supportablity then why does it still exist?  
>>
>> I believe you are one of the most knowledgeable person in the mail list
>> on upstreaming, any support is welcome.
> 
> It should be upstreamable but I doubt it would make it past the maintainer.
> Mostly because it supports DPDK which he is not in favor of but also since
> it is a specialized interface only usable by DPDK, ie. not a general infrastructure.
> 

I was looking to see if we could get more or less the same interface put
in either af_packet or vhost directly. It would work nicely with the XDP
patches where we want to forward a packet into userspace without having
to build skb, etc. So it wouldn't be _just_ a DPDK interface at that
point.

I was hoping to look into it in Jan/Feb I need to wrap a few other
things up first.

.John

Re: [dpdk-dev] KNI broken again with 4.9 kernel

[Mcnamara, John]

> -----Original Message-----
> From: dev [mailto:dev-bounces@dpdk.org] On Behalf Of Stephen Hemminger
> Sent: Wednesday, December 14, 2016 11:41 PM
> To: dev@dpdk.org
> Subject: [dpdk-dev] KNI broken again with 4.9 kernel
> 
> /build/lib/librte_eal/linuxapp/kni/igb_main.c:2317:21: error:
> initialization from incompatible pointer type [-Werror=incompatible-
> pointer-types]
>   .ndo_set_vf_vlan = igb_ndo_set_vf_vlan,
>                      ^~~~~~~~~~~~~~~~~~~
> 
> I am sure Ferruh Yigit will fix it.
> 
> Which raises a couple of questions:
>  1. Why is DPDK still keeping KNI support for Intel specific ethtool
> functionality.
>     This always breaks, is code bloat, and means a 3rd copy of base code
> (Linux, DPDK PMD, + KNI)
> 
>  2. Why is KNI not upstream?
>     If not acceptable due to security or supportablity then why does it
> still exist?
> 
>  3. If not upstream, then maintainer should track upstream kernel changes
> and fix DPDK before
>     kernel is released.  The ABI is normally set early in the rc cycle
> weeks before release.


Hi Stephen,

On point 2: The feedback we have always received is that the KNI code isn't upstreamable. Do you think there is an upstream path? 

> If not acceptable due to security or supportablity then why does it
> still exist?

The most commonly expressed reason when we have asked this question in the past (and we did again at Userspace a few months ago) is that the people who use it want the performance.

On point 3: We do have an internal continuous integration system that runs nightly compiles of DPDK against the latest kernel and flags any issues.

John

Re: [dpdk-dev] KNI broken again with 4.9 kernel

[Jay Rolette]

On Thu, Dec 15, 2016 at 6:01 AM, Mcnamara, John <john.mcnamara@intel.com>
wrote:

>
> > -----Original Message-----
> > From: dev [mailto:dev-bounces@dpdk.org] On Behalf Of Stephen Hemminger
> > Sent: Wednesday, December 14, 2016 11:41 PM
> > To: dev@dpdk.org
> > Subject: [dpdk-dev] KNI broken again with 4.9 kernel
> >
> > /build/lib/librte_eal/linuxapp/kni/igb_main.c:2317:21: error:
> > initialization from incompatible pointer type [-Werror=incompatible-
> > pointer-types]
> >   .ndo_set_vf_vlan = igb_ndo_set_vf_vlan,
> >                      ^~~~~~~~~~~~~~~~~~~
> >
> > I am sure Ferruh Yigit will fix it.
> >
> > Which raises a couple of questions:
> >  1. Why is DPDK still keeping KNI support for Intel specific ethtool
> > functionality.
> >     This always breaks, is code bloat, and means a 3rd copy of base code
> > (Linux, DPDK PMD, + KNI)
> >
> >  2. Why is KNI not upstream?
> >     If not acceptable due to security or supportablity then why does it
> > still exist?
> >
> >  3. If not upstream, then maintainer should track upstream kernel changes
> > and fix DPDK before
> >     kernel is released.  The ABI is normally set early in the rc cycle
> > weeks before release.
>
>
> Hi Stephen,
>
> On point 2: The feedback we have always received is that the KNI code
> isn't upstreamable. Do you think there is an upstream path?
>
> > If not acceptable due to security or supportablity then why does it
> > still exist?
>
> The most commonly expressed reason when we have asked this question in the
> past (and we did again at Userspace a few months ago) is that the people
> who use it want the performance.
>

We use KNI in our product. In our case, it's because it allows "normal"
non-DPDK apps in the control plane to interact with traffic on the fastpath
as needed. Having everything under the sun live in DPDK's essentially flat
memory space is not great for security or stability.

It helps time to market by being able to use existing programs that
interface to the network via sockets instead of having to limit ourselves
to the relatively tiny set of libraries out there that work directly DPDK.

Double bonus on the time-to-market argument since we can implement
functionality in other higher-level languages as appropriate.

Performance-wise, KNI is "ok" but not great. It's not clear to me why it is
so much slower than using a NIC normally (not via DPDK) via the Linux
network stack. Copying data between sk_buf and mbuf is never going to be
cheap, but comparing that to what happens through all the kernel network
stack layers, the end result seems too slow.

That said, it's still faster than TAP/TUN interfaces and similar approaches.

Jay

On point 3: We do have an internal continuous integration system that runs
> nightly compiles of DPDK against the latest kernel and flags any issues.
>
> John
>