Re: [dpdk-dev] [PATCH 1/3] kcp: add kernel control path kernel module

[Ferruh Yigit <ferruh.yigit@intel.com>]

On 2/29/2016 10:58 AM, Avi Kivity wrote:
> 
> 
> On 02/29/2016 12:43 PM, Ferruh Yigit wrote:
>> On 2/29/2016 9:43 AM, Avi Kivity wrote:
>>> On 02/28/2016 10:16 PM, Ferruh Yigit wrote:
>>>> On 2/28/2016 3:34 PM, Avi Kivity wrote:
>>>>> On 01/27/2016 06:24 PM, Ferruh Yigit wrote:
>>>>>> This kernel module is based on KNI module, but this one is stripped
>>>>>> version of it and only for control messages, no data transfer
>>>>>> functionality provided.
>>>>>>
>>>>>> This Linux kernel module helps userspace application create virtual
>>>>>> interfaces and when a control command issued into that virtual
>>>>>> interface, module pushes the command to the userspace and gets the
>>>>>> response back for the caller application.
>>>>>>
>>>>>> The Linux tools like ethtool/ifconfig/ip can be used on virtual
>>>>>> interfaces but not ones for related data, like tcpdump.
>>>>>>
>>>>>> In long term this patch intends to replace the KNI and KNI will be
>>>>>> depreciated.
>>>>> Instead of adding yet another out-of-tree kernel module, why not
>>>>> extend
>>>>> the existing in-tree tap driver?  This will make everyone's life
>>>>> easier.
>>>>>
>>>>> Since tap also supports data transfer, an application can also forward
>>>>> packets not intended to it to the kernel, and forward packets from the
>>>>> kernel through the device.
>>>>>
>>>> Hi Avi,
>>>>
>>>> KDP (Kernel Data Path) does what you have described, it is implemented
>>>> as PMD and it benefits from tap driver to data transfer through the
>>>> kernel. It also support custom kernel module for better performance.
>>>>
>>>> For KCP (Kernel Control Path), network driver forwards control commands
>>>> to the userspace driver, I doubt this is something wanted for tun/tap
>>>> driver, so extending tun/tap driver like this can be hard to upstream.
>>> Have you tried asking?  Maybe if you explain it they will be open to the
>>> extension.
>>>
>> Not communicated but tun/tap already doing something different.
>> For KCP, created interface is map of the DPDK port. All data interface
>> shows coming from DPDK port. For example if you get stats information
>> with ifconfig, the values you observe are DPDK port statistics -not
>> statistics of data between userspace and kernelspace, statistics of data
>> forwarded between DPDK ports. If you down the interface, DPDK port
>> stopped, etc...
>>
>> If you extend the tun/tap, it won't be map of the DPDK port, and if you
>> get statistics information from that interface, what do you expect to
>> see, the data transferred between kernel and userspace, or underlying
>> DPDK port forwarding statistics?
> 
> Good point.  But you really have to involve netdev on this, or you'll
> live out-of-tree forever.
> 
Why do we need to touch netdev?

A simple network driver, similar to kcp, can be solution.

This driver implements all net_device_ops and ethtool_ops in a way to
forward everything to the userspace via netlink. All needs to know about
userspace driver is it's unique id. Any userspace application, not only
DPDK drivers, can listen the netlink messages and response to the
requests come to itself.

This kind of driver is not big or complicated, kcp already does %90 of
what described above.

>> Extending tun/tap in a way we want, forwarding all control commands to
>> userspace, will break the current tun/tap, this doesn't looks like a
>> valid option to me.
> 
> It's possible to enhance it while preserving backwards compatibility, by
> enabling a feature flag (statistics from userspace).
> 
>> For data path, using tun/tap is OK and we are already doing it, for the
>> control path I believe we need a new driver.
>>
>>> Certainly it will be better to have KCP and KDP use the same kernel
>>> interface name; so we'll need to either add data path support to kcp
>>> (causing duplication with tap), or add control path support to tap. I
>>> think the latter is preferable.
>>>
>> Why it is better to have same interface? Anyone who is not interested
>> with kernel data path may want to control DPDK ports using common tools,
>> or want to get some basic information and stats using ethtool or
>> ifconfig. Why we need to bind two different functionality together?
> 
> Having two interfaces will be confusing for the user.  If I wish to
> firewall data packets coming from the dpdk port, do I set firewall rules
> on dpdk0 or tap0?
> 
Agreed that it is confusing to have two interfaces.

I think if user wants to use both data and control paths, a way can be
found to end up with single interface, using module params or something
else. Two different drivers for data and control not conflict with each
other and can cooperate.
But to work on this first both KCP and KDP should go in.

> I don't think it matters whether you extend tap, or add a data path to
> kcp, but if you want to upstream it, it needs to be blessed by netdev.
> 
I still think not good idea to merge these two, because they may be used
independently, but we can improve how they work together.

>>
>>>> We are investigating about adding a native support to Linux kernel for
>>>> KCP, but there is no task started for this right now, any support is
>>>> welcome.
>>>>
>>>>
>