From: Ferruh Yigit <ferruh.yigit@intel.com>
To: Avi Kivity <avi@scylladb.com>, dev@dpdk.org
Subject: Re: [dpdk-dev] [PATCH 1/3] kcp: add kernel control path kernel module
Date: Mon, 29 Feb 2016 14:35:57 +0000 [thread overview]
Message-ID: <56D4574D.6050902@intel.com> (raw)
In-Reply-To: <56D42E0B.6030109@scylladb.com>
On 2/29/2016 11:39 AM, Avi Kivity wrote:
>
>
> On 02/29/2016 01:27 PM, Ferruh Yigit wrote:
>> On 2/29/2016 10:58 AM, Avi Kivity wrote:
>>>
>>> On 02/29/2016 12:43 PM, Ferruh Yigit wrote:
>>>> On 2/29/2016 9:43 AM, Avi Kivity wrote:
>>>>> On 02/28/2016 10:16 PM, Ferruh Yigit wrote:
>>>>>> On 2/28/2016 3:34 PM, Avi Kivity wrote:
>>>>>>> On 01/27/2016 06:24 PM, Ferruh Yigit wrote:
>>>>>>>> This kernel module is based on KNI module, but this one is stripped
>>>>>>>> version of it and only for control messages, no data transfer
>>>>>>>> functionality provided.
>>>>>>>>
>>>>>>>> This Linux kernel module helps userspace application create virtual
>>>>>>>> interfaces and when a control command issued into that virtual
>>>>>>>> interface, module pushes the command to the userspace and gets the
>>>>>>>> response back for the caller application.
>>>>>>>>
>>>>>>>> The Linux tools like ethtool/ifconfig/ip can be used on virtual
>>>>>>>> interfaces but not ones for related data, like tcpdump.
>>>>>>>>
>>>>>>>> In long term this patch intends to replace the KNI and KNI will be
>>>>>>>> depreciated.
>>>>>>> Instead of adding yet another out-of-tree kernel module, why not
>>>>>>> extend
>>>>>>> the existing in-tree tap driver? This will make everyone's life
>>>>>>> easier.
>>>>>>>
>>>>>>> Since tap also supports data transfer, an application can also
>>>>>>> forward
>>>>>>> packets not intended to it to the kernel, and forward packets
>>>>>>> from the
>>>>>>> kernel through the device.
>>>>>>>
>>>>>> Hi Avi,
>>>>>>
>>>>>> KDP (Kernel Data Path) does what you have described, it is
>>>>>> implemented
>>>>>> as PMD and it benefits from tap driver to data transfer through the
>>>>>> kernel. It also support custom kernel module for better performance.
>>>>>>
>>>>>> For KCP (Kernel Control Path), network driver forwards control
>>>>>> commands
>>>>>> to the userspace driver, I doubt this is something wanted for tun/tap
>>>>>> driver, so extending tun/tap driver like this can be hard to
>>>>>> upstream.
>>>>> Have you tried asking? Maybe if you explain it they will be open
>>>>> to the
>>>>> extension.
>>>>>
>>>> Not communicated but tun/tap already doing something different.
>>>> For KCP, created interface is map of the DPDK port. All data interface
>>>> shows coming from DPDK port. For example if you get stats information
>>>> with ifconfig, the values you observe are DPDK port statistics -not
>>>> statistics of data between userspace and kernelspace, statistics of
>>>> data
>>>> forwarded between DPDK ports. If you down the interface, DPDK port
>>>> stopped, etc...
>>>>
>>>> If you extend the tun/tap, it won't be map of the DPDK port, and if you
>>>> get statistics information from that interface, what do you expect to
>>>> see, the data transferred between kernel and userspace, or underlying
>>>> DPDK port forwarding statistics?
>>> Good point. But you really have to involve netdev on this, or you'll
>>> live out-of-tree forever.
>>>
>> Why do we need to touch netdev?
>
> By netdev, I meant the mailing list. If you don't touch it, your driver
> will remain out-of-tree forever.
>
Sorry, I thought you are suggesting updating netdev (struct net_device)
for this.
>> A simple network driver, similar to kcp, can be solution.
>>
>> This driver implements all net_device_ops and ethtool_ops in a way to
>> forward everything to the userspace via netlink. All needs to know about
>> userspace driver is it's unique id. Any userspace application, not only
>> DPDK drivers, can listen the netlink messages and response to the
>> requests come to itself.
>>
>> This kind of driver is not big or complicated, kcp already does %90 of
>> what described above.
>
> I am not arguing against kcp. It fulfills an important need. This is
> my argument:
>
> 1. having multiple interfaces for the control and data path is bad for
> the user
> 2. therefore, we need to either add tap functionality to kcp, or add kcp
> functionality to tap
> 3. netdev@ is more likely (IMO) to accept additional functionality to
> tap than a new driver, but the only way to know is to engage with them
>
Agreed an incremental update to the tap can be easier to get in, but
this is not really working for us, as explained above.
The concern of having two separate interfaces can be solved without
merging data and control path. I believe this is not a showstopper for
the functionality and can be the incremental improvement.
>>
>>>> Extending tun/tap in a way we want, forwarding all control commands to
>>>> userspace, will break the current tun/tap, this doesn't looks like a
>>>> valid option to me.
>>> It's possible to enhance it while preserving backwards compatibility, by
>>> enabling a feature flag (statistics from userspace).
>>>
>>>> For data path, using tun/tap is OK and we are already doing it, for the
>>>> control path I believe we need a new driver.
>>>>
>>>>> Certainly it will be better to have KCP and KDP use the same kernel
>>>>> interface name; so we'll need to either add data path support to kcp
>>>>> (causing duplication with tap), or add control path support to tap. I
>>>>> think the latter is preferable.
>>>>>
>>>> Why it is better to have same interface? Anyone who is not interested
>>>> with kernel data path may want to control DPDK ports using common
>>>> tools,
>>>> or want to get some basic information and stats using ethtool or
>>>> ifconfig. Why we need to bind two different functionality together?
>>> Having two interfaces will be confusing for the user. If I wish to
>>> firewall data packets coming from the dpdk port, do I set firewall rules
>>> on dpdk0 or tap0?
>>>
>> Agreed that it is confusing to have two interfaces.
>>
>> I think if user wants to use both data and control paths, a way can be
>> found to end up with single interface, using module params or something
>> else. Two different drivers for data and control not conflict with each
>> other and can cooperate.
>
> Module parameters are for module-wide options. A single module can
> support multiple interfaces, so module parameters don't apply.
>
> Let's make it simple for the users, even if it is more complex for us
> (and by us, I mean you, unfortunately).
The reason I insist on separate modules is same reason with you,
user/customer rules, not because it is easier. With KCP, DPDK ports
become visible to Linux world, and this has nothing to do with sending
packages to kernel. Decoupling the KCP from data path can give more use
case for KCP, more user can use it.
Also it is possible to make control path more dynamic, you can make them
appear, get some info, set something, and remove them back. This is not
something wanted for slow data path.
This is a design choice to separate responsibilities.
>
>> But to work on this first both KCP and KDP should go in.
>
> Go in where? It doesn't help if it goes into dpdk.git and then netdev@
> rejects it.
>
Go in to dpdk.git can be good start. It helps on a few things:
1- Get feedback on users, get more requirements
2- In communication with netdev@, helps us showing the existing use case.
3- Linux upstream process is risky and may take long time, DPDK users
may not access these for a long time, and if it makes the Linux kernel,
it may be late for some requirements.
I think it is good to include these in dpdk.org, and in parallel work on
upstreaming. Update the code according comments from both communities.
And if this makes into Linux kernel, we can drop from DPDK.
>>
>>> I don't think it matters whether you extend tap, or add a data path to
>>> kcp, but if you want to upstream it, it needs to be blessed by netdev.
>>>
>> I still think not good idea to merge these two, because they may be used
>> independently, but we can improve how they work together.
>>
>
> From a developer's perspective, maybe so, but from a user's perspective,
> there should be exactly one interface for the functionality exposed by
> kcp and kdp.
next prev parent reply other threads:[~2016-02-29 14:37 UTC|newest]
Thread overview: 82+ messages / expand[flat|nested] mbox.gz Atom feed top
[not found] <1453911849-16562-1-git-send-email-ferruh.yigit@intel.com>
2016-01-27 16:24 ` Ferruh Yigit
2016-01-28 9:49 ` Remy Horton
2016-01-28 13:50 ` Ferruh Yigit
2016-02-28 15:34 ` Avi Kivity
2016-02-28 20:16 ` Ferruh Yigit
2016-02-29 9:43 ` Avi Kivity
2016-02-29 10:43 ` Ferruh Yigit
2016-02-29 10:58 ` Avi Kivity
2016-02-29 11:06 ` Thomas Monjalon
2016-02-29 11:35 ` Ferruh Yigit
2016-02-29 15:05 ` Ferruh Yigit
2016-02-29 15:19 ` Panu Matilainen
2016-02-29 15:27 ` Thomas Monjalon
2016-02-29 16:04 ` Panu Matilainen
2016-02-29 14:33 ` Jay Rolette
2016-03-01 22:40 ` Bruce Richardson
2016-03-02 2:02 ` Stephen Hemminger
2016-03-02 8:27 ` Panu Matilainen
2016-03-02 10:47 ` Vincent JARDIN
2016-03-02 10:51 ` Jim Thompson
2016-03-02 12:03 ` Vincent JARDIN
2016-03-02 22:51 ` Jim Thompson
2016-03-02 11:21 ` Thomas Monjalon
2016-03-02 22:35 ` Thomas Monjalon
2016-03-03 8:31 ` Panu Matilainen
2016-03-03 10:05 ` Ferruh Yigit
2016-03-03 10:11 ` Thomas Monjalon
2016-03-03 10:51 ` Panu Matilainen
2016-03-10 0:04 ` Thomas Monjalon
2016-03-10 6:31 ` Vincent JARDIN
2016-03-02 22:18 ` Jay Rolette
2016-03-03 10:11 ` Ferruh Yigit
2016-03-03 16:59 ` Stephen Hemminger
2016-03-03 18:18 ` Ferruh Yigit
2016-02-29 11:27 ` Ferruh Yigit
2016-02-29 11:39 ` Avi Kivity
2016-02-29 14:35 ` Ferruh Yigit [this message]
2016-02-29 20:11 ` Stephen Hemminger
2016-03-01 0:35 ` Ferruh Yigit
2016-01-27 16:24 ` [dpdk-dev] [PATCH 2/3] rte_ctrl_if: add control interface library Ferruh Yigit
2016-01-28 11:14 ` Remy Horton
2016-01-28 13:15 ` Ferruh Yigit
2016-01-28 13:24 ` Jay Rolette
2016-01-28 13:56 ` Ferruh Yigit
2016-01-28 13:57 ` Ananyev, Konstantin
2016-01-28 14:22 ` Yigit, Ferruh
2016-01-27 16:24 ` [dpdk-dev] [PATCH 3/3] examples/ethtool: add control interface support to the application Ferruh Yigit
2016-02-12 13:45 ` [dpdk-dev] [PATCH v2 0/3] Use common Linux tools to control DPDK ports Ferruh Yigit
2016-02-12 13:45 ` [dpdk-dev] [PATCH v2 1/3] kcp: add kernel control path kernel module Ferruh Yigit
2016-02-12 13:45 ` [dpdk-dev] [PATCH v2 2/3] rte_ctrl_if: add control interface library Ferruh Yigit
2016-02-17 19:58 ` Ananyev, Konstantin
2016-02-18 10:43 ` Yigit, Ferruh
2016-02-12 13:45 ` [dpdk-dev] [PATCH v2 3/3] examples/ethtool: add control interface support to the application Ferruh Yigit
2016-02-17 19:39 ` Ananyev, Konstantin
2016-02-18 10:11 ` Yigit, Ferruh
2016-02-26 14:10 ` [dpdk-dev] [PATCH v3 0/4] Use common Linux tools to control DPDK ports Ferruh Yigit
2016-02-26 14:10 ` [dpdk-dev] [PATCH v3 1/4] lib/librte_ethtool: move librte_ethtool form examples to lib folder Ferruh Yigit
2016-02-26 14:10 ` [dpdk-dev] [PATCH v3 2/4] kcp: add kernel control path kernel module Ferruh Yigit
2016-03-01 1:02 ` Stephen Hemminger
2016-03-01 15:53 ` Ferruh Yigit
2016-02-26 14:10 ` [dpdk-dev] [PATCH v3 3/4] rte_ctrl_if: add control interface library Ferruh Yigit
2016-02-26 14:10 ` [dpdk-dev] [PATCH v3 4/4] examples/ethtool: add control interface support to the application Ferruh Yigit
2016-02-29 9:33 ` [dpdk-dev] [PATCH v3 0/4] Use common Linux tools to control DPDK ports Remy Horton
2016-03-01 15:41 ` [dpdk-dev] [PATCH v4 " Ferruh Yigit
2016-03-01 15:41 ` [dpdk-dev] [PATCH v4 1/4] lib/librte_ethtool: move librte_ethtool form examples to lib folder Ferruh Yigit
2016-03-01 15:41 ` [dpdk-dev] [PATCH v4 2/4] kcp: add kernel control path kernel module Ferruh Yigit
2016-03-01 23:06 ` Stephen Hemminger
2016-03-02 11:05 ` Ferruh Yigit
2016-03-01 23:09 ` Stephen Hemminger
2016-03-01 23:10 ` Stephen Hemminger
2016-03-02 11:06 ` Ferruh Yigit
2016-03-01 15:41 ` [dpdk-dev] [PATCH v4 3/4] rte_ctrl_if: add control interface library Ferruh Yigit
2016-03-01 15:42 ` [dpdk-dev] [PATCH v4 4/4] examples/ethtool: add control interface support to the application Ferruh Yigit
2016-03-09 11:41 ` [dpdk-dev] [PATCH v5 0/4] Use common Linux tools to control DPDK ports Ferruh Yigit
2016-03-09 11:41 ` [dpdk-dev] [PATCH v5 1/4] lib/librte_ethtool: move librte_ethtool form examples to lib folder Ferruh Yigit
2016-03-09 11:41 ` [dpdk-dev] [PATCH v5 2/4] kcp: add kernel control path kernel module Ferruh Yigit
2016-03-09 11:41 ` [dpdk-dev] [PATCH v5 3/4] rte_ctrl_if: add control interface library Ferruh Yigit
2016-03-09 11:41 ` [dpdk-dev] [PATCH v5 4/4] examples/ethtool: add control interface support to the application Ferruh Yigit
2016-03-09 12:23 ` Ananyev, Konstantin
2016-03-14 15:31 ` [dpdk-dev] [PATCH v5 0/4] Use common Linux tools to control DPDK ports Ferruh Yigit
2016-03-14 17:40 ` Jay Rolette
2016-03-15 0:00 ` Ferruh Yigit
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=56D4574D.6050902@intel.com \
--to=ferruh.yigit@intel.com \
--cc=avi@scylladb.com \
--cc=dev@dpdk.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).