DPDK patches and discussions
 help / color / mirror / Atom feed
* [dpdk-dev] KNI alternatives
@ 2021-01-13 17:06 Thomas Monjalon
  2021-01-13 17:10 ` Stephen Hemminger
  0 siblings, 1 reply; 6+ messages in thread
From: Thomas Monjalon @ 2021-01-13 17:06 UTC (permalink / raw)
  To: dev
  Cc: ferruh.yigit, kevin.traynor, bruce.richardson, hemant.agrawal,
	jerinj, olivier.matz, maxime.coquelin, stephen

Hi,

As discussed today in the techboard meeting, KNI has probably
better alternatives today without using an out-of-tree module.
Virtio-user is a good candidate to replace KNI.
What is the performance of TAP?
Is there a way to leverage af_packet, af_xdp, or even pcap interfaces?

In order to avoid using the KNI out-of-tree module,
we should make the librte_kni compatible with an other interface.
The big plan is then to move the KNI module out of the main DPDK repo,
as we did for igb_uio.

The first step of this plan might be to document pros & cons
of the KNI alternatives inside the KNI documentation:
	https://doc.dpdk.org/guides/prog_guide/kernel_nic_interface.html

As a start, this doc should be better referenced:
	https://doc.dpdk.org/guides/howto/virtio_user_as_exceptional_path.html

Note: I won't do this update myself, so feel free to step up!
Thanks



^ permalink raw reply	[flat|nested] 6+ messages in thread

* Re: [dpdk-dev] KNI alternatives
  2021-01-13 17:06 [dpdk-dev] KNI alternatives Thomas Monjalon
@ 2021-01-13 17:10 ` Stephen Hemminger
  2021-01-13 18:17   ` Igor Ryzhov
  0 siblings, 1 reply; 6+ messages in thread
From: Stephen Hemminger @ 2021-01-13 17:10 UTC (permalink / raw)
  To: Thomas Monjalon
  Cc: dev, Ferruh Yigit, kevin.traynor, bruce.richardson,
	hemant.agrawal, jerinj, olivier.matz, maxime.coquelin

Last time I tried.
Virtio user was as fast as KNI and consumed less cpu. Was seeing 10mpps
Tap was much slower. Like 1mpps.

Vpp uses virtio user.

Sorry for top post. Only have phone internet

On Wed, Jan 13, 2021, 9:06 AM Thomas Monjalon <thomas@monjalon.net> wrote:

> Hi,
>
> As discussed today in the techboard meeting, KNI has probably
> better alternatives today without using an out-of-tree module.
> Virtio-user is a good candidate to replace KNI.
> What is the performance of TAP?
> Is there a way to leverage af_packet, af_xdp, or even pcap interfaces?
>
> In order to avoid using the KNI out-of-tree module,
> we should make the librte_kni compatible with an other interface.
> The big plan is then to move the KNI module out of the main DPDK repo,
> as we did for igb_uio.
>
> The first step of this plan might be to document pros & cons
> of the KNI alternatives inside the KNI documentation:
>         https://doc.dpdk.org/guides/prog_guide/kernel_nic_interface.html
>
> As a start, this doc should be better referenced:
>
> https://doc.dpdk.org/guides/howto/virtio_user_as_exceptional_path.html
>
> Note: I won't do this update myself, so feel free to step up!
> Thanks
>
>
>

^ permalink raw reply	[flat|nested] 6+ messages in thread

* Re: [dpdk-dev] KNI alternatives
  2021-01-13 17:10 ` Stephen Hemminger
@ 2021-01-13 18:17   ` Igor Ryzhov
  2021-01-13 18:45     ` Thomas Monjalon
  0 siblings, 1 reply; 6+ messages in thread
From: Igor Ryzhov @ 2021-01-13 18:17 UTC (permalink / raw)
  To: Stephen Hemminger
  Cc: Thomas Monjalon, dev, Ferruh Yigit, kevin.traynor, Richardson,
	Bruce, hemant.agrawal, Jerin Jacob, Olivier Matz,
	maxime.coquelin

On Wed, Jan 13, 2021 at 8:10 PM Stephen Hemminger <
stephen@networkplumber.org> wrote:

> Last time I tried.
> Virtio user was as fast as KNI and consumed less cpu. Was seeing 10mpps
> Tap was much slower. Like 1mpps.
>
> Vpp uses virtio user.
>
> Sorry for top post. Only have phone internet
>
> On Wed, Jan 13, 2021, 9:06 AM Thomas Monjalon <thomas@monjalon.net> wrote:
>
> > Hi,
> >
> > As discussed today in the techboard meeting, KNI has probably
> > better alternatives today without using an out-of-tree module.
> > Virtio-user is a good candidate to replace KNI.
> > What is the performance of TAP?
> > Is there a way to leverage af_packet, af_xdp, or even pcap interfaces?


Performance is not the only question. The advantage of KNI we are currently
using is
the ability to control the DPDK interfaces by the kernel.
For example, to implement bonding in the DPDK application, it is possible
to create KNI
pair for each physical interface, create a bond interface in Linux over
those KNI interfaces
and just pass LACP packets between the app and the kernel. The kernel
itself will control
MACs, MTU, etc. of underlying interfaces. AFAIK it's not possible with
virtio-user or tap.
Am I wrong?


>

>
> > In order to avoid using the KNI out-of-tree module,
> > we should make the librte_kni compatible with an other interface.
> > The big plan is then to move the KNI module out of the main DPDK repo,
> > as we did for igb_uio.
> >
> > The first step of this plan might be to document pros & cons
> > of the KNI alternatives inside the KNI documentation:
> >         https://doc.dpdk.org/guides/prog_guide/kernel_nic_interface.html
> >
> > As a start, this doc should be better referenced:
> >
> > https://doc.dpdk.org/guides/howto/virtio_user_as_exceptional_path.html
> >
> > Note: I won't do this update myself, so feel free to step up!
> > Thanks
> >
> >
> >
>

^ permalink raw reply	[flat|nested] 6+ messages in thread

* Re: [dpdk-dev] KNI alternatives
  2021-01-13 18:17   ` Igor Ryzhov
@ 2021-01-13 18:45     ` Thomas Monjalon
  2021-01-13 19:53       ` Igor Ryzhov
  0 siblings, 1 reply; 6+ messages in thread
From: Thomas Monjalon @ 2021-01-13 18:45 UTC (permalink / raw)
  To: Igor Ryzhov
  Cc: Stephen Hemminger, dev, Ferruh Yigit, kevin.traynor, Richardson,
	Bruce, hemant.agrawal, Jerin Jacob, Olivier Matz,
	maxime.coquelin

13/01/2021 19:17, Igor Ryzhov:
> On Wed, Jan 13, 2021 at 8:10 PM Stephen Hemminger <
> stephen@networkplumber.org> wrote:
> > On Wed, Jan 13, 2021, 9:06 AM Thomas Monjalon <thomas@monjalon.net> wrote:
> >
> > > Hi,
> > >
> > > As discussed today in the techboard meeting, KNI has probably
> > > better alternatives today without using an out-of-tree module.
> > > Virtio-user is a good candidate to replace KNI.
> > > What is the performance of TAP?
> > > Is there a way to leverage af_packet, af_xdp, or even pcap interfaces?
> > 
> > Last time I tried.
> > Virtio user was as fast as KNI and consumed less cpu. Was seeing 10mpps
> > Tap was much slower. Like 1mpps.
> 
> Performance is not the only question. The advantage of KNI we are currently
> using is
> the ability to control the DPDK interfaces by the kernel.
> For example, to implement bonding in the DPDK application, it is possible
> to create KNI
> pair for each physical interface, create a bond interface in Linux over
> those KNI interfaces
> and just pass LACP packets between the app and the kernel. The kernel
> itself will control
> MACs, MTU, etc. of underlying interfaces. AFAIK it's not possible with
> virtio-user or tap.
> Am I wrong?

I see at least 2 alternatives for bonding with kernel management:
	- mlx5 bonding
	- af_xdp interface for most of NICs




^ permalink raw reply	[flat|nested] 6+ messages in thread

* Re: [dpdk-dev] KNI alternatives
  2021-01-13 18:45     ` Thomas Monjalon
@ 2021-01-13 19:53       ` Igor Ryzhov
  2021-01-14  5:50         ` Stephen Hemminger
  0 siblings, 1 reply; 6+ messages in thread
From: Igor Ryzhov @ 2021-01-13 19:53 UTC (permalink / raw)
  To: Thomas Monjalon
  Cc: Stephen Hemminger, dev, Ferruh Yigit, kevin.traynor, Richardson,
	Bruce, hemant.agrawal, Jerin Jacob, Olivier Matz,
	maxime.coquelin

On Wed, Jan 13, 2021 at 9:45 PM Thomas Monjalon <thomas@monjalon.net> wrote:

> 13/01/2021 19:17, Igor Ryzhov:
> > On Wed, Jan 13, 2021 at 8:10 PM Stephen Hemminger <
> > stephen@networkplumber.org> wrote:
> > > On Wed, Jan 13, 2021, 9:06 AM Thomas Monjalon <thomas@monjalon.net>
> wrote:
> > >
> > > > Hi,
> > > >
> > > > As discussed today in the techboard meeting, KNI has probably
> > > > better alternatives today without using an out-of-tree module.
> > > > Virtio-user is a good candidate to replace KNI.
> > > > What is the performance of TAP?
> > > > Is there a way to leverage af_packet, af_xdp, or even pcap
> interfaces?
> > >
> > > Last time I tried.
> > > Virtio user was as fast as KNI and consumed less cpu. Was seeing 10mpps
> > > Tap was much slower. Like 1mpps.
> >
> > Performance is not the only question. The advantage of KNI we are
> currently
> > using is
> > the ability to control the DPDK interfaces by the kernel.
> > For example, to implement bonding in the DPDK application, it is possible
> > to create KNI
> > pair for each physical interface, create a bond interface in Linux over
> > those KNI interfaces
> > and just pass LACP packets between the app and the kernel. The kernel
> > itself will control
> > MACs, MTU, etc. of underlying interfaces. AFAIK it's not possible with
> > virtio-user or tap.
> > Am I wrong?
>
> I see at least 2 alternatives for bonding with kernel management:
>         - mlx5 bonding
>         - af_xdp interface for most of NICs
>

Bonding is just an example. My point is that KNI allows changing MAC, MTU,
rx mode and admin status of a NIC used by DPDK application - all using
standard Linux utilities without any need for additional APIs provided by
the app.

Yes, Mellanox NICs also allow this but it's just one of many drivers
supported by DPDK.
I'll dig into af_xdp, thanks.

^ permalink raw reply	[flat|nested] 6+ messages in thread

* Re: [dpdk-dev] KNI alternatives
  2021-01-13 19:53       ` Igor Ryzhov
@ 2021-01-14  5:50         ` Stephen Hemminger
  0 siblings, 0 replies; 6+ messages in thread
From: Stephen Hemminger @ 2021-01-14  5:50 UTC (permalink / raw)
  To: Igor Ryzhov
  Cc: Thomas Monjalon, dev, Ferruh Yigit, kevin.traynor, Richardson,
	Bruce, hemant.agrawal, Jerin Jacob, Olivier Matz,
	maxime.coquelin

Controlling DPDK interfaces with KNI is known to be broken.  Kni calls
usepspace with network mutex held. The problem is not fixable.

The kernel does provide netlink and devlink API's but there is no generic
solution with DPDK.

On Wed, Jan 13, 2021, 11:54 AM Igor Ryzhov <iryzhov@nfware.com> wrote:

>
>
> On Wed, Jan 13, 2021 at 9:45 PM Thomas Monjalon <thomas@monjalon.net>
> wrote:
>
>> 13/01/2021 19:17, Igor Ryzhov:
>> > On Wed, Jan 13, 2021 at 8:10 PM Stephen Hemminger <
>> > stephen@networkplumber.org> wrote:
>> > > On Wed, Jan 13, 2021, 9:06 AM Thomas Monjalon <thomas@monjalon.net>
>> wrote:
>> > >
>> > > > Hi,
>> > > >
>> > > > As discussed today in the techboard meeting, KNI has probably
>> > > > better alternatives today without using an out-of-tree module.
>> > > > Virtio-user is a good candidate to replace KNI.
>> > > > What is the performance of TAP?
>> > > > Is there a way to leverage af_packet, af_xdp, or even pcap
>> interfaces?
>> > >
>> > > Last time I tried.
>> > > Virtio user was as fast as KNI and consumed less cpu. Was seeing
>> 10mpps
>> > > Tap was much slower. Like 1mpps.
>> >
>> > Performance is not the only question. The advantage of KNI we are
>> currently
>> > using is
>> > the ability to control the DPDK interfaces by the kernel.
>> > For example, to implement bonding in the DPDK application, it is
>> possible
>> > to create KNI
>> > pair for each physical interface, create a bond interface in Linux over
>> > those KNI interfaces
>> > and just pass LACP packets between the app and the kernel. The kernel
>> > itself will control
>> > MACs, MTU, etc. of underlying interfaces. AFAIK it's not possible with
>> > virtio-user or tap.
>> > Am I wrong?
>>
>> I see at least 2 alternatives for bonding with kernel management:
>>         - mlx5 bonding
>>         - af_xdp interface for most of NICs
>>
>
> Bonding is just an example. My point is that KNI allows changing MAC, MTU,
> rx mode and admin status of a NIC used by DPDK application - all using
> standard Linux utilities without any need for additional APIs provided by
> the app.
>
> Yes, Mellanox NICs also allow this but it's just one of many drivers
> supported by DPDK.
> I'll dig into af_xdp, thanks.
>

^ permalink raw reply	[flat|nested] 6+ messages in thread

end of thread, other threads:[~2021-01-14  7:53 UTC | newest]

Thread overview: 6+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2021-01-13 17:06 [dpdk-dev] KNI alternatives Thomas Monjalon
2021-01-13 17:10 ` Stephen Hemminger
2021-01-13 18:17   ` Igor Ryzhov
2021-01-13 18:45     ` Thomas Monjalon
2021-01-13 19:53       ` Igor Ryzhov
2021-01-14  5:50         ` Stephen Hemminger

DPDK patches and discussions

This inbox may be cloned and mirrored by anyone:

	git clone --mirror http://inbox.dpdk.org/dev/0 dev/git/0.git

	# If you have public-inbox 1.1+ installed, you may
	# initialize and index your mirror using the following commands:
	public-inbox-init -V2 dev dev/ http://inbox.dpdk.org/dev \
		dev@dpdk.org
	public-inbox-index dev

Example config snippet for mirrors.
Newsgroup available over NNTP:
	nntp://inbox.dpdk.org/inbox.dpdk.dev


AGPL code for this site: git clone https://public-inbox.org/public-inbox.git