* [dpdk-dev] KNI alternatives
@ 2021-01-13 17:06 Thomas Monjalon
2021-01-13 17:10 ` Stephen Hemminger
0 siblings, 1 reply; 6+ messages in thread
From: Thomas Monjalon @ 2021-01-13 17:06 UTC (permalink / raw)
To: dev
Cc: ferruh.yigit, kevin.traynor, bruce.richardson, hemant.agrawal,
jerinj, olivier.matz, maxime.coquelin, stephen
Hi,
As discussed today in the techboard meeting, KNI has probably
better alternatives today without using an out-of-tree module.
Virtio-user is a good candidate to replace KNI.
What is the performance of TAP?
Is there a way to leverage af_packet, af_xdp, or even pcap interfaces?
In order to avoid using the KNI out-of-tree module,
we should make the librte_kni compatible with an other interface.
The big plan is then to move the KNI module out of the main DPDK repo,
as we did for igb_uio.
The first step of this plan might be to document pros & cons
of the KNI alternatives inside the KNI documentation:
https://doc.dpdk.org/guides/prog_guide/kernel_nic_interface.html
As a start, this doc should be better referenced:
https://doc.dpdk.org/guides/howto/virtio_user_as_exceptional_path.html
Note: I won't do this update myself, so feel free to step up!
Thanks
^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: [dpdk-dev] KNI alternatives
2021-01-13 17:06 [dpdk-dev] KNI alternatives Thomas Monjalon
@ 2021-01-13 17:10 ` Stephen Hemminger
2021-01-13 18:17 ` Igor Ryzhov
0 siblings, 1 reply; 6+ messages in thread
From: Stephen Hemminger @ 2021-01-13 17:10 UTC (permalink / raw)
To: Thomas Monjalon
Cc: dev, Ferruh Yigit, kevin.traynor, bruce.richardson,
hemant.agrawal, jerinj, olivier.matz, maxime.coquelin
Last time I tried.
Virtio user was as fast as KNI and consumed less cpu. Was seeing 10mpps
Tap was much slower. Like 1mpps.
Vpp uses virtio user.
Sorry for top post. Only have phone internet
On Wed, Jan 13, 2021, 9:06 AM Thomas Monjalon <thomas@monjalon.net> wrote:
> Hi,
>
> As discussed today in the techboard meeting, KNI has probably
> better alternatives today without using an out-of-tree module.
> Virtio-user is a good candidate to replace KNI.
> What is the performance of TAP?
> Is there a way to leverage af_packet, af_xdp, or even pcap interfaces?
>
> In order to avoid using the KNI out-of-tree module,
> we should make the librte_kni compatible with an other interface.
> The big plan is then to move the KNI module out of the main DPDK repo,
> as we did for igb_uio.
>
> The first step of this plan might be to document pros & cons
> of the KNI alternatives inside the KNI documentation:
> https://doc.dpdk.org/guides/prog_guide/kernel_nic_interface.html
>
> As a start, this doc should be better referenced:
>
> https://doc.dpdk.org/guides/howto/virtio_user_as_exceptional_path.html
>
> Note: I won't do this update myself, so feel free to step up!
> Thanks
>
>
>
^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: [dpdk-dev] KNI alternatives
2021-01-13 17:10 ` Stephen Hemminger
@ 2021-01-13 18:17 ` Igor Ryzhov
2021-01-13 18:45 ` Thomas Monjalon
0 siblings, 1 reply; 6+ messages in thread
From: Igor Ryzhov @ 2021-01-13 18:17 UTC (permalink / raw)
To: Stephen Hemminger
Cc: Thomas Monjalon, dev, Ferruh Yigit, kevin.traynor, Richardson,
Bruce, hemant.agrawal, Jerin Jacob, Olivier Matz,
maxime.coquelin
On Wed, Jan 13, 2021 at 8:10 PM Stephen Hemminger <
stephen@networkplumber.org> wrote:
> Last time I tried.
> Virtio user was as fast as KNI and consumed less cpu. Was seeing 10mpps
> Tap was much slower. Like 1mpps.
>
> Vpp uses virtio user.
>
> Sorry for top post. Only have phone internet
>
> On Wed, Jan 13, 2021, 9:06 AM Thomas Monjalon <thomas@monjalon.net> wrote:
>
> > Hi,
> >
> > As discussed today in the techboard meeting, KNI has probably
> > better alternatives today without using an out-of-tree module.
> > Virtio-user is a good candidate to replace KNI.
> > What is the performance of TAP?
> > Is there a way to leverage af_packet, af_xdp, or even pcap interfaces?
Performance is not the only question. The advantage of KNI we are currently
using is
the ability to control the DPDK interfaces by the kernel.
For example, to implement bonding in the DPDK application, it is possible
to create KNI
pair for each physical interface, create a bond interface in Linux over
those KNI interfaces
and just pass LACP packets between the app and the kernel. The kernel
itself will control
MACs, MTU, etc. of underlying interfaces. AFAIK it's not possible with
virtio-user or tap.
Am I wrong?
>
>
> > In order to avoid using the KNI out-of-tree module,
> > we should make the librte_kni compatible with an other interface.
> > The big plan is then to move the KNI module out of the main DPDK repo,
> > as we did for igb_uio.
> >
> > The first step of this plan might be to document pros & cons
> > of the KNI alternatives inside the KNI documentation:
> > https://doc.dpdk.org/guides/prog_guide/kernel_nic_interface.html
> >
> > As a start, this doc should be better referenced:
> >
> > https://doc.dpdk.org/guides/howto/virtio_user_as_exceptional_path.html
> >
> > Note: I won't do this update myself, so feel free to step up!
> > Thanks
> >
> >
> >
>
^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: [dpdk-dev] KNI alternatives
2021-01-13 18:17 ` Igor Ryzhov
@ 2021-01-13 18:45 ` Thomas Monjalon
2021-01-13 19:53 ` Igor Ryzhov
0 siblings, 1 reply; 6+ messages in thread
From: Thomas Monjalon @ 2021-01-13 18:45 UTC (permalink / raw)
To: Igor Ryzhov
Cc: Stephen Hemminger, dev, Ferruh Yigit, kevin.traynor, Richardson,
Bruce, hemant.agrawal, Jerin Jacob, Olivier Matz,
maxime.coquelin
13/01/2021 19:17, Igor Ryzhov:
> On Wed, Jan 13, 2021 at 8:10 PM Stephen Hemminger <
> stephen@networkplumber.org> wrote:
> > On Wed, Jan 13, 2021, 9:06 AM Thomas Monjalon <thomas@monjalon.net> wrote:
> >
> > > Hi,
> > >
> > > As discussed today in the techboard meeting, KNI has probably
> > > better alternatives today without using an out-of-tree module.
> > > Virtio-user is a good candidate to replace KNI.
> > > What is the performance of TAP?
> > > Is there a way to leverage af_packet, af_xdp, or even pcap interfaces?
> >
> > Last time I tried.
> > Virtio user was as fast as KNI and consumed less cpu. Was seeing 10mpps
> > Tap was much slower. Like 1mpps.
>
> Performance is not the only question. The advantage of KNI we are currently
> using is
> the ability to control the DPDK interfaces by the kernel.
> For example, to implement bonding in the DPDK application, it is possible
> to create KNI
> pair for each physical interface, create a bond interface in Linux over
> those KNI interfaces
> and just pass LACP packets between the app and the kernel. The kernel
> itself will control
> MACs, MTU, etc. of underlying interfaces. AFAIK it's not possible with
> virtio-user or tap.
> Am I wrong?
I see at least 2 alternatives for bonding with kernel management:
- mlx5 bonding
- af_xdp interface for most of NICs
^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: [dpdk-dev] KNI alternatives
2021-01-13 18:45 ` Thomas Monjalon
@ 2021-01-13 19:53 ` Igor Ryzhov
2021-01-14 5:50 ` Stephen Hemminger
0 siblings, 1 reply; 6+ messages in thread
From: Igor Ryzhov @ 2021-01-13 19:53 UTC (permalink / raw)
To: Thomas Monjalon
Cc: Stephen Hemminger, dev, Ferruh Yigit, kevin.traynor, Richardson,
Bruce, hemant.agrawal, Jerin Jacob, Olivier Matz,
maxime.coquelin
On Wed, Jan 13, 2021 at 9:45 PM Thomas Monjalon <thomas@monjalon.net> wrote:
> 13/01/2021 19:17, Igor Ryzhov:
> > On Wed, Jan 13, 2021 at 8:10 PM Stephen Hemminger <
> > stephen@networkplumber.org> wrote:
> > > On Wed, Jan 13, 2021, 9:06 AM Thomas Monjalon <thomas@monjalon.net>
> wrote:
> > >
> > > > Hi,
> > > >
> > > > As discussed today in the techboard meeting, KNI has probably
> > > > better alternatives today without using an out-of-tree module.
> > > > Virtio-user is a good candidate to replace KNI.
> > > > What is the performance of TAP?
> > > > Is there a way to leverage af_packet, af_xdp, or even pcap
> interfaces?
> > >
> > > Last time I tried.
> > > Virtio user was as fast as KNI and consumed less cpu. Was seeing 10mpps
> > > Tap was much slower. Like 1mpps.
> >
> > Performance is not the only question. The advantage of KNI we are
> currently
> > using is
> > the ability to control the DPDK interfaces by the kernel.
> > For example, to implement bonding in the DPDK application, it is possible
> > to create KNI
> > pair for each physical interface, create a bond interface in Linux over
> > those KNI interfaces
> > and just pass LACP packets between the app and the kernel. The kernel
> > itself will control
> > MACs, MTU, etc. of underlying interfaces. AFAIK it's not possible with
> > virtio-user or tap.
> > Am I wrong?
>
> I see at least 2 alternatives for bonding with kernel management:
> - mlx5 bonding
> - af_xdp interface for most of NICs
>
Bonding is just an example. My point is that KNI allows changing MAC, MTU,
rx mode and admin status of a NIC used by DPDK application - all using
standard Linux utilities without any need for additional APIs provided by
the app.
Yes, Mellanox NICs also allow this but it's just one of many drivers
supported by DPDK.
I'll dig into af_xdp, thanks.
^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: [dpdk-dev] KNI alternatives
2021-01-13 19:53 ` Igor Ryzhov
@ 2021-01-14 5:50 ` Stephen Hemminger
0 siblings, 0 replies; 6+ messages in thread
From: Stephen Hemminger @ 2021-01-14 5:50 UTC (permalink / raw)
To: Igor Ryzhov
Cc: Thomas Monjalon, dev, Ferruh Yigit, kevin.traynor, Richardson,
Bruce, hemant.agrawal, Jerin Jacob, Olivier Matz,
maxime.coquelin
Controlling DPDK interfaces with KNI is known to be broken. Kni calls
usepspace with network mutex held. The problem is not fixable.
The kernel does provide netlink and devlink API's but there is no generic
solution with DPDK.
On Wed, Jan 13, 2021, 11:54 AM Igor Ryzhov <iryzhov@nfware.com> wrote:
>
>
> On Wed, Jan 13, 2021 at 9:45 PM Thomas Monjalon <thomas@monjalon.net>
> wrote:
>
>> 13/01/2021 19:17, Igor Ryzhov:
>> > On Wed, Jan 13, 2021 at 8:10 PM Stephen Hemminger <
>> > stephen@networkplumber.org> wrote:
>> > > On Wed, Jan 13, 2021, 9:06 AM Thomas Monjalon <thomas@monjalon.net>
>> wrote:
>> > >
>> > > > Hi,
>> > > >
>> > > > As discussed today in the techboard meeting, KNI has probably
>> > > > better alternatives today without using an out-of-tree module.
>> > > > Virtio-user is a good candidate to replace KNI.
>> > > > What is the performance of TAP?
>> > > > Is there a way to leverage af_packet, af_xdp, or even pcap
>> interfaces?
>> > >
>> > > Last time I tried.
>> > > Virtio user was as fast as KNI and consumed less cpu. Was seeing
>> 10mpps
>> > > Tap was much slower. Like 1mpps.
>> >
>> > Performance is not the only question. The advantage of KNI we are
>> currently
>> > using is
>> > the ability to control the DPDK interfaces by the kernel.
>> > For example, to implement bonding in the DPDK application, it is
>> possible
>> > to create KNI
>> > pair for each physical interface, create a bond interface in Linux over
>> > those KNI interfaces
>> > and just pass LACP packets between the app and the kernel. The kernel
>> > itself will control
>> > MACs, MTU, etc. of underlying interfaces. AFAIK it's not possible with
>> > virtio-user or tap.
>> > Am I wrong?
>>
>> I see at least 2 alternatives for bonding with kernel management:
>> - mlx5 bonding
>> - af_xdp interface for most of NICs
>>
>
> Bonding is just an example. My point is that KNI allows changing MAC, MTU,
> rx mode and admin status of a NIC used by DPDK application - all using
> standard Linux utilities without any need for additional APIs provided by
> the app.
>
> Yes, Mellanox NICs also allow this but it's just one of many drivers
> supported by DPDK.
> I'll dig into af_xdp, thanks.
>
^ permalink raw reply [flat|nested] 6+ messages in thread
end of thread, other threads:[~2021-01-14 7:53 UTC | newest]
Thread overview: 6+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2021-01-13 17:06 [dpdk-dev] KNI alternatives Thomas Monjalon
2021-01-13 17:10 ` Stephen Hemminger
2021-01-13 18:17 ` Igor Ryzhov
2021-01-13 18:45 ` Thomas Monjalon
2021-01-13 19:53 ` Igor Ryzhov
2021-01-14 5:50 ` Stephen Hemminger
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).