Re: [dpdk-dev] DPDK Features for Q1 2015

Re: [dpdk-dev] DPDK Features for Q1 2015

[Zhu, Heqing]

> -----Original Message-----
> From: dev [mailto:dev-bounces@dpdk.org] On Behalf Of Liang, Cunming
> Sent: Wednesday, October 22, 2014 8:06 AM
> To: Zhou, Danny; Thomas Monjalon; O'driscoll, Tim
> Cc: dev@dpdk.org; Fastabend, John R; Ronciak, John
> Subject: Re: [dpdk-dev] DPDK Features for Q1 2015
> 
> > >
> > > This design allows to keep the configuration code in one place: the
> kernel.
> > > In the meantime, we are trying to add a lot of code to configure the
> > > NICs, which looks to be a duplication of effort.
> > > Why should we have two ways of configuring e.g. flow director?

[heqing] There will be multiple choices for DPDK usage model if/after this feature is available, 
the customer can choose the DPDK with or without the bifurcated driver. 

> [Liang, Cunming] The HW sometimes provides additional ability than existing
> abstraction API.
> On that time(HW ability is a superset to the abstraction wrapper, e.g. flow
> director), we need to provide another choice.
> Ethtools is good, but can't apply anything supported in NIC.
> Bifurcated driver considers a lot on reusing existing rx/tx routine.
> We'll send RFC patch soon if kernel patch moving fast.
> 
> > -----Original Message-----
> > From: dev [mailto:dev-bounces@dpdk.org] On Behalf Of Zhou, Danny
> > Sent: Wednesday, October 22, 2014 10:44 PM
> > To: Thomas Monjalon; O'driscoll, Tim
> > Cc: dev@dpdk.org; Fastabend, John R; Ronciak, John
> > Subject: Re: [dpdk-dev] DPDK Features for Q1 2015
> >
> > Thomas,
> >
> > In terms of the bifurcated driver, it is actually the same thing.
> > Specifically, the bifurcated driver PMD in DPDK depends on kernel
> > code(af_packet and 10G/40G NIC) changes. Once the kernel patches are
> > upstreamed, the corresponding DPDK PMDs patches will be submitted to
> > dpdk.org. John Fastabend and John Ronciak are working with very
> > closely to achieve the same goal.
> >
> > -Danny
> >
> > > -----Original Message-----
> > > From: dev [mailto:dev-bounces@dpdk.org] On Behalf Of Thomas
> Monjalon
> > > Sent: Wednesday, October 22, 2014 10:21 PM
> > > To: O'driscoll, Tim
> > > Cc: dev@dpdk.org; Fastabend, John R; Ronciak, John
> > > Subject: Re: [dpdk-dev] DPDK Features for Q1 2015
> > >
> > > Thanks Tim for sharing your plan.
> > > It's really helpful to improve community collaboration.
> > >
> > > I'm sure it's going to generate some interesting discussions.
> > > Please take care to discuss such announce on dev list only.
> > > The announce@dpdk.org list is moderated to keep a low traffic.
> > >
> > > I would like to open discussion about a really important feature,
> > > showed last week by John Fastabend and John Ronciak during LinuxCon:
> > >
> > > > Bifurcated Driver: With the Bifurcated Driver, the kernel will
> > > > retain direct control of the NIC, and will assign specific queue pairs to
> DPDK.
> > > > Configuration of the NIC is controlled by the kernel via ethtool.
> > >
> > > This design allows to keep the configuration code in one place: the
> kernel.
> > > In the meantime, we are trying to add a lot of code to configure the
> > > NICs, which looks to be a duplication of effort.
> > > Why should we have two ways of configuring e.g. flow director?
> > >
> > > Since you at Intel, you'll be supporting your code, I am fine for
> > > duplication, but I feel it's worth arguing why both should be available
> instead of one.
> > >
> > > --
> > > Thomas

[dpdk-dev] [dpdk-announce] DPDK Features for Q1 2015

[O'driscoll, Tim]

We're starting to plan our DPDK features for next year. We're planning to have a DPDK 2.0 release at the end of March, and we'd like to inform the community of the features that we hope to submit to that release. The current list of features, along with brief descriptions, is included below.

There will naturally be some changes to this list as work on these features progresses. Some will inevitably turn out to be bigger than anticipated and will need to be deferred to a later date, while other priorities will arise and need to be accommodated. So, this list will be subject to change, and should be taken as guidance on what we hope to submit, not a commitment.

Our aim in providing this information now is to solicit input from the community. We'd like to make sure we avoid duplication or conflicts with work that others are planning, so we'd be interested in hearing any plans that others in the community have for contributions to DPDK in this timeframe. This will allow us to build a complete picture and ensure we avoid duplication of effort. 

I'm sure people will have questions, and will be looking for more information on these features. Further details will be provided by the individual developers over the next few months. We aim to make better use of the RFC process in this release, and provide early outlines of the features so that we can obtain community feedback as soon as possible.

We also said at the recent DPDK Summit that we would investigate holding regular community conference calls. We'll be scheduling the first of these calls soon, and will use this to discuss the 2.0 (Q1 2015) features in a bit more detail.


2.0 (Q1 2015) DPDK Features:
Bifurcated Driver: With the Bifurcated Driver, the kernel will retain direct control of the NIC, and will assign specific queue pairs to DPDK. Configuration of the NIC is controlled by the kernel via ethtool.

Support the new Intel SoC platform, along with the embedded 10GbE NIC.

Packet Reordering: Assign a sequence number to packets on Rx, and then provide the ability to reorder on Tx to preserve the original order.

Packet Distributor (phase 2): Implement the following enhancements to the Packet Distributor that was originally delivered in the DPDK 1.7 release: performance improvements; the ability for packets from a flow to be processed by multiple worker cores in parallel and then reordered on Tx using the Packet Reordering feature; the ability to have multiple Distributors which share Worker cores.

Support Multiple Threads per Core: Use Linux cgroups to allow multiple threads to run on a single core. This would be useful in situations where a DPDK thread does not require the full resources of a core.

Support the Fedora 21 OS.

Support the host interface for Intel's next generation Ethernet switch. This only covers basic support for the host interface. Support for additional features will be added later.

Cuckoo Hash: A new hash algorithm was implemented as part of the Cuckoo Switch project (see http://www.cs.cmu.edu/~dongz/papers/cuckooswitch.pdf), and shows some promising performance results. This needs to be modified to make it more generic, and then incorporated into DPDK.

Provide DPDK support for uio_pci_generic.

Integrated Qemu Userspace vHost: Modify Userspace vHost to use Qemu version 2.1, and remove the need for the kernel module (cuse.ko).

PCI Hot Plug: When you migrate a VM, you need hot plug support as the new VF on the new hardware you are running on post-migration needs to be initialized. With an emulated NIC migration is seamless as all configuration for the NIC is within the RAM of the VM and the hypervisor. With a VF you have actual hardware in the picture which needs to be set up properly.

Additional XL710/X710 40-Gigabit Ethernet Controller Features:	Support for additional XL710/X710 40-Gigabit Ethernet Controller features, including bandwidth and QoS management, NVGRE and other network overlay support, TSO, IEEE1588, DCB support.  SR-IOV switching and port mirroring.

Single Virtio Driver: Merge existing Virtio drivers into a single implementation, incorporating the best features from each of the existing drivers.

X32 ABI: This is an application binary interface project for the Linux kernel that allows programs to take advantage of the benefits of x86-64 (larger number of CPU registers, better floating-point performance, faster position-independent code shared libraries, function parameters passed via registers, faster syscall instruction) while using 32-bit pointers and thus avoiding the overhead of 64-bit pointers.

AVX2 ACL: Modify ACL library to use AVX2 instructions to improve performance.

Interrupt mode for PMD: Allow DPDK process to transition to interrupt mode when load is low so that other processes can run, or else power can be saved. This will increase latency/jitter.

DPDK Headroom: Provide a mechanism to indicate how much headroom (spare capacity) exists in a DPDK process.


Thanks,
Tim

Re: [dpdk-dev] DPDK Features for Q1 2015

[Thomas Monjalon]

Thanks Tim for sharing your plan.
It's really helpful to improve community collaboration.

I'm sure it's going to generate some interesting discussions.
Please take care to discuss such announce on dev list only.
The announce@dpdk.org list is moderated to keep a low traffic.

I would like to open discussion about a really important feature,
showed last week by John Fastabend and John Ronciak during LinuxCon:

> Bifurcated Driver: With the Bifurcated Driver, the kernel will retain
> direct control of the NIC, and will assign specific queue pairs to DPDK.
> Configuration of the NIC is controlled by the kernel via ethtool.

This design allows to keep the configuration code in one place: the kernel.
In the meantime, we are trying to add a lot of code to configure the NICs,
which looks to be a duplication of effort.
Why should we have two ways of configuring e.g. flow director?

Since you at Intel, you'll be supporting your code, I am fine for duplication,
but I feel it's worth arguing why both should be available instead of one.

-- 
Thomas

Re: [dpdk-dev] DPDK Features for Q1 2015

[Zhou, Danny]

Thomas,

In terms of the bifurcated driver, it is actually the same thing. Specifically, the bifurcated 
driver PMD in DPDK depends on kernel code(af_packet and 10G/40G NIC) changes. Once the
kernel patches are upstreamed, the corresponding DPDK PMDs patches will be 
submitted to dpdk.org. John Fastabend and John Ronciak are working with very
closely to achieve the same goal.

-Danny

> -----Original Message-----
> From: dev [mailto:dev-bounces@dpdk.org] On Behalf Of Thomas Monjalon
> Sent: Wednesday, October 22, 2014 10:21 PM
> To: O'driscoll, Tim
> Cc: dev@dpdk.org; Fastabend, John R; Ronciak, John
> Subject: Re: [dpdk-dev] DPDK Features for Q1 2015
> 
> Thanks Tim for sharing your plan.
> It's really helpful to improve community collaboration.
> 
> I'm sure it's going to generate some interesting discussions.
> Please take care to discuss such announce on dev list only.
> The announce@dpdk.org list is moderated to keep a low traffic.
> 
> I would like to open discussion about a really important feature,
> showed last week by John Fastabend and John Ronciak during LinuxCon:
> 
> > Bifurcated Driver: With the Bifurcated Driver, the kernel will retain
> > direct control of the NIC, and will assign specific queue pairs to DPDK.
> > Configuration of the NIC is controlled by the kernel via ethtool.
> 
> This design allows to keep the configuration code in one place: the kernel.
> In the meantime, we are trying to add a lot of code to configure the NICs,
> which looks to be a duplication of effort.
> Why should we have two ways of configuring e.g. flow director?
> 
> Since you at Intel, you'll be supporting your code, I am fine for duplication,
> but I feel it's worth arguing why both should be available instead of one.
> 
> --
> Thomas

Re: [dpdk-dev] DPDK Features for Q1 2015

[Liang, Cunming]

> >
> > This design allows to keep the configuration code in one place: the kernel.
> > In the meantime, we are trying to add a lot of code to configure the NICs,
> > which looks to be a duplication of effort.
> > Why should we have two ways of configuring e.g. flow director?
[Liang, Cunming] The HW sometimes provides additional ability than existing abstraction API.
On that time(HW ability is a superset to the abstraction wrapper, e.g. flow director), we need to provide another choice.
Ethtools is good, but can't apply anything supported in NIC.
Bifurcated driver considers a lot on reusing existing rx/tx routine.
We'll send RFC patch soon if kernel patch moving fast.

> -----Original Message-----
> From: dev [mailto:dev-bounces@dpdk.org] On Behalf Of Zhou, Danny
> Sent: Wednesday, October 22, 2014 10:44 PM
> To: Thomas Monjalon; O'driscoll, Tim
> Cc: dev@dpdk.org; Fastabend, John R; Ronciak, John
> Subject: Re: [dpdk-dev] DPDK Features for Q1 2015
> 
> Thomas,
> 
> In terms of the bifurcated driver, it is actually the same thing. Specifically, the
> bifurcated
> driver PMD in DPDK depends on kernel code(af_packet and 10G/40G NIC)
> changes. Once the
> kernel patches are upstreamed, the corresponding DPDK PMDs patches will be
> submitted to dpdk.org. John Fastabend and John Ronciak are working with very
> closely to achieve the same goal.
> 
> -Danny
> 
> > -----Original Message-----
> > From: dev [mailto:dev-bounces@dpdk.org] On Behalf Of Thomas Monjalon
> > Sent: Wednesday, October 22, 2014 10:21 PM
> > To: O'driscoll, Tim
> > Cc: dev@dpdk.org; Fastabend, John R; Ronciak, John
> > Subject: Re: [dpdk-dev] DPDK Features for Q1 2015
> >
> > Thanks Tim for sharing your plan.
> > It's really helpful to improve community collaboration.
> >
> > I'm sure it's going to generate some interesting discussions.
> > Please take care to discuss such announce on dev list only.
> > The announce@dpdk.org list is moderated to keep a low traffic.
> >
> > I would like to open discussion about a really important feature,
> > showed last week by John Fastabend and John Ronciak during LinuxCon:
> >
> > > Bifurcated Driver: With the Bifurcated Driver, the kernel will retain
> > > direct control of the NIC, and will assign specific queue pairs to DPDK.
> > > Configuration of the NIC is controlled by the kernel via ethtool.
> >
> > This design allows to keep the configuration code in one place: the kernel.
> > In the meantime, we are trying to add a lot of code to configure the NICs,
> > which looks to be a duplication of effort.
> > Why should we have two ways of configuring e.g. flow director?
> >
> > Since you at Intel, you'll be supporting your code, I am fine for duplication,
> > but I feel it's worth arguing why both should be available instead of one.
> >
> > --
> > Thomas