Re: [dpdk-dev] Beyond DPDK 2.0

[Neil Horman <nhorman@tuxdriver.com>]

On Fri, May 08, 2015 at 12:26:39PM +0200, Hobywan Kenoby wrote:
> 
> > Sounds like you want something like libc, but DPDK is a system like a user
> > space OS more then it is a collection of functions that are independent
> > like strlen, strcpy, memcpy, printf or ... Some parts of DPDK are
> > independent and can be used as you suggest, but the real performance
> > sections are tied together.
> > 
> > >> Regards,
> > >> ++Keith
> 
> This is indeed quite a statement. DPDK is not just a
> bunch of NIC drivers, but "a user space OS" (DPDK 1.0 had a baremetal
> boot: why did it disappeared?).
> 
>  
> 
> Why Linux or Windows do not integrate DPDK concepts to
> catch up performance wise? Is it something so deep like the "Big
> Kernel Lock" that took so many years to get rid of?
> 
Some optimizations are being looked at in the kernel (more deeply ingrained use
of accelerators/offloads like cam management/flow steering, checksum & encap
offloads, tx batching, etc)  Those are features which the kernel can
opportunistically take advantage of in a hardware agnostic fashion.

Some optimizations simply aren't worth the effort to take into a general purpose
OS that seeks to support multiple arches.  Many of the DPDK optimizations
utilize instruction families like AVX or SSE, which, while potentially useful in
some situations have equal potential to be catastrophic to non network-i/o bound
workloads. 
>  
> 
> My assumption is that all current kernels have been
> built with one implicit hypothesis: the memory is much faster than cpu. This is
Thats not entirely true.  Or more to the point, its not true in any way thats
relevant to a comparison between DPDK and the Linux network stack.  Linux is as
careful with its cache management as DPDK is (arguably more so, as it has to
juggle multiple workloads instead of the single purpose workload that DPDK is
designed for).  The difference is that Linux often has to ignore some
performance improvements because it has the additional responsibiilty of
providing secruity and isolation to multiple processes on multiple
architectures.

> the opposite today. DPDK internal structure has been adapted to the new
> paradigm where the TLBs, the memory bandwidth are the scarce resources to manage. So I
> guess Linux and Windows will not be able to integrate DPDK concepts for
> performance anytime soon, if ever...
> 
This is the case with every bit of software.  Memory bandwidth is always a
scarce resoruce to manage.  The difference is that general purpose operating
systems consider protection/layering/isolation to be of equal or greater
importance than performance.  Tradeoffs have to be made.  Linux in general
strives to isolate hardware from applications both functionally and physically
so as to ensure that there is minimal risk in one process adversely affecting
the other.  The tradeoff is that the Linux device model can't just do anything
it wants to improve performance.

Converserly, DPDK is all about performance.  Up until recently (and likely still
somewhat in the future), you have to rebuild your application every time you
move to a new version of DPDK, because the API fluctuated with every release to
eek out additional performance.  The DPDK can optimize using vectorized x86
instructions and other cpu specific features througout because it is in the
position to only worry about a very narrow field of architectures.

>  
> 
> Reading the list carefully, I expect disk block PMDs
> (and block framework?) to come next.
> 
>  
> 
> Beyond DPDK 2.0: is it time to accept the fact that
> DPDK community is actually paving the way to the next generation lightweight,
> high performance, para-virtualized OS? Is it a DPDK task? Another project ?
> Should we rename DPDK to PVDK?
> 
>  
> 
> - HK
> 
>