[dpdk-dev] FW: OVS with DPDK Meetup notes

[dpdk-dev] FW: OVS with DPDK Meetup notes

[Mcnamara, John]

Cross posting this from users@dpdk.org to get great visibility.


> -----Original Message-----
> From: dev [mailto:dev-bounces@openvswitch.org] On Behalf Of Traynor, Kevin
> Sent: Thursday, November 26, 2015 5:56 PM
> To: dev@openvswitch.org
> Cc: users@dpdk.org
> Subject: [ovs-dev] OVS with DPDK Meetup notes


 
Hi All,

Just wanted to post some summary notes on the recent OVS with DPDK Meetup we had after the OVS conference. Thanks to everyone for the often lively discussion. 
I've collated and condensed Maryam's notes (Thank you Maryam) with my own. 
Corrections and additions are welcome.

Usability
==========
* Command line options
  - Create defaults which can be overridden on the command line.
  -c 0x1
  -n 4 (this will be in DPDK 2.2 also)
  -socket-mem 1024,0 

* igb / vfio / uio
  - Update documentation to explain these better.

* 1GB v 2MB Hugepages
  - Test to see if significant performance difference between them and if not,
    update documentation to preference 2 MB pages. 

* Assignment of queues to cores/PMD threads
  - Would like to allow for better load balancing of traffic across cores.
  - Add appctl command to get the current queue/core assignments (currently 
    it's only in logs).
  - Discussion about relative merits of trying to do something better than 
    current scheme automatically or allowing a user interface to configure 
    manually. General consensus seemed to be that while it might be difficult 
    to do well automatically, it would be good to investigate and exhaust that 
    possibility before adding a manual user config.
  - Possibility of using DPDK light weight threads discussed.

* Debugging
  - A tcpdump type solution would be nice. Follow DPDK and leverage.

* More flexible port names / auto-discovery of ports
  - Follow DPDK and leverage.

* Single binary for OVS/OVS with DPDK and static vs. dynamic linking
  - Discussion around deployment and what the best model is.
  - Flavio has posted a mail on this
   http://openvswitch.org/pipermail/dev/2015-November/062599.html

* Documentation
  - Make INSTALL.DPDK.md guide cover the most common use cases only, and split 
    the other stuff into a less common/advanced guide. 

Testing
=======
* General agreement in the room that VSPERF pushing a nightly test report for
  OVS/OVS with DPDK to the ovs-build ML would be useful.

* Static Analysis
  - Is there an open source static analysis tool that the community could all use?

Features
========
* Multiqueue vhost-user
  - Looks really promising - will help us scale out performance to the VM. 

* vhost-cuse
  - Still used by some. Give notice to deprecate as there's a maintenance cost. 
    2 releases was mentioned, but I'm not sure that's clear to a user without a 
    regular release cadence. 

* dpdkr/ivshmem
  - Still useful. Check/Update documentation to ensure limitations are clear.


Performance
===========
* Some new optimizations coming in DPDK for vhost performance that we can
  leverage.
* Scalability
  - Performance is scaling well across cores (See Gerald Roger's OVS conf 
    presentation for reference)


At that point everyone was "conferenced out", but tentative plan to arrange another meetup in ~6 months' time if there's enough discussion items to warrant it.

Thanks,
Kevin.

Re: [dpdk-dev] [ovs-dev] OVS with DPDK Meetup notes

[Traynor, Kevin]

> -----Original Message-----
> From: Flavio Leitner [mailto:fbl@sysclose.org]
> Sent: Monday, November 30, 2015 11:51 PM
> To: Traynor, Kevin
> Cc: dev@openvswitch.org; users@dpdk.org
> Subject: Re: [ovs-dev] OVS with DPDK Meetup notes
> 
> On Thu, Nov 26, 2015 at 05:56:08PM +0000, Traynor, Kevin wrote:
> > Hi All,
> >
> > Just wanted to post some summary notes on the recent OVS with DPDK Meetup
> we
> > had after the OVS conference. Thanks to everyone for the often lively
> discussion.
> > I've collated and condensed Maryam's notes (Thank you Maryam) with my own.
> > Corrections and additions are welcome.
> 
> Thanks for having organized the event and for the good notes.
> 
> 
> > Usability
> > ==========
> > * Single binary for OVS/OVS with DPDK and static vs. dynamic linking
> >   - Discussion around deployment and what the best model is.
> >   - Flavio has posted a mail on this
> >    http://openvswitch.org/pipermail/dev/2015-November/062599.html
> 
> Let us know if you find a performance difference between static vs
> dynamic linking.  We might be able to accommodate both options in
> the same spec, but it seems we should go with shared linking only
> to keep it simple for now.
> 

Yes, will do. I seem to recall from when we looked at this on a previous
project it was a few hundred kpps but it was a long time ago, so I'm not
certain how many.

> 
> > Features
> > ========
> > * Multiqueue vhost-user
> >   - Looks really promising - will help us scale out performance to the VM.
> 
> I see that vhost PMD is moving and if it gets accepted, it would
> be a nice clean up for OVS.  Do you know if there is someone working
> on this already?

I agree, it should simplify the code a lot. Ciara reviewed it and did a
quick integration to see if the api would work. The patch was churning quite
a bit, so we decided to hold off doing any more work with it for the time being.

> 
> > * dpdkr/ivshmem
> >   - Still useful. Check/Update documentation to ensure limitations are
> clear.
> 
> Yeah, same thing here.
> 
> Thanks,
> fbl

Re: [dpdk-dev] [ovs-dev] OVS with DPDK Meetup notes

[Loftus, Ciara]

> >
> > On Thu, Nov 26, 2015 at 05:56:08PM +0000, Traynor, Kevin wrote:
> > > Hi All,
> > >
> > > Just wanted to post some summary notes on the recent OVS with DPDK
> Meetup
> > we
> > > had after the OVS conference. Thanks to everyone for the often lively
> > discussion.
> > > I've collated and condensed Maryam's notes (Thank you Maryam) with
> my own.
> > > Corrections and additions are welcome.
> >
> > Thanks for having organized the event and for the good notes.
> >
> >
> > > Usability
> > > ==========
> > > * Single binary for OVS/OVS with DPDK and static vs. dynamic linking
> > >   - Discussion around deployment and what the best model is.
> > >   - Flavio has posted a mail on this
> > >    http://openvswitch.org/pipermail/dev/2015-November/062599.html
> >
> > Let us know if you find a performance difference between static vs
> > dynamic linking.  We might be able to accommodate both options in
> > the same spec, but it seems we should go with shared linking only
> > to keep it simple for now.
> >
> 
> Yes, will do. I seem to recall from when we looked at this on a previous
> project it was a few hundred kpps but it was a long time ago, so I'm not
> certain how many.
> 
> >
> > > Features
> > > ========
> > > * Multiqueue vhost-user
> > >   - Looks really promising - will help us scale out performance to the VM.
> >
> > I see that vhost PMD is moving and if it gets accepted, it would
> > be a nice clean up for OVS.  Do you know if there is someone working
> > on this already?
> 
> I agree, it should simplify the code a lot. Ciara reviewed it and did a
> quick integration to see if the api would work. The patch was churning quite
> a bit, so we decided to hold off doing any more work with it for the time
> being.

Correct, the vHost PMD really cleans things up and removes the need for a lot of code in netdev-dpdk. The netdev_class for phy ports and vhost-user ports could be pretty much the same, except for the construct functions.

> 
> >
> > > * dpdkr/ivshmem
> > >   - Still useful. Check/Update documentation to ensure limitations are
> > clear.
> >
> > Yeah, same thing here.
> >
> > Thanks,
> > fbl

Re: [dpdk-dev] [ovs-dev] OVS with DPDK Meetup notes

[Gray, Mark D]

> >
> > I agree, it should simplify the code a lot. Ciara reviewed it and did
> > a quick integration to see if the api would work. The patch was
> > churning quite a bit, so we decided to hold off doing any more work
> > with it for the time being.
> 
> Correct, the vHost PMD really cleans things up and removes the need for a
> lot of code in netdev-dpdk. The netdev_class for phy ports and vhost-user
> ports could be pretty much the same, except for the construct functions.
> 
There seems to be a general direction in DPDK to move everything under a PMD
if we make the main interface into DPDK a pmd, we can simplify the OVS code base
and make it easier to add additional port-types in the future without
modifying OVS code.