[dpdk-dev] Performance issue with uio_pci_generic driver

[dpdk-dev] Performance issue with uio_pci_generic driver

[Simon Jouet]

Hi everyone,

First off I would like to thanks tmonjalo, Harry Van Harren and Bruce Richardson for the input they gave while I was trying to figure out the issue and pushing me to report the problem here 😊

Okay, so I was trying out some basic sanity benchmarks with DPDK before doing anything more complicated and surprisingly I was getting lower than gigabit speed for minimum packet size running l2fwd (or l3fwd for that matter).

The setup is very simple I’ve got two machine with Intel x710 quad port NICs one is running DPDK l2fwd and the other is running MoonGen for the performance benchmark.

After much debugging and trying to modify parameters one by one, giving up after nothing worked and setting up ovs-dpdk I noticed from the ovs documentation that the kernel module to load were uio and igb_uio while I was previous loading uio_pci_generic as mentioned in the DPDK getting started guide. I simply changed the kernel module and l2fwd went from 700Mbps to 10G line-rate.  Bruce said that shouldn’t be the case and the performance should be similar regardless of the driver loaded ...

Here is the full log of the experiment, if you’re interested:
https://gist.github.com/simon-jouet/178e1d302afef5c6a642

Best regards,
Simon

Re: [dpdk-dev] Performance issue with uio_pci_generic driver

[Bruce Richardson]

On Wed, Mar 16, 2016 at 11:59:34PM +0000, Simon Jouet wrote:
> Hi everyone,
> 
> First off I would like to thanks tmonjalo, Harry Van Harren and Bruce Richardson for the input they gave while I was trying to figure out the issue and pushing me to report the problem here 😊
> 
> Okay, so I was trying out some basic sanity benchmarks with DPDK before doing anything more complicated and surprisingly I was getting lower than gigabit speed for minimum packet size running l2fwd (or l3fwd for that matter).
> 
> The setup is very simple I’ve got two machine with Intel x710 quad port NICs one is running DPDK l2fwd and the other is running MoonGen for the performance benchmark.
> 
> After much debugging and trying to modify parameters one by one, giving up after nothing worked and setting up ovs-dpdk I noticed from the ovs documentation that the kernel module to load were uio and igb_uio while I was previous loading uio_pci_generic as mentioned in the DPDK getting started guide. I simply changed the kernel module and l2fwd went from 700Mbps to 10G line-rate.  Bruce said that shouldn’t be the case and the performance should be similar regardless of the driver loaded ...
> 
> Here is the full log of the experiment, if you’re interested:
> https://gist.github.com/simon-jouet/178e1d302afef5c6a642
> 
> Best regards,
> Simon
> 
Thanks Simon for reporting this here!
It seems very strange to me that the particular module being used for uio has
such an impact on performance. Has anyone else seen anything similar?

/Bruce

Re: [dpdk-dev] Performance issue with uio_pci_generic driver

[Simon Jouet]

If it’s of any use for debugging I would be willing to setup teamviewer or something like that for a person to remote desktop in a VM and have an SSH access to both servers (all of this is being firewalls that are out of my control so I can’t give direct SSH access). The two servers are for experimentation purposes only so there is nothing critical running.



From: Bruce Richardson<mailto:bruce.richardson@intel.com>
Sent: 18 March 2016 03:11
To: Simon Jouet<mailto:Simon.Jouet@glasgow.ac.uk>
Cc: dev@dpdk.org<mailto:dev@dpdk.org>
Subject: Re: [dpdk-dev] Performance issue with uio_pci_generic driver



On Wed, Mar 16, 2016 at 11:59:34PM +0000, Simon Jouet wrote:
> Hi everyone,
>
> First off I would like to thanks tmonjalo, Harry Van Harren and Bruce Richardson for the input they gave while I was trying to figure out the issue and pushing me to report the problem here 😊
>
> Okay, so I was trying out some basic sanity benchmarks with DPDK before doing anything more complicated and surprisingly I was getting lower than gigabit speed for minimum packet size running l2fwd (or l3fwd for that matter).
>
> The setup is very simple I’ve got two machine with Intel x710 quad port NICs one is running DPDK l2fwd and the other is running MoonGen for the performance benchmark.
>
> After much debugging and trying to modify parameters one by one, giving up after nothing worked and setting up ovs-dpdk I noticed from the ovs documentation that the kernel module to load were uio and igb_uio while I was previous loading uio_pci_generic as mentioned in the DPDK getting started guide. I simply changed the kernel module and l2fwd went from 700Mbps to 10G line-rate.  Bruce said that shouldn’t be the case and the performance should be similar regardless of the driver loaded ...
>
> Here is the full log of the experiment, if you’re interested:
> https://gist.github.com/simon-jouet/178e1d302afef5c6a642
>
> Best regards,
> Simon
>
Thanks Simon for reporting this here!
It seems very strange to me that the particular module being used for uio has
such an impact on performance. Has anyone else seen anything similar?

/Bruce