On May 24, 2013, at 7:41 AM, Thomas Monjalon <thomas.monjalon@6wind.com> wrote:

> Hello,
> 
> 24/05/2013 16:11, Patrick Mahan :
>> Intel Xeon E5-2690 (8 physical, 16 virtual)
> 
> How many CPU sockets have you ?

This is a Dell PowerEdge T620, it has two sockets, but only one has a CPU in it.

> 
>> 64 Gbyte DDR3 memory
>> Intel 82599EB-SPF dual port 10GE interface
>> CentOS 6.4 (2.6.32-358.6.1.el6.x86_64)
>> The 82599 is in a 16x PCI-e slot.
> 
> Check the datasheet of your motherboard.
> Are you sure it is wired as a 16x PCI-e ?

As far as I can tell from the specs on the Dell site - www.dell.com/us/business/p/poweredge-t620/pd

> Is it connected to the right NUMA node ?

Yes, it's in slot labeled: PCIE_G3_x16 (cpu1).  The interfaces show up as p2p1 and p2p2.

> 
>> I have it attached to an IXIA box.  I have been running the app 'testpmd'
>> in iofwd mode with 2K rx/tx descriptors and 512 burst/mbcache.  I have been
>> varying the # of queues and unfortunately, I am not seeing full line rate.
> 
> What is your command line ?
> 

sudo build/app/testpmd -b 0000:03:00.0 -b 0000:03:00.1 -c<coremask> -n3 -- --nb-cores=<ncores> --nb-ports=2 --rxd=2048 --rxd=2048 --mbcache=512 --burst=512 --rxd=<nqueues> --txq=<nqueues>

Where I am using the following to determine cores, coremask and nqueues:

    ncores = nqueues * 2 // this actually the number of ports being tested
    coremask = (1 << (ncores + 1)) - 1

So for say, 3 rx/tx queues -

    ncores = 3 * 2 = 6
    coremask = (1 << (6 + 1)) - 1 = 127 (0x7f)

Now that I remember it I had to fix testpmd to allocate enough mbufs.

>> I am seeing about 20-24% droppage on the receive side.  It doesn't seem to
>> matter the # of queues.
> 
> If queues are polled by different cores, it should matter.

I assume you mean different physical cores, yes?

There is only one physical core, but each 'forwarding' thread is on a separate core.

> 
>> Question 1: Is 'testpmd' the best application for this type of testing?  If
>> not, which program?  Or do I need to roll my own?
> 
> testpmd is the right application for performance benchmark.
> It is also possible to use examples l2fwd/l3fwd but you should keep testpmd.

I am just starting with testpmd to get a feel for raw throughput.  I want to test l2 and l3 soon but I may loose access to the Ixia.

> 
>> Question 2: I have blacklisted the Intel i350 ports on the motherboard and
>> am using ssh to access the platform.  Could this be affecting the test?
> 
> You mean i350 is used for ssh ? It shouldn't significantly affect your test.

Okay, I noticed that they scanned by the pci dpdk layer.

Thanks,

Patrick

> 
> -- 
> Thomas