From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from ns.mahan.org (unknown [67.116.10.138]) by dpdk.org (Postfix) with ESMTP id 7E3776931 for ; Fri, 24 May 2013 20:32:41 +0200 (CEST) Received: from [192.168.71.41] (localhost [127.0.0.1]) (authenticated bits=0) by ns.mahan.org (8.14.5/8.14.5) with ESMTP id r4OIWfdr013173; Fri, 24 May 2013 11:32:42 -0700 (PDT) (envelope-from mahan@mahan.org) References: <519F74F6.3000903@mahan.org> <201305241641.38896.thomas.monjalon@6wind.com> Mime-Version: 1.0 (1.0) In-Reply-To: <201305241641.38896.thomas.monjalon@6wind.com> Content-Type: multipart/alternative; boundary=Apple-Mail-FFA7C240-C261-4914-9791-71B6D72EC946 Content-Transfer-Encoding: 7bit Message-Id: X-Mailer: iPad Mail (10B329) From: Patrick Mahan Date: Fri, 24 May 2013 11:32:43 -0700 To: Thomas Monjalon Cc: "dev@dpdk.org" Subject: Re: [dpdk-dev] Best example for showing throughput? X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: patches and discussions about DPDK List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 24 May 2013 18:32:42 -0000 --Apple-Mail-FFA7C240-C261-4914-9791-71B6D72EC946 Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: quoted-printable On May 24, 2013, at 7:41 AM, Thomas Monjalon wro= te: > Hello, >=20 > 24/05/2013 16:11, Patrick Mahan : >> Intel Xeon E5-2690 (8 physical, 16 virtual) >=20 > How many CPU sockets have you ? This is a Dell PowerEdge T620, it has two sockets, but only one has a CPU in= it. >=20 >> 64 Gbyte DDR3 memory >> Intel 82599EB-SPF dual port 10GE interface >> CentOS 6.4 (2.6.32-358.6.1.el6.x86_64) >> The 82599 is in a 16x PCI-e slot. >=20 > Check the datasheet of your motherboard. > Are you sure it is wired as a 16x PCI-e ? As far as I can tell from the specs on the Dell site - www.dell.com/us/busin= ess/p/poweredge-t620/pd > Is it connected to the right NUMA node ? Yes, it's in slot labeled: PCIE_G3_x16 (cpu1). The interfaces show up as p2= p1 and p2p2. >=20 >> I have it attached to an IXIA box. I have been running the app 'testpmd'= >> in iofwd mode with 2K rx/tx descriptors and 512 burst/mbcache. I have be= en >> varying the # of queues and unfortunately, I am not seeing full line rate= . >=20 > What is your command line ? >=20 sudo build/app/testpmd -b 0000:03:00.0 -b 0000:03:00.1 -c -n3 -- -= -nb-cores=3D --nb-ports=3D2 --rxd=3D2048 --rxd=3D2048 --mbcache=3D51= 2 --burst=3D512 --rxd=3D --txq=3D Where I am using the following to determine cores, coremask and nqueues: ncores =3D nqueues * 2 // this actually the number of ports being tested= coremask =3D (1 << (ncores + 1)) - 1 So for say, 3 rx/tx queues - ncores =3D 3 * 2 =3D 6 coremask =3D (1 << (6 + 1)) - 1 =3D 127 (0x7f) Now that I remember it I had to fix testpmd to allocate enough mbufs. >> I am seeing about 20-24% droppage on the receive side. It doesn't seem t= o >> matter the # of queues. >=20 > If queues are polled by different cores, it should matter. I assume you mean different physical cores, yes? There is only one physical core, but each 'forwarding' thread is on a separa= te core. >=20 >> Question 1: Is 'testpmd' the best application for this type of testing? I= f >> not, which program? Or do I need to roll my own? >=20 > testpmd is the right application for performance benchmark. > It is also possible to use examples l2fwd/l3fwd but you should keep testpm= d. I am just starting with testpmd to get a feel for raw throughput. I want to= test l2 and l3 soon but I may loose access to the Ixia. >=20 >> Question 2: I have blacklisted the Intel i350 ports on the motherboard an= d >> am using ssh to access the platform. Could this be affecting the test? >=20 > You mean i350 is used for ssh ? It shouldn't significantly affect your tes= t. Okay, I noticed that they scanned by the pci dpdk layer. Thanks, Patrick >=20 > --=20 > Thomas --Apple-Mail-FFA7C240-C261-4914-9791-71B6D72EC946 Content-Type: text/html; charset=utf-8 Content-Transfer-Encoding: quoted-printable

On May 24, 20= 13, at 7:41 AM, Thomas Monjalon <thomas.monjalon@6wind.com> wrote:

Hello,
24/05/2013 16:11, Patrick Mahan :
Intel Xeon E5-2690 (8 physical, 16 virtual)<= br>

How many CPU sockets have you ?

=
This is a Dell PowerEd= ge T620, it has two sockets, but only one has a CPU in it.


64 Gbyte DDR3 memory
=
Intel 82599EB-SPF dual port 10G= E interface
CentOS 6.= 4 (2.6.32-358.6.1.el6.x86_64)
The 82599 is in a 16x PCI-e slot.

Check the datasheet of your motherboard.
Are yo= u sure it is wired as a 16x PCI-e ?

As far as I can tell from the specs on the Dell site - = ;www.dell.com/us/business/p/poweredge-= t620/pd

Is it connected to the right NUMA node ?

Yes, it's in slot labeled: PCIE_G3_x16 (cpu1). &nbs= p;The interfaces show up as p2p1 and p2p2.


I have it attached to an IXIA box.  I have b= een running the app 'testpmd'
in iofwd mode with 2K rx/tx descriptors and 512 burst/mbcache. &nbs= p;I have been
varying= the # of queues and unfortunately, I am not seeing full line rate.

What is your command line ?
=

sudo build/app/testpmd -b= 0000:03:00.0 -b 0000:03:00.1 -c<coremask> -n3 -- --nb-cores=3D<nco= res> --nb-ports=3D2 --rxd=3D2048 --rxd=3D2048 --mbcache=3D512 --burst=3D5= 12 --rxd=3D<nqueues> --txq=3D<nqueues>

= Where I am using the following to determine cores, coremask and nqueues:

    ncores =3D nqueues * 2 // this actually t= he number of ports being tested
    coremask =3D (1 <= < (ncores + 1)) - 1

So for say, 3 rx/tx queues -=

    ncores =3D 3 * 2 =3D 6
&nb= sp;   coremask =3D (1 << (6 + 1)) - 1 =3D 127 (0x7f)
Now that I remember it I had to fix testpmd to allocate enough m= bufs.

I am seeing abo= ut 20-24% droppage on the receive side.  It doesn't seem to
<= /blockquote>
matter the # of queues.

If queues are polled by different core= s, it should matter.

I assume yo= u mean different physical cores, yes?

There is only= one physical core, but each 'forwarding' thread is on a separate core.

Question 1: Is 'tes= tpmd' the best application for this type of testing?  If
not, which program?  Or do I n= eed to roll my own?

testpmd is= the right application for performance benchmark.
It is also= possible to use examples l2fwd/l3fwd but you should keep testpmd.

I am just starting with testpmd to get a f= eel for raw throughput.  I want to test l2 and l3 soon but I may loose a= ccess to the Ixia.


Question 2: I have blacklisted the Intel i350 ports on the moth= erboard and
am using s= sh to access the platform.  Could this be affecting the test?

You mean i350 is used for ssh ? It shou= ldn't significantly affect your test.

=
Okay, I noticed that they scanned by the pci dpdk layer.
Thanks,

Patrick


--
Thomas
= --Apple-Mail-FFA7C240-C261-4914-9791-71B6D72EC946--