From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mo1.mail-out.ovh.net (4.mo1.mail-out.ovh.net [46.105.76.26]) by dpdk.org (Postfix) with ESMTP id 714E56938 for ; Wed, 12 Feb 2014 15:13:50 +0100 (CET) Received: from mail406.ha.ovh.net (b9.ovh.net [213.186.33.59]) by mo1.mail-out.ovh.net (Postfix) with SMTP id 151B5100822F for ; Wed, 12 Feb 2014 15:22:09 +0100 (CET) Received: from b0.ovh.net (HELO queueout) (213.186.33.50) by b0.ovh.net with SMTP; 12 Feb 2014 16:20:05 +0200 Received: from lneuilly-152-23-9-75.w193-252.abo.wanadoo.fr (HELO pcdeff) (ff@ozog.com@193.252.40.75) by ns0.ovh.net with SMTP; 12 Feb 2014 16:20:02 +0200 From: =?iso-8859-1?Q?Fran=E7ois-Fr=E9d=E9ric_Ozog?= To: "'Prashant Upadhyaya'" References: <026601cf27e8$49bf1830$dd3d4890$@gmail.com> In-Reply-To: Date: Wed, 12 Feb 2014 15:11:47 +0100 Message-ID: <000001cf27fc$5e98fc80$1bcaf580$@com> MIME-Version: 1.0 Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: quoted-printable X-Mailer: Microsoft Office Outlook 12.0 Thread-Index: AQLuYlzM410cRTj90mDif8LTqIfiZJhzClBQgAADIUCAACXhsA== Content-Language: fr X-Ovh-Tracer-Id: 9444611370659600601 X-Ovh-Remote: 193.252.40.75 (lneuilly-152-23-9-75.w193-252.abo.wanadoo.fr) X-Ovh-Local: 213.186.33.20 (ns0.ovh.net) X-OVH-SPAMSTATE: OK X-OVH-SPAMSCORE: -100 X-OVH-SPAMCAUSE: gggruggvucftvghtrhhoucdtuddrfeejtddrjedvucetufdoteggodetrfcurfhrohhfihhlvgemucfqggfjnecuuegrihhlohhuthemuceftddtnecusecvtfgvtghiphhivghnthhsucdlqddutddtmd X-Spam-Check: DONE|U 0.5/N X-VR-SPAMSTATE: OK X-VR-SPAMSCORE: -100 X-VR-SPAMCAUSE: gggruggvucftvghtrhhoucdtuddrfeejtddrkeehucetufdoteggodetrfcurfhrohhfihhlvgemucfqggfjnecuuegrihhlohhuthemuceftddtnecusecvtfgvtghiphhivghnthhsucdlqddutddtmd Cc: dev@dpdk.org Subject: Re: [dpdk-dev] NUMA CPU Sockets and DPDK X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: patches and discussions about DPDK List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 12 Feb 2014 14:13:50 -0000 Hi Prashant, May be you could monitor RAM, QPI and PCIe activity with http://software.intel.com/en-us/articles/intel-performance-counter-monito= r-a -better-way-to-measure-cpu-utilization It may ease investigating the issue. Fran=E7ois-Fr=E9d=E9ric =20 > -----Message d'origine----- > De=A0: dev [mailto:dev-bounces@dpdk.org] De la part de Prashant = Upadhyaya > Envoy=E9=A0: mercredi 12 f=E9vrier 2014 13:03 > =C0=A0: Etai Lev Ran > Cc=A0: dev@dpdk.org > Objet=A0: Re: [dpdk-dev] NUMA CPU Sockets and DPDK >=20 > Hi Etai, >=20 > Ofcourse all DPDK threads consume 100 % (unless some waits are = introduced > for some power saving etc., all typical DPDK threads are while(1) = loops) > When I said core 1 is unusually busy, I meant to say that it is not = able to > read beyond 2 Gbps or so and the packets are dropping at NIC. > (I have my own custom way of calculating the cpu utilization of core 1 > based on how many empty polls were done and how many polls got me data > which I then process) On the 8 core machine with single socket, the = core 1 > was being able to lift successfully much higher data rates, hence the > question. >=20 > Regards > -Prashant >=20 >=20 > -----Original Message----- > From: Etai Lev Ran [mailto:elevran@gmail.com] > Sent: Wednesday, February 12, 2014 5:18 PM > To: Prashant Upadhyaya > Cc: dev@dpdk.org > Subject: RE: [dpdk-dev] NUMA CPU Sockets and DPDK >=20 > Hi Prashant, >=20 > Based on our experience, using DPDK cross CPU sockets may indeed = result in > some performance degradation (~10% for our application vs. staying in > socket. YMMV based on HW, application structure, etc.). >=20 > Regarding CPU utilization on core 1, the one picking up traffic: = perhaps I > had misunderstood your comment, but I would expect it to always be = close to > 100% since it's polling the device via the PMD and not driven by > interrupts. >=20 > Regards, > Etai >=20 > -----Original Message----- > From: dev [mailto:dev-bounces@dpdk.org] On Behalf Of Prashant = Upadhyaya > Sent: Wednesday, February 12, 2014 1:28 PM > To: dev@dpdk.org > Subject: [dpdk-dev] NUMA CPU Sockets and DPDK >=20 > Hi guys, >=20 > What has been your experience of using DPDK based app's in NUMA mode = with > multiple sockets where some cores are present on one socket and other cores > on some other socket. >=20 > I am migrating my application from one intel machine with 8 cores, all = in > one socket to a 32 core machine where 16 cores are in one socket and = 16 > other cores in the second socket. > My core 0 does all initialization for mbuf's, nic ports, queues etc. = and > uses SOCKET_ID_ANY for socket related parameters. >=20 > The usecase works, but I think I am running into performance issues on = the > 32 core machine. > The lscpu output on my 32 core machine shows the following - NUMA = node0 > CPU(s): 0,2,4,6,8,10,12,14,16,18,20,22,24,26,28,30 > NUMA node1 CPU(s): 1,3,5,7,9,11,13,15,17,19,21,23,25,27,29,31 > I am using core 1 to lift all the data from a single queue of an = 82599EB > port and I see that the cpu utilization for this core 1 is way too = high > even for lifting traffic of 1 Gbps with packet size of 650 bytes. >=20 > In general, does one need to be careful in working with multiple = sockets > and so forth, any comments would be helpful. >=20 > Regards > -Prashant >=20 >=20 >=20 >=20 >=20 > =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= > =3D > =3D=3D=3D > Please refer to http://www.aricent.com/legal/email_disclaimer.html > for important disclosures regarding this electronic communication. > =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= > =3D > =3D=3D=3D >=20 >=20 >=20 >=20 >=20 > =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= > =3D=3D=3D=3D > Please refer to http://www.aricent.com/legal/email_disclaimer.html > for important disclosures regarding this electronic communication. > =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= > =3D=3D=3D=3D