From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from jaguar.aricent.com (jaguar.aricent.com [121.241.96.11]) by dpdk.org (Postfix) with ESMTP id 750676944 for ; Wed, 12 Feb 2014 13:02:02 +0100 (CET) Received: from jaguar.aricent.com (localhost [127.0.0.1]) by postfix.imss71 (Postfix) with ESMTP id 630F336B4C; Wed, 12 Feb 2014 17:33:05 +0530 (IST) Received: from GUREXHT02.ASIAN.AD.ARICENT.COM (gurexht02.asian.ad.aricent.com [10.203.171.138]) (using TLSv1 with cipher RC4-MD5 (128/128 bits)) (No client certificate requested) by jaguar.aricent.com (Postfix) with ESMTP id 4D1B836B65; Wed, 12 Feb 2014 17:33:05 +0530 (IST) Received: from GUREXMB01.asian.ad.aricent.com ([10.203.171.134]) by GUREXHT02.ASIAN.AD.ARICENT.COM ([10.203.171.138]) with mapi; Wed, 12 Feb 2014 17:33:05 +0530 From: Prashant Upadhyaya To: Etai Lev Ran Date: Wed, 12 Feb 2014 17:33:02 +0530 Thread-Topic: [dpdk-dev] NUMA CPU Sockets and DPDK Thread-Index: AQLuYlzM410cRTj90mDif8LTqIfiZJhzClBQgAADIUA= Message-ID: References: <026601cf27e8$49bf1830$dd3d4890$@gmail.com> In-Reply-To: <026601cf27e8$49bf1830$dd3d4890$@gmail.com> Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: acceptlanguage: en-US Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 X-TM-AS-MML: No Cc: "dev@dpdk.org" Subject: Re: [dpdk-dev] NUMA CPU Sockets and DPDK X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: patches and discussions about DPDK List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 12 Feb 2014 12:02:02 -0000 Hi Etai, Ofcourse all DPDK threads consume 100 % (unless some waits are introduced f= or some power saving etc., all typical DPDK threads are while(1) loops) When I said core 1 is unusually busy, I meant to say that it is not able to= read beyond 2 Gbps or so and the packets are dropping at NIC. (I have my own custom way of calculating the cpu utilization of core 1 base= d on how many empty polls were done and how many polls got me data which I = then process) On the 8 core machine with single socket, the core 1 was being able to lift= successfully much higher data rates, hence the question. Regards -Prashant -----Original Message----- From: Etai Lev Ran [mailto:elevran@gmail.com] Sent: Wednesday, February 12, 2014 5:18 PM To: Prashant Upadhyaya Cc: dev@dpdk.org Subject: RE: [dpdk-dev] NUMA CPU Sockets and DPDK Hi Prashant, Based on our experience, using DPDK cross CPU sockets may indeed result in = some performance degradation (~10% for our application vs. staying in socke= t. YMMV based on HW, application structure, etc.). Regarding CPU utilization on core 1, the one picking up traffic: perhaps I = had misunderstood your comment, but I would expect it to always be close to= 100% since it's polling the device via the PMD and not driven by interrup= ts. Regards, Etai -----Original Message----- From: dev [mailto:dev-bounces@dpdk.org] On Behalf Of Prashant Upadhyaya Sent: Wednesday, February 12, 2014 1:28 PM To: dev@dpdk.org Subject: [dpdk-dev] NUMA CPU Sockets and DPDK Hi guys, What has been your experience of using DPDK based app's in NUMA mode with m= ultiple sockets where some cores are present on one socket and other cores = on some other socket. I am migrating my application from one intel machine with 8 cores, all in o= ne socket to a 32 core machine where 16 cores are in one socket and 16 othe= r cores in the second socket. My core 0 does all initialization for mbuf's, nic ports, queues etc. and us= es SOCKET_ID_ANY for socket related parameters. The usecase works, but I think I am running into performance issues on the 32 core machine. The lscpu output on my 32 core machine shows the following - NUMA node0 CPU(s): 0,2,4,6,8,10,12,14,16,18,20,22,24,26,28,30 NUMA node1 CPU(s): 1,3,5,7,9,11,13,15,17,19,21,23,25,27,29,31 I am using core 1 to lift all the data from a single queue of an 82599EB po= rt and I see that the cpu utilization for this core 1 is way too high even = for lifting traffic of 1 Gbps with packet size of 650 bytes. In general, does one need to be careful in working with multiple sockets an= d so forth, any comments would be helpful. Regards -Prashant =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D =3D=3D=3D Please refer to http://www.aricent.com/legal/email_disclaimer.html for important disclosures regarding this electronic communication. =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D =3D=3D=3D =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D Please refer to http://www.aricent.com/legal/email_disclaimer.html for important disclosures regarding this electronic communication. =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D