From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mga02.intel.com (mga02.intel.com [134.134.136.20]) by dpdk.org (Postfix) with ESMTP id 96CD46882 for ; Mon, 3 Mar 2014 21:10:31 +0100 (CET) Received: from orsmga001.jf.intel.com ([10.7.209.18]) by orsmga101.jf.intel.com with ESMTP; 03 Mar 2014 12:11:58 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="4.97,579,1389772800"; d="scan'208";a="465826531" Received: from orsmsx106.amr.corp.intel.com ([10.22.225.133]) by orsmga001.jf.intel.com with ESMTP; 03 Mar 2014 12:11:58 -0800 Received: from orsmsx115.amr.corp.intel.com (10.22.240.11) by ORSMSX106.amr.corp.intel.com (10.22.225.133) with Microsoft SMTP Server (TLS) id 14.3.123.3; Mon, 3 Mar 2014 12:11:57 -0800 Received: from orsmsx103.amr.corp.intel.com ([169.254.2.166]) by ORSMSX115.amr.corp.intel.com ([10.22.240.11]) with mapi id 14.03.0123.003; Mon, 3 Mar 2014 12:11:57 -0800 From: "Jayakumar, Muthurajan" To: Jane Shen , "dev@dpdk.org" Thread-Topic: Physical core vs. hyper threaded core Thread-Index: Ac82Vrk+hXP+o9JVR2ubvxJCfNUDfAAKrqegAADiLNAAAFfE0AABW5oAACNItzAAANNIoA== Date: Mon, 3 Mar 2014 20:11:56 +0000 Message-ID: <5D695A7F6F10504DBD9B9187395A21797C6E78AC@ORSMSX103.amr.corp.intel.com> References: <96144595DCDF66419CB40342CCD31E2E3E441B43@mbx024-e1-nj-4.exch024.domain.local> <5D695A7F6F10504DBD9B9187395A21797C6E7656@ORSMSX103.amr.corp.intel.com> <96144595DCDF66419CB40342CCD31E2E3E441C63@mbx024-e1-nj-4.exch024.domain.local> <5D695A7F6F10504DBD9B9187395A21797C6E767A@ORSMSX103.amr.corp.intel.com> <5D695A7F6F10504DBD9B9187395A21797C6E76A0@ORSMSX103.amr.corp.intel.com> <96144595DCDF66419CB40342CCD31E2E3E4430D6@mbx024-e1-nj-4.exch024.domain.local> In-Reply-To: <96144595DCDF66419CB40342CCD31E2E3E4430D6@mbx024-e1-nj-4.exch024.domain.local> Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-originating-ip: [10.22.254.140] Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Subject: Re: [dpdk-dev] Physical core vs. hyper threaded core X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: patches and discussions about DPDK List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 03 Mar 2014 20:10:32 -0000 Thanks Jane.=20 Statically allocated are resources like load buffer, store buffer, reorder = buffer and small page ITB.=20 In addition, there are other resources that are replicated, competitively s= hared. Below are couple of links that you will find useful.=20 http://software.intel.com/en-us/articles/performance-insights-to-intel-hype= r-threading-technology http://software.intel.com/en-us/articles/avoid-resource-contention-on-hyper= -threading-technology-enabled-systems Thanks, -----Original Message----- From: Jane Shen [mailto:jshen@mavenir.com]=20 Sent: Monday, March 03, 2014 11:50 AM To: Jayakumar, Muthurajan; dev@dpdk.org Subject: RE: Physical core vs. hyper threaded core Thanks Muthurajan. " because there are some resources statically partitioned" What are these resources and in general what types of operations from the s= oftware will be impacted? Like memory access, IO access etc.? Under what condition would sharing sibling lcores for 2 DPDK threads would = be recommended (or at least worth trying)?=20 Thanks, -Jane -----Original Message----- From: Jayakumar, Muthurajan [mailto:muthurajan.jayakumar@intel.com]=20 Sent: Sunday, March 02, 2014 8:54 PM To: Jayakumar, Muthurajan; Jane Shen; dev@dpdk.org Subject: RE: Physical core vs. hyper threaded core In your scenario, NO interference with DPDK. Perfect. -----Original Message----- From: dev [mailto:dev-bounces@dpdk.org] On Behalf Of Jayakumar, Muthurajan Sent: Sunday, March 02, 2014 6:23 PM To: Jane Shen; dev@dpdk.org Subject: Re: [dpdk-dev] Physical core vs. hyper threaded core Thanks Jane. Great. So interference with DPDK. Perfect.=20 The reason for using physical core is, you get more resources compared to u= sing in hyperthreaded mode this is because there are some resources statica= lly partitioned and so when one disables hyperthreaded mode and use physica= l core, then all the resources that are in the core are available. So, when all the work - Rx, Processing, Tx are to be done by one core itsel= f, then using full physical core without hyperthreading may be the better c= hoice.=20 Whereas, in case one application demands pipelined implementation and you a= re choosing 2 lcores to implement, then one may be better off enabling hype= rthreading and pipelining the functionality between the siblings because in= this case inter-lcore communication cost is more important. Thanks, =20 -----Original Message----- From: Jane Shen [mailto:jshen@mavenir.com]=20 Sent: Sunday, March 02, 2014 6:14 PM To: Jayakumar, Muthurajan; dev@dpdk.org Subject: RE: Physical core vs. hyper threaded core Thanks Muthurajan. We were testing with core 0-7 to DPDK and 8-15 to Linux SIP processes. The = core numbers are based on the Linux /etc/cpuinfo. These processes don't hav= e direct coupling with DPDK. They may share some memory with the DPDK via I= PC. What was the reason that it is recommended to use the physical core only fo= r DPDK? Based on your comment below, sounds like that restriction should be= removed. Thanks, -Jane -----Original Message----- From: Jayakumar, Muthurajan [mailto:muthurajan.jayakumar@intel.com]=20 Sent: Sunday, March 02, 2014 7:49 PM To: Jane Shen; dev@dpdk.org Subject: RE: Physical core vs. hyper threaded core Jane,=20 Great. You are correct. Have tried enabling hyperthreading and it works.=20 For example, if we want to have the functionality partitioning such that Rx= + Packet Processing + Tx =3D all of these three functions can be done in = 2 cores - By positioning Rx in one lcore and by positioning Packet process= ing and Tx in the sibling hyperthread lcore of the same physical core, you = get tight coupling because L1 cache and L2 cache are shared between the hyp= erthreaded cores belonging to same physical core. =20 Curious to know - in your configuration, the SIP based signaling threads -= =20 Option A) are they sharing sibling of DPDK threads? Option B) Or all DPDK threads are tightly coupled with sibling threads and = SIP based signaling threads are on separate cores? If it is Option B) more tight coupling within the DPDK threads and less in= terference from signaling threads.=20 Thanks,=20 -----Original Message----- From: dev [mailto:dev-bounces@dpdk.org] On Behalf Of Jane Shen Sent: Sunday, March 02, 2014 4:56 PM To: dev@dpdk.org Subject: [dpdk-dev] Physical core vs. hyper threaded core Hi, I understand that DPDK should use the physical core. But here is what we te= sted: - Enable HT - Assign 8 cores of the CPU (an 8-core Sandybridge CPU) to DPDK. Surprisingly enough, we noticed that the remaining 8 cores (b/c there are t= otal of 16 cores after HT) can still handle other Linux processes which are= SIP based signaling transactions. Anybody can shed some light on how this worked? Is there anybody tried simi= lar thing? What has been your experience? Thanks, -Jane