From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 2AB8845A3C for ; Fri, 27 Sep 2024 05:23:05 +0200 (CEST) Received: from mails.dpdk.org (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id A4CB0400EF; Fri, 27 Sep 2024 05:23:04 +0200 (CEST) Received: from sonic310-25.consmr.mail.ne1.yahoo.com (sonic310-25.consmr.mail.ne1.yahoo.com [66.163.186.206]) by mails.dpdk.org (Postfix) with ESMTP id 9F56C4003C for ; Fri, 27 Sep 2024 05:23:03 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yahoo.com; s=s2048; t=1727407383; bh=uRYj2dxqkalxsqk7dvGK8FKU6B+ohcccdO/kZ5vmZW4=; h=Date:From:To:Cc:In-Reply-To:References:Subject:From:Subject:Reply-To; b=BBTRQRVcUGLl45z2bndTyo3XlQqnoWwC58Fl6JqFHISa/HZR5DlfXWcLAwEbJO5i3mca8+IRIuY5ECUslHMR6lbnuH8B6ZkmnznJ5yyCC1tn0L68GDH2uNceBywP2TQXV8cGweEihBAjxVYucvf3Wwe6jsMfCr+quwr9/d2oiICdzDuyVLSRPYz5vRCD1ngMU0L+cv+/0nGE4XNzTtXV+0vdzYKYy8x7D35j+pUs0fW39PXZ+hBYRPJO3lupm8eYkrig+fPhDH8FS7IShE03m2dEdlu2tH9zYFJRiAEfdjl/mQny7uXpGMnjqE6rjwNVwGEN/PYTZivlULK1vzBJDA== X-SONIC-DKIM-SIGN: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yahoo.com; s=s2048; t=1727407383; bh=FsJIsoIgZIyXV8RkO2qQ1XEj19oID9C6ztZnoneIKNL=; h=X-Sonic-MF:Date:From:To:Subject:From:Subject; b=P93uNEehBTwpcl0sU39A60kkitnjw998pD5sPPLEc7+be5QxnsPv3nd/bK3shiOhmdURqfvejKo5joVAl0kuJH3q5Jq61DR5qtmbhJggCOLSOtLwL90K3VUjo3g8XGyAwEwNIwfHUaMGq3Y5Ki5mjr6ep1fsOF8x092xp/HaskhB/xYatND5wAo1OIM73P+tmJ68mI67BDx0HRZ8x1h3eaNUprk0SLS2Q+ljZsM4XIAQMHhFKkf9LF1zcvCUH9MGUQF24mclSyN8votsAoGQ1cJkfUtlsJ5YHC4n+AqRuOexj4Soa2QirtJUZfTNJEwk9eLcZlQkFiLJ7QmFXRluMw== X-YMail-OSG: Qnb8RIcVM1lfXYOnzKINZOHy9n01fE58QK5__y6DGAhPZ8Mz8vK5DwTz8VPp7c7 wA5mVv_Zmlo2neypBg_kAiFwDwH3p1H_Stmui5f3JbA8UkjpswRpaVT6yFRskej_SM8X7uHXcE0_ kuHzuyWWjDBCFRgUUT2b2CukkBKJFfGF3WnHnZxCbstqiGLDwmAfjtCCBvt_XB71hg_F9TkiEYsx .jGqs4wd1QPRKGZkJEFpBfs.USFXhxxOzC0Gq5j1Ra76aBgXa3FCxmnsYuue58U6sxh_6D.EdS.s iVdlp_1_tW5z3GNad8NeI2xpUg0xsK5hha.dSKKJ0eqCdOlgITBrIVYgBImgMzXijZ8YYqXcI8rK Yo.5KQtOF4KS2mIrub2Tw2eGymHlXeDdcYxTxIV.ihOVLMAl8pNN.uVy22iKTumr5N2cv2AD_DcY tFrOXGkmq8wxffFgoejr5aasA0fm82uDOF479T9VPXL7YUNgCB6feuWNgxOSJNe3XtsPIuHYbwVS EZXOqZ7HiZNKUQ3IOdAJK6WUAr_VAFsHHGkhowmZMCcIVj.wy2jUYXMLiDXgFCbxBwaD_491dwql e9JkEoltPGL8.hfvGOEUltsMYsHAimaVWM1IPIB7DGHz.WZgk965EQLGu_JflzQoaMpe5Yz0rjol bU.0iQx_n31kPHjG985zvRlCKVm5G2vvKOmDO3tah__b5mktFUNjuYihdAEPVeBJI4ITT5lLASU0 _rIoa4PRbP1Wkypjx3MgMG5eVCFS6XdME2.no2Ma.UbMpz9fnJDYPt.f5koIDQHIc5TR4jc7cSx. RCKoA.Bj5tHsBvDI6i8DJpyD6xt2yl4dFGELge2Uy.f1pDExV.7uHIhkQiCzEVAAP176ILBrFBvA g_H1mn.LCCUjPwrPN0WeUPKSRAVvF7chNpIk7.lt4ecB_1YwsFumhjGT7hyNMiz.x2Tm7pgDU74F wnuerFaVurlw8smZep855VSi2NYsLhydTyUqg9CccgAQ.cYnwwz3Lh6lXqCo7Wmt31ojfN8P3RiQ BPLvBJ6qXEu8nkPsoJzNkBXE1hFWW26edwaoGWJk3zcXhkPqnhLsh3HXHgh4Tlh3NGKec6cXHXaR f5LP4o2z0ZQQr29rN3F33.x_CPSBgoMl0Yo0AJyUCmrTRrOwJ4VxeoR3iMHOluFxGqyI98.UDE7X yR2DSVuGGN6YtvICuYrX0cHlLyz9jqBZw1wK7bagxOMwOeCUBiUc5xaFCyyGrmxwI1dDdoMUkyq8 yaTtGtHLx_Cfy5C4NIILRKoDRGAsafy.5PWuW_RKTBw.YtCOr4nCkIjg6nurOr5VEijuZv11_0Q6 aWATRf9aM3qmjUPjUH7b1YLuOt2JYd_R3.9WSfzr0sBg3BJAdDs_6gDd23FFQiH8mHOu7pAlSNe5 Y3FY.uGLUlmcdMRXVc.9GmV7u.fWWFrQ8W4AmFSMp6tfPmH3ZeTlpZ1Q7In_IjydwC7D6vE0WC0Z pNNFHxNXbwFSJsuF46_ez0FjhncaogI9p9Q7w_fxvCFGYkQnpNnkAkfDUIFhp9oVZG731S1MLx_. 94p9pHoSwn66YATLhj0KgN6ZlPi3IW_vNTS3l4vrLt68uVi6lloC1AFGz4oWGsfEN.UeNiQlYjy3 lVVuTO1T5QhInD9PTLdTHC4c9baxIiyFLzLsMM2Hy_ySpoZ9_f1UaRmwZfK0e_E3J7vBhsyxXEGt feKdViDzLAGRthNoWDBdVpcDf9BSoli0zm7bSk6hKiQonTX6.CwsZVl7Lha7y2.gedNfW_L0oyEP aJpDucW_7LrZ6lQyXgpNrxMr4kJLLJbCxci52JIrKVdUMBKw5uxZpgOM.RdRK2PS_PunM9d9BEvJ peDCBD7WhYpFQpmlShdgzuRqEKiRJkhvMNjBRg0tdOC88uVw1g1mOCzO54JCPKMaEvLDpyk7TRaL hhApaYXt17ycUAWEUhP.BEQGGS_GBuEfZK9fBGCs601bCI3O.6X_sUQNW9bBLbyFTeMHO61XiQF8 28UTlouSjNjef7b1lwneo4T62_7xJ4hsrdDkO53dDkpDrBnVZlc.R3AqiPlXUfSgL71lYDAS0ZAc 6uysSYJThZHnPF5aBQ9AK280laTGRA0vXuJQ0B5Zhpcmt9hgqbqUpldDi9LmgL3JnEf6JVuF955i Iucxz8UX86QLiTYHoI3rJbzX0riVKXma_ClOrmcbYcjIInWnSgtjWhx2aAF7MixFBUMBkNyw_OC3 7lHQwImpYiNx1SARpmN1CVVgmycK6d7P1j8R.bYoRF0OTaI8- X-Sonic-MF: X-Sonic-ID: 2606db01-3e38-4816-9627-ebc83ad53b53 Received: from sonic.gate.mail.ne1.yahoo.com by sonic310.consmr.mail.ne1.yahoo.com with HTTP; Fri, 27 Sep 2024 03:23:03 +0000 Date: Fri, 27 Sep 2024 03:23:01 +0000 (UTC) From: amit sehas To: Stephen Hemminger Cc: Nishant Verma , "users@dpdk.org" Message-ID: <781570616.13263184.1727407381739@mail.yahoo.com> In-Reply-To: <894778529.13304006.1727406828202@mail.yahoo.com> References: <1987164393.11670398.1727125003663.ref@mail.yahoo.com> <1987164393.11670398.1727125003663@mail.yahoo.com> <1299564509.11731667.1727133474900@mail.yahoo.com> <2025533199.11789856.1727143607670@mail.yahoo.com> <2042269904.11975457.1727188849962@mail.yahoo.com> <20240924093813.29a01783@fedora> <26643152.12164440.1727210825368@mail.yahoo.com> <810098753.12902128.1727353971978@mail.yahoo.com> <47151973.13053687.1727369764729@mail.yahoo.com> <610700496.13049973.1727370197636@mail.yahoo.com> <20240926200327.6f74ae2d@hermes.local> <894778529.13304006.1727406828202@mail.yahoo.com> Subject: Re: core performance MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable X-Mailer: WebService/1.1.22645 YMailNorrin X-BeenThere: users@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK usage discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: users-bounces@dpdk.org For a weaker AWS instance this is what we find with cpu_layout.py: =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D Core and Socket Information (as reported by '/sys/devices/system/cpu') =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D cores =3D=C2=A0 [0, 1] sockets =3D=C2=A0 [0] =C2=A0 =C2=A0 =C2=A0 =C2=A0Socket 0 =C2=A0 =C2=A0 =C2=A0 =C2=A0-------- Core 0 [0, 2] Core 1 [1, 3] So from this we deduce that logical core 0 and logical core 2 are on the sa= me physical core and 1, 3 are on the other physical core,=C2=A0 learnt something valuable to= day ... regards On Thursday, September 26, 2024 at 08:13:48 PM PDT, amit sehas wrote:=20 Thanks for the suggestion, i didnt even know about cpu_layout.py ... i will= definitely try it.... i made some more measurements and so far this is the= hypothesis: 1) 8 hyperthreads are not the same as 8 CPUs, the scale up is not linear. 2) the vCPU cache allocation per logical CPU thread is also important, if 2= threads are running the same code on the same physical core but 2 differen= t logical cores then we will not have cores competing with each other. 3) try to run dissimilar code on the logical cores that run on the same phy= sical core ... the cpu map is deinitely worth figuring out ... regards On Thursday, September 26, 2024 at 08:03:30 PM PDT, Stephen Hemminger wrote:=20 On Thu, 26 Sep 2024 17:03:17 +0000 (UTC) amit sehas wrote: > If there is a way to determine: >=20 > vCPU thread utilization numbers over a period of time, such as a few hour= s >=20 > or which processes are consuming the most CPU >=20 > top always indicates that the server is consuming the most CPU. >=20 > Now i am begining to wonder if 8 vCPU threads really are capable of runni= ng 6 high intensity threads or only 4 such threads? Dont know >=20 > Also tried to utilize=C2=A0pthread_setschedparam() explicitly on some of = the threads, it made no difference to the performance. But if we do it on m= ore than 1-2 threads then it hangs the whole system. >=20 > This is primarily a matter of CPU scheduling, and if we restirct context = switching on even 2 critical threads we have a win. >=20 >=20 Some other recommendations. =C2=A0 - avoid CPU 0 you can't isolate it, and it has other stuff that has = to run there =C2=A0 =C2=A0 if you have main thread that sleeps, and worker threads that = poll, then =C2=A0 =C2=A0 go ahead and put main on cpu 0. =C2=A0 - don't put two active polling cores on shared hyper-thread. =C2=A0 =C2=A0 You can used DPDK's cpu_layout.py script to show this. For example: $ ./usertools/cpu_layout.py=20 =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D Core and Socket Information (as reported by '/sys/devices/system/cpu') =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D cores =3D=C2=A0 [0, 1, 2, 3] sockets =3D=C2=A0 [0] =C2=A0 =C2=A0 =C2=A0 Socket 0=C2=A0 =C2=A0=20 =C2=A0 =C2=A0 =C2=A0 --------=C2=A0 =C2=A0=20 Core 0 [0, 4]=C2=A0 =C2=A0 =C2=A0=20 Core 1 [1, 5]=C2=A0 =C2=A0 =C2=A0=20 Core 2 [2, 6]=C2=A0 =C2=A0 =C2=A0=20 Core 3 [3, 7]=20 On this system, don't poll on cores 0 and 4 (system activity). Use lcore 1, 2, 3=20