From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 28AC145A04 for ; Tue, 24 Sep 2024 01:18:00 +0200 (CEST) Received: from mails.dpdk.org (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id A167D4027D; Tue, 24 Sep 2024 01:17:59 +0200 (CEST) Received: from sonic310-25.consmr.mail.ne1.yahoo.com (sonic310-25.consmr.mail.ne1.yahoo.com [66.163.186.206]) by mails.dpdk.org (Postfix) with ESMTP id 313314027C for ; Tue, 24 Sep 2024 01:17:57 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yahoo.com; s=s2048; t=1727133476; bh=kat/PXFGBqFWL7M0Qs6ghsE26IIfoVPM+epiofhYtoc=; h=Date:From:To:Cc:In-Reply-To:References:Subject:From:Subject:Reply-To; b=f8ffv6ZCM+/+cpMG4Cfwc/S57Xes1kcluXFqz9FamYcexpsP2aYKkkgSa5CSmnRzN/8jAI/amG8hBY/P1e0LF4ntUbm8ZguQvYhPJgMF9zvy/UHH9ZxFB5C6zMoJUwGBYt7qz+r6PWVgY8LL/M5jPlqqEDLUJi0fxxUW7lbp51DekgZkXDwzaYAFXbh4cKApJRhS4UgrlZtT3EsAa6iBvZtLcFR+RftONaQprt0dewFL+40pEvUXoIAopJ2+5FYjARkRt8C+UFTVJrxboRfYkCqJS9jIVyA7sOB3FAvYzb+4D2jhSBlMdbclYo+ySUD66kkIerBsca04IoBS6XoF9Q== X-SONIC-DKIM-SIGN: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yahoo.com; s=s2048; t=1727133476; bh=q1WUUjNzFQJZ8fB7zW9yYBipwgk+M7VGVh2gH7ZU8F1=; h=X-Sonic-MF:Date:From:To:Subject:From:Subject; b=jnicadZxeLRWnteonsspYw4IU+1ywI8uUzp0Jqqwq248hVs7jLYUTVoaiQcXfbGX2jfy7sQVnMG4jtsgPTiDZMELDTUXmG6fRRHQk+WFJHs6etdAkCwXHQd45gUVWxahbgtUuvV2sTDW6zgg7zj7z3tgR9XlD0Syqi6W+M4xYjnbU4D1zLyeZnBF9dDDt0xzRQPSOqoEeK7lxn3NZDjmnccU33tYgwtGA5QBVOxC9EA/lirrfvq8P2YFW1NMQlqamLERLMH1AtmLoKas7+COD58zA8fM7qDcQxiuoxeA/0V5+zak8llgOpzU49lKym/f9YKICRNa2xhUyqx4Tf3ZeA== X-YMail-OSG: PTr.aqoVM1nXcAZMiHb4CiMWRP3fceQ3nTLtDE.FwxJHxzKmjVxll.ctgoB1GYk NYau3tgLiEYzw8CWjOAjaCgOOa.RVTxB3tDgf1WkY.MC6wdAh5QWe6s_vKUR2D6nEebwyF0nr7xV mPk0b0j2Wf_6MHuX977h3JMOpl6ahQrWhhwa6.Yb94r4fN.0idJ6i5m_HW5xF.H3SgLa1KQYKf2m Y1aewDZ7lTAESwZhbpLOotHzYNppdLSMbioYzFtGsqRIWqHR67ukNTpmsVB1IkUjF_XXWZ6N4fBz Tr3TNxX415nA8h0voG80_OJgBY4Q93P9raFIeBZjGHEGzuR3quVWNrwFr8NSFMGw.woRgGziXEPt eLCWiL07_7tS1tiAqqyzGmiO45.kELdMZBNYNuXTmpdcroQzBUKRvvMDZJQrhHPbN2SpOO7u6b1z MJBcKrrUvTRREOiGkuRR7HDBbrj9zkeGbppAZ57Mo_ySrhpmeX8sBfw6JLqvJk9lkEEWzvpkCRy2 Yg9OxE5kUWgNkwTs9UnALRgAwVIMEcHZMYjq54aVf6TEHtWipzdrxjzMjIVkt4XVJns9yxF7_7Ax YDoV3GqPZsDzPgy9YcKf9lSG95YWEWkv_Ud1TbicR3At6_zIUxGI_bYvmnRf86LNPWfGz9NoCsLP u6YPWOcWCZxQMvhy7lwIjeTdJI2Bv.leRfSxUagR5E4MvDNrFXEdBKNYVGguHxpO.r2wc9NtTuHn PM6Kv2NBTrV2q3JDL_R2zIpBS04fXZsIGuemHqC.55.FHBhuJt00ShmI3DNf6WLU85HLP_lnWRUr 7Lu1A2gIvd9UVk6cslXGng1CQbUv7OCj1QfGNXA39w9O379Ps5OUAH0leL6zl_Un9MDkRnaAlfDd XEyQn0OpBx5mQiZhJsqRB7c8Z12Wt0XcxPm_pWSixEwqUvvXfWsSqXWNh4.ws4PFckogZt_nTLIA y9UC1Y2ruXYw1H6S3v2RoFhBVAxltMYEPULmvZzYnD4mWwU9kbGEkNtMiSqI16WG.WXsT1ZWANJS kjsXAAhQfXgWl17ALnagA38Mj5bSDbo16R6MFqL57QgV4YhaLzmTBKipL40EfoZgFy3Os.2pdKg. VTfjv0q.BDwn5dUujGof6Ze25bA6laDrXOEOS3osWv1EVp9.nGH3vbYBHBnYb2UcGtB6K7qwyZMq AXi4LRc6fyvwU8fgFe8vQioqtA20iafKEkeytBXR9UEtmCKxHK.7zWeh5bNfhbVusQWX5h1UipoD wQkiJ.3FMtuOqwLPHOMWQwUuwSXIehpAvZjx3MW9dbV1l9lKjdR5yY8XIGd_MeXgslrqACq1l.lE NZw8S_4yFL_jr6wyqd_lZrHv_6jYEZl3PVoDQBEWm_mjeefmRx6m5X2cwYturtPg0d5kgibPrpfO awHw22Okf0TBRiwf.ik7cMhwfRA1cXyZNMSh3FojPPmdE2l2Uvq2xln3dzPYgc_ih7ZzRzwwm5o7 GG9mnbcjSprZLeTKh2HSL.TsLxwxtIWBbwcWPkD8jFRdUu8JctvU7eWVBT6_g_dLI3d5yC3_e.mD iKT8lj1oIu1ywp3zZkveNazE5uD3xJlIE6o68Ba48vRcNWzRAdzD3lYXulsKfAxphTbJihovVjmF u7nTMrHO.9bAJm5bNI9BiYuI0NbB1mRWzecVmDutkR1IUqjvEl6ZVnu7VyqyqXz4sL0DTvnCA2hb hAG.5pucqHAeemTbAsjKDHBawWUllbdb42EDneJAX17XBlW0py90562yiLXDrArGz4Eq9UYDNsMl SnIkhCQbY1TkRd87TpGacJtqGhcP.PbJs3ZIOju95fWK5pQ4Vw3dXR7Z3M0emgsyYwF7qTC54joU Z8016xVYNo2rWuPfUcTHP4U_OJC6XdVMD1xha0hzl8e3DoYpwzmwDv2ERm2tLZDWe6ueRa9EVNFA 7erwMFDP_22f4wU0H59hz6pMJagcDNqPwd43KuQjFbn4hb.Y_q3uC4zvuHyliDDd8mKuOQoKI2x. F.kZITM5eOX.TCDeZK6vfytK0ZN_5QKmJdUMVvMOW1p5jW7vLBv_qjN1jtJj_q3_XOFZV.gFLsH5 tpYia8h0VQebfGhLWLzRIdwTgiYPrXoDDsfDDIn.vjSzFYEW0t814FN8pEAUJyUrV0CXPHO28rf. jmF3vfcAEATAfDhrFq9IlOixl4TIFrJ3yRtRwXjov0_T93iDt7VIJrwwEBDL3vD2Ji6.DCKwIiK0 Gk.yn5X5lSU5d94FK0EsPNzygBLQtWhSVIpqLM7Bc56I_aVXMv234a.vsa0qNxK62Aj1bhq2doaA h7RfRi9UdEA-- X-Sonic-MF: X-Sonic-ID: dde265bc-1613-4d01-856b-f186f6481c90 Received: from sonic.gate.mail.ne1.yahoo.com by sonic310.consmr.mail.ne1.yahoo.com with HTTP; Mon, 23 Sep 2024 23:17:56 +0000 Date: Mon, 23 Sep 2024 23:17:54 +0000 (UTC) From: amit sehas To: Wisam Jaddo , Nishant Verma Cc: "users@dpdk.org" Message-ID: <1299564509.11731667.1727133474900@mail.yahoo.com> In-Reply-To: References: <1987164393.11670398.1727125003663.ref@mail.yahoo.com> <1987164393.11670398.1727125003663@mail.yahoo.com> Subject: Re: core performance MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable X-Mailer: WebService/1.1.22645 YMailNorrin X-BeenThere: users@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK usage discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: users-bounces@dpdk.org Thanks for the responses, this is on AWS, which is utilizing Xeon with hype= rthreading. Not utilizing hyperthreading is not an option. After trying a few things i am narrowing down on the following approach: only for the critical threads we could utilize:=C2=A0 rte_thread_set_priori= ty to=C2=A0RTE_THREAD_PRIORITY_REALTIME_CRITICAL however this API requires a rte_thread_t parameter, if we utilize rte_eal_r= emote_launch, we are not provided with this parameter. I am searching through the code to see if there is an API where i can obtai= n the rte_thread_t for the current thread that was launched with rte_eal_re= mote_launch. regards On Monday, September 23, 2024 at 03:18:11 PM PDT, Nishant Verma wrote:=20 Also make sure all core you are using are physical core not the logical cor= e.=C2=A0 Secondly, check your core isolation options and apply them accordingly. . Regards, Nishant Verma On Mon, Sep 23, 2024 at 6:04=E2=80=AFPM Wisam Jaddo wro= te: > Hello Amit, >=20 >> -----Original Message----- >> From: amit sehas >> Sent: Monday, September 23, 2024 11:57 PM >> To: users@dpdk.org >> Subject: core performance >>=20 >> We are seeing different dpdk threads (launched via=C2=A0rte_eal_remote_l= aunch()), >> demonstrate very different performance. >>=20 >>=20 >>=20 >> After placing counters all over the code, we realize that some threads a= re >> uniformly slow, in other words there is no application level issue that = is >> throttling one thread over the other. We come to the conculsion that eit= her >> the Cores on which they are running are not at the same frequency which >> seems doubtful or the threads are not getting a chance to execute on the= cores >> uniformly. >>=20 >>=20 >>=20 >> It seems that isolcpus has been deprecated in recent versions of linux. >>=20 >>=20 >>=20 >> What is the recommended approach to prevent the kernel from utilizing so= me >> CPU threads, for anything other than the threads that are launched on th= em. >=20 > If you are wishing to run each thread on separate core, try to use rte_ea= l_mp_remote_launch() > instead of rte_eal_remote_launch(), make sure that your CPU is isolated, = and you are passing correct > Cores that were isolated to your app using -c, -l. >=20 >=20 >>=20 >>=20 >>=20 >> Is there some API in dpdk which also helps us determine which CPU core t= he >> thread is pinned to? >>=20 >> I did not find any code in dpdk which actually performed pinning of a th= read to >> a CPU core. >>=20 >>=20 >>=20 >> In our case it is more or less certain that the different threads are si= mply not >> getting the same CPU core time, as a result some are demonstrating highe= r >> throughput than the others ... >>=20 >>=20 >>=20 >> how do we fix this? >=20 > BRs, > Wisam Jaddo >=20