From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from dpdk.org (dpdk.org [92.243.14.124]) by inbox.dpdk.org (Postfix) with ESMTP id C8FF8A0487 for ; Mon, 1 Jul 2019 17:39:14 +0200 (CEST) Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id 2ED8A1B9D7; Mon, 1 Jul 2019 17:39:14 +0200 (CEST) Received: from mga01.intel.com (mga01.intel.com [192.55.52.88]) by dpdk.org (Postfix) with ESMTP id DB0FA1B9C5 for ; Mon, 1 Jul 2019 17:39:12 +0200 (CEST) X-Amp-Result: SKIPPED(no attachment in message) X-Amp-File-Uploaded: False Received: from orsmga004.jf.intel.com ([10.7.209.38]) by fmsmga101.fm.intel.com with ESMTP/TLS/DHE-RSA-AES256-GCM-SHA384; 01 Jul 2019 08:39:11 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.63,439,1557212400"; d="scan'208";a="314921404" Received: from irsmsx151.ger.corp.intel.com ([163.33.192.59]) by orsmga004.jf.intel.com with ESMTP; 01 Jul 2019 08:39:10 -0700 Received: from irsmsx102.ger.corp.intel.com ([169.254.2.159]) by IRSMSX151.ger.corp.intel.com ([169.254.4.183]) with mapi id 14.03.0439.000; Mon, 1 Jul 2019 16:39:10 +0100 From: "Van Haaren, Harry" To: Antoine POLLENUS , "users@dpdk.org" Thread-Topic: What is the best threading technology when using DPDK ? Thread-Index: AdUwD72eUhtDZ/lvQSi78Wj0kqO9awAEWd3g Date: Mon, 1 Jul 2019 15:39:09 +0000 Message-ID: References: <1a2e3c0440bd4c0f849a5ff871a4c289@deltacast.tv> In-Reply-To: <1a2e3c0440bd4c0f849a5ff871a4c289@deltacast.tv> Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-titus-metadata-40: eyJDYXRlZ29yeUxhYmVscyI6IiIsIk1ldGFkYXRhIjp7Im5zIjoiaHR0cDpcL1wvd3d3LnRpdHVzLmNvbVwvbnNcL0ludGVsMyIsImlkIjoiODg5ZGQyMTAtMzg4OS00NWM3LWIzMjYtMmJjYTFjNDkwYjI3IiwicHJvcHMiOlt7Im4iOiJDVFBDbGFzc2lmaWNhdGlvbiIsInZhbHMiOlt7InZhbHVlIjoiQ1RQX05UIn1dfV19LCJTdWJqZWN0TGFiZWxzIjpbXSwiVE1DVmVyc2lvbiI6IjE3LjEwLjE4MDQuNDkiLCJUcnVzdGVkTGFiZWxIYXNoIjoiQ2RhazFvMkRtTkhsSVJwUXlXaWg3NXBraXhsUGF2ZlwvemRDUGY0RTBDNmQyeDgzR2VHMU9HRHNWTXBxQWR1Q08ifQ== x-ctpclassification: CTP_NT dlp-product: dlpe-windows dlp-version: 11.0.600.7 dlp-reaction: no-action x-originating-ip: [163.33.239.180] Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Subject: Re: [dpdk-users] What is the best threading technology when using DPDK ? X-BeenThere: users@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK usage discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: users-bounces@dpdk.org Sender: "users" > -----Original Message----- > From: users [mailto:users-bounces@dpdk.org] On Behalf Of Antoine POLLENUS > Sent: Monday, July 1, 2019 2:20 PM > To: users@dpdk.org > Subject: [dpdk-users] What is the best threading technology when using DP= DK > ? >=20 > Hello, Hi Antoine, > I'm developing a time critical application using DPDK that require to be > multithreaded. I'm wondering what threading technology I should use ? >=20 > - Can I use the standard pthread library and if yes, is there a > trade of in term of performance ? >=20 > - I see on this page that a lthread library also exist but is ki= nd > of limited in term of functionality: > https://doc.dpdk.org/guides/sample_app_ug/performance_thread.html >=20 > - I see also that we can launch a function on another lcore usin= g > the rte_eal_remote_launch(...) >=20 > Is there a recommendation when using DPDK to use a technology threading > technology or another ? Good questions to ask, I'll bullet a few thoughts in reply; - DPDK provides its own threading APIs, that depending on the platform call= s the OS native implementation. For Linux this means pthreads. So by using = DPDK's thread APIs, you're really using pthreads, but with a wrapper layer.= This wrapper layer means that you can recompile against other targets (win= dows support is WIP for DPDK) and you won't have to change your threading c= ode... - Lthreads is a method of scheduling large numbers of work items on a lower= numbers of "real" threads. Think of it as a scheduler implementation (like= any OS has, to multiplex tasks to HW CPU cores). If you are running in a t= ime-critical domain, general practice is to avoid multiplexing, and to dedi= cate resources just to the time critical work. In short; I suggest you run = a DPDK lcore dedicated to the time-critical task, and do not use lthreads. - The DPDK threading APIs use rte_eal_remote_launch() to "spawn" a worker t= hread to a given hardware thread on a CPU. (With hyper-threading, or runnin= g 2 "logical" threads on one "physical" core, this enumeration becomes a li= ttle more complex, but is still valid). DPDK uses this feature to do core-p= inning, which means that a worker pthread is affinitized with a specific ha= rdware-thread on the CPU. This stops the linux scheduler from moving the so= ftware-thread to a different CPU core/thread, which is desirable as you wan= t to minimize jitter for time sensitive workloads. (And switching to a diff= erent CPU core/thread requires work, and hence takes time). - For time sensitive processing, my recommendation would be to spawn a work= er thread into a busy-loop for the time critical task. If possible it is be= st to dedicate that CPU for the task and not put any other work on that thr= ead - this will minimize the jitter/latency. - Investigate the "isolcpus" kernel boot parameter, and IRQ affinities if y= ou have not already done so, to reduce jitter due to Linux scheduler and IR= Q subsystem interference with the DPDK thread. > Regards >=20 > Antoine Pollenus Hope the above helps! Regards, -Harry