From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from dpdk.org (dpdk.org [92.243.14.124]) by inbox.dpdk.org (Postfix) with ESMTP id 198A7A0487 for ; Tue, 2 Jul 2019 07:41:33 +0200 (CEST) Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id 4ED8E5905; Tue, 2 Jul 2019 07:41:32 +0200 (CEST) Received: from mail.deltatec.be (mxbackup1.deltatec.be [62.197.119.12]) by dpdk.org (Postfix) with ESMTP id EED0728EE for ; Tue, 2 Jul 2019 07:41:30 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=deltacast.tv; s=AnsUTM; h=MIME-Version:Content-Transfer-Encoding:Content-Type:In-Reply-To:References:Message-ID:Date:Subject:To:From; bh=gjXCKlbBi8OPhXOlolH+rQYm0lRbcGWtnN1X/Npa3hs=; b=WNa/3l29UvgHUn3Esh9cvMwbANPbXuR26nqa2/8kzfqAQyZ+5b114jrkfVh5hzvlVen+v+ChlYiEDofkz/2NmUFBjZMyhYVgwMna5SwAB+QTCaS8sKjTQhVUmPy7ECg/EZpmxQHJoOpjqMrrsA+UHpomhyjwQHK22VUTo0cuJlU=; Received: from [172.16.4.5] (port=32071 helo=W2K16-SVR-5.office.deltatec.net) by mail.deltatec.be with esmtps (TLSv1.2:AES256-GCM-SHA384:256) (Exim 4.82_1-5b7a7c0-XX) (envelope-from ) id 1hiBXk-0003Ca-1y; Tue, 02 Jul 2019 07:41:28 +0200 Received: from W2K16-SVR-5.office.deltatec.net (172.16.4.5) by W2K16-SVR-5.office.deltatec.net (172.16.4.5) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.1034.26; Tue, 2 Jul 2019 07:41:28 +0200 Received: from W2K16-SVR-5.office.deltatec.net ([::1]) by W2K16-SVR-5.office.deltatec.net ([::1]) with mapi id 15.01.1034.026; Tue, 2 Jul 2019 07:41:28 +0200 From: Antoine POLLENUS To: "Van Haaren, Harry" , "users@dpdk.org" Thread-Topic: What is the best threading technology when using DPDK ? Thread-Index: AdUwD72eUhtDZ/lvQSi78Wj0kqO9awAEWd3gAB3bBnA= Date: Tue, 2 Jul 2019 05:41:28 +0000 Message-ID: <5140278f2ae549919bbbf03e13c93578@deltacast.tv> References: <1a2e3c0440bd4c0f849a5ff871a4c289@deltacast.tv> In-Reply-To: Accept-Language: fr-BE, en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-originating-ip: [172.16.6.165] Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Subject: Re: [dpdk-users] What is the best threading technology when using DPDK ? X-BeenThere: users@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK usage discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: users-bounces@dpdk.org Sender: "users" Thanks a lot for your answer. I can now see clearly what I have to do in my= implementation. Regards Antoine Pollenus -----Original Message----- From: Van Haaren, Harry [mailto:harry.van.haaren@intel.com]=20 Sent: lundi 1 juillet 2019 17:39 To: Antoine POLLENUS ; users@dpdk.org Subject: RE: What is the best threading technology when using DPDK ? > -----Original Message----- > From: users [mailto:users-bounces@dpdk.org] On Behalf Of Antoine=20 > POLLENUS > Sent: Monday, July 1, 2019 2:20 PM > To: users@dpdk.org > Subject: [dpdk-users] What is the best threading technology when using=20 > DPDK ? >=20 > Hello, Hi Antoine, > I'm developing a time critical application using DPDK that require to=20 > be multithreaded. I'm wondering what threading technology I should use ? >=20 > - Can I use the standard pthread library and if yes, is there a > trade of in term of performance ? >=20 > - I see on this page that a lthread library also exist but is ki= nd > of limited in term of functionality: > https://doc.dpdk.org/guides/sample_app_ug/performance_thread.html >=20 > - I see also that we can launch a function on another lcore usin= g > the rte_eal_remote_launch(...) >=20 > Is there a recommendation when using DPDK to use a technology=20 > threading technology or another ? Good questions to ask, I'll bullet a few thoughts in reply; - DPDK provides its own threading APIs, that depending on the platform call= s the OS native implementation. For Linux this means pthreads. So by using = DPDK's thread APIs, you're really using pthreads, but with a wrapper layer.= This wrapper layer means that you can recompile against other targets (win= dows support is WIP for DPDK) and you won't have to change your threading c= ode... - Lthreads is a method of scheduling large numbers of work items on a lower= numbers of "real" threads. Think of it as a scheduler implementation (like= any OS has, to multiplex tasks to HW CPU cores). If you are running in a t= ime-critical domain, general practice is to avoid multiplexing, and to dedi= cate resources just to the time critical work. In short; I suggest you run = a DPDK lcore dedicated to the time-critical task, and do not use lthreads. - The DPDK threading APIs use rte_eal_remote_launch() to "spawn" a worker t= hread to a given hardware thread on a CPU. (With hyper-threading, or runnin= g 2 "logical" threads on one "physical" core, this enumeration becomes a li= ttle more complex, but is still valid). DPDK uses this feature to do core-p= inning, which means that a worker pthread is affinitized with a specific ha= rdware-thread on the CPU. This stops the linux scheduler from moving the so= ftware-thread to a different CPU core/thread, which is desirable as you wan= t to minimize jitter for time sensitive workloads. (And switching to a diff= erent CPU core/thread requires work, and hence takes time). - For time sensitive processing, my recommendation would be to spawn a work= er thread into a busy-loop for the time critical task. If possible it is be= st to dedicate that CPU for the task and not put any other work on that thr= ead - this will minimize the jitter/latency. - Investigate the "isolcpus" kernel boot parameter, and IRQ affinities if y= ou have not already done so, to reduce jitter due to Linux scheduler and IR= Q subsystem interference with the DPDK thread. > Regards >=20 > Antoine Pollenus Hope the above helps! Regards, -Harry