From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mail-wm0-f43.google.com (mail-wm0-f43.google.com [74.125.82.43]) by dpdk.org (Postfix) with ESMTP id 1DE125F1F for ; Fri, 16 Mar 2018 19:20:04 +0100 (CET) Received: by mail-wm0-f43.google.com with SMTP id h21so4653481wmd.1 for ; Fri, 16 Mar 2018 11:20:04 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:in-reply-to:references:from:date:message-id:subject:to :cc; bh=dQTzEh48Sqb51MIfQmus1qK1/isV7hzrV+Et6rF7ac8=; b=kHsbnGUNdHcFkpHIlV06JMQLh1oFiDsSxg19GgxrLFc56EGdJzmKZ1Xef43ICu9h2a 2dKEH0e8K6cv7Jp+ZbDqbHFU1VqJP5OMcKLxZDWh7KsQpRV3lkPlLsSc596Wmm8lDAwQ lE03+Y2z6DKp15YW/jU9wVfr+jCm8n4eT8lbbMPM+JaBm+yI4gcnYy8i3C4UZ+VkoxER 5C/zK0A7tEB8AToL/tlMD7EN3QoWgXtH06yx2U5w0HjemTo747Qvd46YY2fwNoFO/5nJ ZYBvy2jphA9E8rEf+ztBOzyLzenVNRFRL02PSBSs3apJcDxYwjr4C5ban4DrkTqOXZFk U4AQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:in-reply-to:references:from:date :message-id:subject:to:cc; bh=dQTzEh48Sqb51MIfQmus1qK1/isV7hzrV+Et6rF7ac8=; b=BOu77VjQqSQRfT33CAGrZdHkls5acn7McTG4DD8jZtz8s3+33VMV0efoWeKNUAdfnc HxNGpIg+WIljzLVmQKuU25WW5TP3Yk3+rv83X9nZjyKFE3JCORv5Cea1piq+tBQaZ7uI 7d+2RCRjZI9C616AgzrvCVaBpSXzgzD0wEz0QvbeSaqGTP+tVdupu9wcKHh5CWetas2q zJWn3e2Mn9ZhDfC/IG0hujhcfvp1VxOYFx8xhrKEl2z/DBktnwerhJh6/8eORMfk6mSM PNrjj0HQeay94fkCSEf7EW2RI+etRGDYQyzuQSFF/JaKNzezHFVm/zri/tMPyuLyLnLt duow== X-Gm-Message-State: AElRT7G0DV7U2Yt+M1n39f+UCv/YkJrBDuuAo8CS+mT5GZj2XMC4xHoz IJNdjlYhI42j1fjvCwxAwmJknH+qThw34VQzYI0= X-Google-Smtp-Source: AG47ELs+9s1i5Fh6zuT0HHIR6wvz0pKn0d61hIurNZAoksOLXZWg1UHIetiZ/UgZj3n2VxJ8XWjn+BcNSCrx0jgd/yQ= X-Received: by 10.80.170.156 with SMTP id q28mr3634286edc.43.1521224403750; Fri, 16 Mar 2018 11:20:03 -0700 (PDT) MIME-Version: 1.0 Received: by 10.80.203.139 with HTTP; Fri, 16 Mar 2018 11:20:02 -0700 (PDT) In-Reply-To: References: From: Victor Huertas Date: Fri, 16 Mar 2018 19:20:02 +0100 Message-ID: To: "Van Haaren, Harry" Cc: "users@dpdk.org" Content-Type: text/plain; charset="UTF-8" X-Content-Filtered-By: Mailman/MimeDel 2.1.15 Subject: Re: [dpdk-users] running multiple pthreads in the same lcore id X-BeenThere: users@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK usage discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 16 Mar 2018 18:20:04 -0000 Thanks Harry for your quick response, Mmmm. your recommendation on lighweight threads makes me think on suitability of using DPDK in my case. It seems to me that "lightweight threads" is specially recommended when you have to execute in parallel many instances of the same working function (having the same per-packet processing load). Am I right on this assumption? The idea that comes to mind is to break the packet processing chaining application into several processes making then simpler (in terms of multi-threading) and independent packet processing feature. For example: - one process to only handle with packet-capture and packet tx through NICs,: here if I want to support RSS several threads shall be run. - one process to classify and conform the traffic: as many threads as the one run in the previous process to support pipelining. - one process to perform scheduling in order to distribute packet into queues: here DPDK scheduling qos library is not an option for me because I have already implemented my own WFQ and Strict priority Queue Server by my own being able to have hundreds and thousands of independent queues. Here it would be a good candidate for lightweight threading (one for each queue) for example. The following diagram depicts why I mean: [Process for Tx/Rx packets NICs] --> sw ring --> [Process for classif/conform] -->sw ring-->[Process for QoS scheculing] | | |<------------------------------------------- sw ring <---------------------------------------------------------------| Each process would be assigned it own lcores id thanks to the DPDK EAL initialization options. Despite this partition, I will have to launch a good number of phreads, so I am afraid that I will have to use non-EAL pthread functions. However, I use several customized packet metadata that must be preserved between processes (I would use the userdata pointer in the rte_mbuf metadata header in order not to loose the association: packet-->metadata along all the application). The key point here is to combine two things: non-EAL pthreading launching function and the DPDK software rings and memory pools that must be shared by ALL the proceses mentioned before. Do you agree? Regards, 2018-03-16 18:11 GMT+01:00 Van Haaren, Harry : > > From: users [mailto:users-bounces@dpdk.org] On Behalf Of Victor Huertas > > Sent: Friday, March 16, 2018 4:24 PM > > To: users@dpdk.org > > Subject: [dpdk-users] running multiple pthreads in the same lcore id > > > > Hi all, > > Hi Victor, > > > I have developed a simple application which implements a network bridge. > It > > captures packets in one port (port0), sends them into a software ring > > towards a second thread which receives them and sends them out via port1. > > > > Same implementation is performed in the direction port1->port0. So there > > are 4 threads in total. > > > > [port0]-->rx thread0 -->sw ring--> tx thread1-->[port 1] > > [port0]<--rx thread2 <--sw ring<-- tx thread3<--[port 1] > > > > The thing is that this application is a first step of a functionality > > expansion by inserting additional threads (all connected with software > > rings) in the middle of the path so that it performs some packet > > manipulation before delivering them to the destination port: > > > > [port0]-->rx thread0 -->sw ring--> add_thread 1-->sw ring--> add_thread > > N-->sw ring-> tx thread1-->[port 1] > > > > The total number of threads can reach hundreds (much more than number of > > lcores) and I would like to assign every thread to a particular lcore, no > > mattering if more than one thread is being executed by the same lcore. > This > > assignment would be done through an XML config file, which would be > loaded > > at the very beginning of the application execution. > > Understood. > > > > the rte_eal_remote_launch function at rte_launch.h can only launch one > > thread and attach it to one and unique lcore. So imagine that I want to > > launch another thread on the same lcore. DPDK doesn't seem to allow me to > > do that. > > Correct, DPDK uses core pinning as this enables higher performance by > reducing > overhead of context-switching between threads. > > > > I have read that using cgroups could make this possible but I haven't > found > > an example application where DPDK and cgroups are used toghether. > > I would advise to consider some form of "lightweight threads", which don't > actually > perform a thread-switch, but instead run a different work function on the > same > DPDK lcore. > > DPDK provides infrastructure for doing this - which you might like to > investigate. > The "service-cores" library enables running various functions on the same > DPDK lcore. > Refer to the examples/service_cores code, and you can see different work() > functions being > switched between DPDK lcores dynamically (aka, runtime movement of the > work between threads). > > > > what is your recommendation on this as expert users? Is using cgroups an > > acceptable option? > > > > Is DPDK contemplating in a future release to support natively multi > pthread > > on one lcore on the rte_launch.h lib? > > I'm not aware of any such ideas. > > Hope the above helps! -Harry > > > > Thanks for your attention, > > > > -- > > Victor > -- Victor