From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mail.lysator.liu.se (mail.lysator.liu.se [130.236.254.3]) by dpdk.org (Postfix) with ESMTP id 662701B39B for ; Mon, 7 Jan 2019 16:36:16 +0100 (CET) Received: from mail.lysator.liu.se (localhost [127.0.0.1]) by mail.lysator.liu.se (Postfix) with ESMTP id A7C6F40009 for ; Mon, 7 Jan 2019 16:36:15 +0100 (CET) Received: by mail.lysator.liu.se (Postfix, from userid 1004) id 90B5C40005; Mon, 7 Jan 2019 16:36:15 +0100 (CET) X-Spam-Checker-Version: SpamAssassin 3.4.1 (2015-04-28) on bernadotte.lysator.liu.se X-Spam-Level: X-Spam-Status: No, score=-0.9 required=5.0 tests=ALL_TRUSTED,AWL autolearn=disabled version=3.4.1 X-Spam-Score: -0.9 Received: from [192.168.1.59] (host-90-232-172-103.mobileonline.telia.com [90.232.172.103]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.lysator.liu.se (Postfix) with ESMTPSA id 4DFC140004; Mon, 7 Jan 2019 16:36:14 +0100 (CET) To: Venky Venkatesh , "dev@dpdk.org" References: <2D68DFF2-08A3-403B-9570-43D4AD916FD2@paloaltonetworks.com> <8C05BF3C-5D90-4CD0-AA71-1B4C4324E53B@paloaltonetworks.com> <597ee5d1-6db1-54ee-36af-df28ef1654bb@ericsson.com> <2ED77A34-B171-49BF-966A-BAE6A9E1BC66@paloaltonetworks.com> From: =?UTF-8?Q?Mattias_R=c3=b6nnblom?= Message-ID: Date: Mon, 7 Jan 2019 16:36:13 +0100 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:60.0) Gecko/20100101 Thunderbird/60.2.1 MIME-Version: 1.0 In-Reply-To: <2ED77A34-B171-49BF-966A-BAE6A9E1BC66@paloaltonetworks.com> Content-Type: text/plain; charset=utf-8; format=flowed Content-Language: en-US Content-Transfer-Encoding: 8bit X-Virus-Scanned: ClamAV using ClamSMTP Subject: Re: [dpdk-dev] DSW eventdev and multi-process DPDK X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 07 Jan 2019 15:36:16 -0000 On 2018-12-21 20:12, Venky Venkatesh wrote: > > > On 12/21/18, 10:59 AM, "Mattias Rönnblom" wrote: > > On 2018-12-21 19:34, Venky Venkatesh wrote: > > > > > > On 12/21/18, 10:24 AM, "Mattias Rönnblom" wrote: > > > > On 2018-12-21 06:13, Venky Venkatesh wrote: > > > Hi, > > > We are considering using a multi-process mode of the DPDK with the event generators and consumers being spread across multiple processes (on different cores). We are also considering using the DSW eventdev. Is the DSW designed for such a use case? If so, are there some restrictions and something specific that need to be done to make it work correctly? > > > > > > > The purpose of an event device is to do dynamic load balancing across > > multiple cores. Using the DPDK multiple-process support, with its > > requirement of having unique, non-overlapping, core masks works against > > or even defeats this purpose. > > > > [VV]: I don’t understand your last sentence. Suppose I am having multiple packet processing processes (each with a single thread and polling a disjoint set of queues) and each linked to DSW. Each process would invoke the enqueue which will be handled by the DSW linked to that process. Will the DSWs across these processes "collaborate" to get load balancing across the processes? > > > > If the processes are to collaborate, and process packets in the same > pipeline, they will need to share an event device (for example, a DSW > instance). > > However, if you put each of your pipeline stages into a process with a > single worker thread, you will not leave any room for an event device to > load balance, since every eventdev queue will have only a single > consumer linked to it. > > [VV]: Sorry for the ambiguous terminology used by me -- queue (above) referred to port queues and not eventdev queues. Additionally, consider a very simple pipeline -- just 1 stage followed by transmit. Thus each process is pulling packets out of the port queue, enqueue into local DSW, dequeue from local DSW and running this 1 stage pipeline and transmitting. The role of eventdev in this world is to load balance across the processes -- that is what I meant by DSWs collaborate (since they need to exchange load information and do migration handshake). Hope that clarifies. Pls let me know if this will work. I'm not familiar with the details of DPDK multiprocess support, but I think this should work. Again, the DSW instance needs to be shared, and can't be local to the process in case you want to use it to load balance across different DPDK processes. All of the huge page memory is shared, and that's the only memory a DSW event device is using (except for execution stacks of course, which of course doesn't have to be shared).