From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 0FEB5A057F for ; Wed, 19 Oct 2022 01:20:06 +0200 (CEST) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id A01E44069C; Wed, 19 Oct 2022 01:20:05 +0200 (CEST) Received: from mail-pj1-f50.google.com (mail-pj1-f50.google.com [209.85.216.50]) by mails.dpdk.org (Postfix) with ESMTP id 3D52C40041 for ; Wed, 19 Oct 2022 01:20:04 +0200 (CEST) Received: by mail-pj1-f50.google.com with SMTP id cl1so15361955pjb.1 for ; Tue, 18 Oct 2022 16:20:04 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=networkplumber-org.20210112.gappssmtp.com; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:subject:cc:to:from:date:from:to:cc:subject:date :message-id:reply-to; bh=P6TVxWbYMZBrkGANlWLg3Aw1tKf9QLsjtH76JBMuovw=; b=PJfWQGcP1IDrHDuKrUI+bBFL1fzKt8j445VMvmNmU/AjB+300V2gnal1K17yPcF8xA ZqBQ6XdNqJFa2IcNg84kio3jYGAvZ3zOIsim/Nij5fAfuj3st47y0g9cBRV+Ce/eladt km+Dt9mKBC2wtCKhtI0xlLS+qIBr+YHXSXoL0IyGVhrnEVhwPHRRc40W/qqeWIrwX861 Ilu8VlrSfQ994XecyLRiPatNFMBojMxSXkGjsiss7iRzLirc2qcsrSDDo4/vmQ8qotem I5KADEVwm5WV4A1VloRme3OZx5fprdVLB0Jz+O2bvObWHR6xfbqUxx705hwWpOp8jHwF PWPw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:subject:cc:to:from:date:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=P6TVxWbYMZBrkGANlWLg3Aw1tKf9QLsjtH76JBMuovw=; b=hPzEN8tFDA5Sx2Q6Uk7Gl4YOWXa7XvAcDo3X41g/FOUgeJURiU/6HK6XvC5AB8qfIg TR6FGP2Rb+rVy8uvB/usGLgQ57ra7iyTi/sP7epfWnz/ESNqjDiHgdxNZnGRwCOJTdsk M0blYQGx9WiA58Zr6vtXfHrtCt1SQzgFBYj2bw+UE4o5gNcATnz/9SbUX03ef/gTOTTQ YK9lpj9NCC8TJ29DAp3Sngz8ssXpSrRI2ggQri8Shg/LS3TKD/TD7ucR4y0QMKmDDfdh bvvUv+M54j8AoXgxXmJ8hc2BEYIz8sF3LyA+e8mS9khtUUP6z3eOCjw1LThjDb4sXULk 1KcA== X-Gm-Message-State: ACrzQf25qA7eRTj01Ad2rEk2fvtgyw0fCCXvzOWN88jRNskU5Pegy/7M 2c7PIkOlPT46FCrjwwfzgPs1Bg== X-Google-Smtp-Source: AMsMyM6La5eEHh/KhKJi/DyQaXWdRnpBEqs1XUPpU6CCGUzf01HgjwQjgj1RXWanyYy8Mjdm6p/+/w== X-Received: by 2002:a17:90b:4d0e:b0:1f7:ae99:4d7f with SMTP id mw14-20020a17090b4d0e00b001f7ae994d7fmr41146605pjb.200.1666135203213; Tue, 18 Oct 2022 16:20:03 -0700 (PDT) Received: from hermes.local (204-195-120-218.wavecable.com. [204.195.120.218]) by smtp.gmail.com with ESMTPSA id g13-20020aa79dcd000000b005668b26ade0sm9485263pfq.136.2022.10.18.16.20.02 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 18 Oct 2022 16:20:03 -0700 (PDT) Date: Tue, 18 Oct 2022 16:20:01 -0700 From: Stephen Hemminger To: "Juan Pablo L." Cc: "users@dpdk.org" Subject: Re: lcores clarification Message-ID: <20221018162001.58965de1@hermes.local> In-Reply-To: References: <20221018154106.101b064f@hermes.local> MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit X-BeenThere: users@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK usage discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: users-bounces@dpdk.org On Tue, 18 Oct 2022 23:04:48 +0000 Juan Pablo L. wrote: > Hi Stephen, thank you for your advise, I understand that, like you said the same applies to other things not only dpdk, but I was asking to run different lcores on a CPU .. I understand that a lcore is a thread (pthread in this case) running on a CPU .. I also understand that many lcores can run on same CPU( right ?) and said lcores have sperate per-lcore variables, env, etc so one lcore is a single thread running on a while(resource is busy) NOT sharing anything with the other lcores running on same CPU that is each lcore has its own ring for example waiting for packages coming from different sockets, etc ... and the lcore is NOT running threads inside or anything like that ... > > I have to admit that I am trying to understand this whole lcore,CPU, scheduling, etc in DPDK still ... thank you for your input! > ________________________________ > From: Stephen Hemminger > Sent: Tuesday, October 18, 2022 10:41 PM > To: Juan Pablo L. > Cc: users@dpdk.org > Subject: Re: lcores clarification > > On Tue, 18 Oct 2022 22:06:04 +0000 > Juan Pablo L. wrote: > > > Hellos guys, I have a doubt that I have not been able to clarify with the docs, so maybe you can help me please ... > > > > I have a process that I want to run on a set of CPUs (performing the same task) but I want to run all of them at the same time, so my question is: > > > > if I configure something like "--lcores='2@(5-7)'" that will make my lcore with id 2 run simultaneously (at the same time in parallel) on CPUs 5,6,7 or it will run my lcore id 2 only one time at any given time on either CPU 5,6 or 7 ? > > > > would it be better if I configure different lcore id for the same and assign it to individual CPUs ? , e.g: "--lcores='2@5,3@6,4@7'" even though lcore id 2,3,4 are really the same job ? > > > > I think my question is just a matter of what is the best way to configure EAL to accomplish what I want, that is to run the same process on all CPU available concurrently, even though, I believe, option 2 seems to be the obvious way I just want to make sure I am not doing something that DPDK could do for me already .... > > > > A bonus question if its not too much, since lcores are really pthreads (in linux) and I have isolated the CPUs from the kernel scheduler, my guess is that if I have different lcores (performing the same or different tasks) running on the same CPU DPDK will be doing the scheduling right ? .. > > > > thank you very much!!!! > > > > If you have multiple threads running on same lcore, you better not use locks or rings, > or mbufs, or most of the DPDK. Because these features all work like: > > while (resource is busy) > spin wait > > So if one thread has a resource and gets preempted. When another thread runs > and tries to acquire the same resource; the thread will spin until scheduler > decides to run the other thread. > > This is not unique to DPDK, the same problem is described in the pthread_spinlock > man page. Running multiple DPDK lcores on same cpu risks the scheduler hitting you and will hurt performance. Running a DPDK lcore with an affinity cpuset that spans multiple cpu's will not be faster. It just will cause the scheduler to move that thread among cpus. The fastest is to do the default and pin the lcore to a single cpu; and use CPU isolation so that no other process runs on that CPU. Sure you can do lots of weird and wonderful things to rearrange lcores but your just making the application unstable.