From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 73644A0C4B; Thu, 19 Aug 2021 23:33:52 +0200 (CEST) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 5B56B4013F; Thu, 19 Aug 2021 23:33:52 +0200 (CEST) Received: from mail-pl1-f181.google.com (mail-pl1-f181.google.com [209.85.214.181]) by mails.dpdk.org (Postfix) with ESMTP id C1BFA4003D for ; Thu, 19 Aug 2021 23:33:51 +0200 (CEST) Received: by mail-pl1-f181.google.com with SMTP id o10so4787497plg.0 for ; Thu, 19 Aug 2021 14:33:51 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=networkplumber-org.20150623.gappssmtp.com; s=20150623; h=date:from:to:cc:subject:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=eJzIKLzTylV2SZDt6sdyIHBbz7PDN2u/a90x0uBnF9c=; b=zBxwFZsVIIlJuzRZFIizMyAOPOicMyiHi+0fR3yDO2+ZmBc3HP77ELDbzu54JkuvZn c1c7mzyN3SL8wFEcSIrshXdtF23q/cujxd4eBVzCiOGMlpCV8+b0wnxQSqjh7AJDPQ4C +NUM1FeGZJHpm4s5UUDMJdgAswsVsU7eGUYfdm640knKBDaTHMyK8pP+wl3xd0zDSRjP KmEEvzX3KfP9Pg+0zfCLGvadXvi/XnWbUgoBQ5hBZiMh7cBwQIHNGWf96Yl8RK8eqLy/ NhcsOjp2iFonOsoRRZ+hCHUCKiN5V5Q6ZNfoRlPzHUkIXTObWRJD8wBAy+SrCCunKXOW vM8Q== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:from:to:cc:subject:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=eJzIKLzTylV2SZDt6sdyIHBbz7PDN2u/a90x0uBnF9c=; b=uLSmfaDY1W5WS9GDKpreRKdSDX+ZSsTd+a3t+rhTHAjHM8bSpcoDOUY17nKeA1f56/ C6tcfpTI5xL7RYz0mdZQ7HAnDystYhg8Qd/chYG1/+RZZB7eL87C8Bnkj5L3YuIBUBOB RYj0VJo06tbbdrsYjTZDeOHOblfr9fLqI/wZC8veYRjgs1kBSj5kobS5F8ZwNYN1DzFR ZTPQhxqvCNTTs77GIR90a4rSpUDMGEOKnTGNmSUOCxmgvc75rPwaBglBznZ6AUCyDhZQ 82xuz4Z77+LU5q0R++qizi4JxX6VydUo+KZFlzFXekXgWtic9bh8cvgB3HmdHmxH8Ot8 MWwQ== X-Gm-Message-State: AOAM5311lxxVNMERJEgmAEG1+VLDkCBt6HCfL2mS+BuZ5zQPrwT4CVY8 2C4ZFgZmA4iZZq/j7HdCkFLgLg== X-Google-Smtp-Source: ABdhPJzaPD8P7KyUYtJaX4O99fsmhbbWDxQ5oi3/jmfAEHShbsMPwhIuQUBaUvZcDZfMlnDGmy7SZQ== X-Received: by 2002:a17:902:8f90:b029:12d:1b48:efd8 with SMTP id z16-20020a1709028f90b029012d1b48efd8mr13743141plo.23.1629408830915; Thu, 19 Aug 2021 14:33:50 -0700 (PDT) Received: from hermes.local (204-195-33-123.wavecable.com. [204.195.33.123]) by smtp.gmail.com with ESMTPSA id r9sm4560314pfh.135.2021.08.19.14.33.50 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 19 Aug 2021 14:33:50 -0700 (PDT) Date: Thu, 19 Aug 2021 14:33:47 -0700 From: Stephen Hemminger To: Narcisa Ana Maria Vasile Cc: Bruce Richardson , dev@dpdk.org, thomas@monjalon.net, dmitry.kozliuk@gmail.com, khot@microsoft.com, navasile@microsoft.com, dmitrym@microsoft.com, roretzla@microsoft.com, talshn@nvidia.com, ocardona@microsoft.com, david.marchand@redhat.com, pallavi.kadam@intel.com Message-ID: <20210819143347.1f47d17d@hermes.local> In-Reply-To: <20210819213019.GA26150@linuxonhyperv3.guj3yctzbm1etfxqx2vob5hsef.xx.internal.cloudapp.net> References: <1627925546-29982-1-git-send-email-navasile@linux.microsoft.com> <1628017291-3756-1-git-send-email-navasile@linux.microsoft.com> <1628017291-3756-10-git-send-email-navasile@linux.microsoft.com> <20210818142833.73027ecf@hermes.local> <20210819213019.GA26150@linuxonhyperv3.guj3yctzbm1etfxqx2vob5hsef.xx.internal.cloudapp.net> MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Subject: Re: [dpdk-dev] [PATCH v13 09/10] eal: add EAL argument for setting thread priority X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" On Thu, 19 Aug 2021 14:30:19 -0700 Narcisa Ana Maria Vasile wrote: > On Thu, Aug 19, 2021 at 10:06:06AM +0100, Bruce Richardson wrote: > > On Wed, Aug 18, 2021 at 02:28:33PM -0700, Stephen Hemminger wrote: > > > On Tue, 3 Aug 2021 12:01:30 -0700 Narcisa Ana Maria Vasile > > > wrote: > > > > > > > +static int +eal_parse_thread_priority(const char *arg) +{ + > > > > struct internal_config *internal_conf = + > > > > eal_get_internal_configuration(); + enum rte_thread_priority priority; > > > > + + if (!strncmp("normal", arg, sizeof("normal"))) + > > > > priority = RTE_THREAD_PRIORITY_NORMAL; + else if > > > > (!strncmp("realtime", arg, sizeof("realtime"))) + priority = > > > > RTE_THREAD_PRIORITY_REALTIME_CRITICAL; + else + return -1; > > > > + + internal_conf->thread_priority = priority; + return 0; +} + > > > > > > In my experience using real time priority is dangerous and risks > > > starvation and deadlock. The problem is that DPDK applications are > > > typically 100% CPU poll mode with no system calls; but the kernel has a > > > number of worker threads that can be required on those CPUs. > > > > > > The typical failure is a disk completion interrupt happens on a CPU that > > > is being used by DPDK lcore thread. With RT priority, the kernel thread > > > to process that I/O completion never runs because the RT user thread has > > > higher priority than the kernel I/O completion thread. > > > > > > It maybe possible to workaround this with lots of hand crafting through > > > sysfs to reassign threads and irq's. Also, later kernels with full RT > > > might also help. > > > > > > Before putting this in as part of DPDK in EAL, a full set of testing and > > > documentation of how to configure these kind of applications and systems > > > is needed. > > > > > I would tend to agree caution here, based on my experience of having locked > > up a number of systems in the past when testing running DPDK apps with RT > > priority! > > Thank you for the comments! I've added this option since it was requested by > multiple users. I understand RT priority causes issues on Linux platforms. > On Windows we want to be able to use REALTIME priority in certain scenarios. > > Would it be acceptable to replace this option with a "HIGH_PRIORITY" one > and keep it realtime on Windows and choose a higher (but non-realtime) option on Linux? > However, there are 2 issues here: > * We will have different behaviors between the 2 platforms. > * Not sure if I can set a normal but higher priority on Linux. SCHED_OTHER only allows > one value and Linux "nice" values don't help. If anyone knows of a way to accomplish > this on Linux, please do advise. > Alternatively, we can have this option for Windows only. > > In the meantime, I've removed this patch from this patchset in v14 as the cmdline option is not > being enabled yet, as DmitryK noted. > > I think on Linux it should produce a big a*** warning message.