From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 5B442A0C4B; Thu, 19 Aug 2021 23:30:22 +0200 (CEST) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id DA0254013F; Thu, 19 Aug 2021 23:30:21 +0200 (CEST) Received: from linux.microsoft.com (linux.microsoft.com [13.77.154.182]) by mails.dpdk.org (Postfix) with ESMTP id 4FB854003D for ; Thu, 19 Aug 2021 23:30:20 +0200 (CEST) Received: by linux.microsoft.com (Postfix, from userid 1059) id 9410620C33B4; Thu, 19 Aug 2021 14:30:19 -0700 (PDT) DKIM-Filter: OpenDKIM Filter v2.11.0 linux.microsoft.com 9410620C33B4 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linux.microsoft.com; s=default; t=1629408619; bh=li1qrNyUM7PZRKSxGvm04LntmdrNQXvrscVY1/z1oh8=; h=Date:From:To:Cc:Subject:References:In-Reply-To:From; b=B6OLdYdgo1F4tuMAQIxV4prlp40/Dw+Q4LMUwU/qPkywa1vJz3Wda9howpSUoEi6B tXSrH2cHc49Ij8X5tUChiEtV1yYexiEbDDTSg1hqnQJWi+t759f31EKkgWldpAKW5p bOlvYnVacu68aM9p41vAmHT6/9Zm4Bigq1i9ahcc= Date: Thu, 19 Aug 2021 14:30:19 -0700 From: Narcisa Ana Maria Vasile To: Bruce Richardson Cc: Stephen Hemminger , dev@dpdk.org, thomas@monjalon.net, dmitry.kozliuk@gmail.com, khot@microsoft.com, navasile@microsoft.com, dmitrym@microsoft.com, roretzla@microsoft.com, talshn@nvidia.com, ocardona@microsoft.com, david.marchand@redhat.com, pallavi.kadam@intel.com Message-ID: <20210819213019.GA26150@linuxonhyperv3.guj3yctzbm1etfxqx2vob5hsef.xx.internal.cloudapp.net> References: <1627925546-29982-1-git-send-email-navasile@linux.microsoft.com> <1628017291-3756-1-git-send-email-navasile@linux.microsoft.com> <1628017291-3756-10-git-send-email-navasile@linux.microsoft.com> <20210818142833.73027ecf@hermes.local> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: User-Agent: Mutt/1.5.21 (2010-09-15) Subject: Re: [dpdk-dev] [PATCH v13 09/10] eal: add EAL argument for setting thread priority X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" On Thu, Aug 19, 2021 at 10:06:06AM +0100, Bruce Richardson wrote: > On Wed, Aug 18, 2021 at 02:28:33PM -0700, Stephen Hemminger wrote: > > On Tue, 3 Aug 2021 12:01:30 -0700 Narcisa Ana Maria Vasile > > wrote: > > > > > +static int +eal_parse_thread_priority(const char *arg) +{ + > > > struct internal_config *internal_conf = + > > > eal_get_internal_configuration(); + enum rte_thread_priority priority; > > > + + if (!strncmp("normal", arg, sizeof("normal"))) + > > > priority = RTE_THREAD_PRIORITY_NORMAL; + else if > > > (!strncmp("realtime", arg, sizeof("realtime"))) + priority = > > > RTE_THREAD_PRIORITY_REALTIME_CRITICAL; + else + return -1; > > > + + internal_conf->thread_priority = priority; + return 0; +} + > > > > In my experience using real time priority is dangerous and risks > > starvation and deadlock. The problem is that DPDK applications are > > typically 100% CPU poll mode with no system calls; but the kernel has a > > number of worker threads that can be required on those CPUs. > > > > The typical failure is a disk completion interrupt happens on a CPU that > > is being used by DPDK lcore thread. With RT priority, the kernel thread > > to process that I/O completion never runs because the RT user thread has > > higher priority than the kernel I/O completion thread. > > > > It maybe possible to workaround this with lots of hand crafting through > > sysfs to reassign threads and irq's. Also, later kernels with full RT > > might also help. > > > > Before putting this in as part of DPDK in EAL, a full set of testing and > > documentation of how to configure these kind of applications and systems > > is needed. > > > I would tend to agree caution here, based on my experience of having locked > up a number of systems in the past when testing running DPDK apps with RT > priority! Thank you for the comments! I've added this option since it was requested by multiple users. I understand RT priority causes issues on Linux platforms. On Windows we want to be able to use REALTIME priority in certain scenarios. Would it be acceptable to replace this option with a "HIGH_PRIORITY" one and keep it realtime on Windows and choose a higher (but non-realtime) option on Linux? However, there are 2 issues here: * We will have different behaviors between the 2 platforms. * Not sure if I can set a normal but higher priority on Linux. SCHED_OTHER only allows one value and Linux "nice" values don't help. If anyone knows of a way to accomplish this on Linux, please do advise. Alternatively, we can have this option for Windows only. In the meantime, I've removed this patch from this patchset in v14 as the cmdline option is not being enabled yet, as DmitryK noted.