From mboxrd@z Thu Jan 1 00:00:00 1970
Return-Path: <dev-bounces@dpdk.org>
Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124])
by inbox.dpdk.org (Postfix) with ESMTP id 3B0C0A0C47;
Thu, 19 Aug 2021 11:06:16 +0200 (CEST)
Received: from [217.70.189.124] (localhost [127.0.0.1])
by mails.dpdk.org (Postfix) with ESMTP id B0E424067E;
Thu, 19 Aug 2021 11:06:15 +0200 (CEST)
Received: from mga04.intel.com (mga04.intel.com [192.55.52.120])
by mails.dpdk.org (Postfix) with ESMTP id 23F7D40141
for <dev@dpdk.org>; Thu, 19 Aug 2021 11:06:13 +0200 (CEST)
X-IronPort-AV: E=McAfee;i="6200,9189,10080"; a="214671820"
X-IronPort-AV: E=Sophos;i="5.84,334,1620716400"; d="scan'208";a="214671820"
Received: from fmsmga007.fm.intel.com ([10.253.24.52])
by fmsmga104.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384;
19 Aug 2021 02:06:13 -0700
X-IronPort-AV: E=Sophos;i="5.84,334,1620716400"; d="scan'208";a="451321909"
Received: from bricha3-mobl.ger.corp.intel.com ([10.252.21.121])
by fmsmga007-auth.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-SHA;
19 Aug 2021 02:06:10 -0700
Date: Thu, 19 Aug 2021 10:06:06 +0100
From: Bruce Richardson <bruce.richardson@intel.com>
To: Stephen Hemminger <stephen@networkplumber.org>
Cc: Narcisa Ana Maria Vasile <navasile@linux.microsoft.com>, dev@dpdk.org,
thomas@monjalon.net, dmitry.kozliuk@gmail.com, khot@microsoft.com,
navasile@microsoft.com, dmitrym@microsoft.com,
roretzla@microsoft.com, talshn@nvidia.com, ocardona@microsoft.com,
david.marchand@redhat.com, pallavi.kadam@intel.com
Message-ID: <YR4e/pbVcoE21eHA@bricha3-MOBL.ger.corp.intel.com>
References: <1627925546-29982-1-git-send-email-navasile@linux.microsoft.com>
<1628017291-3756-1-git-send-email-navasile@linux.microsoft.com>
<1628017291-3756-10-git-send-email-navasile@linux.microsoft.com>
<20210818142833.73027ecf@hermes.local>
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
In-Reply-To: <20210818142833.73027ecf@hermes.local>
Subject: Re: [dpdk-dev] [PATCH v13 09/10] eal: add EAL argument for setting
thread priority
X-BeenThere: dev@dpdk.org
X-Mailman-Version: 2.1.29
Precedence: list
List-Id: DPDK patches and discussions <dev.dpdk.org>
List-Unsubscribe: <https://mails.dpdk.org/options/dev>,
<mailto:dev-request@dpdk.org?subject=unsubscribe>
List-Archive: <http://mails.dpdk.org/archives/dev/>
List-Post: <mailto:dev@dpdk.org>
List-Help: <mailto:dev-request@dpdk.org?subject=help>
List-Subscribe: <https://mails.dpdk.org/listinfo/dev>,
<mailto:dev-request@dpdk.org?subject=subscribe>
Errors-To: dev-bounces@dpdk.org
Sender: "dev" <dev-bounces@dpdk.org>
On Wed, Aug 18, 2021 at 02:28:33PM -0700, Stephen Hemminger wrote:
> On Tue, 3 Aug 2021 12:01:30 -0700 Narcisa Ana Maria Vasile
> <navasile@linux.microsoft.com> wrote:
>
> > +static int +eal_parse_thread_priority(const char *arg) +{ +
> > struct internal_config *internal_conf = +
> > eal_get_internal_configuration(); + enum rte_thread_priority priority;
> > + + if (!strncmp("normal", arg, sizeof("normal"))) +
> > priority = RTE_THREAD_PRIORITY_NORMAL; + else if
> > (!strncmp("realtime", arg, sizeof("realtime"))) + priority =
> > RTE_THREAD_PRIORITY_REALTIME_CRITICAL; + else + return -1;
> > + + internal_conf->thread_priority = priority; + return 0; +} +
>
> In my experience using real time priority is dangerous and risks
> starvation and deadlock. The problem is that DPDK applications are
> typically 100% CPU poll mode with no system calls; but the kernel has a
> number of worker threads that can be required on those CPUs.
>
> The typical failure is a disk completion interrupt happens on a CPU that
> is being used by DPDK lcore thread. With RT priority, the kernel thread
> to process that I/O completion never runs because the RT user thread has
> higher priority than the kernel I/O completion thread.
>
> It maybe possible to workaround this with lots of hand crafting through
> sysfs to reassign threads and irq's. Also, later kernels with full RT
> might also help.
>
> Before putting this in as part of DPDK in EAL, a full set of testing and
> documentation of how to configure these kind of applications and systems
> is needed.
>
I would tend to agree caution here, based on my experience of having locked
up a number of systems in the past when testing running DPDK apps with RT
priority!