From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 77B07A0C52; Wed, 18 Aug 2021 23:28:38 +0200 (CEST) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id F00024069E; Wed, 18 Aug 2021 23:28:37 +0200 (CEST) Received: from mail-pg1-f173.google.com (mail-pg1-f173.google.com [209.85.215.173]) by mails.dpdk.org (Postfix) with ESMTP id 453F74067E for ; Wed, 18 Aug 2021 23:28:37 +0200 (CEST) Received: by mail-pg1-f173.google.com with SMTP id e7so3699970pgk.2 for ; Wed, 18 Aug 2021 14:28:37 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=networkplumber-org.20150623.gappssmtp.com; s=20150623; h=date:from:to:cc:subject:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=a+L9/STc/VowSo/nEGq7I++5bBLaVOEm19fCAUyc0Hs=; b=xOvgMrsxDlLD23kVEsRItAjjysb0+zbeLb3Ty4fUtRBwztVqZOqI1+fdc6m7nHG3gY 8qyYeKwpxOZCdlKKWlvjZ1z3BgOk2gupewGoedoF31iv3vXA5qI0aDu3q/poKomZ09at MOTFcuDJOapZYpEbX3G2ITUcB0J57WYPuP0L9vqRmD0eC/+xWLo1HSlg2xinUzERKXTt yN9GSzcbr2Ht58D+i2Qc45Qzhi3n5nELI4/1FOm0meUsyzOayn9E0DQTsV2SUqgwQCy1 XVLCwE2paG1gwQLfPUQJ/bkjo2HpKo9HY7FcDC/wVCD8g6XH4Fs6QSx8JHNFbsuWDtRd 2Gzg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:from:to:cc:subject:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=a+L9/STc/VowSo/nEGq7I++5bBLaVOEm19fCAUyc0Hs=; b=oAA19qkwfRaKF4Ihx3aWJ15KDGFh/fgGyPydMGLdJ9COG9gBFm6hj7spw+XU9e45iB ma3K1Klpxkwbr8isjdHFeupRkqPfd1IRVIF0YB72mcOJWlJd21y5nEy03QAZVzdc1+rc PcCQACQi7WZNW6/VfwrbXwWfX9jEYXagV1stIlvS+/EhPqQFNJYDtZssDLrjKt+mCb/S uu7dhXZrxXS4yrz+k83L4AJHwzCBX8j40U3lFqcueD4+E4lb8cQfh6BRsWFGj4ysGY5T NzQXPqku2GQY6Y/njrqCzUWmSdDYRIq3NVDUfoZnf1RzGAM/JsYFEvbWsI/GLhWKyU7+ zs9Q== X-Gm-Message-State: AOAM531lAYrP2JKAqPYgelyLT2xZi2Omyc05BijPqx1up3NXPYktcr7F AfP3/VFM+C//LYSkS6mcFosO9Q== X-Google-Smtp-Source: ABdhPJzq0bN8DI370jST/xzC+CbOjwkEuXb1kfquIljXKd5XPX1DjX8erB/IKNI4M7Z0h+sQiURFIQ== X-Received: by 2002:a63:d14c:: with SMTP id c12mr10669611pgj.412.1629322116407; Wed, 18 Aug 2021 14:28:36 -0700 (PDT) Received: from hermes.local (204-195-33-123.wavecable.com. [204.195.33.123]) by smtp.gmail.com with ESMTPSA id ep21sm711592pjb.18.2021.08.18.14.28.35 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 18 Aug 2021 14:28:36 -0700 (PDT) Date: Wed, 18 Aug 2021 14:28:33 -0700 From: Stephen Hemminger To: Narcisa Ana Maria Vasile Cc: dev@dpdk.org, thomas@monjalon.net, dmitry.kozliuk@gmail.com, khot@microsoft.com, navasile@microsoft.com, dmitrym@microsoft.com, roretzla@microsoft.com, talshn@nvidia.com, ocardona@microsoft.com, bruce.richardson@intel.com, david.marchand@redhat.com, pallavi.kadam@intel.com Message-ID: <20210818142833.73027ecf@hermes.local> In-Reply-To: <1628017291-3756-10-git-send-email-navasile@linux.microsoft.com> References: <1627925546-29982-1-git-send-email-navasile@linux.microsoft.com> <1628017291-3756-1-git-send-email-navasile@linux.microsoft.com> <1628017291-3756-10-git-send-email-navasile@linux.microsoft.com> MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Subject: Re: [dpdk-dev] [PATCH v13 09/10] eal: add EAL argument for setting thread priority X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" On Tue, 3 Aug 2021 12:01:30 -0700 Narcisa Ana Maria Vasile wrote: > +static int > +eal_parse_thread_priority(const char *arg) > +{ > + struct internal_config *internal_conf = > + eal_get_internal_configuration(); > + enum rte_thread_priority priority; > + > + if (!strncmp("normal", arg, sizeof("normal"))) > + priority = RTE_THREAD_PRIORITY_NORMAL; > + else if (!strncmp("realtime", arg, sizeof("realtime"))) > + priority = RTE_THREAD_PRIORITY_REALTIME_CRITICAL; > + else > + return -1; > + > + internal_conf->thread_priority = priority; > + return 0; > +} > + In my experience using real time priority is dangerous and risks starvation and deadlock. The problem is that DPDK applications are typically 100% CPU poll mode with no system calls; but the kernel has a number of worker threads that can be required on those CPUs. The typical failure is a disk completion interrupt happens on a CPU that is being used by DPDK lcore thread. With RT priority, the kernel thread to process that I/O completion never runs because the RT user thread has higher priority than the kernel I/O completion thread. It maybe possible to workaround this with lots of hand crafting through sysfs to reassign threads and irq's. Also, later kernels with full RT might also help. Before putting this in as part of DPDK in EAL, a full set of testing and documentation of how to configure these kind of applications and systems is needed. I can't recommend this going in at this time.