From mboxrd@z Thu Jan 1 00:00:00 1970
Return-Path: <dev-bounces@dpdk.org>
Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124])
by inbox.dpdk.org (Postfix) with ESMTP id 5D9DBA0A02;
Thu, 14 Jan 2021 10:36:51 +0100 (CET)
Received: from [217.70.189.124] (localhost [127.0.0.1])
by mails.dpdk.org (Postfix) with ESMTP id D85C814101F;
Thu, 14 Jan 2021 10:36:50 +0100 (CET)
Received: from us-smtp-delivery-124.mimecast.com
(us-smtp-delivery-124.mimecast.com [63.128.21.124])
by mails.dpdk.org (Postfix) with ESMTP id 75EE114101E
for <dev@dpdk.org>; Thu, 14 Jan 2021 10:36:49 +0100 (CET)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com;
s=mimecast20190719; t=1610617008;
h=from:from:reply-to:subject:subject:date:date:message-id:message-id:
to:to:cc:cc:mime-version:mime-version:content-type:content-type:
content-transfer-encoding:content-transfer-encoding:
in-reply-to:in-reply-to:references:references;
bh=GZevHUMqFZOAGKSRlEe5F1ids66ZXmWfu2LXOsE5QnU=;
b=LDVlLDtWaZJ+rvtK0fNSry7KHfI6hoVMjxvDPYaMldY+75AqIUtCmJlZ9/bwOIecrgnZ1j
NWcjL9lBS6Lsmc/jLQSFgfB2nCSERQAQ4GITrM3cO1Dx3F0xinCRKrM4SzfJDQpdqDuzwW
34YveR/XJJpxzGs+JMlFOyhWwCmwTgU=
Received: from mail-vk1-f200.google.com (mail-vk1-f200.google.com
[209.85.221.200]) (Using TLS) by relay.mimecast.com with ESMTP id
us-mta-440-uaLHFS-pOKONnim55WZWAg-1; Thu, 14 Jan 2021 04:36:45 -0500
X-MC-Unique: uaLHFS-pOKONnim55WZWAg-1
Received: by mail-vk1-f200.google.com with SMTP id h21so2114984vke.22
for <dev@dpdk.org>; Thu, 14 Jan 2021 01:36:45 -0800 (PST)
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
d=1e100.net; s=20161025;
h=x-gm-message-state:mime-version:references:in-reply-to:from:date
:message-id:subject:to:cc:content-transfer-encoding;
bh=GZevHUMqFZOAGKSRlEe5F1ids66ZXmWfu2LXOsE5QnU=;
b=Dlf3P7QAirc5WE2k4Dvp1toF1ALgUumDB/0htA2Lka9shl5pgVFEgRmk7w2usGejm8
XHzu4Z/52JwM3APhx3nUYEkaekSgP0lUysGxuoXytZS8FUvv6Oj7DPLaSkgNr6qLTFfw
4OurKb/JZlZxGLDpqaLFCyPn4zYDdSEHSwc+BbRpLXgv+wJ9oemj3bVlm3mG0Q9B6E8J
ZFKN2+lXM5TVolx7SsOGqj2xpx+hNQzaLHzKVhMV3oV78X67xE4C2Mj8MDjkmYF9UP1W
08BoNW8si8uMallZ8LyAumgcTHSvWgaBbNo6zHSgjBHROH8NycMWwNKj4Yuq7v1ydV+c
qf7Q==
X-Gm-Message-State: AOAM531OIoXkzd4wRL8PaMjyesB+VSuSJyz8Ws0z1MydMPvdYgOrb0eG
MVWSj+DCimLdzcUh+P4VToCIu3ArAU3w81UFI/Uw+N0/2VSEnV1HF6dGRDGHT5mh1NskTFqm25C
V7YX35hB/FlVSq1r8Fmo=
X-Received: by 2002:a67:3093:: with SMTP id w141mr661124vsw.27.1610617004641;
Thu, 14 Jan 2021 01:36:44 -0800 (PST)
X-Google-Smtp-Source: ABdhPJx7lqa6xJYKhnB1lBD/TAPhE/a+v3g4am2EVA5CGXE3afziQgu4yosU4gTb4EuKM2mGq414kaHYdk5CW7uPJTk=
X-Received: by 2002:a67:3093:: with SMTP id w141mr661101vsw.27.1610617004249;
Thu, 14 Jan 2021 01:36:44 -0800 (PST)
MIME-Version: 1.0
References: <cover.1610377084.git.anatoly.burakov@intel.com>
<cover.1610473000.git.anatoly.burakov@intel.com>
In-Reply-To: <cover.1610473000.git.anatoly.burakov@intel.com>
From: David Marchand <david.marchand@redhat.com>
Date: Thu, 14 Jan 2021 10:36:32 +0100
Message-ID: <CAJFAV8zhA8eTSJ_dk_1yGZCBQ_PTS7TiwUZZdAUOe-LzxE2dcQ@mail.gmail.com>
To: Anatoly Burakov <anatoly.burakov@intel.com>, Ray Kinsella <mdr@ashroe.eu>
Cc: dev <dev@dpdk.org>, Thomas Monjalon <thomas@monjalon.net>,
"Ananyev, Konstantin" <konstantin.ananyev@intel.com>,
Timothy McDaniel <timothy.mcdaniel@intel.com>,
David Hunt <david.hunt@intel.com>,
Bruce Richardson <bruce.richardson@intel.com>, chris.macnamara@intel.com,
Kevin Traynor <ktraynor@redhat.com>
Authentication-Results: relay.mimecast.com;
auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=dmarchan@redhat.com
X-Mimecast-Spam-Score: 0
X-Mimecast-Originator: redhat.com
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable
Subject: Re: [dpdk-dev] [PATCH v16 00/11] Add PMD power management
X-BeenThere: dev@dpdk.org
X-Mailman-Version: 2.1.29
Precedence: list
List-Id: DPDK patches and discussions <dev.dpdk.org>
List-Unsubscribe: <https://mails.dpdk.org/options/dev>,
<mailto:dev-request@dpdk.org?subject=unsubscribe>
List-Archive: <http://mails.dpdk.org/archives/dev/>
List-Post: <mailto:dev@dpdk.org>
List-Help: <mailto:dev-request@dpdk.org?subject=help>
List-Subscribe: <https://mails.dpdk.org/listinfo/dev>,
<mailto:dev-request@dpdk.org?subject=subscribe>
Errors-To: dev-bounces@dpdk.org
Sender: "dev" <dev-bounces@dpdk.org>
On Tue, Jan 12, 2021 at 6:37 PM Anatoly Burakov
<anatoly.burakov@intel.com> wrote:
>
> This patchset proposes a simple API for Ethernet drivers to cause the
> CPU to enter a power-optimized state while waiting for packets to
> arrive. There are multiple proposed mechanisms to achieve said power
> savings: simple frequency scaling, idle loop, and monitoring the Rx
> queue for incoming packages. The latter is achieved through cooperation
> with the NIC driver that will allow us to know address of wake up event,
> and wait for writes on that address.
>
> On IA, this is achieved through using UMONITOR/UMWAIT instructions. They
> are used in their raw opcode form because there is no widespread
> compiler support for them yet. Still, the API is made generic enough to
> hopefully support other architectures, if they happen to implement
> similar instructions.
>
> To achieve power savings, there is a very simple mechanism used: we're
> counting empty polls, and if a certain threshold is reached, we employ
> one of the suggested power management schemes automatically, from within
> a Rx callback inside the PMD. Once there's traffic again, the empty poll
> counter is reset.
>
> This patchset also introduces a few changes into existing power
> management-related intrinsics, namely to provide a native way of waking
> up a sleeping core without application being responsible for it, as well
> as general robustness improvements. There's quite a bit of locking going
> on, but these locks are per-thread and very little (if any) contention
> is expected, so the performance impact shouldn't be that bad (and in any
> case the locking happens when we're about to sleep anyway).
>
> Why are we putting it into ethdev as opposed to leaving this up to the
> application? Our customers specifically requested a way to do it with
> minimal changes to the application code. The current approach allows to
> just flip a switch and automatically have power savings.
>
> Things of note:
>
> - Only 1:1 core to queue mapping is supported, meaning that each lcore
> must at most handle RX on a single queue
If we want to save power, it is likely we would poll more rxqs on a thread.
> - Support 3 type policies. Monitor/Pause/Frequency Scaling
> - Power management is enabled per-queue
> - The API doesn't extend to other device types
>
> v16:
> - Implemented Konstantin's suggestions and comments
> - Added return values to the API
- This revision breaks SPDK build (reported by UNH):
http://mails.dpdk.org/archives/test-report/2021-January/174069.html
- Build is broken for ARM and PPC at patch:
86491d5bd4 - (HEAD) eal: add monitor wakeup function (25 minutes ago)
<Anatoly Burakov>
Only pasting the ARM failure:
ninja: Entering directory `/home/dmarchan/builds/build-arm64-host-clang'
[1/297] Compiling C object
'lib/76b5a35@@rte_eal@sta/librte_eal_arm_rte_power_intrinsics.c.o'.
FAILED: lib/76b5a35@@rte_eal@sta/librte_eal_arm_rte_power_intrinsics.c.o
aarch64-linux-gnu-gcc -Ilib/76b5a35@@rte_eal@sta -Ilib
-I../../dpdk/lib -I. -I../../dpdk/ -Iconfig -I../../dpdk/config
-Ilib/librte_eal/include -I../../dpdk/lib/librte_eal/include
-Ilib/librte_eal/linux/include
-I../../dpdk/lib/librte_eal/linux/include -Ilib/librte_eal/arm/include
-I../../dpdk/lib/librte_eal/arm/include -Ilib/librte_eal/common
-I../../dpdk/lib/librte_eal/common -Ilib/librte_eal
-I../../dpdk/lib/librte_eal -Ilib/librte_kvargs
-I../../dpdk/lib/librte_kvargs
-Ilib/librte_telemetry/../librte_metrics
-I../../dpdk/lib/librte_telemetry/../librte_metrics
-Ilib/librte_telemetry -I../../dpdk/lib/librte_telemetry
-fdiagnostics-color=3Dalways -pipe -D_FILE_OFFSET_BITS=3D64 -Wall
-Winvalid-pch -Werror -O2 -g -include rte_config.h -Wextra -Wcast-qual
-Wdeprecated -Wformat -Wformat-nonliteral -Wformat-security
-Wmissing-declarations -Wmissing-prototypes -Wnested-externs
-Wold-style-definition -Wpointer-arith -Wsign-compare
-Wstrict-prototypes -Wundef -Wwrite-strings -Wno-packed-not-aligned
-Wno-missing-field-initializers -D_GNU_SOURCE -fPIC -march=3Darmv8-a+crc
-DALLOW_EXPERIMENTAL_API -DALLOW_INTERNAL_API -Wno-format-truncation
'-DABI_VERSION=3D"21.1"' -DRTE_LIBEAL_USE_GETENTROPY -MD -MQ
'lib/76b5a35@@rte_eal@sta/librte_eal_arm_rte_power_intrinsics.c.o' -MF
'lib/76b5a35@@rte_eal@sta/librte_eal_arm_rte_power_intrinsics.c.o.d'
-o 'lib/76b5a35@@rte_eal@sta/librte_eal_arm_rte_power_intrinsics.c.o'
-c ../../dpdk/lib/librte_eal/arm/rte_power_intrinsics.c
../../dpdk/lib/librte_eal/arm/rte_power_intrinsics.c:35:1: error:
conflicting types for =E2=80=98rte_power_monitor_wakeup=E2=80=99
rte_power_monitor_wakeup(const unsigned int lcore_id)
^~~~~~~~~~~~~~~~~~~~~~~~
In file included from
../../dpdk/lib/librte_eal/arm/include/rte_power_intrinsics.h:14,
from ../../dpdk/lib/librte_eal/arm/rte_power_intrinsics.c:=
5:
../../dpdk/lib/librte_eal/include/generic/rte_power_intrinsics.h:79:5:
note: previous declaration of =E2=80=98rte_power_monitor_wakeup=E2=80=99 wa=
s here
int rte_power_monitor_wakeup(const unsigned int lcore_id);
^~~~~~~~~~~~~~~~~~~~~~~~
ninja: build stopped: subcommand failed.
- The ABI check is still not happy as I reported earlier.
Reproduced on v16 (GHA had a hiccup on this revision, but previous
ones had the failure too):
1 Changed variable:
[C] 'rte_eth_dev rte_eth_devices[]' was changed at rte_ethdev_core.h:196:=
1:
type of variable changed:
array element type 'struct rte_eth_dev' changed:
type size hasn't changed
1 data member change:
type of 'const eth_dev_ops* rte_eth_dev::dev_ops' changed:
in pointed to type 'const eth_dev_ops':
in unqualified underlying type 'struct eth_dev_ops' at
rte_ethdev_driver.h:789:1:
type size changed from 6208 to 6272 (in bits)
1 data member insertion:
'eth_get_monitor_addr_t
eth_dev_ops::get_monitor_addr', at offset 6208 (in bits) at
rte_ethdev_driver.h:940:1
no data member changes (94 filtered);
type size hasn't changed
Error: ABI issue reported for 'abidiff --suppr
/home/dmarchan/dpdk/devtools/../devtools/libabigail.abignore
--no-added-syms --headers-dir1
/home/dmarchan/abi/v20.11/build-gcc-static/usr/local/include
--headers-dir2 /home/dmarchan/builds/build-gcc-static/install/usr/local/inc=
lude
/home/dmarchan/abi/v20.11/build-gcc-static/dump/librte_ethdev.dump
/home/dmarchan/builds/build-gcc-static/install/dump/librte_ethdev.dump'
ABIDIFF_ABI_CHANGE, this change requires a review (abidiff flagged
this as a potential issue).
One solution is to add an exception on the eth_dev_ops structure.
--- a/devtools/libabigail.abignore
+++ b/devtools/libabigail.abignore
@@ -7,3 +7,7 @@
symbol_version =3D INTERNAL
[suppress_variable]
symbol_version =3D INTERNAL
+
+; Explicit ignore for driver-only ABI
+[suppress_type]
+ name =3D eth_dev_ops
--=20
David marchand