From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 1A24AA0A03; Mon, 18 Jan 2021 16:24:55 +0100 (CET) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id C46E8140F65; Mon, 18 Jan 2021 16:24:53 +0100 (CET) Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [216.205.24.124]) by mails.dpdk.org (Postfix) with ESMTP id 7862A140F61 for ; Mon, 18 Jan 2021 16:24:52 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1610983491; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: in-reply-to:in-reply-to:references:references; bh=188oFpFi760/QVfYQ5rOyVWh5ASy/5bl2PgJxmBOSvs=; b=C6RTiDWU9b4mAZFTdVdwozOjOy0f6Mn7GqCWUdQfBnG8+UrFcnctU/AGu2s2jWsOZRWIUr VT0Ilk7xg8TxqntpDKbzUhE7BoeAjoTj8ZODij8576KqhX9LVmKqhs2WRGal7MDEaWV7CS WgDBx3VIC6Wv9JJM7J8VaCl8AVR8TtE= Received: from mail-vk1-f198.google.com (mail-vk1-f198.google.com [209.85.221.198]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-195-60ip9tlrMpauipNLxHC-Kw-1; Mon, 18 Jan 2021 10:24:48 -0500 X-MC-Unique: 60ip9tlrMpauipNLxHC-Kw-1 Received: by mail-vk1-f198.google.com with SMTP id h15so9409698vka.10 for ; Mon, 18 Jan 2021 07:24:48 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=188oFpFi760/QVfYQ5rOyVWh5ASy/5bl2PgJxmBOSvs=; b=SwZoODN8Xiwm2sJLK5YeFV90LL6Ijl3pZe84BePmODjxgU7fJQpMLtmF5qSjc9hWKp 4RplJH3KAfBFAcMC2iloEyrVJS0kWq3MlKjnkuUK7SCOcCaSGwTwclV8JdjAINOw1qm1 DAezGf642CH+AvQfqYgeaE6X4CdwIr4unhvF2clB4i9CVJTMh9L2CcHYGcp65v4TpRNT Qn/9ENeX7OTVdGycvr4qp1ywuee15X6gFuGo/uP8XDNiAyk8smbi8M6p4t8GtThWslf7 i2aLJGOBJSgSJ7ChiBlti1T5HmlZUSZfFmZlGECGMvWzOdQxz/uvB6ygspI/oRcGp3Q0 elXA== X-Gm-Message-State: AOAM530gb+S1Fx22GPK19Fjy/N0JeP93wcsdKtqE7h2rLS0YUHq8UCOA Z2Fde4lT9hZ5DJ81CMZsu1N1/3slL6Ey1zSOKyQtXbRAjEuv27orfjDQFiR2lrlck73FDNQ7hDY AChhC9bRtpWSMlsyMKSw= X-Received: by 2002:ab0:6b:: with SMTP id 98mr16827307uai.86.1610983488136; Mon, 18 Jan 2021 07:24:48 -0800 (PST) X-Google-Smtp-Source: ABdhPJzqMfrLSDdzK8GBTUjHq+KaI2pdJknsLfJ4PWPuAJM1n2bAEd8SvTp9aeU6Pq/1ACom8k+AlG2AsG7Lj8odPx0= X-Received: by 2002:ab0:6b:: with SMTP id 98mr16827299uai.86.1610983487860; Mon, 18 Jan 2021 07:24:47 -0800 (PST) MIME-Version: 1.0 References: In-Reply-To: From: David Marchand Date: Mon, 18 Jan 2021 16:24:36 +0100 Message-ID: To: Anatoly Burakov Cc: dev , Thomas Monjalon , "Ananyev, Konstantin" , Timothy McDaniel , David Hunt , Bruce Richardson , chris.macnamara@intel.com Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=dmarchan@redhat.com X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-Type: text/plain; charset="UTF-8" Subject: Re: [dpdk-dev] [PATCH v17 00/11] Add PMD power management X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" On Thu, Jan 14, 2021 at 3:46 PM Anatoly Burakov wrote: > > This patchset proposes a simple API for Ethernet drivers to cause the > CPU to enter a power-optimized state while waiting for packets to > arrive. There are multiple proposed mechanisms to achieve said power > savings: simple frequency scaling, idle loop, and monitoring the Rx > queue for incoming packages. The latter is achieved through cooperation > with the NIC driver that will allow us to know address of wake up event, > and wait for writes on that address. > > On IA, this is achieved through using UMONITOR/UMWAIT instructions. They > are used in their raw opcode form because there is no widespread > compiler support for them yet. Still, the API is made generic enough to > hopefully support other architectures, if they happen to implement > similar instructions. > > To achieve power savings, there is a very simple mechanism used: we're > counting empty polls, and if a certain threshold is reached, we employ > one of the suggested power management schemes automatically, from within > a Rx callback inside the PMD. Once there's traffic again, the empty poll > counter is reset. > > This patchset also introduces a few changes into existing power > management-related intrinsics, namely to provide a native way of waking > up a sleeping core without application being responsible for it, as well > as general robustness improvements. There's quite a bit of locking going > on, but these locks are per-thread and very little (if any) contention > is expected, so the performance impact shouldn't be that bad (and in any > case the locking happens when we're about to sleep anyway). > > Why are we putting it into ethdev as opposed to leaving this up to the > application? Our customers specifically requested a way to do it with > minimal changes to the application code. The current approach allows to > just flip a switch and automatically have power savings. > > Things of note: > > - Only 1:1 core to queue mapping is supported, meaning that each lcore > must at most handle RX on a single queue > - Support 3 type policies. Monitor/Pause/Frequency Scaling > - Power management is enabled per-queue > - The API doesn't extend to other device types > > v17: > - Added exception for ethdev driver-only ABI > - Added memory barriers for monitor/wakeup (Konstantin) > - Fixed compiled issues on non-x86 platforms (hopefully!) SPDK build is still broken. http://mails.dpdk.org/archives/test-report/2021-January/174840.html ==== 20 line log output for Ubuntu 18.04 (dpdk_compile_spdk): ==== rte_power_pmd_mgmt.c:(.text.experimental+0x1cc): undefined reference to `rte_eth_add_rx_callback' rte_power_pmd_mgmt.c:(.text.experimental+0x1f8): undefined reference to `rte_eth_get_monitor_addr' rte_power_pmd_mgmt.c:(.text.experimental+0x37f): undefined reference to `rte_eth_dev_logtype' /dpdk/build/lib/librte_power.a(librte_power_rte_power_pmd_mgmt.c.o): In function `rte_power_pmd_mgmt_queue_disable': rte_power_pmd_mgmt.c:(.text.experimental+0x42a): undefined reference to `rte_eth_dev_is_valid_port' rte_power_pmd_mgmt.c:(.text.experimental+0x4e7): undefined reference to `rte_eth_remove_rx_callback' rte_power_pmd_mgmt.c:(.text.experimental+0x536): undefined reference to `rte_eth_remove_rx_callback' rte_power_pmd_mgmt.c:(.text.experimental+0x54d): undefined reference to `rte_eth_dev_logtype' collect2: error: ld returned 1 exit status /spdk/mk/spdk.app.mk:65: recipe for target 'iscsi_fuzz' failed /spdk/mk/spdk.subdirs.mk:44: recipe for target 'iscsi_fuzz' failed /spdk/mk/spdk.subdirs.mk:44: recipe for target 'fuzz' failed make[4]: *** [iscsi_fuzz] Error 1 make[3]: *** [iscsi_fuzz] Error 2 make[2]: *** [fuzz] Error 2 /spdk/mk/spdk.subdirs.mk:44: recipe for target 'app' failed make[1]: *** [app] Error 2 /spdk/mk/spdk.subdirs.mk:44: recipe for target 'test' failed make: *** [test] Error 2 [2] Error running command. I guess this is because of the added dependency of rte_ethdev to rte_power. Afaics, SPDK does not use pkg-config: https://github.com/spdk/spdk/blob/master/lib/env_dpdk/env.mk#L53 -- David Marchand