From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: <dev-bounces@dpdk.org> Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 9863BA0C49; Wed, 7 Jul 2021 13:54:43 +0200 (CEST) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 848A1406FF; Wed, 7 Jul 2021 13:54:43 +0200 (CEST) Received: from mga17.intel.com (mga17.intel.com [192.55.52.151]) by mails.dpdk.org (Postfix) with ESMTP id 35589406B4 for <dev@dpdk.org>; Wed, 7 Jul 2021 13:54:42 +0200 (CEST) X-IronPort-AV: E=McAfee;i="6200,9189,10037"; a="189665840" X-IronPort-AV: E=Sophos;i="5.83,331,1616482800"; d="scan'208";a="189665840" Received: from fmsmga004.fm.intel.com ([10.253.24.48]) by fmsmga107.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 07 Jul 2021 04:54:35 -0700 X-IronPort-AV: E=Sophos;i="5.83,331,1616482800"; d="scan'208";a="482143330" Received: from dhunt5-mobl5.ger.corp.intel.com (HELO [10.252.24.114]) ([10.252.24.114]) by fmsmga004-auth.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 07 Jul 2021 04:54:33 -0700 To: Anatoly Burakov <anatoly.burakov@intel.com>, dev@dpdk.org Cc: konstantin.ananyev@intel.com, ciara.loftus@intel.com References: <cover.1625498488.git.anatoly.burakov@intel.com> <cover.1625654882.git.anatoly.burakov@intel.com> <afb0da82b0b05f8580ff6a5b6ff22e3a7842aa15.1625654882.git.anatoly.burakov@intel.com> From: David Hunt <david.hunt@intel.com> Message-ID: <443d7e34-f8eb-d6cb-9b0e-37980c64f4cf@intel.com> Date: Wed, 7 Jul 2021 12:54:31 +0100 User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:78.0) Gecko/20100101 Thunderbird/78.11.0 MIME-Version: 1.0 In-Reply-To: <afb0da82b0b05f8580ff6a5b6ff22e3a7842aa15.1625654882.git.anatoly.burakov@intel.com> Content-Type: text/plain; charset=utf-8; format=flowed Content-Transfer-Encoding: 7bit Content-Language: en-GB Subject: Re: [dpdk-dev] [PATCH v7 5/7] power: support callbacks for multiple Rx queues X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions <dev.dpdk.org> List-Unsubscribe: <https://mails.dpdk.org/options/dev>, <mailto:dev-request@dpdk.org?subject=unsubscribe> List-Archive: <http://mails.dpdk.org/archives/dev/> List-Post: <mailto:dev@dpdk.org> List-Help: <mailto:dev-request@dpdk.org?subject=help> List-Subscribe: <https://mails.dpdk.org/listinfo/dev>, <mailto:dev-request@dpdk.org?subject=subscribe> Errors-To: dev-bounces@dpdk.org Sender: "dev" <dev-bounces@dpdk.org> On 7/7/2021 11:48 AM, Anatoly Burakov wrote: > Currently, there is a hard limitation on the PMD power management > support that only allows it to support a single queue per lcore. This is > not ideal as most DPDK use cases will poll multiple queues per core. > > The PMD power management mechanism relies on ethdev Rx callbacks, so it > is very difficult to implement such support because callbacks are > effectively stateless and have no visibility into what the other ethdev > devices are doing. This places limitations on what we can do within the > framework of Rx callbacks, but the basics of this implementation are as > follows: > > - Replace per-queue structures with per-lcore ones, so that any device > polled from the same lcore can share data > - Any queue that is going to be polled from a specific lcore has to be > added to the list of queues to poll, so that the callback is aware of > other queues being polled by the same lcore > - Both the empty poll counter and the actual power saving mechanism is > shared between all queues polled on a particular lcore, and is only > activated when all queues in the list were polled and were determined > to have no traffic. > - The limitation on UMWAIT-based polling is not removed because UMWAIT > is incapable of monitoring more than one address. > > Also, while we're at it, update and improve the docs. > > Signed-off-by: Anatoly Burakov <anatoly.burakov@intel.com> > --- > > Notes: > v7: > - Fix bug where initial sleep target was always set to zero > - Fix logic in handling of n_queues_ready_to_sleep counter > - Update documentation on hardware requirements > > v6: > - Track each individual queue sleep status (Konstantin) > - Fix segfault (Dave) > > v5: > - Remove the "power save queue" API and replace it with mechanism suggested by > Konstantin > > v3: > - Move the list of supported NICs to NIC feature table > > v2: > - Use a TAILQ for queues instead of a static array > - Address feedback from Konstantin > - Add additional checks for stopped queues > > doc/guides/nics/features.rst | 10 + > doc/guides/prog_guide/power_man.rst | 69 ++-- > doc/guides/rel_notes/release_21_08.rst | 3 + > lib/power/rte_power_pmd_mgmt.c | 456 +++++++++++++++++++------ > 4 files changed, 402 insertions(+), 136 deletions(-) > --snip-- Thanks Anatoly. Not seeing the rollover now, and power savings are back as expected when low traffic in both monitor and pause modes. All previous issues seem now to be resolved. Patch set LGTM. Tested-by: David Hunt <david.hunt@intel.com>