From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from dpdk.org (dpdk.org [92.243.14.124]) by inbox.dpdk.org (Postfix) with ESMTP id CA2A0A04DB; Thu, 15 Oct 2020 18:53:07 +0200 (CEST) Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id 433711DC98; Thu, 15 Oct 2020 18:53:06 +0200 (CEST) Received: from mga04.intel.com (mga04.intel.com [192.55.52.120]) by dpdk.org (Postfix) with ESMTP id 8BE241DC86 for ; Thu, 15 Oct 2020 18:53:04 +0200 (CEST) IronPort-SDR: vigr7trTh/t54mgnLd55GIPRfkza6t37Cr9Li39c7SgYX8Ki2HtHAPZrvDibijCekN0BoJe91D oV1u6wTecBuA== X-IronPort-AV: E=McAfee;i="6000,8403,9775"; a="163778160" X-IronPort-AV: E=Sophos;i="5.77,379,1596524400"; d="scan'208";a="163778160" X-Amp-Result: SKIPPED(no attachment in message) X-Amp-File-Uploaded: False Received: from orsmga008.jf.intel.com ([10.7.209.65]) by fmsmga104.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 15 Oct 2020 09:53:02 -0700 IronPort-SDR: G1McDKYo9a9RtHw+A07irrxQVT7ONQswpxR7DC7ct6h12n9W6zsl0aYUhW8+DpUC52YO12KH/H eXVG9yoXVTDw== X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.77,379,1596524400"; d="scan'208";a="346912981" Received: from orsmsx601.amr.corp.intel.com ([10.22.229.14]) by orsmga008.jf.intel.com with ESMTP; 15 Oct 2020 09:53:02 -0700 Received: from orsmsx601.amr.corp.intel.com (10.22.229.14) by ORSMSX601.amr.corp.intel.com (10.22.229.14) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.1713.5; Thu, 15 Oct 2020 09:53:01 -0700 Received: from ORSEDG602.ED.cps.intel.com (10.7.248.7) by orsmsx601.amr.corp.intel.com (10.22.229.14) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.1713.5 via Frontend Transport; Thu, 15 Oct 2020 09:53:01 -0700 Received: from NAM12-DM6-obe.outbound.protection.outlook.com (104.47.59.172) by edgegateway.intel.com (134.134.137.103) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.1.1713.5; Thu, 15 Oct 2020 09:53:00 -0700 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=C7MlK0JrVsRONtzwQXANzv/mEiGUqA6/6RAeFSTkDm6tT4V1pwrSYgqFPC4o5H60IXh7WC3S1kCZYI36cPwM+6V9kfUlJzxbX5Y5V5tb4tOOJpjabGSQRN7Y2wshwqfm+TczLq08ZLRtlQAPrvyZ0NvOVaYOhHtdPQc6jH61ptUnCs5esW+PaCmXBuKVnFkUv/g0DbnjFR73VPYL9hQa0dIiZhCJIf7YfS5t+402BtQrvp89CMJPuFYSQ2BaYxWQRRHd/3oJxeTwtnJ+dq7RL7ey4K2egOUl5BLT5JB1vDsM84afQ6+aLgfVqCqDuS54ELspxq4Nzs6x/abCE3QedA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=Gd57gzeceV/shffvx3cb5yR1+Auz3cQCyvIAZxvHn7Y=; b=hdtbHJrDsU1Ujkn/M6fn1jJqWXWhkDnTDBSP0lY51XdPsxucRkVgBSFFUoO/lAl0c6wAIEJbPqejxV69vJxPlErfUkby9LK/X5awfZ1TwPKZE4BlegkBklrZmMbm06XMLd5SBLMc2q7UYUAXvgUD9LKGB6w9bP5eI5mVY1xkLhavAKBS59J49iva/mSryV8sUPagPpFyLZBSpdq3VNPqkXdgSqACkJf5iHYET+5O/dtqCRnBWw80NmIf7y3H/N0TM3XcgwDHzSP/thfhAN41ktMuOiwMIbyGCzltczGmKvpB7s0+P7tkOPFNS0Mh8ZwElCCayTri6OA3ZgHLdC4AHQ== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=intel.com; dmarc=pass action=none header.from=intel.com; dkim=pass header.d=intel.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=intel.onmicrosoft.com; s=selector2-intel-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=Gd57gzeceV/shffvx3cb5yR1+Auz3cQCyvIAZxvHn7Y=; b=bcWd8d2C94S/Cs3PH0nAAbDmPchPYW4AET5wUCkCTPl5bdpsQ2DpqRrl41j2mEWvSpTNoyhDQ6mLUk1WOb/fG/pd9kKZ0IBDvQ9EkqNoCuOxbNZbcVofJt7gQ6VgcH8fixhODQl8tI4dokHvBK5dNH/GH/qJK+GlNB5rho85Qvo= Received: from BYAPR11MB3301.namprd11.prod.outlook.com (2603:10b6:a03:7f::26) by BY5PR11MB4101.namprd11.prod.outlook.com (2603:10b6:a03:18e::10) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.3455.28; Thu, 15 Oct 2020 16:52:58 +0000 Received: from BYAPR11MB3301.namprd11.prod.outlook.com ([fe80::f5a4:3f6b:ade3:296b]) by BYAPR11MB3301.namprd11.prod.outlook.com ([fe80::f5a4:3f6b:ade3:296b%3]) with mapi id 15.20.3477.021; Thu, 15 Oct 2020 16:52:58 +0000 From: "Ananyev, Konstantin" To: "Burakov, Anatoly" , "dev@dpdk.org" CC: "Ma, Liang J" , "Hunt, David" , Ray Kinsella , Neil Horman , "jerinjacobk@gmail.com" , "Richardson, Bruce" , "thomas@monjalon.net" , "McDaniel, Timothy" , "Eads, Gage" , "Macnamara, Chris" Thread-Topic: [PATCH v7 05/10] power: add PMD power management API and callback Thread-Index: AQHWoutklzNp1e+juU+1cY7dH1zqIqmY4LAw Date: Thu, 15 Oct 2020 16:52:58 +0000 Message-ID: References: In-Reply-To: Accept-Language: en-GB, en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: dlp-product: dlpe-windows dlp-reaction: no-action dlp-version: 11.5.1.3 authentication-results: intel.com; dkim=none (message not signed) header.d=none;intel.com; dmarc=none action=none header.from=intel.com; x-originating-ip: [46.7.39.127] x-ms-publictraffictype: Email x-ms-office365-filtering-correlation-id: c668e6ca-f509-4cec-0ea0-08d8712ac54b x-ms-traffictypediagnostic: BY5PR11MB4101: x-ld-processed: 46c98d88-e344-4ed4-8496-4ed7712e255d,ExtAddr x-ms-exchange-transport-forked: True x-microsoft-antispam-prvs: x-ms-oob-tlc-oobclassifiers: OLM:8882; x-ms-exchange-senderadcheck: 1 x-microsoft-antispam: BCL:0; x-microsoft-antispam-message-info: BoyjWY05ZEFvAjpKwu1qzlZcJj6ZAaBKkQ3P67aYR0Y7qLl2LsaInQo8HT51SkSZug6GLg21QWrGVGdpEuOdIvYmW8ZgUxGL/YAMaFCTg5QHhQdMkq0cWtqDWwMVTxizYzxbVJIraQ5YKNeG36k9uKV5aGDS+wfRFGPQgBzxHmi6MN2MmtDrU/5H4RVpIRgjxbBblME5Yj3jOj9FJlo7IyN4NCjXQbicMBNdkHipfgsgkk0DcfbIAXxbAoea3JLFlkyzYikuGw88WZr7i1htTq9RgkfBunkgj0ZF4Kq3h2y7qeHlEUbUKdzwwKDDUlQ6 x-forefront-antispam-report: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:BYAPR11MB3301.namprd11.prod.outlook.com; PTR:; CAT:NONE; SFS:(4636009)(366004)(396003)(136003)(376002)(39860400002)(346002)(7696005)(71200400001)(6506007)(478600001)(4326008)(33656002)(52536014)(2906002)(8676002)(54906003)(110136005)(8936002)(83380400001)(107886003)(186003)(5660300002)(26005)(76116006)(66446008)(66556008)(66476007)(66946007)(64756008)(86362001)(9686003)(316002)(55016002); DIR:OUT; SFP:1102; x-ms-exchange-antispam-messagedata: 7mDk9qw7NEo6J1V8prf+BmX1IoU7Hsqr3eu09nAYreKohbKETRtENfDnvjDiryCqYjtdsrj1drHTaohpTw1n7nK3g1Cyo+eY18QlB6nqU+RF9CqSZ4ZJ5Mj3tenpMcRSIOC0xXBUm+U+vnz3u8oIqMwm9N6yQBwsZ13WKmjLY3Ohkq0dMId7G7zeQHs2sD+uEmTI32Fw+oDLZmCZrnNsRCpoJ5873Lr7ZMMYzpOS4V3AZsnYOfFvUXLI78klls4fC3Ez5cLi9PoF3TuD0OZCnqKd8oAsN/9JcAsf2W1pAG6WresQ3QA40VQ+Q0ieGFXYm+CMCMRJGk88m3vp/2sMkrusEEdx+pWupGOLrlXDsiYGPwi41KMs2x4xbwaklaV9YZZf73YW9u97uCNPjIcednXgKdosqunI3NohkPnilAw4pAp7E1KuSOEKfnduFHxQirX5EbL7L+JmCFTZOV7hW+oU3u/vZvYVLufsSG2ahH444F5tQMbkBS04wAENbrqby5AoWDyQfP1CBdfqdcW1zmM8RulS6WVKAzYykumgPV0eEhNtfpsVJmToHwTmbO7UKE79ychUoainuQIkyH5jsYlTX6phqEhj+6PVd7YM4DTWy7GufW9oKRUxCGiC0cer5sXFV4DXcJAz3FoKRBOIRA== Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-AuthSource: BYAPR11MB3301.namprd11.prod.outlook.com X-MS-Exchange-CrossTenant-Network-Message-Id: c668e6ca-f509-4cec-0ea0-08d8712ac54b X-MS-Exchange-CrossTenant-originalarrivaltime: 15 Oct 2020 16:52:58.3961 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Hosted X-MS-Exchange-CrossTenant-id: 46c98d88-e344-4ed4-8496-4ed7712e255d X-MS-Exchange-CrossTenant-mailboxtype: HOSTED X-MS-Exchange-CrossTenant-userprincipalname: EAeapQ59f/5I7r//W8V5oRIM77TBRXhk1Lt8j0AAlJB/SjP6k6DIQUeW5QLgVGh8hVI1Rm/0V16WODIbBRpMezl3SLEXk5tIlcPhGyHznE0= X-MS-Exchange-Transport-CrossTenantHeadersStamped: BY5PR11MB4101 X-OriginatorOrg: intel.com Subject: Re: [dpdk-dev] [PATCH v7 05/10] power: add PMD power management API and callback X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" > Add a simple on/off switch that will enable saving power when no > packets are arriving. It is based on counting the number of empty > polls and, when the number reaches a certain threshold, entering an > architecture-defined optimized power state that will either wait > until a TSC timestamp expires, or when packets arrive. >=20 > This API mandates a core-to-single-queue mapping (that is, multiple > queued per device are supported, but they have to be polled on different > cores). >=20 > This design is using PMD RX callbacks. >=20 > 1. UMWAIT/UMONITOR: >=20 > When a certain threshold of empty polls is reached, the core will go > into a power optimized sleep while waiting on an address of next RX > descriptor to be written to. >=20 > 2. Pause instruction >=20 > Instead of move the core into deeper C state, this method uses the > pause instruction to avoid busy polling. >=20 > 3. Frequency scaling > Reuse existing DPDK power library to scale up/down core frequency > depending on traffic volume. >=20 > Signed-off-by: Liang Ma > Signed-off-by: Anatoly Burakov > Acked-by: David Hunt > --- >=20 > Notes: > v7: > - Fixed race condition (Konstantin) > - Slight rework of the structure of monitor code > - Added missing inline for wakeup >=20 > v6: > - Added wakeup mechanism for UMWAIT > - Removed memory allocation (everything is now allocated statically) > - Fixed various typos and comments > - Check for invalid queue ID > - Moved release notes to this patch >=20 > v5: > - Make error checking more robust > - Prevent initializing scaling if ACPI or PSTATE env wasn't set > - Prevent initializing UMWAIT path if PMD doesn't support get_wake_= addr > - Add some debug logging > - Replace x86-specific code path to generic path using the intrinsic = check >=20 > doc/guides/rel_notes/release_20_11.rst | 11 + > lib/librte_power/meson.build | 5 +- > lib/librte_power/rte_power_pmd_mgmt.c | 320 +++++++++++++++++++++++++ > lib/librte_power/rte_power_pmd_mgmt.h | 92 +++++++ > lib/librte_power/rte_power_version.map | 4 + > 5 files changed, 430 insertions(+), 2 deletions(-) > create mode 100644 lib/librte_power/rte_power_pmd_mgmt.c > create mode 100644 lib/librte_power/rte_power_pmd_mgmt.h >=20 > diff --git a/doc/guides/rel_notes/release_20_11.rst b/doc/guides/rel_note= s/release_20_11.rst > index 4c6a615ce9..a814c67d54 100644 > --- a/doc/guides/rel_notes/release_20_11.rst > +++ b/doc/guides/rel_notes/release_20_11.rst > @@ -204,6 +204,17 @@ New Features >=20 > * Added support to update subport rate dynamically. >=20 > +* **Add PMD power management mechanism** > + > + 3 new Ethernet PMD power management mechanism is added through existin= g > + RX callback infrastructure. > + > + * Add power saving scheme based on UMWAIT instruction (x86 only) > + * Add power saving scheme based on ``rte_pause()`` > + * Add power saving scheme based on frequency scaling through the power= library > + * Add new EXPERIMENTAL API ``rte_power_pmd_mgmt_queue_enable()`` > + * Add new EXPERIMENTAL API ``rte_power_pmd_mgmt_queue_disable()`` > + >=20 > Removed Items > ------------- .... > + > +int > +rte_power_pmd_mgmt_queue_enable(unsigned int lcore_id, > + uint16_t port_id, uint16_t queue_id, > + enum rte_power_pmd_mgmt_type mode) > +{ > + struct rte_eth_dev *dev; > + struct pmd_queue_cfg *queue_cfg; > + int ret; > + > + RTE_ETH_VALID_PORTID_OR_ERR_RET(port_id, -EINVAL); > + dev =3D &rte_eth_devices[port_id]; > + > + /* check if queue id is valid */ > + if (queue_id >=3D dev->data->nb_rx_queues || > + queue_id >=3D RTE_MAX_QUEUES_PER_PORT) { > + return -EINVAL; > + } > + > + queue_cfg =3D &port_cfg[port_id][queue_id]; > + > + if (queue_cfg->pwr_mgmt_state =3D=3D PMD_MGMT_ENABLED) { > + ret =3D -EINVAL; > + goto end; > + } > + > + switch (mode) { > + case RTE_POWER_MGMT_TYPE_WAIT: > + { > + /* check if rte_power_monitor is supported */ > + uint64_t dummy_expected, dummy_mask; > + struct rte_cpu_intrinsics i; > + volatile void *dummy_addr; > + uint8_t dummy_sz; > + > + rte_cpu_get_intrinsics_support(&i); > + > + if (!i.power_monitor) { > + RTE_LOG(DEBUG, POWER, "Monitoring intrinsics are not supported\n"); > + ret =3D -ENOTSUP; > + goto end; > + } > + > + /* check if the device supports the necessary PMD API */ > + if (rte_eth_get_wake_addr(port_id, queue_id, > + &dummy_addr, &dummy_expected, > + &dummy_mask, &dummy_sz) =3D=3D -ENOTSUP) { > + RTE_LOG(DEBUG, POWER, "The device does not support rte_eth_rxq_ring_a= ddr_get\n"); > + ret =3D -ENOTSUP; > + goto end; > + } > + /* initialize UMWAIT spinlock */ > + rte_spinlock_init(&queue_cfg->umwait_lock); Still looks excessive and possibly error prone to me. Apart from that: Acked-by: Konstantin Ananyev > + > + /* initialize data before enabling the callback */ > + queue_cfg->empty_poll_stats =3D 0; > + queue_cfg->cb_mode =3D mode; > + queue_cfg->pwr_mgmt_state =3D PMD_MGMT_ENABLED; > + > + queue_cfg->cur_cb =3D rte_eth_add_rx_callback(port_id, queue_id, > + clb_umwait, NULL); > + break; > + } > + case RTE_POWER_MGMT_TYPE_SCALE: > + { > + enum power_management_env env; > + /* only PSTATE and ACPI modes are supported */ > + if (!rte_power_check_env_supported(PM_ENV_ACPI_CPUFREQ) && > + !rte_power_check_env_supported( > + PM_ENV_PSTATE_CPUFREQ)) { > + RTE_LOG(DEBUG, POWER, "Neither ACPI nor PSTATE modes are supported\n"= ); > + ret =3D -ENOTSUP; > + goto end; > + } > + /* ensure we could initialize the power library */ > + if (rte_power_init(lcore_id)) { > + ret =3D -EINVAL; > + goto end; > + } > + /* ensure we initialized the correct env */ > + env =3D rte_power_get_env(); > + if (env !=3D PM_ENV_ACPI_CPUFREQ && > + env !=3D PM_ENV_PSTATE_CPUFREQ) { > + RTE_LOG(DEBUG, POWER, "Neither ACPI nor PSTATE modes were initialized= \n"); > + ret =3D -ENOTSUP; > + goto end; > + } > + /* initialize data before enabling the callback */ > + queue_cfg->empty_poll_stats =3D 0; > + queue_cfg->cb_mode =3D mode; > + queue_cfg->pwr_mgmt_state =3D PMD_MGMT_ENABLED; > + > + queue_cfg->cur_cb =3D rte_eth_add_rx_callback(port_id, > + queue_id, clb_scale_freq, NULL); > + break; > + } > + case RTE_POWER_MGMT_TYPE_PAUSE: > + /* initialize data before enabling the callback */ > + queue_cfg->empty_poll_stats =3D 0; > + queue_cfg->cb_mode =3D mode; > + queue_cfg->pwr_mgmt_state =3D PMD_MGMT_ENABLED; > + > + queue_cfg->cur_cb =3D rte_eth_add_rx_callback(port_id, queue_id, > + clb_pause, NULL); > + break; > + } > + ret =3D 0; > + > +end: > + return ret; > +} > +