From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 4D30742B2C; Wed, 17 May 2023 16:44:07 +0200 (CEST) Received: from mails.dpdk.org (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 2412D406B7; Wed, 17 May 2023 16:44:07 +0200 (CEST) Received: from mga02.intel.com (mga02.intel.com [134.134.136.20]) by mails.dpdk.org (Postfix) with ESMTP id E02A140041 for ; Wed, 17 May 2023 16:44:04 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1684334645; x=1715870645; h=message-id:date:subject:to:cc:references:from: in-reply-to:content-transfer-encoding:mime-version; bh=lfrlFtL7sUILdyyUQVZtBHpjGjqaKTMvI3loc2bVKNw=; b=cj077tw4ELnrtdYON5TB2oVYTdzbvflBuV0iE7h3eXTdmiOu1oJW03FF IqTLE9TvbaFuOw3Em7rfwiNhjBNMvoueBfa6bhba4+WxP3N6fAWp6GGk4 G3D/PVPF6IZFJkUVIPY/AhLYNCSIe+7AwmiFUFSEBdscDPb2axD2cQyyZ a4eubHU1pdaS194GDy94WrFRo46F7HA8eYERU0Mv1zVTON8FS+MJdQ1kT E301x01LRF0ajAf76dZ+72SDLFy2YLHjlv8yWnyCTecKC2cT4a+nK9guP kzosRTdbWjteKTF/0kpHw7anJvQNs0kjHg4Vhk8T9F1A88T0BCmccCuFx Q==; X-IronPort-AV: E=McAfee;i="6600,9927,10713"; a="341163203" X-IronPort-AV: E=Sophos;i="5.99,282,1677571200"; d="scan'208";a="341163203" Received: from orsmga006.jf.intel.com ([10.7.209.51]) by orsmga101.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 17 May 2023 07:44:03 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=McAfee;i="6600,9927,10713"; a="679308262" X-IronPort-AV: E=Sophos;i="5.99,282,1677571200"; d="scan'208";a="679308262" Received: from orsmsx601.amr.corp.intel.com ([10.22.229.14]) by orsmga006.jf.intel.com with ESMTP; 17 May 2023 07:44:03 -0700 Received: from orsmsx610.amr.corp.intel.com (10.22.229.23) by ORSMSX601.amr.corp.intel.com (10.22.229.14) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.23; Wed, 17 May 2023 07:44:03 -0700 Received: from orsmsx610.amr.corp.intel.com (10.22.229.23) by ORSMSX610.amr.corp.intel.com (10.22.229.23) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.23; Wed, 17 May 2023 07:44:03 -0700 Received: from orsedg603.ED.cps.intel.com (10.7.248.4) by orsmsx610.amr.corp.intel.com (10.22.229.23) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.23 via Frontend Transport; Wed, 17 May 2023 07:44:03 -0700 Received: from NAM10-BN7-obe.outbound.protection.outlook.com (104.47.70.109) by edgegateway.intel.com (134.134.137.100) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.1.2507.23; Wed, 17 May 2023 07:44:02 -0700 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=nM5j5/KCf69F8Okq+FK9cIxlm5Uk/uOHwTciMlM4aKhP7NCTVefGgHJlDBId5ME0oMLrzJ8y/ZZKiuns9vJ0SlRas/DKiqfslQ2ndgerYCMEx1vjbTsy7kAM7w5UgFMQprwRYvodOjuy+23DVWwCnJjhUKwRWFRejFEM0pWswC/SOV4Aty6cCq0o6bO/dh40gJ+oXvbooHxF9n0TD5Z1cL0jp876uFOdLZfKekhKeb90QUyvwpR/+SiLUGI5VPbt6XTUrzMkFxkv3QuXFF3wzL4LD+5M01hpMQr+Vd2YfgwP0BLjMztGvhMW8d7j8aB70dR/d3k9xQrmfrwUqtTAdA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=NenUYxYM3/mkItHhDRF4hmPZ9hf/0exyQ9+KjNoitoo=; b=IV1+L7tMejJSgtjAy4pWq2e7dEi5v8pglCNytcAUXiLa7KuEz6Z52jY0hwx8IFKDt66sy79qFRqBj0VF8F4nGVFquv+6/yHLCMYwUGCAdq2nw1ah2kvQSvKAyJNLWjHKsi0U6/ILSi1hUUByki+4t6b97ticXj5aj6+XzRAIm+mKMZfjRQ02oRG0jwhRdOnOPLKnuUorpw/8o3Qa5MqIFVzLqyfd71pgTfXTRqmXr87QZ/qimodh03/H6kU70iqBbHng0k+C5ojlgsSs0+gqgWP201Bu2eZRjHswlbihgZxLnaWlLwnZcLNw46cE42shq8tFZt+dQzO+b8dxxaoQ4Q== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=intel.com; dmarc=pass action=none header.from=intel.com; dkim=pass header.d=intel.com; arc=none Authentication-Results: dkim=none (message not signed) header.d=none;dmarc=none action=none header.from=intel.com; Received: from DM4PR11MB6502.namprd11.prod.outlook.com (2603:10b6:8:89::7) by CY8PR11MB7268.namprd11.prod.outlook.com (2603:10b6:930:9b::9) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.6387.30; Wed, 17 May 2023 14:43:55 +0000 Received: from DM4PR11MB6502.namprd11.prod.outlook.com ([fe80::49c0:aa4c:e5b4:e718]) by DM4PR11MB6502.namprd11.prod.outlook.com ([fe80::49c0:aa4c:e5b4:e718%5]) with mapi id 15.20.6387.033; Wed, 17 May 2023 14:43:55 +0000 Message-ID: <1cecb5a2-3089-d862-8614-77bb6bea3f17@intel.com> Date: Wed, 17 May 2023 15:43:50 +0100 User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:102.0) Gecko/20100101 Firefox/102.0 Thunderbird/102.11.0 Subject: Re: [RFC PATCH 4/5] power: add eventdev support for power management Content-Language: en-US To: Sivaprasad Tummala , , , CC: References: <20230419095427.563185-1-sivaprasad.tummala@amd.com> <20230419095427.563185-4-sivaprasad.tummala@amd.com> From: "Burakov, Anatoly" In-Reply-To: <20230419095427.563185-4-sivaprasad.tummala@amd.com> Content-Type: text/plain; charset="UTF-8"; format=flowed Content-Transfer-Encoding: 7bit X-ClientProxiedBy: LO2P265CA0154.GBRP265.PROD.OUTLOOK.COM (2603:10a6:600:9::22) To DM4PR11MB6502.namprd11.prod.outlook.com (2603:10b6:8:89::7) MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: DM4PR11MB6502:EE_|CY8PR11MB7268:EE_ X-MS-Office365-Filtering-Correlation-Id: b7919f42-64ba-43f7-ee5d-08db56e5238e X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: 9XeB2L8PGhqF6OFbIlZ4XYFQetyylE5MlCu4J1YAknOEjj1tD+nlUzq2d5h5eIRdSMrRuDoF+Ou/7fc24crSf8b3OgGP7KE4C7huBm7oau7mJhkQq7uRiRXm+jsxPdzvEGInZ46OL2RRveW4wACssGfkg3Qj8l6f5TUFD5vrZEDQMhI5a6BqE2gdpNpBiOzrdpX3uUpArEYtM0dzmmozYw7yNvM1OJS4jAeKLRp9vSXHsBKNbsPCaWxxJXqMFHrLxL8haTgLQfwEr93i3c0k3WE8BI5P55PDKAFHgnoYCwzE6dvdhaYS7cb9DiASqnzoGdWqafC4AJgDGUJC3Iz8+qRfLEBzmE3/5kJZbxVq84al+4lDMkGhRnd9+I/j2C712EgZsBlrJIpICX1a7ry7Nom9N/2Yio0NangHNg8VkZXc4vsBiQwKs/tpuHikkH7qrDh/PAoQ+1jOckW+7F7x2cy4L08yAn3tAPWKFVi59Vi1nZyPySXe+p5rKwf0ySNVbn+VhZ8KRqKG9aKmmRfybWG7m/vUQXm2I1UjxqE13T0VCMQAgsltiKWR4PF4BwQRMb7j86Ozy/pvIkE1vK0ZiIOs6YFi0g95M4LarV6AXZTh5BvZlxu1d0YqAClLe2NBV4FoJlIe1cSiBBQmICz5Pw== X-Forefront-Antispam-Report: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:DM4PR11MB6502.namprd11.prod.outlook.com; PTR:; CAT:NONE; SFS:(13230028)(366004)(39860400002)(136003)(376002)(396003)(346002)(451199021)(26005)(2906002)(83380400001)(31686004)(2616005)(6506007)(186003)(36756003)(38100700002)(53546011)(6512007)(31696002)(86362001)(5660300002)(6486002)(82960400001)(8676002)(6666004)(66556008)(66476007)(6636002)(4326008)(478600001)(66946007)(41300700001)(8936002)(316002)(45980500001)(43740500002); DIR:OUT; SFP:1102; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: =?utf-8?B?UExzaUJwR2d3TjRtbVpuRXlvenpwdzdpR3krdEJMQVRpQzNiWlI1QXlGOUIw?= =?utf-8?B?U3lhZFF5STV0OFFVc3pHSlNEY2J2TGcveGt2NkNuMm9XSForNG5yZjRHdnkw?= =?utf-8?B?Q0pjZUluM0FwR0JxMHY3SitYMjlNZHFDSlY0L29PRUxyVWRRaS9TQm9hZmtU?= =?utf-8?B?T0kvT051UE1TYmxBaFlUUVFGbVFtaVh2aXdPZGhMWElNZ1F5WE92WXcxdFFY?= =?utf-8?B?dVBvWWRmNUhyMC9MTDJ5OXBLMHJwbWw5NnV4TmNnd0s0RzBkZW5JbzdqVE1Z?= =?utf-8?B?ZVk2c3dZZ0QyR05ad1E3blV6ZVNYU1pQVGFxMHdmUThyKzYrRHdoMnBsbzNE?= =?utf-8?B?dFhzUHArRTNkWnVHazVjVEVRb0E5cVdpZGs2cnFudHRqKy95aFpjRHFJcSs5?= =?utf-8?B?a1lnMm1DUnNwTU5PZHpmWDVwWVlyN29xTExQVy9nbUJWNjhrR3dOUVBBTThx?= =?utf-8?B?MWNDeWZPZDNrckFOQ1NhN2NIblcrOVpBK0I1QXQzSUNFVjNUTUNNVHg2QXlx?= =?utf-8?B?TURVZU14M0RUVWZxYmNJRmF6Q2pPdjlaVnpiRmlTUG53dVZSYVZYT294Uzho?= =?utf-8?B?TmlXQXhkcGMwZ1c0cmp1R3RrKzA3Mm1aQ3c1azFkRHhpZGxhWVlDK2ZLT3Vp?= =?utf-8?B?ZnhKSkdtOENjVjZ2NGVmSHBwUk9MeHB5Zk42czhUM00yVTFwQm90TlRSejhT?= =?utf-8?B?cXovcUkzUjgrRzNheWRiSTNFem52VG1CNU5pdG83MUtRSmV2Sm9tM1Q4U1NE?= =?utf-8?B?bVM0NlVrMndHZ1NnSTZFZXRQNDAvNlFNZTEvazAvamkyczFtWVNTdDRtbXBR?= =?utf-8?B?dTVnT0wxSmhvbmpSYWd2TmZ3ZWsremtiR3dpZklCbWNjRFhLaVh3aEsyYTA5?= =?utf-8?B?ZDFuTFptVjVGdEVsUkN4L0VxRXhCa1Q2bUpVRFZOOG81QjRYTHJJa2ZnSll0?= =?utf-8?B?TTdVdTFvS1FNMDlMZ1FxbVY1Z3JVNkVFZHRLT1lFRHpHdVRtTXhVQVlod0FU?= =?utf-8?B?OVBwandURTh2cmRQNEtqckRDdWU4VHkxRXp4V21FcWs4ZUV4b01LNzNBRk85?= =?utf-8?B?R1lhTVExekQ3ZlJ2Y3RzTkRLR2ZkRDRnaVV4WjZ3VEJta3RqSGRzSkhObWJ1?= =?utf-8?B?TDhUSmZuck1ha1dzN3BsdlBqcHNndkxrcWN6ZnJGSEJkZWVLY0lkSTd6UFky?= =?utf-8?B?U05WbGxSdmNhTGhObmEvRnR1STdRTi9wZndxQlhlcGg4MWVmVEZyVWU5YnZ4?= =?utf-8?B?Z0JKamdUSUVkQmNld0pONEd4anpoUGlRbUNVWmdqbSt6VUpQSWZUK0orYUQ1?= =?utf-8?B?QUtTM2ttOEFpU1FrS1pKMmdBM0VkcWV3Mk9IWGFLTFlLRFE3c2l0b3YyL3V5?= =?utf-8?B?TnJnVEY5NHJVbkhRblRrVWJ1NTZYWkJHTlFNMVU0a3UzOFJJR2duVG5USGJN?= =?utf-8?B?aEFnVndNTlg0WFowTytBbDlmS1ZiU1A4RFhuUklOeXNuNEoxemorZzkyZTlS?= =?utf-8?B?Yk1wSHo5U3kycktjRGVaNjFLMTY3OTZwRmlCNXZSRmxOMmkvSHVIYjNVVGYx?= =?utf-8?B?ZzlmSWYzLzRmUVpUWTRlR2tvUUdKR3R4c2d1TVR0RTBQTjhxQmx6ZDdYMEJ3?= =?utf-8?B?QllYY3hybGlHTzMxRUVQd1hBUkdkcHJoZjZUVDFjVWxIMDdmL3hVRHVaS3oz?= =?utf-8?B?WUQwUFNiVGMyRU5YcWJub2lma1NIb2tMSUlJNXI3V0p3ejBwckJVdytPR20v?= =?utf-8?B?ZnpJS3BCdkp6VkMyalJyQ3ZRYjlFK3E2R3BhdkNYQ1JPeUhDQjZ5ajR5enhO?= =?utf-8?B?c1hFRi9xZWFwSzdLRC9teVVwRmFTWHU2NHJhQzV4N3doMk9tSTVlMUdndXpK?= =?utf-8?B?MUpyTVBZTCtjVHovTUdRQXFybmVFTE9VcElYcFBlMDh2aEo0YngyOWY4UG11?= =?utf-8?B?eE9uNURDS1NSRzBUd3VKa2Rhdzhyb1pOemxtUlFzUHlKbnhoREN4bGl2OXBy?= =?utf-8?B?ZHorOHFNOThaNlg2U3U3cjQrd1JDbkJReGI3MFZna1dHb0ZneDgvck94algw?= =?utf-8?B?Q2JmcER2cjNkcjEwMDZ5T0M3UGt4cEoyaW9OZUZHb3RuenVFd3NVaWR5aUVG?= =?utf-8?B?MXNPSW4rbVRWdHFaWmc1TFZrSDFOa0l0cEM3VmJpV2tJQ3kvLzBpSURHVmdY?= =?utf-8?B?eHc9PQ==?= X-MS-Exchange-CrossTenant-Network-Message-Id: b7919f42-64ba-43f7-ee5d-08db56e5238e X-MS-Exchange-CrossTenant-AuthSource: DM4PR11MB6502.namprd11.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 17 May 2023 14:43:54.9805 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 46c98d88-e344-4ed4-8496-4ed7712e255d X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: a8kT9JPI2KFXDgYfH8rvZsHZeLucNRh7MXLdoiHFH/M2kPqjv8kox31M/cy7QzP+z/H/t6C5e8sRrzcE6/zfZ1oIep3AptGe8bXOIGElLRg= X-MS-Exchange-Transport-CrossTenantHeadersStamped: CY8PR11MB7268 X-OriginatorOrg: intel.com X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org On 4/19/2023 10:54 AM, Sivaprasad Tummala wrote: > Add eventdev support to enable power saving when no events > are arriving. It is based on counting the number of empty > polls and, when the number reaches a certain threshold, entering > an architecture-defined optimized power state that will either wait > until a TSC timestamp expires, or when events arrive. > > This API mandates a core-to-single-port mapping (i.e. one core polling > multiple ports of event device is not supported). This should be ok > as the general use case will have one CPU core using one port to > enqueue/dequeue events from an eventdev. > > This design is using Eventdev PMD Dequeue callbacks. > > 1. MWAITX/MONITORX: > > When a certain threshold of empty polls is reached, the core will go > into a power optimized sleep while waiting on an address of next RX > descriptor to be written to. > > 2. Pause instruction > > This method uses the pause instruction to avoid busy polling. > > Signed-off-by: Sivaprasad Tummala > --- Hi, few comments > + > +static uint16_t > +evt_clb_pause(uint8_t dev_id __rte_unused, uint8_t port_id __rte_unused, > + struct rte_event *ev __rte_unused, > + uint16_t nb_events, void *arg) > +{ > + const unsigned int lcore = rte_lcore_id(); > + struct queue_list_entry *queue_conf = arg; > + struct pmd_core_cfg *lcore_conf; > + const bool empty = nb_events == 0; > + uint32_t pause_duration = rte_power_pmd_mgmt_get_pause_duration(); > + > + lcore_conf = &lcore_cfgs[lcore]; > + > + if (likely(!empty)) > + /* early exit */ > + queue_reset(lcore_conf, queue_conf); > + else { > + /* can this queue sleep? */ > + if (!queue_can_sleep(lcore_conf, queue_conf)) > + return nb_events; > + > + /* can this lcore sleep? */ > + if (!lcore_can_sleep(lcore_conf)) > + return nb_events; > + > + uint64_t i; > + for (i = 0; i < global_data.pause_per_us * pause_duration; i++) > + rte_pause(); Why not add support for TPAUSE? This is generic code, ethdev code path supports it, and most of this function is duplicated from ethdev implementation. I wish we could unify them somehow, but I can't think of an elegant way to do it off the top of my head. > + > + /* we need this in various places */ > + rte_cpu_get_intrinsics_support(&global_data.intrinsics_support); > + > + switch (mode) { > + case RTE_POWER_MGMT_TYPE_MONITOR: > + /* check if we can add a new port */ > + ret = check_evt_monitor(lcore_cfg, &qdata); > + if (ret < 0) > + goto end; > + > + clb = evt_clb_umwait; > + break; > + case RTE_POWER_MGMT_TYPE_PAUSE: > + /* figure out various time-to-tsc conversions */ > + if (global_data.tsc_per_us == 0) > + calc_tsc(); > + > + clb = evt_clb_pause; > + break; > + default: > + RTE_LOG(DEBUG, POWER, "Invalid power management type\n"); Technically, if we specify "scale" here, the power management scheme would be *unsupported* rather than *invalid*, and thus should return -ENOTSUP rather than -EINVAL. Also, since this is generic code, theoretically this code could in fact support SCALE mode? Would it make sense for eventdev to use that scheme? > +/** > + * @warning > + * @b EXPERIMENTAL: this API may change, or be removed, without prior notice. > + * > + * Disable power management on a specified Ethernet device Rx queue and lcore. > + * > + * @note This function is not thread-safe. > + * > + * @warning This function must be called when all affected Ethernet queues are > + * stopped and no Rx/Tx is in progress! > + * > + * @param lcore_id > + * The lcore the Rx queue is polled from. > + * @param dev_id > + * The identifier of the device. > + * @param port_id > + * Event port identifier of the Event device. > + * @return > + * 0 on success > + * <0 on error > + */ > +__rte_experimental > +int > +rte_power_eventdev_pmgmt_port_disable(unsigned int lcore_id, > + uint8_t dev_id, uint8_t port_id); It would've been nice if we didn't have to reimplement the same logic for every new device type, but seeing how we do not have any unified driver API, I don't have any bright ideas on how to do it better. -- Thanks, Anatoly