From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id D3FB642A4D; Wed, 3 May 2023 10:14:01 +0200 (CEST) Received: from mails.dpdk.org (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 641794114B; Wed, 3 May 2023 10:14:01 +0200 (CEST) Received: from NAM12-DM6-obe.outbound.protection.outlook.com (mail-dm6nam12on2053.outbound.protection.outlook.com [40.107.243.53]) by mails.dpdk.org (Postfix) with ESMTP id AE9C741144 for ; Wed, 3 May 2023 10:13:59 +0200 (CEST) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=VZloedG9oZQ/883sKr+EzZXaXcP4ZMt00dLAPdOXckjbHYvt3vIGs1O9Luuhw5ne4gIe72WNcPbbgT8A4Ser/LQcXW9JQI+QWTVD69/R505DVerC96IZZ1yBynPx4s0Jr8kcINiaEvmazLLKUlouXJvHTx+wLANDHkhttGQzGxce/PAQvRv9wCP4KklAKJXVhdU0aZiFhFw/I9U/yuwWtJIUpPMoEdNQMbp/lnBudMzwiAEuLsVI4nc4Iio+qwV3BdZxDOd3mq56tgKFzevEBxJECnHyVWxby9nQ38yPFcYEoJ4uTpoh/ENA80zl5IUwqdiZW+1zzt6TsbrKekEAgQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=If8sIpt6fgc09f93LZ+YT4Ct8GRpJ28WblOx9UOI9h4=; b=ecUmSDEgYxGXeZRccGU4aj3FSTD8gGbYW66C+oN+Z8wSmWvDx7IA6cRKN8Q2Itc86xS3Yb7dQDQ1V+lZq1civgDLkqUf5YFsOTVyBHIzmAuDBGPIEeH83jVNa8lH6P1eGbct6mcXvbNk/tKBcHb/kSwc5Eca3llkvrUAeWbjQCxv0ThslYODenkuWybZu1+KxiLAGBxLUB5px7fjzglSsSQZrnzE7BBIHyEtuY09qAocS1khJZUz1trlDEU8rAfOfebcJAJXHPurWis/JcsmMVf5TF/uNV4/8qYF5uYOcyJZN7mSZOnclEdM97SrUf6iGVQdc3biNRlEDu/d5GlSMQ== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=amd.com; dmarc=pass action=none header.from=amd.com; dkim=pass header.d=amd.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amd.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=If8sIpt6fgc09f93LZ+YT4Ct8GRpJ28WblOx9UOI9h4=; b=M/MO3iIp4HHgKufTIIo5iWt4QD4DvNaAJs47u3hZdx6LPynJ+lFfkMyK+Pn5S8DkH1Po2aJWTIF9ahQYWjvMiJupbzzpuIim4vr8CZLOhXcZO5yIn2JOdh468eVkEcD1cHwZSFbjvAmRJPxHfCVxQivzLDdJ6lbfWnSmvUcZxqo= Authentication-Results: dkim=none (message not signed) header.d=none;dmarc=none action=none header.from=amd.com; Received: from CH2PR12MB4294.namprd12.prod.outlook.com (2603:10b6:610:a9::11) by CH3PR12MB9172.namprd12.prod.outlook.com (2603:10b6:610:198::7) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.6340.31; Wed, 3 May 2023 08:13:57 +0000 Received: from CH2PR12MB4294.namprd12.prod.outlook.com ([fe80::e818:77ea:75b5:f8cc]) by CH2PR12MB4294.namprd12.prod.outlook.com ([fe80::e818:77ea:75b5:f8cc%5]) with mapi id 15.20.6363.022; Wed, 3 May 2023 08:13:57 +0000 Message-ID: Date: Wed, 3 May 2023 09:13:00 +0100 User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:102.0) Gecko/20100101 Thunderbird/102.10.1 To: Jerin Jacob Cc: Sivaprasad Tummala , david.hunt@intel.com, jerinj@marvell.com, harry.van.haaren@intel.com, dev@dpdk.org, Pavan Nikhilesh , "McDaniel, Timothy" , Shijith Thotton , Hemant Agrawal , Sachin Saxena , =?UTF-8?Q?Mattias_R=c3=b6nnblom?= , Peter Mccarthy , Liang Ma References: <20230419095427.563185-1-sivaprasad.tummala@amd.com> <324d5201-da95-f926-5580-f74ca5c09799@amd.com> Content-Language: en-US From: Ferruh Yigit Subject: Re: [RFC PATCH 1/5] eventdev: add power monitoring API on event port In-Reply-To: Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-ClientProxiedBy: LO4P265CA0048.GBRP265.PROD.OUTLOOK.COM (2603:10a6:600:2ac::8) To CH2PR12MB4294.namprd12.prod.outlook.com (2603:10b6:610:a9::11) MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: CH2PR12MB4294:EE_|CH3PR12MB9172:EE_ X-MS-Office365-Filtering-Correlation-Id: 5e0c5507-2d76-45ef-3782-08db4bae5786 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: r2KDxsSpA70qXekoRSmn0/1uOFNv5dib2TdOQ9SohCXZvW1AfBBOICCllwOLwHlmAfjPlWT9pNqZ6rgBcOWU8hxkr8FAUfcHhDtYBkxZ1L2kFhhCZCWktmlgFHzHUm3ZwIIQsn6PJ9SbDXDb1xT6Dn5UyXUZ78U+tRa0HH1jpOKMoIDgcmYGk0G0x4n5pfLkbA8SQND1lA7JZ7lbddxeOusRJsBnU8fkgpy4MPmTw/G8C/PRfbsrcYpsTdzHCi2OkeoIselpKADexYLxuVX/pHhpQP89YQktzozwejghF6AMHeqLyHgSB9xd3F58Svks77RTMWHGk1yzZgJfbeVFMb6U8dq1r96DGFxp5C1Peil3ppj0UiUGQggksQZ7xi6u/Nnipbxi6JZmz1hMbWw06HzgygW3r6uT72tI8ik2IDrdLjWSmgDZht4JMrgD3s7wyEJLnqFPaoBhO2qjhv0Gd2/XFmHtLRDDG3AgRRFW92q3Eq8NUa8i+YP4CcA2iQ6mwi+KENnUcE+mbdXkBG8faT0zQq9CVKazHoW4yvyWUBrz7pgw1hNB7LymW8PknYBDp7LGld8KiuEHetKtTVXV2lB+oyo4oP36Y7bnRx0rtP9nKDX8FpM9cwmdRwLh1Mt2N2XcCeRbRYKL0byVqIbqGg== X-Forefront-Antispam-Report: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:CH2PR12MB4294.namprd12.prod.outlook.com; PTR:; CAT:NONE; SFS:(13230028)(4636009)(39860400002)(366004)(376002)(396003)(346002)(136003)(451199021)(26005)(186003)(53546011)(6506007)(6512007)(66556008)(83380400001)(2616005)(316002)(66476007)(41300700001)(66946007)(5660300002)(6916009)(8676002)(6486002)(8936002)(4326008)(44832011)(478600001)(54906003)(7416002)(6666004)(2906002)(36756003)(86362001)(31696002)(38100700002)(31686004)(66899021)(45980500001)(43740500002); DIR:OUT; SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: =?utf-8?B?akUrWnRSenRiUmo0M0QzbkkySFVQYXRRUXdpTTRMRE9yYXJubnlPdURxYVY0?= =?utf-8?B?MGVIQjYvc0M4cytkOE5Vc1pXRzFtOVlUaFdyUVZCVGRwNlZaeDQzd2U4OUk0?= =?utf-8?B?Vm5qa1RoV2xvQUt5dytYM2JlaXdHTTA3dlJveWxJbHZhaWdoZEVRNGx1bjJW?= =?utf-8?B?aXRkaXpna1lVWVRNRjZPVlQzTEVSYmcxeFZ4UEkranJiZnFsaDIySGJXYktU?= =?utf-8?B?bEQvU1Q3NzJnN1czQTBDNVJzaVN0blV1dGhRdWk1cGFMNWw1aFNjTTIwK3ll?= =?utf-8?B?Um1PQ0FNRUF6czNta3BxME9OaGErZ0FUS2VFUitoeWNMVkkzV1VXazZHTkJH?= =?utf-8?B?c2NtUTUvZnB2dGVtOTZERUkrTzRCbmh2OVFzaXpNWFpjeFM5QXRBdVdYblFp?= =?utf-8?B?NjF4TCtjRXh5UnRKSHRHd21LbHhCaU5DdW1jMFJxS2FPdS8wVktocTdlTTFM?= =?utf-8?B?RkE5L3FBYzJCQURXdjFnRmVERHFpdUZDeC9UV2tSZGJ1Mm16bUptK1REL1lR?= =?utf-8?B?eENCRm9YejFHbTh3QWRDWWIvbEl2WFllMDNGeEdxYmJUVmsyRkFhSkdYWXhC?= =?utf-8?B?M3BhWVFCdGwzNzMwVWNIOGhRTllSa04zQUFoZUx2bEhZSzRtK0oraWhIT3VQ?= =?utf-8?B?NVhGYUtReWRSdHVHSlhJL0tDUkQzdk5EdGw2UmdVb1VqTFFKa2l0Rk9aRE9B?= =?utf-8?B?bEFaaGY4cGcrdDdMQWorVU5WWEp3WWZPMkVzT09DK1lUalhQUU5ITFNDNHJi?= =?utf-8?B?dEc3M0Z4WFB0OW5BWHBGVHVuWDBXaXNMY0JhS3l3dEcwUVlkVk5HNlV6bExu?= =?utf-8?B?NWFjQ1ZkT0Y5SlZCWmRZd0FXV0lsMmhVVFpHN0lrTVlMeGNvVFNTSWN1dlRk?= =?utf-8?B?cU1zK1lQcHRzSGoyYUw5S0FvMmU3d1FHa1R0bGpoTWFhVk9WZFFYT3FZNnRH?= =?utf-8?B?VHVabHZBZ3plR2ZwNXB3RElYa1lpeHV2bjVISXZHd0xEcEd0NnRnQ0FUNjV3?= =?utf-8?B?bFVuRm9BT2MzQnlvZFBldjFwSk9YbXlMWlJia1JPay9ySmVKMjUvRnpZVm05?= =?utf-8?B?c2YzQVoxTDVIYnVpNERoUmRZREtIQk13MjRaYjFJVXl6VEE5cFJzVlhnZTBx?= =?utf-8?B?ZGlKZlhwUzlUQUpJbVBldnRTSC9wRXB1dmdaM0M5QnJpSTIvdW8yZHEzNk53?= =?utf-8?B?azdRYlhSMG52Uzl6ZnVKUWIzdjRSRDBxd1hoWUxDeUVaNThVQnNnN3JuMVBl?= =?utf-8?B?RzNIUGN3MGthYlFlelJnQXlNQ2lzbnk5UmVLZVovTzBiekNEdE9ZQnpZQ1pF?= =?utf-8?B?cXZZYkVHRTVxbWt2aDZ3c0JEK3czeXNhRXVUREFtN3F1YlkwTTBqZ2k5MTE5?= =?utf-8?B?QUNHMCtDSzVUck5tOUtoZ1R5OWEwOGtCYTh0YXI3SUlwVjF4cmZaN2dNNlV2?= =?utf-8?B?WDNXS0ZzaTdlTTBiVEllcXpIR2xZd1NndllXV2pCUjhtaVgyN0hKcDR4QVhR?= =?utf-8?B?cjF5MTRLMmQ4NldObExaN3lWSTY4eGpiQUVmeFNVaUluVE02L29SYnNWMXFs?= =?utf-8?B?ejBqazhTaWpNZ1U5OURuWTBxYlZZRTkxV0hYak5IdzRzdHJNRS90T3g2NjF4?= =?utf-8?B?Zm5rL0FDRFdHSUkwMWZQdTduK3FqWnkyREVSYzUwZXBEMis5RGVINzdpV2ZY?= =?utf-8?B?SnVGODJHQTEvalNXQTdtK0JvenpaUGZ2c2xoQUo5OTJ6N3MrVWRWb2hTQ2s0?= =?utf-8?B?TnBPRjllUEZWM2xJNnVuUEFaUVBmdjRYM3N5V2NxT1ZYR1EwUUlnc0N3N1hF?= =?utf-8?B?OS9RUVJEZ1YzdVFjd0N4ckFwRFV3Ukk2SDg0ZnVpb0xqcTNJcVYveCtyZ1hN?= =?utf-8?B?L05ad1MxTTBKbE5zTElDbk1QT3BhQnVNQ2xNd3VralZEVWZrYnJnQ3U1UTF6?= =?utf-8?B?UExoSlFqZU9reXdmTWMya2wrdlk1R2NqWk0xQkV3dUxURE5LckZLK3VxUDZW?= =?utf-8?B?MmxoSE5QekNxeE9jQ2tLRUhWWnBqU3IvSTgrRkdnMVVwN0FxWGIwQXp3SkZz?= =?utf-8?B?OU02TVFPUTZ2L0tLS0ZwSCtWeExZSUNqOHQyWk9RVFRFN1N5Q05OazNvbWxm?= =?utf-8?Q?P3ubHNdr3yjJO+c0wdj4Hj2/6?= X-OriginatorOrg: amd.com X-MS-Exchange-CrossTenant-Network-Message-Id: 5e0c5507-2d76-45ef-3782-08db4bae5786 X-MS-Exchange-CrossTenant-AuthSource: CH2PR12MB4294.namprd12.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 03 May 2023 08:13:57.0259 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: G8O8ToUHklf4SUx97xtcrO8PDxk/GYy9NHcuPIzSOKYOMVkASh14u08Hte+tDM2J X-MS-Exchange-Transport-CrossTenantHeadersStamped: CH3PR12MB9172 X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org On 5/3/2023 8:58 AM, Jerin Jacob wrote: > On Tue, May 2, 2023 at 4:49 PM Ferruh Yigit wrote: >> >> On 4/25/2023 5:09 AM, Jerin Jacob wrote: >>> On Mon, Apr 24, 2023 at 9:36 PM Ferruh Yigit wrote: >>>> >>>> On 4/19/2023 11:15 AM, Jerin Jacob wrote: >>>>> On Wed, Apr 19, 2023 at 3:24 PM Sivaprasad Tummala >>>>> wrote: >>>>>> >>>>>> A new API to allow power monitoring condition on event port to >>>>>> optimize power when no events are arriving on an event port for >>>>>> the worker core to process in an eventdev based pipelined application. >>>>>> >>>>>> Signed-off-by: Sivaprasad Tummala >>>>>> + * >>>>>> + * @param dev_id >>>>>> + * Eventdev id >>>>>> + * @param port_id >>>>>> + * Eventdev port id >>>>>> + * @param pmc >>>>>> + * The pointer to power-optimized monitoring condition structure. >>>>>> + * >>>>>> + * @return >>>>>> + * - 0: Success. >>>>>> + * -ENOTSUP: Operation not supported. >>>>>> + * -EINVAL: Invalid parameters. >>>>>> + * -ENODEV: Invalid device ID. >>>>>> + */ >>>>>> +__rte_experimental >>>>>> +int >>>>>> +rte_event_port_get_monitor_addr(uint8_t dev_id, uint8_t port_id, >>>>>> + struct rte_power_monitor_cond *pmc); >>>>> >>>>> + eventdev driver maintainers >>>>> >>>>> I think, we don't need to expose this application due to applications >>>>> 1)To make applications to be transparent whether power saving is enabled or not? >>>>> 2)Some HW and Arch already supports power managent in driver and in HW >>>>> (Not using CPU architecture directly) >>>>> >>>>> If so, that will be translated to following, >>>>> a) Add rte_event_port_power_saving_ena_dis(uint8_t dev_id, uint8_t >>>>> port_id, bool ena) for controlling power saving in slowpath. >>>>> b) Create reusable PMD private function based on the CPU architecture >>>>> power saving primitive to cover the PMD don't have native power saving >>>>> support. >>>>> c)Update rte_event_dequeue_burst() burst of PMD callback to use (b). >>>>> >>>>> >>>> >>>> Hi Jerin, >>> >>> Hi Ferruh, >>> >>>> >>>> ethdev approach seems applied here. >>> >>> Understands that. But none of the NIC HW supports power management at >>> HW level like eventdev, so that way >>> for what we are doing for ethdev is a correct abstraction for ethdev. >>> >> >> What I understand is there is HW based event manager and SW based ones, >> SW based ones can benefit more from CPU power optimizations, for HW >> event managers if there is not enough benefit they can just ignore the >> feature. >> >>>> >>>> In ethdev, 'rte_event_port_get_monitor_addr()' equivalent is >>>> 'rte_eth_get_monitor_addr()'. >>>> >>>> Although 'rte_eth_get_monitor_addr()' is public API, it is currently >>>> only called from Rx/Tx callback functions implemented in the power library. >>>> But I assume intention to make it public is to enable users to implement >>>> their own callback functions that has custom algorithm for the power >>>> management. >>> >>> If there is a use case for customizing with own callback, we can provide that. >>> Provided NULL is valid with default algorithm. >>> >>>> >>>> And probably same is true for the 'rte_event_port_get_monitor_addr()'. >>>> >>>> >>>> Also instead of implementing power features for withing PMDs, isn't it >>>> better to have a common eventdev layer for it? >>> >>> We can have rte_evetdev_pmd_* APIs as non-public APIs. >>> My only objection is to NOT introduce _monitor_ APIs at eventdev level, >>> Instead, _monitor_ is one way to do it in SW, So we need higher level >>> of abstraction. >>> >> >> I see, this seems a trade off between flexibility and usability. If >> application has access to _monitor_ APIs, they can be more flexible to >> implement their own logic. > > OK. > >> >> Another option can be application provides the policy with an API and >> monitor API used to realize the policy, but for this case it can be >> challenge to find and implement correct policies. > > OK. If we can enumerate the policies, then it will be ideal. > On plus side, there will not be any changes in needed in lib/power/ > If we are talking about a power framework that user defines policies, I expect parsing/defining policies will be in the power library and will require changes in the power library anyway. But as mentioned above it is difficult to define a proper policy, this is not really related to eventdev, more a power library issue. We can continue to provide flexibility to user in eventdev and discuss the policy if a wider forum. > >> >>>> >>>> For the PMDs benefit from HW event manager, just not implementing >>>> .get_monitor_addr() dev_ops will make them free from power related APIs. >>> >>> But application fast path code gets diverged by exposing low level primitives. >>> >> >> I am not clear with concern above, but for application that use default >> callbacks, 'rte_power_eventdev_pmgmt_port_enable()' needs to be called >> to enable this feature, if not called datapath is not impacted. >> And if not dequeue callback added at all, custom or default, data path >> is not impacted at all. > > Concerns are around following code[1] when callback is not registered > for this use case. > In eventdev, we are using _one packet at a time_ for a lot of use case > with latency critical workload like L1 processing. > On such cases, the following code will add up. > > [1] > cb = __atomic_load_n((void **)&fp_ops->ev_port.clbk[port_id], > __ATOMIC_RELAXED); > if (unlikely(cb != NULL)) > nb_rx = rte_event_dequeue_callbacks(dev_id, port_id, ev, nb_rx, cb); > > I see two options, > 1) Enumerate the power policy and let driver implement through > non-public PMD helper functions > OR > 2)Move the power management callback to driver via non-public PMD > helper functions to avoid cost of > PMDs where power managment done in HW and to remove above extra check > when NO callback is registered[1] > Got it, yes there is an additional check with event callbacks, we can add a compiler flag around it as done in ethdev to let it not compiled when not needed, will it work?