Re: [RFC PATCH 1/5] eventdev: add power monitoring API on event port

[Ferruh Yigit <ferruh.yigit@amd.com>]

On 4/25/2023 7:19 AM, Mattias Rönnblom wrote:
> On 2023-04-24 18:06, Ferruh Yigit wrote:
>> On 4/19/2023 11:15 AM, Jerin Jacob wrote:
>>> On Wed, Apr 19, 2023 at 3:24 PM Sivaprasad Tummala
>>> <sivaprasad.tummala@amd.com> wrote:
>>>>
>>>> A new API to allow power monitoring condition on event port to
>>>> optimize power when no events are arriving on an event port for
>>>> the worker core to process in an eventdev based pipelined application.
>>>>
>>>> Signed-off-by: Sivaprasad Tummala <sivaprasad.tummala@amd.com>
>>>> + *
>>>> + * @param dev_id
>>>> + *   Eventdev id
>>>> + * @param port_id
>>>> + *   Eventdev port id
>>>> + * @param pmc
>>>> + *   The pointer to power-optimized monitoring condition structure.
>>>> + *
>>>> + * @return
>>>> + *   - 0: Success.
>>>> + *   -ENOTSUP: Operation not supported.
>>>> + *   -EINVAL: Invalid parameters.
>>>> + *   -ENODEV: Invalid device ID.
>>>> + */
>>>> +__rte_experimental
>>>> +int
>>>> +rte_event_port_get_monitor_addr(uint8_t dev_id, uint8_t port_id,
>>>> +               struct rte_power_monitor_cond *pmc);
>>>
>>> + eventdev driver maintainers
>>>
>>> I think, we don't need to expose this application due to applications
>>> 1)To make applications to be transparent whether power saving is enabled or not?
>>> 2)Some HW and Arch already supports power managent in driver and in HW
>>> (Not using  CPU architecture directly)
>>>
>>> If so, that will be translated to following,
>>> a) Add rte_event_port_power_saving_ena_dis(uint8_t dev_id, uint8_t
>>> port_id, bool ena) for controlling power saving in slowpath.
>>> b) Create reusable PMD private function based on the CPU architecture
>>> power saving primitive to cover the PMD don't have native power saving
>>> support.
>>> c)Update rte_event_dequeue_burst() burst of PMD callback to use (b).
>>>
>>>
>>
>> Hi Jerin,
>>
>> ethdev approach seems applied here.
>>
>> In ethdev, 'rte_event_port_get_monitor_addr()' equivalent is
>> 'rte_eth_get_monitor_addr()'.
>>
>> Although 'rte_eth_get_monitor_addr()' is public API, it is currently
>> only called from Rx/Tx callback functions implemented in the power library.
>> But I assume intention to make it public is to enable users to implement
>> their own callback functions that has custom algorithm for the power
>> management.
>>
>> And probably same is true for the 'rte_event_port_get_monitor_addr()'.
>>
>>
>> Also instead of implementing power features for withing PMDs, isn't it
>> better to have a common eventdev layer for it?
>>
> 
> To allow that question to be answered, I think you need to be more 
> specific what are "power features".
> 
>  From what it seems to me, the get_monitor_addr() family of functions 
> address the pretty narrow case of allowing umwait (or the non-x86 
> equivalent) to be used to wait for new events. It leaves all the heavy 
> lifting to the app, which needs to figure out how loaded each CPU core 
> is, what backlog of work there is, how to shuffle work around to get the 
> most out of the power, how to translate wall-clock latency requirements 
> into the equation, what CPU (and/or accelerator/NIC-level) power 
> features to employ (e.g., DVFS, sleep states, umwait), etc.
> 
> In the context of Eventdev, optimizing for power may include packing 
> more flows into the same port, in low-load situations. Keeping a few 
> cores relatively busy, and the rest in some deep sleep state may well be 
> the best solution for certain (most?) systems. For such a feature to 
> work, the event device must be in the loop, but the mechanics could (and 
> should) be generic. Eventdev could also control DVFS.
> 
> A reasonably generic power management mechanism could go into Eventdev a 
> combination of the event device drivers, and some generic functions). 
> (Various policies would still need to come from the app.)
> 
> I think keeping this kind of functionality in Eventdev works well 
> provided the only source of work is Eventdev events (i.e., most or all 
> fast path lcores are "pure" event-based lcores). No non-eventdev timer 
> wheels, no non-eventdev lookaside accelerator or I/O device access, no 
> control plane rings to poll, etc.
> 
> If such a model is too limiting, another option is to put the central 
> power management function in the service framework (with a lot of help 
> from Eventdev, RTE timer, and other sources of work as well).
> 

Hi Mattias,

The current power management features referred in the scope of this
patch is around umwait use case as you mentioned.

It has default callbacks that application benefit with minimal
involvement from application, but if application wants more
sophisticated algorithm, needs to implement its own functions.

And I agree to have more comprehensive power management, it has benefit
but it has to start somewhere and we can grow it more by time. Also it
requires more support from community, not just from some vendors.

I think it is a good start to enable some HW features for power
management and make existing APIs more HW agnostic.

>> For the PMDs benefit from HW event manager, just not implementing
>> .get_monitor_addr() dev_ops will make them free from power related APIs.
>>
>>
>>
>