drivers use of service cores

drivers use of service cores

[Thomas Monjalon]

Hello,

I think we can improve the developer experience for using service cores
from a driver, like finding or allocating a service core.
We may take some code and ideas from sfc and nfp drivers,
like in these functions:
	nfp_map_service()
	sfc_mae_counter_service_register()
	sfc_get_service_lcore()

If it is not possible to use a service core,
we could default to using a control thread.
So the driver would never fail because of a thread initialization.

What do you think about proposing such a high level API
in order to get more drivers using it?

RE: drivers use of service cores

[Van Haaren, Harry]

> -----Original Message-----
> From: Thomas Monjalon <thomas@monjalon.net>
> Sent: Tuesday, September 5, 2023 4:40 PM
> To: Van Haaren, Harry <harry.van.haaren@intel.com>;
> david.marchand@redhat.com; Richardson, Bruce
> <bruce.richardson@intel.com>; Andrew Rybchenko
> <andrew.rybchenko@oktetlabs.ru>; Chaoyong He
> <chaoyong.he@corigine.com>; Niklas Soderlund
> <niklas.soderlund@corigine.com>
> Cc: ferruh.yigit@amd.com; dev@dpdk.org; Tyler Retzlaff
> <roretzla@linux.microsoft.com>
> Subject: drivers use of service cores
> 
> Hello,

Hi All,

For context, Thomas and I (and a few others) had a brief discussion about this topic
at userspace in Dublin earlier this week.  I have a bit of better understanding of the
problem-space, and we made some progress in technical solutions too.

> I think we can improve the developer experience for using service cores
> from a driver, like finding or allocating a service core.
> We may take some code and ideas from sfc and nfp drivers,
> like in these functions:
> 	nfp_map_service()
> 	sfc_mae_counter_service_register()
> 	sfc_get_service_lcore()
> 
> If it is not possible to use a service core, we could default to using a control thread.
> So the driver would never fail because of a thread initialization.

There was input from a few people that "hidden threads" that their DPDK application
doesn't know about can cause issues (e.g. a driver creating a thread "behind the application's back").
I think Thomas suggested a callback function the application could hook-into, to either accept/decline
the drivers "request" to create a thread.

The default could be "accept" if the application doesn't hook the callback, allowing drivers to default to
achieving work, and allowing power-users to manually handle specific threading-requirements. I have
not strong preference here, just writing down the discussions and feedback from Userspace.

> What do you think about proposing such a high level API
> in order to get more drivers using it?

I believe service-cores was required to transparently enable certain use-cases of HW-acceleration,
Initially Eventdev/SW PMD, but it is of course possible for other components in DPDK to use it.

I do recall some folks had concerns over "scope creep" when initially discussing service-cores upstreaming, and perhaps they're right.
I'm not sure how much more functionality is desired here, vs better usability of the service-cores APIs. Perhaps a POC patch of the
NFP, SFC, etc use-cases would help drive towards a code-level discussion?

Regards, -Harry

RE: drivers use of service cores

[Morten Brørup]

+Chris Brezovec, took part in the discussion in Dublin

> From: Van Haaren, Harry [mailto:harry.van.haaren@intel.com]
> Sent: Friday, 15 September 2023 14.59
> 
> > From: Thomas Monjalon <thomas@monjalon.net>
> > Sent: Tuesday, September 5, 2023 4:40 PM
> >
> > Hello,
> 
> Hi All,
> 
> For context, Thomas and I (and a few others) had a brief discussion
> about this topic
> at userspace in Dublin earlier this week.  I have a bit of better
> understanding of the
> problem-space, and we made some progress in technical solutions too.
> 
> > I think we can improve the developer experience for using service
> cores
> > from a driver, like finding or allocating a service core.
> > We may take some code and ideas from sfc and nfp drivers,
> > like in these functions:
> > 	nfp_map_service()
> > 	sfc_mae_counter_service_register()
> > 	sfc_get_service_lcore()
> >
> > If it is not possible to use a service core, we could default to using
> a control thread.
> > So the driver would never fail because of a thread initialization.
> 
> There was input from a few people that "hidden threads" that their DPDK
> application
> doesn't know about can cause issues (e.g. a driver creating a thread
> "behind the application's back").
> I think Thomas suggested a callback function the application could hook-
> into, to either accept/decline
> the drivers "request" to create a thread.
> 
> The default could be "accept" if the application doesn't hook the
> callback, allowing drivers to default to
> achieving work, and allowing power-users to manually handle specific
> threading-requirements. I have
> not strong preference here, just writing down the discussions and
> feedback from Userspace.
> 
> > What do you think about proposing such a high level API
> > in order to get more drivers using it?
> 
> I believe service-cores was required to transparently enable certain
> use-cases of HW-acceleration,
> Initially Eventdev/SW PMD, but it is of course possible for other
> components in DPDK to use it.
> 
> I do recall some folks had concerns over "scope creep" when initially
> discussing service-cores upstreaming, and perhaps they're right.
> I'm not sure how much more functionality is desired here, vs better
> usability of the service-cores APIs. Perhaps a POC patch of the
> NFP, SFC, etc use-cases would help drive towards a code-level
> discussion?

Since the discussion in Dublin, I have given this some thoughts. Here's what I think...

My key objection is this: CPU cycles and CPU cores might be scarce resources. And even though a driver has some "important" work to do, the application might have some other work to do, which is more important and perhaps also timing sensitive. It is a core design principle of DPDK that the application is in full control! Drivers should not have the ability to take away resources from the application; resources should be explicitly allocated and given to the driver by the application.

If I have a piece of single- or dual-core hardware, I certainly don't want any drivers to be in control of when they can spend my CPU cycles.

It should be obvious that if a driver needs extra CPU resources (in the form of dedicated CPU cores or unregistered non-EAL threads), the driver should document this special requirement.

I don't know the drivers' detailed needs for additional threads, but I guess that voluntary scheduling would be a good solution for this. And service cores is a kind of voluntary scheduling. So instead of offering a fallback to unregistered non-EAL threads for such drivers, we could require that applications (using such drivers) must make the service cores infrastructure available to these drivers. This would leave the application in control of CPU scheduling.

<just joking>
Now, let's introduce some of the foreseen scope creep: The service cores scheduler could be updated to support real-time requirements, so the application's real-time deadlines are not exceeded because of some driver hogging the CPU.
</just joking>

Unregistered non-EAL threads should not be used for timing sensitive or deadline-bound tasks. They might be affected by "noisy neighbors" taking away their CPU time.

Although I don't have a solution, I remain critical: We can either add a "quick fix" to continue supporting the mess introduced by these drivers, or we can stop to analyze the situation and provide a proper solution.

Re: drivers use of service cores

[Stephen Hemminger]

On Fri, 15 Sep 2023 15:54:02 +0200
Morten Brørup <mb@smartsharesystems.com> wrote:

> > For context, Thomas and I (and a few others) had a brief discussion
> > about this topic
> > at userspace in Dublin earlier this week.  I have a bit of better
> > understanding of the
> > problem-space, and we made some progress in technical solutions too.
> >   
> > > I think we can improve the developer experience for using service  
> > cores  
> > > from a driver, like finding or allocating a service core.
> > > We may take some code and ideas from sfc and nfp drivers,
> > > like in these functions:
> > > 	nfp_map_service()
> > > 	sfc_mae_counter_service_register()
> > > 	sfc_get_service_lcore()
> > >
> > > If it is not possible to use a service core, we could default to using  
> > a control thread.  
> > > So the driver would never fail because of a thread initialization.  
> > 
> > There was input from a few people that "hidden threads" that their DPDK
> > application
> > doesn't know about can cause issues (e.g. a driver creating a thread
> > "behind the application's back").
> > I think Thomas suggested a callback function the application could hook-
> > into, to either accept/decline
> > the drivers "request" to create a thread.
> > 
> > The default could be "accept" if the application doesn't hook the
> > callback, allowing drivers to default to
> > achieving work, and allowing power-users to manually handle specific
> > threading-requirements. I have
> > not strong preference here, just writing down the discussions and
> > feedback from Userspace.
> >   
> > > What do you think about proposing such a high level API
> > > in order to get more drivers using it?  

> > I believe service-cores was required to transparently enable certain
> > use-cases of HW-acceleration,
> > Initially Eventdev/SW PMD, but it is of course possible for other
> > components in DPDK to use it.
> > 
> > I do recall some folks had concerns over "scope creep" when initially
> > discussing service-cores upstreaming, and perhaps they're right.
> > I'm not sure how much more functionality is desired here, vs better
> > usability of the service-cores APIs. Perhaps a POC patch of the
> > NFP, SFC, etc use-cases would help drive towards a code-level
> > discussion?  
> 
> Since the discussion in Dublin, I have given this some thoughts. Here's what I think...
> 
> My key objection is this: CPU cycles and CPU cores might be scarce resources. And even though a driver has some "important" work to do, the application might have some other work to do, which is more important and perhaps also timing sensitive. It is a core design principle of DPDK that the application is in full control! Drivers should not have the ability to take away resources from the application; resources should be explicitly allocated and given to the driver by the application.


In the embedded world, it is common that cores and other resources are scarce.
At my previous role, the number of available cores and memory was controlled by cgroups
and the actual allocation decisions were done high up in the project management
team. I doubt this is a unique case.

Also, when a network virtual appliance is run in a VM. The number of cores
in the VM impacts the cost. You pay a lot more for more cores.
Bottom line, cores are not free.

Many applications are not setup to use service lcores. And some are
not even setup to use timers.