Re: [dpdk-dev] [PATCH v2 0/4] Introduce IF proxy library

[Thomas Monjalon <thomas@monjalon.net>]

02/04/2020 15:48, Andrzej Ostruszka [C]:
> On 3/26/20 6:42 PM, Andrzej Ostruszka wrote:
> > On 3/25/20 12:11 PM, Morten Brørup wrote:
> [...]
> >> And I am still strongly opposed to the callback method:
> > 
> > Noted - however for now I would like to keep them.  I don't have much
> > experience with this library so if they prove to be inadequate then we
> > will remove them.  Right now they seem to add some flexibility that I like:
> > - if something should be changed globally and once (and it is safe to do
> >   so!) then it can be done from the callback
> > - if something can be prepared once and consumed later by lcores then it
> >   can be done in callback and the callback returns 0 so that event is
> >   still queued and lcores (under assumption that queues are per lcore)
> >   pick up what has been prepared.
> 
> Morten
> 
> I've been thinking about this a bit and would like to know your (and
> others) opinion about following proposed enhancement.
> 
> Right now, how queues are used is left to the application decision (per
> lcore, per port, ...) - and I intend to keep it that way - but they are
> "match all".  What I mean by that is that (unlike callbacks where you
> have separate per event type) queue has no chance to be selective.
> 
> So if someone would like to go with queues only they he would have to
> coordinate between queues (or their "owners") which one does the
> handling of an event that supposedly should be handled only once.
> 
> Let's take this forwarding example - the queues are per lcore and each
> lcore keeps its own copy of ARP table (hash) so when the change is
> noticed the event is queued to all registered queue, each lcore updates
> its own copy and everything is OK.  However the routing is global (and
> right now is updated from callback) and if no callback is used for that
> then the event would be queued to all lcores and application would need
> to select the one which does the update.
> 
> Would that be easier/better to register queue together with a bitmask of
> event types that given queue is accepting?  Than during setup phase
> application would select just one queue to handle "global" events and
> the logic of event handling for lcores should be simplier.
> 
> Let me know what you think.

I think we want to avoid complicate design.
So let's choose between callback and message queue.
I vote for message queue because it can handle any situation,
and it allows to control the context of the event processing.
The other reason is that I believe we need message queueing for
other purposes in DPDK (ex: multi-process, telemetry).

You start thinking about complex message management.
And I start thinking about other usages of message queueing.
So I think it is the right time to introduce a generic messaging in DPDK.
Note: the IPC rte_mp should be built on top of such generic messaging.

If you agree, we can start a new email thread to better discuss
the generic messaging sub-system.
I describe here the 3 properties I have in mind:

1/ Message policy
One very important rule in DPDK is to let the control to the application.
So the messaging policy must be managed by the application via DPDK API.

2/ Message queue
It seems we should rely on ZeroMQ. Here is why:
http://zguide.zeromq.org/page:all#Why-We-Needed-ZeroMQ

3/ Message format
I am not sure whether we can manage with "simple strings", TLV,
or should we use something more complex like protobuf?