DPDK patches and discussions
 help / color / mirror / Atom feed
From: "Morten Brørup" <mb@smartsharesystems.com>
To: "Jerin Jacob" <jerinjacobk@gmail.com>,
	"Thomas Monjalon" <thomas@monjalon.net>,
	"Andrzej Ostruszka [C]" <aostruszka@marvell.com>
Cc: "David Marchand" <david.marchand@redhat.com>,
	"dpdk-dev" <dev@dpdk.org>,
	"Richardson, Bruce" <bruce.richardson@intel.com>,
	"Anatoly Burakov" <anatoly.burakov@intel.com>
Subject: Re: [dpdk-dev] [PATCH v2 0/4] Introduce IF proxy library
Date: Fri, 3 Apr 2020 23:18:39 +0200	[thread overview]
Message-ID: <98CBD80474FA8B44BF855DF32C47DC35C60F40@smartserver.smartshare.dk> (raw)
In-Reply-To: <CALBAE1NfNNnoEmWfjrErcQr2+rWUQQae5drWiNt5FG5DzhZcpg@mail.gmail.com>

> From: dev [mailto:dev-bounces@dpdk.org] On Behalf Of Jerin Jacob
> Sent: Friday, April 3, 2020 9:09 PM
> 
> On Fri, Apr 3, 2020 at 10:49 PM Thomas Monjalon <thomas@monjalon.net>
> wrote:
> >
> > 02/04/2020 15:48, Andrzej Ostruszka [C]:
> > > On 3/26/20 6:42 PM, Andrzej Ostruszka wrote:
> > > > On 3/25/20 12:11 PM, Morten Brørup wrote:
> > > [...]
> > > >> And I am still strongly opposed to the callback method:
> > > >
> > > > Noted - however for now I would like to keep them.  I don't have
> much
> > > > experience with this library so if they prove to be inadequate
> then we
> > > > will remove them.  Right now they seem to add some flexibility
> that I like:
> > > > - if something should be changed globally and once (and it is
> safe to do
> > > >   so!) then it can be done from the callback
> > > > - if something can be prepared once and consumed later by lcores
> then it
> > > >   can be done in callback and the callback returns 0 so that
> event is
> > > >   still queued and lcores (under assumption that queues are per
> lcore)
> > > >   pick up what has been prepared.
> > >
> > > Morten
> > >
> > > I've been thinking about this a bit and would like to know your
> (and
> > > others) opinion about following proposed enhancement.
> > >
> > > Right now, how queues are used is left to the application decision
> (per
> > > lcore, per port, ...) - and I intend to keep it that way - but they
> are
> > > "match all".  What I mean by that is that (unlike callbacks where
> you
> > > have separate per event type) queue has no chance to be selective.
> > >
> > > So if someone would like to go with queues only they he would have
> to
> > > coordinate between queues (or their "owners") which one does the
> > > handling of an event that supposedly should be handled only once.
> > >
> > > Let's take this forwarding example - the queues are per lcore and
> each
> > > lcore keeps its own copy of ARP table (hash) so when the change is
> > > noticed the event is queued to all registered queue, each lcore
> updates
> > > its own copy and everything is OK.  However the routing is global
> (and
> > > right now is updated from callback) and if no callback is used for
> that
> > > then the event would be queued to all lcores and application would
> need
> > > to select the one which does the update.
> > >
> > > Would that be easier/better to register queue together with a
> bitmask of
> > > event types that given queue is accepting?  Than during setup phase
> > > application would select just one queue to handle "global" events
> and
> > > the logic of event handling for lcores should be simplier.
> > >
> > > Let me know what you think.
> >
> > I think we want to avoid complicate design.
> > So let's choose between callback and message queue.
> > I vote for message queue because it can handle any situation,
> > and it allows to control the context of the event processing.
> 
> IMO, it should be left to application decision, Application can use
> either callback or
> message queue based on their design and I don't think, DPDK needs to
> enforce certain model.
> On the upside, Giving two options, the application can choose the right
> model.
> The simple use case like updating the global routing table, The
> callback scheme would be more than enough.
> The downside of pushing the architecture to message queue would
> be that application either need to create additional control thread to
> poll or call select()
> get the event or in worst case check the message queue emptiness in
> fastpath.
> So why to enforce?
> 
> Thoughts?

A message queue would not require an additional control thread. It would use the existing control thread that the application already has.

I think you are missing an important point:

The application needs to handle all control plane interactions, not just control plane interactions related to the interface proxy library.

So the application already has (or needs to add) mechanisms in place for this. E.g. if a control plane event (from the interface proxy library or some other trigger) needs to be distributed across a single or multiple data plane lcores, the application already has (or needs to add) a mechanism for doing it. Adding a specific mechanism only in this library does not help all the other control plane interactions the application needs to handle. Actually it does the opposite: it requires that the application handles events from the interface proxy library in a specific way that is different from the way the application already handles other control plane events.

So I'm also voting for simplicity: A single event queue, leaving it up to the application how to handle these events.

> > The other reason is that I believe we need message queueing for
> > other purposes in DPDK (ex: multi-process, telemetry).
> 
> As far as I know, telemetry is using Linux socket fro IPC, I am not
> sure
> why do we need to standardize message queue infra? Becasue, each use
> case is different.

I think Thomas is suggesting that we consider the generic case of interaction with the control plane, as I described above. Not just interaction with the interface proxy events.

> >
> > You start thinking about complex message management.
> > And I start thinking about other usages of message queueing.
> > So I think it is the right time to introduce a generic messaging in
> DPDK.
> > Note: the IPC rte_mp should be built on top of such generic
> messaging.
> >
> > If you agree, we can start a new email thread to better discuss
> > the generic messaging sub-system.

I agree that it should be separated from the interface proxy library.

And yes, DPDK is missing a generic framework - or at least a "best practices" description - for interaction between the control plane and the data plane. So far, every DPDK application developer has to come up with his own.

> > I describe here the 3 properties I have in mind:
> >
> > 1/ Message policy
> > One very important rule in DPDK is to let the control to the
> application.
> > So the messaging policy must be managed by the application via DPDK
> API.
> 
> Do you mean send() and recv() should be wrapped around DPDK call?
> 
> >
> > 2/ Message queue
> > It seems we should rely on ZeroMQ. Here is why:
> > http://zguide.zeromq.org/page:all#Why-We-Needed-ZeroMQ
> 
> IMO, ZeroMQ used for IPC over network etc. In this case, the purpose is
> to pass the Netlink message IN THE SAME SYSTEM to application.
> Do you need external library dependency? On the same system or
> multiprocess application, our rte_ring would be more than enough.
> Right?
> If not, please enumerate the use case.
> 
> >
> > 3/ Message format
> > I am not sure whether we can manage with "simple strings", TLV,
> > or should we use something more complex like protobuf?

Lean and mean is the way to go. A binary format, please. No more JSON or similar bloated encoding!

> 
> In this use case, we are relying the Netlink message to application at
> least
> in Linux case. I think the message should be similar to Netlink message
> and give
> provision for other OS'es such as scheme.
> 
> Why reinvent the wheel?


  reply	other threads:[~2020-04-03 21:18 UTC|newest]

Thread overview: 64+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2020-03-06 16:41 [dpdk-dev] [PATCH " Andrzej Ostruszka
2020-03-06 16:41 ` [dpdk-dev] [PATCH 1/4] lib: introduce IF Proxy library Andrzej Ostruszka
2020-03-31 12:36   ` Harman Kalra
2020-03-31 15:37     ` Andrzej Ostruszka [C]
2020-04-01  5:29   ` Varghese, Vipin
2020-04-01 20:08     ` Andrzej Ostruszka [C]
2020-04-08  3:04       ` Varghese, Vipin
2020-04-08 18:13         ` Andrzej Ostruszka [C]
2020-03-06 16:41 ` [dpdk-dev] [PATCH 2/4] if_proxy: add library documentation Andrzej Ostruszka
2020-03-06 16:41 ` [dpdk-dev] [PATCH 3/4] if_proxy: add simple functionality test Andrzej Ostruszka
2020-03-06 16:41 ` [dpdk-dev] [PATCH 4/4] if_proxy: add example application Andrzej Ostruszka
2020-03-06 17:17 ` [dpdk-dev] [PATCH 0/4] Introduce IF proxy library Andrzej Ostruszka
2020-03-10 11:10 ` [dpdk-dev] [PATCH v2 " Andrzej Ostruszka
2020-03-10 11:10   ` [dpdk-dev] [PATCH v2 1/4] lib: introduce IF Proxy library Andrzej Ostruszka
2020-07-02  0:34     ` Stephen Hemminger
2020-07-07 20:13       ` Andrzej Ostruszka [C]
2020-07-08 16:07         ` Morten Brørup
2020-07-09  8:43           ` Andrzej Ostruszka [C]
2020-07-22  0:40             ` Thomas Monjalon
2020-07-22  8:45               ` Jerin Jacob
2020-07-22  8:56                 ` Thomas Monjalon
2020-07-22  9:09                   ` Jerin Jacob
2020-07-22  9:27                     ` Thomas Monjalon
2020-07-22  9:54                       ` Jerin Jacob
2020-07-23 14:09                         ` [dpdk-dev] [EXT] " Andrzej Ostruszka [C]
2020-03-10 11:10   ` [dpdk-dev] [PATCH v2 2/4] if_proxy: add library documentation Andrzej Ostruszka
2020-03-10 11:10   ` [dpdk-dev] [PATCH v2 3/4] if_proxy: add simple functionality test Andrzej Ostruszka
2020-03-10 11:10   ` [dpdk-dev] [PATCH v2 4/4] if_proxy: add example application Andrzej Ostruszka
2020-03-25  8:08   ` [dpdk-dev] [PATCH v2 0/4] Introduce IF proxy library David Marchand
2020-03-25 11:11     ` Morten Brørup
2020-03-26 17:42       ` Andrzej Ostruszka
2020-04-02 13:48         ` Andrzej Ostruszka [C]
2020-04-03 17:19           ` Thomas Monjalon
2020-04-03 19:09             ` Jerin Jacob
2020-04-03 21:18               ` Morten Brørup [this message]
2020-04-03 21:57                 ` Thomas Monjalon
2020-04-04 10:18                   ` Jerin Jacob
2020-04-10 10:41                     ` Morten Brørup
2020-04-04 18:30             ` Andrzej Ostruszka [C]
2020-04-04 19:58               ` Thomas Monjalon
2020-04-10 10:03                 ` Morten Brørup
2020-04-10 12:28                   ` Jerin Jacob
2020-03-26 12:41     ` Andrzej Ostruszka
2020-03-30 19:23       ` Andrzej Ostruszka
2020-04-03 21:42   ` Thomas Monjalon
2020-04-04 18:07     ` Andrzej Ostruszka [C]
2020-04-04 19:51       ` Thomas Monjalon
2020-04-16 16:11 ` [dpdk-dev] [PATCH " Stephen Hemminger
2020-04-16 16:49   ` Jerin Jacob
2020-04-16 17:04     ` Stephen Hemminger
2020-04-16 17:26       ` Andrzej Ostruszka [C]
2020-04-16 17:27       ` Jerin Jacob
2020-04-16 17:12     ` Andrzej Ostruszka [C]
2020-04-16 17:19       ` Stephen Hemminger
2020-05-04  8:53 ` [dpdk-dev] [PATCH v3 " Andrzej Ostruszka
2020-05-04  8:53   ` [dpdk-dev] [PATCH v3 1/4] lib: introduce IF Proxy library Andrzej Ostruszka
2020-05-04  8:53   ` [dpdk-dev] [PATCH v3 2/4] if_proxy: add library documentation Andrzej Ostruszka
2020-05-04  8:53   ` [dpdk-dev] [PATCH v3 3/4] if_proxy: add simple functionality test Andrzej Ostruszka
2020-05-04  8:53   ` [dpdk-dev] [PATCH v3 4/4] if_proxy: add example application Andrzej Ostruszka
2020-06-22  9:21 ` [dpdk-dev] [PATCH v4 0/4] Introduce IF proxy library Andrzej Ostruszka
2020-06-22  9:21   ` [dpdk-dev] [PATCH v4 1/4] lib: introduce IF Proxy library Andrzej Ostruszka
2020-06-22  9:21   ` [dpdk-dev] [PATCH v4 2/4] if_proxy: add library documentation Andrzej Ostruszka
2020-06-22  9:21   ` [dpdk-dev] [PATCH v4 3/4] if_proxy: add simple functionality test Andrzej Ostruszka
2020-06-22  9:21   ` [dpdk-dev] [PATCH v4 4/4] if_proxy: add example application Andrzej Ostruszka

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=98CBD80474FA8B44BF855DF32C47DC35C60F40@smartserver.smartshare.dk \
    --to=mb@smartsharesystems.com \
    --cc=anatoly.burakov@intel.com \
    --cc=aostruszka@marvell.com \
    --cc=bruce.richardson@intel.com \
    --cc=david.marchand@redhat.com \
    --cc=dev@dpdk.org \
    --cc=jerinjacobk@gmail.com \
    --cc=thomas@monjalon.net \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).