DPDK patches and discussions
 help / color / mirror / Atom feed
From: Thomas Monjalon <thomas@monjalon.net>
To: Jerin Jacob <jerinjacobk@gmail.com>,
	"Andrzej Ostruszka [C]" <aostruszka@marvell.com>,
	Morten Brørup <mb@smartsharesystems.com>
Cc: David Marchand <david.marchand@redhat.com>,
	dpdk-dev <dev@dpdk.org>,
	"Richardson, Bruce" <bruce.richardson@intel.com>,
	Anatoly Burakov <anatoly.burakov@intel.com>
Subject: Re: [dpdk-dev] [PATCH v2 0/4] Introduce IF proxy library
Date: Fri, 03 Apr 2020 23:57:46 +0200
Message-ID: <2394105.vtBmWVcJkq@xps> (raw)
In-Reply-To: <98CBD80474FA8B44BF855DF32C47DC35C60F40@smartserver.smartshare.dk>

03/04/2020 23:18, Morten Brørup:
> From: dev [mailto:dev-bounces@dpdk.org] On Behalf Of Jerin Jacob
> > Thomas Monjalon <thomas@monjalon.net> wrote:
> > > 02/04/2020 15:48, Andrzej Ostruszka [C]:
> > > > On 3/26/20 6:42 PM, Andrzej Ostruszka wrote:
> > > > > On 3/25/20 12:11 PM, Morten Brørup wrote:
> > > > [...]
> > > > >> And I am still strongly opposed to the callback method:
> > > > >
> > > > > Noted - however for now I would like to keep them.  I don't have
> > much
> > > > > experience with this library so if they prove to be inadequate
> > then we
> > > > > will remove them.  Right now they seem to add some flexibility
> > that I like:
> > > > > - if something should be changed globally and once (and it is
> > safe to do
> > > > >   so!) then it can be done from the callback
> > > > > - if something can be prepared once and consumed later by lcores
> > then it
> > > > >   can be done in callback and the callback returns 0 so that
> > event is
> > > > >   still queued and lcores (under assumption that queues are per
> > lcore)
> > > > >   pick up what has been prepared.
> > > >
> > > > Morten
> > > >
> > > > I've been thinking about this a bit and would like to know your
> > (and
> > > > others) opinion about following proposed enhancement.
> > > >
> > > > Right now, how queues are used is left to the application decision
> > (per
> > > > lcore, per port, ...) - and I intend to keep it that way - but they
> > are
> > > > "match all".  What I mean by that is that (unlike callbacks where
> > you
> > > > have separate per event type) queue has no chance to be selective.
> > > >
> > > > So if someone would like to go with queues only they he would have
> > to
> > > > coordinate between queues (or their "owners") which one does the
> > > > handling of an event that supposedly should be handled only once.
> > > >
> > > > Let's take this forwarding example - the queues are per lcore and
> > each
> > > > lcore keeps its own copy of ARP table (hash) so when the change is
> > > > noticed the event is queued to all registered queue, each lcore
> > updates
> > > > its own copy and everything is OK.  However the routing is global
> > (and
> > > > right now is updated from callback) and if no callback is used for
> > that
> > > > then the event would be queued to all lcores and application would
> > need
> > > > to select the one which does the update.
> > > >
> > > > Would that be easier/better to register queue together with a
> > bitmask of
> > > > event types that given queue is accepting?  Than during setup phase
> > > > application would select just one queue to handle "global" events
> > and
> > > > the logic of event handling for lcores should be simplier.
> > > >
> > > > Let me know what you think.
> > >
> > > I think we want to avoid complicate design.
> > > So let's choose between callback and message queue.
> > > I vote for message queue because it can handle any situation,
> > > and it allows to control the context of the event processing.
> > 
> > IMO, it should be left to application decision, Application can use
> > either callback or
> > message queue based on their design and I don't think, DPDK needs to
> > enforce certain model.
> > On the upside, Giving two options, the application can choose the right
> > model.
> > The simple use case like updating the global routing table, The
> > callback scheme would be more than enough.
> > The downside of pushing the architecture to message queue would
> > be that application either need to create additional control thread to
> > poll or call select()
> > get the event or in worst case check the message queue emptiness in
> > fastpath.
> > So why to enforce?
> > 
> > Thoughts?
> 
> A message queue would not require an additional control thread. It would use the existing control thread that the application already has.
> 
> I think you are missing an important point:
> 
> The application needs to handle all control plane interactions,
> not just control plane interactions related to the interface proxy library.

Yes this is the point.

> So the application already has (or needs to add) mechanisms in place for this. E.g. if a control plane event (from the interface proxy library or some other trigger) needs to be distributed across a single or multiple data plane lcores, the application already has (or needs to add) a mechanism for doing it. Adding a specific mechanism only in this library does not help all the other control plane interactions the application needs to handle. Actually it does the opposite: it requires that the application handles events from the interface proxy library in a specific way that is different from the way the application already handles other control plane events.
> 
> So I'm also voting for simplicity: A single event queue, leaving it up to the application how to handle these events.
> 
> > > The other reason is that I believe we need message queueing for
> > > other purposes in DPDK (ex: multi-process, telemetry).
> > 
> > As far as I know, telemetry is using Linux socket fro IPC, I am not
> > sure
> > why do we need to standardize message queue infra? Becasue, each use
> > case is different.
> 
> I think Thomas is suggesting that we consider the generic case of
> interaction with the control plane, as I described above.
> Not just interaction with the interface proxy events.
> 
> > >
> > > You start thinking about complex message management.
> > > And I start thinking about other usages of message queueing.
> > > So I think it is the right time to introduce a generic messaging in
> > DPDK.
> > > Note: the IPC rte_mp should be built on top of such generic
> > messaging.
> > >
> > > If you agree, we can start a new email thread to better discuss
> > > the generic messaging sub-system.
> 
> I agree that it should be separated from the interface proxy library.
> 
> And yes, DPDK is missing a generic framework - or at least a "best practices" description - for interaction between the control plane and the data plane. So far, every DPDK application developer has to come up with his own.
> 
> > > I describe here the 3 properties I have in mind:
> > >
> > > 1/ Message policy
> > > One very important rule in DPDK is to let the control to the
> > application.
> > > So the messaging policy must be managed by the application via DPDK
> > API.
> > 
> > Do you mean send() and recv() should be wrapped around DPDK call?

I am thinking about something a bit more complex with handlers
registration and default handlers in each DPDK library.


> > > 2/ Message queue
> > > It seems we should rely on ZeroMQ. Here is why:
> > > http://zguide.zeromq.org/page:all#Why-We-Needed-ZeroMQ
> > 
> > IMO, ZeroMQ used for IPC over network etc. In this case, the purpose is
> > to pass the Netlink message IN THE SAME SYSTEM to application.
> > Do you need external library dependency? On the same system or
> > multiprocess application, our rte_ring would be more than enough.
> > Right?
> > If not, please enumerate the use case.

Network communication will allow standardizing a DPDK remote control.
With ZeroMQ, it comes for free.


> > > 3/ Message format
> > > I am not sure whether we can manage with "simple strings", TLV,
> > > or should we use something more complex like protobuf?
> 
> Lean and mean is the way to go. A binary format, please.
> No more JSON or similar bloated encoding!

JSON, as other text encoding as one advantage: it is readable
when debugging.
But I tend to agree that TLV is probably a good fit.

> > In this use case, we are relying the Netlink message to application at
> > least
> > in Linux case. I think the message should be similar to Netlink message
> > and give
> > provision for other OS'es such as scheme.
> > 
> > Why reinvent the wheel?

I agree, we should not re-encode Netlink.
With a TLV format, we can just encapsulate Netlink for the generic
channel, and give it a message type to dispatch the message to the
right hansler.




  reply	other threads:[~2020-04-03 21:57 UTC|newest]

Thread overview: 64+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2020-03-06 16:41 [dpdk-dev] [PATCH " Andrzej Ostruszka
2020-03-06 16:41 ` [dpdk-dev] [PATCH 1/4] lib: introduce IF Proxy library Andrzej Ostruszka
2020-03-31 12:36   ` Harman Kalra
2020-03-31 15:37     ` Andrzej Ostruszka [C]
2020-04-01  5:29   ` Varghese, Vipin
2020-04-01 20:08     ` Andrzej Ostruszka [C]
2020-04-08  3:04       ` Varghese, Vipin
2020-04-08 18:13         ` Andrzej Ostruszka [C]
2020-03-06 16:41 ` [dpdk-dev] [PATCH 2/4] if_proxy: add library documentation Andrzej Ostruszka
2020-03-06 16:41 ` [dpdk-dev] [PATCH 3/4] if_proxy: add simple functionality test Andrzej Ostruszka
2020-03-06 16:41 ` [dpdk-dev] [PATCH 4/4] if_proxy: add example application Andrzej Ostruszka
2020-03-06 17:17 ` [dpdk-dev] [PATCH 0/4] Introduce IF proxy library Andrzej Ostruszka
2020-03-10 11:10 ` [dpdk-dev] [PATCH v2 " Andrzej Ostruszka
2020-03-10 11:10   ` [dpdk-dev] [PATCH v2 1/4] lib: introduce IF Proxy library Andrzej Ostruszka
2020-07-02  0:34     ` Stephen Hemminger
2020-07-07 20:13       ` Andrzej Ostruszka [C]
2020-07-08 16:07         ` Morten Brørup
2020-07-09  8:43           ` Andrzej Ostruszka [C]
2020-07-22  0:40             ` Thomas Monjalon
2020-07-22  8:45               ` Jerin Jacob
2020-07-22  8:56                 ` Thomas Monjalon
2020-07-22  9:09                   ` Jerin Jacob
2020-07-22  9:27                     ` Thomas Monjalon
2020-07-22  9:54                       ` Jerin Jacob
2020-07-23 14:09                         ` [dpdk-dev] [EXT] " Andrzej Ostruszka [C]
2020-03-10 11:10   ` [dpdk-dev] [PATCH v2 2/4] if_proxy: add library documentation Andrzej Ostruszka
2020-03-10 11:10   ` [dpdk-dev] [PATCH v2 3/4] if_proxy: add simple functionality test Andrzej Ostruszka
2020-03-10 11:10   ` [dpdk-dev] [PATCH v2 4/4] if_proxy: add example application Andrzej Ostruszka
2020-03-25  8:08   ` [dpdk-dev] [PATCH v2 0/4] Introduce IF proxy library David Marchand
2020-03-25 11:11     ` Morten Brørup
2020-03-26 17:42       ` Andrzej Ostruszka
2020-04-02 13:48         ` Andrzej Ostruszka [C]
2020-04-03 17:19           ` Thomas Monjalon
2020-04-03 19:09             ` Jerin Jacob
2020-04-03 21:18               ` Morten Brørup
2020-04-03 21:57                 ` Thomas Monjalon [this message]
2020-04-04 10:18                   ` Jerin Jacob
2020-04-10 10:41                     ` Morten Brørup
2020-04-04 18:30             ` Andrzej Ostruszka [C]
2020-04-04 19:58               ` Thomas Monjalon
2020-04-10 10:03                 ` Morten Brørup
2020-04-10 12:28                   ` Jerin Jacob
2020-03-26 12:41     ` Andrzej Ostruszka
2020-03-30 19:23       ` Andrzej Ostruszka
2020-04-03 21:42   ` Thomas Monjalon
2020-04-04 18:07     ` Andrzej Ostruszka [C]
2020-04-04 19:51       ` Thomas Monjalon
2020-04-16 16:11 ` [dpdk-dev] [PATCH " Stephen Hemminger
2020-04-16 16:49   ` Jerin Jacob
2020-04-16 17:04     ` Stephen Hemminger
2020-04-16 17:26       ` Andrzej Ostruszka [C]
2020-04-16 17:27       ` Jerin Jacob
2020-04-16 17:12     ` Andrzej Ostruszka [C]
2020-04-16 17:19       ` Stephen Hemminger
2020-05-04  8:53 ` [dpdk-dev] [PATCH v3 " Andrzej Ostruszka
2020-05-04  8:53   ` [dpdk-dev] [PATCH v3 1/4] lib: introduce IF Proxy library Andrzej Ostruszka
2020-05-04  8:53   ` [dpdk-dev] [PATCH v3 2/4] if_proxy: add library documentation Andrzej Ostruszka
2020-05-04  8:53   ` [dpdk-dev] [PATCH v3 3/4] if_proxy: add simple functionality test Andrzej Ostruszka
2020-05-04  8:53   ` [dpdk-dev] [PATCH v3 4/4] if_proxy: add example application Andrzej Ostruszka
2020-06-22  9:21 ` [dpdk-dev] [PATCH v4 0/4] Introduce IF proxy library Andrzej Ostruszka
2020-06-22  9:21   ` [dpdk-dev] [PATCH v4 1/4] lib: introduce IF Proxy library Andrzej Ostruszka
2020-06-22  9:21   ` [dpdk-dev] [PATCH v4 2/4] if_proxy: add library documentation Andrzej Ostruszka
2020-06-22  9:21   ` [dpdk-dev] [PATCH v4 3/4] if_proxy: add simple functionality test Andrzej Ostruszka
2020-06-22  9:21   ` [dpdk-dev] [PATCH v4 4/4] if_proxy: add example application Andrzej Ostruszka

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=2394105.vtBmWVcJkq@xps \
    --to=thomas@monjalon.net \
    --cc=anatoly.burakov@intel.com \
    --cc=aostruszka@marvell.com \
    --cc=bruce.richardson@intel.com \
    --cc=david.marchand@redhat.com \
    --cc=dev@dpdk.org \
    --cc=jerinjacobk@gmail.com \
    --cc=mb@smartsharesystems.com \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link

DPDK patches and discussions

This inbox may be cloned and mirrored by anyone:

	git clone --mirror https://inbox.dpdk.org/dev/0 dev/git/0.git

	# If you have public-inbox 1.1+ installed, you may
	# initialize and index your mirror using the following commands:
	public-inbox-init -V2 dev dev/ https://inbox.dpdk.org/dev \
		dev@dpdk.org
	public-inbox-index dev

Example config snippet for mirrors.
Newsgroup available over NNTP:
	nntp://inbox.dpdk.org/inbox.dpdk.dev


AGPL code for this site: git clone https://public-inbox.org/public-inbox.git