From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from dpdk.org (dpdk.org [92.243.14.124]) by inbox.dpdk.org (Postfix) with ESMTP id 3D778A0562; Fri, 3 Apr 2020 23:57:52 +0200 (CEST) Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id 887071C12B; Fri, 3 Apr 2020 23:57:51 +0200 (CEST) Received: from out1-smtp.messagingengine.com (out1-smtp.messagingengine.com [66.111.4.25]) by dpdk.org (Postfix) with ESMTP id 150C71C125 for ; Fri, 3 Apr 2020 23:57:50 +0200 (CEST) Received: from compute7.internal (compute7.nyi.internal [10.202.2.47]) by mailout.nyi.internal (Postfix) with ESMTP id 7D4385C0167; Fri, 3 Apr 2020 17:57:49 -0400 (EDT) Received: from mailfrontend2 ([10.202.2.163]) by compute7.internal (MEProxy); Fri, 03 Apr 2020 17:57:49 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=monjalon.net; h= from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding:content-type; s=mesmtp; bh=Xkpt+9zb5kDu6kZHcx1fm7Yx6VuubNVK6j3/muFdzwE=; b=QJfLY3PHl9ri FeyS7YumUXAPALyjpzbrxBrRilAEly0iDEjvFUk91J3YH8No8dHpWFhrS6mEvVHh dZ8yy6nNOBtdBUV7+nFC/zWHwWjyVrZUulkWOupAi4lih/pGdcGel1H5iwVGxmUN XJRktz4apeiBw57oUodz2DUTTV3noNM= DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d= messagingengine.com; h=cc:content-transfer-encoding:content-type :date:from:in-reply-to:message-id:mime-version:references :subject:to:x-me-proxy:x-me-proxy:x-me-sender:x-me-sender :x-sasl-enc; s=fm2; bh=Xkpt+9zb5kDu6kZHcx1fm7Yx6VuubNVK6j3/muFdz wE=; b=QiB6mRw84IWStYgdXp21uCIPEIlWhdK0PIGqnj5Sj4RoDIrTdBbDISP6V lIB8c7/Hqoqg/gYGbrkM4VnJzLIiO9LSQYNXiO4QyCLHChGvVPo9CHGesFuWv9vw byAP+uS8fgKrvOMIwrBLIZ3M+Us448G0V/rXWu/7krm7WIi6VY+Y3vOWKKW1aXDo wHyIMMcdVpac3nFF034e6TjPeVrpDZ0ZZRJhT6CLEx/VmhFx0yID4NsrozvbZ1Qp Aw8WxBvyf1cSQBtmnt5WbhTpnq3INrdIqBwMaxJcWxGC09sKt8JJH7Of3MOOoK0v ZB0urpUepeoZEFwaFziVWF9C3EuGA== X-ME-Sender: X-ME-Proxy-Cause: gggruggvucftvghtrhhoucdtuddrgeduhedrtdejgddthecutefuodetggdotefrodftvf curfhrohhfihhlvgemucfhrghsthforghilhdpqfgfvfdpuffrtefokffrpgfnqfghnecu uegrihhlohhuthemuceftddtnecusecvtfgvtghiphhivghnthhsucdlqddutddtmdenuc fjughrpefhvffufffkjghfggfgtgesthhqredttddtudenucfhrhhomhepvfhhohhmrghs ucfoohhnjhgrlhhonhcuoehthhhomhgrshesmhhonhhjrghlohhnrdhnvghtqeenucffoh hmrghinhepiigvrhhomhhqrdhorhhgnecukfhppeejjedrudefgedrvddtfedrudekgeen ucevlhhushhtvghrufhiiigvpedtnecurfgrrhgrmhepmhgrihhlfhhrohhmpehthhhomh grshesmhhonhhjrghlohhnrdhnvght X-ME-Proxy: Received: from xps.localnet (184.203.134.77.rev.sfr.net [77.134.203.184]) by mail.messagingengine.com (Postfix) with ESMTPA id 35A54306CFCA; Fri, 3 Apr 2020 17:57:48 -0400 (EDT) From: Thomas Monjalon To: Jerin Jacob , "Andrzej Ostruszka [C]" , Morten =?ISO-8859-1?Q?Br=F8rup?= Cc: David Marchand , dpdk-dev , "Richardson, Bruce" , Anatoly Burakov Date: Fri, 03 Apr 2020 23:57:46 +0200 Message-ID: <2394105.vtBmWVcJkq@xps> In-Reply-To: <98CBD80474FA8B44BF855DF32C47DC35C60F40@smartserver.smartshare.dk> References: <20200306164104.15528-1-aostruszka@marvell.com> <98CBD80474FA8B44BF855DF32C47DC35C60F40@smartserver.smartshare.dk> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="iso-8859-1" Subject: Re: [dpdk-dev] [PATCH v2 0/4] Introduce IF proxy library X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" 03/04/2020 23:18, Morten Br=F8rup: > From: dev [mailto:dev-bounces@dpdk.org] On Behalf Of Jerin Jacob > > Thomas Monjalon wrote: > > > 02/04/2020 15:48, Andrzej Ostruszka [C]: > > > > On 3/26/20 6:42 PM, Andrzej Ostruszka wrote: > > > > > On 3/25/20 12:11 PM, Morten Br=F8rup wrote: > > > > [...] > > > > >> And I am still strongly opposed to the callback method: > > > > > > > > > > Noted - however for now I would like to keep them. I don't have > > much > > > > > experience with this library so if they prove to be inadequate > > then we > > > > > will remove them. Right now they seem to add some flexibility > > that I like: > > > > > - if something should be changed globally and once (and it is > > safe to do > > > > > so!) then it can be done from the callback > > > > > - if something can be prepared once and consumed later by lcores > > then it > > > > > can be done in callback and the callback returns 0 so that > > event is > > > > > still queued and lcores (under assumption that queues are per > > lcore) > > > > > pick up what has been prepared. > > > > > > > > Morten > > > > > > > > I've been thinking about this a bit and would like to know your > > (and > > > > others) opinion about following proposed enhancement. > > > > > > > > Right now, how queues are used is left to the application decision > > (per > > > > lcore, per port, ...) - and I intend to keep it that way - but they > > are > > > > "match all". What I mean by that is that (unlike callbacks where > > you > > > > have separate per event type) queue has no chance to be selective. > > > > > > > > So if someone would like to go with queues only they he would have > > to > > > > coordinate between queues (or their "owners") which one does the > > > > handling of an event that supposedly should be handled only once. > > > > > > > > Let's take this forwarding example - the queues are per lcore and > > each > > > > lcore keeps its own copy of ARP table (hash) so when the change is > > > > noticed the event is queued to all registered queue, each lcore > > updates > > > > its own copy and everything is OK. However the routing is global > > (and > > > > right now is updated from callback) and if no callback is used for > > that > > > > then the event would be queued to all lcores and application would > > need > > > > to select the one which does the update. > > > > > > > > Would that be easier/better to register queue together with a > > bitmask of > > > > event types that given queue is accepting? Than during setup phase > > > > application would select just one queue to handle "global" events > > and > > > > the logic of event handling for lcores should be simplier. > > > > > > > > Let me know what you think. > > > > > > I think we want to avoid complicate design. > > > So let's choose between callback and message queue. > > > I vote for message queue because it can handle any situation, > > > and it allows to control the context of the event processing. > >=20 > > IMO, it should be left to application decision, Application can use > > either callback or > > message queue based on their design and I don't think, DPDK needs to > > enforce certain model. > > On the upside, Giving two options, the application can choose the right > > model. > > The simple use case like updating the global routing table, The > > callback scheme would be more than enough. > > The downside of pushing the architecture to message queue would > > be that application either need to create additional control thread to > > poll or call select() > > get the event or in worst case check the message queue emptiness in > > fastpath. > > So why to enforce? > >=20 > > Thoughts? >=20 > A message queue would not require an additional control thread. It would = use the existing control thread that the application already has. >=20 > I think you are missing an important point: >=20 > The application needs to handle all control plane interactions, > not just control plane interactions related to the interface proxy librar= y. Yes this is the point. > So the application already has (or needs to add) mechanisms in place for = this. E.g. if a control plane event (from the interface proxy library or so= me other trigger) needs to be distributed across a single or multiple data = plane lcores, the application already has (or needs to add) a mechanism for= doing it. Adding a specific mechanism only in this library does not help a= ll the other control plane interactions the application needs to handle. Ac= tually it does the opposite: it requires that the application handles event= s from the interface proxy library in a specific way that is different from= the way the application already handles other control plane events. >=20 > So I'm also voting for simplicity: A single event queue, leaving it up to= the application how to handle these events. >=20 > > > The other reason is that I believe we need message queueing for > > > other purposes in DPDK (ex: multi-process, telemetry). > >=20 > > As far as I know, telemetry is using Linux socket fro IPC, I am not > > sure > > why do we need to standardize message queue infra? Becasue, each use > > case is different. >=20 > I think Thomas is suggesting that we consider the generic case of > interaction with the control plane, as I described above. > Not just interaction with the interface proxy events. >=20 > > > > > > You start thinking about complex message management. > > > And I start thinking about other usages of message queueing. > > > So I think it is the right time to introduce a generic messaging in > > DPDK. > > > Note: the IPC rte_mp should be built on top of such generic > > messaging. > > > > > > If you agree, we can start a new email thread to better discuss > > > the generic messaging sub-system. >=20 > I agree that it should be separated from the interface proxy library. >=20 > And yes, DPDK is missing a generic framework - or at least a "best practi= ces" description - for interaction between the control plane and the data p= lane. So far, every DPDK application developer has to come up with his own. >=20 > > > I describe here the 3 properties I have in mind: > > > > > > 1/ Message policy > > > One very important rule in DPDK is to let the control to the > > application. > > > So the messaging policy must be managed by the application via DPDK > > API. > >=20 > > Do you mean send() and recv() should be wrapped around DPDK call? I am thinking about something a bit more complex with handlers registration and default handlers in each DPDK library. > > > 2/ Message queue > > > It seems we should rely on ZeroMQ. Here is why: > > > http://zguide.zeromq.org/page:all#Why-We-Needed-ZeroMQ > >=20 > > IMO, ZeroMQ used for IPC over network etc. In this case, the purpose is > > to pass the Netlink message IN THE SAME SYSTEM to application. > > Do you need external library dependency? On the same system or > > multiprocess application, our rte_ring would be more than enough. > > Right? > > If not, please enumerate the use case. Network communication will allow standardizing a DPDK remote control. With ZeroMQ, it comes for free. > > > 3/ Message format > > > I am not sure whether we can manage with "simple strings", TLV, > > > or should we use something more complex like protobuf? >=20 > Lean and mean is the way to go. A binary format, please. > No more JSON or similar bloated encoding! JSON, as other text encoding as one advantage: it is readable when debugging. But I tend to agree that TLV is probably a good fit. > > In this use case, we are relying the Netlink message to application at > > least > > in Linux case. I think the message should be similar to Netlink message > > and give > > provision for other OS'es such as scheme. > >=20 > > Why reinvent the wheel? I agree, we should not re-encode Netlink. With a TLV format, we can just encapsulate Netlink for the generic channel, and give it a message type to dispatch the message to the right hansler.