Re: [dpdk-dev] [RFC PATCH 0/3] introduce IF proxy library

["Morten Brørup" <mb@smartsharesystems.com>]

> -----Original Message-----
> From: dev [mailto:dev-bounces@dpdk.org] On Behalf Of Jerin Jacob
> Sent: Thursday, January 16, 2020 8:15 AM
> 
> On Wed, Jan 15, 2020 at 11:45 PM Morten Brørup
> <mb@smartsharesystems.com> wrote:
> 
> > > > > IMO, in order to have effective use of data plane, the control
> > > > > plane has to be integrated together in an OS-independent way.
> > > > >
> > > >
> > > > Also remember that not all DPDK applications need an IP stack
> resembling
> > > what Linux has. E.g. the SmartShare StraightShaper is a transparent
> > > bandwidth optimization appliance, and it doesn't perform any
> routing, it
> > > doesn't use any O/S-like features in the data path, and thus it
> doesn't
> > > need to integrate with the IP stack in the O/S. (The management
> interface
> > > uses the Linux IP stack, but it is completely isolated from the
> DPDK
> > > application's data plane.) The same can be said about e.g. T-Rex.
> > > >
> > > > Obviously, not all DPDK applications use all DPDK libraries, and
> since
> > > I'm not obligated to use it, I'm not strongly opposed against it. I
> only
> > > question its usefulness outside of the specific use case of
> replacing the
> > > fast path in the Linux kernel.
> > >
> > > Of course, We still follow the "À la carte" model, where we are not
> > > forcing to use the library in the end-user application. You can
> always
> > > use whatever control path that makes sense with the end-user
> > > applications.
> > > But if some application wants to write control plane SW that needs
> to
> > > work Linux/FreeBSD/Windows or other RTOS then it can be used (Again
> if
> > > someone wishes to do so).
> > > We can also provide the means for NOPing out the callbacks or
> override
> > > with something it is the specific end-user library as well, so that
> > > complete flexibly will be still with the application wrt the usage.
> > >
> >
> > OK, you convinced me that a general API for interfacing to the O/S
> control plane might be useful. So let me switch from arguing against it
> to providing some constructive feedback:
> 
> Good news :-)
> 
> >
> > You should consider that most DPDK APIs are not thread safe, meaning
> that their internal structures cannot be manipulated/reconfigured by a
> control plane thread while data plane threads are accessing them. E.g.
> a route cannot be added in the DPDK route library while it is also
> being used by for lookups by a DPDK data plane thread. The same goes
> for the hash table library. This means that callbacks are probably not
> the right design pattern.
> 
> I think, we can have only two design patterns for this case.
> 
> 1) push model(i.e callback). In this case, DP gets the callback, if it
> is not the correct time to apply the configuration then DP can store
> it in its own queue and pull it latter.
> 2) pull model. In this case, the library stores the events. When DP
> needs the events, it can pull the events from the library.
> 
> Do you have any other model in mind? and what is your preference among
> two?
> 

This library interfaces to the O/S on the one side, and a DPDK application on the other side.

Looking at the interface towards the DPDK application, I would personally prefer a pull model. It will allow the DPDK application to handle the events when it is convenient and safe for the DPDK application to manipulate its non-thread safe data structures.

Looking at the interface towards the O/S, Linux Netlink is well defined and described in RFC 3549, and message queues (e.g. DPDK rings) seem like a perfect match for this.

I don't know enough about the Windows network stack to tell if the same applies here, so you should look into this before proceeding. On the other hand, the "memif" Memory Interface PMD is Linux only; so you could also consider limiting your library support to operating systems often being used as routers, i.e. Linux and BSD, and explicitly omit Windows support.

I have no preferences about the message format, but since Linux Netlink is described in an RFC, you could consider using this exact message format or a closely related message format. The RFC authors probably thought this through very thoroughly.

> >
> > AFAIK, the DPDK documentation doesn't mention any "best practices"
> for interaction between the control plane and data plans threads, so I
> understand why you chose a design pattern similar to the NIC Link
> Status Change interrupt design pattern.
> >
> > Furthermore, I have now skimmed the other parts of your patch set. If
> I got it right, it looks like there's a limit of 64 callbacks; this
> will probably not suffice in the long run.
> 
> Yes. We will increase it.
> 
> > And on the administrative side, I assume one of you guys will
> volunteer as the maintainer of this library?
> 
> Yes

Great.

-Morten