Re: [dpdk-dev] DPDK and forked processes

["Eads, Gage" <gage.eads@intel.com>]

As this discussion has broad implications for DPDK, is it a good candidate for a techboard meeting topic? 

> -----Original Message-----
> From: Burakov, Anatoly
> Sent: Monday, July 16, 2018 10:09 AM
> To: Eads, Gage <gage.eads@intel.com>; dev@dpdk.org
> Subject: Re: DPDK and forked processes
> 
> On 16-Jul-18 4:00 PM, Eads, Gage wrote:
> > Hi all,
> >
> > Does DPDK support forking secondary processes after executing
> > rte_eal_init()? The l2fwd_fork example and at least one application
> > (OpenEM: https://sourceforge.net/projects/eventmachine/) use this
> > model, and they do so by fixing up the EAL internals (e.g. manually
> > changing process_type from primary to secondary) at the start of the
> > child process. This feels like a hack, and I can’t find any
> > documentation describing this model.
> >
> > Moreover, this approach doesn’t appear to be compatible with recent
> > EAL changes. For instance, the multi-process communication creates a
> > couple handler threads (“rte_mp_handle” and “rte_mp_async”) during EAL
> > initialization. The child processes won’t inherit these threads, and
> > so won’t be able to participate in multi-process comms. This means the
> > reworked memory subsystem and upcoming device hotplug support
> > (http://mails.dpdk.org/archives/dev/2018-July/107704.html) won’t work
> > with this fork-after-init model.
> >
> > This is just one example – there may be other features/subsystems that
> > won’t work. As far as I can tell there is no official stance (though
> > the l2fwd_fork example implies it’s supported, IMO); I think either
> > DPDK should either drop the example and not support this model, or
> > support it and either document its limitations or resolve them. This
> > model could be an interesting way to run multi-process DPDK on an
> > ASLR-enabled system, but supporting this wouldn’t be trivial.
> >
> > Thanks,
> >
> > Gage
> >
> 
> I think it's a very bad idea to use such a model in recent versions of DPDK. As you
> have correctly pointed out, IPC will not work in such a scenario, and given how
> our memory subsystem relies on IPC, this is a recipe for memory corruption and
> divergent memory maps (since technically both initial and forked processes
> believe they are primary).
> 
> Even hacking rte_config to make DPDK think it's a secondary process will not
> work, because the initialization has already completed, but all of the threads
> (IPC, interrupt, etc.) are gone and correct IPC socket was not created, which
> means the process becomes invisible to the primary for all intents and purposes.
> 
> We _could_ introduce some kind of "official DPDK fork" function that would fork
> the process and then restart interrupt, IPC etc. stuff on an already running
> instance of DPDK, but that seems like a workaround for a problem that shouldn't
> exist in the first place, because such usage is fundamentally incompatible with
> DPDK as it stands now.
> 
> --
> Thanks,
> Anatoly