Re: DPDK multiprocessing issue: what if primary process dies

[Stephen Hemminger <stephen@networkplumber.org>]

On Mon, 10 Mar 2025 17:29:09 +0100
Mikhail Malofeev <mdmalofeev@gmail.com> wrote:

> Hello everyone,
> 
> I am new to DPDK and I am implementing a DPDK-based application which might
> be running as a multi-process application. My understanding is that one of
> the processes should be the primary one, responsible for initializing and
> managing the hugepages. All the other processes should be secondary,
> meaning that they are connected to the primary process during DPDK
> initialization to get information about hugepages (please correct me if I
> am mistaken here).

That is one usage model. But secondary processes add additional overhead
and are certainly not required.

> I am interested in the scenario where there are multiple DPDK processes,
> and the primary process dies. As far as I understand, existing secondary
> processes will continue running; however, I will not be able to start new
> DPDK processes:

Once primary process dies, the DPDK application is not in a viable state.
The most common model is to use a service manager (like systemd) to recover
if primary process dies.

The secondary processes should all be monitoring for notification that
the primary dies, and exit then.

> 1) I cannot start a new primary process because it will try to initialize
> hugepages and this will fail since some of them are in use by existing
> secondary processes (source:
> https://stackoverflow.com/questions/74602244/dpdk-multi-process-kill-a-primary-process-and-restart-as-a-secondary-doesnt-wo
> )
> 2) I cannot start a new secondary process because there is no primary
> process to connect to and get the hugepages info, so the secondary process
> won't be able to complete the initialization.
> 
> Hence, if my primary process terminates, what should I do in order to start
> it again?
> 
> I have come up with two possible solutions:
> 1) If the primary process dies, I will have to kill all the secondary
> processes and restart everything. This solution is not ideal because I want
> to minimize the amount of time when processes are not running.
> 2) Have a dummy primary process that does not contain any critical business
> logic and is solely responsible for DPDK initialization. Then, all the
> business-critical applications must be secondary processes. If one of them
> dies, I can simply restart it without interrupting other processes. This
> solution sounds better than the first, but it is still not perfect as it
> requires running an additional dummy process which I would prefer to avoid.
> 
> Could anyone please advise me on the idiomatic way of dealing with these
> multiprocessing issues?
> 

Use systemd, and do full restart.