RE: [PATCH] common/mlx5: adjust fork call with the new kernel API

[Slava Ovsiienko <viacheslavo@nvidia.com>]

> -----Original Message-----
> From: Stephen Hemminger <stephen@networkplumber.org>
> Sent: Wednesday, May 24, 2023 5:50 PM
> To: Erez Ferber <erezf@nvidia.com>
> Cc: dev@dpdk.org; Slava Ovsiienko <viacheslavo@nvidia.com>; Matan Azrad
> <matan@nvidia.com>; Raslan Darawsheh <rasland@nvidia.com>;
> stable@dpdk.org
> Subject: Re: [PATCH] common/mlx5: adjust fork call with the new kernel API
> 
> On Wed, 24 May 2023 15:01:40 +0300
> <erezf@nvidia.com> wrote:
> 
> > From: Erez Ferber <erezf@nvidia.com>
> >
> > While doing process fork() the operating system remaps all the parent
> > process's memory to the address space of the child process and
> > activates the Copy-on-Write mechanics - it duplicates physical pages
> > once memory writing happens in the child process. Sometimes memory
> > duplication is not allowed - for example, if the page contains
> > hardware queue descriptors. To handle similar issues the rdma-core
> > library should be prepared for forking.
> >
> > The ibv_fork_init() prepares the library to track all the related
> > memory and prevent it from forking using madvise() system API. This
> > approach allows fork, but not all the memory is forked to the child
> > process and, application should care not to touch pages where the
> > parent application allocated the rdma-core objects.
> >
> > The newer kernels propose an option of copy-on-fork for DMA pages and
> > tracking all the memory and disabling it for the forking is no longer
> > needed. The new API routine ibv_is_fork_initialized() should be
> > involved to decide if library initialization for forking is required.
> >
> > Fixes: 0e83b8e536 ("net/mlx5: move rdma-core calls to separate file")
> > Cc: stable@dpdk.org
> > Signed-off-by: Erez Ferber <erezf@nvidia.com>
> 
Hi,

> I don't think DPDK applications should fork(), and lots other parts of the
> shared huge pages will break if an application does this.

I agree - application should not, we have the secondary/primary processes approach.
Nonetheless, we have the real use case - DPDK application does fork() and works well.
Without mlx5 PMD 😊. With mlx5 it ran into some troubles. Now we have the solution.

With best regards,
Slava