From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id F41B7A04FD for ; Fri, 8 Apr 2022 23:08:25 +0200 (CEST) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id CA57340041; Fri, 8 Apr 2022 23:08:25 +0200 (CEST) Received: from mail-lf1-f49.google.com (mail-lf1-f49.google.com [209.85.167.49]) by mails.dpdk.org (Postfix) with ESMTP id AD6824003F for ; Fri, 8 Apr 2022 23:08:24 +0200 (CEST) Received: by mail-lf1-f49.google.com with SMTP id t25so17053491lfg.7 for ; Fri, 08 Apr 2022 14:08:24 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :cc; bh=FwAFHNQd9Ke5aqGmM2Z3h5Yo/rSJDtVBrI1q/wmSLb0=; b=akhVqv4vTGl3ug7Ry/fZUPSE0q4mlgVo2D3RQLTbXPBxXIOApK2Dg1dlIaofM2UYRq t7766JGrSJmkf1tIow3aib+Y5SZEKOPwQMbIp9uqGsdqTkHur9cIWmlXpooQ8SHoEz4X hoXsIm+G382Yt5skTg2DatDDEy2ILcjfUvUPAAXir0vl6ILLI61olly26aUKUsynySOY 6ZDi8vG59vd2aoL0MwTxE1u8uVfQ127SYzo7rfx9rn+3p2TpCbWjYQJkK2RGRB7TdkNd JUh/MsRO//7EdDY+O1oMPLlPdGNEpeeWzftGj6hSKUVxlxczNEd/vgfvb1mE7iZNvqI2 JeMA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=FwAFHNQd9Ke5aqGmM2Z3h5Yo/rSJDtVBrI1q/wmSLb0=; b=STQtuVUjZFhxdrdos1fO5Hix1IAIFTXuKzmP8zkeMlqUdBbUzNcp+UBpw4V+kRyctQ PJfgOHHShmJIy1SwxExMZTFIttG6O46MKrI/CPst28uAkcr+e8F+EP2HCgdTWrJL2nfV gIgEsCZ1O7GQoAEq+TngV9xqccrnbLTtpEtVgcBV6NhtlR7OMkuLc7BntRKCRa1CN5oy GLrnzBxff46+buA8G6kkWWAgs7KSqLnUQg86v3GQH6n5qcaO+JA2VeH/Vh6qrdyGS7rL m1qvh7FqvJme6GAZRZOP2jcXBFcmZccwxLqmKyABakpYMtrPk5xPEah7euzNbchiJkfF Ging== X-Gm-Message-State: AOAM532wNB8WajpaQWNlr7T/Py9BBTT8RdZa0OtVhJ+wYsUyIuonq7Ft Ir+SfZcEAuWr/O0H96tFy9E8iUu6yQgjm9E4k9k= X-Google-Smtp-Source: ABdhPJzJOJn4lIQ090hDwcvcyGzrrKi/NTpek1cLa0ord5arWveM76PptbS0UfxICRb2xBkswqKOApKkubrynye62sU= X-Received: by 2002:a19:6d0b:0:b0:448:5b24:c198 with SMTP id i11-20020a196d0b000000b004485b24c198mr13606665lfc.132.1649452103951; Fri, 08 Apr 2022 14:08:23 -0700 (PDT) MIME-Version: 1.0 References: <20220408162629.372dfd0d@sovereign> In-Reply-To: <20220408162629.372dfd0d@sovereign> From: Antonio Di Bacco Date: Fri, 8 Apr 2022 23:08:12 +0200 Message-ID: Subject: Re: Shared memory between two primary DPDK processes To: Dmitry Kozlyuk Cc: users@dpdk.org Content-Type: multipart/alternative; boundary="000000000000bae87405dc2afe8f" X-BeenThere: users@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK usage discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: users-bounces@dpdk.org --000000000000bae87405dc2afe8f Content-Type: text/plain; charset="UTF-8" Il giorno ven 8 apr 2022 alle ore 15:26 Dmitry Kozlyuk < dmitry.kozliuk@gmail.com> ha scritto: > 2022-04-08 14:31 (UTC+0200), Antonio Di Bacco: > > I know that it is possible to share memory between a primary and > secondary > > process using rte_memzone_reserve_aligned to allocate memory in primary > > that is "seen" also by the secondary. If we have two primary processes > > (started with different file-prefix) the same approach is not feasible. I > > wonder how to share a chunk of memory hosted on a hugepage between two > > primaries. > > > > Regards. > > Hi Antonio, > > Correction: all hugepages allocated by DPDK are shared > between primary and secondary processes, not only memzones. > > I assume we're talking about processes within one host, > because your previous similar question was about sharing memory between > hosts > (as we have discussed offline), which is out of scope for DPDK. > > As for the question directly, you need to map the same part of the same > file > in the second primary as the hugepage is mapped from in the first primary. > I don't recommend to work with file paths, because their management > is not straightforward (--single-file-segments, for one) and is > undocumented. > > There is a way to share DPDK memory segment file descriptors. > Although public, this DPDK API is dangerous in the sense that you must > clearly understand what you're doing and how DPDK works. > Hence the question: what is the task you need this sharing for? > Maybe there is a simpler way. > > 1. In the first primary: > > mz = rte_memzone_reserve() > ms = rte_mem_virt2memseg(mz->addr) > fd = rte_memseg_get_fd(ms) > offset = rte_memseg_get_fd_offset(ms) > > 2. Use Unix domain sockets with SCM_RIGHTS > to send "fd" and "offset" to the second primary. > > 3. In the second primary, after receiving "fd" and "offset": > > flags = MAP_SHARED | MAP_HUGE | (30 << MAP_HUGE_SHIFT) > addr = mmap(fd, offset, flags) > > Note that "mz" may consist of multiple "ms" depending on the sizes > of the zone and hugepages, and on the zone alignment. > Also "addr" may (and probably will) differ from "mz->addr". > It is possible to pass "mz->addr" and try to force it, > like DPDK does for primary/secondary. > Thank you Dmitry, it is really incredible how deep your knowledge is. I will give it a try. --000000000000bae87405dc2afe8f Content-Type: text/html; charset="UTF-8" Content-Transfer-Encoding: quoted-printable


=
Il giorno ven 8 apr 2022 alle ore 15:= 26 Dmitry Kozlyuk <dmitry.ko= zliuk@gmail.com> ha scritto:
2022-04-08 14:31 (UTC+0200), Antonio Di Bacco:
> I know that it is possible to share memory between a primary and secon= dary
> process using rte_memzone_reserve_aligned to allocate memory in primar= y
> that is "seen" also by the secondary. If we have two primary= processes
> (started with different file-prefix) the same approach is not feasible= . I
> wonder how to share a chunk of memory hosted on a hugepage between two=
> primaries.
>
> Regards.

Hi Antonio,

Correction: all hugepages allocated by DPDK are shared
between primary and secondary processes, not only memzones.

I assume we're talking about processes within one host,
because your previous similar question was about sharing memory between hos= ts
(as we have discussed offline), which is out of scope for DPDK.

As for the question directly, you need to map the same part of the same fil= e
in the second primary as the hugepage is mapped from in the first primary.<= br> I don't recommend to work with file paths, because their management
is not straightforward (--single-file-segments, for one) and is undocumente= d.

There is a way to share DPDK memory segment file descriptors.
Although public, this DPDK API is dangerous in the sense that you must
clearly understand what you're doing and how DPDK works.
Hence the question: what is the task you need this sharing for?
Maybe there is a simpler way.

1. In the first primary:

=C2=A0 =C2=A0 =C2=A0 =C2=A0 mz =3D rte_memzone_reserve()
=C2=A0 =C2=A0 =C2=A0 =C2=A0 ms =3D rte_mem_virt2memseg(mz->addr)
=C2=A0 =C2=A0 =C2=A0 =C2=A0 fd =3D rte_memseg_get_fd(ms)
=C2=A0 =C2=A0 =C2=A0 =C2=A0 offset =3D rte_memseg_get_fd_offset(ms)

2. Use Unix domain sockets with SCM_RIGHTS
=C2=A0 =C2=A0to send "fd" and "offset" to the second pr= imary.

3. In the second primary, after receiving "fd" and "offset&q= uot;:

=C2=A0 =C2=A0 =C2=A0 =C2=A0 flags =3D MAP_SHARED | MAP_HUGE | (30 << = MAP_HUGE_SHIFT)
=C2=A0 =C2=A0 =C2=A0 =C2=A0 addr =3D mmap(fd, offset, flags)

Note that "mz" may consist of multiple "ms" depending o= n the sizes
of the zone and hugepages, and on the zone alignment.
Also "addr" may (and probably will) differ from "mz->addr= ".
It is possible to pass "mz->addr" and try to force it,
like DPDK does for primary/secondary.

<= br>
Thank you Dmitry, it is really incredible how deep your knowl= edge is. I will give it a try.=C2=A0
--000000000000bae87405dc2afe8f--