From mboxrd@z Thu Jan 1 00:00:00 1970
Return-Path: <users-bounces@dpdk.org>
Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124])
by inbox.dpdk.org (Postfix) with ESMTP id 273C5A0540
for <public@inbox.dpdk.org>; Thu, 7 Jul 2022 00:14:31 +0200 (CEST)
Received: from [217.70.189.124] (localhost [127.0.0.1])
by mails.dpdk.org (Postfix) with ESMTP id AB8EE4069D;
Thu, 7 Jul 2022 00:14:30 +0200 (CEST)
Received: from mail-oa1-f43.google.com (mail-oa1-f43.google.com
[209.85.160.43]) by mails.dpdk.org (Postfix) with ESMTP id 3E98940691
for <users@dpdk.org>; Thu, 7 Jul 2022 00:14:30 +0200 (CEST)
Received: by mail-oa1-f43.google.com with SMTP id
586e51a60fabf-10c0430e27dso11582812fac.4
for <users@dpdk.org>; Wed, 06 Jul 2022 15:14:30 -0700 (PDT)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112;
h=mime-version:references:in-reply-to:from:date:message-id:subject:to
:cc; bh=J/vbeBfrCbCrKoIscUA2/rvGP1PRvWXtKdA9Rs+dLzE=;
b=HpG3epFAfXPKsChVYQqFcSaqzQNs9R9M/Y+UZkjmtzMBgh+HoW/0KGy23DQjCPOu/c
TvXPsyNE9OP/VzxlpxJXZCVEZnT1XlrOT0XnCsFgisw8FElBn90dQIRdDpjCnCzb7lJP
Ya39bb6xYMwElc7HtU7l3mlknuj55anhXeAgatqVuXRTigvufA6McgcTvneWcNfUBCXY
+nzHS7zxhZBMCEF8n6dpL+gZ2zUzHzLWuCbXZmRzcGKLRYde8+5qGR6onhhnQa5eTixT
TVFT+/T5yfeGn06NUbHcWSs0bXhlfwpjRI7zwS3l/MMZDTnv48zwV7RUSWgvdLLrzzar
AbyQ==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
d=1e100.net; s=20210112;
h=x-gm-message-state:mime-version:references:in-reply-to:from:date
:message-id:subject:to:cc;
bh=J/vbeBfrCbCrKoIscUA2/rvGP1PRvWXtKdA9Rs+dLzE=;
b=y7ywKRq0hjLb20Ds+QiV+gsi7QXZb+C+ZgeQxnr3TsgipmkuaniKG04Cg9cGt1yIx0
vs3QrC4BXOZp8HYB3FpQf+9hVCON9u1sXVmCTPRPT6BvSXxJDYmSiu8A5R3/sn1TY+SC
JxvuWsF1E4pmH+egTtVC01nDlGIxvZm6BqtI0x6ZUlPOmUMQM2qXt0TWdobm5jmODxER
nltAruRf5IaUDaiIJyt/VpO3ClMvSawUnZf3/5WuekSHhA13V5YNUjvb4Ve+WTVLO2mG
unu+Ivc1YP3xVc6yvg5iaxgoeP9s6R8rdlig7BhVgwwZ3GFsIkxM9496wUGhPohnngcX
1Aug==
X-Gm-Message-State: AJIora8L5jPjW6GKUZl5BjoyPmeqyYjjz9DjuLogFbEPr89jQFRjHhDi
8DW+0o1H3CJwIjjWjIM81XlslUKx4p8Mp+j2uDM=
X-Google-Smtp-Source: AGRyM1uG5SfYDG7s57w02OO6ln2e8LJLdhXeWu380GAiK7YJrmk3P7uAu4NY4DaPVeIxG/zkdyNTIPDbyanZVedv30Q=
X-Received: by 2002:a05:6870:f203:b0:108:4a52:da42 with SMTP id
t3-20020a056870f20300b001084a52da42mr584246oao.59.1657145669456; Wed, 06 Jul
2022 15:14:29 -0700 (PDT)
MIME-Version: 1.0
References: <CAO8pfFmiSYf=z5kK4EBwJkyJEpSCUrCqZspbNb3dC8nEPipUBw@mail.gmail.com>
<20220408162629.372dfd0d@sovereign>
In-Reply-To: <20220408162629.372dfd0d@sovereign>
From: Antonio Di Bacco <a.dibacco.ks@gmail.com>
Date: Thu, 7 Jul 2022 00:14:18 +0200
Message-ID: <CAO8pfFmnzJmYUd7i0tvZmCB6xjeNbWKRrL8-ecZDui8Q15EwfA@mail.gmail.com>
Subject: Re: Shared memory between two primary DPDK processes
To: Dmitry Kozlyuk <dmitry.kozliuk@gmail.com>
Cc: users@dpdk.org
Content-Type: text/plain; charset="UTF-8"
X-BeenThere: users@dpdk.org
X-Mailman-Version: 2.1.29
Precedence: list
List-Id: DPDK usage discussions <users.dpdk.org>
List-Unsubscribe: <https://mails.dpdk.org/options/users>,
<mailto:users-request@dpdk.org?subject=unsubscribe>
List-Archive: <http://mails.dpdk.org/archives/users/>
List-Post: <mailto:users@dpdk.org>
List-Help: <mailto:users-request@dpdk.org?subject=help>
List-Subscribe: <https://mails.dpdk.org/listinfo/users>,
<mailto:users-request@dpdk.org?subject=subscribe>
Errors-To: users-bounces@dpdk.org
Dear Dmitry,
I tried to follow this approach and if I allocate 1GB on primary
process number 1, then I can mmap that memory on the primary process
number 2.
I also tried to convert the virt addr of the allocation made in
primary 1 to phys and then I converted the virt addr returned by mmap
in primary 2 and I got the same phys addr.
Unfortunately, if I try to allocated only 10 MB for example in primary
1, then mmap in primary 2 succeeds but it seems that this virt addr
doesn't correspond to the same phys memory as in primary 1.
In the primary 2, the mmap is used like this:
int flags = MAP_SHARED | MAP_HUGETLB ;
uint64_t* addr = (uint64_t*) mmap(NULL, sz, PROT_READ|PROT_WRITE,
flags, my_mem_fd, off);
On Fri, Apr 8, 2022 at 3:26 PM Dmitry Kozlyuk <dmitry.kozliuk@gmail.com> wrote:
>
> 2022-04-08 14:31 (UTC+0200), Antonio Di Bacco:
> > I know that it is possible to share memory between a primary and secondary
> > process using rte_memzone_reserve_aligned to allocate memory in primary
> > that is "seen" also by the secondary. If we have two primary processes
> > (started with different file-prefix) the same approach is not feasible. I
> > wonder how to share a chunk of memory hosted on a hugepage between two
> > primaries.
> >
> > Regards.
>
> Hi Antonio,
>
> Correction: all hugepages allocated by DPDK are shared
> between primary and secondary processes, not only memzones.
>
> I assume we're talking about processes within one host,
> because your previous similar question was about sharing memory between hosts
> (as we have discussed offline), which is out of scope for DPDK.
>
> As for the question directly, you need to map the same part of the same file
> in the second primary as the hugepage is mapped from in the first primary.
> I don't recommend to work with file paths, because their management
> is not straightforward (--single-file-segments, for one) and is undocumented.
>
> There is a way to share DPDK memory segment file descriptors.
> Although public, this DPDK API is dangerous in the sense that you must
> clearly understand what you're doing and how DPDK works.
> Hence the question: what is the task you need this sharing for?
> Maybe there is a simpler way.
>
> 1. In the first primary:
>
> mz = rte_memzone_reserve()
> ms = rte_mem_virt2memseg(mz->addr)
> fd = rte_memseg_get_fd(ms)
> offset = rte_memseg_get_fd_offset(ms)
>
> 2. Use Unix domain sockets with SCM_RIGHTS
> to send "fd" and "offset" to the second primary.
>
> 3. In the second primary, after receiving "fd" and "offset":
>
> flags = MAP_SHARED | MAP_HUGE | (30 << MAP_HUGE_SHIFT)
> addr = mmap(fd, offset, flags)
>
> Note that "mz" may consist of multiple "ms" depending on the sizes
> of the zone and hugepages, and on the zone alignment.
> Also "addr" may (and probably will) differ from "mz->addr".
> It is possible to pass "mz->addr" and try to force it,
> like DPDK does for primary/secondary.
>