From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 4A287A00C4 for ; Mon, 18 Apr 2022 19:34:22 +0200 (CEST) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id D38244014F; Mon, 18 Apr 2022 19:34:21 +0200 (CEST) Received: from mail-lf1-f49.google.com (mail-lf1-f49.google.com [209.85.167.49]) by mails.dpdk.org (Postfix) with ESMTP id CE68240141 for ; Mon, 18 Apr 2022 19:34:20 +0200 (CEST) Received: by mail-lf1-f49.google.com with SMTP id w1so1840766lfa.4 for ; Mon, 18 Apr 2022 10:34:20 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :cc; bh=2xQVc6m1CYGC3yVSWSN/JBlkLfTthUGiFF0akKRLcqc=; b=VpUGAcjq13K+hw77wqa/wDZpPfDmQntY5+wod+hA9MFi0yu4BUBxyVXrZKX0ffX6yA WVW+cHvCulsoICO2B5X9dX0H0aHFApyhGArkNbD3rjchEeLEngUC5d0SBApRaSQLtQg9 UZxrg1+qbJuPXhLdtdRhAiObsKJJr4gXmXz8xR00vaguseNBZ4x4t6iUaYqoug9cfyxI Z13zSBlSn7gEJRs3JfXSBzuiPC3T6CpDBJzRXVC3Tesyt7MeiHdXnvQwcivf47/UplSc 3UHF26f8a0B482met5FdY9B7IQaRhnzVwfqiMtU2edB3J/QcPhsAfgFtxVECga7jauT3 igvg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=2xQVc6m1CYGC3yVSWSN/JBlkLfTthUGiFF0akKRLcqc=; b=fdODDqESzhLcfWeaiZdg3FULL7NG8GffEavL150v5W+q4g/FSMXDhcuMDT868twoPm GkG4r8jFm8UMaiyB8I3zNTZ+6Rr496rbExqBazsCeqBMp1SQs/UC9eNDFaJgbhAjjT89 XaEnjDQIA63zJPg+KflHs3LfRuEcoa7C717sSK44c2Qu3TO+iN9ZSxDGC0Q37WcnWy+p l7TDYNhlro4NGGpTzWwcLjElcgDoGa7W+gg4zaaXfkq6MeFebxx/3cOb1Y/0izaqfDx7 Sd17oeG//tJ1ffcCyCnsnqoywKPska/bQhuACXftXy8nRqIAIxlLbi6vpBj7oW2y/Xs1 dDzA== X-Gm-Message-State: AOAM530hZnMD/aDHiaGXpDOhTOCKM6xLAxnsKSCpKPZJHJa1lELwvzIB tmuZY/q7CeADvkI2G1EpB3JiIh04U3rTy2NijBU= X-Google-Smtp-Source: ABdhPJxFD2iaGb0rI5qu/gogOtfhHZgs5PkTv9kTotq7BE4kqVbjdVecT3SePZ6Pszksox0y7Pj4XkED4HJFI2yUcrc= X-Received: by 2002:a05:6512:ac9:b0:470:e6d0:1bd8 with SMTP id n9-20020a0565120ac900b00470e6d01bd8mr6937752lfu.614.1650303259060; Mon, 18 Apr 2022 10:34:19 -0700 (PDT) MIME-Version: 1.0 References: <20220408162629.372dfd0d@sovereign> <20220411203011.4df9f6f4@sovereign> <20220414220148.0d638532@sovereign> In-Reply-To: From: Antonio Di Bacco Date: Mon, 18 Apr 2022 19:34:06 +0200 Message-ID: Subject: Re: Shared memory between two primary DPDK processes To: Dmitry Kozlyuk Cc: users@dpdk.org Content-Type: multipart/alternative; boundary="000000000000872b9f05dcf12b9e" X-BeenThere: users@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK usage discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: users-bounces@dpdk.org --000000000000872b9f05dcf12b9e Content-Type: text/plain; charset="UTF-8" At the end I tried the pidfd_getfd syscall that is working really fine and giving me back a "clone" fd of an fd in that was opened from another process. I tested it opening a text file in the first process and after cloning the fd , I could really read the file also in the second process. Now the weird thing: 1) In the first process I allocate- a huge page, then get the fd 2) In the second process I get my "clone" fd and do an mmap, it works but if I write on that memory, the first process cannot see what I wrote int second_process(int remote_pid, int remote_mem_fd) { printf("remote_pid %d remote_mem_fd %d\n", remote_pid, remote_mem_fd); int pidfd = syscall(__NR_pidfd_open, remote_pid, 0); int my_mem_fd = syscall(438, pidfd, remote_mem_fd, 0); printf("my_mem_fd %d\n", my_mem_fd); // This is nice int flags = MAP_SHARED | MAP_HUGETLB | (30 << MAP_HUGE_SHIFT); uint64_t* addr = (uint64_t*) mmap(NULL, 1024 * 1024 * 1024, PROT_READ|PROT_WRITE, flags, my_mem_fd, 0); if (addr == -1) perror("mmap"); *addr = 0x0101010102020202; } Il giorno gio 14 apr 2022 alle ore 21:51 Antonio Di Bacco < a.dibacco.ks@gmail.com> ha scritto: > > > Il giorno gio 14 apr 2022 alle ore 21:01 Dmitry Kozlyuk < > dmitry.kozliuk@gmail.com> ha scritto: > >> 2022-04-14 10:20 (UTC+0200), Antonio Di Bacco: >> [...] >> > Ok, after having a look to memif I managed to exchange the fd between >> the >> > two processes and it works. >> > Anyway the procedure seems a little bit clunky and I think I'm going to >> use >> > the new SYSCALL pidfd_getfd >> > to achieve the same result. In your opinion this method (getfd_pidfd) >> > could also work if the two DPDK processes >> > are inside different docker containers? >> >> Honestly, I've just learned about pidfd_getfd() from you. >> But I know that containers use PID namespaces, so there's a question >> how you will obtain the pidfd of a process in another container. >> >> In general, any method of sharing FD will work. >> Remember that you also need offset and size. >> Given that some channel is required to share those, >> I think Unix domain socket is still the preferred way. >> >> > Or is there another mechanims like using handles to hugepages present in >> > the filesystem to share between two >> > different containers? >> >> FD is needed for mmap(). >> You need to either pass the FD or open() the same hugepage file by path. >> I advise against using paths because they are not a part of DPDK API >> contract. >> > > Thank you very much Dmitry, your answers are always enlightening. > I'm going to ask a different question on the dpdk.org about the best > practice to share memory between two dpdk processes running in different > containers. > --000000000000872b9f05dcf12b9e Content-Type: text/html; charset="UTF-8" Content-Transfer-Encoding: quoted-printable
At the end I tried the pidfd_getfd syscall that is working= really fine and giving me back a "clone" fd of an fd in that was= opened from another process. I tested it opening a text file in the first = process=C2=A0 and after cloning the fd , I could really read the file also = in the second process.=C2=A0
Now the weird thing:
1) In the f= irst process I allocate- a huge page, then get the fd
2) In the s= econd process I get my "clone" fd and do an mmap, it works but if= I write on that memory, the first process cannot see what I wrote
=C2=A0=C2=A0
int second_process(int remote_pid, int remote_mem_= fd) {

=C2=A0 =C2=A0 =C2=A0 =C2=A0 printf("rem= ote_pid %d remote_mem_fd %d\n", remote_pid, remote_mem_fd);
=C2=A0 = =C2=A0 =C2=A0 =C2=A0 int pidfd =3D syscall(__NR_pidfd_open, remote_pid, 0);=

=C2=A0 =C2=A0 =C2=A0 =C2=A0 int my_mem_fd =3D syscall(438, pidfd, r= emote_mem_fd, 0);
=C2=A0 =C2=A0 =C2=A0 =C2=A0 printf("my_mem_fd %d\= n", my_mem_fd);=C2=A0 =C2=A0// This is nice

=C2=A0 =C2=A0 =C2= =A0 =C2=A0 int flags =3D MAP_SHARED | MAP_HUGETLB | (30 << MAP_HUGE_S= HIFT);
=C2=A0 =C2=A0 =C2=A0 =C2=A0 uint64_t* addr =3D (uint64_t*) mmap(N= ULL, 1024 * 1024 * 1024, PROT_READ|PROT_WRITE, flags, my_mem_fd, 0);
=C2= =A0 =C2=A0 =C2=A0 =C2=A0 if (addr =3D=3D -1)
=C2=A0 =C2=A0 =C2=A0 =C2=A0= =C2=A0 =C2=A0 perror("mmap");
=C2=A0 =C2=A0 =C2=A0 =C2=A0 *ad= dr =3D 0x0101010102020202;
}


Il giorno gi= o 14 apr 2022 alle ore 21:51 Antonio Di Bacco <a.dibacco.ks@gmail.com> ha scritto:


Il giorno gio 14 apr 2022 alle ore 21:01 Dmitry Kozlyuk <dmitry.kozliuk@gmail= .com> ha scritto:
2022-04-14 10:20 (UTC+0200), Antonio Di Bacco:
[...]
> Ok, after having a look to memif I managed to exchange the fd=C2=A0 be= tween the
> two processes and it works.
> Anyway the procedure seems a little bit clunky and I think I'm goi= ng to use
> the new SYSCALL pidfd_getfd
> to achieve the same result.=C2=A0 In your opinion this method (getfd_p= idfd)
> could also work if the two DPDK processes
> are inside different docker containers?

Honestly, I've just learned about pidfd_getfd() from you.
But I know that containers use PID namespaces, so there's a question how you will obtain the pidfd of a process in another container.

In general, any method of sharing FD will work.
Remember that you also need offset and size.
Given that some channel is required to share those,
I think Unix domain socket is still the preferred way.

> Or is there another mechanims like using handles to hugepages present = in
> the filesystem to share between two
> different containers?

FD is needed for mmap().
You need to either pass the FD or open() the same hugepage file by path. I advise against using paths because they are not a part of DPDK API contra= ct.

Thank you very much Dmitry, your an= swers are always enlightening.
I'm going to ask a different q= uestion on the dpdk.org a= bout the best practice to share memory between two dpdk processes running i= n different containers.=C2=A0
--000000000000872b9f05dcf12b9e--