* Can the DPDK multi-process feature be used in Docker containers? @ 2021-11-17 13:47 Staffan Wiklund 2021-11-18 6:11 ` Cliff Burdick 0 siblings, 1 reply; 7+ messages in thread From: Staffan Wiklund @ 2021-11-17 13:47 UTC (permalink / raw) To: users [-- Attachment #1: Type: text/plain, Size: 617 bytes --] Hello I wonder if it is possible to use the DPDK multi-process feature in Docker containers? That is, can a DPDK application execute in a Docker container and share its DPDK memory with another DPDK application executing in another Docker container using the DPDK multi-process feature? For example if the DPDK example mp_server executes in one Docker container: <build_dir>/examples/dpdk-mp_server -l 1-2 -n 4 -- -p 3 -n 2 and the DPDK example mp_client executes in another Docker container: <build_dir>/examples/dpdk-mp_client -l 3 -n 4 --proc-type=auto -- -n 0 Is this possible to implement? Thanks Staffan [-- Attachment #2: Type: text/html, Size: 1997 bytes --] ^ permalink raw reply [flat|nested] 7+ messages in thread
* Re: Can the DPDK multi-process feature be used in Docker containers? 2021-11-17 13:47 Can the DPDK multi-process feature be used in Docker containers? Staffan Wiklund @ 2021-11-18 6:11 ` Cliff Burdick 2021-11-18 6:20 ` Li Feng 0 siblings, 1 reply; 7+ messages in thread From: Cliff Burdick @ 2021-11-18 6:11 UTC (permalink / raw) To: Staffan Wiklund; +Cc: users [-- Attachment #1: Type: text/plain, Size: 890 bytes --] Yes. Try mapping the dpdk metadata directory (/var/run/dpdk) into both containers from the host. You should be able to do the normal dual process methods. On Wed, Nov 17, 2021, 05:58 Staffan Wiklund <staffan491@gmail.com> wrote: > Hello > > I wonder if it is possible to use the DPDK multi-process feature in Docker > containers? > > That is, can a DPDK application execute in a Docker container and share its > DPDK memory with another DPDK application executing in another Docker > container > using the DPDK multi-process feature? > > For example if the DPDK example mp_server executes in one Docker container: > <build_dir>/examples/dpdk-mp_server -l 1-2 -n 4 -- -p 3 -n 2 > > and the DPDK example mp_client executes in another Docker container: > > <build_dir>/examples/dpdk-mp_client -l 3 -n 4 --proc-type=auto -- -n 0 > > Is this possible to implement? > > Thanks > Staffan > > > [-- Attachment #2: Type: text/html, Size: 2356 bytes --] ^ permalink raw reply [flat|nested] 7+ messages in thread
* Re: Can the DPDK multi-process feature be used in Docker containers? 2021-11-18 6:11 ` Cliff Burdick @ 2021-11-18 6:20 ` Li Feng 2021-11-18 6:23 ` Cliff Burdick 0 siblings, 1 reply; 7+ messages in thread From: Li Feng @ 2021-11-18 6:20 UTC (permalink / raw) To: Cliff Burdick; +Cc: Staffan Wiklund, users On Thu, Nov 18, 2021 at 2:11 PM Cliff Burdick <shaklee3@gmail.com> wrote: > > Yes. Try mapping the dpdk metadata directory (/var/run/dpdk) into both containers from the host. You should be able to do the normal dual process methods. > By the way, if I want to run two separate dpdk processes in two different dockers, what should I do? e.g. Running two processes: examples/dpdk-mp_server These two dockers have mapped the same /dev/hugepages directories. I have tested, the dpdk will crash because the hugepages are mixed. Thanks. > > On Wed, Nov 17, 2021, 05:58 Staffan Wiklund <staffan491@gmail.com> wrote: >> >> Hello >> >> I wonder if it is possible to use the DPDK multi-process feature in Docker containers? >> >> That is, can a DPDK application execute in a Docker container and share its >> DPDK memory with another DPDK application executing in another Docker container >> using the DPDK multi-process feature? >> >> For example if the DPDK example mp_server executes in one Docker container: >> <build_dir>/examples/dpdk-mp_server -l 1-2 -n 4 -- -p 3 -n 2 >> >> and the DPDK example mp_client executes in another Docker container: >> >> <build_dir>/examples/dpdk-mp_client -l 3 -n 4 --proc-type=auto -- -n 0 >> >> Is this possible to implement? >> >> Thanks >> Staffan >> >> ^ permalink raw reply [flat|nested] 7+ messages in thread
* Re: Can the DPDK multi-process feature be used in Docker containers? 2021-11-18 6:20 ` Li Feng @ 2021-11-18 6:23 ` Cliff Burdick 2021-11-18 6:47 ` Li Feng 0 siblings, 1 reply; 7+ messages in thread From: Cliff Burdick @ 2021-11-18 6:23 UTC (permalink / raw) To: Li Feng; +Cc: Staffan Wiklund, users [-- Attachment #1: Type: text/plain, Size: 1590 bytes --] I don't remember that being a problem. Are you starting them with two different file prefixes and mounting the hugepages directory into both containers? On Wed, Nov 17, 2021, 22:20 Li Feng <fengli@smartx.com> wrote: > On Thu, Nov 18, 2021 at 2:11 PM Cliff Burdick <shaklee3@gmail.com> wrote: > > > > Yes. Try mapping the dpdk metadata directory (/var/run/dpdk) into both > containers from the host. You should be able to do the normal dual process > methods. > > > By the way, if I want to run two separate dpdk processes in two > different dockers, what should I do? > e.g. Running two processes: examples/dpdk-mp_server > These two dockers have mapped the same /dev/hugepages directories. > I have tested, the dpdk will crash because the hugepages are mixed. > > Thanks. > > > > On Wed, Nov 17, 2021, 05:58 Staffan Wiklund <staffan491@gmail.com> > wrote: > >> > >> Hello > >> > >> I wonder if it is possible to use the DPDK multi-process feature in > Docker containers? > >> > >> That is, can a DPDK application execute in a Docker container and share > its > >> DPDK memory with another DPDK application executing in another Docker > container > >> using the DPDK multi-process feature? > >> > >> For example if the DPDK example mp_server executes in one Docker > container: > >> <build_dir>/examples/dpdk-mp_server -l 1-2 -n 4 -- -p 3 -n 2 > >> > >> and the DPDK example mp_client executes in another Docker container: > >> > >> <build_dir>/examples/dpdk-mp_client -l 3 -n 4 --proc-type=auto -- -n 0 > >> > >> Is this possible to implement? > >> > >> Thanks > >> Staffan > >> > >> > [-- Attachment #2: Type: text/html, Size: 2287 bytes --] ^ permalink raw reply [flat|nested] 7+ messages in thread
* Re: Can the DPDK multi-process feature be used in Docker containers? 2021-11-18 6:23 ` Cliff Burdick @ 2021-11-18 6:47 ` Li Feng 2021-11-18 7:12 ` Staffan Wiklund 0 siblings, 1 reply; 7+ messages in thread From: Li Feng @ 2021-11-18 6:47 UTC (permalink / raw) To: Cliff Burdick; +Cc: Staffan Wiklund, users On Thu, Nov 18, 2021 at 2:23 PM Cliff Burdick <shaklee3@gmail.com> wrote: > > I don't remember that being a problem. Are you starting them with two different file prefixes and mounting the hugepages directory into both containers? > I'm running the spdk processes, so the hugepage files are created with prefix "spdk-<PID>", both containers are mounting the same /dev/hugepages. > > On Wed, Nov 17, 2021, 22:20 Li Feng <fengli@smartx.com> wrote: >> >> On Thu, Nov 18, 2021 at 2:11 PM Cliff Burdick <shaklee3@gmail.com> wrote: >> > >> > Yes. Try mapping the dpdk metadata directory (/var/run/dpdk) into both containers from the host. You should be able to do the normal dual process methods. >> > >> By the way, if I want to run two separate dpdk processes in two >> different dockers, what should I do? >> e.g. Running two processes: examples/dpdk-mp_server >> These two dockers have mapped the same /dev/hugepages directories. >> I have tested, the dpdk will crash because the hugepages are mixed. >> >> Thanks. >> > >> > On Wed, Nov 17, 2021, 05:58 Staffan Wiklund <staffan491@gmail.com> wrote: >> >> >> >> Hello >> >> >> >> I wonder if it is possible to use the DPDK multi-process feature in Docker containers? >> >> >> >> That is, can a DPDK application execute in a Docker container and share its >> >> DPDK memory with another DPDK application executing in another Docker container >> >> using the DPDK multi-process feature? >> >> >> >> For example if the DPDK example mp_server executes in one Docker container: >> >> <build_dir>/examples/dpdk-mp_server -l 1-2 -n 4 -- -p 3 -n 2 >> >> >> >> and the DPDK example mp_client executes in another Docker container: >> >> >> >> <build_dir>/examples/dpdk-mp_client -l 3 -n 4 --proc-type=auto -- -n 0 >> >> >> >> Is this possible to implement? >> >> >> >> Thanks >> >> Staffan >> >> >> >> ^ permalink raw reply [flat|nested] 7+ messages in thread
* Re: Can the DPDK multi-process feature be used in Docker containers? 2021-11-18 6:47 ` Li Feng @ 2021-11-18 7:12 ` Staffan Wiklund 2021-11-18 17:38 ` Cliff Burdick 0 siblings, 1 reply; 7+ messages in thread From: Staffan Wiklund @ 2021-11-18 7:12 UTC (permalink / raw) To: Li Feng; +Cc: Cliff Burdick, users [-- Attachment #1: Type: text/plain, Size: 2204 bytes --] Cliff, Many thanks for your help! Mapping /var/run/dpdk into both containers as you said made it possible to use the multi-process feature in Docker containers. Regards Staffan Den tors 18 nov. 2021 kl 07:47 skrev Li Feng <fengli@smartx.com>: > On Thu, Nov 18, 2021 at 2:23 PM Cliff Burdick <shaklee3@gmail.com> wrote: > > > > I don't remember that being a problem. Are you starting them with two > different file prefixes and mounting the hugepages directory into both > containers? > > > I'm running the spdk processes, so the hugepage files are created with > prefix "spdk-<PID>", both containers are mounting the same > /dev/hugepages. > > > > > > On Wed, Nov 17, 2021, 22:20 Li Feng <fengli@smartx.com> wrote: > >> > >> On Thu, Nov 18, 2021 at 2:11 PM Cliff Burdick <shaklee3@gmail.com> > wrote: > >> > > >> > Yes. Try mapping the dpdk metadata directory (/var/run/dpdk) into > both containers from the host. You should be able to do the normal dual > process methods. > >> > > >> By the way, if I want to run two separate dpdk processes in two > >> different dockers, what should I do? > >> e.g. Running two processes: examples/dpdk-mp_server > >> These two dockers have mapped the same /dev/hugepages directories. > >> I have tested, the dpdk will crash because the hugepages are mixed. > >> > >> Thanks. > >> > > >> > On Wed, Nov 17, 2021, 05:58 Staffan Wiklund <staffan491@gmail.com> > wrote: > >> >> > >> >> Hello > >> >> > >> >> I wonder if it is possible to use the DPDK multi-process feature in > Docker containers? > >> >> > >> >> That is, can a DPDK application execute in a Docker container and > share its > >> >> DPDK memory with another DPDK application executing in another > Docker container > >> >> using the DPDK multi-process feature? > >> >> > >> >> For example if the DPDK example mp_server executes in one Docker > container: > >> >> <build_dir>/examples/dpdk-mp_server -l 1-2 -n 4 -- -p 3 -n 2 > >> >> > >> >> and the DPDK example mp_client executes in another Docker container: > >> >> > >> >> <build_dir>/examples/dpdk-mp_client -l 3 -n 4 --proc-type=auto -- -n > 0 > >> >> > >> >> Is this possible to implement? > >> >> > >> >> Thanks > >> >> Staffan > >> >> > >> >> > [-- Attachment #2: Type: text/html, Size: 3350 bytes --] ^ permalink raw reply [flat|nested] 7+ messages in thread
* Re: Can the DPDK multi-process feature be used in Docker containers? 2021-11-18 7:12 ` Staffan Wiklund @ 2021-11-18 17:38 ` Cliff Burdick 0 siblings, 0 replies; 7+ messages in thread From: Cliff Burdick @ 2021-11-18 17:38 UTC (permalink / raw) To: Staffan Wiklund; +Cc: Li Feng, users [-- Attachment #1: Type: text/plain, Size: 2553 bytes --] Li, I would suggest pasting your errors. It sounds like a separate issue from Staffan, and I know for sure it works in DPDK (never used SPDK), so I'd be curious to see what the problem is. On Wed, Nov 17, 2021 at 11:13 PM Staffan Wiklund <staffan491@gmail.com> wrote: > Cliff, > > Many thanks for your help! > Mapping /var/run/dpdk into both containers as you said made it possible to > use the multi-process feature in Docker containers. > > Regards > Staffan > > Den tors 18 nov. 2021 kl 07:47 skrev Li Feng <fengli@smartx.com>: > >> On Thu, Nov 18, 2021 at 2:23 PM Cliff Burdick <shaklee3@gmail.com> wrote: >> > >> > I don't remember that being a problem. Are you starting them with two >> different file prefixes and mounting the hugepages directory into both >> containers? >> > >> I'm running the spdk processes, so the hugepage files are created with >> prefix "spdk-<PID>", both containers are mounting the same >> /dev/hugepages. >> >> >> > >> > On Wed, Nov 17, 2021, 22:20 Li Feng <fengli@smartx.com> wrote: >> >> >> >> On Thu, Nov 18, 2021 at 2:11 PM Cliff Burdick <shaklee3@gmail.com> >> wrote: >> >> > >> >> > Yes. Try mapping the dpdk metadata directory (/var/run/dpdk) into >> both containers from the host. You should be able to do the normal dual >> process methods. >> >> > >> >> By the way, if I want to run two separate dpdk processes in two >> >> different dockers, what should I do? >> >> e.g. Running two processes: examples/dpdk-mp_server >> >> These two dockers have mapped the same /dev/hugepages directories. >> >> I have tested, the dpdk will crash because the hugepages are mixed. >> >> >> >> Thanks. >> >> > >> >> > On Wed, Nov 17, 2021, 05:58 Staffan Wiklund <staffan491@gmail.com> >> wrote: >> >> >> >> >> >> Hello >> >> >> >> >> >> I wonder if it is possible to use the DPDK multi-process feature in >> Docker containers? >> >> >> >> >> >> That is, can a DPDK application execute in a Docker container and >> share its >> >> >> DPDK memory with another DPDK application executing in another >> Docker container >> >> >> using the DPDK multi-process feature? >> >> >> >> >> >> For example if the DPDK example mp_server executes in one Docker >> container: >> >> >> <build_dir>/examples/dpdk-mp_server -l 1-2 -n 4 -- -p 3 -n 2 >> >> >> >> >> >> and the DPDK example mp_client executes in another Docker container: >> >> >> >> >> >> <build_dir>/examples/dpdk-mp_client -l 3 -n 4 --proc-type=auto -- >> -n 0 >> >> >> >> >> >> Is this possible to implement? >> >> >> >> >> >> Thanks >> >> >> Staffan >> >> >> >> >> >> >> > [-- Attachment #2: Type: text/html, Size: 3920 bytes --] ^ permalink raw reply [flat|nested] 7+ messages in thread
end of thread, other threads:[~2021-11-18 17:38 UTC | newest] Thread overview: 7+ messages (download: mbox.gz / follow: Atom feed) -- links below jump to the message on this page -- 2021-11-17 13:47 Can the DPDK multi-process feature be used in Docker containers? Staffan Wiklund 2021-11-18 6:11 ` Cliff Burdick 2021-11-18 6:20 ` Li Feng 2021-11-18 6:23 ` Cliff Burdick 2021-11-18 6:47 ` Li Feng 2021-11-18 7:12 ` Staffan Wiklund 2021-11-18 17:38 ` Cliff Burdick
This is a public inbox, see mirroring instructions for how to clone and mirror all data and code used for this inbox; as well as URLs for NNTP newsgroup(s).