From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id E2FA845D48 for ; Tue, 19 Nov 2024 22:40:18 +0100 (CET) Received: from mails.dpdk.org (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id AA12F4025D; Tue, 19 Nov 2024 22:40:18 +0100 (CET) Received: from smtp1.cs.Stanford.EDU (smtp1.cs.stanford.edu [171.64.64.25]) by mails.dpdk.org (Postfix) with ESMTP id 8075C4021F for ; Tue, 19 Nov 2024 22:40:16 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=cs.stanford.edu; s=cs2308; h=Content-Type:Cc:To:Subject:Message-ID:Date: From:In-Reply-To:References:MIME-Version:Sender:Reply-To: Content-Transfer-Encoding:Content-ID:Content-Description:Resent-Date: Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Id: List-Help:List-Unsubscribe:List-Subscribe:List-Post:List-Owner:List-Archive; bh=OEaTJslGVT/gbqIrV2ygG2fyZBDdci1IPiwrMht+9OI=; t=1732052416; x=1732916416; b=XEHJaKDVrCOI9zsbH+tacbcHzzIlq46nZuj8oHl7vjMCTkGIz3QQbbRZ/e4ga0ei5qyAvXWzVVc uSbHUAW9AJa7I3YFnS9ji5lErOcY2oV2fBwXPk6Ddz5MX7fY5tIteQUSsSiZuDPLugOm5a3dHLu/0 aCYbUQDGJWk2lsY8oLvNCmD7ipz2XfNng4P0I+tCwkVjZB3kkrpn4Wh13Ls/cutx0/HbC/P0bwYj/ 3YIQIbj6aHUEvxWYOO6Vm9Sf9kHQTaYVsLZOzJ1sErbEO6Hdp9vCf6ktrwBg+vjZAca1Hr7xhEYBN aKhnm0JlPwO1iGqr6+nRnxK+UM2UvbDaI7pw==; Received: from mail-pj1-f70.google.com ([209.85.216.70]:60544) by smtp1.cs.Stanford.EDU with esmtpsa (TLS1.2) tls TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256 (Exim 4.94.2) (envelope-from ) id 1tDVxG-0006w8-RD for users@dpdk.org; Tue, 19 Nov 2024 13:40:15 -0800 Received: by mail-pj1-f70.google.com with SMTP id 98e67ed59e1d1-2e5efb04da7so1376203a91.0 for ; Tue, 19 Nov 2024 13:40:14 -0800 (PST) X-Gm-Message-State: AOJu0YzOSfXNmX2o6U7vVdaC7Adb4N0bon711HCkZDFKSbeqMG0d7HSV +NxOUSm+47H7MgfuT8VlmvPHiZsush5ujMsgryidFvtHtorm2IfeMec/0lnCPdr4hXJUtrtRZW+ 1vxxwPGgEGMLe+Um/grEkothM2fpNAYhSMaOrMcrBWP7DiAA7BMLE1ZDFUNARr2bROWGTxnjGM0 enyK/6GDJPaCc4UyKJy7xbslFeVqAv6Z/yv54AKxuAC/g= X-Received: by 2002:a17:90b:224b:b0:2ea:a25d:3bc1 with SMTP id 98e67ed59e1d1-2eaca7cf152mr300118a91.28.1732052414368; Tue, 19 Nov 2024 13:40:14 -0800 (PST) X-Google-Smtp-Source: AGHT+IELXyBPVR34mrp/CToFLMfq4bpIW/CZoC02tf7mPoIvWs5NHuV+58QuDy9ZKwz+x2opVdtKMEMGtHxKtFq8j4U= X-Received: by 2002:a17:90b:224b:b0:2ea:a25d:3bc1 with SMTP id 98e67ed59e1d1-2eaca7cf152mr300107a91.28.1732052414211; Tue, 19 Nov 2024 13:40:14 -0800 (PST) MIME-Version: 1.0 References: <20241119132903.12fefa8c@hermes.local> In-Reply-To: <20241119132903.12fefa8c@hermes.local> From: Thea Corinne Rossman Date: Tue, 19 Nov 2024 13:39:38 -0800 X-Gmail-Original-Message-ID: Message-ID: Subject: Re: Containernet (Docker/Container Networking) with DPDK? To: stephen@networkplumber.org Cc: users@dpdk.org Content-Type: multipart/alternative; boundary="000000000000e1d35806274ae085" X-proofpoint-id: ef3f9bb5-395f-4fcc-a797-849143d52903 X-Spam-Score: 2.6 X-Spam-Level: ** X-Spam-Checker-Version: SpamAssassin on smtp1.cs.Stanford.EDU X-Scan-Signature: cb1c78d2334ace1af8fbe5bfe4207783 X-BeenThere: users@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK usage discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: users-bounces@dpdk.org --000000000000e1d35806274ae085 Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable This is SO helpful -- thank you so much. One follow-up question regarding NICs: can multiple containers on the same host share the same PCI device? If I have a host NIC with (say) VFIO driver binding, do I have to split it with some kind of SR-IOV so that each container has its own "NIC" binding? Or, when running DPDK's "devbind" script, can I set up each one with the same PCI address? On Tue, Nov 19, 2024 at 1:29=E2=80=AFPM Stephen Hemminger < stephen@networkplumber.org> wrote: > On Tue, 19 Nov 2024 12:53:02 -0800 > Thea Corinne Rossman wrote: > > > I'm following up on this to ask a more specific question, since my firs= t > > question was a bit all over the place. This is regarding the interplay > > between the host and the containers when setting up containers that can > run > > DPDK applications. > > > > Based on what I've found so far, it looks like I will have to fully > > configure DPDK on the host and then mount devices onto each container, > even > > if there's no need to connect the containers to the outside world. (Is > this > > correct?) If so, I don't fully understand this, since a > container/container > > network should be self-contained. > > > > - Why do we need to set up DPDK on the host? (Why isn't the containe= r > > enough?) > > - Why do we need to set up a DPDK-compatible driver on the host NICs= ? > If > > the containers are on the same machine, exchanging packets, why woul= d > the > > host NIC be involved at all? Nothing is going in or out. > > - Why do we need to configure hugepages on the host and then mount > them > > on the container? Why can't you just configure this on the > containers? Is > > this something that can't be emulated? > > > > Containers are a made up construct. They are made by setting permissions > for > namespaces and groups for resources. > > In most cases, DPDK works by passing through the raw hardware (PCI device= ) > to the userspace application. To make it work with a container system you > need to either acquire the resource in an restricted environment and then > allow access to that resource in the restricted container; or you need > to give the restricted container environment lots of privileges to > configure > and setup the raw hardware directly. In the latter case, there really is > no point in having containers. > > Same applies to hugepages. I don't think hugepages are namespaced either= . > --000000000000e1d35806274ae085 Content-Type: text/html; charset="UTF-8" Content-Transfer-Encoding: quoted-printable
This is SO helpful -- thank you so much.

One follow-up question regarding NICs: can multiple containers on th= e same host share the same PCI device? If I have a host NIC with (say) VFIO= driver binding, do I have to split it with some kind of SR-IOV so that eac= h container has its own "NIC" binding? Or, when running DPDK'= s "devbind" script, can I set up each one with the same PCI addre= ss?=C2=A0

On Tue, Nov 19, 2024 at 1:29=E2=80=AFPM Stephen Hemmin= ger <stephen@networkplumbe= r.org> wrote:
On Tue, 19 Nov 2024 12:53:02= -0800
Thea Corinne Rossman <thea.rossman@cs.stanford.edu> wrote:

> I'm following up on this to ask a more specific question, since my= first
> question was a bit all over the place. This is regarding the interplay=
> between the host and the containers when setting up containers that ca= n run
> DPDK applications.
>
> Based on what I've found so far, it looks like I will have to full= y
> configure DPDK on the host and then mount devices onto each container,= even
> if there's no need to connect the containers to the outside world.= (Is this
> correct?) If so, I don't fully understand this, since a container/= container
> network should be self-contained.
>
>=C2=A0 =C2=A0 - Why do we need to set up DPDK on the host? (Why isn'= ;t the container
>=C2=A0 =C2=A0 enough?)
>=C2=A0 =C2=A0 - Why do we need to set up a DPDK-compatible driver on th= e host NICs? If
>=C2=A0 =C2=A0 the containers are on the same machine, exchanging packet= s, why would the
>=C2=A0 =C2=A0 host NIC be involved at all? Nothing is going in or out.<= br> >=C2=A0 =C2=A0 - Why do we need to configure hugepages on the host and t= hen mount them
>=C2=A0 =C2=A0 on the container? Why can't you just configure this o= n the containers? Is
>=C2=A0 =C2=A0 this something that can't be emulated?
>

Containers are a made up construct. They are made by setting permissions fo= r
namespaces and groups for resources.

In most cases, DPDK works by passing through the raw hardware (PCI device)<= br> to the userspace application. To make it work with a container system you need to either acquire the resource in an restricted environment and then allow access to that resource in the restricted container; or you need
to give the restricted container environment lots of privileges to configur= e
and setup the raw hardware directly. In the latter case, there really is no point in having containers.

Same applies to hugepages.=C2=A0 I don't think hugepages are namespaced= either.
--000000000000e1d35806274ae085--