From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 758B645D48 for ; Wed, 20 Nov 2024 00:30:51 +0100 (CET) Received: from mails.dpdk.org (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 65C9A4027E; Wed, 20 Nov 2024 00:30:51 +0100 (CET) Received: from fout-a6-smtp.messagingengine.com (fout-a6-smtp.messagingengine.com [103.168.172.149]) by mails.dpdk.org (Postfix) with ESMTP id DDECB4025D for ; Wed, 20 Nov 2024 00:30:50 +0100 (CET) Received: from phl-compute-06.internal (phl-compute-06.phl.internal [10.202.2.46]) by mailfout.phl.internal (Postfix) with ESMTP id 788AD1380710; Tue, 19 Nov 2024 18:30:50 -0500 (EST) Received: from phl-mailfrontend-01 ([10.202.2.162]) by phl-compute-06.internal (MEProxy); Tue, 19 Nov 2024 18:30:50 -0500 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=monjalon.net; h= cc:cc:content-transfer-encoding:content-type:content-type:date :date:from:from:in-reply-to:in-reply-to:message-id:mime-version :references:reply-to:subject:subject:to:to; s=fm3; t=1732059050; x=1732145450; bh=gE0uELKEECCfWp6vnNBMOdzlXQMTipVnDaL9zI4V0sg=; b= wHp5pukNiFZXbwjB5CKJltuYKwuzT9mcUyiz2Kv9RVSmuMs3IgwKjHADArjtSDfw mieieGkOxp5z7G//eNrI70gl22MVzFKaleD3HRfaBl2olcNJMjmKWZZj2IsjDngL BMPiOfo68KRgI0v8d4pFRHTgXgr8s9qDtkCcWamBGi+hjdc5ayO3I/nLiaYtCTbR Z88giNSkl9k6LBVHOPfoDWVY1FQJcWMcsxY2jF9qgzvxIdvrwR1G/a0F1OYWEZJI 00GFWUqB5g/wRBmCjpfIt60klnkaNKvxJSBdCJmU7v0H7hQmw4q6aHJZsjaGJrwm Cp58tRbi7cR41aoiYbUPqA== DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d= messagingengine.com; h=cc:cc:content-transfer-encoding :content-type:content-type:date:date:feedback-id:feedback-id :from:from:in-reply-to:in-reply-to:message-id:mime-version :references:reply-to:subject:subject:to:to:x-me-proxy :x-me-sender:x-me-sender:x-sasl-enc; s=fm3; t=1732059050; x= 1732145450; bh=gE0uELKEECCfWp6vnNBMOdzlXQMTipVnDaL9zI4V0sg=; b=A 8BpcmzWJVCc8DWDQmKIkBDLQrHasAO4fn6uMJv8e5qVynyJ2b+kiIN2JWjlkywJe 3Iz6Mm/4gmtl5j5PcVU2YjpsN+4lSZZMkpF/8Vtax2r++9MLb0dP1YufcWuN5hUG xNZYNrCqUqwh4ESKhC7bNgfv6vlGMukGTYVlgX3XltW9RFqRQBI4XbPiudFh+3/8 kHpkmzs/IInUf69mz3y5XskhJc6t9wOCruYpB8VQZeGEjp8Q+P9w7LNZMSRci9A8 uQ1kKnooHBC3dNRog2kQ0MN4fwrf+PLhQI0JGE1SVluUKWkyLRme7XSKj7EGAaDH yg1+jmv0OIXhyikg0RRNA== X-ME-Sender: X-ME-Received: X-ME-Proxy-Cause: gggruggvucftvghtrhhoucdtuddrgeefuddrfeefgddutdcutefuodetggdotefrodftvf curfhrohhfihhlvgemucfhrghsthforghilhdpggftfghnshhusghstghrihgsvgdpuffr tefokffrpgfnqfghnecuuegrihhlohhuthemuceftddtnecusecvtfgvtghiphhivghnth hsucdlqddutddtmdenucfjughrpefhvfevufffkfgjfhgggfgtsehtqhertddttdejnecu hfhrohhmpefvhhhomhgrshcuofhonhhjrghlohhnuceothhhohhmrghssehmohhnjhgrlh honhdrnhgvtheqnecuggftrfgrthhtvghrnhepffdtuefhhedvkeelleevffdvlefhleeh vdegtddvvdduueeivedtgfejvddugeefnecuffhomhgrihhnpeguphgukhdrohhrghenuc evlhhushhtvghrufhiiigvpedtnecurfgrrhgrmhepmhgrihhlfhhrohhmpehthhhomhgr shesmhhonhhjrghlohhnrdhnvghtpdhnsggprhgtphhtthhopeefpdhmohguvgepshhmth hpohhuthdprhgtphhtthhopehthhgvrgdrrhhoshhsmhgrnhestghsrdhsthgrnhhfohhr ugdrvgguuhdprhgtphhtthhopehsthgvphhhvghnsehnvghtfihorhhkphhluhhmsggvrh drohhrghdprhgtphhtthhopehushgvrhhsseguphgukhdrohhrgh X-ME-Proxy: Feedback-ID: i47234305:Fastmail Received: by mail.messagingengine.com (Postfix) with ESMTPA; Tue, 19 Nov 2024 18:30:49 -0500 (EST) From: Thomas Monjalon To: Thea Corinne Rossman Cc: stephen@networkplumber.org, users@dpdk.org Subject: Re: Containernet (Docker/Container Networking) with DPDK? Date: Wed, 20 Nov 2024 00:30:48 +0100 Message-ID: <28399425.gRfpFWEtPU@thomas> In-Reply-To: References: <2390764.BjyWNHgNrj@thomas> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" X-BeenThere: users@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK usage discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: users-bounces@dpdk.org 20/11/2024 00:23, Thea Corinne Rossman: > Conceptually, for two containers on the same host, how would exchanging > traffic work under-the-hood? Specifically, how is the physical NIC > involved, if at all? Traffic between containers on the same host does not need to use a PCI devi= ce. You can check virtio-user for this usage: https://doc.dpdk.org/guides/howto/virtio_user_for_container_networking.html > For example, on a physical host: for TX, a userspace application writes a > packet to host memory and pushes its physical address/metadata to the > appropriate NIC's TX queue. The NIC uses the physical address + DMA to > avoid a copy when serializing/sending. (Similar for RX in the other > direction, where the NIC writes to memory.) >=20 > I'm not sure how this would translate to a containerized case on a single > host, since traffic shouldn't need to exit and the container network has > its own namespace. Say that two different containers have NICs mapped to > different PCI addresses (split device). If container A appends to its TX > queue, what happens next? >=20 > Thanks again for your help. >=20 > On Tue, Nov 19, 2024 at 2:14=E2=80=AFPM Thomas Monjalon wrote: >=20 > > 19/11/2024 22:39, Thea Corinne Rossman: > > > This is SO helpful -- thank you so much. > > > > > > One follow-up question regarding NICs: can multiple containers on the > > same > > > host share the same PCI device? If I have a host NIC with (say) VFIO > > driver > > > binding, do I have to split it with some kind of SR-IOV so that each > > > container has its own "NIC" binding? Or, when running DPDK's "devbind" > > > script, can I set up each one with the same PCI address? > > > > You need to split devices. > > SR-IOV VF may help, or you can use SF which was designed exactly for th= is, > > with the help of the Linux auxiliary bus.