From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id D561845D48 for ; Wed, 20 Nov 2024 00:23:40 +0100 (CET) Received: from mails.dpdk.org (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 61F6740288; Wed, 20 Nov 2024 00:23:40 +0100 (CET) Received: from smtp1.cs.Stanford.EDU (smtp1.cs.stanford.edu [171.64.64.25]) by mails.dpdk.org (Postfix) with ESMTP id B05484025D for ; Wed, 20 Nov 2024 00:23:38 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=cs.stanford.edu; s=cs2308; h=Content-Type:Cc:To:Subject:Message-ID:Date: From:In-Reply-To:References:MIME-Version:Sender:Reply-To: Content-Transfer-Encoding:Content-ID:Content-Description:Resent-Date: Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Id: List-Help:List-Unsubscribe:List-Subscribe:List-Post:List-Owner:List-Archive; bh=7GQVOGqooP6NBI4x282EFp/EZWQevCXpjza0PFdI2dU=; t=1732058618; x=1732922618; b=cipRRcJE04Bk4+jticidheBPvm2HMTyX7Th74n3ZXgbB8dVb1Sc+jMC29Yzyup/1Ene3Nt5FZOJ MEzarWFDNf3/S/3JE58OzXlpwRzt3HaleK//uemD0Qkb0rqNngVU6Ygw70pEBImXL6TTs+fq42zog lGBlvvLp14eLkehK+KfNLwTHrv9cJ4hSod+qAzKE3fEtNjozww8GpVz6e1dpbdrXDHAIQ6twcI7X5 S7x2JhXRnYqt9kcRq+jOYKrm//4d1JHknLU2hp2wqUb9pq9Y0G4k7idrY+67Q0w7HLjWv1B6IbF9S icSWmPsAijsR+G6jt84FmEMj4eT/mjMKp9iA==; Received: from mail-pj1-f69.google.com ([209.85.216.69]:61620) by smtp1.cs.Stanford.EDU with esmtpsa (TLS1.2) tls TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256 (Exim 4.94.2) (envelope-from ) id 1tDXZJ-0006AQ-6G for users@dpdk.org; Tue, 19 Nov 2024 15:23:37 -0800 Received: by mail-pj1-f69.google.com with SMTP id 98e67ed59e1d1-2ea2dc1a51fso1448403a91.2 for ; Tue, 19 Nov 2024 15:23:37 -0800 (PST) X-Forwarded-Encrypted: i=1; AJvYcCVUfNDvYEw0MUxqNRIMW/1C+vaa2bQFJmptwsEYCFFIm2Gbl/v8fJiiYBu8L2HaRJkZf+PhGg==@dpdk.org X-Gm-Message-State: AOJu0YyMZbsXJ74Y6PhK4KoKcUenn9wiUMhqqB4uC+pCX2BOJKz2FGrf zdgmRwog4XC2pb90cES7mUfJnFlvcdBSio8a/yD+mj/zHxwvpaLFVmfiAQMR34LcCLuyL6FkRJi yCmAznLFyQ+UjlhmKCiho83QjeOi6dkonu4ZWlJkyyJvjT0ZtqOUcr9ijEclbdGs9S/D6c/DVRP jzWPfvsNeTsgc6ef3r6aEKErn3+mVQ41qT+lGxZkDtP5I= X-Gm-Gg: ASbGncstReMeG+86yD1kEul6HE8yeXDyDow0E9y6BYZd1/AlRVHVGEsjsTNUZ1Nol7W BPwDtd7OhtK1JSUT/l/CIdwDs+ae8tw== X-Received: by 2002:a17:90b:2d8c:b0:2ea:b2d7:4a35 with SMTP id 98e67ed59e1d1-2eaca6c7064mr691784a91.2.1732058616670; Tue, 19 Nov 2024 15:23:36 -0800 (PST) X-Google-Smtp-Source: AGHT+IFn3gWJXrL2iW+pE7D/Igb7/+Pz5b0cX9vF8WN2U3xjcYU6teKJzX1Q01tq5EELCg87UeQ6Wta+p4284kgaTP8= X-Received: by 2002:a17:90b:2d8c:b0:2ea:b2d7:4a35 with SMTP id 98e67ed59e1d1-2eaca6c7064mr691772a91.2.1732058616512; Tue, 19 Nov 2024 15:23:36 -0800 (PST) MIME-Version: 1.0 References: <20241119132903.12fefa8c@hermes.local> <2390764.BjyWNHgNrj@thomas> In-Reply-To: <2390764.BjyWNHgNrj@thomas> From: Thea Corinne Rossman Date: Tue, 19 Nov 2024 15:23:00 -0800 X-Gmail-Original-Message-ID: Message-ID: Subject: Re: Containernet (Docker/Container Networking) with DPDK? To: Thomas Monjalon Cc: stephen@networkplumber.org, users@dpdk.org Content-Type: multipart/alternative; boundary="0000000000009168f406274c5292" X-proofpoint-id: ef3f9bb5-395f-4fcc-a797-849143d52903 X-Spam-Score: 0.8 X-Spam-Level: X-Spam-Checker-Version: SpamAssassin on smtp1.cs.Stanford.EDU X-Scan-Signature: 6c66c47a22907c63296a3ecdec1f9d62 X-BeenThere: users@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK usage discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: users-bounces@dpdk.org --0000000000009168f406274c5292 Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable Conceptually, for two containers on the same host, how would exchanging traffic work under-the-hood? Specifically, how is the physical NIC involved, if at all? For example, on a physical host: for TX, a userspace application writes a packet to host memory and pushes its physical address/metadata to the appropriate NIC's TX queue. The NIC uses the physical address + DMA to avoid a copy when serializing/sending. (Similar for RX in the other direction, where the NIC writes to memory.) I'm not sure how this would translate to a containerized case on a single host, since traffic shouldn't need to exit and the container network has its own namespace. Say that two different containers have NICs mapped to different PCI addresses (split device). If container A appends to its TX queue, what happens next? Thanks again for your help. On Tue, Nov 19, 2024 at 2:14=E2=80=AFPM Thomas Monjalon wrote: > 19/11/2024 22:39, Thea Corinne Rossman: > > This is SO helpful -- thank you so much. > > > > One follow-up question regarding NICs: can multiple containers on the > same > > host share the same PCI device? If I have a host NIC with (say) VFIO > driver > > binding, do I have to split it with some kind of SR-IOV so that each > > container has its own "NIC" binding? Or, when running DPDK's "devbind" > > script, can I set up each one with the same PCI address? > > You need to split devices. > SR-IOV VF may help, or you can use SF which was designed exactly for this= , > with the help of the Linux auxiliary bus. > > > > --0000000000009168f406274c5292 Content-Type: text/html; charset="UTF-8" Content-Transfer-Encoding: quoted-printable
Conceptually, for two containers on the same host, how wou= ld exchanging traffic work under-the-hood? Specifically, how is the physica= l NIC involved, if at all?

For example, on a physical ho= st: for TX, a userspace application writes a packet to host memory and push= es its physical address/metadata to the appropriate NIC's TX queue. The= NIC uses the physical address=C2=A0+ DMA to avoid a copy when serializing/= sending. (Similar for RX in the other direction, where the NIC writes to me= mory.)=C2=A0

I'm not sure how this would trans= late to a containerized case on a single host, since traffic shouldn't = need to exit and the container network has its own namespace. Say that two = different containers have NICs mapped to different PCI addresses (split dev= ice). If container A appends to its TX queue, what happens next?=C2=A0

Thanks again for your help.=C2=A0

On Tue, Nov 19,= 2024 at 2:14=E2=80=AFPM Thomas Monjalon <thomas@monjalon.net> wrote:
19/11/2024 22:39, Thea Corinne Rossman:
> This is SO helpful -- thank you so much.
>
> One follow-up question regarding NICs: can multiple containers on the = same
> host share the same PCI device? If I have a host NIC with (say) VFIO d= river
> binding, do I have to split it with some kind of SR-IOV so that each > container has its own "NIC" binding? Or, when running DPDK&#= 39;s "devbind"
> script, can I set up each one with the same PCI address?

You need to split devices.
SR-IOV VF may help, or you can use SF which was designed exactly for this,<= br> with the help of the Linux auxiliary bus.



--0000000000009168f406274c5292--