From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: <users-bounces@dpdk.org> Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 75733A0A02 for <public@inbox.dpdk.org>; Thu, 20 May 2021 23:19:26 +0200 (CEST) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 0324640143; Thu, 20 May 2021 23:19:26 +0200 (CEST) Received: from mail-ej1-f46.google.com (mail-ej1-f46.google.com [209.85.218.46]) by mails.dpdk.org (Postfix) with ESMTP id E643C40041 for <users@dpdk.org>; Thu, 20 May 2021 23:19:24 +0200 (CEST) Received: by mail-ej1-f46.google.com with SMTP id s22so27239248ejv.12 for <users@dpdk.org>; Thu, 20 May 2021 14:19:24 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :cc; bh=DZz61qXWYKgRAUWOSi5AW6tpKS9CjTvtcK1m8xK+Hek=; b=WyhBCHl8LV8iQ3zyFfHiOv7rdLsS1e64Z+zlMhRIr855Ju5QVXO9PdgHTEdkFz4siw a9DOsmFprciG3hVsU/eGD68XSbqBcJKWMhl5cz/bkDeTP7J9KWW4z/RJfMoenqBUX/Mg pz/oI4/bvlwxHtAku03IYkW+68itpSgSviy/DsjcDISl16/hvPBBiEzLIm8xsBINyRw5 yflTH2E3KHi26PA9MwL84lSRUY4Vmirt6I6HEBgt3q2vtOpLl/eY88Lam9QFtSEAj7uw aiR226qXnvDidSDYNv0kJlN3HF1DN4cAvnx+NeApV+D/Y1FUJ0+Dm6wHkUia3TxbMeTP mSzw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=DZz61qXWYKgRAUWOSi5AW6tpKS9CjTvtcK1m8xK+Hek=; b=nXvXDimbNR4qPkJiYwPv0hn6s0aIVIyB8Wgv/taMBkr/sin/UJNVjFs+BCX9GWmnnU MnhQc5H1d3ldq2yR8pga6GDJ45fEmtxa46T+RqHcIV4XxAmUCYxpb2k6aCXexCbdvVtp EBCMboUbVFJegWeNkkiShLPSUe6tvScfxOVIZ3v4ZTEKrMp7e+/M5mZ/IUCWqwtsinoL RGxolfkU9adZXRqhtx/UFyynM6+mWejrOtyQ7OshkkdurJXNepFFgdi5kLsL4jj1rUQx SVDAbZPnmeLen2vHWrDGfB0iJZ9q5GqaTPTNsUi4azzuer+djEc7Jc6LHhe5UQyv29H+ 2L8A== X-Gm-Message-State: AOAM530As69wkFnoeoQVqhR8YveghkbMlWZQevel2oSquS6wABlqPr8D AuxhXdDQgRl9Xev53W+nWORbhk1HVvD1F6yMH5SoffiZ X-Google-Smtp-Source: ABdhPJzHT1HFq4I0w+RhubZoAaV09KWqsVLq7rWIrToS+CHWUDkmjAgk3bLmkI5//fl0cMvtYjh7M3F+mo2dhV4IDFE= X-Received: by 2002:a17:907:20b0:: with SMTP id pw16mr6727699ejb.48.1621545564572; Thu, 20 May 2021 14:19:24 -0700 (PDT) MIME-Version: 1.0 References: <CAEm3rU=P9r4U6ZK+y60CUh9-dVoVPvjgQ5-a1Euq83Ftbh7_Qw@mail.gmail.com> <20210520135418.21ee4d11@hermes.local> In-Reply-To: <20210520135418.21ee4d11@hermes.local> From: Sudharshan Krishnakumar <sudhar.ckrishna@gmail.com> Date: Thu, 20 May 2021 14:19:13 -0700 Message-ID: <CAEm3rUnWPxkz7OsCx2-_NkAYYq5eGthFQX2WmffmVG9+Dbu_fA@mail.gmail.com> To: stephen@networkplumber.org Cc: users@dpdk.org Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Content-Filtered-By: Mailman/MimeDel 2.1.29 Subject: Re: [dpdk-users] access to different ports on the same NIC by 2 DPDK applications X-BeenThere: users@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK usage discussions <users.dpdk.org> List-Unsubscribe: <https://mails.dpdk.org/options/users>, <mailto:users-request@dpdk.org?subject=unsubscribe> List-Archive: <http://mails.dpdk.org/archives/users/> List-Post: <mailto:users@dpdk.org> List-Help: <mailto:users-request@dpdk.org?subject=help> List-Subscribe: <https://mails.dpdk.org/listinfo/users>, <mailto:users-request@dpdk.org?subject=subscribe> Errors-To: users-bounces@dpdk.org Sender: "users" <users-bounces@dpdk.org> Thanks for your inputs. So, does this mean, when ports are bound to VFIO driver(which uses IOMMU), we can run 2 DPDK apps concurrently, only if the 2 apps are accessing ports on 2 different Network cards. In other words, when ports are bound to VFIO driver, if 2 DPDK apps are running concurrently on the same system, then we need 2 Network cards installed on that system. So, I am guessing with 2 Network cards, under /dev, they would enumerate as 2 distinct paths, for example-> /dev/vfio/1 and /dev/vfio/2. Is this a correct understanding, please let me know. On Thu, May 20, 2021 at 1:54 PM Stephen Hemminger < stephen@networkplumber.org> wrote: > On Thu, 20 May 2021 13:34:35 -0700 > Sudharshan Krishnakumar <sudhar.ckrishna@gmail.com> wrote: > > > Hi All, > > > > Have a Qs related to-> access to different ports on the same NIC by 2 > DPDK > > applications. > > > > > > > > Setup: 4-Port 10 Gig NIC card with ports bound for DPDK usage. > > > > > > > > With intel=E2=80=99s igb_uio kernel driver (used for binding NIC=E2=80= =99s ports) using > > dpdk-setup.sh, at the /dev level, I see 4 device paths created, > > > > /dev/uio0, /dev/uio1, /dev/ui02, /dev/uio3 > > > > > > > > And I was able to run 2 DPDK applications concurrently(by blacklisting > the > > ports in EAL parameters), and they were able to take ownership of > different > > ports on the same NIC. > > > > For example DPDK_APP1 can take access of ports->0,1 and DPDK_APP2 can > take > > access of ports->2,3. > > > > > > > > Currently, I am using another DPDK app that relies on binding to VFIO > > driver. > > > > > > > > So I am trying to switch to binding NIC=E2=80=99s 4 ports to VFIO drive= r. > > > > > > > > Enabled VTIO in BIOS and added intel_iommu=3Don in kernel boot-param in > GRUB. > > > > > > > > After binding all 4 ports to VFIO driver, using dpdk-setup,sh, at the > /dev > > level, I only see these 2 device paths created-> /dev/vfio/vfio(control > > channel) > > > > and /dev/vfio/1(essentially just 1 path). > > > > > > > > I am able to run one DPDK application, for example-> PacketGen tool. > > > > But not able to run 2 DPDK applications concurrently accessing differen= t > > ports on the same NIC, due to DPDK application not being able to get > access > > to the ports. > > > > For example, if I launch DPDK_APP1 first, it can get access to the port= s, > > but DPDK_APP2 is NOT able to get access, and vice-versa. > > > > Note that both apps are trying to access different ports on the same NI= C, > > and also ports have been black-listed as part of EAL params to the DPDK > > apps. > > > > I know this is possible with igb_uio driver. > > > > > > > > Please let me know, if it is possible to do this when ports are bound t= o > > VFIO driver. > > > > Thanks, > > > > Sudha > > The issue is that the IOMMU can't map two different areas at once > to same device. You might be able to use DPDK primary/secondary process > model to do this. >