From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 2662EA0A02 for ; Thu, 20 May 2021 22:54:35 +0200 (CEST) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 9E63940143; Thu, 20 May 2021 22:54:34 +0200 (CEST) Received: from mail-pg1-f173.google.com (mail-pg1-f173.google.com [209.85.215.173]) by mails.dpdk.org (Postfix) with ESMTP id 35D1940041 for ; Thu, 20 May 2021 22:54:33 +0200 (CEST) Received: by mail-pg1-f173.google.com with SMTP id t30so12687565pgl.8 for ; Thu, 20 May 2021 13:54:33 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=networkplumber-org.20150623.gappssmtp.com; s=20150623; h=date:from:to:cc:subject:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=kIif3P+UojzjiryCiHbZ4rmvRqt2XyKgCqOqabvwxP0=; b=VZduogiI8r0exSZD9DCrmEB2Mi49oU9l0Shw7ZWDriaarHOTAC2ZT/IrPjf2PNEdAq QMOKr1gWgGL455Vodj7EWi2703ob/T+BgGBip/qNUViaWxV/MpKf6JcYsqrOM0vuNeiz AJBDt+LtsWL7ubKKYXcqfthcflRXmiiwDazujJjBHrlilHYJofUnxTMoHya3Sm1WAXox F9gJoSYdJ6iVx87f1yx5BYIzKiJB5tHnSvixytRIz56ZL80jFBbfDgEMuYBXSFx7Q/4b ih53eJiobdhq7zcHmMth450nIVSXWdvkhFyy8lvie2GPnSf9928Po0U9PLQEZuGMPL2T NcMg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:from:to:cc:subject:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=kIif3P+UojzjiryCiHbZ4rmvRqt2XyKgCqOqabvwxP0=; b=ar4A46RBaR4FxE3jTFJ3KNact216bsoYRoj7P2FpIVNjwCHL9MX4DwNhPmxE9EmUnD cOy27uKPhEU/A/GQcEyrDcvSa9CgI3XiRVI7n2oMewdeI48Bpe3x9SdWE9Rz88Uc0Bxp gDr6S2sNbYbLmxV0v4ngSVb1dHGX/Mwh4pGLaCmBeDOS4SZxOF1wL8jISoQ1PkDGxc5+ CXQehYjYdoBzqJ/4iOuUD4hdEL2VIf8qGZwJAQV3U604xZEwZUJqJzCiwNULXTe7TP2F ieWxANdu6WhZTyVQ5KLT+RUAAbAme7LIbKFeUSdGo21KabrfF5coXDn9uW99VVMWl0Xy Vgzg== X-Gm-Message-State: AOAM531HRHUCd0MZzL2U9JJVxznZy6nlMVpRcByoNU7z9T6VLou/TnKt o93/5d+8yo9b3C1m0mMqosQDBA== X-Google-Smtp-Source: ABdhPJzrMBSKWinCQcygCxSx3rIbxTgn6+AVbpgV2WWaYleqaYKa8L9uZ3aWF3ke9FFbCL4MD36AQQ== X-Received: by 2002:a63:ee10:: with SMTP id e16mr6428212pgi.135.1621544072247; Thu, 20 May 2021 13:54:32 -0700 (PDT) Received: from hermes.local (76-14-218-44.or.wavecable.com. [76.14.218.44]) by smtp.gmail.com with ESMTPSA id 23sm2101351pfn.192.2021.05.20.13.54.31 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 20 May 2021 13:54:31 -0700 (PDT) Date: Thu, 20 May 2021 13:54:18 -0700 From: Stephen Hemminger To: Sudharshan Krishnakumar Cc: users@dpdk.org Message-ID: <20210520135418.21ee4d11@hermes.local> In-Reply-To: References: MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable Subject: Re: [dpdk-users] access to different ports on the same NIC by 2 DPDK applications X-BeenThere: users@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK usage discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: users-bounces@dpdk.org Sender: "users" On Thu, 20 May 2021 13:34:35 -0700 Sudharshan Krishnakumar wrote: > Hi All, >=20 > Have a Qs related to-> access to different ports on the same NIC by 2 DPDK > applications. >=20 >=20 >=20 > Setup: 4-Port 10 Gig NIC card with ports bound for DPDK usage. >=20 >=20 >=20 > With intel=E2=80=99s igb_uio kernel driver (used for binding NIC=E2=80=99= s ports) using > dpdk-setup.sh, at the /dev level, I see 4 device paths created, >=20 > /dev/uio0, /dev/uio1, /dev/ui02, /dev/uio3 >=20 >=20 >=20 > And I was able to run 2 DPDK applications concurrently(by blacklisting the > ports in EAL parameters), and they were able to take ownership of differe= nt > ports on the same NIC. >=20 > For example DPDK_APP1 can take access of ports->0,1 and DPDK_APP2 can take > access of ports->2,3. >=20 >=20 >=20 > Currently, I am using another DPDK app that relies on binding to VFIO > driver. >=20 >=20 >=20 > So I am trying to switch to binding NIC=E2=80=99s 4 ports to VFIO driver. >=20 >=20 >=20 > Enabled VTIO in BIOS and added intel_iommu=3Don in kernel boot-param in G= RUB. >=20 >=20 >=20 > After binding all 4 ports to VFIO driver, using dpdk-setup,sh, at the /dev > level, I only see these 2 device paths created-> /dev/vfio/vfio(control > channel) >=20 > and /dev/vfio/1(essentially just 1 path). >=20 >=20 >=20 > I am able to run one DPDK application, for example-> PacketGen tool. >=20 > But not able to run 2 DPDK applications concurrently accessing different > ports on the same NIC, due to DPDK application not being able to get acce= ss > to the ports. >=20 > For example, if I launch DPDK_APP1 first, it can get access to the ports, > but DPDK_APP2 is NOT able to get access, and vice-versa. >=20 > Note that both apps are trying to access different ports on the same NIC, > and also ports have been black-listed as part of EAL params to the DPDK > apps. >=20 > I know this is possible with igb_uio driver. >=20 >=20 >=20 > Please let me know, if it is possible to do this when ports are bound to > VFIO driver. >=20 > Thanks, >=20 > Sudha The issue is that the IOMMU can't map two different areas at once to same device. You might be able to use DPDK primary/secondary process model to do this.