DPDK usage discussions
 help / color / mirror / Atom feed
From: David Christensen <drc@linux.vnet.ibm.com>
To: fwefew 4t4tg <7532yahoo@gmail.com>, users@dpdk.org
Subject: Re: If or how one gets an IP address associated with a vfio-pci bound NIC
Date: Wed, 3 Nov 2021 11:11:28 -0700
Message-ID: <0aa050c0-fa95-a4cd-bb58-ba3dfef8146d@linux.vnet.ibm.com> (raw)
In-Reply-To: <CA+Tq66WLO09=y3U5RjyvU5wvnH0fMySgb5jKaRSxhu3PZpvobQ@mail.gmail.com>



On 11/2/21 4:14 PM, fwefew 4t4tg wrote:
> I'm trying to use DPDK on AWS i3.metal instances. I have the code built 
> with AWS vfio-patches. In order to be logged into the machine on one NIC 
> while having a free ENA NIC for DPDK, I attached a second NIC. 
> ./dpdk-devbind.py is able to ZjQcmQRYFpfptBannerStart
> This Message Is From an External Sender
> This message came from outside your organization.
> ZjQcmQRYFpfptBannerEnd
> 
> I'm trying to use DPDK on AWS i3.metal instances. I have the code built 
> with AWS vfio-patches. In order to be logged into the machine on one NIC 
> while having a free ENA NIC for DPDK, I attached a second NIC.
> 
> ./dpdk-devbind.py is able to see the second NIC, and bind to it. *All 
> that's working fine. However, by default this 2nd NIC does not have an 
> IP address.*
> 
> Meanwhile code needs a hostname or IP address of the client and server. 
> How do I get an IP address associated with this 2nd NIC? 

I don't think you understand the intent behind the DPDK framework. 
You're passing control of the NIC to a user application.  That means you 
don't receive any benefits of the kernel's networking stack.  The user 
application you use will need to handle all network services, including 
it's own TCP/IP stack if required.

If you're using the bundled DPDK testpmd application then there's no 
need to assign an IP address to the interface.  The testpmd app can 
build and send/receive ANY type of network packet, though it's mostly 
only used to verify functionality provided by the DPDK framework.  If 
you're WRITING a network application then DPDK might be what you want, 
but if you have a specific network function in mind then you're likely 
looking for an application that USES DPDK.

And do I need
> to do some sys-admin work to ensure traffic in and out of the DPDK bound 
> vfio-pci NIC is kept separate from the first NIC?
> 
> As far as I can see the correct approach is to:
> 
> # setup second NIC to have an IP address and make sure UP before 
> dpdk-devbind:
> * sudo ip addr add <some-valid-ipv4-addr> dev ens1 label ens1:1
> * sudo ip lin set ens1 u[
> 
> before I do DPDK bind.
> 
> The NIC, when AWS adds it, starts off down without an IP address by default:
> 
> ubuntu$ lspci | grep Ether
> 
> 04:00.0 Ethernet controller: Amazon.com, Inc. Elastic Network Adapter (ENA)
> 05:00.0 Ethernet controller: Amazon.com, Inc. Elastic Network Adapter (ENA)
> ubuntu$ sudo ip a
> 1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN 
> group default qlen 1000
>      link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
>      inet 127.0.0.1/8 <http://127.0.0.1/8> scope host lo
>         valid_lft forever preferred_lft forever
>      inet6 ::1/128 scope host
>         valid_lft forever preferred_lft forever
> 2: ens785: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 9001 qdisc mq state UP 
> group default qlen 1000
>      link/ether 0a:0f:1f:db:ca:73 brd ff:ff:ff:ff:ff:ff
>      inet 172.31.17.144/20 <http://172.31.17.144/20> brd 172.31.31.255 
> scope global dynamic ens785
>         valid_lft 3544sec preferred_lft 3544sec
>      inet6 fe80::80f:1fff:fedb:ca73/64 scope link
>         valid_lft forever preferred_lft forever
> *3: ens1: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN group 
> default qlen 1000
>      link/ether 0a:06:15:14:95:05 brd ff:ff:ff:ff:ff:ff
> *
> Once I bind 'ens1' dpdk-devbind reports it as bound -AND- it no longer 
> appears in `ip a`:

This is expected.  You've removed the NIC from the kernel's control and 
bound it to the vfio_pci driver, which allows the NIC to be controlled 
entirely by a user application.

> 
> Network devices using DPDK-compatible driver
> ============================================
> 0000:05:00.0 'Elastic Network Adapter (ENA) ec20' drv=vfio-pci unused=ena
> 
> Network devices using kernel driver
> ===================================
> 0000:04:00.0 'Elastic Network Adapter (ENA) ec20' if=ens785 drv=ena 
> unused=vfio-pci *Active*
> 
> $ ip a
> ubuntu@ip-172-31-17-144:~/Scripts$ ip a
> 1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN 
> group default qlen 1000
>      link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
>      inet 127.0.0.1/8 <http://127.0.0.1/8> scope host lo
>         valid_lft forever preferred_lft forever
>      inet6 ::1/128 scope host
>         valid_lft forever preferred_lft forever
> 2: ens785: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 9001 qdisc mq state UP 
> group default qlen 1000
>      link/ether 0a:0f:1f:db:ca:73 brd ff:ff:ff:ff:ff:ff
>      inet 172.31.17.144/20 <http://172.31.17.144/20> brd 172.31.31.255 
> scope global dynamic ens785
>         valid_lft 3314sec preferred_lft 3314sec
>      inet6 fe80::80f:1fff:fedb:ca73/64 scope link
>         valid_lft forever preferred_lft forever

Everything seems in order here.  If you can share what you're trying to 
accomplish with DPDK we might be able to provide better guidance.

Dave

  reply	other threads:[~2021-11-03 18:11 UTC|newest]

Thread overview: 4+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2021-11-02 23:14 fwefew 4t4tg
2021-11-03 18:11 ` David Christensen [this message]
2021-11-04  1:40   ` fwefew 4t4tg
2021-11-04 17:14     ` David Christensen

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=0aa050c0-fa95-a4cd-bb58-ba3dfef8146d@linux.vnet.ibm.com \
    --to=drc@linux.vnet.ibm.com \
    --cc=7532yahoo@gmail.com \
    --cc=users@dpdk.org \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link

DPDK usage discussions

This inbox may be cloned and mirrored by anyone:

	git clone --mirror http://inbox.dpdk.org/users/0 users/git/0.git

	# If you have public-inbox 1.1+ installed, you may
	# initialize and index your mirror using the following commands:
	public-inbox-init -V2 users users/ http://inbox.dpdk.org/users \
		users@dpdk.org
	public-inbox-index users

Example config snippet for mirrors.
Newsgroup available over NNTP:
	nntp://inbox.dpdk.org/inbox.dpdk.users


AGPL code for this site: git clone https://public-inbox.org/public-inbox.git