From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 7D734A0C4B for ; Wed, 3 Nov 2021 00:14:53 +0100 (CET) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 5327740E03; Wed, 3 Nov 2021 00:14:53 +0100 (CET) Received: from mail-io1-f47.google.com (mail-io1-f47.google.com [209.85.166.47]) by mails.dpdk.org (Postfix) with ESMTP id B123340689 for ; Wed, 3 Nov 2021 00:14:52 +0100 (CET) Received: by mail-io1-f47.google.com with SMTP id g8so606493iob.10 for ; Tue, 02 Nov 2021 16:14:52 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=mime-version:from:date:message-id:subject:to; bh=czigUzu9UfG5e9Ne9U4UatjjImuTD1u+8DxEC0Cigsk=; b=depdcdjsLi14uFzYBbSfgD51wO6B1Li6kV3PjL6y5yIRd5lKoC8E2QW1NTPtuaCWiF SIUYpC2onyQ5z7jaHYtyC7RLZNqOrhIFY+DBlaC77CNtcmr+c3499AkkDknStefFp4Jb fVRlu8GDYuuRTqS+l1upapO7jHySqDVLpJTuPtteOjm7agqKC5AhxtT95VGzlTenVNeH szEuo+tuHd4+wIr5wStIdG3e1c1TPg5wN3rw7ENT5S2jaN6AmIEndXpw1FFDCt1Qm9jz jNzuxm1hRpqGVL9GXu6JTJSPlrDmFp9SDW46hEj0zFPpaFrlI5PcA6rVy/dcvMk9VKOo xvOA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:mime-version:from:date:message-id:subject:to; bh=czigUzu9UfG5e9Ne9U4UatjjImuTD1u+8DxEC0Cigsk=; b=SDJ2P2n7Eb5XYX37g7X8zt3ocZMBSvmGTjPjQeeLfh0cvqPtISdzPR57EloJnjSxdi ZwK/EauXxHMCTdl+e0IE4vCsltvDuXtRGQuLKrUVpkJ4mKbzR3KHkbovqLSfPUuftycA WjZyQK9X+vfels0znq5ziY1FiaCNoTYr7YTxqpQXThQLvEFhHRKi7F7hsEJTAfgg/DP2 HDAmeJ6S63yzslhf49bW4pODqbO+VmcWSIJspRU24eB5GStD7qWEyoosq/gpiaFopi8n E8aF0WVQVsn4OtqCpgs3DuafyAPWctLg9cqybz3QXxb6rlG4YLVy0a6m255oakrrYaQT n5oA== X-Gm-Message-State: AOAM5307lT6Jm+hE42WXHLb5a5IoUSOrVQ4EBJpldq038TIgtfvCVGRU SqozhSt1Lt48l5jf268e4AJj0gB/rbmpJuEZLQMLlaUtZ0o= X-Google-Smtp-Source: ABdhPJzVj7zdYhgCJwKI0awSGQcLd0d9bKG0bXIEgdh3g8wxiXJRet6r/XdR+Q4W4SXXwzV2l1oxS++aYmlyCLIG15I= X-Received: by 2002:a02:a884:: with SMTP id l4mr29672313jam.44.1635894891851; Tue, 02 Nov 2021 16:14:51 -0700 (PDT) MIME-Version: 1.0 From: fwefew 4t4tg <7532yahoo@gmail.com> Date: Tue, 2 Nov 2021 19:14:41 -0400 Message-ID: Subject: If or how one gets an IP address associated with a vfio-pci bound NIC To: users@dpdk.org Content-Type: multipart/alternative; boundary="000000000000eb310405cfd67578" X-BeenThere: users@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK usage discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: users-bounces@dpdk.org --000000000000eb310405cfd67578 Content-Type: text/plain; charset="UTF-8" I'm trying to use DPDK on AWS i3.metal instances. I have the code built with AWS vfio-patches. In order to be logged into the machine on one NIC while having a free ENA NIC for DPDK, I attached a second NIC. ./dpdk-devbind.py is able to see the second NIC, and bind to it. *All that's working fine. However, by default this 2nd NIC does not have an IP address.* Meanwhile code needs a hostname or IP address of the client and server. How do I get an IP address associated with this 2nd NIC? And do I need to do some sys-admin work to ensure traffic in and out of the DPDK bound vfio-pci NIC is kept separate from the first NIC? As far as I can see the correct approach is to: # setup second NIC to have an IP address and make sure UP before dpdk-devbind: * sudo ip addr add dev ens1 label ens1:1 * sudo ip lin set ens1 u[ before I do DPDK bind. The NIC, when AWS adds it, starts off down without an IP address by default: ubuntu$ lspci | grep Ether 04:00.0 Ethernet controller: Amazon.com, Inc. Elastic Network Adapter (ENA) 05:00.0 Ethernet controller: Amazon.com, Inc. Elastic Network Adapter (ENA) ubuntu$ sudo ip a 1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 inet 127.0.0.1/8 scope host lo valid_lft forever preferred_lft forever inet6 ::1/128 scope host valid_lft forever preferred_lft forever 2: ens785: mtu 9001 qdisc mq state UP group default qlen 1000 link/ether 0a:0f:1f:db:ca:73 brd ff:ff:ff:ff:ff:ff inet 172.31.17.144/20 brd 172.31.31.255 scope global dynamic ens785 valid_lft 3544sec preferred_lft 3544sec inet6 fe80::80f:1fff:fedb:ca73/64 scope link valid_lft forever preferred_lft forever *3: ens1: mtu 1500 qdisc noop state DOWN group default qlen 1000 link/ether 0a:06:15:14:95:05 brd ff:ff:ff:ff:ff:ff* Once I bind 'ens1' dpdk-devbind reports it as bound -AND- it no longer appears in `ip a`: Network devices using DPDK-compatible driver ============================================ 0000:05:00.0 'Elastic Network Adapter (ENA) ec20' drv=vfio-pci unused=ena Network devices using kernel driver =================================== 0000:04:00.0 'Elastic Network Adapter (ENA) ec20' if=ens785 drv=ena unused=vfio-pci *Active* $ ip a ubuntu@ip-172-31-17-144:~/Scripts$ ip a 1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 inet 127.0.0.1/8 scope host lo valid_lft forever preferred_lft forever inet6 ::1/128 scope host valid_lft forever preferred_lft forever 2: ens785: mtu 9001 qdisc mq state UP group default qlen 1000 link/ether 0a:0f:1f:db:ca:73 brd ff:ff:ff:ff:ff:ff inet 172.31.17.144/20 brd 172.31.31.255 scope global dynamic ens785 valid_lft 3314sec preferred_lft 3314sec inet6 fe80::80f:1fff:fedb:ca73/64 scope link valid_lft forever preferred_lft forever --000000000000eb310405cfd67578 Content-Type: text/html; charset="UTF-8" Content-Transfer-Encoding: quoted-printable

I'm trying to use DPDK= on AWS=C2=A0i3.metal instances. I have the code built with AWS vfio-patche= s. In order to be logged into the machine on one NIC while having a free EN= A NIC for DPDK, I attached a second NIC.

./dpdk-devbind.py is able t= o see the second NIC, and bind to it.=C2=A0All that's working fine. = However, by default this 2nd NIC does not have an IP address.

Meanwhile code need= s a hostname or IP address of the client and server. How do I get an IP add= ress associated with this 2nd NIC? And do I need to do some sys-admin work = to ensure traffic=C2=A0in and out of the DPDK bound vfio-pci NIC is kept se= parate from the first NIC?

As far as I can see the correct approach= is to:

# setup second NIC to have an IP address and make sure UP be= fore dpdk-devbind:
*=C2=A0
sudo ip addr add <some-valid-ipv4-ad= dr> dev ens1 label ens1:1
* sudo ip = lin set ens1 u[

before I do DPDK bind.

The NIC, when AWS=C2= =A0adds it, starts off down without an IP address by default:

ubuntu$ lspci | grep E= ther

04:00.0 Ethernet controller: Amazon.com, Inc. Elastic Networ= k Adapter (ENA)
05:00.0 Ethernet controller: Amazon.com, Inc. Elastic Ne= twork Adapter (ENA)
ubuntu$ sudo ip a
1: lo: <LOOPBACK,UP,LOWER_UP= > mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000
=C2= =A0 =C2=A0 link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
=C2=A0 = =C2=A0 inet 127.0.0.1/8 scope host lo=C2=A0 =C2=A0 =C2=A0 =C2=A0valid_lft forever preferred_lft forever
=C2= =A0 =C2=A0 inet6 ::1/128 scope host
=C2=A0 =C2=A0 =C2=A0 =C2=A0valid_lf= t forever preferred_lft forever
2: ens785: <BROADCAST,MULTICAST,UP,LO= WER_UP> mtu 9001 qdisc mq state UP group default qlen 1000
=C2=A0 =C2= =A0 link/ether 0a:0f:1f:db:ca:73 brd ff:ff:ff:ff:ff:ff
=C2=A0 =C2=A0 ine= t 172.31.17.144/20 brd 172.31.31.25= 5 scope global dynamic ens785
=C2=A0 =C2=A0 =C2=A0 =C2=A0valid_lft 3544s= ec preferred_lft 3544sec
=C2=A0 =C2=A0 inet6 fe80::80f:1fff:fedb:ca73/64= scope link
=C2=A0 =C2=A0 =C2=A0 =C2=A0valid_lft forever preferred_lft = forever
3: ens1: <BROADCAST,MULTICAST> mtu 1500 qdisc noop stat= e DOWN group default qlen 1000
=C2=A0 =C2=A0 link/ether 0a:06:15:14:95:0= 5 brd ff:ff:ff:ff:ff:ff

Once I bind 'ens1' dpdk-devbind = reports it as bound -AND- it no longer appears in `ip a`:

Network de= vices using DPDK-compatible driver
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D
0000:05:00.0 'Elastic Network Adapter (ENA) ec= 20' drv=3Dvfio-pci unused=3Dena

Network devices using kernel dri= ver
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D
0000:04:00.0 'Elastic Networ= k Adapter (ENA) ec20' if=3Dens785 drv=3Dena unused=3Dvfio-pci *Active*<= br>
$ ip a
ubuntu@ip-172-31-17-144:~/Scripts$ ip a
1: lo: <LOOP= BACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default ql= en 1000
=C2=A0 =C2=A0 link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00= :00
=C2=A0 =C2=A0 inet 127.0.0.1/8 sc= ope host lo
=C2=A0 =C2=A0 =C2=A0 =C2=A0valid_lft forever preferred_lft f= orever
=C2=A0 =C2=A0 inet6 ::1/128 scope host
=C2=A0 =C2=A0 =C2=A0 = =C2=A0valid_lft forever preferred_lft forever
2: ens785: <BROADCAST,M= ULTICAST,UP,LOWER_UP> mtu 9001 qdisc mq state UP group default qlen 1000=
=C2=A0 =C2=A0 link/ether 0a:0f:1f:db:ca:73 brd ff:ff:ff:ff:ff:ff
=C2= =A0 =C2=A0 inet 172.31.17.144/20 br= d 172.31.31.255 scope global dynamic ens785
=C2=A0 =C2=A0 =C2=A0 =C2=A0v= alid_lft 3314sec preferred_lft 3314sec
=C2=A0 =C2=A0 inet6 fe80::80f:1ff= f:fedb:ca73/64 scope link
=C2=A0 =C2=A0 =C2=A0 =C2=A0valid_lft forever = preferred_lft forever
--000000000000eb310405cfd67578--