From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mx1.redhat.com (mx1.redhat.com [209.132.183.28]) by dpdk.org (Postfix) with ESMTP id 791501B506; Wed, 11 Jul 2018 16:28:30 +0200 (CEST) Received: from smtp.corp.redhat.com (int-mx02.intmail.prod.int.phx2.redhat.com [10.5.11.12]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id C1CDE3078AB1; Wed, 11 Jul 2018 14:28:29 +0000 (UTC) Received: from colo-mx.corp.redhat.com (colo-mx01.intmail.prod.int.phx2.redhat.com [10.5.11.20]) by smtp.corp.redhat.com (Postfix) with ESMTPS id E1DBB5FCAB; Wed, 11 Jul 2018 14:28:27 +0000 (UTC) Received: from zmail18.collab.prod.int.phx2.redhat.com (zmail18.collab.prod.int.phx2.redhat.com [10.5.83.21]) by colo-mx.corp.redhat.com (Postfix) with ESMTP id 4D2F318037ED; Wed, 11 Jul 2018 14:28:27 +0000 (UTC) Date: Wed, 11 Jul 2018 10:28:26 -0400 (EDT) From: Bala Sankaran To: Ciara Loftus Cc: Aaron Conole , Keith Wiles , users@dpdk.org, dev@dpdk.org, Maxime Coquelin , Ian Stokes Message-ID: <2107272165.5096308.1531319306822.JavaMail.zimbra@redhat.com> In-Reply-To: <74F120C019F4A64C9B78E802F6AD4CC278FCF6E2@IRSMSX106.ger.corp.intel.com> References: <1882345510.2557424.1530213493803.JavaMail.zimbra@redhat.com> <329369015.3963546.1530802400841.JavaMail.zimbra@redhat.com> <1706940680.3976131.1530805573796.JavaMail.zimbra@redhat.com> <74F120C019F4A64C9B78E802F6AD4CC278FC9F12@IRSMSX106.ger.corp.intel.com> <537677734.4883876.1531244244731.JavaMail.zimbra@redhat.com> <74F120C019F4A64C9B78E802F6AD4CC278FCF6E2@IRSMSX106.ger.corp.intel.com> MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit X-Originating-IP: [10.18.25.217, 10.4.195.24] Thread-Topic: [dpdk-users] Traffic doesn't forward on virtual devices Thread-Index: AQHUF7Amdnv0lRkeoUCXZ2BXFt7UrKSIHHTA3XUGOq2ijIrgIGx3qpcA X-Scanned-By: MIMEDefang 2.79 on 10.5.11.12 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.48]); Wed, 11 Jul 2018 14:28:29 +0000 (UTC) Subject: Re: [dpdk-dev] [dpdk-users] Traffic doesn't forward on virtual devices X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 11 Jul 2018 14:28:30 -0000 ----- Original Message ----- > From: "Ciara Loftus" > To: "Bala Sankaran" > Cc: "Aaron Conole" , "Keith Wiles" , users@dpdk.org, dev@dpdk.org, "Maxime > Coquelin" , "Ian Stokes" > Sent: Wednesday, July 11, 2018 4:12:38 AM > Subject: RE: [dpdk-users] Traffic doesn't forward on virtual devices > > > > > > > > > Bala Sankaran writes: > > > > > > > > > Perfect! > > > > > > > > > > Thanks for the help. > > > > > > > > > > ----- Original Message ----- > > > > >> From: "Keith Wiles" > > > > >> To: "Bala Sankaran" > > > > >> Cc: users@dpdk.org, "Aaron Conole" > > > > >> Sent: Thursday, July 5, 2018 11:41:46 AM > > > > >> Subject: Re: [dpdk-users] Traffic doesn't forward on virtual devices > > > > >> > > > > >> > > > > >> > > > > >> > On Jul 5, 2018, at 9:53 AM, Bala Sankaran > > > > wrote: > > > > >> > > > > > >> > Greetings, > > > > >> > > > > > >> > I am currently using dpdk version 17.11.2. I see that there are a > > > > >> > few > > > > other > > > > >> > revisions in 17.11.3, followed by the latest stable version of > > > > >> > 18.02.2. > > > > >> > > > > > >> > Based on the issues I have faced so far (see Original > > > > >> > Message below), would you suggest that I go for > > > > >> > another version? If yes, which one? In essence, my question is, > > would > > > > >> > resorting to a different version of dpdk solve my current issue of > > > > >> > virtqueue id being invalid? > > > > >> > > > > > >> > Any help is much appreciated. > > > > >> > > > > >> From a support perspective using the latest version 18.05 or the > > > > >> long > > > > >> term > > > > >> supported version 17.11.3 is easier for most to help. I would pick > > > > >> the > > > > >> latest release 18.05 myself. As for fixing this problem I do not > > > > >> know. > > > > >> You > > > > >> can look into the MAINTAINERS file and find the maintainers of > > > > >> area(s) > > > > and > > > > >> include them in the CC line on your questions as sometimes they miss > > the > > > > >> emails as the volume can be high at times. > > > > > > > > Thanks Keith. > > > > > > > > I took a quick look and it seems like the queues are not setting up > > > > correctly between OvS and testpmd? Probably there's a step missing > > > > somewhere, although nothing in either the netdev-dpdk.c from OvS nor > > the > > > > rte_ethdev was obvious to stand out to me. > > > > > > > > I've CC'd Maxime, Ian, and Ciara - maybe they have a better idea to > > > > try? > > > > > > Hi, > > > > > > I think the appropriate driver to use in this test on the test-pmd side > > > might > > > be virtio-user. > > > Follow the same steps just change your vdev test-pmd arguments to: > > > --vdev='net_virtio_user0,path=/usr/local/var/run/openvswitch/vhu0' > > > > > > Thanks, > > > Ciara > > > > > > > Thank you for your response. > > > > I tried using virtio-user, but I face an error that says: Failed to prepare > > memory for vhost-user. > > The command I ran is as below: > > > > [root@localhost openvswitch]# testpmd --socket-mem=1024 -- > > vdev='net_virtio_user1,path=/usr/local/var/run/openvswitch/vhu1,server= > > 1' --vdev='net_tap1,iface=tap1' --file-prefix page1 -- -i > > EAL: Detected 4 lcore(s) > > EAL: Detected 1 NUMA nodes > > EAL: Multi-process socket /var/run/dpdk/page1/mp_socket > > EAL: Probing VFIO support... > > EAL: WARNING: cpu flags constant_tsc=yes nonstop_tsc=no -> using > > unreliable clock cycles ! > > rte_pmd_tap_probe(): Initializing pmd_tap for net_tap1 as tap1 > > Interactive-mode selected > > Warning: NUMA should be configured manually by using --port-numa-config > > and --ring-numa-config parameters along with --numa. > > testpmd: create a new mbuf pool : n=171456, > > size=2176, socket=0 > > testpmd: preferred mempool ops selected: ring_mp_mc > > Configuring Port 0 (socket 0) > > virtio_user_server_reconnect(): WARNING: Some features 0x1801 are not > > supported by vhost-user! > > get_hugepage_file_info(): Exceed maximum of 8 > > prepare_vhost_memory_user(): Failed to prepare memory for vhost-user > > Port 0: DA:60:01:0C:4B:29 > > Configuring Port 1 (socket 0) > > Port 1: D2:5A:94:68:AF:B3 > > Checking link statuses... > > > > Port 0: LSC event > > Done > > > > I tried increasing the socket-memory, I checked /proc/meminfo and found > > there were over > > 1280 free hugepages. > > So my understanding is that this is not an issue where I don't have enough > > hugepages. > > > > Can you provide leads on what's wrong here? > > Hi, > > The limitations section for Virtio User > https://doc.dpdk.org/guides/howto/virtio_user_for_container_networking.html#limitations > states that: > > " Cannot work when there are more than VHOST_MEMORY_MAX_NREGIONS(8) > hugepages. If you have more regions (especially when 2MB hugepages are > used), the option, --single-file-segments, can help to reduce the number of > shared files." > > Suggest using the --single-file-segments option on the test-pmd command line > or failing that increasing your hugepage size to 1G > https://doc.dpdk.org/guides/linux_gsg/sys_reqs.html#use-of-hugepages-in-the-linux-environment > > Thanks, > Ciara Thanks a ton. Looks like I had just missed out on adding the single-file-segments argument. After addition to my existing command structure, testpmd is setup and further, I am able to ping the tap devices. Thanks Bala. > > > > > > > > > > > >> > > > > > >> > Thanks > > > > >> > > > > > >> > ----- Original Message ----- > > > > >> >> From: "Bala Sankaran" > > > > >> >> To: users@dpdk.org > > > > >> >> Cc: "Aaron Conole" > > > > >> >> Sent: Thursday, June 28, 2018 3:18:13 PM > > > > >> >> Subject: Traffic doesn't forward on virtual devices > > > > >> >> > > > > >> >> > > > > >> >> Hello team, > > > > >> >> > > > > >> >> I am working on a project to do PVP tests on dpdk. As a first > > > > >> >> step, I > > > > >> >> would > > > > >> >> like to get traffic flow between tap devices. I'm in process of > > > > >> >> setting up > > > > >> >> the architecture, in which I've used testpmd to forward traffic > > > > between > > > > >> >> two > > > > >> >> virtual devices(tap and vhost users) over a bridge. > > > > >> >> > > > > >> >> While I'm at it, I've identified that the internal dev_attached > > > > >> >> flag > > > > >> >> never > > > > >> >> gets set to 1 from the rte_eth_vhost.c file. I've tried to > > > > >> >> manually > > > > >> >> set it > > > > >> >> to 1 in the start routine, but I just see that the queue index > > > > >> >> being > > > > >> >> referenced is out of range. > > > > >> >> > > > > >> >> I'm not sure how to proceed. Has anyone had luck using testpmd > > to > > > > >> >> communicate with vhost-user devices? If yes, any hints on a > > > > workaround? > > > > >> >> > > > > >> >> Here's how I configured my setup after installing dpdk and > > > > openvswitch: > > > > >> >> > > > > >> >> 1. To start ovs-ctl: > > > > >> >> /usr/local/share/openvswitch/scripts/ovs-ctl start > > > > >> >> > > > > >> >> 2. Setup hugepages: > > > > >> >> echo '2048' > /proc/sys/vm/nr_hugepages > > > > >> >> > > > > >> >> 3. Add a new network namespace: > > > > >> >> ip netns add ns1 > > > > >> >> > > > > >> >> 4. Add and set a bridge: > > > > >> >> ovs-vsctl add-br dpdkbr0 -- set Bridge dpdkbr0 > > datapath_type=netdev > > > > >> >> options:vhost-server-path=/usr/local/var/run/openvswitch/vhu0 > > > > >> >> ovs-vsctl show > > > > >> >> > > > > >> >> 5. Add a vhost user to the bridge created: > > > > >> >> ovs-vsctl add-port dpdkbr0 vhu0 -- set Interface vhu0 > > > > >> >> type=dpdkvhostuserclient > > > > >> >> > > > > >> >> 6. Execute bash on the network namespace: > > > > >> >> ip netns exec ns1 bash > > > > >> >> > > > > >> >> 7. Use testpmd and connect the namespaces: > > > > >> >> testpmd --socket-mem=512 > > > > >> >> -- > > > > > > vdev='eth_vhost0,iface=/usr/local/var/run/openvswitch/vhu0,queues=1' > > > > >> >> --vdev='net_tap0,iface=tap0' --file-prefix page0 -- -i > > > > >> >> > > > > >> >> > > > > >> >> I repeated steps 3 - 7 for another network namespace on the same > > > > bridge. > > > > >> >> Following this, in fresh terminals, I assigned IP addresses to > > > > >> >> the > > > > >> >> tap > > > > >> >> devices created and tried pinging them. From port statistics, > > > > >> >> I identified the above mentioned issue with the dev_attached and > > > > queue > > > > >> >> statistics. > > > > >> >> > > > > >> >> I would greatly appreciate any help from your end. > > > > >> >> > > > > >> >> Thanks. > > > > >> >> > > > > >> >> ------------------------------------------------- > > > > >> >> Bala Sankaran > > > > >> >> Networking Services Intern > > > > >> >> Red Hat Inc ., > > > > >> >> > > > > >> > ------------------------------------------------- > > > > >> > Bala Sankaran > > > > >> > Networking Services Intern > > > > >> > > > > >> Regards, > > > > >> Keith > > > > >> > > > > >> > > > > > > > > > > -------------------------------------------------- > > > > > Bala Sankaran > > > > > Networking Services Intern > > > > > Red Hat Inc ., > > > > > > > Thanks. > > Bala. >