* Re: [dpdk-dev] [dpdk-users] Traffic doesn't forward on virtual devices [not found] ` <1706940680.3976131.1530805573796.JavaMail.zimbra@redhat.com> @ 2018-07-09 18:10 ` Aaron Conole 2018-07-10 8:20 ` Loftus, Ciara 0 siblings, 1 reply; 5+ messages in thread From: Aaron Conole @ 2018-07-09 18:10 UTC (permalink / raw) To: Bala Sankaran Cc: Keith Wiles, users, dev, Maxime Coquelin, Ian Stokes, Ciara Loftus Bala Sankaran <bsankara@redhat.com> writes: > Perfect! > > Thanks for the help. > > ----- Original Message ----- >> From: "Keith Wiles" <keith.wiles@intel.com> >> To: "Bala Sankaran" <bsankara@redhat.com> >> Cc: users@dpdk.org, "Aaron Conole" <aconole@redhat.com> >> Sent: Thursday, July 5, 2018 11:41:46 AM >> Subject: Re: [dpdk-users] Traffic doesn't forward on virtual devices >> >> >> >> > On Jul 5, 2018, at 9:53 AM, Bala Sankaran <bsankara@redhat.com> wrote: >> > >> > Greetings, >> > >> > I am currently using dpdk version 17.11.2. I see that there are a few other >> > revisions in 17.11.3, followed by the latest stable version of 18.02.2. >> > >> > Based on the issues I have faced so far (see Original >> > Message below), would you suggest that I go for >> > another version? If yes, which one? In essence, my question is, would >> > resorting to a different version of dpdk solve my current issue of >> > virtqueue id being invalid? >> > >> > Any help is much appreciated. >> >> From a support perspective using the latest version 18.05 or the long term >> supported version 17.11.3 is easier for most to help. I would pick the >> latest release 18.05 myself. As for fixing this problem I do not know. You >> can look into the MAINTAINERS file and find the maintainers of area(s) and >> include them in the CC line on your questions as sometimes they miss the >> emails as the volume can be high at times. Thanks Keith. I took a quick look and it seems like the queues are not setting up correctly between OvS and testpmd? Probably there's a step missing somewhere, although nothing in either the netdev-dpdk.c from OvS nor the rte_ethdev was obvious to stand out to me. I've CC'd Maxime, Ian, and Ciara - maybe they have a better idea to try? >> > >> > Thanks >> > >> > ----- Original Message ----- >> >> From: "Bala Sankaran" <bsankara@redhat.com> >> >> To: users@dpdk.org >> >> Cc: "Aaron Conole" <aconole@redhat.com> >> >> Sent: Thursday, June 28, 2018 3:18:13 PM >> >> Subject: Traffic doesn't forward on virtual devices >> >> >> >> >> >> Hello team, >> >> >> >> I am working on a project to do PVP tests on dpdk. As a first step, I >> >> would >> >> like to get traffic flow between tap devices. I'm in process of setting up >> >> the architecture, in which I've used testpmd to forward traffic between >> >> two >> >> virtual devices(tap and vhost users) over a bridge. >> >> >> >> While I'm at it, I've identified that the internal dev_attached flag never >> >> gets set to 1 from the rte_eth_vhost.c file. I've tried to manually set it >> >> to 1 in the start routine, but I just see that the queue index being >> >> referenced is out of range. >> >> >> >> I'm not sure how to proceed. Has anyone had luck using testpmd to >> >> communicate with vhost-user devices? If yes, any hints on a workaround? >> >> >> >> Here's how I configured my setup after installing dpdk and openvswitch: >> >> >> >> 1. To start ovs-ctl: >> >> /usr/local/share/openvswitch/scripts/ovs-ctl start >> >> >> >> 2. Setup hugepages: >> >> echo '2048' > /proc/sys/vm/nr_hugepages >> >> >> >> 3. Add a new network namespace: >> >> ip netns add ns1 >> >> >> >> 4. Add and set a bridge: >> >> ovs-vsctl add-br dpdkbr0 -- set Bridge dpdkbr0 datapath_type=netdev >> >> options:vhost-server-path=/usr/local/var/run/openvswitch/vhu0 >> >> ovs-vsctl show >> >> >> >> 5. Add a vhost user to the bridge created: >> >> ovs-vsctl add-port dpdkbr0 vhu0 -- set Interface vhu0 >> >> type=dpdkvhostuserclient >> >> >> >> 6. Execute bash on the network namespace: >> >> ip netns exec ns1 bash >> >> >> >> 7. Use testpmd and connect the namespaces: >> >> testpmd --socket-mem=512 >> >> --vdev='eth_vhost0,iface=/usr/local/var/run/openvswitch/vhu0,queues=1' >> >> --vdev='net_tap0,iface=tap0' --file-prefix page0 -- -i >> >> >> >> >> >> I repeated steps 3 - 7 for another network namespace on the same bridge. >> >> Following this, in fresh terminals, I assigned IP addresses to the tap >> >> devices created and tried pinging them. From port statistics, >> >> I identified the above mentioned issue with the dev_attached and queue >> >> statistics. >> >> >> >> I would greatly appreciate any help from your end. >> >> >> >> Thanks. >> >> >> >> ------------------------------------------------- >> >> Bala Sankaran >> >> Networking Services Intern >> >> Red Hat Inc ., >> >> >> > ------------------------------------------------- >> > Bala Sankaran >> > Networking Services Intern >> >> Regards, >> Keith >> >> > > -------------------------------------------------- > Bala Sankaran > Networking Services Intern > Red Hat Inc ., ^ permalink raw reply [flat|nested] 5+ messages in thread
* Re: [dpdk-dev] [dpdk-users] Traffic doesn't forward on virtual devices 2018-07-09 18:10 ` [dpdk-dev] [dpdk-users] Traffic doesn't forward on virtual devices Aaron Conole @ 2018-07-10 8:20 ` Loftus, Ciara 2018-07-10 17:37 ` Bala Sankaran 0 siblings, 1 reply; 5+ messages in thread From: Loftus, Ciara @ 2018-07-10 8:20 UTC (permalink / raw) To: Aaron Conole, Bala Sankaran Cc: Wiles, Keith, users, dev, Maxime Coquelin, Stokes, Ian > > Bala Sankaran <bsankara@redhat.com> writes: > > > Perfect! > > > > Thanks for the help. > > > > ----- Original Message ----- > >> From: "Keith Wiles" <keith.wiles@intel.com> > >> To: "Bala Sankaran" <bsankara@redhat.com> > >> Cc: users@dpdk.org, "Aaron Conole" <aconole@redhat.com> > >> Sent: Thursday, July 5, 2018 11:41:46 AM > >> Subject: Re: [dpdk-users] Traffic doesn't forward on virtual devices > >> > >> > >> > >> > On Jul 5, 2018, at 9:53 AM, Bala Sankaran <bsankara@redhat.com> > wrote: > >> > > >> > Greetings, > >> > > >> > I am currently using dpdk version 17.11.2. I see that there are a few > other > >> > revisions in 17.11.3, followed by the latest stable version of 18.02.2. > >> > > >> > Based on the issues I have faced so far (see Original > >> > Message below), would you suggest that I go for > >> > another version? If yes, which one? In essence, my question is, would > >> > resorting to a different version of dpdk solve my current issue of > >> > virtqueue id being invalid? > >> > > >> > Any help is much appreciated. > >> > >> From a support perspective using the latest version 18.05 or the long term > >> supported version 17.11.3 is easier for most to help. I would pick the > >> latest release 18.05 myself. As for fixing this problem I do not know. You > >> can look into the MAINTAINERS file and find the maintainers of area(s) > and > >> include them in the CC line on your questions as sometimes they miss the > >> emails as the volume can be high at times. > > Thanks Keith. > > I took a quick look and it seems like the queues are not setting up > correctly between OvS and testpmd? Probably there's a step missing > somewhere, although nothing in either the netdev-dpdk.c from OvS nor the > rte_ethdev was obvious to stand out to me. > > I've CC'd Maxime, Ian, and Ciara - maybe they have a better idea to try? Hi, I think the appropriate driver to use in this test on the test-pmd side might be virtio-user. Follow the same steps just change your vdev test-pmd arguments to: --vdev='net_virtio_user0,path=/usr/local/var/run/openvswitch/vhu0' Thanks, Ciara > > >> > > >> > Thanks > >> > > >> > ----- Original Message ----- > >> >> From: "Bala Sankaran" <bsankara@redhat.com> > >> >> To: users@dpdk.org > >> >> Cc: "Aaron Conole" <aconole@redhat.com> > >> >> Sent: Thursday, June 28, 2018 3:18:13 PM > >> >> Subject: Traffic doesn't forward on virtual devices > >> >> > >> >> > >> >> Hello team, > >> >> > >> >> I am working on a project to do PVP tests on dpdk. As a first step, I > >> >> would > >> >> like to get traffic flow between tap devices. I'm in process of setting up > >> >> the architecture, in which I've used testpmd to forward traffic > between > >> >> two > >> >> virtual devices(tap and vhost users) over a bridge. > >> >> > >> >> While I'm at it, I've identified that the internal dev_attached flag never > >> >> gets set to 1 from the rte_eth_vhost.c file. I've tried to manually set it > >> >> to 1 in the start routine, but I just see that the queue index being > >> >> referenced is out of range. > >> >> > >> >> I'm not sure how to proceed. Has anyone had luck using testpmd to > >> >> communicate with vhost-user devices? If yes, any hints on a > workaround? > >> >> > >> >> Here's how I configured my setup after installing dpdk and > openvswitch: > >> >> > >> >> 1. To start ovs-ctl: > >> >> /usr/local/share/openvswitch/scripts/ovs-ctl start > >> >> > >> >> 2. Setup hugepages: > >> >> echo '2048' > /proc/sys/vm/nr_hugepages > >> >> > >> >> 3. Add a new network namespace: > >> >> ip netns add ns1 > >> >> > >> >> 4. Add and set a bridge: > >> >> ovs-vsctl add-br dpdkbr0 -- set Bridge dpdkbr0 datapath_type=netdev > >> >> options:vhost-server-path=/usr/local/var/run/openvswitch/vhu0 > >> >> ovs-vsctl show > >> >> > >> >> 5. Add a vhost user to the bridge created: > >> >> ovs-vsctl add-port dpdkbr0 vhu0 -- set Interface vhu0 > >> >> type=dpdkvhostuserclient > >> >> > >> >> 6. Execute bash on the network namespace: > >> >> ip netns exec ns1 bash > >> >> > >> >> 7. Use testpmd and connect the namespaces: > >> >> testpmd --socket-mem=512 > >> >> -- > vdev='eth_vhost0,iface=/usr/local/var/run/openvswitch/vhu0,queues=1' > >> >> --vdev='net_tap0,iface=tap0' --file-prefix page0 -- -i > >> >> > >> >> > >> >> I repeated steps 3 - 7 for another network namespace on the same > bridge. > >> >> Following this, in fresh terminals, I assigned IP addresses to the tap > >> >> devices created and tried pinging them. From port statistics, > >> >> I identified the above mentioned issue with the dev_attached and > queue > >> >> statistics. > >> >> > >> >> I would greatly appreciate any help from your end. > >> >> > >> >> Thanks. > >> >> > >> >> ------------------------------------------------- > >> >> Bala Sankaran > >> >> Networking Services Intern > >> >> Red Hat Inc ., > >> >> > >> > ------------------------------------------------- > >> > Bala Sankaran > >> > Networking Services Intern > >> > >> Regards, > >> Keith > >> > >> > > > > -------------------------------------------------- > > Bala Sankaran > > Networking Services Intern > > Red Hat Inc ., ^ permalink raw reply [flat|nested] 5+ messages in thread
* Re: [dpdk-dev] [dpdk-users] Traffic doesn't forward on virtual devices 2018-07-10 8:20 ` Loftus, Ciara @ 2018-07-10 17:37 ` Bala Sankaran 2018-07-11 8:12 ` Loftus, Ciara 0 siblings, 1 reply; 5+ messages in thread From: Bala Sankaran @ 2018-07-10 17:37 UTC (permalink / raw) To: Ciara Loftus Cc: Aaron Conole, Keith Wiles, users, dev, Maxime Coquelin, Ian Stokes ----- Original Message ----- > From: "Ciara Loftus" <ciara.loftus@intel.com> > To: "Aaron Conole" <aconole@redhat.com>, "Bala Sankaran" <bsankara@redhat.com> > Cc: "Keith Wiles" <keith.wiles@intel.com>, users@dpdk.org, dev@dpdk.org, "Maxime Coquelin" > <maxime.coquelin@redhat.com>, "Ian Stokes" <ian.stokes@intel.com> > Sent: Tuesday, July 10, 2018 4:20:31 AM > Subject: RE: [dpdk-users] Traffic doesn't forward on virtual devices > > > > > Bala Sankaran <bsankara@redhat.com> writes: > > > > > Perfect! > > > > > > Thanks for the help. > > > > > > ----- Original Message ----- > > >> From: "Keith Wiles" <keith.wiles@intel.com> > > >> To: "Bala Sankaran" <bsankara@redhat.com> > > >> Cc: users@dpdk.org, "Aaron Conole" <aconole@redhat.com> > > >> Sent: Thursday, July 5, 2018 11:41:46 AM > > >> Subject: Re: [dpdk-users] Traffic doesn't forward on virtual devices > > >> > > >> > > >> > > >> > On Jul 5, 2018, at 9:53 AM, Bala Sankaran <bsankara@redhat.com> > > wrote: > > >> > > > >> > Greetings, > > >> > > > >> > I am currently using dpdk version 17.11.2. I see that there are a few > > other > > >> > revisions in 17.11.3, followed by the latest stable version of > > >> > 18.02.2. > > >> > > > >> > Based on the issues I have faced so far (see Original > > >> > Message below), would you suggest that I go for > > >> > another version? If yes, which one? In essence, my question is, would > > >> > resorting to a different version of dpdk solve my current issue of > > >> > virtqueue id being invalid? > > >> > > > >> > Any help is much appreciated. > > >> > > >> From a support perspective using the latest version 18.05 or the long > > >> term > > >> supported version 17.11.3 is easier for most to help. I would pick the > > >> latest release 18.05 myself. As for fixing this problem I do not know. > > >> You > > >> can look into the MAINTAINERS file and find the maintainers of area(s) > > and > > >> include them in the CC line on your questions as sometimes they miss the > > >> emails as the volume can be high at times. > > > > Thanks Keith. > > > > I took a quick look and it seems like the queues are not setting up > > correctly between OvS and testpmd? Probably there's a step missing > > somewhere, although nothing in either the netdev-dpdk.c from OvS nor the > > rte_ethdev was obvious to stand out to me. > > > > I've CC'd Maxime, Ian, and Ciara - maybe they have a better idea to try? > > Hi, > > I think the appropriate driver to use in this test on the test-pmd side might > be virtio-user. > Follow the same steps just change your vdev test-pmd arguments to: > --vdev='net_virtio_user0,path=/usr/local/var/run/openvswitch/vhu0' > > Thanks, > Ciara > Thank you for your response. I tried using virtio-user, but I face an error that says: Failed to prepare memory for vhost-user. The command I ran is as below: [root@localhost openvswitch]# testpmd --socket-mem=1024 --vdev='net_virtio_user1,path=/usr/local/var/run/openvswitch/vhu1,server=1' --vdev='net_tap1,iface=tap1' --file-prefix page1 -- -i EAL: Detected 4 lcore(s) EAL: Detected 1 NUMA nodes EAL: Multi-process socket /var/run/dpdk/page1/mp_socket EAL: Probing VFIO support... EAL: WARNING: cpu flags constant_tsc=yes nonstop_tsc=no -> using unreliable clock cycles ! rte_pmd_tap_probe(): Initializing pmd_tap for net_tap1 as tap1 Interactive-mode selected Warning: NUMA should be configured manually by using --port-numa-config and --ring-numa-config parameters along with --numa. testpmd: create a new mbuf pool <mbuf_pool_socket_0>: n=171456, size=2176, socket=0 testpmd: preferred mempool ops selected: ring_mp_mc Configuring Port 0 (socket 0) virtio_user_server_reconnect(): WARNING: Some features 0x1801 are not supported by vhost-user! get_hugepage_file_info(): Exceed maximum of 8 prepare_vhost_memory_user(): Failed to prepare memory for vhost-user Port 0: DA:60:01:0C:4B:29 Configuring Port 1 (socket 0) Port 1: D2:5A:94:68:AF:B3 Checking link statuses... Port 0: LSC event Done I tried increasing the socket-memory, I checked /proc/meminfo and found there were over 1280 free hugepages. So my understanding is that this is not an issue where I don't have enough hugepages. Can you provide leads on what's wrong here? > > > > >> > > > >> > Thanks > > >> > > > >> > ----- Original Message ----- > > >> >> From: "Bala Sankaran" <bsankara@redhat.com> > > >> >> To: users@dpdk.org > > >> >> Cc: "Aaron Conole" <aconole@redhat.com> > > >> >> Sent: Thursday, June 28, 2018 3:18:13 PM > > >> >> Subject: Traffic doesn't forward on virtual devices > > >> >> > > >> >> > > >> >> Hello team, > > >> >> > > >> >> I am working on a project to do PVP tests on dpdk. As a first step, I > > >> >> would > > >> >> like to get traffic flow between tap devices. I'm in process of > > >> >> setting up > > >> >> the architecture, in which I've used testpmd to forward traffic > > between > > >> >> two > > >> >> virtual devices(tap and vhost users) over a bridge. > > >> >> > > >> >> While I'm at it, I've identified that the internal dev_attached flag > > >> >> never > > >> >> gets set to 1 from the rte_eth_vhost.c file. I've tried to manually > > >> >> set it > > >> >> to 1 in the start routine, but I just see that the queue index being > > >> >> referenced is out of range. > > >> >> > > >> >> I'm not sure how to proceed. Has anyone had luck using testpmd to > > >> >> communicate with vhost-user devices? If yes, any hints on a > > workaround? > > >> >> > > >> >> Here's how I configured my setup after installing dpdk and > > openvswitch: > > >> >> > > >> >> 1. To start ovs-ctl: > > >> >> /usr/local/share/openvswitch/scripts/ovs-ctl start > > >> >> > > >> >> 2. Setup hugepages: > > >> >> echo '2048' > /proc/sys/vm/nr_hugepages > > >> >> > > >> >> 3. Add a new network namespace: > > >> >> ip netns add ns1 > > >> >> > > >> >> 4. Add and set a bridge: > > >> >> ovs-vsctl add-br dpdkbr0 -- set Bridge dpdkbr0 datapath_type=netdev > > >> >> options:vhost-server-path=/usr/local/var/run/openvswitch/vhu0 > > >> >> ovs-vsctl show > > >> >> > > >> >> 5. Add a vhost user to the bridge created: > > >> >> ovs-vsctl add-port dpdkbr0 vhu0 -- set Interface vhu0 > > >> >> type=dpdkvhostuserclient > > >> >> > > >> >> 6. Execute bash on the network namespace: > > >> >> ip netns exec ns1 bash > > >> >> > > >> >> 7. Use testpmd and connect the namespaces: > > >> >> testpmd --socket-mem=512 > > >> >> -- > > vdev='eth_vhost0,iface=/usr/local/var/run/openvswitch/vhu0,queues=1' > > >> >> --vdev='net_tap0,iface=tap0' --file-prefix page0 -- -i > > >> >> > > >> >> > > >> >> I repeated steps 3 - 7 for another network namespace on the same > > bridge. > > >> >> Following this, in fresh terminals, I assigned IP addresses to the > > >> >> tap > > >> >> devices created and tried pinging them. From port statistics, > > >> >> I identified the above mentioned issue with the dev_attached and > > queue > > >> >> statistics. > > >> >> > > >> >> I would greatly appreciate any help from your end. > > >> >> > > >> >> Thanks. > > >> >> > > >> >> ------------------------------------------------- > > >> >> Bala Sankaran > > >> >> Networking Services Intern > > >> >> Red Hat Inc ., > > >> >> > > >> > ------------------------------------------------- > > >> > Bala Sankaran > > >> > Networking Services Intern > > >> > > >> Regards, > > >> Keith > > >> > > >> > > > > > > -------------------------------------------------- > > > Bala Sankaran > > > Networking Services Intern > > > Red Hat Inc ., > Thanks. Bala. ^ permalink raw reply [flat|nested] 5+ messages in thread
* Re: [dpdk-dev] [dpdk-users] Traffic doesn't forward on virtual devices 2018-07-10 17:37 ` Bala Sankaran @ 2018-07-11 8:12 ` Loftus, Ciara 2018-07-11 14:28 ` Bala Sankaran 0 siblings, 1 reply; 5+ messages in thread From: Loftus, Ciara @ 2018-07-11 8:12 UTC (permalink / raw) To: Bala Sankaran Cc: Aaron Conole, Wiles, Keith, users, dev, Maxime Coquelin, Stokes, Ian > > > > > > Bala Sankaran <bsankara@redhat.com> writes: > > > > > > > Perfect! > > > > > > > > Thanks for the help. > > > > > > > > ----- Original Message ----- > > > >> From: "Keith Wiles" <keith.wiles@intel.com> > > > >> To: "Bala Sankaran" <bsankara@redhat.com> > > > >> Cc: users@dpdk.org, "Aaron Conole" <aconole@redhat.com> > > > >> Sent: Thursday, July 5, 2018 11:41:46 AM > > > >> Subject: Re: [dpdk-users] Traffic doesn't forward on virtual devices > > > >> > > > >> > > > >> > > > >> > On Jul 5, 2018, at 9:53 AM, Bala Sankaran <bsankara@redhat.com> > > > wrote: > > > >> > > > > >> > Greetings, > > > >> > > > > >> > I am currently using dpdk version 17.11.2. I see that there are a few > > > other > > > >> > revisions in 17.11.3, followed by the latest stable version of > > > >> > 18.02.2. > > > >> > > > > >> > Based on the issues I have faced so far (see Original > > > >> > Message below), would you suggest that I go for > > > >> > another version? If yes, which one? In essence, my question is, > would > > > >> > resorting to a different version of dpdk solve my current issue of > > > >> > virtqueue id being invalid? > > > >> > > > > >> > Any help is much appreciated. > > > >> > > > >> From a support perspective using the latest version 18.05 or the long > > > >> term > > > >> supported version 17.11.3 is easier for most to help. I would pick the > > > >> latest release 18.05 myself. As for fixing this problem I do not know. > > > >> You > > > >> can look into the MAINTAINERS file and find the maintainers of area(s) > > > and > > > >> include them in the CC line on your questions as sometimes they miss > the > > > >> emails as the volume can be high at times. > > > > > > Thanks Keith. > > > > > > I took a quick look and it seems like the queues are not setting up > > > correctly between OvS and testpmd? Probably there's a step missing > > > somewhere, although nothing in either the netdev-dpdk.c from OvS nor > the > > > rte_ethdev was obvious to stand out to me. > > > > > > I've CC'd Maxime, Ian, and Ciara - maybe they have a better idea to try? > > > > Hi, > > > > I think the appropriate driver to use in this test on the test-pmd side might > > be virtio-user. > > Follow the same steps just change your vdev test-pmd arguments to: > > --vdev='net_virtio_user0,path=/usr/local/var/run/openvswitch/vhu0' > > > > Thanks, > > Ciara > > > > Thank you for your response. > > I tried using virtio-user, but I face an error that says: Failed to prepare > memory for vhost-user. > The command I ran is as below: > > [root@localhost openvswitch]# testpmd --socket-mem=1024 -- > vdev='net_virtio_user1,path=/usr/local/var/run/openvswitch/vhu1,server= > 1' --vdev='net_tap1,iface=tap1' --file-prefix page1 -- -i > EAL: Detected 4 lcore(s) > EAL: Detected 1 NUMA nodes > EAL: Multi-process socket /var/run/dpdk/page1/mp_socket > EAL: Probing VFIO support... > EAL: WARNING: cpu flags constant_tsc=yes nonstop_tsc=no -> using > unreliable clock cycles ! > rte_pmd_tap_probe(): Initializing pmd_tap for net_tap1 as tap1 > Interactive-mode selected > Warning: NUMA should be configured manually by using --port-numa-config > and --ring-numa-config parameters along with --numa. > testpmd: create a new mbuf pool <mbuf_pool_socket_0>: n=171456, > size=2176, socket=0 > testpmd: preferred mempool ops selected: ring_mp_mc > Configuring Port 0 (socket 0) > virtio_user_server_reconnect(): WARNING: Some features 0x1801 are not > supported by vhost-user! > get_hugepage_file_info(): Exceed maximum of 8 > prepare_vhost_memory_user(): Failed to prepare memory for vhost-user > Port 0: DA:60:01:0C:4B:29 > Configuring Port 1 (socket 0) > Port 1: D2:5A:94:68:AF:B3 > Checking link statuses... > > Port 0: LSC event > Done > > I tried increasing the socket-memory, I checked /proc/meminfo and found > there were over > 1280 free hugepages. > So my understanding is that this is not an issue where I don't have enough > hugepages. > > Can you provide leads on what's wrong here? Hi, The limitations section for Virtio User https://doc.dpdk.org/guides/howto/virtio_user_for_container_networking.html#limitations states that: " Cannot work when there are more than VHOST_MEMORY_MAX_NREGIONS(8) hugepages. If you have more regions (especially when 2MB hugepages are used), the option, --single-file-segments, can help to reduce the number of shared files." Suggest using the --single-file-segments option on the test-pmd command line or failing that increasing your hugepage size to 1G https://doc.dpdk.org/guides/linux_gsg/sys_reqs.html#use-of-hugepages-in-the-linux-environment Thanks, Ciara > > > > > > > >> > > > > >> > Thanks > > > >> > > > > >> > ----- Original Message ----- > > > >> >> From: "Bala Sankaran" <bsankara@redhat.com> > > > >> >> To: users@dpdk.org > > > >> >> Cc: "Aaron Conole" <aconole@redhat.com> > > > >> >> Sent: Thursday, June 28, 2018 3:18:13 PM > > > >> >> Subject: Traffic doesn't forward on virtual devices > > > >> >> > > > >> >> > > > >> >> Hello team, > > > >> >> > > > >> >> I am working on a project to do PVP tests on dpdk. As a first step, I > > > >> >> would > > > >> >> like to get traffic flow between tap devices. I'm in process of > > > >> >> setting up > > > >> >> the architecture, in which I've used testpmd to forward traffic > > > between > > > >> >> two > > > >> >> virtual devices(tap and vhost users) over a bridge. > > > >> >> > > > >> >> While I'm at it, I've identified that the internal dev_attached flag > > > >> >> never > > > >> >> gets set to 1 from the rte_eth_vhost.c file. I've tried to manually > > > >> >> set it > > > >> >> to 1 in the start routine, but I just see that the queue index being > > > >> >> referenced is out of range. > > > >> >> > > > >> >> I'm not sure how to proceed. Has anyone had luck using testpmd > to > > > >> >> communicate with vhost-user devices? If yes, any hints on a > > > workaround? > > > >> >> > > > >> >> Here's how I configured my setup after installing dpdk and > > > openvswitch: > > > >> >> > > > >> >> 1. To start ovs-ctl: > > > >> >> /usr/local/share/openvswitch/scripts/ovs-ctl start > > > >> >> > > > >> >> 2. Setup hugepages: > > > >> >> echo '2048' > /proc/sys/vm/nr_hugepages > > > >> >> > > > >> >> 3. Add a new network namespace: > > > >> >> ip netns add ns1 > > > >> >> > > > >> >> 4. Add and set a bridge: > > > >> >> ovs-vsctl add-br dpdkbr0 -- set Bridge dpdkbr0 > datapath_type=netdev > > > >> >> options:vhost-server-path=/usr/local/var/run/openvswitch/vhu0 > > > >> >> ovs-vsctl show > > > >> >> > > > >> >> 5. Add a vhost user to the bridge created: > > > >> >> ovs-vsctl add-port dpdkbr0 vhu0 -- set Interface vhu0 > > > >> >> type=dpdkvhostuserclient > > > >> >> > > > >> >> 6. Execute bash on the network namespace: > > > >> >> ip netns exec ns1 bash > > > >> >> > > > >> >> 7. Use testpmd and connect the namespaces: > > > >> >> testpmd --socket-mem=512 > > > >> >> -- > > > > vdev='eth_vhost0,iface=/usr/local/var/run/openvswitch/vhu0,queues=1' > > > >> >> --vdev='net_tap0,iface=tap0' --file-prefix page0 -- -i > > > >> >> > > > >> >> > > > >> >> I repeated steps 3 - 7 for another network namespace on the same > > > bridge. > > > >> >> Following this, in fresh terminals, I assigned IP addresses to the > > > >> >> tap > > > >> >> devices created and tried pinging them. From port statistics, > > > >> >> I identified the above mentioned issue with the dev_attached and > > > queue > > > >> >> statistics. > > > >> >> > > > >> >> I would greatly appreciate any help from your end. > > > >> >> > > > >> >> Thanks. > > > >> >> > > > >> >> ------------------------------------------------- > > > >> >> Bala Sankaran > > > >> >> Networking Services Intern > > > >> >> Red Hat Inc ., > > > >> >> > > > >> > ------------------------------------------------- > > > >> > Bala Sankaran > > > >> > Networking Services Intern > > > >> > > > >> Regards, > > > >> Keith > > > >> > > > >> > > > > > > > > -------------------------------------------------- > > > > Bala Sankaran > > > > Networking Services Intern > > > > Red Hat Inc ., > > > > Thanks. > Bala. ^ permalink raw reply [flat|nested] 5+ messages in thread
* Re: [dpdk-dev] [dpdk-users] Traffic doesn't forward on virtual devices 2018-07-11 8:12 ` Loftus, Ciara @ 2018-07-11 14:28 ` Bala Sankaran 0 siblings, 0 replies; 5+ messages in thread From: Bala Sankaran @ 2018-07-11 14:28 UTC (permalink / raw) To: Ciara Loftus Cc: Aaron Conole, Keith Wiles, users, dev, Maxime Coquelin, Ian Stokes ----- Original Message ----- > From: "Ciara Loftus" <ciara.loftus@intel.com> > To: "Bala Sankaran" <bsankara@redhat.com> > Cc: "Aaron Conole" <aconole@redhat.com>, "Keith Wiles" <keith.wiles@intel.com>, users@dpdk.org, dev@dpdk.org, "Maxime > Coquelin" <maxime.coquelin@redhat.com>, "Ian Stokes" <ian.stokes@intel.com> > Sent: Wednesday, July 11, 2018 4:12:38 AM > Subject: RE: [dpdk-users] Traffic doesn't forward on virtual devices > > > > > > > > > Bala Sankaran <bsankara@redhat.com> writes: > > > > > > > > > Perfect! > > > > > > > > > > Thanks for the help. > > > > > > > > > > ----- Original Message ----- > > > > >> From: "Keith Wiles" <keith.wiles@intel.com> > > > > >> To: "Bala Sankaran" <bsankara@redhat.com> > > > > >> Cc: users@dpdk.org, "Aaron Conole" <aconole@redhat.com> > > > > >> Sent: Thursday, July 5, 2018 11:41:46 AM > > > > >> Subject: Re: [dpdk-users] Traffic doesn't forward on virtual devices > > > > >> > > > > >> > > > > >> > > > > >> > On Jul 5, 2018, at 9:53 AM, Bala Sankaran <bsankara@redhat.com> > > > > wrote: > > > > >> > > > > > >> > Greetings, > > > > >> > > > > > >> > I am currently using dpdk version 17.11.2. I see that there are a > > > > >> > few > > > > other > > > > >> > revisions in 17.11.3, followed by the latest stable version of > > > > >> > 18.02.2. > > > > >> > > > > > >> > Based on the issues I have faced so far (see Original > > > > >> > Message below), would you suggest that I go for > > > > >> > another version? If yes, which one? In essence, my question is, > > would > > > > >> > resorting to a different version of dpdk solve my current issue of > > > > >> > virtqueue id being invalid? > > > > >> > > > > > >> > Any help is much appreciated. > > > > >> > > > > >> From a support perspective using the latest version 18.05 or the > > > > >> long > > > > >> term > > > > >> supported version 17.11.3 is easier for most to help. I would pick > > > > >> the > > > > >> latest release 18.05 myself. As for fixing this problem I do not > > > > >> know. > > > > >> You > > > > >> can look into the MAINTAINERS file and find the maintainers of > > > > >> area(s) > > > > and > > > > >> include them in the CC line on your questions as sometimes they miss > > the > > > > >> emails as the volume can be high at times. > > > > > > > > Thanks Keith. > > > > > > > > I took a quick look and it seems like the queues are not setting up > > > > correctly between OvS and testpmd? Probably there's a step missing > > > > somewhere, although nothing in either the netdev-dpdk.c from OvS nor > > the > > > > rte_ethdev was obvious to stand out to me. > > > > > > > > I've CC'd Maxime, Ian, and Ciara - maybe they have a better idea to > > > > try? > > > > > > Hi, > > > > > > I think the appropriate driver to use in this test on the test-pmd side > > > might > > > be virtio-user. > > > Follow the same steps just change your vdev test-pmd arguments to: > > > --vdev='net_virtio_user0,path=/usr/local/var/run/openvswitch/vhu0' > > > > > > Thanks, > > > Ciara > > > > > > > Thank you for your response. > > > > I tried using virtio-user, but I face an error that says: Failed to prepare > > memory for vhost-user. > > The command I ran is as below: > > > > [root@localhost openvswitch]# testpmd --socket-mem=1024 -- > > vdev='net_virtio_user1,path=/usr/local/var/run/openvswitch/vhu1,server= > > 1' --vdev='net_tap1,iface=tap1' --file-prefix page1 -- -i > > EAL: Detected 4 lcore(s) > > EAL: Detected 1 NUMA nodes > > EAL: Multi-process socket /var/run/dpdk/page1/mp_socket > > EAL: Probing VFIO support... > > EAL: WARNING: cpu flags constant_tsc=yes nonstop_tsc=no -> using > > unreliable clock cycles ! > > rte_pmd_tap_probe(): Initializing pmd_tap for net_tap1 as tap1 > > Interactive-mode selected > > Warning: NUMA should be configured manually by using --port-numa-config > > and --ring-numa-config parameters along with --numa. > > testpmd: create a new mbuf pool <mbuf_pool_socket_0>: n=171456, > > size=2176, socket=0 > > testpmd: preferred mempool ops selected: ring_mp_mc > > Configuring Port 0 (socket 0) > > virtio_user_server_reconnect(): WARNING: Some features 0x1801 are not > > supported by vhost-user! > > get_hugepage_file_info(): Exceed maximum of 8 > > prepare_vhost_memory_user(): Failed to prepare memory for vhost-user > > Port 0: DA:60:01:0C:4B:29 > > Configuring Port 1 (socket 0) > > Port 1: D2:5A:94:68:AF:B3 > > Checking link statuses... > > > > Port 0: LSC event > > Done > > > > I tried increasing the socket-memory, I checked /proc/meminfo and found > > there were over > > 1280 free hugepages. > > So my understanding is that this is not an issue where I don't have enough > > hugepages. > > > > Can you provide leads on what's wrong here? > > Hi, > > The limitations section for Virtio User > https://doc.dpdk.org/guides/howto/virtio_user_for_container_networking.html#limitations > states that: > > " Cannot work when there are more than VHOST_MEMORY_MAX_NREGIONS(8) > hugepages. If you have more regions (especially when 2MB hugepages are > used), the option, --single-file-segments, can help to reduce the number of > shared files." > > Suggest using the --single-file-segments option on the test-pmd command line > or failing that increasing your hugepage size to 1G > https://doc.dpdk.org/guides/linux_gsg/sys_reqs.html#use-of-hugepages-in-the-linux-environment > > Thanks, > Ciara Thanks a ton. Looks like I had just missed out on adding the single-file-segments argument. After addition to my existing command structure, testpmd is setup and further, I am able to ping the tap devices. Thanks Bala. > > > > > > > > > > > >> > > > > > >> > Thanks > > > > >> > > > > > >> > ----- Original Message ----- > > > > >> >> From: "Bala Sankaran" <bsankara@redhat.com> > > > > >> >> To: users@dpdk.org > > > > >> >> Cc: "Aaron Conole" <aconole@redhat.com> > > > > >> >> Sent: Thursday, June 28, 2018 3:18:13 PM > > > > >> >> Subject: Traffic doesn't forward on virtual devices > > > > >> >> > > > > >> >> > > > > >> >> Hello team, > > > > >> >> > > > > >> >> I am working on a project to do PVP tests on dpdk. As a first > > > > >> >> step, I > > > > >> >> would > > > > >> >> like to get traffic flow between tap devices. I'm in process of > > > > >> >> setting up > > > > >> >> the architecture, in which I've used testpmd to forward traffic > > > > between > > > > >> >> two > > > > >> >> virtual devices(tap and vhost users) over a bridge. > > > > >> >> > > > > >> >> While I'm at it, I've identified that the internal dev_attached > > > > >> >> flag > > > > >> >> never > > > > >> >> gets set to 1 from the rte_eth_vhost.c file. I've tried to > > > > >> >> manually > > > > >> >> set it > > > > >> >> to 1 in the start routine, but I just see that the queue index > > > > >> >> being > > > > >> >> referenced is out of range. > > > > >> >> > > > > >> >> I'm not sure how to proceed. Has anyone had luck using testpmd > > to > > > > >> >> communicate with vhost-user devices? If yes, any hints on a > > > > workaround? > > > > >> >> > > > > >> >> Here's how I configured my setup after installing dpdk and > > > > openvswitch: > > > > >> >> > > > > >> >> 1. To start ovs-ctl: > > > > >> >> /usr/local/share/openvswitch/scripts/ovs-ctl start > > > > >> >> > > > > >> >> 2. Setup hugepages: > > > > >> >> echo '2048' > /proc/sys/vm/nr_hugepages > > > > >> >> > > > > >> >> 3. Add a new network namespace: > > > > >> >> ip netns add ns1 > > > > >> >> > > > > >> >> 4. Add and set a bridge: > > > > >> >> ovs-vsctl add-br dpdkbr0 -- set Bridge dpdkbr0 > > datapath_type=netdev > > > > >> >> options:vhost-server-path=/usr/local/var/run/openvswitch/vhu0 > > > > >> >> ovs-vsctl show > > > > >> >> > > > > >> >> 5. Add a vhost user to the bridge created: > > > > >> >> ovs-vsctl add-port dpdkbr0 vhu0 -- set Interface vhu0 > > > > >> >> type=dpdkvhostuserclient > > > > >> >> > > > > >> >> 6. Execute bash on the network namespace: > > > > >> >> ip netns exec ns1 bash > > > > >> >> > > > > >> >> 7. Use testpmd and connect the namespaces: > > > > >> >> testpmd --socket-mem=512 > > > > >> >> -- > > > > > > vdev='eth_vhost0,iface=/usr/local/var/run/openvswitch/vhu0,queues=1' > > > > >> >> --vdev='net_tap0,iface=tap0' --file-prefix page0 -- -i > > > > >> >> > > > > >> >> > > > > >> >> I repeated steps 3 - 7 for another network namespace on the same > > > > bridge. > > > > >> >> Following this, in fresh terminals, I assigned IP addresses to > > > > >> >> the > > > > >> >> tap > > > > >> >> devices created and tried pinging them. From port statistics, > > > > >> >> I identified the above mentioned issue with the dev_attached and > > > > queue > > > > >> >> statistics. > > > > >> >> > > > > >> >> I would greatly appreciate any help from your end. > > > > >> >> > > > > >> >> Thanks. > > > > >> >> > > > > >> >> ------------------------------------------------- > > > > >> >> Bala Sankaran > > > > >> >> Networking Services Intern > > > > >> >> Red Hat Inc ., > > > > >> >> > > > > >> > ------------------------------------------------- > > > > >> > Bala Sankaran > > > > >> > Networking Services Intern > > > > >> > > > > >> Regards, > > > > >> Keith > > > > >> > > > > >> > > > > > > > > > > -------------------------------------------------- > > > > > Bala Sankaran > > > > > Networking Services Intern > > > > > Red Hat Inc ., > > > > > > > Thanks. > > Bala. > ^ permalink raw reply [flat|nested] 5+ messages in thread
end of thread, other threads:[~2018-07-11 14:28 UTC | newest] Thread overview: 5+ messages (download: mbox.gz / follow: Atom feed) -- links below jump to the message on this page -- [not found] <1882345510.2557424.1530213493803.JavaMail.zimbra@redhat.com> [not found] ` <329369015.3963546.1530802400841.JavaMail.zimbra@redhat.com> [not found] ` <FA078466-7060-44E9-908A-D696272FC83B@intel.com> [not found] ` <1706940680.3976131.1530805573796.JavaMail.zimbra@redhat.com> 2018-07-09 18:10 ` [dpdk-dev] [dpdk-users] Traffic doesn't forward on virtual devices Aaron Conole 2018-07-10 8:20 ` Loftus, Ciara 2018-07-10 17:37 ` Bala Sankaran 2018-07-11 8:12 ` Loftus, Ciara 2018-07-11 14:28 ` Bala Sankaran
This is a public inbox, see mirroring instructions for how to clone and mirror all data and code used for this inbox; as well as URLs for NNTP newsgroup(s).