[dpdk-users] Traffic doesn't forward on virtual devices

[dpdk-users] Traffic doesn't forward on virtual devices

[Bala Sankaran]

Hello team,

I am working on a project to do PVP tests on dpdk. As a first step, I would like to get traffic flow between tap devices. I'm in process of setting up the architecture, in which I've used testpmd to forward traffic between two virtual devices(tap and vhost users) over a bridge. 

While I'm at it, I've identified that the internal dev_attached flag never gets set to 1 from the rte_eth_vhost.c file. I've tried to manually set it to 1 in the start routine, but I just see that the queue index being referenced is out of range.
 
I'm not sure how to proceed.  Has anyone had luck using testpmd to communicate with vhost-user devices?  If yes, any hints on a workaround?

Here's how I configured my setup after installing dpdk and openvswitch:

1. To start ovs-ctl:
/usr/local/share/openvswitch/scripts/ovs-ctl start

2. Setup hugepages: 
echo '2048' > /proc/sys/vm/nr_hugepages

3. Add a new network namespace:
ip netns add ns1

4. Add and set a bridge:
ovs-vsctl add-br dpdkbr0 -- set Bridge dpdkbr0 datapath_type=netdev options:vhost-server-path=/usr/local/var/run/openvswitch/vhu0
ovs-vsctl show

5. Add a vhost user to the bridge created:
ovs-vsctl add-port dpdkbr0 vhu0 -- set Interface vhu0 type=dpdkvhostuserclient

6. Execute bash on the network namespace:
ip netns exec ns1 bash

7. Use testpmd and connect the namespaces:
testpmd --socket-mem=512 --vdev='eth_vhost0,iface=/usr/local/var/run/openvswitch/vhu0,queues=1'  --vdev='net_tap0,iface=tap0' --file-prefix page0 -- -i


I repeated steps 3 - 7 for another network namespace on the same bridge. Following this, in fresh terminals, I assigned IP addresses to the tap devices created and tried pinging them. From port statistics, 
I identified the above mentioned issue with the dev_attached and queue statistics.

I would greatly appreciate any help from your end.

Thanks. 

-------------------------------------------------
Bala Sankaran
Networking Services Intern
Red Hat Inc .,

Re: [dpdk-users] Traffic doesn't forward on virtual devices

[Bala Sankaran]

Greetings,

I am currently using dpdk version 17.11.2. I see that there are a few other 
revisions in 17.11.3, followed by the latest stable version of 18.02.2. 

Based on the issues I have faced so far (see Original 
Message below), would you suggest that  I go for 
another version? If yes, which one? In essence, my question is, would 
resorting to a different version of dpdk solve my current issue of 
virtqueue id being invalid? 

Any help is much appreciated.

Thanks

----- Original Message -----
> From: "Bala Sankaran" <bsankara@redhat.com>
> To: users@dpdk.org
> Cc: "Aaron Conole" <aconole@redhat.com>
> Sent: Thursday, June 28, 2018 3:18:13 PM
> Subject: Traffic doesn't forward on virtual devices
> 
> 
> Hello team,
> 
> I am working on a project to do PVP tests on dpdk. As a first step, I would
> like to get traffic flow between tap devices. I'm in process of setting up
> the architecture, in which I've used testpmd to forward traffic between two
> virtual devices(tap and vhost users) over a bridge.
> 
> While I'm at it, I've identified that the internal dev_attached flag never
> gets set to 1 from the rte_eth_vhost.c file. I've tried to manually set it
> to 1 in the start routine, but I just see that the queue index being
> referenced is out of range.
>  
> I'm not sure how to proceed.  Has anyone had luck using testpmd to
> communicate with vhost-user devices?  If yes, any hints on a workaround?
> 
> Here's how I configured my setup after installing dpdk and openvswitch:
> 
> 1. To start ovs-ctl:
> /usr/local/share/openvswitch/scripts/ovs-ctl start
> 
> 2. Setup hugepages:
> echo '2048' > /proc/sys/vm/nr_hugepages
> 
> 3. Add a new network namespace:
> ip netns add ns1
> 
> 4. Add and set a bridge:
> ovs-vsctl add-br dpdkbr0 -- set Bridge dpdkbr0 datapath_type=netdev
> options:vhost-server-path=/usr/local/var/run/openvswitch/vhu0
> ovs-vsctl show
> 
> 5. Add a vhost user to the bridge created:
> ovs-vsctl add-port dpdkbr0 vhu0 -- set Interface vhu0
> type=dpdkvhostuserclient
> 
> 6. Execute bash on the network namespace:
> ip netns exec ns1 bash
> 
> 7. Use testpmd and connect the namespaces:
> testpmd --socket-mem=512
> --vdev='eth_vhost0,iface=/usr/local/var/run/openvswitch/vhu0,queues=1'
> --vdev='net_tap0,iface=tap0' --file-prefix page0 -- -i
> 
> 
> I repeated steps 3 - 7 for another network namespace on the same bridge.
> Following this, in fresh terminals, I assigned IP addresses to the tap
> devices created and tried pinging them. From port statistics,
> I identified the above mentioned issue with the dev_attached and queue
> statistics.
> 
> I would greatly appreciate any help from your end.
> 
> Thanks.
> 
> -------------------------------------------------
> Bala Sankaran
> Networking Services Intern
> Red Hat Inc .,
> 
-------------------------------------------------
Bala Sankaran
Networking Services Intern

Re: [dpdk-users] Traffic doesn't forward on virtual devices

[Wiles, Keith]

> On Jul 5, 2018, at 9:53 AM, Bala Sankaran <bsankara@redhat.com> wrote:
> 
> Greetings,
> 
> I am currently using dpdk version 17.11.2. I see that there are a few other 
> revisions in 17.11.3, followed by the latest stable version of 18.02.2. 
> 
> Based on the issues I have faced so far (see Original 
> Message below), would you suggest that  I go for 
> another version? If yes, which one? In essence, my question is, would 
> resorting to a different version of dpdk solve my current issue of 
> virtqueue id being invalid? 
> 
> Any help is much appreciated.

>From a support perspective using the latest version 18.05 or the long term supported version 17.11.3 is easier for most to help. I would pick the latest release 18.05 myself. As for fixing this problem I do not know. You can look into the MAINTAINERS file and find the maintainers of area(s) and include them in the CC line on your questions as sometimes they miss the emails as the volume can be high at times.

> 
> Thanks
> 
> ----- Original Message -----
>> From: "Bala Sankaran" <bsankara@redhat.com>
>> To: users@dpdk.org
>> Cc: "Aaron Conole" <aconole@redhat.com>
>> Sent: Thursday, June 28, 2018 3:18:13 PM
>> Subject: Traffic doesn't forward on virtual devices
>> 
>> 
>> Hello team,
>> 
>> I am working on a project to do PVP tests on dpdk. As a first step, I would
>> like to get traffic flow between tap devices. I'm in process of setting up
>> the architecture, in which I've used testpmd to forward traffic between two
>> virtual devices(tap and vhost users) over a bridge.
>> 
>> While I'm at it, I've identified that the internal dev_attached flag never
>> gets set to 1 from the rte_eth_vhost.c file. I've tried to manually set it
>> to 1 in the start routine, but I just see that the queue index being
>> referenced is out of range.
>> 
>> I'm not sure how to proceed.  Has anyone had luck using testpmd to
>> communicate with vhost-user devices?  If yes, any hints on a workaround?
>> 
>> Here's how I configured my setup after installing dpdk and openvswitch:
>> 
>> 1. To start ovs-ctl:
>> /usr/local/share/openvswitch/scripts/ovs-ctl start
>> 
>> 2. Setup hugepages:
>> echo '2048' > /proc/sys/vm/nr_hugepages
>> 
>> 3. Add a new network namespace:
>> ip netns add ns1
>> 
>> 4. Add and set a bridge:
>> ovs-vsctl add-br dpdkbr0 -- set Bridge dpdkbr0 datapath_type=netdev
>> options:vhost-server-path=/usr/local/var/run/openvswitch/vhu0
>> ovs-vsctl show
>> 
>> 5. Add a vhost user to the bridge created:
>> ovs-vsctl add-port dpdkbr0 vhu0 -- set Interface vhu0
>> type=dpdkvhostuserclient
>> 
>> 6. Execute bash on the network namespace:
>> ip netns exec ns1 bash
>> 
>> 7. Use testpmd and connect the namespaces:
>> testpmd --socket-mem=512
>> --vdev='eth_vhost0,iface=/usr/local/var/run/openvswitch/vhu0,queues=1'
>> --vdev='net_tap0,iface=tap0' --file-prefix page0 -- -i
>> 
>> 
>> I repeated steps 3 - 7 for another network namespace on the same bridge.
>> Following this, in fresh terminals, I assigned IP addresses to the tap
>> devices created and tried pinging them. From port statistics,
>> I identified the above mentioned issue with the dev_attached and queue
>> statistics.
>> 
>> I would greatly appreciate any help from your end.
>> 
>> Thanks.
>> 
>> -------------------------------------------------
>> Bala Sankaran
>> Networking Services Intern
>> Red Hat Inc .,
>> 
> -------------------------------------------------
> Bala Sankaran
> Networking Services Intern

Regards,
Keith

Re: [dpdk-users] Traffic doesn't forward on virtual devices

[Bala Sankaran]

Perfect!

Thanks for the help.

----- Original Message -----
> From: "Keith Wiles" <keith.wiles@intel.com>
> To: "Bala Sankaran" <bsankara@redhat.com>
> Cc: users@dpdk.org, "Aaron Conole" <aconole@redhat.com>
> Sent: Thursday, July 5, 2018 11:41:46 AM
> Subject: Re: [dpdk-users] Traffic doesn't forward on virtual devices
> 
> 
> 
> > On Jul 5, 2018, at 9:53 AM, Bala Sankaran <bsankara@redhat.com> wrote:
> > 
> > Greetings,
> > 
> > I am currently using dpdk version 17.11.2. I see that there are a few other
> > revisions in 17.11.3, followed by the latest stable version of 18.02.2.
> > 
> > Based on the issues I have faced so far (see Original
> > Message below), would you suggest that  I go for
> > another version? If yes, which one? In essence, my question is, would
> > resorting to a different version of dpdk solve my current issue of
> > virtqueue id being invalid?
> > 
> > Any help is much appreciated.
> 
> From a support perspective using the latest version 18.05 or the long term
> supported version 17.11.3 is easier for most to help. I would pick the
> latest release 18.05 myself. As for fixing this problem I do not know. You
> can look into the MAINTAINERS file and find the maintainers of area(s) and
> include them in the CC line on your questions as sometimes they miss the
> emails as the volume can be high at times.
> 
> > 
> > Thanks
> > 
> > ----- Original Message -----
> >> From: "Bala Sankaran" <bsankara@redhat.com>
> >> To: users@dpdk.org
> >> Cc: "Aaron Conole" <aconole@redhat.com>
> >> Sent: Thursday, June 28, 2018 3:18:13 PM
> >> Subject: Traffic doesn't forward on virtual devices
> >> 
> >> 
> >> Hello team,
> >> 
> >> I am working on a project to do PVP tests on dpdk. As a first step, I
> >> would
> >> like to get traffic flow between tap devices. I'm in process of setting up
> >> the architecture, in which I've used testpmd to forward traffic between
> >> two
> >> virtual devices(tap and vhost users) over a bridge.
> >> 
> >> While I'm at it, I've identified that the internal dev_attached flag never
> >> gets set to 1 from the rte_eth_vhost.c file. I've tried to manually set it
> >> to 1 in the start routine, but I just see that the queue index being
> >> referenced is out of range.
> >> 
> >> I'm not sure how to proceed.  Has anyone had luck using testpmd to
> >> communicate with vhost-user devices?  If yes, any hints on a workaround?
> >> 
> >> Here's how I configured my setup after installing dpdk and openvswitch:
> >> 
> >> 1. To start ovs-ctl:
> >> /usr/local/share/openvswitch/scripts/ovs-ctl start
> >> 
> >> 2. Setup hugepages:
> >> echo '2048' > /proc/sys/vm/nr_hugepages
> >> 
> >> 3. Add a new network namespace:
> >> ip netns add ns1
> >> 
> >> 4. Add and set a bridge:
> >> ovs-vsctl add-br dpdkbr0 -- set Bridge dpdkbr0 datapath_type=netdev
> >> options:vhost-server-path=/usr/local/var/run/openvswitch/vhu0
> >> ovs-vsctl show
> >> 
> >> 5. Add a vhost user to the bridge created:
> >> ovs-vsctl add-port dpdkbr0 vhu0 -- set Interface vhu0
> >> type=dpdkvhostuserclient
> >> 
> >> 6. Execute bash on the network namespace:
> >> ip netns exec ns1 bash
> >> 
> >> 7. Use testpmd and connect the namespaces:
> >> testpmd --socket-mem=512
> >> --vdev='eth_vhost0,iface=/usr/local/var/run/openvswitch/vhu0,queues=1'
> >> --vdev='net_tap0,iface=tap0' --file-prefix page0 -- -i
> >> 
> >> 
> >> I repeated steps 3 - 7 for another network namespace on the same bridge.
> >> Following this, in fresh terminals, I assigned IP addresses to the tap
> >> devices created and tried pinging them. From port statistics,
> >> I identified the above mentioned issue with the dev_attached and queue
> >> statistics.
> >> 
> >> I would greatly appreciate any help from your end.
> >> 
> >> Thanks.
> >> 
> >> -------------------------------------------------
> >> Bala Sankaran
> >> Networking Services Intern
> >> Red Hat Inc .,
> >> 
> > -------------------------------------------------
> > Bala Sankaran
> > Networking Services Intern
> 
> Regards,
> Keith
> 
> 

--------------------------------------------------
Bala Sankaran
Networking Services Intern
Red Hat Inc .,

Re: [dpdk-users] Traffic doesn't forward on virtual devices

[Aaron Conole]

Bala Sankaran <bsankara@redhat.com> writes:

> Perfect!
>
> Thanks for the help.
>
> ----- Original Message -----
>> From: "Keith Wiles" <keith.wiles@intel.com>
>> To: "Bala Sankaran" <bsankara@redhat.com>
>> Cc: users@dpdk.org, "Aaron Conole" <aconole@redhat.com>
>> Sent: Thursday, July 5, 2018 11:41:46 AM
>> Subject: Re: [dpdk-users] Traffic doesn't forward on virtual devices
>> 
>> 
>> 
>> > On Jul 5, 2018, at 9:53 AM, Bala Sankaran <bsankara@redhat.com> wrote:
>> > 
>> > Greetings,
>> > 
>> > I am currently using dpdk version 17.11.2. I see that there are a few other
>> > revisions in 17.11.3, followed by the latest stable version of 18.02.2.
>> > 
>> > Based on the issues I have faced so far (see Original
>> > Message below), would you suggest that  I go for
>> > another version? If yes, which one? In essence, my question is, would
>> > resorting to a different version of dpdk solve my current issue of
>> > virtqueue id being invalid?
>> > 
>> > Any help is much appreciated.
>> 
>> From a support perspective using the latest version 18.05 or the long term
>> supported version 17.11.3 is easier for most to help. I would pick the
>> latest release 18.05 myself. As for fixing this problem I do not know. You
>> can look into the MAINTAINERS file and find the maintainers of area(s) and
>> include them in the CC line on your questions as sometimes they miss the
>> emails as the volume can be high at times.

Thanks Keith.

I took a quick look and it seems like the queues are not setting up
correctly between OvS and testpmd?  Probably there's a step missing
somewhere, although nothing in either the netdev-dpdk.c from OvS nor the
rte_ethdev was obvious to stand out to me.

I've CC'd Maxime, Ian, and Ciara - maybe they have a better idea to try?

>> > 
>> > Thanks
>> > 
>> > ----- Original Message -----
>> >> From: "Bala Sankaran" <bsankara@redhat.com>
>> >> To: users@dpdk.org
>> >> Cc: "Aaron Conole" <aconole@redhat.com>
>> >> Sent: Thursday, June 28, 2018 3:18:13 PM
>> >> Subject: Traffic doesn't forward on virtual devices
>> >> 
>> >> 
>> >> Hello team,
>> >> 
>> >> I am working on a project to do PVP tests on dpdk. As a first step, I
>> >> would
>> >> like to get traffic flow between tap devices. I'm in process of setting up
>> >> the architecture, in which I've used testpmd to forward traffic between
>> >> two
>> >> virtual devices(tap and vhost users) over a bridge.
>> >> 
>> >> While I'm at it, I've identified that the internal dev_attached flag never
>> >> gets set to 1 from the rte_eth_vhost.c file. I've tried to manually set it
>> >> to 1 in the start routine, but I just see that the queue index being
>> >> referenced is out of range.
>> >> 
>> >> I'm not sure how to proceed.  Has anyone had luck using testpmd to
>> >> communicate with vhost-user devices?  If yes, any hints on a workaround?
>> >> 
>> >> Here's how I configured my setup after installing dpdk and openvswitch:
>> >> 
>> >> 1. To start ovs-ctl:
>> >> /usr/local/share/openvswitch/scripts/ovs-ctl start
>> >> 
>> >> 2. Setup hugepages:
>> >> echo '2048' > /proc/sys/vm/nr_hugepages
>> >> 
>> >> 3. Add a new network namespace:
>> >> ip netns add ns1
>> >> 
>> >> 4. Add and set a bridge:
>> >> ovs-vsctl add-br dpdkbr0 -- set Bridge dpdkbr0 datapath_type=netdev
>> >> options:vhost-server-path=/usr/local/var/run/openvswitch/vhu0
>> >> ovs-vsctl show
>> >> 
>> >> 5. Add a vhost user to the bridge created:
>> >> ovs-vsctl add-port dpdkbr0 vhu0 -- set Interface vhu0
>> >> type=dpdkvhostuserclient
>> >> 
>> >> 6. Execute bash on the network namespace:
>> >> ip netns exec ns1 bash
>> >> 
>> >> 7. Use testpmd and connect the namespaces:
>> >> testpmd --socket-mem=512
>> >> --vdev='eth_vhost0,iface=/usr/local/var/run/openvswitch/vhu0,queues=1'
>> >> --vdev='net_tap0,iface=tap0' --file-prefix page0 -- -i
>> >> 
>> >> 
>> >> I repeated steps 3 - 7 for another network namespace on the same bridge.
>> >> Following this, in fresh terminals, I assigned IP addresses to the tap
>> >> devices created and tried pinging them. From port statistics,
>> >> I identified the above mentioned issue with the dev_attached and queue
>> >> statistics.
>> >> 
>> >> I would greatly appreciate any help from your end.
>> >> 
>> >> Thanks.
>> >> 
>> >> -------------------------------------------------
>> >> Bala Sankaran
>> >> Networking Services Intern
>> >> Red Hat Inc .,
>> >> 
>> > -------------------------------------------------
>> > Bala Sankaran
>> > Networking Services Intern
>> 
>> Regards,
>> Keith
>> 
>> 
>
> --------------------------------------------------
> Bala Sankaran
> Networking Services Intern
> Red Hat Inc .,

Re: [dpdk-users] Traffic doesn't forward on virtual devices

[Loftus, Ciara]

> 
> Bala Sankaran <bsankara@redhat.com> writes:
> 
> > Perfect!
> >
> > Thanks for the help.
> >
> > ----- Original Message -----
> >> From: "Keith Wiles" <keith.wiles@intel.com>
> >> To: "Bala Sankaran" <bsankara@redhat.com>
> >> Cc: users@dpdk.org, "Aaron Conole" <aconole@redhat.com>
> >> Sent: Thursday, July 5, 2018 11:41:46 AM
> >> Subject: Re: [dpdk-users] Traffic doesn't forward on virtual devices
> >>
> >>
> >>
> >> > On Jul 5, 2018, at 9:53 AM, Bala Sankaran <bsankara@redhat.com>
> wrote:
> >> >
> >> > Greetings,
> >> >
> >> > I am currently using dpdk version 17.11.2. I see that there are a few
> other
> >> > revisions in 17.11.3, followed by the latest stable version of 18.02.2.
> >> >
> >> > Based on the issues I have faced so far (see Original
> >> > Message below), would you suggest that  I go for
> >> > another version? If yes, which one? In essence, my question is, would
> >> > resorting to a different version of dpdk solve my current issue of
> >> > virtqueue id being invalid?
> >> >
> >> > Any help is much appreciated.
> >>
> >> From a support perspective using the latest version 18.05 or the long term
> >> supported version 17.11.3 is easier for most to help. I would pick the
> >> latest release 18.05 myself. As for fixing this problem I do not know. You
> >> can look into the MAINTAINERS file and find the maintainers of area(s)
> and
> >> include them in the CC line on your questions as sometimes they miss the
> >> emails as the volume can be high at times.
> 
> Thanks Keith.
> 
> I took a quick look and it seems like the queues are not setting up
> correctly between OvS and testpmd?  Probably there's a step missing
> somewhere, although nothing in either the netdev-dpdk.c from OvS nor the
> rte_ethdev was obvious to stand out to me.
> 
> I've CC'd Maxime, Ian, and Ciara - maybe they have a better idea to try?

Hi,

I think the appropriate driver to use in this test on the test-pmd side might be virtio-user.
Follow the same steps just change your vdev test-pmd arguments to:
--vdev='net_virtio_user0,path=/usr/local/var/run/openvswitch/vhu0'

Thanks,
Ciara

> 
> >> >
> >> > Thanks
> >> >
> >> > ----- Original Message -----
> >> >> From: "Bala Sankaran" <bsankara@redhat.com>
> >> >> To: users@dpdk.org
> >> >> Cc: "Aaron Conole" <aconole@redhat.com>
> >> >> Sent: Thursday, June 28, 2018 3:18:13 PM
> >> >> Subject: Traffic doesn't forward on virtual devices
> >> >>
> >> >>
> >> >> Hello team,
> >> >>
> >> >> I am working on a project to do PVP tests on dpdk. As a first step, I
> >> >> would
> >> >> like to get traffic flow between tap devices. I'm in process of setting up
> >> >> the architecture, in which I've used testpmd to forward traffic
> between
> >> >> two
> >> >> virtual devices(tap and vhost users) over a bridge.
> >> >>
> >> >> While I'm at it, I've identified that the internal dev_attached flag never
> >> >> gets set to 1 from the rte_eth_vhost.c file. I've tried to manually set it
> >> >> to 1 in the start routine, but I just see that the queue index being
> >> >> referenced is out of range.
> >> >>
> >> >> I'm not sure how to proceed.  Has anyone had luck using testpmd to
> >> >> communicate with vhost-user devices?  If yes, any hints on a
> workaround?
> >> >>
> >> >> Here's how I configured my setup after installing dpdk and
> openvswitch:
> >> >>
> >> >> 1. To start ovs-ctl:
> >> >> /usr/local/share/openvswitch/scripts/ovs-ctl start
> >> >>
> >> >> 2. Setup hugepages:
> >> >> echo '2048' > /proc/sys/vm/nr_hugepages
> >> >>
> >> >> 3. Add a new network namespace:
> >> >> ip netns add ns1
> >> >>
> >> >> 4. Add and set a bridge:
> >> >> ovs-vsctl add-br dpdkbr0 -- set Bridge dpdkbr0 datapath_type=netdev
> >> >> options:vhost-server-path=/usr/local/var/run/openvswitch/vhu0
> >> >> ovs-vsctl show
> >> >>
> >> >> 5. Add a vhost user to the bridge created:
> >> >> ovs-vsctl add-port dpdkbr0 vhu0 -- set Interface vhu0
> >> >> type=dpdkvhostuserclient
> >> >>
> >> >> 6. Execute bash on the network namespace:
> >> >> ip netns exec ns1 bash
> >> >>
> >> >> 7. Use testpmd and connect the namespaces:
> >> >> testpmd --socket-mem=512
> >> >> --
> vdev='eth_vhost0,iface=/usr/local/var/run/openvswitch/vhu0,queues=1'
> >> >> --vdev='net_tap0,iface=tap0' --file-prefix page0 -- -i
> >> >>
> >> >>
> >> >> I repeated steps 3 - 7 for another network namespace on the same
> bridge.
> >> >> Following this, in fresh terminals, I assigned IP addresses to the tap
> >> >> devices created and tried pinging them. From port statistics,
> >> >> I identified the above mentioned issue with the dev_attached and
> queue
> >> >> statistics.
> >> >>
> >> >> I would greatly appreciate any help from your end.
> >> >>
> >> >> Thanks.
> >> >>
> >> >> -------------------------------------------------
> >> >> Bala Sankaran
> >> >> Networking Services Intern
> >> >> Red Hat Inc .,
> >> >>
> >> > -------------------------------------------------
> >> > Bala Sankaran
> >> > Networking Services Intern
> >>
> >> Regards,
> >> Keith
> >>
> >>
> >
> > --------------------------------------------------
> > Bala Sankaran
> > Networking Services Intern
> > Red Hat Inc .,

Re: [dpdk-users] Traffic doesn't forward on virtual devices

[Bala Sankaran]

----- Original Message -----
> From: "Ciara Loftus" <ciara.loftus@intel.com>
> To: "Aaron Conole" <aconole@redhat.com>, "Bala Sankaran" <bsankara@redhat.com>
> Cc: "Keith Wiles" <keith.wiles@intel.com>, users@dpdk.org, dev@dpdk.org, "Maxime Coquelin"
> <maxime.coquelin@redhat.com>, "Ian Stokes" <ian.stokes@intel.com>
> Sent: Tuesday, July 10, 2018 4:20:31 AM
> Subject: RE: [dpdk-users] Traffic doesn't forward on virtual devices
> 
> > 
> > Bala Sankaran <bsankara@redhat.com> writes:
> > 
> > > Perfect!
> > >
> > > Thanks for the help.
> > >
> > > ----- Original Message -----
> > >> From: "Keith Wiles" <keith.wiles@intel.com>
> > >> To: "Bala Sankaran" <bsankara@redhat.com>
> > >> Cc: users@dpdk.org, "Aaron Conole" <aconole@redhat.com>
> > >> Sent: Thursday, July 5, 2018 11:41:46 AM
> > >> Subject: Re: [dpdk-users] Traffic doesn't forward on virtual devices
> > >>
> > >>
> > >>
> > >> > On Jul 5, 2018, at 9:53 AM, Bala Sankaran <bsankara@redhat.com>
> > wrote:
> > >> >
> > >> > Greetings,
> > >> >
> > >> > I am currently using dpdk version 17.11.2. I see that there are a few
> > other
> > >> > revisions in 17.11.3, followed by the latest stable version of
> > >> > 18.02.2.
> > >> >
> > >> > Based on the issues I have faced so far (see Original
> > >> > Message below), would you suggest that  I go for
> > >> > another version? If yes, which one? In essence, my question is, would
> > >> > resorting to a different version of dpdk solve my current issue of
> > >> > virtqueue id being invalid?
> > >> >
> > >> > Any help is much appreciated.
> > >>
> > >> From a support perspective using the latest version 18.05 or the long
> > >> term
> > >> supported version 17.11.3 is easier for most to help. I would pick the
> > >> latest release 18.05 myself. As for fixing this problem I do not know.
> > >> You
> > >> can look into the MAINTAINERS file and find the maintainers of area(s)
> > and
> > >> include them in the CC line on your questions as sometimes they miss the
> > >> emails as the volume can be high at times.
> > 
> > Thanks Keith.
> > 
> > I took a quick look and it seems like the queues are not setting up
> > correctly between OvS and testpmd?  Probably there's a step missing
> > somewhere, although nothing in either the netdev-dpdk.c from OvS nor the
> > rte_ethdev was obvious to stand out to me.
> > 
> > I've CC'd Maxime, Ian, and Ciara - maybe they have a better idea to try?
> 
> Hi,
> 
> I think the appropriate driver to use in this test on the test-pmd side might
> be virtio-user.
> Follow the same steps just change your vdev test-pmd arguments to:
> --vdev='net_virtio_user0,path=/usr/local/var/run/openvswitch/vhu0'
> 
> Thanks,
> Ciara
> 

Thank you for your response.

I tried using virtio-user, but I face an error that says: Failed to prepare memory for vhost-user.
The command I ran is as below: 

[root@localhost openvswitch]# testpmd --socket-mem=1024 --vdev='net_virtio_user1,path=/usr/local/var/run/openvswitch/vhu1,server=1'  --vdev='net_tap1,iface=tap1' --file-prefix page1 -- -i
EAL: Detected 4 lcore(s)
EAL: Detected 1 NUMA nodes
EAL: Multi-process socket /var/run/dpdk/page1/mp_socket
EAL: Probing VFIO support...
EAL: WARNING: cpu flags constant_tsc=yes nonstop_tsc=no -> using unreliable clock cycles !
rte_pmd_tap_probe(): Initializing pmd_tap for net_tap1 as tap1
Interactive-mode selected
Warning: NUMA should be configured manually by using --port-numa-config and --ring-numa-config parameters along with --numa.
testpmd: create a new mbuf pool <mbuf_pool_socket_0>: n=171456, size=2176, socket=0
testpmd: preferred mempool ops selected: ring_mp_mc
Configuring Port 0 (socket 0)
virtio_user_server_reconnect(): WARNING: Some features 0x1801 are not supported by vhost-user!
get_hugepage_file_info(): Exceed maximum of 8
prepare_vhost_memory_user(): Failed to prepare memory for vhost-user
Port 0: DA:60:01:0C:4B:29
Configuring Port 1 (socket 0)
Port 1: D2:5A:94:68:AF:B3
Checking link statuses...

Port 0: LSC event
Done

I tried increasing the socket-memory, I checked /proc/meminfo and found there were over 
1280 free hugepages. 
So my understanding is that this is not an issue where I don't have enough hugepages.

Can you provide leads on what's wrong here?

> > 
> > >> >
> > >> > Thanks
> > >> >
> > >> > ----- Original Message -----
> > >> >> From: "Bala Sankaran" <bsankara@redhat.com>
> > >> >> To: users@dpdk.org
> > >> >> Cc: "Aaron Conole" <aconole@redhat.com>
> > >> >> Sent: Thursday, June 28, 2018 3:18:13 PM
> > >> >> Subject: Traffic doesn't forward on virtual devices
> > >> >>
> > >> >>
> > >> >> Hello team,
> > >> >>
> > >> >> I am working on a project to do PVP tests on dpdk. As a first step, I
> > >> >> would
> > >> >> like to get traffic flow between tap devices. I'm in process of
> > >> >> setting up
> > >> >> the architecture, in which I've used testpmd to forward traffic
> > between
> > >> >> two
> > >> >> virtual devices(tap and vhost users) over a bridge.
> > >> >>
> > >> >> While I'm at it, I've identified that the internal dev_attached flag
> > >> >> never
> > >> >> gets set to 1 from the rte_eth_vhost.c file. I've tried to manually
> > >> >> set it
> > >> >> to 1 in the start routine, but I just see that the queue index being
> > >> >> referenced is out of range.
> > >> >>
> > >> >> I'm not sure how to proceed.  Has anyone had luck using testpmd to
> > >> >> communicate with vhost-user devices?  If yes, any hints on a
> > workaround?
> > >> >>
> > >> >> Here's how I configured my setup after installing dpdk and
> > openvswitch:
> > >> >>
> > >> >> 1. To start ovs-ctl:
> > >> >> /usr/local/share/openvswitch/scripts/ovs-ctl start
> > >> >>
> > >> >> 2. Setup hugepages:
> > >> >> echo '2048' > /proc/sys/vm/nr_hugepages
> > >> >>
> > >> >> 3. Add a new network namespace:
> > >> >> ip netns add ns1
> > >> >>
> > >> >> 4. Add and set a bridge:
> > >> >> ovs-vsctl add-br dpdkbr0 -- set Bridge dpdkbr0 datapath_type=netdev
> > >> >> options:vhost-server-path=/usr/local/var/run/openvswitch/vhu0
> > >> >> ovs-vsctl show
> > >> >>
> > >> >> 5. Add a vhost user to the bridge created:
> > >> >> ovs-vsctl add-port dpdkbr0 vhu0 -- set Interface vhu0
> > >> >> type=dpdkvhostuserclient
> > >> >>
> > >> >> 6. Execute bash on the network namespace:
> > >> >> ip netns exec ns1 bash
> > >> >>
> > >> >> 7. Use testpmd and connect the namespaces:
> > >> >> testpmd --socket-mem=512
> > >> >> --
> > vdev='eth_vhost0,iface=/usr/local/var/run/openvswitch/vhu0,queues=1'
> > >> >> --vdev='net_tap0,iface=tap0' --file-prefix page0 -- -i
> > >> >>
> > >> >>
> > >> >> I repeated steps 3 - 7 for another network namespace on the same
> > bridge.
> > >> >> Following this, in fresh terminals, I assigned IP addresses to the
> > >> >> tap
> > >> >> devices created and tried pinging them. From port statistics,
> > >> >> I identified the above mentioned issue with the dev_attached and
> > queue
> > >> >> statistics.
> > >> >>
> > >> >> I would greatly appreciate any help from your end.
> > >> >>
> > >> >> Thanks.
> > >> >>
> > >> >> -------------------------------------------------
> > >> >> Bala Sankaran
> > >> >> Networking Services Intern
> > >> >> Red Hat Inc .,
> > >> >>
> > >> > -------------------------------------------------
> > >> > Bala Sankaran
> > >> > Networking Services Intern
> > >>
> > >> Regards,
> > >> Keith
> > >>
> > >>
> > >
> > > --------------------------------------------------
> > > Bala Sankaran
> > > Networking Services Intern
> > > Red Hat Inc .,
> 

Thanks.
Bala.

Re: [dpdk-users] Traffic doesn't forward on virtual devices

[Loftus, Ciara]

> > >
> > > Bala Sankaran <bsankara@redhat.com> writes:
> > >
> > > > Perfect!
> > > >
> > > > Thanks for the help.
> > > >
> > > > ----- Original Message -----
> > > >> From: "Keith Wiles" <keith.wiles@intel.com>
> > > >> To: "Bala Sankaran" <bsankara@redhat.com>
> > > >> Cc: users@dpdk.org, "Aaron Conole" <aconole@redhat.com>
> > > >> Sent: Thursday, July 5, 2018 11:41:46 AM
> > > >> Subject: Re: [dpdk-users] Traffic doesn't forward on virtual devices
> > > >>
> > > >>
> > > >>
> > > >> > On Jul 5, 2018, at 9:53 AM, Bala Sankaran <bsankara@redhat.com>
> > > wrote:
> > > >> >
> > > >> > Greetings,
> > > >> >
> > > >> > I am currently using dpdk version 17.11.2. I see that there are a few
> > > other
> > > >> > revisions in 17.11.3, followed by the latest stable version of
> > > >> > 18.02.2.
> > > >> >
> > > >> > Based on the issues I have faced so far (see Original
> > > >> > Message below), would you suggest that  I go for
> > > >> > another version? If yes, which one? In essence, my question is,
> would
> > > >> > resorting to a different version of dpdk solve my current issue of
> > > >> > virtqueue id being invalid?
> > > >> >
> > > >> > Any help is much appreciated.
> > > >>
> > > >> From a support perspective using the latest version 18.05 or the long
> > > >> term
> > > >> supported version 17.11.3 is easier for most to help. I would pick the
> > > >> latest release 18.05 myself. As for fixing this problem I do not know.
> > > >> You
> > > >> can look into the MAINTAINERS file and find the maintainers of area(s)
> > > and
> > > >> include them in the CC line on your questions as sometimes they miss
> the
> > > >> emails as the volume can be high at times.
> > >
> > > Thanks Keith.
> > >
> > > I took a quick look and it seems like the queues are not setting up
> > > correctly between OvS and testpmd?  Probably there's a step missing
> > > somewhere, although nothing in either the netdev-dpdk.c from OvS nor
> the
> > > rte_ethdev was obvious to stand out to me.
> > >
> > > I've CC'd Maxime, Ian, and Ciara - maybe they have a better idea to try?
> >
> > Hi,
> >
> > I think the appropriate driver to use in this test on the test-pmd side might
> > be virtio-user.
> > Follow the same steps just change your vdev test-pmd arguments to:
> > --vdev='net_virtio_user0,path=/usr/local/var/run/openvswitch/vhu0'
> >
> > Thanks,
> > Ciara
> >
> 
> Thank you for your response.
> 
> I tried using virtio-user, but I face an error that says: Failed to prepare
> memory for vhost-user.
> The command I ran is as below:
> 
> [root@localhost openvswitch]# testpmd --socket-mem=1024 --
> vdev='net_virtio_user1,path=/usr/local/var/run/openvswitch/vhu1,server=
> 1'  --vdev='net_tap1,iface=tap1' --file-prefix page1 -- -i
> EAL: Detected 4 lcore(s)
> EAL: Detected 1 NUMA nodes
> EAL: Multi-process socket /var/run/dpdk/page1/mp_socket
> EAL: Probing VFIO support...
> EAL: WARNING: cpu flags constant_tsc=yes nonstop_tsc=no -> using
> unreliable clock cycles !
> rte_pmd_tap_probe(): Initializing pmd_tap for net_tap1 as tap1
> Interactive-mode selected
> Warning: NUMA should be configured manually by using --port-numa-config
> and --ring-numa-config parameters along with --numa.
> testpmd: create a new mbuf pool <mbuf_pool_socket_0>: n=171456,
> size=2176, socket=0
> testpmd: preferred mempool ops selected: ring_mp_mc
> Configuring Port 0 (socket 0)
> virtio_user_server_reconnect(): WARNING: Some features 0x1801 are not
> supported by vhost-user!
> get_hugepage_file_info(): Exceed maximum of 8
> prepare_vhost_memory_user(): Failed to prepare memory for vhost-user
> Port 0: DA:60:01:0C:4B:29
> Configuring Port 1 (socket 0)
> Port 1: D2:5A:94:68:AF:B3
> Checking link statuses...
> 
> Port 0: LSC event
> Done
> 
> I tried increasing the socket-memory, I checked /proc/meminfo and found
> there were over
> 1280 free hugepages.
> So my understanding is that this is not an issue where I don't have enough
> hugepages.
> 
> Can you provide leads on what's wrong here?

Hi,

The limitations section for Virtio User https://doc.dpdk.org/guides/howto/virtio_user_for_container_networking.html#limitations states that:

" Cannot work when there are more than VHOST_MEMORY_MAX_NREGIONS(8) hugepages. If you have more regions (especially when 2MB hugepages are used), the option, --single-file-segments, can help to reduce the number of shared files."

Suggest using the --single-file-segments option on the test-pmd command line or failing that increasing your hugepage size to 1G https://doc.dpdk.org/guides/linux_gsg/sys_reqs.html#use-of-hugepages-in-the-linux-environment

Thanks,
Ciara

> 
> > >
> > > >> >
> > > >> > Thanks
> > > >> >
> > > >> > ----- Original Message -----
> > > >> >> From: "Bala Sankaran" <bsankara@redhat.com>
> > > >> >> To: users@dpdk.org
> > > >> >> Cc: "Aaron Conole" <aconole@redhat.com>
> > > >> >> Sent: Thursday, June 28, 2018 3:18:13 PM
> > > >> >> Subject: Traffic doesn't forward on virtual devices
> > > >> >>
> > > >> >>
> > > >> >> Hello team,
> > > >> >>
> > > >> >> I am working on a project to do PVP tests on dpdk. As a first step, I
> > > >> >> would
> > > >> >> like to get traffic flow between tap devices. I'm in process of
> > > >> >> setting up
> > > >> >> the architecture, in which I've used testpmd to forward traffic
> > > between
> > > >> >> two
> > > >> >> virtual devices(tap and vhost users) over a bridge.
> > > >> >>
> > > >> >> While I'm at it, I've identified that the internal dev_attached flag
> > > >> >> never
> > > >> >> gets set to 1 from the rte_eth_vhost.c file. I've tried to manually
> > > >> >> set it
> > > >> >> to 1 in the start routine, but I just see that the queue index being
> > > >> >> referenced is out of range.
> > > >> >>
> > > >> >> I'm not sure how to proceed.  Has anyone had luck using testpmd
> to
> > > >> >> communicate with vhost-user devices?  If yes, any hints on a
> > > workaround?
> > > >> >>
> > > >> >> Here's how I configured my setup after installing dpdk and
> > > openvswitch:
> > > >> >>
> > > >> >> 1. To start ovs-ctl:
> > > >> >> /usr/local/share/openvswitch/scripts/ovs-ctl start
> > > >> >>
> > > >> >> 2. Setup hugepages:
> > > >> >> echo '2048' > /proc/sys/vm/nr_hugepages
> > > >> >>
> > > >> >> 3. Add a new network namespace:
> > > >> >> ip netns add ns1
> > > >> >>
> > > >> >> 4. Add and set a bridge:
> > > >> >> ovs-vsctl add-br dpdkbr0 -- set Bridge dpdkbr0
> datapath_type=netdev
> > > >> >> options:vhost-server-path=/usr/local/var/run/openvswitch/vhu0
> > > >> >> ovs-vsctl show
> > > >> >>
> > > >> >> 5. Add a vhost user to the bridge created:
> > > >> >> ovs-vsctl add-port dpdkbr0 vhu0 -- set Interface vhu0
> > > >> >> type=dpdkvhostuserclient
> > > >> >>
> > > >> >> 6. Execute bash on the network namespace:
> > > >> >> ip netns exec ns1 bash
> > > >> >>
> > > >> >> 7. Use testpmd and connect the namespaces:
> > > >> >> testpmd --socket-mem=512
> > > >> >> --
> > >
> vdev='eth_vhost0,iface=/usr/local/var/run/openvswitch/vhu0,queues=1'
> > > >> >> --vdev='net_tap0,iface=tap0' --file-prefix page0 -- -i
> > > >> >>
> > > >> >>
> > > >> >> I repeated steps 3 - 7 for another network namespace on the same
> > > bridge.
> > > >> >> Following this, in fresh terminals, I assigned IP addresses to the
> > > >> >> tap
> > > >> >> devices created and tried pinging them. From port statistics,
> > > >> >> I identified the above mentioned issue with the dev_attached and
> > > queue
> > > >> >> statistics.
> > > >> >>
> > > >> >> I would greatly appreciate any help from your end.
> > > >> >>
> > > >> >> Thanks.
> > > >> >>
> > > >> >> -------------------------------------------------
> > > >> >> Bala Sankaran
> > > >> >> Networking Services Intern
> > > >> >> Red Hat Inc .,
> > > >> >>
> > > >> > -------------------------------------------------
> > > >> > Bala Sankaran
> > > >> > Networking Services Intern
> > > >>
> > > >> Regards,
> > > >> Keith
> > > >>
> > > >>
> > > >
> > > > --------------------------------------------------
> > > > Bala Sankaran
> > > > Networking Services Intern
> > > > Red Hat Inc .,
> >
> 
> Thanks.
> Bala.

Re: [dpdk-users] Traffic doesn't forward on virtual devices

[Bala Sankaran]

----- Original Message -----
> From: "Ciara Loftus" <ciara.loftus@intel.com>
> To: "Bala Sankaran" <bsankara@redhat.com>
> Cc: "Aaron Conole" <aconole@redhat.com>, "Keith Wiles" <keith.wiles@intel.com>, users@dpdk.org, dev@dpdk.org, "Maxime
> Coquelin" <maxime.coquelin@redhat.com>, "Ian Stokes" <ian.stokes@intel.com>
> Sent: Wednesday, July 11, 2018 4:12:38 AM
> Subject: RE: [dpdk-users] Traffic doesn't forward on virtual devices
> 
> > > >
> > > > Bala Sankaran <bsankara@redhat.com> writes:
> > > >
> > > > > Perfect!
> > > > >
> > > > > Thanks for the help.
> > > > >
> > > > > ----- Original Message -----
> > > > >> From: "Keith Wiles" <keith.wiles@intel.com>
> > > > >> To: "Bala Sankaran" <bsankara@redhat.com>
> > > > >> Cc: users@dpdk.org, "Aaron Conole" <aconole@redhat.com>
> > > > >> Sent: Thursday, July 5, 2018 11:41:46 AM
> > > > >> Subject: Re: [dpdk-users] Traffic doesn't forward on virtual devices
> > > > >>
> > > > >>
> > > > >>
> > > > >> > On Jul 5, 2018, at 9:53 AM, Bala Sankaran <bsankara@redhat.com>
> > > > wrote:
> > > > >> >
> > > > >> > Greetings,
> > > > >> >
> > > > >> > I am currently using dpdk version 17.11.2. I see that there are a
> > > > >> > few
> > > > other
> > > > >> > revisions in 17.11.3, followed by the latest stable version of
> > > > >> > 18.02.2.
> > > > >> >
> > > > >> > Based on the issues I have faced so far (see Original
> > > > >> > Message below), would you suggest that  I go for
> > > > >> > another version? If yes, which one? In essence, my question is,
> > would
> > > > >> > resorting to a different version of dpdk solve my current issue of
> > > > >> > virtqueue id being invalid?
> > > > >> >
> > > > >> > Any help is much appreciated.
> > > > >>
> > > > >> From a support perspective using the latest version 18.05 or the
> > > > >> long
> > > > >> term
> > > > >> supported version 17.11.3 is easier for most to help. I would pick
> > > > >> the
> > > > >> latest release 18.05 myself. As for fixing this problem I do not
> > > > >> know.
> > > > >> You
> > > > >> can look into the MAINTAINERS file and find the maintainers of
> > > > >> area(s)
> > > > and
> > > > >> include them in the CC line on your questions as sometimes they miss
> > the
> > > > >> emails as the volume can be high at times.
> > > >
> > > > Thanks Keith.
> > > >
> > > > I took a quick look and it seems like the queues are not setting up
> > > > correctly between OvS and testpmd?  Probably there's a step missing
> > > > somewhere, although nothing in either the netdev-dpdk.c from OvS nor
> > the
> > > > rte_ethdev was obvious to stand out to me.
> > > >
> > > > I've CC'd Maxime, Ian, and Ciara - maybe they have a better idea to
> > > > try?
> > >
> > > Hi,
> > >
> > > I think the appropriate driver to use in this test on the test-pmd side
> > > might
> > > be virtio-user.
> > > Follow the same steps just change your vdev test-pmd arguments to:
> > > --vdev='net_virtio_user0,path=/usr/local/var/run/openvswitch/vhu0'
> > >
> > > Thanks,
> > > Ciara
> > >
> > 
> > Thank you for your response.
> > 
> > I tried using virtio-user, but I face an error that says: Failed to prepare
> > memory for vhost-user.
> > The command I ran is as below:
> > 
> > [root@localhost openvswitch]# testpmd --socket-mem=1024 --
> > vdev='net_virtio_user1,path=/usr/local/var/run/openvswitch/vhu1,server=
> > 1'  --vdev='net_tap1,iface=tap1' --file-prefix page1 -- -i
> > EAL: Detected 4 lcore(s)
> > EAL: Detected 1 NUMA nodes
> > EAL: Multi-process socket /var/run/dpdk/page1/mp_socket
> > EAL: Probing VFIO support...
> > EAL: WARNING: cpu flags constant_tsc=yes nonstop_tsc=no -> using
> > unreliable clock cycles !
> > rte_pmd_tap_probe(): Initializing pmd_tap for net_tap1 as tap1
> > Interactive-mode selected
> > Warning: NUMA should be configured manually by using --port-numa-config
> > and --ring-numa-config parameters along with --numa.
> > testpmd: create a new mbuf pool <mbuf_pool_socket_0>: n=171456,
> > size=2176, socket=0
> > testpmd: preferred mempool ops selected: ring_mp_mc
> > Configuring Port 0 (socket 0)
> > virtio_user_server_reconnect(): WARNING: Some features 0x1801 are not
> > supported by vhost-user!
> > get_hugepage_file_info(): Exceed maximum of 8
> > prepare_vhost_memory_user(): Failed to prepare memory for vhost-user
> > Port 0: DA:60:01:0C:4B:29
> > Configuring Port 1 (socket 0)
> > Port 1: D2:5A:94:68:AF:B3
> > Checking link statuses...
> > 
> > Port 0: LSC event
> > Done
> > 
> > I tried increasing the socket-memory, I checked /proc/meminfo and found
> > there were over
> > 1280 free hugepages.
> > So my understanding is that this is not an issue where I don't have enough
> > hugepages.
> > 
> > Can you provide leads on what's wrong here?
> 
> Hi,
> 
> The limitations section for Virtio User
> https://doc.dpdk.org/guides/howto/virtio_user_for_container_networking.html#limitations
> states that:
> 
> " Cannot work when there are more than VHOST_MEMORY_MAX_NREGIONS(8)
> hugepages. If you have more regions (especially when 2MB hugepages are
> used), the option, --single-file-segments, can help to reduce the number of
> shared files."
> 
> Suggest using the --single-file-segments option on the test-pmd command line
> or failing that increasing your hugepage size to 1G
> https://doc.dpdk.org/guides/linux_gsg/sys_reqs.html#use-of-hugepages-in-the-linux-environment
> 
> Thanks,
> Ciara

Thanks a ton. 

Looks like I had just missed out on adding the single-file-segments argument.
After addition to my existing command structure, testpmd is setup and further,
I am able to ping the tap devices. 

Thanks
Bala.

> 
> > 
> > > >
> > > > >> >
> > > > >> > Thanks
> > > > >> >
> > > > >> > ----- Original Message -----
> > > > >> >> From: "Bala Sankaran" <bsankara@redhat.com>
> > > > >> >> To: users@dpdk.org
> > > > >> >> Cc: "Aaron Conole" <aconole@redhat.com>
> > > > >> >> Sent: Thursday, June 28, 2018 3:18:13 PM
> > > > >> >> Subject: Traffic doesn't forward on virtual devices
> > > > >> >>
> > > > >> >>
> > > > >> >> Hello team,
> > > > >> >>
> > > > >> >> I am working on a project to do PVP tests on dpdk. As a first
> > > > >> >> step, I
> > > > >> >> would
> > > > >> >> like to get traffic flow between tap devices. I'm in process of
> > > > >> >> setting up
> > > > >> >> the architecture, in which I've used testpmd to forward traffic
> > > > between
> > > > >> >> two
> > > > >> >> virtual devices(tap and vhost users) over a bridge.
> > > > >> >>
> > > > >> >> While I'm at it, I've identified that the internal dev_attached
> > > > >> >> flag
> > > > >> >> never
> > > > >> >> gets set to 1 from the rte_eth_vhost.c file. I've tried to
> > > > >> >> manually
> > > > >> >> set it
> > > > >> >> to 1 in the start routine, but I just see that the queue index
> > > > >> >> being
> > > > >> >> referenced is out of range.
> > > > >> >>
> > > > >> >> I'm not sure how to proceed.  Has anyone had luck using testpmd
> > to
> > > > >> >> communicate with vhost-user devices?  If yes, any hints on a
> > > > workaround?
> > > > >> >>
> > > > >> >> Here's how I configured my setup after installing dpdk and
> > > > openvswitch:
> > > > >> >>
> > > > >> >> 1. To start ovs-ctl:
> > > > >> >> /usr/local/share/openvswitch/scripts/ovs-ctl start
> > > > >> >>
> > > > >> >> 2. Setup hugepages:
> > > > >> >> echo '2048' > /proc/sys/vm/nr_hugepages
> > > > >> >>
> > > > >> >> 3. Add a new network namespace:
> > > > >> >> ip netns add ns1
> > > > >> >>
> > > > >> >> 4. Add and set a bridge:
> > > > >> >> ovs-vsctl add-br dpdkbr0 -- set Bridge dpdkbr0
> > datapath_type=netdev
> > > > >> >> options:vhost-server-path=/usr/local/var/run/openvswitch/vhu0
> > > > >> >> ovs-vsctl show
> > > > >> >>
> > > > >> >> 5. Add a vhost user to the bridge created:
> > > > >> >> ovs-vsctl add-port dpdkbr0 vhu0 -- set Interface vhu0
> > > > >> >> type=dpdkvhostuserclient
> > > > >> >>
> > > > >> >> 6. Execute bash on the network namespace:
> > > > >> >> ip netns exec ns1 bash
> > > > >> >>
> > > > >> >> 7. Use testpmd and connect the namespaces:
> > > > >> >> testpmd --socket-mem=512
> > > > >> >> --
> > > >
> > vdev='eth_vhost0,iface=/usr/local/var/run/openvswitch/vhu0,queues=1'
> > > > >> >> --vdev='net_tap0,iface=tap0' --file-prefix page0 -- -i
> > > > >> >>
> > > > >> >>
> > > > >> >> I repeated steps 3 - 7 for another network namespace on the same
> > > > bridge.
> > > > >> >> Following this, in fresh terminals, I assigned IP addresses to
> > > > >> >> the
> > > > >> >> tap
> > > > >> >> devices created and tried pinging them. From port statistics,
> > > > >> >> I identified the above mentioned issue with the dev_attached and
> > > > queue
> > > > >> >> statistics.
> > > > >> >>
> > > > >> >> I would greatly appreciate any help from your end.
> > > > >> >>
> > > > >> >> Thanks.
> > > > >> >>
> > > > >> >> -------------------------------------------------
> > > > >> >> Bala Sankaran
> > > > >> >> Networking Services Intern
> > > > >> >> Red Hat Inc .,
> > > > >> >>
> > > > >> > -------------------------------------------------
> > > > >> > Bala Sankaran
> > > > >> > Networking Services Intern
> > > > >>
> > > > >> Regards,
> > > > >> Keith
> > > > >>
> > > > >>
> > > > >
> > > > > --------------------------------------------------
> > > > > Bala Sankaran
> > > > > Networking Services Intern
> > > > > Red Hat Inc .,
> > >
> > 
> > Thanks.
> > Bala.
>