Re: [dpdk-dev] DPDK OVS on Ubuntu 14.04# Issue's Resolved# Getting memory backing issues with qemu parameter passing

["Czesnowicz, Przemyslaw" <przemyslaw.czesnowicz@intel.com>]

You should be able to clone networking-ovs-dpdk, switch to kilo branch,  and run
python setup.py install
in the root of networking-ovs-dpdk, that should install agent and mech driver.
Then you would need to enable mech driver (ovsdpdk) on the controller in the /etc/neutron/plugins/ml2/ml2_conf.ini
And run the right agent on the computes (networking-ovs-dpdk-agent).


There should be pip packeges of networking-ovs-dpdk available shortly, I’ll let you know when that happens.

Przemek

From: Abhijeet Karve [mailto:abhijeet.karve@tcs.com]
Sent: Thursday, December 24, 2015 6:42 PM
To: Czesnowicz, Przemyslaw
Cc: dev@dpdk.org; discuss@openvswitch.org; Gray, Mark D
Subject: RE: [dpdk-dev] DPDK OVS on Ubuntu 14.04# Issue's Resolved# Getting memory backing issues with qemu parameter passing

Hi Przemek,

Thank you so much for your quick response.

The guide(https://github.com/openstack/networking-ovs-dpdk/blob/stable/kilo/doc/source/getstarted/ubuntu.rst) which you have suggested that is for openstack vhost user installations with devstack.
Can't we have any reference for including ovs-dpdk mechanisam driver for openstack Ubuntu distribution which we are following for
compute+controller node setup?"

We are facing below listed issues With the current approach of setting up openstack kilo interactively + replacing ovs with ovs-dpdk enabled and Instance creation in openstack with
passing that instance id to QEMU command line which further passes the vhost-user sockets to instances for enabling the DPDK libraries in it.


1. Created a flavor m1.hugepages which is backed by hugepage memory, unable to spawn instance with this flavor – Getting a issue like: No matching hugetlbfs for the number of hugepages assigned to the flavor.
2. Passing socket info to instances via qemu manually and instnaces created are not persistent.

Now as you suggested, we are looking in enabling ovsdpdk ml2 mechanism driver and agent all of that in our openstack ubuntu distribution.

Would be really appriciate if get any help or ref with explanation.

We are using compute + controller node setup and we are using following software platform on compute node:
_____________
Openstack: Kilo
Distribution: Ubuntu 14.04
OVS Version: 2.4.0
DPDK 2.0.0
_____________

Thanks,
Abhijeet Karve





From:        "Czesnowicz, Przemyslaw" <przemyslaw.czesnowicz@intel.com<mailto:przemyslaw.czesnowicz@intel.com>>
To:        Abhijeet Karve <abhijeet.karve@tcs.com<mailto:abhijeet.karve@tcs.com>>
Cc:        "dev@dpdk.org<mailto:dev@dpdk.org>" <dev@dpdk.org<mailto:dev@dpdk.org>>, "discuss@openvswitch.org<mailto:discuss@openvswitch.org>" <discuss@openvswitch.org<mailto:discuss@openvswitch.org>>, "Gray, Mark D" <mark.d.gray@intel.com<mailto:mark.d.gray@intel.com>>
Date:        12/17/2015 06:32 PM
Subject:        RE: [dpdk-dev] DPDK OVS on Ubuntu 14.04# Issue's Resolved# Successfully setup DPDK OVS with vhostuser
________________________________



I haven’t tried that approach not sure if that would work, it seems clunky.

If you enable ovsdpdk ml2 mechanism driver and agent all of that (add ports to ovs with the right type, pass the sockets to qemu) would be done by OpenStack.

Przemek

From: Abhijeet Karve [mailto:abhijeet.karve@tcs.com]
Sent: Thursday, December 17, 2015 12:41 PM
To: Czesnowicz, Przemyslaw
Cc: dev@dpdk.org<mailto:dev@dpdk.org>; discuss@openvswitch.org<mailto:discuss@openvswitch.org>; Gray, Mark D
Subject: RE: [dpdk-dev] DPDK OVS on Ubuntu 14.04# Issue's Resolved# Successfully setup DPDK OVS with vhostuser

Hi Przemek,

Thank you so much for sharing the ref guide.

Would be appreciate if clear one doubt.

At present we are setting up openstack kilo interactively and further replacing ovs with ovs-dpdk enabled.
Once the above setup done, We are creating instance in openstack and passing that instance id to QEMU command line which further passes the vhost-user sockets to instances, enabling the DPDK libraries in it.

Isn't this the correct way of integrating ovs-dpdk with openstack?


Thanks & Regards
Abhijeet Karve




From:        "Czesnowicz, Przemyslaw" <przemyslaw.czesnowicz@intel.com<mailto:przemyslaw.czesnowicz@intel.com>>
To:        Abhijeet Karve <abhijeet.karve@tcs.com<mailto:abhijeet.karve@tcs.com>>
Cc:        "dev@dpdk.org<mailto:dev@dpdk.org>" <dev@dpdk.org<mailto:dev@dpdk.org>>, "discuss@openvswitch.org<mailto:discuss@openvswitch.org>" <discuss@openvswitch.org<mailto:discuss@openvswitch.org>>, "Gray, Mark D" <mark.d.gray@intel.com<mailto:mark.d.gray@intel.com>>
Date:        12/17/2015 05:27 PM
Subject:        RE: [dpdk-dev] DPDK OVS on Ubuntu 14.04# Issue's Resolved# Successfully setup DPDK OVS with vhostuser
________________________________




HI Abhijeet,

For Kilo you need to use ovsdpdk mechanism driver and a matching agent to integrate ovs-dpdk with OpenStack.

The guide you are following only talks about running ovs-dpdk not how it should be integrated with OpenStack.

Please follow this guide:
https://github.com/openstack/networking-ovs-dpdk/blob/stable/kilo/doc/source/getstarted/ubuntu.rst

Best regards
Przemek


From: Abhijeet Karve [mailto:abhijeet.karve@tcs.com]
Sent: Wednesday, December 16, 2015 9:37 AM
To: Czesnowicz, Przemyslaw
Cc: dev@dpdk.org<mailto:dev@dpdk.org>; discuss@openvswitch.org<mailto:discuss@openvswitch.org>; Gray, Mark D
Subject: RE: [dpdk-dev] DPDK OVS on Ubuntu 14.04# Issue's Resolved# Successfully setup DPDK OVS with vhostuser

Hi Przemek,


We have configured the accelerated data path between a physical interface to the VM using openvswitch netdev-dpdk with vhost-user support. The VM created with this special data path and vhost library, I am calling as DPDK instance.

If assigning ip manually to the newly created Cirros VM instance, We are able to make 2 VM's to communicate on the same compute node. Else it's not associating any ip through DHCP though DHCP is in compute node only.

Yes it's a compute + controller node setup and we are using following software platform on compute node:
_____________
Openstack: Kilo
Distribution: Ubuntu 14.04
OVS Version: 2.4.0
DPDK 2.0.0
_____________

We are following the intel guide https://software.intel.com/en-us/blogs/2015/06/09/building-vhost-user-for-ovs-today-using-dpdk-200

When doing "ovs-vsctl show" in compute node, it shows below output:
_____________________________________________
ovs-vsctl show
c2ec29a5-992d-4875-8adc-1265c23e0304
  Bridge br-ex
      Port phy-br-ex
          Interface phy-br-ex
              type: patch
              options: {peer=int-br-ex}
      Port br-ex
          Interface br-ex
              type: internal
  Bridge br-tun
      fail_mode: secure
      Port br-tun
          Interface br-tun
              type: internal
      Port patch-int
          Interface patch-int
              type: patch
              options: {peer=patch-tun}
  Bridge br-int
      fail_mode: secure
      Port "qvo0ae19a43-b6"
          tag: 2
          Interface "qvo0ae19a43-b6"
      Port br-int
          Interface br-int
              type: internal
      Port "qvo31c89856-a2"
          tag: 1
          Interface "qvo31c89856-a2"
      Port patch-tun
          Interface patch-tun
              type: patch
              options: {peer=patch-int}
      Port int-br-ex
          Interface int-br-ex
              type: patch
              options: {peer=phy-br-ex}
      Port "qvo97fef28a-ec"
          tag: 2
          Interface "qvo97fef28a-ec"
  Bridge br-dpdk
      Port br-dpdk
          Interface br-dpdk
              type: internal
  Bridge "br0"
      Port "br0"
          Interface "br0"
              type: internal
      Port "dpdk0"
          Interface "dpdk0"
              type: dpdk
      Port "vhost-user-2"
          Interface "vhost-user-2"
              type: dpdkvhostuser
      Port "vhost-user-0"
          Interface "vhost-user-0"
              type: dpdkvhostuser
      Port "vhost-user-1"
          Interface "vhost-user-1"
              type: dpdkvhostuser
  ovs_version: "2.4.0"
root@dpdk:~#
_____________________________________________

Open flows output in bridge in compute node are as below:
_____________________________________________
root@dpdk:~# ovs-ofctl dump-flows br-tun
NXST_FLOW reply (xid=0x4):
cookie=0x0, duration=71796.741s, table=0, n_packets=519, n_bytes=33794, idle_age=19982, hard_age=65534, priority=1,in_port=1 actions=resubmit(,2)
cookie=0x0, duration=71796.700s, table=0, n_packets=0, n_bytes=0, idle_age=65534, hard_age=65534, priority=0 actions=drop
cookie=0x0, duration=71796.649s, table=2, n_packets=0, n_bytes=0, idle_age=65534, hard_age=65534, priority=0,dl_dst=00:00:00:00:00:00/01:00:00:00:00:00 actions=resubmit(,20)
cookie=0x0, duration=71796.610s, table=2, n_packets=519, n_bytes=33794, idle_age=19982, hard_age=65534, priority=0,dl_dst=01:00:00:00:00:00/01:00:00:00:00:00 actions=resubmit(,22)
cookie=0x0, duration=71794.631s, table=3, n_packets=0, n_bytes=0, idle_age=65534, hard_age=65534, priority=1,tun_id=0x5c actions=mod_vlan_vid:2,resubmit(,10)
cookie=0x0, duration=71794.316s, table=3, n_packets=0, n_bytes=0, idle_age=65534, hard_age=65534, priority=1,tun_id=0x57 actions=mod_vlan_vid:1,resubmit(,10)
cookie=0x0, duration=71796.565s, table=3, n_packets=0, n_bytes=0, idle_age=65534, hard_age=65534, priority=0 actions=drop
cookie=0x0, duration=71796.522s, table=4, n_packets=0, n_bytes=0, idle_age=65534, hard_age=65534, priority=0 actions=drop
cookie=0x0, duration=71796.481s, table=10, n_packets=0, n_bytes=0, idle_age=65534, hard_age=65534, priority=1 actions=learn(table=20,hard_timeout=300,priority=1,NXM_OF_VLAN_TCI[0..11],NXM_OF_ETH_DST[]=NXM_OF_ETH_SRC[],load:0->NXM_OF_VLAN_TCI[],load:NXM_NX_TUN_ID[]->NXM_NX_TUN_ID[],output:NXM_OF_IN_PORT[]),output:1
cookie=0x0, duration=71796.439s, table=20, n_packets=0, n_bytes=0, idle_age=65534, hard_age=65534, priority=0 actions=resubmit(,22)
cookie=0x0, duration=71796.398s, table=22, n_packets=519, n_bytes=33794, idle_age=19982, hard_age=65534, priority=0 actions=drop
root@dpdk:~#
root@dpdk:~#
root@dpdk:~#
root@dpdk:~# ovs-ofctl dump-flows br-tun
int NXST_FLOW reply (xid=0x4):
cookie=0x0, duration=71801.275s, table=0, n_packets=0, n_bytes=0, idle_age=65534, hard_age=65534, priority=2,in_port=10 actions=drop
cookie=0x0, duration=71801.862s, table=0, n_packets=661, n_bytes=48912, idle_age=19981, hard_age=65534, priority=1 actions=NORMAL
cookie=0x0, duration=71801.817s, table=23, n_packets=0, n_bytes=0, idle_age=65534, hard_age=65534, priority=0 actions=drop
root@dpdk:~#
_____________________________________________


Further we don't know what all the network changes(Packet Flow addition) if required for associating IP address through the DHCP.

Would be really appreciate if have clarity on DHCP flow establishment.



Thanks & Regards
Abhijeet Karve





From:        "Czesnowicz, Przemyslaw" <przemyslaw.czesnowicz@intel.com<mailto:przemyslaw.czesnowicz@intel.com>>
To:        Abhijeet Karve <abhijeet.karve@tcs.com<mailto:abhijeet.karve@tcs.com>>, "Gray, Mark D" <mark.d.gray@intel.com<mailto:mark.d.gray@intel.com>>
Cc:        "dev@dpdk.org<mailto:dev@dpdk.org>" <dev@dpdk.org<mailto:dev@dpdk.org>>, "discuss@openvswitch.org<mailto:discuss@openvswitch.org>" <discuss@openvswitch.org<mailto:discuss@openvswitch.org>>
Date:        12/15/2015 09:13 PM
Subject:        RE: [dpdk-dev] DPDK OVS on Ubuntu 14.04# Issue's Resolved# Successfully setup DPDK OVS with vhostuser
________________________________





Hi Abhijeet,

If you answer below questions it will help me understand your problem.

What do you mean by DPDK instance?
Are you able to communicate with other VM's on the same compute node?
Can you check if the DHCP requests arrive on the controller node? (I'm assuming this is at least compute+ controller setup)

Best regards
Przemek

> -----Original Message-----
> From: dev [mailto:dev-bounces@dpdk.org] On Behalf Of Abhijeet Karve
> Sent: Tuesday, December 15, 2015 5:56 AM
> To: Gray, Mark D
> Cc: dev@dpdk.org<mailto:dev@dpdk.org>; discuss@openvswitch.org<mailto:discuss@openvswitch.org>
> Subject: Re: [dpdk-dev] DPDK OVS on Ubuntu 14.04# Issue's Resolved#
> Successfully setup DPDK OVS with vhostuser
>
> Dear All,
>
> After seting up system boot parameters as shown below, the issue is
> resolved now & we are able to successfully setup openvswitch netdev-dpdk
> with vhostuser support.
>
> __________________________________________________________
> _______________________________________________________
> Setup 2 sets of huge pages with different sizes. One for Vhost and another
> for Guest VM.
>          Edit /etc/default/grub.
>             GRUB_CMDLINE_LINUX="iommu=pt intel_iommu=on  hugepagesz=1G
> hugepages=10 hugepagesz=2M hugepages=4096"
>          # update-grub
>        - Mount the huge pages into different directory.
>           # sudo mount -t hugetlbfs nodev /mnt/huge_2M -o pagesize=2M
>           # sudo mount -t hugetlbfs nodev /mnt/huge_1G -o pagesize=1G
> __________________________________________________________
> _______________________________________________________
>
> At present we are facing an issue in Testing DPDK application on setup. In our
> scenario, We have DPDK instance launched on top of the Openstack Kilo
> compute node. Not able to assign DHCP IP from controller.
>
>
> Thanks & Regards
> Abhijeet Karve
>
> =====-----=====-----=====
> Notice: The information contained in this e-mail message and/or
> attachments to it may contain confidential or privileged information. If you
> are not the intended recipient, any dissemination, use, review, distribution,
> printing or copying of the information contained in this e-mail message
> and/or attachments to it are strictly prohibited. If you have received this
> communication in error, please notify us by reply e-mail or telephone and
> immediately and permanently delete the message and any attachments.
> Thank you
>