From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mga02.intel.com (mga02.intel.com [134.134.136.20]) by dpdk.org (Postfix) with ESMTP id C68CC7F24 for ; Fri, 10 Oct 2014 02:19:35 +0200 (CEST) Received: from orsmga002.jf.intel.com ([10.7.209.21]) by orsmga101.jf.intel.com with ESMTP; 09 Oct 2014 17:26:58 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.04,688,1406617200"; d="scan'208";a="616142366" Received: from fmsmsx106.amr.corp.intel.com ([10.18.124.204]) by orsmga002.jf.intel.com with ESMTP; 09 Oct 2014 17:26:58 -0700 Received: from fmsmsx101.amr.corp.intel.com (10.18.124.199) by FMSMSX106.amr.corp.intel.com (10.18.124.204) with Microsoft SMTP Server (TLS) id 14.3.195.1; Thu, 9 Oct 2014 17:26:58 -0700 Received: from shsmsx104.ccr.corp.intel.com (10.239.4.70) by fmsmsx101.amr.corp.intel.com (10.18.124.199) with Microsoft SMTP Server (TLS) id 14.3.195.1; Thu, 9 Oct 2014 17:26:57 -0700 Received: from shsmsx101.ccr.corp.intel.com ([169.254.1.203]) by SHSMSX104.ccr.corp.intel.com ([169.254.5.230]) with mapi id 14.03.0195.001; Fri, 10 Oct 2014 08:26:55 +0800 From: "Xie, Huawei" To: "Murthy, Krishna J" , "dev@dpdk.org" Thread-Topic: [dpdk-dev] [README]:Running DPDK in a LXC-based Container Thread-Index: Ac/i/d7CY64D/WpN3EueVAbqy3oGTABHXQKw Date: Fri, 10 Oct 2014 00:26:55 +0000 Message-ID: References: <910C5016C9E77747B5B12EF34B54ADA92A3FA2CE@IRSMSX108.ger.corp.intel.com> In-Reply-To: <910C5016C9E77747B5B12EF34B54ADA92A3FA2CE@IRSMSX108.ger.corp.intel.com> Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-originating-ip: [10.239.127.40] Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Subject: Re: [dpdk-dev] [README]:Running DPDK in a LXC-based Container X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: patches and discussions about DPDK List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 10 Oct 2014 00:19:36 -0000 Here is the docker step, it is quite straightforward. Basically they are th= e same as LXC. UIO: docker run -i -t --device=3D/dev/uio0:/dev/uio0 --device=3D/dev/uio1:/dev/= uio1 -v /mnt:/mnt fedora20-dpdk-2 /bin/bash VFIO: docker run -i -t --privileged --device=3D/dev/vfio/vfio:/dev/vfio/vfio --= device=3D/dev/vfio/22:/dev/vfio/22 -v /mnt:/mnt -v /var/run:/var/run f= edora20-dpdk-2 /bin/bash=20 There are open/ioctl calls on vfio device files, and I haven't figured out = exactly the capability we need to give to docker for each sys call, otherwi= se we could remove --privilege option. MP process: With /var/run or /var/run/.rte_* mounted to docker, we could run multiple p= rocess between dockers. =20 (We could generate the dpdk cfg files in /var/run/dpdk, then we could mount= whole directories to docker without specify each file). > -----Original Message----- > From: dev [mailto:dev-bounces@dpdk.org] On Behalf Of Murthy, Krishna J > Sent: Wednesday, October 08, 2014 6:44 AM > To: dev@dpdk.org > Subject: [dpdk-dev] [README]:Running DPDK in a LXC-based Container >=20 > Running DPDK in a LXC-based Container > =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D > =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D > Author: Krishnamurthy Jambur >=20 > Note: This README assumes you have the a LXC based container running on > the IA platform. This was tested on Windriver OVP Linux 6.0 >=20 > In this README we are will go through some of the unique steps involved > in attaching a NIC to the LXC-based Container and the running a sample > DPDP application in the container. >=20 > 1. Attaching NIC device to the container > ---------------------------------------- > Let's assume for this example that you would like to have a management > interface > and then separate dedicated 10G NIC to run DPDK. With this requirement yo= ur > LXC > config file should have one networking sections one for veth. For the 10G= NIC > we > need to use uio device file system >=20 > veth uses a paired set of network interfaces, one inside the container sp= ecified > by the option lxc.network.name, and one in the host specified by the opti= on > lxc.network.pair. The latter is attached automatically to a bridge in the= host > if the name of the bridge is specified by the option lxc.network.link. >=20 > # networking-veth > lxc.network.type =3D veth > lxc.network.link =3D virbr0 > lxc.network.veth.pair =3D vc0 > lxc.network.name =3D eth0 > lxc.network.hwaddr =3D ac:de:48:34:01:01 > lxc.network.flags =3D up > lxc.network.ipv4 =3D 192.168.122.12/24 > lxc.network.ipv4.gateway =3D 192.168.122.1 >=20 > once you have this your LXC container will have an IP using bridge. Next = step > is to install DPDK igb_uio on the host and then bind it to the the 10G Ni= antic > NIC ports as usual. Once installed do ls -l /dev/uio* on the host. These = are the > uioX interface corresponding to the the NIC ports to which DPDK igb_uio i= s > binded. DPDK application will use these device files. In order to provide= access > to these device files inside the LXC container not down the major number = [e.g. > in this case 249] and add this config to the LXC config file >=20 > lxc.cgroup.devices.allow =3D c 249:* rwm >=20 > once you start the container then you need to create these uio0, uio1,... > devices nodes inside the container file system using mknod command as usu= al > with the same major and minor numbers. >=20 > mknod /dev/uio3 c 249 0 > mknod /dev/uio3 c 249 3 > mknod /dev/uio2 c 249 2 > mknod /dev/uio1 c 249 1 >=20 > 2. Host allocated Huge page access inside container > ---------------------------------------------------- > For this set it is assumes that you would have pre-allocated hugepages an= d > mounted on the on the host. Once you have this set-up then you need to pr= ovide > access to hugetlbfs file system to the container using the LXC config fil= e. >=20 > lxc.mount.entry =3D /mnt/huge mnt/huge none bind,create=3Ddir 0 0 >=20 > once you have this DPDK application will be able to use the host mounted = huge > pages in the container. >=20 > Next step is run the DPDK application as usual in the LXC Container >=20 > Regards, > Krishna >=20 > -------------------------------------------------------------- > Intel Shannon Limited > Registered in Ireland > Registered Office: Collinstown Industrial Park, Leixlip, County Kildare > Registered Number: 308263 > Business address: Dromore House, East Park, Shannon, Co. Clare >=20 > This e-mail and any attachments may contain confidential material for the= sole > use of the intended recipient(s). Any review or distribution by others is= strictly > prohibited. If you are not the intended recipient, please contact the sen= der and > delete all copies. >=20