From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from out1134-202.mail.aliyun.com (out1134-202.mail.aliyun.com [42.120.134.202]) by dpdk.org (Postfix) with ESMTP id 8A38B7F2C for ; Fri, 7 Nov 2014 16:10:00 +0100 (CET) X-Alimail-AntiSpam: AC=CONTINUE; BC=0.0234163|-1; FP=0|0|0|0|0|-1|-1|-1; HT=r41g08146; MF=liang.xu@cinfotech.cn; PH=DW; RN=4; RT=4; SR=0; Received: from WS-web (liang.xu@cinfotech.cn[222.65.239.251]) by r41g06010.xy2.aliyun.com at Fri, 07 Nov 2014 23:19:26 +0800 Date: Fri, 07 Nov 2014 23:19:26 +0800 From: "XU Liang" To: "=?UTF-8?B?QnVyYWtvdiwgQW5hdG9seQ==?=" , "dev@dpdk.org" Message-ID: X-Mailer: Alimail-Mailagent revision 2667797 MIME-Version: 1.0 References: <1415193919-17361-1-git-send-email-liang.xu@cinfotech.cn> <1415372269-8723-1-git-send-email-liang.xu@cinfotech.cn>, C6ECDF3AB251BE4894318F4E4512369780C07A7E@IRSMSX109.ger.corp.intel.com In-Reply-To: C6ECDF3AB251BE4894318F4E4512369780C07A7E@IRSMSX109.ger.corp.intel.com x-aliyun-mail-creator: Webmail4_2670074_M3LTW96aWxsYS81LjAgKE1hY2ludG9zaDsgSW50ZWwgTWFjIE9TIFggMTBfMTBfMSkgQXBwbGVXZWJLaXQvNTM3LjM2IChLSFRNTCwgbGlrZSBHZWNrbykgQ2hyb21lLzM4LjAuMjEyNS4xMTEgU2FmYXJpLzUzNy4zNg==vN Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Content-Filtered-By: Mailman/MimeDel 2.1.15 Subject: Re: [dpdk-dev] =?utf-8?q?=5BPATCH_v7=5D_eal=3A_map_uio_resources_afte?= =?utf-8?q?r_hugepages=2E?= X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list Reply-To: XU Liang List-Id: patches and discussions about DPDK List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 07 Nov 2014 15:10:02 -0000 I got it. Just some guys don't like global variable. I'm not sure what is DPDK= code style.------------------------------------------------------------------= From:Burakov, Anatoly Time:2014 Nov 7 (Fri) 23 : 15= To:=E5=BE=90=E4=BA=AE , dev@dpdk.org Cc:t= homas.monjalon@6wind.com , De Lara Guarch, Pablo Subject:RE: [PATCH v7] eal: map uio resources af= ter hugepages.=0AUm, not sure I agree with this implementation. I think a clea= ner way would be to put the requested_addr in pci_uio_map_resource and pci_vfi= o_map_resource (or rather, put it in include/eal_pci_init.h, like extern void = *requested_addr) but make actual use of it in pci_uio/vfio_map_resource only (= and leave all of this out of eal_pci.c at all). That will also rid you of the = necessity to pass around pointers to pointers.=0A=0A(in that case I would also= rename requested_addr to pci_map_addr or something, to make it less vague) =0A= =0AThanks,=0AAnatoly=0A=0A-----Original Message-----=0AFrom: lxu [mailto:liang= .xu@cinfotech.cn] =0ASent: Friday, November 7, 2014 2:58 PM=0ATo: dev@dpdk.org= =0ACc: Burakov, Anatoly; thomas.monjalon@6wind.com; De Lara Guarch, Pablo=0ASu= bject: [PATCH v7] eal: map uio resources after hugepages.=0A=0AA multiple proc= ess DPDK application must mmap hugepages and pci resources into same virtual a= ddresses. By default the virtual addresses chosen by the primary process autom= atically when calling the mmap. But sometime the chosen virtual addresses isn'= t usable at secondary process. Such as the secondary process linked with more = libraries than primary process. The library has been mapped into this virtual = address. The command line parameter 'base-virtaddr' has been added for this si= tuation. If it's configured, the hugepages will be mapped into this base addre= ss. But the virtual address of uio resource mapped still does not refer to the= parameter. In that case "EAL: pci_map_resource(): cannot mmap" will be got.=0A= =0AThis patch try to map uio resources after hugepages. So the error can be re= solved by set base-virtaddr into free virtual address space.=0A=0ASigned-off-b= y: lxu =0A---=0A lib/librte_eal/linuxapp/eal/eal_pci.c = | 25 ++++++++++++++++++++--=0A lib/librte_eal/linuxapp/eal/eal_pc= i_uio.c | 6 ++++--=0A lib/librte_eal/linuxapp/eal/eal_pci_vfio.c = | 4 +++-=0A lib/librte_eal/linuxapp/eal/include/eal_pci_init.h | 4 ++--= =0A 4 files changed, 32 insertions(+), 7 deletions(-)=0A=0Adiff --git a/lib/li= brte_eal/linuxapp/eal/eal_pci.c b/lib/librte_eal/linuxapp/eal/eal_pci.c=0Ainde= x 5fe3961..aef6f5e 100644=0A--- a/lib/librte_eal/linuxapp/eal/eal_pci.c=0A+++ = b/lib/librte_eal/linuxapp/eal/eal_pci.c=0A@@ -483,15 +483,36 @@ pci_config_spa= ce_set(struct rte_pci_device *dev) } #endif=0A =0A+static void *=0A+pci_find= _max_end_va(void)=0A+{=0A+ const struct rte_memseg * seg =3D rte_eal_get_physm= em_layout();=0A+ const struct rte_memseg * last =3D seg;=0A+ unsigned i =3D 0;= =0A+=0A+ for (i=3D0; iaddr =3D=3D= NULL)=0A+ break;=0A+=0A+ if(seg->addr > last->addr)=0A+ last =3D seg;=0A= +=0A+ }=0A+ return RTE_PTR_ADD(last->addr, last->len); }=0A+=0A static int=0A = pci_map_device(struct rte_pci_device *dev) {=0A int ret, mapped =3D 0;=0A+ s= tatic void * requested_addr;=0A+ if(NULL =3D=3D requested_addr)=0A+ requested= _addr =3D pci_find_max_end_va();=0A =0A /* try mapping the NIC resources usin= g VFIO if it exists */ #ifdef VFIO_PRESENT=0A if (pci_vfio_is_enabled()) {=0A= - ret =3D pci_vfio_map_resource(dev);=0A+ ret =3D pci_vfio_map_resource(dev,= &requested_addr);=0A if (ret =3D=3D 0)=0A mapped =3D 1;=0A else if (re= t < 0)=0A@@ -500,7 +521,7 @@ pci_map_device(struct rte_pci_device *dev) #endi= f=0A /* map resources for devices that use igb_uio */=0A if (!mapped) {=0A- = ret =3D pci_uio_map_resource(dev);=0A+ ret =3D pci_uio_map_resource(dev, &re= quested_addr);=0A if (ret !=3D 0)=0A return ret;=0A }=0Adiff --git a/lib= /librte_eal/linuxapp/eal/eal_pci_uio.c b/lib/librte_eal/linuxapp/eal/eal_pci_u= io.c=0Aindex 7e62266..e92124e 100644=0A--- a/lib/librte_eal/linuxapp/eal/eal_p= ci_uio.c=0A+++ b/lib/librte_eal/linuxapp/eal/eal_pci_uio.c=0A@@ -275,7 +275,7 = @@ pci_get_uio_dev(struct rte_pci_device *dev, char *dstbuf,=0A =0A /* map the= PCI resource of a PCI device in virtual memory */ int -pci_uio_map_resource(= struct rte_pci_device *dev)=0A+pci_uio_map_resource(struct rte_pci_device *dev= , void **requested_addr)=0A {=0A int i, j;=0A char dirname[PATH_MAX];=0A@@ -= 371,10 +371,12 @@ pci_uio_map_resource(struct rte_pci_device *dev)=0A if (m= aps[j].addr !=3D NULL)=0A fail =3D 1;=0A else {=0A- mapaddr =3D pci_= map_resource(NULL, fd, (off_t)offset,=0A+ mapaddr =3D pci_map_resource(*req= uested_addr, fd, (off_t)offset,=0A (size_t)maps[j].size);=0A if (map= addr =3D=3D NULL)=0A fail =3D 1;=0A+ else=0A+ *requested_addr =3D = RTE_PTR_ADD(mapaddr, maps[j].size);=0A }=0A =0A if (fail) {=0Adiff --git= a/lib/librte_eal/linuxapp/eal/eal_pci_vfio.c b/lib/librte_eal/linuxapp/eal/ea= l_pci_vfio.c=0Aindex c776ddc..2102adf 100644=0A--- a/lib/librte_eal/linuxapp/e= al/eal_pci_vfio.c=0A+++ b/lib/librte_eal/linuxapp/eal/eal_pci_vfio.c=0A@@ -515= ,7 +515,7 @@ clear_current_group(void)=0A * primary and secondary processes f= ollow almost exactly the same path=0A */=0A int=0A-pci_vfio_map_resource(stru= ct rte_pci_device *dev)=0A+pci_vfio_map_resource(struct rte_pci_device *dev, v= oid =0A+**requested_addr)=0A {=0A struct vfio_group_status group_status =3D {= =0A .argsz =3D sizeof(group_status)=0A@@ -720,6 +720,7 @@ pci_vfio_map_reso= urce(struct rte_pci_device *dev)=0A if (i =3D=3D msix_bar)=0A continue;=0A= =0A+ maps[i].addr =3D *requested_addr;=0A bar_addr =3D pci_map_resource(ma= ps[i].addr, vfio_dev_fd, reg.offset,=0A reg.size);=0A =0A@@ -732,6 +733,7 = @@ pci_vfio_map_resource(struct rte_pci_device *dev)=0A return -1;=0A }=0A= =0A+ *requested_addr =3D bar_addr;=0A maps[i].addr =3D bar_addr;=0A maps= [i].offset =3D reg.offset;=0A maps[i].size =3D reg.size;=0Adiff --git a/lib/= librte_eal/linuxapp/eal/include/eal_pci_init.h b/lib/librte_eal/linuxapp/eal/i= nclude/eal_pci_init.h=0Aindex d758bee..e14fa36 100644=0A--- a/lib/librte_eal/l= inuxapp/eal/include/eal_pci_init.h=0A+++ b/lib/librte_eal/linuxapp/eal/include= /eal_pci_init.h=0A@@ -63,7 +63,7 @@ void *pci_map_resource(void *requested_add= r, int fd, off_t offset,=0A size_t size);=0A =0A /* map IGB_UIO resource pro= totype */=0A-int pci_uio_map_resource(struct rte_pci_device *dev);=0A+int pci_= uio_map_resource(struct rte_pci_device *dev, void =0A+**requested_addr);=0A =0A= #ifdef VFIO_PRESENT=0A =0A@@ -74,7 +74,7 @@ int pci_vfio_is_enabled(void); i= nt pci_vfio_mp_sync_setup(void);=0A =0A /* map VFIO resource prototype */=0A-i= nt pci_vfio_map_resource(struct rte_pci_device *dev);=0A+int pci_vfio_map_reso= urce(struct rte_pci_device *dev, void =0A+**requested_addr);=0A int pci_vfio_g= et_group_fd(int iommu_group_fd); int pci_vfio_get_container_fd(void);=0A =0A-= -=0A1.9.1 >From bruce.richardson@intel.com Fri Nov 7 16:21:25 2014 Return-Path: Received: from mga01.intel.com (mga01.intel.com [192.55.52.88]) by dpdk.org (Postfix) with ESMTP id 0ACF67F29 for ; Fri, 7 Nov 2014 16:21:24 +0100 (CET) Received: from fmsmga001.fm.intel.com ([10.253.24.23]) by fmsmga101.fm.intel.com with ESMTP; 07 Nov 2014 07:30:56 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.07,333,1413270000"; d="scan'208";a="618893192" Received: from gcgray-mobl3.ger.corp.intel.com ([10.252.29.251]) by fmsmga001.fm.intel.com with SMTP; 07 Nov 2014 07:30:54 -0800 Received: by (sSMTP sendmail emulation); Fri, 07 Nov 2014 15:30:53 +0025 Date: Fri, 7 Nov 2014 15:30:53 +0000 From: Bruce Richardson To: Manoj Viswanath Message-ID: <20141107153053.GA10376@bricha3-MOBL3> References: <20141105101246.GA9856@bricha3-MOBL3> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: Organization: Intel Shannon Ltd. User-Agent: Mutt/1.5.23 (2014-03-12) Cc: dev@dpdk.org Subject: Re: [dpdk-dev] Ports not detected by IGB_UIO in DPDK 1.7.1 in QEMU_KVM environment X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: patches and discussions about DPDK List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 07 Nov 2014 15:21:26 -0000 On Fri, Nov 07, 2014 at 08:31:34PM +0530, Manoj Viswanath wrote: > Hi Bruce, > > I was not doing anything specific for binding the NICs to IGB_UIO (like > invoking "dpdk_nic_bind.py" script explicitly) when using my application > with DPDK 1.6.0. The e1000 devices assigned via virt-manager to the VM were > automatically getting picked up and initialized by IGB_UIO within each VM. > > The same is not working with DPDK 1.7.1 now. > > I tried exporting the "dpdk_nic_bind.py" script into my VM (running DPDK > 1.7.1) and tried to check the status. The emulated devices were shown as > neither bound to kernel nor to IGB_UIO as evident from below output:- > > <---------------------------------------------------------------------------------------------------> > Network devices using DPDK-compatible driver > ============================================ > > > Network devices using kernel driver > =================================== > 0000:00:03.0 'Virtio network device' if= drv=virtio-pci unused=igb_uio > > Other network devices > ===================== > 0000:00:04.0 '82540EM Gigabit Ethernet Controller' unused=igb_uio > 0000:00:05.0 '82540EM Gigabit Ethernet Controller' unused=igb_uio > <---------------------------------------------------------------------------------------------------> > > When i tried to forcefully bind the NICs using the "--bind=igb_uio" option Was there any output of the dpdk_nic_bind script? What does the output of it with --status show afterwards? Regards, /Bruce