From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mga02.intel.com (mga02.intel.com [134.134.136.20]) by dpdk.org (Postfix) with ESMTP id 0F1117E8E for ; Mon, 10 Nov 2014 10:44:23 +0100 (CET) Received: from orsmga001.jf.intel.com ([10.7.209.18]) by orsmga101.jf.intel.com with ESMTP; 10 Nov 2014 01:54:06 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.07,351,1413270000"; d="scan'208";a="605210142" Received: from irsmsx102.ger.corp.intel.com ([163.33.3.155]) by orsmga001.jf.intel.com with ESMTP; 10 Nov 2014 01:53:58 -0800 Received: from irsmsx153.ger.corp.intel.com (163.33.192.75) by IRSMSX102.ger.corp.intel.com (163.33.3.155) with Microsoft SMTP Server (TLS) id 14.3.195.1; Mon, 10 Nov 2014 09:53:23 +0000 Received: from irsmsx109.ger.corp.intel.com ([169.254.13.101]) by IRSMSX153.ger.corp.intel.com ([169.254.9.23]) with mapi id 14.03.0195.001; Mon, 10 Nov 2014 09:53:23 +0000 From: "Burakov, Anatoly" To: Liang Xu , "dev@dpdk.org" Thread-Topic: [PATCH] eal: map PCI memory resources after hugepages Thread-Index: AQHP+wSiEAb9hfd8fkOi8Y61mgTq6JxZoD9Q Date: Mon, 10 Nov 2014 09:53:22 +0000 Message-ID: References: <1415417532-4363-1-git-send-email-liang.xu@cinfotech.cn> In-Reply-To: <1415417532-4363-1-git-send-email-liang.xu@cinfotech.cn> Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-originating-ip: [163.33.239.181] Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Subject: Re: [dpdk-dev] [PATCH] eal: map PCI memory resources after hugepages X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: patches and discussions about DPDK List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 10 Nov 2014 09:44:25 -0000 Hi Liang I don't think that overriding the value passed to pci_map_resource as argum= ent is the way to go. While it results in less code, it looks weird, in my = opinion at least, as I believe tracking the correctness of address being re= quested should be the responsibility of the caller, i.e. either UIO or VFIO= code. Which is why I keep insisting that you make requested_pci_addr globa= l to linuxapp EAL PCI section and put it into include/eal_pci_init.h. Would= you mind if I made a patch for this issue based on your code? Thanks, Anatoly -----Original Message----- From: Liang Xu [mailto:liang.xu@cinfotech.cn]=20 Sent: Saturday, November 8, 2014 3:32 AM To: dev@dpdk.org Cc: Burakov, Anatoly; thomas.monjalon@6wind.com Subject: [PATCH] eal: map PCI memory resources after hugepages A multiple process DPDK application must mmap hugepages and pci resources i= nto same virtual addresses. By default the virtual addresses chosen by the = primary process automatically when calling the mmap. But sometime the chose= n virtual addresses isn't usable at secondary process. Such as the secondar= y process linked with more libraries than primary process. The library has = been mapped into this virtual address. The command line parameter 'base-vir= taddr' has been added for this situation. If it's configured, the hugepages= will be mapped into this base address. But the virtual address of pci reso= urces mapped still does not refer to the parameter. In that case "EAL: pci_= map_resource(): cannot mmap"=20 will be got. This patch try to map pci resources after hugepages. So the error can be re= solved by set base-virtaddr into free virtual address space. Signed-off-by: Liang Xu --- lib/librte_eal/linuxapp/eal/eal_pci.c | 32 +++++++++++++++++++++++++++++++= - 1 file changed, 31 insertions(+), 1 deletion(-) diff --git a/lib/librte_eal/linuxapp/eal/eal_pci.c b/lib/librte_eal/linuxap= p/eal/eal_pci.c index ddb0535..502eef2 100644 --- a/lib/librte_eal/linuxapp/eal/eal_pci.c +++ b/lib/librte_eal/linuxapp/eal/eal_pci.c @@ -97,14 +97,42 @@ error: return -1; } =20 +static void * +pci_find_max_end_va(void) +{ + const struct rte_memseg *seg =3D rte_eal_get_physmem_layout(); + const struct rte_memseg *last =3D seg; + unsigned i =3D 0; + + for (i =3D 0; i < RTE_MAX_MEMSEG; i++, seg++) { + if (seg->addr =3D=3D NULL) + break; + + if (seg->addr > last->addr) + last =3D seg; + + } + return RTE_PTR_ADD(last->addr, last->len); } + /* map a particular resource from a file */ void * pci_map_resource(void= *requested_addr, int fd, off_t offset, size_t size) { void *mapaddr; =20 + /* By default the PCI memory resource will be mapped after hugepages */ + static void *default_map_addr; + if (NULL =3D=3D requested_addr) { + if (NULL =3D=3D default_map_addr) + default_map_addr =3D pci_find_max_end_va(); + mapaddr =3D default_map_addr; + } else { + mapaddr =3D requested_addr; + } + /* Map the PCI memory resource of device */ - mapaddr =3D mmap(requested_addr, size, PROT_READ | PROT_WRITE, + mapaddr =3D mmap(mapaddr, size, PROT_READ | PROT_WRITE, MAP_SHARED, fd, offset); if (mapaddr =3D=3D MAP_FAILED || (requested_addr !=3D NULL && mapaddr !=3D requested_addr)) { @@ -114,6 = +142,8 @@ pci_map_resource(void *requested_addr, int fd, off_t offset, size= _t size) strerror(errno), mapaddr); goto fail; } + if (NULL =3D=3D requested_addr) + default_map_addr =3D RTE_PTR_ADD(mapaddr, size); =20 RTE_LOG(DEBUG, EAL, " PCI memory mapped at %p\n", mapaddr); =20 -- 1.9.1