From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from out1134-233.mail.aliyun.com (out1134-233.mail.aliyun.com [42.120.134.233]) by dpdk.org (Postfix) with ESMTP id A51B3590B for ; Sat, 8 Nov 2014 04:22:54 +0100 (CET) X-Alimail-AntiSpam: AC=CONTINUE; BC=0.07771052|-1; FP=0|0|0|0|0|-1|-1|-1; HT=r46d02008; MF=liang.xu@cinfotech.cn; PH=DS; RN=3; RT=3; SR=0; Received: from localhost.localdomain(mailfrom:liang.xu@cinfotech.cn ip:222.65.239.251) by smtp.aliyun-inc.com(10.147.11.247); Sat, 08 Nov 2014 11:32:24 +0800 From: Liang Xu To: dev@dpdk.org Date: Sat, 8 Nov 2014 11:32:12 +0800 Message-Id: <1415417532-4363-1-git-send-email-liang.xu@cinfotech.cn> X-Mailer: git-send-email 1.9.1 Subject: [dpdk-dev] [PATCH] eal: map PCI memory resources after hugepages X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: patches and discussions about DPDK List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 08 Nov 2014 03:22:56 -0000 A multiple process DPDK application must mmap hugepages and pci resources into same virtual addresses. By default the virtual addresses chosen by the primary process automatically when calling the mmap. But sometime the chosen virtual addresses isn't usable at secondary process. Such as the secondary process linked with more libraries than primary process. The library has been mapped into this virtual address. The command line parameter 'base-virtaddr' has been added for this situation. If it's configured, the hugepages will be mapped into this base address. But the virtual address of pci resources mapped still does not refer to the parameter. In that case "EAL: pci_map_resource(): cannot mmap" will be got. This patch try to map pci resources after hugepages. So the error can be resolved by set base-virtaddr into free virtual address space. Signed-off-by: Liang Xu --- lib/librte_eal/linuxapp/eal/eal_pci.c | 32 +++++++++++++++++++++++++++++++- 1 file changed, 31 insertions(+), 1 deletion(-) diff --git a/lib/librte_eal/linuxapp/eal/eal_pci.c b/lib/librte_eal/linuxapp/eal/eal_pci.c index ddb0535..502eef2 100644 --- a/lib/librte_eal/linuxapp/eal/eal_pci.c +++ b/lib/librte_eal/linuxapp/eal/eal_pci.c @@ -97,14 +97,42 @@ error: return -1; } +static void * +pci_find_max_end_va(void) +{ + const struct rte_memseg *seg = rte_eal_get_physmem_layout(); + const struct rte_memseg *last = seg; + unsigned i = 0; + + for (i = 0; i < RTE_MAX_MEMSEG; i++, seg++) { + if (seg->addr == NULL) + break; + + if (seg->addr > last->addr) + last = seg; + + } + return RTE_PTR_ADD(last->addr, last->len); +} + /* map a particular resource from a file */ void * pci_map_resource(void *requested_addr, int fd, off_t offset, size_t size) { void *mapaddr; + /* By default the PCI memory resource will be mapped after hugepages */ + static void *default_map_addr; + if (NULL == requested_addr) { + if (NULL == default_map_addr) + default_map_addr = pci_find_max_end_va(); + mapaddr = default_map_addr; + } else { + mapaddr = requested_addr; + } + /* Map the PCI memory resource of device */ - mapaddr = mmap(requested_addr, size, PROT_READ | PROT_WRITE, + mapaddr = mmap(mapaddr, size, PROT_READ | PROT_WRITE, MAP_SHARED, fd, offset); if (mapaddr == MAP_FAILED || (requested_addr != NULL && mapaddr != requested_addr)) { @@ -114,6 +142,8 @@ pci_map_resource(void *requested_addr, int fd, off_t offset, size_t size) strerror(errno), mapaddr); goto fail; } + if (NULL == requested_addr) + default_map_addr = RTE_PTR_ADD(mapaddr, size); RTE_LOG(DEBUG, EAL, " PCI memory mapped at %p\n", mapaddr); -- 1.9.1