[dpdk-dev] 答复：[PATCH v2] eal: map uio resources after hugepages when the base_virtaddr is configured.

[徐亮 <liang.xu@cinfotech.cn>]

When user configure base_virtaddr, we should believe they can take care it.
In my case, I always check /proc/xxxx/maps to find a huge free address space, such as 0x20 0000 0000, to map all the hugepages and uio resource. 
------------------------------------------------------------------发件人：Burakov, Anatoly <anatoly.burakov@intel.com>发送时间：2014年11月6日(星期四) 22:29收件人：徐亮 <liang.xu@cinfotech.cn>，dev@dpdk.org <dev@dpdk.org>主　题：RE: [PATCH v2] eal: map uio resources after hugepages when the base_virtaddr is configured.
Few nitpicks.

Static variables are always initialized to 0, so "= NULL" isn't necessary, a declaration will suffice. Also, we have a macro RTE_PTR_ADD to add numbers to pointers, I think it would be better to use those. Otherwise, looks fine to me.

I still feel uneasy about depending on nothing being mapped directly after hugepages (perhaps we could do mmap(bar_size) before trying pci_map_resource, and increment requested_addr until we find a free spot?), but I imagine this case would be quite rare, so probably it's not worth the added kludge.

Thanks,
Anatoly

-----Original Message-----
From: lxu [mailto:liang.xu@cinfotech.cn] 
Sent: Thursday, November 6, 2014 2:12 PM
To: dev@dpdk.org
Cc: Burakov, Anatoly
Subject: [PATCH v2] eal: map uio resources after hugepages when the base_virtaddr is configured.

---
 lib/librte_eal/linuxapp/eal/eal_pci_uio.c | 29 ++++++++++++++++++++++++++++-
 1 file changed, 28 insertions(+), 1 deletion(-)

diff --git a/lib/librte_eal/linuxapp/eal/eal_pci_uio.c b/lib/librte_eal/linuxapp/eal/eal_pci_uio.c
index 7e62266..a591da3 100644
--- a/lib/librte_eal/linuxapp/eal/eal_pci_uio.c
+++ b/lib/librte_eal/linuxapp/eal/eal_pci_uio.c
@@ -273,6 +273,24 @@ pci_get_uio_dev(struct rte_pci_device *dev, char *dstbuf,
 	return uio_num;
 }
 
+static inline const struct rte_memseg *
+get_physmem_last(void)
+{
+	const struct rte_memseg * seg = rte_eal_get_physmem_layout();
+	const struct rte_memseg * last = seg;
+	unsigned i = 0;
+
+	for (i=0; i<RTE_MAX_MEMSEG; i++, seg++) {
+		if (seg->addr == NULL)
+			break;
+
+		if(seg->addr > last->addr)
+		 	last = seg;
+
+	}
+	return last;
+}
+
 /* map the PCI resource of a PCI device in virtual memory */
 int
 pci_uio_map_resource(struct rte_pci_device *dev)
@@ -290,6 +308,13 @@ pci_uio_map_resource(struct rte_pci_device *dev)
 	struct mapped_pci_resource *uio_res;
 	struct pci_map *maps;
 
+	/* map uio resource into user required virtual address */
+	static void * requested_addr = NULL;
+	if (internal_config.base_virtaddr && NULL == requested_addr) {
+		const struct rte_memseg * last = get_physmem_last();
+		requested_addr = (void *)(last->addr_64 + last->len);
+	}
+
 	dev->intr_handle.fd = -1;
 	dev->intr_handle.type = RTE_INTR_HANDLE_UNKNOWN;
 
@@ -371,10 +396,12 @@ pci_uio_map_resource(struct rte_pci_device *dev)
 			if (maps[j].addr != NULL)
 				fail = 1;
 			else {
-				mapaddr = pci_map_resource(NULL, fd, (off_t)offset,
+				mapaddr = pci_map_resource(requested_addr, fd, (off_t)offset,
 						(size_t)maps[j].size);
 				if (mapaddr == NULL)
 					fail = 1;
+				else if (NULL != requested_addr)
+					requested_addr = (uint8_t *)mapaddr + maps[j].size;
 			}
 
 			if (fail) {
-- 
1.9.1
From anatoly.burakov@intel.com  Thu Nov  6 15:54:34 2014
Return-Path: <anatoly.burakov@intel.com>
Received: from mga01.intel.com (mga01.intel.com [192.55.52.88])
 by dpdk.org (Postfix) with ESMTP id 23B417E75
 for <dev@dpdk.org>; Thu,  6 Nov 2014 15:54:31 +0100 (CET)
Received: from fmsmga001.fm.intel.com ([10.253.24.23])
 by fmsmga101.fm.intel.com with ESMTP; 06 Nov 2014 07:02:45 -0800
X-ExtLoop1: 1
X-IronPort-AV: E=Sophos;i="5.07,326,1413270000"; d="scan'208";a="618245383"
Received: from irsmsx101.ger.corp.intel.com ([163.33.3.153])
 by fmsmga001.fm.intel.com with ESMTP; 06 Nov 2014 06:59:25 -0800
Received: from irsmsx107.ger.corp.intel.com (163.33.3.99) by
 IRSMSX101.ger.corp.intel.com (163.33.3.153) with Microsoft SMTP Server (TLS)
 id 14.3.195.1; Thu, 6 Nov 2014 14:58:10 +0000
Received: from irsmsx109.ger.corp.intel.com ([169.254.13.101]) by
 IRSMSX107.ger.corp.intel.com ([169.254.10.154]) with mapi id 14.03.0195.001;
 Thu, 6 Nov 2014 14:58:09 +0000
From: "Burakov, Anatoly" <anatoly.burakov@intel.com>
To: lxu <liang.xu@cinfotech.cn>, "dev@dpdk.org" <dev@dpdk.org>
Thread-Topic: [PATCH v3] eal: map uio resources after hugepages when the
 base_virtaddr is configured.
Thread-Index: AQHP+dCl0ziZwg+l7UqDsOao12IOWZxTsFEg
Date: Thu, 6 Nov 2014 14:58:09 +0000
Message-ID: <C6ECDF3AB251BE4894318F4E4512369780C075E8@IRSMSX109.ger.corp.intel.com>
References: <1415193919-17361-1-git-send-email-liang.xu@cinfotech.cn>
 <1415285223-7662-1-git-send-email-liang.xu@cinfotech.cn>
In-Reply-To: <1415285223-7662-1-git-send-email-liang.xu@cinfotech.cn>
Accept-Language: en-US
Content-Language: en-US
X-MS-Has-Attach:
X-MS-TNEF-Correlator:
x-originating-ip: [163.33.239.181]
Content-Type: text/plain; charset="us-ascii"
Content-Transfer-Encoding: quoted-printable
MIME-Version: 1.0
Subject: Re: [dpdk-dev] [PATCH v3] eal: map uio resources after hugepages
 when the base_virtaddr is configured.
X-BeenThere: dev@dpdk.org
X-Mailman-Version: 2.1.15
Precedence: list
List-Id: patches and discussions about DPDK <dev.dpdk.org>
List-Unsubscribe: <http://dpdk.org/ml/options/dev>,
 <mailto:dev-request@dpdk.org?subject=unsubscribe>
List-Archive: <http://dpdk.org/ml/archives/dev/>
List-Post: <mailto:dev@dpdk.org>
List-Help: <mailto:dev-request@dpdk.org?subject=help>
List-Subscribe: <http://dpdk.org/ml/listinfo/dev>,
 <mailto:dev-request@dpdk.org?subject=subscribe>
X-List-Received-Date: Thu, 06 Nov 2014 14:54:34 -0000

+					requested_addr = (uint8_t *)mapaddr + maps[j].size;

RTE_PTR_ADD?

Thanks,
Anatoly

-----Original Message-----
From: lxu [mailto:liang.xu@cinfotech.cn] 
Sent: Thursday, November 6, 2014 2:47 PM
To: dev@dpdk.org
Cc: Burakov, Anatoly
Subject: [PATCH v3] eal: map uio resources after hugepages when the base_virtaddr is configured.

---
 lib/librte_eal/linuxapp/eal/eal_pci_uio.c | 29 ++++++++++++++++++++++++++++-
 1 file changed, 28 insertions(+), 1 deletion(-)

diff --git a/lib/librte_eal/linuxapp/eal/eal_pci_uio.c b/lib/librte_eal/linuxapp/eal/eal_pci_uio.c
index 7e62266..3a218d0 100644
--- a/lib/librte_eal/linuxapp/eal/eal_pci_uio.c
+++ b/lib/librte_eal/linuxapp/eal/eal_pci_uio.c
@@ -273,6 +273,24 @@ pci_get_uio_dev(struct rte_pci_device *dev, char *dstbuf,
 	return uio_num;
 }
 
+static inline const struct rte_memseg *
+get_physmem_last(void)
+{
+	const struct rte_memseg * seg = rte_eal_get_physmem_layout();
+	const struct rte_memseg * last = seg;
+	unsigned i = 0;
+
+	for (i=0; i<RTE_MAX_MEMSEG; i++, seg++) {
+		if (seg->addr == NULL)
+			break;
+
+		if(seg->addr > last->addr)
+		 	last = seg;
+
+	}
+	return last;
+}
+
 /* map the PCI resource of a PCI device in virtual memory */  int  pci_uio_map_resource(struct rte_pci_device *dev) @@ -290,6 +308,13 @@ pci_uio_map_resource(struct rte_pci_device *dev)
 	struct mapped_pci_resource *uio_res;
 	struct pci_map *maps;
 
+	/* map uio resource into user required virtual address */
+	static void * requested_addr;
+	if (internal_config.base_virtaddr && NULL == requested_addr) {
+		const struct rte_memseg * last = get_physmem_last();
+		requested_addr = RTE_PTR_ADD(last->addr, last->len);
+	}
+
 	dev->intr_handle.fd = -1;
 	dev->intr_handle.type = RTE_INTR_HANDLE_UNKNOWN;
 
@@ -371,10 +396,12 @@ pci_uio_map_resource(struct rte_pci_device *dev)
 			if (maps[j].addr != NULL)
 				fail = 1;
 			else {
-				mapaddr = pci_map_resource(NULL, fd, (off_t)offset,
+				mapaddr = pci_map_resource(requested_addr, fd, (off_t)offset,
 						(size_t)maps[j].size);
 				if (mapaddr == NULL)
 					fail = 1;
+				else if (NULL != requested_addr)
+					requested_addr = (uint8_t *)mapaddr + maps[j].size;
 			}
 
 			if (fail) {
--
1.9.1