DPDK patches and discussions
 help / color / mirror / Atom feed
From: Anatoly Burakov <anatoly.burakov@intel.com>
To: dev@dpdk.org
Cc: stable@dpdk.org
Subject: [dpdk-dev] [PATCH v3] eal/mem: preallocate VA space in no-huge mode
Date: Fri,  7 Feb 2020 11:11:14 +0000	[thread overview]
Message-ID: <5ebcce5b7df8fa0620b3d0fd6bf67a0668b942ac.1581073866.git.anatoly.burakov@intel.com> (raw)
In-Reply-To: <99d72c1b91e3fce36713b529ab47e8aa3fd78454.1579885526.git.anatoly.burakov@intel.com>

When --no-huge mode is used, the memory is currently allocated with
mmap(NULL, ...). This is fine in most cases, but can fail in cases
where DPDK is run on a machine with an IOMMU that is of more limited
address width than that of a VA, because we're not specifying the
address hint for mmap() call.

Fix it by preallocating VA space before mapping it.

Cc: stable@dpdk.org

Signed-off-by: Anatoly Burakov <anatoly.burakov@intel.com>
---

Notes:
    v3:
    - Fix mmap flags used in place of offset
    - Fix using internal_config.memory in place of mem_sz
    - Add additional address sanity check
    
    v2:
    - Add unmap on unsuccessful mmap
    
    I couldn't figure out which specific commit has introduced
    the issue, so there's no fix tag. The most likely candidate
    is one that introduced the DMA mask thing in the first place
    but i'm not sure.

 lib/librte_eal/linux/eal/eal_memory.c | 24 ++++++++++++++++++++----
 1 file changed, 20 insertions(+), 4 deletions(-)

diff --git a/lib/librte_eal/linux/eal/eal_memory.c b/lib/librte_eal/linux/eal/eal_memory.c
index 5604c2a7c0..7a9c97ff88 100644
--- a/lib/librte_eal/linux/eal/eal_memory.c
+++ b/lib/librte_eal/linux/eal/eal_memory.c
@@ -1340,6 +1340,8 @@ eal_legacy_hugepage_init(void)
 
 	/* hugetlbfs can be disabled */
 	if (internal_config.no_hugetlbfs) {
+		void *prealloc_addr;
+		size_t mem_sz;
 		struct rte_memseg_list *msl;
 		int n_segs, cur_seg, fd, flags;
 #ifdef MEMFD_SUPPORTED
@@ -1395,17 +1397,31 @@ eal_legacy_hugepage_init(void)
 			}
 		}
 #endif
-		addr = mmap(NULL, internal_config.memory, PROT_READ | PROT_WRITE,
-				flags, fd, 0);
-		if (addr == MAP_FAILED) {
+		/* preallocate address space for the memory, so that it can be
+		 * fit into the DMA mask.
+		 */
+		mem_sz = internal_config.memory;
+		prealloc_addr = eal_get_virtual_area(
+				NULL, &mem_sz, page_sz, 0, 0);
+		if (prealloc_addr == NULL) {
+			RTE_LOG(ERR, EAL,
+					"%s: reserving memory area failed: "
+					"%s\n",
+					__func__, strerror(errno));
+			return -1;
+		}
+		addr = mmap(prealloc_addr, mem_sz, PROT_READ | PROT_WRITE,
+				flags | MAP_FIXED, fd, 0);
+		if (addr == MAP_FAILED || addr != prealloc_addr) {
 			RTE_LOG(ERR, EAL, "%s: mmap() failed: %s\n", __func__,
 					strerror(errno));
+			munmap(prealloc_addr, mem_sz);
 			return -1;
 		}
 		msl->base_va = addr;
 		msl->page_sz = page_sz;
 		msl->socket_id = 0;
-		msl->len = internal_config.memory;
+		msl->len = mem_sz;
 		msl->heap = 1;
 
 		/* we're in single-file segments mode, so only the segment list
-- 
2.17.1

  parent reply	other threads:[~2020-02-07 11:11 UTC|newest]

Thread overview: 10+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2020-01-24 17:01 [dpdk-dev] [PATCH] " Anatoly Burakov
2020-01-24 17:05 ` [dpdk-dev] [PATCH v2] " Anatoly Burakov
2020-02-06 15:39   ` Thomas Monjalon
2020-02-06 21:07     ` Thomas Monjalon
2020-02-07 11:11   ` Anatoly Burakov [this message]
2020-03-25 14:39     ` [dpdk-dev] [dpdk-stable] [PATCH v3] " David Marchand
2020-03-26 17:06       ` Burakov, Anatoly
2020-03-27  6:33         ` David Marchand
     [not found]       ` <6473a4fe45d8437285fd8a1e931a1cb8@intel.com>
2020-03-27  2:23         ` Zhou, JunX W
2020-03-27 10:16       ` David Marchand

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=5ebcce5b7df8fa0620b3d0fd6bf67a0668b942ac.1581073866.git.anatoly.burakov@intel.com \
    --to=anatoly.burakov@intel.com \
    --cc=dev@dpdk.org \
    --cc=stable@dpdk.org \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).