From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from dpdk.org (dpdk.org [92.243.14.124]) by dpdk.space (Postfix) with ESMTP id B8A90A00B9 for ; Sun, 31 Mar 2019 10:44:03 +0200 (CEST) Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id 61CA91DB8; Sun, 31 Mar 2019 10:44:02 +0200 (CEST) Received: from mellanox.co.il (mail-il-dmz.mellanox.com [193.47.165.129]) by dpdk.org (Postfix) with ESMTP id 5450B1DB8 for ; Sun, 31 Mar 2019 10:44:01 +0200 (CEST) Received: from Internal Mail-Server by MTLPINE1 (envelope-from shahafs@mellanox.com) with ESMTPS (AES256-SHA encrypted); 31 Mar 2019 11:43:57 +0300 Received: from unicorn01.mtl.labs.mlnx. (unicorn01.mtl.labs.mlnx [10.7.12.62]) by labmailer.mlnx (8.13.8/8.13.8) with ESMTP id x2V8hvLF000550; Sun, 31 Mar 2019 11:43:57 +0300 From: Shahaf Shuler To: anatoly.burakov@intel.com, thomas@monjalon.net Cc: dev@dpdk.org, stable@dpdk.org, alejandro.lucero@netronome.com Date: Sun, 31 Mar 2019 11:43:48 +0300 Message-Id: <20190331084348.4740-1-shahafs@mellanox.com> X-Mailer: git-send-email 2.12.0 In-Reply-To: <20190321202156.117496-1-shahafs@mellanox.com> References: <20190321202156.117496-1-shahafs@mellanox.com> Subject: [dpdk-dev] [PATCH v2] mem: limit use of address hint X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" Content-Type: text/plain; charset="UTF-8" Message-ID: <20190331084348.pjh48cAhzvop-qH_oCq5UiqHsffSjVoGdSID91v6pUU@z> patch[1] added an address hint as starting address for 64 bit systems in case an explicit base virtual address was not set by the user. The justification for such hint was to help devices that work in VA mode and has a address range limitation to work smoothly with the eal memory subsystem. While the base address value selected may work fine for the eal initialization, it easily breaks when trying to register external memory using rte_extmem_register API. Trying to register anonymous memory on RH x86_64 machine took several minutes, during them the function eal_get_virtual_area repeatedly scanned for a good VA candidate. The attempt to guess which VA address will be free for mapping will always result in not portable, error prone code: * different application may use different libraries along w/ DPDK. One can never guess which library was called first and how much virtual memory it consumed. * external memory can be registered at any time in the application run time. In order not to break the existing secondary process design, this patch only limits the max number of tries that will be done with the address hint. When the number of tries exceeds the threshold the code will use the suggested address from kernel. Fixes: 1df21702873d ("mem: use address hint for mapping hugepages") Cc: stable@dpdk.org Cc: alejandro.lucero@netronome.com [1] commit 1df21702873d ("mem: use address hint for mapping hugepages") Signed-off-by: Shahaf Shuler --- On v2: * instead of a complete remove of the hint limit the number of tries we allow. --- lib/librte_eal/common/eal_common_memory.c | 13 +++++++++---- 1 file changed, 9 insertions(+), 4 deletions(-) diff --git a/lib/librte_eal/common/eal_common_memory.c b/lib/librte_eal/common/eal_common_memory.c index c9da69b164..5ae8d0124d 100644 --- a/lib/librte_eal/common/eal_common_memory.c +++ b/lib/librte_eal/common/eal_common_memory.c @@ -56,6 +56,7 @@ static uint64_t system_page_sz; static uint64_t baseaddr = 0x100000000; #endif +#define MAX_MMAP_WITH_DEFINED_ADDR_TRIES 5 void * eal_get_virtual_area(void *requested_addr, size_t *size, size_t page_sz, int flags, int mmap_flags) @@ -63,6 +64,7 @@ eal_get_virtual_area(void *requested_addr, size_t *size, bool addr_is_hint, allow_shrink, unmap, no_align; uint64_t map_sz; void *mapped_addr, *aligned_addr; + uint8_t try = 0; if (system_page_sz == 0) system_page_sz = sysconf(_SC_PAGESIZE); @@ -118,11 +120,14 @@ eal_get_virtual_area(void *requested_addr, size_t *size, if (mapped_addr != MAP_FAILED && addr_is_hint && mapped_addr != requested_addr) { - /* hint was not used. Try with another offset */ - munmap(mapped_addr, map_sz); - mapped_addr = MAP_FAILED; + try++; next_baseaddr = RTE_PTR_ADD(next_baseaddr, page_sz); - requested_addr = next_baseaddr; + if (try <= MAX_MMAP_WITH_DEFINED_ADDR_TRIES) { + /* hint was not used. Try with another offset */ + munmap(mapped_addr, map_sz); + mapped_addr = MAP_FAILED; + requested_addr = next_baseaddr; + } } } while ((allow_shrink || addr_is_hint) && mapped_addr == MAP_FAILED && *size > 0); -- 2.12.0