From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mga12.intel.com (mga12.intel.com [192.55.52.136]) by dpdk.org (Postfix) with ESMTP id 401D71D940 for ; Mon, 11 Jun 2018 22:55:45 +0200 (CEST) X-Amp-Result: SKIPPED(no attachment in message) X-Amp-File-Uploaded: False Received: from orsmga002.jf.intel.com ([10.7.209.21]) by fmsmga106.fm.intel.com with ESMTP/TLS/DHE-RSA-AES256-GCM-SHA384; 11 Jun 2018 13:55:44 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.51,211,1526367600"; d="scan'208";a="66165832" Received: from irvmail001.ir.intel.com ([163.33.26.43]) by orsmga002.jf.intel.com with ESMTP; 11 Jun 2018 13:55:43 -0700 Received: from sivswdev01.ir.intel.com (sivswdev01.ir.intel.com [10.237.217.45]) by irvmail001.ir.intel.com (8.14.3/8.13.6/MailSET/Hub) with ESMTP id w5BKtgke026365 for ; Mon, 11 Jun 2018 21:55:42 +0100 Received: from sivswdev01.ir.intel.com (localhost [127.0.0.1]) by sivswdev01.ir.intel.com with ESMTP id w5BKtgkv021555 for ; Mon, 11 Jun 2018 21:55:42 +0100 Received: (from aburakov@localhost) by sivswdev01.ir.intel.com with LOCAL id w5BKtgRs021548 for dev@dpdk.org; Mon, 11 Jun 2018 21:55:42 +0100 From: Anatoly Burakov To: dev@dpdk.org Date: Mon, 11 Jun 2018 21:55:33 +0100 Message-Id: X-Mailer: git-send-email 1.7.0.7 Subject: [dpdk-dev] [PATCH 0/9] mem: reduce memory fragmentation X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 11 Jun 2018 20:55:46 -0000 This patchset is mostly dealing with changes fbarray, but it is actually about reducing fragmentation in Linuxapp memalloc. We allocate hugepages from lower VA to higher VA. However, our malloc heap allocates addresses from higher VA to lower VA. This results in a situation where, whenever new page is allocated, malloc starts to allocate memory from the top, leaving fragmented space between new allocation's leftover and previous leftover. Over time, this leads to lots of free elements sitting at page boundaries, small enough to be useful but large enough to have an impact on memory fragmentation in certain circumstances. To fix this, we need to allocate memory from higher VA first. However, in order to do that, we need the ability to search fbarray in reverse, which is currently not supported. Adding this support is what most of this patchset is about. First 4 patches fix some issues in existing fbarray implementation and remove some code duplication, preparing for adding of new functionality. Next 3 patches add new functionality - reverse search of used/free slots, mirroring already existing functionality in semantics and capable of returning identical results but in reverse order. Patch 8 adds unit tests for fbarray, testing both existing and new functionality. Finally, patch 9 changes memalloc to look up free slots in memseg list in reverse order. No other changes is necessary, as all other code can handle segments, wherever they are allocated. Anatoly Burakov (9): fbarray: fix errno values returned from functions fbarray: reduce duplication in find_contig code fbarray: reduce duplication in find_next_n code fbarray: reduce duplication in find_next code fbarray: add reverse find_free/used fbarray: add reverse find n used/free fbarray: add reverse find contig used/free test: add fbarray autotests memalloc: allocate memory in reverse lib/librte_eal/common/eal_common_fbarray.c | 493 ++++++++++++++--- lib/librte_eal/common/include/rte_fbarray.h | 114 ++++ lib/librte_eal/linuxapp/eal/eal_memalloc.c | 3 +- lib/librte_eal/rte_eal_version.map | 6 + test/test/Makefile | 1 + test/test/test_fbarray.c | 576 ++++++++++++++++++++ 6 files changed, 1119 insertions(+), 74 deletions(-) create mode 100644 test/test/test_fbarray.c -- 2.17.1