From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from dpdk.org (dpdk.org [92.243.14.124]) by inbox.dpdk.org (Postfix) with ESMTP id 7C3CAA00BE; Wed, 29 Apr 2020 01:50:26 +0200 (CEST) Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id 393F31D644; Wed, 29 Apr 2020 01:50:25 +0200 (CEST) Received: from mail-lj1-f193.google.com (mail-lj1-f193.google.com [209.85.208.193]) by dpdk.org (Postfix) with ESMTP id D8E851D640 for ; Wed, 29 Apr 2020 01:50:23 +0200 (CEST) Received: by mail-lj1-f193.google.com with SMTP id u15so726182ljd.3 for ; Tue, 28 Apr 2020 16:50:23 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=VPL3dnPxuwdR5ruYLJW3hSb4AZz5WcrFmVenpuEhzWI=; b=bwVNIo/svDLaTW5eZPlxNY6OnI1Qsnrf6uM7/S9GYAFqe0v9NeZVKCjb49xlWwgp3F xaywt9n+0ULI53Jc6LeeRlEF2fXk83jZxD9Qtg4q6uKqvmGOWvRXKQLcE3BWygl6mgKU 7t4WibxF/tpNCGHPY0NcXqmTVQBxhQrwxOLofqKuDD2qVseSVfAq+yrJWvjqDsr1FkQo WgAnl7s54IDogEbPsnjQOEZ3HCGXv50aY/kDZUhnhsGkemI48wUPWo1mOwRrv49Xptrz VB2j2a05vSeLjncMmhDHBVs8x+bVXOnyXoqEpn4RUL8rvwk2qme8qkFwtTenUTHPx8nh u5zQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=VPL3dnPxuwdR5ruYLJW3hSb4AZz5WcrFmVenpuEhzWI=; b=LUlQElmVBtBPTGdsnhsR7/EJ5rrQyYEnVZHCdLTdLvy0dUmVYSJWVrk0TWasTcOrPq X3oK+LF72LfMgJcfrgUAxepOwGkMGqJN2jgVvp8gZGMaejM+7OhuWisc7DpTaOQwChWK u0W9bXZZJoKAEpKbLGqBEP0PoqwZGTf+1Drr3dc2nt4dx6j9+Tjjmv55GZHmNjmcEQhk pZxJ81DYmuizjRuYsb2m7xu4UxQNvs3iJFyU7jBGIzu0cmk8sfVNK5QqAQguDAI30dnG kloIadsmcRNKWTsNLOfzqsxl8E0820p13+Y4pw6qXGcW4PDTgJH39tA5N0vuzTnw4qJa 4Ffw== X-Gm-Message-State: AGi0PuYc+sNwb6da3j8X8l6gr4iAYzfFHuY68+fJ/Jt863wDa06tIVTL AApyC0V1Wb2GVNbztJ+yVedRv9YK X-Google-Smtp-Source: APiQypJ4KdSb7+yrALOsADSR/gY2105S8zC1r+tm7pZ8dTyNZVm7XQqZ4ywlId36VksXYog520Yomw== X-Received: by 2002:a2e:b17a:: with SMTP id a26mr18045905ljm.215.1588117822696; Tue, 28 Apr 2020 16:50:22 -0700 (PDT) Received: from localhost.localdomain (broadband-37-110-65-23.ip.moscow.rt.ru. [37.110.65.23]) by smtp.googlemail.com with ESMTPSA id g10sm183339lfc.95.2020.04.28.16.50.21 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 28 Apr 2020 16:50:21 -0700 (PDT) From: Dmitry Kozlyuk To: dev@dpdk.org Cc: "Dmitry Malloy (MESHCHANINOV)" , Narcisa Ana Maria Vasile , Fady Bader , Tal Shnaiderman , Dmitry Kozlyuk Date: Wed, 29 Apr 2020 02:50:07 +0300 Message-Id: <20200428235015.2820677-1-dmitry.kozliuk@gmail.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20200410164342.1194634-1-dmitry.kozliuk@gmail.com> References: <20200410164342.1194634-1-dmitry.kozliuk@gmail.com> MIME-Version: 1.0 Content-Transfer-Encoding: 8bit Subject: [dpdk-dev] [PATCH v4 0/8] Windows basic memory management X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" Note: symbols and release notes updated for v20.05, despite this patch now targeting v20.08, because proper sections don't yet exist. This patchset implements basic MM with the following features: * Hugepages are dynamically allocated in user-mode. * Only 2MB hugepages are supported. * IOVA is always PA, obtained through kernel-mode driver. * No 32-bit support (presumably not demanded). * Ni multi-process support (it is forcefully disabled). * No-huge mode for testing with IOVA unavailable. New EAL public functions for memory mapping are introduced to mitigate OS differences in DPDK libraries and applications: * rte_mem_map * rte_mem_unmap * rte_mem_lock * rte_get_page_size To support common MM routines, internal wrappers for low-level memory reservation and file management are introduced. These changes affect Linux and FreeBSD EAL. Shared code is placed unded /unix/ subdirectory (suggested by Thomas). Also, entire is imported from FreeBSD, replacing existing partial import. There is already a license exception for this file. Windows MM duplicates quite a lot of code from Linux EAL: * eal_memalloc_alloc_seg_bulk * eal_memalloc_free_seg_bulk * calc_num_pages_per_socket * rte_eal_hugepage_init Perhaps this should be left as-is until Windows MM evolves into having some specific requirements for these parts. Notes on checkpatch warnings: * No space after comma / no space before closing parent in macros--- definitely a false-positive, unclear how to suppress this. * Issues from imported BSD code---probably should be ignored? --- v4: * Rebase on ToT, drop patches merged into master. * Rearrange patches to split Windows code (Jerin). * Fix Linux and FreeBSD build with make (Ophir). * Use int instead of enum to hold a set of flags (Anatoly). * Rename eal_mem_reserve items and fix their description (Anatoly). * Add eal_mem_set_dump() wrapper around madvise (Anatoly). * Don't claim Windows Server 2016 support due to lack of API (Tal). * Replace enum rte_page_sizes with a set of #defines (Jerin). * Fix documentation, SPDX tags, logging (Thomas). v3: * Fix Linux build on and aarch64 and 32-bit x86 (reported by CI). * Fix logic and error handling while allocating segments. * Fix Unix rte_mem_map(): return NULL on failure. * Fix some checkpatch.sh issues: * Do not return positive errno, use DWORD for GetLastError(). * Make dpdk-kmods source files non-executable. * Improve GSG for Windows Server (suggested by Ranjit Menon). v2: * Rebase on ToT. Move all new code shared between Linux and FreeBSD to /unix/ subdirectory, also factor out some existing code there. * Improve description of Clang issue with rte_page_sizes on Windows. Restore -fstrict-enum for EAL. Check running, not target compiler. * Use EAL prefix for private facilities instead if RTE. * Improve documentation comments for new functions. * Remove co-installer for virt2phys. Add a typecast for clarity. * Document virt2phys in user guide, improve its own README. * Explicitly and forcefully disable multi-process. Dmitry Kozlyuk (8): eal: replace rte_page_sizes with a set of constants eal: introduce internal wrappers for file operations eal: introduce memory management wrappers eal: extract common code for memseg list initialization eal/windows: replace sys/queue.h with a complete one from FreeBSD eal/windows: improve CPU and NUMA node detection eal/windows: initialize hugepage info eal/windows: implement basic memory management config/meson.build | 12 +- doc/guides/rel_notes/release_20_05.rst | 2 + doc/guides/windows_gsg/build_dpdk.rst | 20 - doc/guides/windows_gsg/index.rst | 1 + doc/guides/windows_gsg/run_apps.rst | 95 ++ lib/librte_eal/common/eal_common_fbarray.c | 56 +- lib/librte_eal/common/eal_common_memory.c | 116 +- lib/librte_eal/common/eal_common_memzone.c | 7 + lib/librte_eal/common/eal_private.h | 155 ++- lib/librte_eal/common/meson.build | 10 + lib/librte_eal/common/rte_malloc.c | 9 + lib/librte_eal/freebsd/Makefile | 5 + lib/librte_eal/freebsd/eal_memory.c | 57 +- lib/librte_eal/include/rte_memory.h | 109 +- lib/librte_eal/linux/Makefile | 5 + lib/librte_eal/linux/eal_memalloc.c | 5 +- lib/librte_eal/linux/eal_memory.c | 68 +- lib/librte_eal/meson.build | 4 + lib/librte_eal/rte_eal_exports.def | 119 ++ lib/librte_eal/rte_eal_version.map | 4 + lib/librte_eal/unix/eal_unix.c | 51 + lib/librte_eal/unix/eal_unix_memory.c | 144 ++ lib/librte_eal/unix/meson.build | 7 + lib/librte_eal/windows/eal.c | 158 +++ lib/librte_eal/windows/eal_hugepages.c | 108 ++ lib/librte_eal/windows/eal_lcore.c | 185 ++- lib/librte_eal/windows/eal_memalloc.c | 418 ++++++ lib/librte_eal/windows/eal_memory.c | 1155 +++++++++++++++++ lib/librte_eal/windows/eal_mp.c | 103 ++ lib/librte_eal/windows/eal_windows.h | 100 ++ lib/librte_eal/windows/include/meson.build | 1 + lib/librte_eal/windows/include/rte_os.h | 4 + .../windows/include/rte_virt2phys.h | 34 + lib/librte_eal/windows/include/rte_windows.h | 2 + lib/librte_eal/windows/include/sys/queue.h | 663 +++++++++- lib/librte_eal/windows/include/unistd.h | 3 + lib/librte_eal/windows/meson.build | 6 + 37 files changed, 3655 insertions(+), 346 deletions(-) create mode 100644 doc/guides/windows_gsg/run_apps.rst create mode 100644 lib/librte_eal/unix/eal_unix.c create mode 100644 lib/librte_eal/unix/eal_unix_memory.c create mode 100644 lib/librte_eal/unix/meson.build create mode 100644 lib/librte_eal/windows/eal_hugepages.c create mode 100644 lib/librte_eal/windows/eal_memalloc.c create mode 100644 lib/librte_eal/windows/eal_memory.c create mode 100644 lib/librte_eal/windows/eal_mp.c create mode 100644 lib/librte_eal/windows/include/rte_virt2phys.h -- 2.25.1