From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from dpdk.org (dpdk.org [92.243.14.124]) by inbox.dpdk.org (Postfix) with ESMTP id 07A2AA0577; Wed, 15 Apr 2020 01:37:33 +0200 (CEST) Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id 3ACAE1D177; Wed, 15 Apr 2020 01:37:33 +0200 (CEST) Received: from mga03.intel.com (mga03.intel.com [134.134.136.65]) by dpdk.org (Postfix) with ESMTP id 291C61D16B for ; Wed, 15 Apr 2020 01:37:31 +0200 (CEST) IronPort-SDR: /nwc6IGDv2q1nyx0fITD7pswy8clEe9MZBs3VrLTOW+1yjMQzRsxWI0dKX0PoLZxHvZ9mmPH/G UA7XblCCjTLg== X-Amp-Result: SKIPPED(no attachment in message) X-Amp-File-Uploaded: False Received: from fmsmga004.fm.intel.com ([10.253.24.48]) by orsmga103.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 14 Apr 2020 16:37:29 -0700 IronPort-SDR: 9QS+dtomA/3weEJSNbnNosSGlsIrSygIkU5sqFoFwhgdUcx3T04h9TZWiUapmsYGUFWWQBXppG vx6ewuAIRovw== X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.72,384,1580803200"; d="scan'208";a="277433870" Received: from pkadam-mobl1.amr.corp.intel.com (HELO [10.212.153.223]) ([10.212.153.223]) by fmsmga004.fm.intel.com with ESMTP; 14 Apr 2020 16:37:29 -0700 To: dev@dpdk.org, Dmitry Kozlyuk References: <20200410164342.1194634-1-dmitry.kozliuk@gmail.com> <20200414194426.1640704-1-dmitry.kozliuk@gmail.com> From: "Kadam, Pallavi" Message-ID: <22b45147-7301-fc4b-1628-70e076c28319@intel.com> Date: Tue, 14 Apr 2020 16:37:28 -0700 User-Agent: Mozilla/5.0 (Windows NT 10.0; WOW64; rv:68.0) Gecko/20100101 Thunderbird/68.5.0 MIME-Version: 1.0 In-Reply-To: <20200414194426.1640704-1-dmitry.kozliuk@gmail.com> Content-Type: text/plain; charset=utf-8; format=flowed Content-Language: en-GB Content-Transfer-Encoding: 7bit Subject: Re: [dpdk-dev] [PATCH v3 00/10] Windows basic memory management X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" On 4/14/2020 12:44 PM, Dmitry Kozlyuk wrote: > Note: no changes in cover letter since v2. > > This patchset implements basic MM with the following features: > > * Hugepages are dynamically allocated in user-mode. > * Only 2MB hugepages are supported. > * IOVA is always PA, obtained through kernel-mode driver. > * No 32-bit support (presumably not demanded). > * Ni multi-process support (it is forcefully disabled). > * No-huge mode for testing with IOVA unavailable. > > > The first commit introduces a new kernel-mode driver, virt2phys. > It translates user-mode virtual addresses into physical addresses. > On Windows community call 2020-04-01 it was decided this driver can be > used for now, later netUIO may pick up its code/interface or not. > > > New EAL public functions for memory mapping are introduced > to mitigate OS differences in DPDK libraries and applications: > > * rte_mem_map > * rte_mem_unmap > * rte_mem_lock > > To support common MM routines, internal wrappers for low-level > memory reservation and file management are introduced. These changes > affect Linux and FreeBSD EAL. Shared code is placed unded /unix/ > subdirectory (suggested by Thomas). > > Also, entire is imported from FreeBSD, replacing existing > partial import. There is already a license exception for this file. > > > Windows MM duplicates quite a lot of code from Linux EAL: > > * eal_memalloc_alloc_seg_bulk > * eal_memalloc_free_seg_bulk > * calc_num_pages_per_socket > * rte_eal_hugepage_init > > Perhaps this should be left as-is until Windows MM evolves into having > some specific requirements for these parts. > > > Notes on checkpatch warnings: > > * No space after comma / no space before closing parent in macros--- > definitely a false-positive, unclear how to suppress this. > > * Issues from imported BSD code---probably should be ignored? > > * Checkpatch is not run against dpdk-kmods (Windows drivers). > > --- > > v3: > > * Fix Linux build on and aarch64 and 32-bit x86 (reported by CI). > * Fix logic and error handling while allocating segments. > * Fix Unix rte_mem_map(): return NULL on failure. > * Fix some checkpatch.sh issues: > * Do not return positive errno, use DWORD for GetLastError(). > * Make dpdk-kmods source files non-executable. > * Improve GSG for Windows Server (suggested by Ranjit Menon). > > v2: > > * Rebase on ToT. Move all new code shared between Linux and FreeBSD > to /unix/ subdirectory, also factor out some existing code there. > * Improve description of Clang issue with rte_page_sizes on Windows. > Restore -fstrict-enum for EAL. Check running, not target compiler. > * Use EAL prefix for private facilities instead if RTE. > * Improve documentation comments for new functions. > * Remove co-installer for virt2phys. Add a typecast for clarity. > * Document virt2phys in user guide, improve its own README. > * Explicitly and forcefully disable multi-process. > > Dmitry Kozlyuk (9): > eal/windows: do not expose private EAL facilities > eal/windows: improve CPU and NUMA node detection > eal/windows: initialize hugepage info > eal: introduce internal wrappers for file operations > eal: introduce memory management wrappers > eal: extract common code for memseg list initialization > eal/windows: fix rte_page_sizes with Clang on Windows > eal/windows: replace sys/queue.h with a complete one from FreeBSD > eal/windows: implement basic memory management > > config/meson.build | 12 +- > doc/guides/windows_gsg/build_dpdk.rst | 20 - > doc/guides/windows_gsg/index.rst | 1 + > doc/guides/windows_gsg/run_apps.rst | 84 ++ > lib/librte_eal/common/eal_common_fbarray.c | 57 +- > lib/librte_eal/common/eal_common_memory.c | 104 +- > lib/librte_eal/common/eal_private.h | 134 +- > lib/librte_eal/common/malloc_heap.c | 1 + > lib/librte_eal/common/meson.build | 9 + > lib/librte_eal/freebsd/eal_memory.c | 55 +- > lib/librte_eal/include/rte_memory.h | 74 ++ > lib/librte_eal/linux/eal_memory.c | 68 +- > lib/librte_eal/meson.build | 4 + > lib/librte_eal/rte_eal_exports.def | 119 ++ > lib/librte_eal/rte_eal_version.map | 4 + > lib/librte_eal/unix/eal.c | 47 + > lib/librte_eal/unix/eal_memory.c | 113 ++ > lib/librte_eal/unix/meson.build | 7 + > lib/librte_eal/windows/eal.c | 160 +++ > lib/librte_eal/windows/eal_hugepages.c | 108 ++ > lib/librte_eal/windows/eal_lcore.c | 187 ++- > lib/librte_eal/windows/eal_memalloc.c | 418 ++++++ > lib/librte_eal/windows/eal_memory.c | 1141 +++++++++++++++++ > lib/librte_eal/windows/eal_mp.c | 103 ++ > lib/librte_eal/windows/eal_thread.c | 1 + > lib/librte_eal/windows/eal_windows.h | 129 ++ > lib/librte_eal/windows/include/meson.build | 2 + > lib/librte_eal/windows/include/pthread.h | 2 + > lib/librte_eal/windows/include/rte_os.h | 48 +- > .../windows/include/rte_virt2phys.h | 34 + > lib/librte_eal/windows/include/rte_windows.h | 43 + > lib/librte_eal/windows/include/sys/queue.h | 663 +++++++++- > lib/librte_eal/windows/include/unistd.h | 3 + > lib/librte_eal/windows/meson.build | 6 + > 34 files changed, 3611 insertions(+), 350 deletions(-) > create mode 100644 doc/guides/windows_gsg/run_apps.rst > create mode 100644 lib/librte_eal/unix/eal.c > create mode 100644 lib/librte_eal/unix/eal_memory.c > create mode 100644 lib/librte_eal/unix/meson.build > create mode 100644 lib/librte_eal/windows/eal_hugepages.c > create mode 100644 lib/librte_eal/windows/eal_memalloc.c > create mode 100644 lib/librte_eal/windows/eal_memory.c > create mode 100644 lib/librte_eal/windows/eal_mp.c > create mode 100644 lib/librte_eal/windows/eal_windows.h > create mode 100644 lib/librte_eal/windows/include/rte_virt2phys.h > create mode 100644 lib/librte_eal/windows/include/rte_windows.h > Tested-by: Pallavi Kadam