From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from dpdk.org (dpdk.org [92.243.14.124]) by inbox.dpdk.org (Postfix) with ESMTP id 0F612A0598; Fri, 10 Apr 2020 18:43:54 +0200 (CEST) Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id D7C1B1D5DA; Fri, 10 Apr 2020 18:43:53 +0200 (CEST) Received: from mail-lj1-f196.google.com (mail-lj1-f196.google.com [209.85.208.196]) by dpdk.org (Postfix) with ESMTP id 9C14B1D5D1 for ; Fri, 10 Apr 2020 18:43:52 +0200 (CEST) Received: by mail-lj1-f196.google.com with SMTP id z26so2493870ljz.11 for ; Fri, 10 Apr 2020 09:43:52 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=cBVLUYScJjUWKd8D8auogiXb/4oXpx6rtajZAUO9djs=; b=B4Xs7Q4LazdtvrXErQ2pbx/WQD7LHm4meDyhwD+IxKQnVRspatsZJLuNud0xe574Tb thrda+noGrjVY5+momJVBeT4TmsIcwArnBo8S0azpJwrxlhsFixOmf1QRjHKwjq2gz1P Cg3+NuIjR1ukKmQynDZOYkHgNCHvvCWpcj5d5LSHaQ+xXlUc3t8SyjiRMhWpsdubN0Ds DArbQ07uRntCQCgKlzv50RE/zp4qBcvO1VFONjJvC42WVuexVpZD1lMaTZWEXe1ehyTZ zwsM+GM/LZOJbrm+pS0VtjehjtmketnGYbK+MJ+kn6zh4GsHdI4nB4InJZ2ScGeSK4sp ABTQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=cBVLUYScJjUWKd8D8auogiXb/4oXpx6rtajZAUO9djs=; b=Kk06hb9m84f6vkLO/8wHSOMWl8sb12zDYXnardthfaF35mgX9Our/fRY5WeMuQiFS8 jcBo4N8chQepJdEeK+7gMsFYcPuKE/aFeVMfIUzDCYUMIQxeovBTbn+HoMtS9qFaoByD zBjcP1ar6ye5uR1lZX6roMNYzth2aTQDBdREolxkzmm3nEEh7bHQ92yYY6jYAC0dksBq 4mzk/41PcX3uGlzbZw0FhXe2vIN2j8rjJ8Sim/CS6zYcFQ03q0FaW9VWB959sbmfpVpw JOwqvBUq6hWivJt59yS+KAdTkaA5ZsBqD7qQXzecorouyp2oJgiZXjsQb+1n5d32SaJf IcIA== X-Gm-Message-State: AGi0PuYvp6fezdp1JoVM7H06PoXf/gbBOici1UMctg0x64wPWOTtZybX cN64K3NQkZAZWmYfwEZD1soFlhACo8QvdA== X-Google-Smtp-Source: APiQypJd8Qsym1C2Kz8Md0muMBpaW6VPMPSPp/9zxeQHVGn0sJxr50jY8MsYdcq+ATlw+wLmf3ZlXQ== X-Received: by 2002:a2e:b610:: with SMTP id r16mr3497027ljn.254.1586537030870; Fri, 10 Apr 2020 09:43:50 -0700 (PDT) Received: from localhost.localdomain (broadband-37-110-65-23.ip.moscow.rt.ru. [37.110.65.23]) by smtp.googlemail.com with ESMTPSA id d21sm1343075ljc.49.2020.04.10.09.43.50 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 10 Apr 2020 09:43:50 -0700 (PDT) From: Dmitry Kozlyuk To: dev@dpdk.org Cc: "Dmitry Malloy (MESHCHANINOV)" , Narcisa Ana Maria Vasile , Fady Bader , Tal Shnaiderman , Dmitry Kozlyuk Date: Fri, 10 Apr 2020 19:43:32 +0300 Message-Id: <20200410164342.1194634-1-dmitry.kozliuk@gmail.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20200330041026.784624-1-dmitry.kozliuk@gmail.com> References: <20200330041026.784624-1-dmitry.kozliuk@gmail.com> MIME-Version: 1.0 Content-Transfer-Encoding: 8bit Subject: [dpdk-dev] [PATCH v2 00/10] eal: Windows basic memory management X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" This patchset implements basic MM with the following features: * Hugepages are dynamically allocated in user-mode. * Only 2MB hugepages are supported. * IOVA is always PA, obtained through kernel-mode driver. * No 32-bit support (presumably not demanded). * Ni multi-process support (it is forcefully disabled). * No-huge mode for testing with IOVA unavailable. The first commit introduces a new kernel-mode driver, virt2phys. It translates user-mode virtual addresses into physical addresses. On Windows community call 2020-04-01 it was decided this driver can be used for now, later netUIO may pick up its code/interface or not. New EAL public functions for memory mapping are introduced to mitigate OS differences in DPDK libraries and applications: * rte_mem_map * rte_mem_unmap * rte_mem_lock To support common MM routines, internal wrappers for low-level memory reservation and file management are introduced. These changes affect Linux and FreeBSD EAL. Shared code is placed unded /unix/ subdirectory (suggested by Thomas). Also, entire is imported from FreeBSD, replacing existing partial import. There is already a license exception for this file. Windows MM duplicates quite a lot of code from Linux EAL: * eal_memalloc_alloc_seg_bulk * eal_memalloc_free_seg_bulk * calc_num_pages_per_socket * rte_eal_hugepage_init Perhaps this should be left as-is until Windows MM evolves into having some specific requirements for these parts. Notes on checkpatch warnings: * No space after comma / no space before closing parent in macros--- definitely a false-positive, unclear how to suppress this. * Issues from imported BSD code---probably should be ignored? * Checkpatch is not run against dpdk-kmods (Windows drivers). --- There were comments from Mellanox and Microsoft outside of this mailing list. Still missing input from Anatoly Burakov. v2: * Rebase on ToT. Move all new code shared between Linux and FreeBSD to /unix/ subdirectory, also factor out some existing code there. * Improve description of Clang issue with rte_page_sizes on Windows. Restore -fstrict-enum for EAL. Check running, not target compiler. * Use EAL prefix for private facilities instead if RTE. * Improve documentation comments for new functions. * Remove co-installer for virt2phys. Add a typecast for clarity. * Document virt2phys in user guide, improve its own README. * Explicitly and forcefully disable multi-process. Dmitry Kozlyuk (9): eal/windows: do not expose private EAL facilities eal/windows: improve CPU and NUMA node detection eal/windows: initialize hugepage info eal: introduce internal wrappers for file operations eal: introduce memory management wrappers eal: extract common code for memseg list initialization eal/windows: fix rte_page_sizes with Clang on Windows eal/windows: replace sys/queue.h with a complete one from FreeBSD eal/windows: implement basic memory management config/meson.build | 12 +- doc/guides/windows_gsg/build_dpdk.rst | 20 - doc/guides/windows_gsg/index.rst | 1 + doc/guides/windows_gsg/run_apps.rst | 77 ++ lib/librte_eal/common/eal_common_fbarray.c | 57 +- lib/librte_eal/common/eal_common_memory.c | 104 +- lib/librte_eal/common/eal_private.h | 134 +- lib/librte_eal/common/malloc_heap.c | 1 + lib/librte_eal/common/meson.build | 9 + lib/librte_eal/freebsd/eal_memory.c | 55 +- lib/librte_eal/include/rte_memory.h | 74 ++ lib/librte_eal/linux/eal_memory.c | 68 +- lib/librte_eal/meson.build | 5 + lib/librte_eal/rte_eal_exports.def | 119 ++ lib/librte_eal/rte_eal_version.map | 4 + lib/librte_eal/unix/eal.c | 47 + lib/librte_eal/unix/eal_memory.c | 112 ++ lib/librte_eal/unix/meson.build | 7 + lib/librte_eal/windows/eal.c | 160 +++ lib/librte_eal/windows/eal_hugepages.c | 108 ++ lib/librte_eal/windows/eal_lcore.c | 187 +-- lib/librte_eal/windows/eal_memalloc.c | 423 ++++++ lib/librte_eal/windows/eal_memory.c | 1133 +++++++++++++++++ lib/librte_eal/windows/eal_mp.c | 103 ++ lib/librte_eal/windows/eal_thread.c | 1 + lib/librte_eal/windows/eal_windows.h | 129 ++ lib/librte_eal/windows/include/meson.build | 2 + lib/librte_eal/windows/include/pthread.h | 2 + lib/librte_eal/windows/include/rte_os.h | 48 +- .../windows/include/rte_virt2phys.h | 34 + lib/librte_eal/windows/include/rte_windows.h | 43 + lib/librte_eal/windows/include/sys/queue.h | 663 +++++++++- lib/librte_eal/windows/include/unistd.h | 3 + lib/librte_eal/windows/meson.build | 4 + 34 files changed, 3599 insertions(+), 350 deletions(-) create mode 100644 doc/guides/windows_gsg/run_apps.rst create mode 100644 lib/librte_eal/unix/eal.c create mode 100644 lib/librte_eal/unix/eal_memory.c create mode 100644 lib/librte_eal/unix/meson.build create mode 100644 lib/librte_eal/windows/eal_hugepages.c create mode 100644 lib/librte_eal/windows/eal_memalloc.c create mode 100644 lib/librte_eal/windows/eal_memory.c create mode 100644 lib/librte_eal/windows/eal_mp.c create mode 100644 lib/librte_eal/windows/eal_windows.h create mode 100644 lib/librte_eal/windows/include/rte_virt2phys.h create mode 100644 lib/librte_eal/windows/include/rte_windows.h -- 2.25.1