DPDK patches and discussions
 help / color / mirror / Atom feed
* [dpdk-dev] [RFC PATCH 0/9] Windows basic memory management
@ 2020-03-30  4:10 Dmitry Kozlyuk
  2020-03-30  4:10 ` [dpdk-dev] [PATCH 1/1] virt2phys: virtual to physical address translator for Windows Dmitry Kozlyuk
                   ` (9 more replies)
  0 siblings, 10 replies; 218+ messages in thread
From: Dmitry Kozlyuk @ 2020-03-30  4:10 UTC (permalink / raw)
  To: dev; +Cc: Dmitry Malloy (MESHCHANINOV), Dmitry Kozlyuk

This RFC implements basic MM with the following features:

* Hugepages are dynamically allocated in user-mode.
* IOVA is always PA, obtained through kernel-mode driver.
* No 32-bit support (presumably not demanded).
* No multi-process support. Note that without --in-memory EAL MM
  will create files in current working (runtime) directory.
* No-huge mode for testing with no IOVA available. IOVA could be obtained
  using Address Windowing Extensions, but it is doubtfully demanded.

Roadmap for Windows [1] proposes that memory management (MM) should be
implemented in basic and advanced stages:

1. Basic MM must be sufficient for core libraries and network PMDs:
2. Advanced MM could address features missing from basic stage.

Advanced memory management discussion is out of scope of this RFC.
Windows community calls suggest is will be focused on security and IOMMU
support. I will post a separate thread with background and suggestions.
Cc'ing Dmitry Malloy (MESHCHANINOV) nevertheless.

Because netUIO is not yet committed to dpdk-kmods, the first commit
introduces a new simple driver, virt2phys. It will almost certainly
become a part of netUIO once it is available for patches. Until then, it
must be installed according to documentation provided with the patch.
User-mode code locates the driver interface by GUID, so transition from
virt2phys to netUIO should not require changes to DPDK.

Hugepages allocation on Windows requires some privilege setup. Please
refer to documentation provided in the "initialize hugepage info" patch.

New EAL public functions for memory mapping are introduced. Their
implementation for Linux and FreeBSD is identical. The upcoming patch series
reorganizing EAL directories will help fixing that [2].

Windows MM duplicates quite a lot of code from Linux EAL:

* eal_memalloc_alloc_seg_bulk
* eal_memalloc_free_seg_bulk
* calc_num_pages_per_socket
* rte_eal_hugepage_init

Need input if it should be left as-is to evolve independently, or some
"common to memory hot-plug" code should be factored out. This
duplication may be reduced naturally when advanced MM is implemented.

Notes on checkpatch warnings:

* No space after comma / no space before closing parent in macros---
  definitely a false-positive, unclear how to suppress this.

* Issues from imported BSD code---probably should be ignored?

* Checkpatch is not run against dpdk-kmods (Windows drivers).

[1]: http://core.dpdk.org/roadmap/windows/
[2]: https://patchwork.dpdk.org/project/dpdk/list/?series=9070

Dmitry Kozlyuk (8):
  eal/windows: do not expose private EAL facilities
  eal/windows: improve CPU and NUMA node detection
  eal/windows: initialize hugepage info
  eal: introduce internal wrappers for file operations
  eal: introduce memory management wrappers
  eal/windows: fix rte_page_sizes with Clang on Windows
  eal/windows: replace sys/queue.h with a complete one from FreeBSD
  eal/windows: implement basic memory management

 config/meson.build                            |   12 +-
 doc/guides/windows_gsg/build_dpdk.rst         |   20 -
 doc/guides/windows_gsg/index.rst              |    1 +
 doc/guides/windows_gsg/run_apps.rst           |   47 +
 lib/librte_eal/common/eal_common_fbarray.c    |   57 +-
 lib/librte_eal/common/eal_common_memory.c     |   50 +-
 lib/librte_eal/common/eal_private.h           |  116 +-
 lib/librte_eal/common/include/rte_memory.h    |   69 +
 lib/librte_eal/common/malloc_heap.c           |    1 +
 lib/librte_eal/freebsd/eal/eal.c              |   40 +
 lib/librte_eal/freebsd/eal/eal_memory.c       |  118 +-
 lib/librte_eal/linux/eal/eal.c                |   40 +
 lib/librte_eal/linux/eal/eal_memory.c         |  117 ++
 lib/librte_eal/meson.build                    |    4 +
 lib/librte_eal/rte_eal_exports.def            |  119 ++
 lib/librte_eal/rte_eal_version.map            |    4 +
 lib/librte_eal/windows/eal/eal.c              |  152 +++
 lib/librte_eal/windows/eal/eal_hugepages.c    |  108 ++
 lib/librte_eal/windows/eal/eal_lcore.c        |  187 ++-
 lib/librte_eal/windows/eal/eal_memalloc.c     |  423 ++++++
 lib/librte_eal/windows/eal/eal_memory.c       | 1166 +++++++++++++++++
 lib/librte_eal/windows/eal/eal_mp.c           |  103 ++
 lib/librte_eal/windows/eal/eal_thread.c       |    1 +
 lib/librte_eal/windows/eal/eal_windows.h      |  113 ++
 lib/librte_eal/windows/eal/include/pthread.h  |    2 +
 lib/librte_eal/windows/eal/include/rte_os.h   |   48 +-
 .../windows/eal/include/rte_virt2phys.h       |   34 +
 .../windows/eal/include/rte_windows.h         |   43 +
 .../windows/eal/include/sys/queue.h           |  663 +++++++++-
 lib/librte_eal/windows/eal/include/unistd.h   |    3 +
 lib/librte_eal/windows/eal/meson.build        |   15 +
 31 files changed, 3626 insertions(+), 250 deletions(-)
 create mode 100644 doc/guides/windows_gsg/run_apps.rst
 create mode 100644 lib/librte_eal/windows/eal/eal_hugepages.c
 create mode 100644 lib/librte_eal/windows/eal/eal_memalloc.c
 create mode 100644 lib/librte_eal/windows/eal/eal_memory.c
 create mode 100644 lib/librte_eal/windows/eal/eal_mp.c
 create mode 100644 lib/librte_eal/windows/eal/eal_windows.h
 create mode 100644 lib/librte_eal/windows/eal/include/rte_virt2phys.h
 create mode 100644 lib/librte_eal/windows/eal/include/rte_windows.h

-- 
2.25.1


^ permalink raw reply	[flat|nested] 218+ messages in thread