Re: [dpdk-dev] [PATCH v4 8/8] eal/windows: implement basic memory management

[Dmitry Kozlyuk <dmitry.kozliuk@gmail.com>]

On 2020-05-06 10:46 GMT+0100 Burakov, Anatoly wrote:
> On 06-May-20 12:20 AM, Dmitry Kozlyuk wrote:
> > On 2020-05-05 17:24 GMT+0100 Burakov, Anatoly wrote:  
> >> On 29-Apr-20 12:50 AM, Dmitry Kozlyuk wrote:
> >> Lots of duplication... I wonder if it would be possible to share at
> >> least some of this code in common. Tracking down bugs because of
> >> duplicated code desync is always a pain...  
> > 
> > This was the main question of the cover letter :)
> > Dmitry Malloy explained to me recently that even internally Windows has
> > no notion of preallocated hugepages and "memory types" (as memseg_primary_init
> > describes it). Since Windows EAL is not going to support multi-process any
> > time soon (if ever), maybe these reservations are not needed and memory manger
> > should create MSLs and enforce socket limits dynamically? This way most of the
> > duplicated code can be removed, I think. Or does MSL reservation serve some
> > other purposes?  
> 
> MSL reservation serves the purpose of dynamically expanding memory 
> usage.

But expansion is limited during init, because alloc_more_mem_on_socket()
works with existing MSLs, correct? No going to change anything there, just
trying to understand MM internals.

> If there is no notion of NUMA nodes or multiple page sizes, then 
> you can greatly simplify the code, but you'd still need *some* usage of 
> MSL's if you plan to support dynamically allocating memory, or 
> supporting externally allocated memory (i assume it's out of scope for 
> now, since you can't do IOVA as VA).

Windows is NUMA-aware and it supports both 2MB and 1GB hugepages (although
Windows EAL does not at the moment, because Win32 API is not yet official).
What I meant is that Windows does not reserve hugepages like Linux does with
vm.nr_hugepages or hugepage-related kernel options. So logic duplicated from
Linux EAL makes sense for Windows. The bulk of it can be extracted to some
common file, but it will not be truly common, rather "everything but
FreeBSD". Against it is a point that Windows MM may change significantly, but
I honestly can't come up with an example of how can those duplicated parts
may require adjustments.

> So, yes, you could greatly simplify the memory management code *if* you 
> were to go FreeBSD way and not allow dynamic page reservation. If you 
> do, however, then i would guess that you'd end up writing something 
> that's largely similar to existing Linux code (minus multiprocess) and 
> so would just be duplicating effort.

-- 
Dmitry Kozlyuk