DPDK patches and discussions
 help / color / mirror / Atom feed
From: Panu Matilainen <pmatilai@redhat.com>
To: Thomas Monjalon <thomas.monjalon@6wind.com>,
	Olivier Matz <olivier.matz@6wind.com>
Cc: dev@dpdk.org, bruce.richardson@intel.com,
	stephen@networkplumber.org, keith.wiles@intel.com
Subject: Re: [dpdk-dev] [PATCH v3 00/35] mempool: rework memory allocation
Date: Fri, 20 May 2016 11:42:47 +0300	[thread overview]
Message-ID: <9d61a426-f580-b39e-1e38-7a30a4937324@redhat.com> (raw)
In-Reply-To: <36612895.rbhkJEDisP@xps13>

On 05/19/2016 03:47 PM, Thomas Monjalon wrote:
> 2016-05-18 13:04, Olivier Matz:
>> This series is a rework of mempool. For those who don't want to read
>> all the cover letter, here is a sumary:
>>
>> - it is not possible to allocate large mempools if there is not enough
>>   contiguous memory, this series solves this issue
>> - introduce new APIs with less arguments: "create, populate, obj_init"
>> - allow to free a mempool
>> - split code in smaller functions, will ease the introduction of ext_handler
>> - remove test-pmd anonymous mempool creation
>> - remove most of dom0-specific mempool code
>> - opens the door for a eal_memory rework: we probably don't need large
>>   contiguous memory area anymore, working with pages would work.
>>
>> This breaks the ABI as it was indicated in the deprecation for 16.04.
>> The API stays almost the same, no modification is needed in examples app
>> or in test-pmd. Only kni and mellanox drivers are slightly modified.
>
> Applied with a small change you sent me to fix mlx build in the middle of the patchset
> and update the removed Xen files in MAINTAINERS file.
>
> Thanks for the big rework!
>

Just noticed this series "breaks" --no-huge as a regular user, commit 
593a084afc2b to be exact:

mmap(NULL, 4194304, PROT_READ|PROT_WRITE, 
MAP_PRIVATE|MAP_ANONYMOUS|MAP_LOCKED, 0, 0) = -1 EAGAIN (Resource 
temporarily unavailable)
write(1, "EAL: rte_eal_hugepage_init: mmap"..., 76EAL: 
rte_eal_hugepage_init: mmap() failed: Resource temporarily unavailable

"Breaks" in quotes because I guess it always was broken (as the 
non-locked pages might not be in physical memory) and because its
possible to adjust resourse limits to allow the operation to succeed.
If you're root, that is.

I was just looking into making the test-suite runnable by a regular user 
with no special privileges, primarily to make it possible to run the 
testsuite as part of rpm package builds (in %check), and no special 
setup or extra privileges can be assumed there. Such tests are of course 
of limited coverage but still better than nothing, and --no-huge was my 
ticket there. Talk about bad timing :)

It'd be fine to have limited subset of tests to run when non-privileged 
but since this one lives inside rte_eal_init() it practically prevents 
everything, unless I'm missing some other magic switch or such. Thoughts?

	- Panu -

  reply	other threads:[~2016-05-20  8:42 UTC|newest]

Thread overview: 150+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2016-03-09 16:19 [dpdk-dev] [RFC " Olivier Matz
2016-03-09 16:19 ` [dpdk-dev] [RFC 01/35] mempool: fix comments and style Olivier Matz
2016-03-09 16:19 ` [dpdk-dev] [RFC 02/35] mempool: replace elt_size by total_elt_size Olivier Matz
2016-03-09 16:19 ` [dpdk-dev] [RFC 03/35] mempool: uninline function to check cookies Olivier Matz
2016-03-09 16:19 ` [dpdk-dev] [RFC 04/35] mempool: use sizeof to get the size of header and trailer Olivier Matz
2016-03-09 16:19 ` [dpdk-dev] [RFC 05/35] mempool: rename mempool_obj_ctor_t as mempool_obj_cb_t Olivier Matz
2016-03-09 16:19 ` [dpdk-dev] [RFC 06/35] mempool: update library version Olivier Matz
2016-03-09 16:19 ` [dpdk-dev] [RFC 07/35] mempool: list objects when added in the mempool Olivier Matz
2016-03-09 16:19 ` [dpdk-dev] [RFC 08/35] mempool: remove const attribute in mempool_walk Olivier Matz
2016-03-09 16:19 ` [dpdk-dev] [RFC 09/35] mempool: use the list to iterate the mempool elements Olivier Matz
2016-03-09 16:19 ` [dpdk-dev] [RFC 10/35] eal: introduce RTE_DECONST macro Olivier Matz
2016-03-09 18:53   ` Stephen Hemminger
2016-03-09 20:47     ` Olivier MATZ
2016-03-09 21:01       ` Stephen Hemminger
2016-03-10  8:11         ` Olivier MATZ
2016-03-11 21:47           ` Stephen Hemminger
2016-03-09 21:22       ` Bruce Richardson
2016-03-10  8:29         ` Olivier MATZ
2016-03-10  9:26           ` Bruce Richardson
2016-03-10 10:05             ` Olivier MATZ
2016-03-09 16:19 ` [dpdk-dev] [RFC 11/35] mempool: use the list to audit all elements Olivier Matz
2016-03-09 16:19 ` [dpdk-dev] [RFC 12/35] mempool: use the list to initialize mempool objects Olivier Matz
2016-03-09 16:19 ` [dpdk-dev] [RFC 13/35] mempool: create the internal ring in a specific function Olivier Matz
2016-03-09 16:19 ` [dpdk-dev] [RFC 14/35] mempool: store physaddr in mempool objects Olivier Matz
2016-03-09 16:19 ` [dpdk-dev] [RFC 15/35] mempool: remove MEMPOOL_IS_CONTIG() Olivier Matz
2016-03-09 16:19 ` [dpdk-dev] [RFC 16/35] mempool: store memory chunks in a list Olivier Matz
2016-03-09 16:19 ` [dpdk-dev] [RFC 17/35] mempool: new function to iterate the memory chunks Olivier Matz
2016-03-09 16:19 ` [dpdk-dev] [RFC 18/35] mempool: simplify xmem_usage Olivier Matz
2016-03-09 16:19 ` [dpdk-dev] [RFC 19/35] mempool: introduce a free callback for memory chunks Olivier Matz
2016-03-09 16:19 ` [dpdk-dev] [RFC 20/35] mempool: make page size optional when getting xmem size Olivier Matz
2016-03-09 16:19 ` [dpdk-dev] [RFC 21/35] mempool: default allocation in several memory chunks Olivier Matz
2016-03-09 16:19 ` [dpdk-dev] [RFC 22/35] eal: lock memory when using no-huge Olivier Matz
2016-03-09 16:19 ` [dpdk-dev] [RFC 23/35] mempool: support no-hugepage mode Olivier Matz
2016-03-09 16:19 ` [dpdk-dev] [RFC 24/35] mempool: replace mempool physaddr by a memzone pointer Olivier Matz
2016-03-09 16:19 ` [dpdk-dev] [RFC 25/35] mempool: introduce a function to free a mempool Olivier Matz
2016-03-09 16:19 ` [dpdk-dev] [RFC 26/35] mempool: introduce a function to create an empty mempool Olivier Matz
2016-03-09 16:19 ` [dpdk-dev] [RFC 27/35] eal/xen: return machine address without knowing memseg id Olivier Matz
2016-03-09 16:19 ` [dpdk-dev] [RFC 28/35] mempool: rework support of xen dom0 Olivier Matz
2016-03-09 16:19 ` [dpdk-dev] [RFC 29/35] mempool: create the internal ring when populating Olivier Matz
2016-03-09 16:19 ` [dpdk-dev] [RFC 30/35] mempool: populate a mempool with anonymous memory Olivier Matz
2016-03-09 16:19 ` [dpdk-dev] [RFC 31/35] test-pmd: remove specific anon mempool code Olivier Matz
2016-03-09 16:19 ` [dpdk-dev] [RFC 32/35] mempool: make mempool populate and free api public Olivier Matz
2016-03-09 16:19 ` [dpdk-dev] [RFC 33/35] mem: avoid memzone/mempool/ring name truncation Olivier Matz
2016-03-09 16:19 ` [dpdk-dev] [RFC 34/35] mempool: new flag when phys contig mem is not needed Olivier Matz
2016-03-09 16:19 ` [dpdk-dev] [RFC 35/35] mempool: update copyright Olivier Matz
2016-03-09 18:52   ` Stephen Hemminger
2016-03-10 14:57     ` Panu Matilainen
2016-03-09 16:44 ` [dpdk-dev] [RFC 00/35] mempool: rework memory allocation Olivier MATZ
2016-03-17  9:05 ` [dpdk-dev] [PATCH] doc: mempool ABI deprecation notice for 16.07 Olivier Matz
2016-04-04 14:38   ` Thomas Monjalon
2016-04-05  9:27     ` Hunt, David
2016-04-05 14:08       ` Wiles, Keith
2016-04-05 15:17         ` Thomas Monjalon
2016-04-14 10:19 ` [dpdk-dev] [PATCH 00/36] mempool: rework memory allocation Olivier Matz
2016-04-14 10:19   ` [dpdk-dev] [PATCH 01/36] mempool: fix comments and style Olivier Matz
2016-04-14 14:15     ` Wiles, Keith
2016-04-14 10:19   ` [dpdk-dev] [PATCH 02/36] mempool: replace elt_size by total_elt_size Olivier Matz
2016-04-14 14:18     ` Wiles, Keith
2016-04-14 10:19   ` [dpdk-dev] [PATCH 03/36] mempool: uninline function to check cookies Olivier Matz
2016-04-14 10:19   ` [dpdk-dev] [PATCH 04/36] mempool: use sizeof to get the size of header and trailer Olivier Matz
2016-04-14 10:19   ` [dpdk-dev] [PATCH 05/36] mempool: rename mempool_obj_ctor_t as mempool_obj_cb_t Olivier Matz
2016-04-14 10:19   ` [dpdk-dev] [PATCH 06/36] mempool: update library version Olivier Matz
2016-04-15 12:38     ` Olivier Matz
2016-04-14 10:19   ` [dpdk-dev] [PATCH 07/36] mempool: list objects when added in the mempool Olivier Matz
2016-04-14 10:19   ` [dpdk-dev] [PATCH 08/36] mempool: remove const attribute in mempool_walk Olivier Matz
2016-04-14 10:19   ` [dpdk-dev] [PATCH 09/36] mempool: remove const qualifier in dump and audit Olivier Matz
2016-04-14 10:19   ` [dpdk-dev] [PATCH 10/36] mempool: use the list to iterate the mempool elements Olivier Matz
2016-04-14 15:33     ` Wiles, Keith
2016-04-15  7:31       ` Olivier Matz
2016-04-15 13:19         ` Wiles, Keith
2016-05-11 10:02     ` [dpdk-dev] [PATCH v2 " Olivier Matz
2016-04-14 10:19   ` [dpdk-dev] [PATCH 11/36] mempool: use the list to audit all elements Olivier Matz
2016-04-14 10:19   ` [dpdk-dev] [PATCH 12/36] mempool: use the list to initialize mempool objects Olivier Matz
2016-04-14 10:19   ` [dpdk-dev] [PATCH 13/36] mempool: create the internal ring in a specific function Olivier Matz
2016-04-14 10:19   ` [dpdk-dev] [PATCH 14/36] mempool: store physaddr in mempool objects Olivier Matz
2016-04-14 15:40     ` Wiles, Keith
2016-04-15  7:34       ` Olivier Matz
2016-04-14 10:19   ` [dpdk-dev] [PATCH 15/36] mempool: remove MEMPOOL_IS_CONTIG() Olivier Matz
2016-04-14 10:19   ` [dpdk-dev] [PATCH 16/36] mempool: store memory chunks in a list Olivier Matz
2016-04-14 10:19   ` [dpdk-dev] [PATCH 17/36] mempool: new function to iterate the memory chunks Olivier Matz
2016-04-14 10:19   ` [dpdk-dev] [PATCH 18/36] mempool: simplify xmem_usage Olivier Matz
2016-04-14 10:19   ` [dpdk-dev] [PATCH 19/36] mempool: introduce a free callback for memory chunks Olivier Matz
2016-04-14 10:19   ` [dpdk-dev] [PATCH 20/36] mempool: make page size optional when getting xmem size Olivier Matz
2016-04-14 10:19   ` [dpdk-dev] [PATCH 21/36] mempool: default allocation in several memory chunks Olivier Matz
2016-04-14 10:19   ` [dpdk-dev] [PATCH 22/36] eal: lock memory when using no-huge Olivier Matz
2016-04-14 10:19   ` [dpdk-dev] [PATCH 23/36] mempool: support no-hugepage mode Olivier Matz
2016-04-14 10:19   ` [dpdk-dev] [PATCH 24/36] mempool: replace mempool physaddr by a memzone pointer Olivier Matz
2016-04-14 10:19   ` [dpdk-dev] [PATCH 25/36] mempool: introduce a function to free a mempool Olivier Matz
2016-04-14 10:19   ` [dpdk-dev] [PATCH 26/36] mempool: introduce a function to create an empty mempool Olivier Matz
2016-04-14 15:57     ` Wiles, Keith
2016-04-15  7:42       ` Olivier Matz
2016-04-15 13:26         ` Wiles, Keith
2016-04-14 10:19   ` [dpdk-dev] [PATCH 27/36] eal/xen: return machine address without knowing memseg id Olivier Matz
2016-04-14 10:19   ` [dpdk-dev] [PATCH 28/36] mempool: rework support of xen dom0 Olivier Matz
2016-04-14 10:19   ` [dpdk-dev] [PATCH 29/36] mempool: create the internal ring when populating Olivier Matz
2016-04-14 10:19   ` [dpdk-dev] [PATCH 30/36] mempool: populate a mempool with anonymous memory Olivier Matz
2016-04-14 10:19   ` [dpdk-dev] [PATCH 31/36] mempool: make mempool populate and free api public Olivier Matz
2016-04-14 10:19   ` [dpdk-dev] [PATCH 32/36] test-pmd: remove specific anon mempool code Olivier Matz
2016-04-14 10:19   ` [dpdk-dev] [PATCH 33/36] mem: avoid memzone/mempool/ring name truncation Olivier Matz
2016-04-14 10:19   ` [dpdk-dev] [PATCH 34/36] mempool: new flag when phys contig mem is not needed Olivier Matz
2016-04-14 10:19   ` [dpdk-dev] [PATCH 35/36] app/test: rework mempool test Olivier Matz
2016-04-14 10:19   ` [dpdk-dev] [PATCH 36/36] mempool: update copyright Olivier Matz
2016-04-14 13:50   ` [dpdk-dev] [PATCH 00/36] mempool: rework memory allocation Wiles, Keith
2016-04-14 14:01     ` Olivier MATZ
2016-04-14 14:03       ` Wiles, Keith
2016-04-14 14:20       ` Hunt, David
2016-05-18 11:04   ` [dpdk-dev] [PATCH v3 00/35] " Olivier Matz
2016-05-18 11:04     ` [dpdk-dev] [PATCH v3 01/35] mempool: rework comments and style Olivier Matz
2016-05-18 11:04     ` [dpdk-dev] [PATCH v3 02/35] mempool: rename element size variables Olivier Matz
2016-05-18 11:04     ` [dpdk-dev] [PATCH v3 03/35] mempool: uninline function to check cookies Olivier Matz
2016-05-18 11:04     ` [dpdk-dev] [PATCH v3 04/35] mempool: use sizeof to get the size of header and trailer Olivier Matz
2016-05-18 11:04     ` [dpdk-dev] [PATCH v3 05/35] mempool: rename object constructor typedef Olivier Matz
2016-05-18 11:04     ` [dpdk-dev] [PATCH v3 06/35] mempool: list objects when added Olivier Matz
2016-05-18 11:04     ` [dpdk-dev] [PATCH v3 07/35] mempool: remove const qualifier when browsing pools Olivier Matz
2016-05-18 11:04     ` [dpdk-dev] [PATCH v3 08/35] mempool: remove const qualifier in dump and audit Olivier Matz
2016-05-18 11:04     ` [dpdk-dev] [PATCH v3 09/35] mempool: use the list to iterate the elements Olivier Matz
2016-05-18 11:04     ` [dpdk-dev] [PATCH v3 10/35] mempool: use the list to audit all elements Olivier Matz
2016-05-18 11:04     ` [dpdk-dev] [PATCH v3 11/35] mempool: use the list to initialize objects Olivier Matz
2016-05-18 11:04     ` [dpdk-dev] [PATCH v3 12/35] mempool: create internal ring in a specific function Olivier Matz
2016-05-18 11:04     ` [dpdk-dev] [PATCH v3 13/35] mempool: store physical address in objects Olivier Matz
2016-05-25 17:51       ` Jain, Deepak K
2016-05-25 19:41         ` Olivier Matz
2016-05-18 11:04     ` [dpdk-dev] [PATCH v3 14/35] mempool: remove macro to check if contiguous Olivier Matz
2016-05-18 11:04     ` [dpdk-dev] [PATCH v3 15/35] mempool: store memory chunks in a list Olivier Matz
2016-05-18 11:04     ` [dpdk-dev] [PATCH v3 16/35] mempool: add function to iterate the memory chunks Olivier Matz
2016-05-18 11:04     ` [dpdk-dev] [PATCH v3 17/35] mempool: simplify the memory usage calculation Olivier Matz
2016-05-18 11:04     ` [dpdk-dev] [PATCH v3 18/35] mempool: introduce a free callback for memory chunks Olivier Matz
2016-05-18 11:04     ` [dpdk-dev] [PATCH v3 19/35] mempool: get memory size with unspecified page size Olivier Matz
2016-05-18 11:04     ` [dpdk-dev] [PATCH v3 20/35] mempool: allocate in several memory chunks by default Olivier Matz
2016-06-01  3:37       ` Ferruh Yigit
2016-05-18 11:04     ` [dpdk-dev] [PATCH v3 21/35] eal: lock memory when not using hugepages Olivier Matz
2016-05-18 11:04     ` [dpdk-dev] [PATCH v3 22/35] mempool: support no hugepage mode Olivier Matz
2016-05-18 11:04     ` [dpdk-dev] [PATCH v3 23/35] mempool: replace physical address by a memzone pointer Olivier Matz
2016-05-18 11:04     ` [dpdk-dev] [PATCH v3 24/35] mempool: introduce a function to free a pool Olivier Matz
2016-05-18 11:04     ` [dpdk-dev] [PATCH v3 25/35] mempool: introduce a function to create an empty pool Olivier Matz
2016-05-18 11:04     ` [dpdk-dev] [PATCH v3 26/35] eal/xen: return machine address without knowing memseg id Olivier Matz
2016-05-18 11:04     ` [dpdk-dev] [PATCH v3 27/35] mempool: rework support of Xen dom0 Olivier Matz
2016-05-18 11:04     ` [dpdk-dev] [PATCH v3 28/35] mempool: create the internal ring when populating Olivier Matz
2016-05-18 11:04     ` [dpdk-dev] [PATCH v3 29/35] mempool: populate with anonymous memory Olivier Matz
2016-05-18 11:04     ` [dpdk-dev] [PATCH v3 30/35] mempool: make mempool populate and free api public Olivier Matz
2016-05-18 11:04     ` [dpdk-dev] [PATCH v3 31/35] app/testpmd: remove anonymous mempool code Olivier Matz
2016-05-18 11:04     ` [dpdk-dev] [PATCH v3 32/35] mem: avoid memzone/mempool/ring name truncation Olivier Matz
2016-05-18 11:04     ` [dpdk-dev] [PATCH v3 33/35] mempool: add flag for removing phys contiguous constraint Olivier Matz
2016-05-18 11:04     ` [dpdk-dev] [PATCH v3 34/35] app/test: rework mempool test Olivier Matz
2016-05-18 11:04     ` [dpdk-dev] [PATCH v3 35/35] doc: update release notes about mempool allocation Olivier Matz
2016-05-19 12:47     ` [dpdk-dev] [PATCH v3 00/35] mempool: rework memory allocation Thomas Monjalon
2016-05-20  8:42       ` Panu Matilainen [this message]
2016-05-20  9:09         ` Thomas Monjalon
2016-05-23  7:43           ` Olivier Matz
2016-06-13 10:27             ` Olivier Matz

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=9d61a426-f580-b39e-1e38-7a30a4937324@redhat.com \
    --to=pmatilai@redhat.com \
    --cc=bruce.richardson@intel.com \
    --cc=dev@dpdk.org \
    --cc=keith.wiles@intel.com \
    --cc=olivier.matz@6wind.com \
    --cc=stephen@networkplumber.org \
    --cc=thomas.monjalon@6wind.com \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).