From: Robert Sanford <rsanford@prolexic.com>
To: Dmitry Vyal <dmitryvyal@gmail.com>
Cc: dev@dpdk.org
Subject: Re: [dpdk-dev] How to fight forwarding performance regression on large mempool sizes.
Date: Fri, 20 Sep 2013 11:34:23 -0400 [thread overview]
Message-ID: <CAFmpvUNEtcM8TtHJhqShRggDKRT3Y0vAY9aiG=28aV9NomQRgA@mail.gmail.com> (raw)
In-Reply-To: <523BEFA3.4070209@gmail.com>
One more point, if you're not doing this already: Allocate 2^N-1 mbufs, not
2^N. According to the code and comments: "The optimum size (in terms of
memory usage) for a mempool is when n is a power of two minus one: n = (2^q
- 1)."
The reason: rte_mempool_create(... n ...) invokes rte_ring_create(...
rte_align32power(n+1) ...), i.e., it wants to allocate n+1 slots in the
ring, but it has to round that up to the next power of 2. So, when you
allocate 2^N bufs, its ring comes with twice as many slots (2^(N+1)).
--
Robert
On Fri, Sep 20, 2013 at 2:48 AM, Dmitry Vyal <dmitryvyal@gmail.com> wrote:
> On 09/19/2013 11:39 PM, Robert Sanford wrote:
>
>> Hi Dmitry,
>>
>> The biggest drop-off seems to be from size 128K to 256K. Are you using
>> 1GB huge pages already (rather than 2MB)?
>>
>> I would think that it would not use over 1GB until you ask for 512K mbufs
>> or more.
>>
>>
> Hi Robert,
>
> Yes, I've been using 1GB pages for a while. My L3 cache is 20MB and mbufs
> are 2240 bytes of size. So something strange indeed happens then we move
> from ~200MB to ~400MB. Any ideas?
>
> Regards,
> Dmitry
>
>
next prev parent reply other threads:[~2013-09-20 15:33 UTC|newest]
Thread overview: 8+ messages / expand[flat|nested] mbox.gz Atom feed top
2013-09-19 7:50 Dmitry Vyal
2013-09-19 19:39 ` Robert Sanford
2013-09-19 19:43 ` Venkatesan, Venky
2013-09-20 6:50 ` Dmitry Vyal
2013-09-20 6:48 ` Dmitry Vyal
2013-09-20 14:41 ` Robert Sanford
2013-09-20 15:34 ` Robert Sanford [this message]
2013-09-22 19:03 ` Dmitry Vyal
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to='CAFmpvUNEtcM8TtHJhqShRggDKRT3Y0vAY9aiG=28aV9NomQRgA@mail.gmail.com' \
--to=rsanford@prolexic.com \
--cc=dev@dpdk.org \
--cc=dmitryvyal@gmail.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).