DPDK patches and discussions
 help / color / mirror / Atom feed
From: "Ananyev, Konstantin" <konstantin.ananyev@intel.com>
To: "Rapelly, Varun" <vrapelly@sonusnet.com>, "dev@dpdk.org" <dev@dpdk.org>
Subject: Re: [dpdk-dev] ACL memory allocation failures
Date: Fri, 26 Feb 2016 12:39:06 +0000	[thread overview]
Message-ID: <2601191342CEEE43887BDE71AB97725836B0BADD@irsmsx105.ger.corp.intel.com> (raw)
In-Reply-To: <BY2PR0301MB1542EFDE36F08FD98A41FB7EB7A70@BY2PR0301MB1542.namprd03.prod.outlook.com>



> 
> Thanks Konstantin.
> 
> Previous allocation error was coming with 1024 huge pages of 2 MB size.
> 
> After increasing the huge pages to 2048, I was able to add another ~140 rules [IPv4 rule data--> with src, dst IP address & port, next
> header ] more, ie., 950 rules were added.

That's strange according to your log, all you need is ~13MB of hugepage memory:
ACL: allocation of 12966528 bytes on socket 0 for ipv4_acl_table1
Wonder what consumed rest of 4GB?
Again do you re-build your table after every rule you add?
If so, then it seems a bit strange approach (and definitely not the fastest one).
What you can do instead: create context; add all your rules into it; build; 

> 
> Logically it did not increase number of rules [expected 2*817, but only 950 were added]. Is it really using huge pages memory only?
> 
> From the code it looks like heap memory. [ ret = malloc_heap_alloc(&mcfg->malloc_heaps[i], type, size, 0, align == 0 ? 1 : align, 0) ]

As I can see from the log it fails at GEN phase, when trying to allocate hugepages for RT table.
At lib/librte_acl/acl_gen.c:509

rte_acl_gen(struct rte_acl_ctx *ctx, struct rte_acl_trie *trie,
        struct rte_acl_bld_trie *node_bld_trie, uint32_t num_tries,
        uint32_t num_categories, uint32_t data_index_sz, size_t max_size)
{
...
mem = rte_zmalloc_socket(ctx->name, total_size, RTE_CACHE_LINE_SIZE,
                        ctx->socket_id);
if (mem == NULL) {
                RTE_LOG(ERR, ACL,
                        "allocation of %zu bytes on socket %d for %s failed\n",
                        total_size, ctx->socket_id, ctx->name);
                return -ENOMEM;
}

Konstantin

> 
> > -----Original Message-----
> > From: dev [mailto:dev-bounces@dpdk.org] On Behalf Of Rapelly, Varun
> > Sent: Friday, February 26, 2016 10:28 AM
> > To: dev@dpdk.org
> > Subject: Re: [dpdk-dev] ACL memory allocation failures
> >
> > Hi All,
> >
> > When I'm trying to configure some 5000+ ACL rules with different
> > source IP addresses, getting ACL memory allocation failure. I'm using DPDK 2.1.
> >
> > [root@ACLISSUE log_2015_10_26_08_19_42]# vim np.log match nodes/bytes
> > used: 816/104448
> > total: 12940832 bytes
> > ACL: Build phase for ACL "ipv4_acl_table2":
> > memory consumed: 947913495
> > ACL: trie 0: number of rules: 816
> > ACL: allocation of 12966528 bytes on socket 0 for ipv4_acl_table1
> > failed
> > ACL: Build phase for ACL "ipv4_acl_table1":
> > memory consumed: 947913495
> > ACL: trie 0: number of rules: 817
> > EAL: Error - exiting with code: 1
> >   Cause: Failed to build ACL trie
> >
> > Again sourced the ACL config file. After adding around 77 again the same error came.
> >
> > total: 14912784 bytes
> > ACL: Build phase for ACL "ipv4_acl_table1":
> > memory consumed: 1040188260
> > ACL: trie 0: number of rules: 893
> > ACL: allocation of 14938480 bytes on socket 0 for ipv4_acl_table2
> > failed
> 
> You are running out of hugepages memory.
> 
> > ACL: Build phase for ACL "ipv4_acl_table2":
> > memory consumed: 1040188260
> > ACL: trie 0: number of rules: 894
> > EAL: Error - exiting with code: 1
> >   Cause: Failed to build ACL trie
> >
> > Where to increase the memory to avoid this issue?
> 
>  Refer to:
> http://dpdk.org/doc/guides/linux_gsg/sys_reqs.html#running-dpdk-applications
> Section 2.3.2
> 
> Konstantin

  reply	other threads:[~2016-02-26 12:39 UTC|newest]

Thread overview: 7+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2016-02-26 10:28 Rapelly, Varun
2016-02-26 10:53 ` Ananyev, Konstantin
2016-02-26 11:55   ` Rapelly, Varun
2016-02-26 12:39     ` Ananyev, Konstantin [this message]
2016-02-29  5:00       ` Rapelly, Varun
2016-02-29 12:06         ` Rapelly, Varun
  -- strict thread matches above, loose matches on Subject: below --
2016-02-26 10:27 Rapelly, Varun

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=2601191342CEEE43887BDE71AB97725836B0BADD@irsmsx105.ger.corp.intel.com \
    --to=konstantin.ananyev@intel.com \
    --cc=dev@dpdk.org \
    --cc=vrapelly@sonusnet.com \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).