DPDK patches and discussions
 help / color / mirror / Atom feed
From: Bruce Richardson <bruce.richardson@intel.com>
To: Kamraan Nasim <knasim@sidebandnetworks.com>
Cc: dev@dpdk.org
Subject: Re: [dpdk-dev] Panic in rte MEMPOOL__mempool_check_cookies()
Date: Thu, 13 Nov 2014 12:59:52 +0000	[thread overview]
Message-ID: <20141113125952.GB9932@bricha3-MOBL3> (raw)
In-Reply-To: <CAPrTskjQaUrk5JsBQyrxXTdFEf4kzqyVG5+XUbUdKeAZSuqAdQ@mail.gmail.com>

On Wed, Nov 12, 2014 at 09:50:50PM -0500, Kamraan Nasim wrote:
> Hi Bruce,
> 
> The issue seems to be triggered by a combination of using the
> CONFIG_RTE_LIBRTE_MEMPOOL_DEBUG  option and contention in the mempool(small
> mempool size).
> 
> I disabled that and the header panics went away.
> 
> Thank you for your help!

No problem, though it does mean that we have a bug in the mempool debug functions
that will have to be investigated and fixed in the future. :-(

> 
> --Kam
> 
> On Fri, Nov 7, 2014 at 4:48 AM, Bruce Richardson <bruce.richardson@intel.com
> > wrote:
> 
> > On Thu, Nov 06, 2014 at 04:28:18PM -0500, Kamraan Nasim wrote:
> > > Greetings,
> > >
> > > I have been hitting this issue fairly consistently for the ixgbe driver
> > >
> > > MEMPOOL: obj=0x7ffeed1f5d00, mempool=0x7ffeecb69bc0,
> > cookie=badbadbadadd2e55
> > > PANIC in __mempool_check_cookies():
> > > MEMPOOL: bad header cookie (get)
> > >
> > > It seems to be a corruption in the mempool bound to my ixgbe port. What I
> > > have observed is that this ONLY happens if I initialize dpdk(i.e. start
> > > dpdk application) AFTER traffic is already flowing in through the port.
> > If
> > > I initialize dpdk and bind BEFORE I start traffic then things seem to
> > work
> > > fine.
> > >
> > > Any clues on why this might be happening?
> > >
> > > A bit stumped, so would really appreciate all the help I can get on this
> > > one.
> > >
> >
> > Hi.
> > Can you confirm if your application works ok with the mempool debug options
> > disabled? The mempool debug options are not often used because they slow
> > things
> > down so it's possible the bug could be in the cookie checking itself.
> >
> > /Bruce
> >
> > > Thanks,
> > > Kam
> > >
> > >
> > > (bt for your reference)
> > >
> > > #2  0x0000000000408cc6 in __rte_panic (funcname=0x571100
> > > "__mempool_check_cookies", format=
> > >     0x568fb0 "MEMPOOL: bad header cookie (get)\n%.0s")
> > >     at
> > >
> > /b/knasim/bandwagon/sbn/src/share/dpdk/lib/librte_eal/linuxapp/eal/eal_debug.c:83
> > > #3  0x00000000004af027 in __mempool_check_cookies (rxq=<value optimized
> > > out>)
> > >     at
> > >
> > /b/knasim/bandwagon/sbn/src/share/dpdk/x86_64-native-linuxapp-gcc/include/rte_mempool.h:357
> > > #4  rte_mempool_get_bulk (rxq=<value optimized out>)
> > >     at
> > >
> > /b/knasim/bandwagon/sbn/src/share/dpdk/x86_64-native-linuxapp-gcc/include/rte_mempool.h:1094
> > > #5  ixgbe_rx_alloc_bufs (rxq=<value optimized out>)
> > >     at
> > >
> > /b/knasim/bandwagon/sbn/src/share/dpdk/lib/librte_pmd_ixgbe/ixgbe_rxtx.c:997
> > > #6  0x00000000004afce9 in rx_recv_pkts (rx_queue=0x7ffeec8edbc0,
> > > rx_pkts=0x900410,
> > >     nb_pkts=<value optimized out>)
> > >     at
> > >
> > /b/knasim/bandwagon/sbn/src/share/dpdk/lib/librte_pmd_ixgbe/ixgbe_rxtx.c:1074
> > > #7  ixgbe_recv_pkts_bulk_alloc (rx_queue=0x7ffeec8edbc0,
> > rx_pkts=0x900410,
> > > nb_pkts=<value optimized out>)
> > >     at
> > >
> > /b/knasim/bandwagon/sbn/src/share/dpdk/lib/librte_pmd_ixgbe/ixgbe_rxtx.c:1124
> > > #8  0x0000000000520d36 in rte_eth_rx_burst (lp=0x900340, n_workers=14,
> > > bsz_rd=<value optimized out>, bsz_wr=
> > >     144, pos_lb=0 '\000') at /usr/lib/dpdk/include/rte_ethdev.h:2368
> >

      reply	other threads:[~2014-11-13 12:49 UTC|newest]

Thread overview: 4+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2014-11-06 21:28 Kamraan Nasim
2014-11-07  9:48 ` Bruce Richardson
2014-11-13  2:50   ` Kamraan Nasim
2014-11-13 12:59     ` Bruce Richardson [this message]

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=20141113125952.GB9932@bricha3-MOBL3 \
    --to=bruce.richardson@intel.com \
    --cc=dev@dpdk.org \
    --cc=knasim@sidebandnetworks.com \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).