[dpdk-dev] backtracing from within the code

[dpdk-dev] backtracing from within the code

[Catalin Vasile]

Hi,

I'm trying to add a feature to DPDK and I'm having a hard time printing a backtrace.
I tried using this[1] functions for printing, but it does not print more than one function. Maybe it lacks the symbols it needs.
I tried compiling with "-rdynamic", but it breaks the compilation with an error of "bad -rpath option".
Can some help me out?

Catalin Vasile

[1] https://www.gnu.org/software/libc/manual/html_node/Backtraces.html

Re: [dpdk-dev] backtracing from within the code

[Dumitrescu, Cristian]

> -----Original Message-----
> From: dev [mailto:dev-bounces@dpdk.org] On Behalf Of Catalin Vasile
> Sent: Friday, June 24, 2016 9:10 AM
> To: dev@dpdk.org
> Subject: [dpdk-dev] backtracing from within the code
> 
> Hi,
> 
> I'm trying to add a feature to DPDK and I'm having a hard time printing a
> backtrace.
> I tried using this[1] functions for printing, but it does not print more than one
> function. Maybe it lacks the symbols it needs.
> I tried compiling with "-rdynamic", but it breaks the compilation with an error
> of "bad -rpath option".
> Can some help me out?
> 
> Catalin Vasile
> 
> [1] https://www.gnu.org/software/libc/manual/html_node/Backtraces.html

Hi Catalin,

You can look at the implementation of function rte_panic(), which is used to dump the call stack and exit.

It eventually calls rte_dump_stack() in file lib/lirte_eal/linuxapp/eal/eal_debug.c, which calls backtrace(), which is probably what you are looking for. 

Regards,
Cristian

Re: [dpdk-dev] backtracing from within the code

[Thomas Monjalon]

2016-06-24 09:25, Dumitrescu, Cristian:
> From: dev [mailto:dev-bounces@dpdk.org] On Behalf Of Catalin Vasile
> > I'm trying to add a feature to DPDK and I'm having a hard time printing a
> > backtrace.
> > I tried using this[1] functions for printing, but it does not print more than one
> > function. Maybe it lacks the symbols it needs.
[...]
> It eventually calls rte_dump_stack() in file lib/lirte_eal/linuxapp/eal/eal_debug.c, which calls backtrace(), which is probably what you are looking for. 

Example:
5: [build/app/testpmd(_start+0x29) [0x416f69]]
4: [/usr/lib/libc.so.6(__libc_start_main+0xf0) [0x7eff3b757610]]
3: [build/app/testpmd(main+0x2ff) [0x416b3f]]
2: [build/app/testpmd(init_port_config+0x88) [0x419a78]]
1: [build/lib/librte_eal.so.2.1(rte_dump_stack+0x18) [0x7eff3c126488]]

Please tell us if you have some cases where rte_dump_stack() does not work.
I do not remember what are the constraints to have it working.
Your binary is not stripped?

Re: [dpdk-dev] backtracing from within the code

[Stephen Hemminger]

On Fri, 24 Jun 2016 12:05:26 +0200
Thomas Monjalon <thomas.monjalon@6wind.com> wrote:

> 2016-06-24 09:25, Dumitrescu, Cristian:
> > From: dev [mailto:dev-bounces@dpdk.org] On Behalf Of Catalin Vasile
> > > I'm trying to add a feature to DPDK and I'm having a hard time printing a
> > > backtrace.
> > > I tried using this[1] functions for printing, but it does not print more than one
> > > function. Maybe it lacks the symbols it needs.
> [...]
> > It eventually calls rte_dump_stack() in file lib/lirte_eal/linuxapp/eal/eal_debug.c, which calls backtrace(), which is probably what you are looking for. 
> 
> Example:
> 5: [build/app/testpmd(_start+0x29) [0x416f69]]
> 4: [/usr/lib/libc.so.6(__libc_start_main+0xf0) [0x7eff3b757610]]
> 3: [build/app/testpmd(main+0x2ff) [0x416b3f]]
> 2: [build/app/testpmd(init_port_config+0x88) [0x419a78]]
> 1: [build/lib/librte_eal.so.2.1(rte_dump_stack+0x18) [0x7eff3c126488]]
> 
> Please tell us if you have some cases where rte_dump_stack() does not work.
> I do not remember what are the constraints to have it working.
> Your binary is not stripped?

The GCC backtrace doesn't work well because it can't find static functions.
I ended up using libunwind to get a better back trace.

Re: [dpdk-dev] backtracing from within the code

[Rosen, Rami]

Hi,
If you are willing to skip static methods and use the GCC backtrace, you can 
try this example (it worked for me, but it was quite a time ago):
http://www.helicontech.co.il/?id=linuxbt

Regards,
Rami Rosen
Intel Corporation

-----Original Message-----
From: dev [mailto:dev-bounces@dpdk.org] On Behalf Of Stephen Hemminger
Sent: Friday, June 24, 2016 8:46 PM
To: Thomas Monjalon <thomas.monjalon@6wind.com>
Cc: Catalin Vasile <cata.vasile@nxp.com>; dev@dpdk.org; Dumitrescu, Cristian <cristian.dumitrescu@intel.com>
Subject: Re: [dpdk-dev] backtracing from within the code

On Fri, 24 Jun 2016 12:05:26 +0200
Thomas Monjalon <thomas.monjalon@6wind.com> wrote:

> 2016-06-24 09:25, Dumitrescu, Cristian:
> > From: dev [mailto:dev-bounces@dpdk.org] On Behalf Of Catalin Vasile
> > > I'm trying to add a feature to DPDK and I'm having a hard time printing a
> > > backtrace.
> > > I tried using this[1] functions for printing, but it does not print more than one
> > > function. Maybe it lacks the symbols it needs.
> [...]
> > It eventually calls rte_dump_stack() in file lib/lirte_eal/linuxapp/eal/eal_debug.c, which calls backtrace(), which is probably what you are looking for. 
> 
> Example:
> 5: [build/app/testpmd(_start+0x29) [0x416f69]]
> 4: [/usr/lib/libc.so.6(__libc_start_main+0xf0) [0x7eff3b757610]]
> 3: [build/app/testpmd(main+0x2ff) [0x416b3f]]
> 2: [build/app/testpmd(init_port_config+0x88) [0x419a78]]
> 1: [build/lib/librte_eal.so.2.1(rte_dump_stack+0x18) [0x7eff3c126488]]
> 
> Please tell us if you have some cases where rte_dump_stack() does not work.
> I do not remember what are the constraints to have it working.
> Your binary is not stripped?

The GCC backtrace doesn't work well because it can't find static functions.
I ended up using libunwind to get a better back trace.

Re: [dpdk-dev] backtracing from within the code

[Matt Laswell]

I've done something similar to what's described in the link below.  But
it's worth pointing out that it's using printf() inside a signal handler,
which isn't safe. If your use case is catching SIGSEGV, for example,
solutions built on printf() will usually work, but can deadlock.  One way
around the problem is to call write() directly, passing it stdout's file
handle.

For example, I have this in my code:
#define WRITE_STRING(fd, s) write (fd, s, strlen (s))

In my signal handlers, I use the above like this:
        WRITE_STRING(STDOUT_FILENO, "Stack trace:\n");

This approach is a little bit more cumbersome to code, but safer.

The last time that I looked the DPDK rte_dump_stack() is using vfprintf(),
which isn't safe in a signal handler.  However, it's been several DPDK
releases since I peeked at the details.

--
Matt Laswell
Principal Software Engineer
infinite io, inc.
laswell@infinite.io


On Sat, Jun 25, 2016 at 9:07 AM, Rosen, Rami <rami.rosen@intel.com> wrote:

> Hi,
> If you are willing to skip static methods and use the GCC backtrace, you
> can
> try this example (it worked for me, but it was quite a time ago):
> http://www.helicontech.co.il/?id=linuxbt
>
> Regards,
> Rami Rosen
> Intel Corporation
>
> -----Original Message-----
> From: dev [mailto:dev-bounces@dpdk.org] On Behalf Of Stephen Hemminger
> Sent: Friday, June 24, 2016 8:46 PM
> To: Thomas Monjalon <thomas.monjalon@6wind.com>
> Cc: Catalin Vasile <cata.vasile@nxp.com>; dev@dpdk.org; Dumitrescu,
> Cristian <cristian.dumitrescu@intel.com>
> Subject: Re: [dpdk-dev] backtracing from within the code
>
> On Fri, 24 Jun 2016 12:05:26 +0200
> Thomas Monjalon <thomas.monjalon@6wind.com> wrote:
>
> > 2016-06-24 09:25, Dumitrescu, Cristian:
> > > From: dev [mailto:dev-bounces@dpdk.org] On Behalf Of Catalin Vasile
> > > > I'm trying to add a feature to DPDK and I'm having a hard time
> printing a
> > > > backtrace.
> > > > I tried using this[1] functions for printing, but it does not print
> more than one
> > > > function. Maybe it lacks the symbols it needs.
> > [...]
> > > It eventually calls rte_dump_stack() in file
> lib/lirte_eal/linuxapp/eal/eal_debug.c, which calls backtrace(), which is
> probably what you are looking for.
> >
> > Example:
> > 5: [build/app/testpmd(_start+0x29) [0x416f69]]
> > 4: [/usr/lib/libc.so.6(__libc_start_main+0xf0) [0x7eff3b757610]]
> > 3: [build/app/testpmd(main+0x2ff) [0x416b3f]]
> > 2: [build/app/testpmd(init_port_config+0x88) [0x419a78]]
> > 1: [build/lib/librte_eal.so.2.1(rte_dump_stack+0x18) [0x7eff3c126488]]
> >
> > Please tell us if you have some cases where rte_dump_stack() does not
> work.
> > I do not remember what are the constraints to have it working.
> > Your binary is not stripped?
>
> The GCC backtrace doesn't work well because it can't find static functions.
> I ended up using libunwind to get a better back trace.
>