Re: [dpdk-dev] Secondary process crash in rte_eal_init

["Burakov, Anatoly" <anatoly.burakov@intel.com>]

On 08-Apr-19 2:07 PM, Mohamed Mahmoud (mmahmoud) wrote:
> Hi Anatoly:
> 
> For the secondary process all we are setting is the –proc-type set to 
> secondary, maybe point me to your test that might give me some clues, 
> also How I can generate detailed crash log ?
> 
> When I checked the init code there are very few places that can cause 
> the panic
> 
>    if(pipe(lcore_config[i].pipe_master2slave) < 0)
> 
> rte_panic("Cannot create pipe\n");
> 
> if(pipe(lcore_config[i].pipe_slave2master) < 0)
> 
> rte_panic("Cannot create pipe\n");
> 
>          lcore_config[i].state = WAIT;
> 
> /* create a thread for each lcore */
> 
>          ret = pthread_create(&lcore_config[i].thread_id, NULL,
> 
>                       eal_thread_loop, NULL);
> 
> if(ret != 0)
> 
> rte_panic("Cannot create thread\n");

Hi,

That would be the most recent DPDK version. The issue i was responding 
to was for version 17.08, which has quite a bit more panic's in init code.

In order for me to diagnose the issue i would have to at least know 
where the panic happens. That information is missing from both reports, 
so i cannot begin to imagine what went wrong, let alone why.

> 
> Thanks,
> 
> Mohamed
> 
> *From: *"Burakov, Anatoly" <anatoly.burakov@intel.com>
> *Date: *Monday, April 8, 2019 at 4:42 AM
> *To: *"Jim Holland (jimholla)" <jimholla@cisco.com>, "dev@dpdk.org" 
> <dev@dpdk.org>
> *Cc: *"Mohamed Mahmoud (mmahmoud)" <mmahmoud@cisco.com>, "Min Tang 
> (mtang2)" <mtang2@cisco.com>
> *Subject: *Re: [dpdk-dev] Secondary process crash in rte_eal_init
> 
> On 06-Apr-19 3:26 PM, Jim Holland (jimholla) wrote:
> 
>     Hi,
> 
>     We're seeing something similar to what is described in thread below.
>     Our product uses dpdk 17.08. Was there ever a resolution to Souvik's
>     issue?
> 
>     Thanks...Jim
> 
>     On 01/04/2015 06:00, Dey, Souvik wrote:
> 
>         Hi All,
> 
>                             We have a single primary application with
>         multiple secondary applications launched on the same cpu core.
>         When the system boots up for the first time ,the primary comes
>         up followed by the secondary process everything works fine. But
>         in between if I try to restart the secondary processes without
>         restart the primary, the secondary process crashes with the
>         below stack trace. Any idea what is going wrong out here.
> 
>         ============================================
> 
>         Stack Trace of the Crash :
> 
>         (gdb) bt
> 
>         #0 0x00007f4dad6e01b5 in raise () from /lib/libc.so.6
> 
>         #1 0x00007f4dad6e2fc0 in abort () from /lib/libc.so.6
> 
>         #2 0x0000000000402545 in __rte_panic ()
> 
>         #3 0x00000000007353f4 in rte_eal_init ()
> 
>         #4 0x0000000000403133 in main (argc=10, argv=0x7fff2c52fea8)
> 
>         ============================================
> 
>         --
> 
>         Regards,
> 
>         Souvik
> 
>     Hi Dey,
> 
>     Could you try to reproduce the issue with one of the multi-process
> 
>     examples from DPDK?
> 
>     Also you could provide some info such as command line options when
> 
>     running both primary/secondary apps, dpdk version, etc...
> 
>     Sergio
> 
> The crash you are seeing is a panic, but i don't see *what* is
> 
> panic'ing, so i cannot diagnose the issue from stack trace alone.
> 
> In general, restarting secondary processes during primary's runtime is a
> 
> supported case, and is indeed the basis for one of our unit tests
> 
> (granted, most secondary process spinups in our unit tests are supposed
> 
> to fail, but some will pass), and i have at various points done
> 
> "secondary process spam" tests firing up tens of thousands of
> 
> secondaries as well. So, crashes on secondary initialization with valid
> 
> parameters is certainly not normal behavior.
> 
> A more detailed crash log would be good to have.
> 
> -- 
> 
> Thanks,
> 
> Anatoly
> 


-- 
Thanks,
Anatoly