From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mga03.intel.com (mga03.intel.com [134.134.136.65]) by dpdk.org (Postfix) with ESMTP id C837F2B95 for ; Mon, 8 Apr 2019 10:42:02 +0200 (CEST) X-Amp-Result: SKIPPED(no attachment in message) X-Amp-File-Uploaded: False Received: from orsmga004.jf.intel.com ([10.7.209.38]) by orsmga103.jf.intel.com with ESMTP/TLS/DHE-RSA-AES256-GCM-SHA384; 08 Apr 2019 01:42:01 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.60,324,1549958400"; d="scan'208";a="289655097" Received: from vzlobin-mobl1.ccr.corp.intel.com (HELO [10.252.26.147]) ([10.252.26.147]) by orsmga004.jf.intel.com with ESMTP; 08 Apr 2019 01:42:00 -0700 To: "Jim Holland (jimholla)" , "dev@dpdk.org" Cc: "Mohamed Mahmoud (mmahmoud)" , "Min Tang (mtang2)" References: From: "Burakov, Anatoly" Message-ID: <839ef3df-c0d3-4705-f510-be517ef8d99f@intel.com> Date: Mon, 8 Apr 2019 09:41:59 +0100 User-Agent: Mozilla/5.0 (Windows NT 10.0; WOW64; rv:60.0) Gecko/20100101 Thunderbird/60.6.1 MIME-Version: 1.0 In-Reply-To: Content-Type: text/plain; charset=utf-8; format=flowed Content-Language: en-US Content-Transfer-Encoding: 7bit Subject: Re: [dpdk-dev] Secondary process crash in rte_eal_init X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 08 Apr 2019 08:42:03 -0000 On 06-Apr-19 3:26 PM, Jim Holland (jimholla) wrote: > Hi, > > We're seeing something similar to what is described in thread below. Our product uses dpdk 17.08. Was there ever a resolution to Souvik's issue? > > Thanks...Jim > > On 01/04/2015 06:00, Dey, Souvik wrote: >> Hi All, >> We have a single primary application with multiple secondary applications launched on the same cpu core. When the system boots up for the first time ,the primary comes up followed by the secondary process everything works fine. But in between if I try to restart the secondary processes without restart the primary, the secondary process crashes with the below stack trace. Any idea what is going wrong out here. >> >> ============================================ >> Stack Trace of the Crash : >> (gdb) bt >> #0 0x00007f4dad6e01b5 in raise () from /lib/libc.so.6 >> #1 0x00007f4dad6e2fc0 in abort () from /lib/libc.so.6 >> #2 0x0000000000402545 in __rte_panic () >> #3 0x00000000007353f4 in rte_eal_init () >> #4 0x0000000000403133 in main (argc=10, argv=0x7fff2c52fea8) >> ============================================ >> >> >> -- >> Regards, >> Souvik >> > Hi Dey, > > Could you try to reproduce the issue with one of the multi-process > examples from DPDK? > Also you could provide some info such as command line options when > running both primary/secondary apps, dpdk version, etc... > > Sergio > The crash you are seeing is a panic, but i don't see *what* is panic'ing, so i cannot diagnose the issue from stack trace alone. In general, restarting secondary processes during primary's runtime is a supported case, and is indeed the basis for one of our unit tests (granted, most secondary process spinups in our unit tests are supposed to fail, but some will pass), and i have at various points done "secondary process spam" tests firing up tens of thousands of secondaries as well. So, crashes on secondary initialization with valid parameters is certainly not normal behavior. A more detailed crash log would be good to have. -- Thanks, Anatoly From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from dpdk.org (dpdk.org [92.243.14.124]) by dpdk.space (Postfix) with ESMTP id D0D57A0096 for ; Mon, 8 Apr 2019 10:42:05 +0200 (CEST) Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id 5A4F52BC9; Mon, 8 Apr 2019 10:42:04 +0200 (CEST) Received: from mga03.intel.com (mga03.intel.com [134.134.136.65]) by dpdk.org (Postfix) with ESMTP id C837F2B95 for ; Mon, 8 Apr 2019 10:42:02 +0200 (CEST) X-Amp-Result: SKIPPED(no attachment in message) X-Amp-File-Uploaded: False Received: from orsmga004.jf.intel.com ([10.7.209.38]) by orsmga103.jf.intel.com with ESMTP/TLS/DHE-RSA-AES256-GCM-SHA384; 08 Apr 2019 01:42:01 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.60,324,1549958400"; d="scan'208";a="289655097" Received: from vzlobin-mobl1.ccr.corp.intel.com (HELO [10.252.26.147]) ([10.252.26.147]) by orsmga004.jf.intel.com with ESMTP; 08 Apr 2019 01:42:00 -0700 To: "Jim Holland (jimholla)" , "dev@dpdk.org" Cc: "Mohamed Mahmoud (mmahmoud)" , "Min Tang (mtang2)" References: From: "Burakov, Anatoly" Message-ID: <839ef3df-c0d3-4705-f510-be517ef8d99f@intel.com> Date: Mon, 8 Apr 2019 09:41:59 +0100 User-Agent: Mozilla/5.0 (Windows NT 10.0; WOW64; rv:60.0) Gecko/20100101 Thunderbird/60.6.1 MIME-Version: 1.0 In-Reply-To: Content-Type: text/plain; charset="UTF-8"; format="flowed" Content-Language: en-US Content-Transfer-Encoding: 7bit Subject: Re: [dpdk-dev] Secondary process crash in rte_eal_init X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" Message-ID: <20190408084159.bE-HSY5YbkmUiTh7iFccwugVFK1T8MVKYkSD7L6kB6A@z> On 06-Apr-19 3:26 PM, Jim Holland (jimholla) wrote: > Hi, > > We're seeing something similar to what is described in thread below. Our product uses dpdk 17.08. Was there ever a resolution to Souvik's issue? > > Thanks...Jim > > On 01/04/2015 06:00, Dey, Souvik wrote: >> Hi All, >> We have a single primary application with multiple secondary applications launched on the same cpu core. When the system boots up for the first time ,the primary comes up followed by the secondary process everything works fine. But in between if I try to restart the secondary processes without restart the primary, the secondary process crashes with the below stack trace. Any idea what is going wrong out here. >> >> ============================================ >> Stack Trace of the Crash : >> (gdb) bt >> #0 0x00007f4dad6e01b5 in raise () from /lib/libc.so.6 >> #1 0x00007f4dad6e2fc0 in abort () from /lib/libc.so.6 >> #2 0x0000000000402545 in __rte_panic () >> #3 0x00000000007353f4 in rte_eal_init () >> #4 0x0000000000403133 in main (argc=10, argv=0x7fff2c52fea8) >> ============================================ >> >> >> -- >> Regards, >> Souvik >> > Hi Dey, > > Could you try to reproduce the issue with one of the multi-process > examples from DPDK? > Also you could provide some info such as command line options when > running both primary/secondary apps, dpdk version, etc... > > Sergio > The crash you are seeing is a panic, but i don't see *what* is panic'ing, so i cannot diagnose the issue from stack trace alone. In general, restarting secondary processes during primary's runtime is a supported case, and is indeed the basis for one of our unit tests (granted, most secondary process spinups in our unit tests are supposed to fail, but some will pass), and i have at various points done "secondary process spam" tests firing up tens of thousands of secondaries as well. So, crashes on secondary initialization with valid parameters is certainly not normal behavior. A more detailed crash log would be good to have. -- Thanks, Anatoly