From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 90B61A00C4; Mon, 18 Apr 2022 17:19:04 +0200 (CEST) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 63F184014F; Mon, 18 Apr 2022 17:19:03 +0200 (CEST) Received: from mx0b-00169c01.pphosted.com (mx0b-00169c01.pphosted.com [67.231.156.123]) by mails.dpdk.org (Postfix) with ESMTP id BC9B740141 for ; Mon, 18 Apr 2022 17:19:01 +0200 (CEST) Received: from pps.filterd (m0281122.ppops.net [127.0.0.1]) by mx0b-00169c01.pphosted.com (8.17.1.5/8.17.1.5) with ESMTP id 23I86djT006555 for ; Mon, 18 Apr 2022 08:19:00 -0700 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=paloaltonetworks.com; h=mime-version : references : in-reply-to : from : date : message-id : subject : to : cc : content-type; s=PPS12012017; bh=0DwJgB0Sjiq0Qp3R3Cd7c2Q/26ljz+2BYsUgTAP0EIE=; b=ieLe2dqCv1HhFyNJB0JBhaJI/9mKH6nc6o7IPjS49saeXaF/jZ0GP+8JvKglcr5jvELf BLB2+76Un3xppy6WSGtps7bPMzL+J3KHtNyET5uJMFg9rpiVNdtoPwTuwsy1eGnehgMR 6A5IL55wZ1bSJRE7Mu3P+UjKO7HFMrBcL/J7dO/wJ3jxIH1HA+me3yOYaAGOpafKBjq0 cMoAMzZf8EE2Tqtagr2E5/bKCsJg3naEsRVmZ/G+rdMDNnQCjcEAfOIWY+OKy+a6vyvb R1gcHa22pElbR6+eYkVz8l1AGQfOhFVvueMAPmnmMwb9VK0w4Bq+GZMEfuB7N67obAxL Tg== Received: from mail-vs1-f69.google.com (mail-vs1-f69.google.com [209.85.217.69]) by mx0b-00169c01.pphosted.com (PPS) with ESMTPS id 3ffudjbh22-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128 verify=NOT) for ; Mon, 18 Apr 2022 08:19:00 -0700 Received: by mail-vs1-f69.google.com with SMTP id a6-20020a67b306000000b0032a14b43a75so1653889vsm.12 for ; Mon, 18 Apr 2022 08:19:00 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=paloaltonetworks-com.20210112.gappssmtp.com; s=20210112; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :cc; bh=0DwJgB0Sjiq0Qp3R3Cd7c2Q/26ljz+2BYsUgTAP0EIE=; b=xnjagcPpe+73tFFz6InqhoErGbKMl63aR60PdS+tdKv18VfGLJ3y3GKE8JL7pFgA5j Y35x33MF3lv6k9A5qZsNu64kL2VhYfuLzWTM2SQaIeMqp3pe49KmBIKAAwQuM0McxNaQ iTbglDqpL8llUdTxV28j+hxYyT3m4OFmp12jlW8GV/g7jSUI6vK2rqzLYyWQUYonRPlG kEBZLfRXshQIb8uPNT1H/RbQQjQFri1M2to1Glqf3y1npOxTtnpld6Wim7TmjA4/1ANo ZsZhFr1G5DRSR5+lsG9A9Ooj/dKsNHxQu9d3bf81KsKTlTRTWX4Wk18zqFsU0qKrtiv7 +kUw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=0DwJgB0Sjiq0Qp3R3Cd7c2Q/26ljz+2BYsUgTAP0EIE=; b=Gmc3X5nxpPevCxeWazVPknnnVPR+OSS1NB9Sr4tuCkJCSXtKQH9Em/2sBy+X9nZIRS d1uy6X+XSNoHemcLWwOxyef1/vylbOGsRKZwJYkA234l5DqbSgXewMpuk+KJO09R280J i4gTDi3k8Vwra4IRRfXaPwDRv/fdm0qkcdMuPCv3JWK8dg2lPqciwjTmZ9XpY43+i1// HWxo0jEHippljSecizlKqdemRy3epdUe9fqZMg0R12XNvtcAXCfZjRlVZ3CpaDNILGWt C5xHez4YBJH2ftAt4Nb3XbB0b1cVGlae2cvYbzD/+b75IdsqW3mZTf30f0teTNJ8bBP9 Hcrg== X-Gm-Message-State: AOAM530ypY+ZAHEbFVfr5CdyaBqbJEMjezUYxQQi964kBkj4sL6d6I6Y x75fpFPQ8RRNQ/Mm4/hnFGRPr1ED2chZlAcNWe4xGKLxqcSV0sdtW5xf2hL7+QFMggGs5ruEYF9 9R9bMD/21YqNKolpelNM= X-Received: by 2002:a05:6102:18a:b0:32a:3b90:b1b8 with SMTP id r10-20020a056102018a00b0032a3b90b1b8mr3303482vsq.15.1650295139749; Mon, 18 Apr 2022 08:18:59 -0700 (PDT) X-Google-Smtp-Source: ABdhPJzv1v0HjAjCEtBGKaExVP4wTvOdDVpabP7+BeMXah0kVfU0FB4ktAVFIAAA+9lh9mK/qDcYrzZ/WkBtsoU+C7Q= X-Received: by 2002:a05:6102:18a:b0:32a:3b90:b1b8 with SMTP id r10-20020a056102018a00b0032a3b90b1b8mr3303472vsq.15.1650295139448; Mon, 18 Apr 2022 08:18:59 -0700 (PDT) MIME-Version: 1.0 References: In-Reply-To: From: Amiya Mohakud Date: Mon, 18 Apr 2022 20:48:48 +0530 Message-ID: Subject: Re: DPDK:20.11.1: net/ena crash while fetching xstats To: dev@dpdk.org Cc: Sachin Kanoje , Megha Punjani , Sharad Saha , Eswar Sadaram Content-Type: multipart/alternative; boundary="0000000000008fd5a805dcef4723" X-Proofpoint-GUID: hbpdNnehCBjsKF8T1mHkhD6TNXrGq0Ck X-Proofpoint-ORIG-GUID: hbpdNnehCBjsKF8T1mHkhD6TNXrGq0Ck X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.205,Aquarius:18.0.858,Hydra:6.0.486,FMLib:17.11.64.514 definitions=2022-04-18_02,2022-04-15_01,2022-02-23_01 X-Proofpoint-Spam-Details: rule=outbound_spam_notspam policy=outbound_spam score=0 bulkscore=0 spamscore=0 suspectscore=0 malwarescore=0 mlxlogscore=796 phishscore=0 mlxscore=0 impostorscore=0 priorityscore=1501 adultscore=0 lowpriorityscore=0 clxscore=1015 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2202240000 definitions=main-2204180091 X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org --0000000000008fd5a805dcef4723 Content-Type: text/plain; charset="UTF-8" + Megha, Sharad and Eswar. On Mon, Apr 18, 2022 at 2:03 PM Amiya Mohakud wrote: > Hi Michal/DPDK-Experts, > > I am facing one issue in net/ena driver while fetching extended stats > (xstats). The DPDK seems to segfault with below backtrace. > > DPDK Version: 20.11.1 > ENA version: 2.2.1 > > > Using host libthread_db library "/lib64/libthread_db.so.1". > > Core was generated by `/opt/dpfs/usr/local/bin/brdagent'. > > Program terminated with signal SIGSEGV, Segmentation fault. > > #0 __memmove_avx_unaligned_erms () at > ../sysdeps/x86_64/multiarch/memmove-vec-unaligned-erms.S:232 > > 232 VMOVU %VEC(0), (%rdi) > > [Current thread is 1 (Thread 0x7fffed93a400 (LWP 5060))] > > > Thread 1 (Thread 0x7fffed93a400 (LWP 5060)): > > #0 __memmove_avx_unaligned_erms () at > ../sysdeps/x86_64/multiarch/memmove-vec-unaligned-erms.S:232 > > #1 0x00007ffff3c246df in ena_com_handle_admin_completion () from > ../lib64/../../lib64/libdpdk.so.20 > > #2 0x00007ffff3c1e7f5 in ena_interrupt_handler_rte () from > ../lib64/../../lib64/libdpdk.so.20 > > #3 0x00007ffff3519902 in eal_intr_thread_main () from > /../lib64/../../lib64/libdpdk.so.20 > > #4 0x00007ffff510714a in start_thread (arg=) at > pthread_create.c:479 > > #5 0x00007ffff561ff23 in clone () at > ../sysdeps/unix/sysv/linux/x86_64/clone.S:95 > > > > > *Background:* > > - This used to work fine with DPDK-19.11.3 , that means there was no > crash observed with the 19.11.3 DPDK version, but now after upgrading to > DPDK 20.11.1, DPDK is crashing with the above trace. > - It looks to me as a DPDK issue. > - I could see multiple fixes/patches in the net/ena area, but not able > to identify which patch would exactly fix this issue. > > For example: > http://git.dpdk.org/dpdk/diff/?h=releases&id=aab58857330bb4bd03f6699bf1ee716f72993774 > > https://inbox.dpdk.org/dev/20210430125725.28796-6-mk@semihalf.com/T/#me99457c706718bb236d1fd8006ee7a0319ce76fc > > > Could you please help here and let me know what patch could fix this issue. > > > Regards > Amiya > --0000000000008fd5a805dcef4723 Content-Type: text/html; charset="UTF-8" Content-Transfer-Encoding: quoted-printable
+ Megha, Sharad and Eswar.

On Mon, Apr 18, 2022 at 2:0= 3 PM Amiya Mohakud <amo= hakud@paloaltonetworks.com> wrote:
Hi Michal/DPDK-Experts,

I am fac= ing one issue in net/ena driver while fetching extended stats (xstats). The= DPDK seems to segfault with below backtrace.=C2=A0

DPDK Version: 20.1= 1.1
ENA version: 2.2.1


Using host libthread_db library "/lib64/libthread_db.so.1"= .

Core was generated by `/opt/dpfs/usr/local/bin/brdagent'.

Program terminated with signal SIGSEGV, Segmentation fault.

#0=C2=A0 __memmove_avx_unaligned_erms () at ../sysdeps/= x86_64/multiarch/memmove-vec-unaligned-erms.S:232

232 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 VMOVU =C2=A0 %VEC(0), (%rdi)

[Current thread is 1 (Thread 0x7fffed93a400 (LWP 5060))]


Thread 1 (Thread 0x7fffed93a400 (LWP 5060)):

#0=C2=A0 __memmove_avx_unaligned_erms () at ../sysdeps/= x86_64/multiarch/memmove-vec-unaligned-erms.S:232

#1=C2=A0 0x00007ffff3c246df in ena_com_handle_admin_com= pletion () from ../lib64/../../lib64/libdpdk.so.20

#2=C2=A0 0x00007ffff3c1e7f5 in ena_interrupt_handler_rt= e () from ../lib64/../../lib64/libdpdk.so.20

#3=C2=A0 0x00007ffff3519902 in eal_intr_thread_main () = from /../lib64/../../lib64/libdpdk.so.20

#4=C2=A0 0x00007ffff510714a in start_thread (arg=3D<= optimized out>) at pthread_create.c:479

#5=C2=A0 0x00007ffff561ff23 in clone () at ../sysdeps/u= nix/sysv/linux/x86_64/clone.S:95



<= br>

Background:
  • This used to work fine with DPDK-19.11.3 , that=C2=A0= means there was no crash observed with the 19.11.3 DPDK version, but now af= ter upgrading to DPDK 20.11.1, DPDK is crashing with the above trace.=C2=A0=
  • It looks to me as a DPDK issue.
  • I could see multiple fixes= /patches in the net/ena area, but not able to identify which patch would ex= actly fix this issue.


Could you plea= se help here and let me know what patch could fix this issue.


Regards
Amiya
--0000000000008fd5a805dcef4723--