From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mailfilter02.viettel.com.vn (mailfilter02.viettel.com.vn [125.235.240.54]) by dpdk.org (Postfix) with ESMTP id E902A16829 for ; Fri, 8 Sep 2017 06:42:10 +0200 (CEST) X-IronPort-AV: E=Sophos;i="5.42,360,1500915600"; d="scan'208";a="55023526" Received: from 125.235.240.44.adsl.viettel.vn (HELO mta1.viettel.com.vn) ([125.235.240.44]) by mailfilter02.viettel.com.vn with ESMTP; 08 Sep 2017 11:42:05 +0700 Received: from localhost (localhost [127.0.0.1]) by mta1.viettel.com.vn (Postfix) with ESMTP id 0151A622682; Fri, 8 Sep 2017 11:41:53 +0700 (ICT) Received: from mta1.viettel.com.vn ([127.0.0.1]) by localhost (mta1.viettel.com.vn [127.0.0.1]) (amavisd-new, port 10032) with ESMTP id SJBkcG1xUaNF; Fri, 8 Sep 2017 11:41:52 +0700 (ICT) Received: from localhost (localhost [127.0.0.1]) by mta1.viettel.com.vn (Postfix) with ESMTP id D109E622688; Fri, 8 Sep 2017 11:41:52 +0700 (ICT) X-Virus-Scanned: amavisd-new at Received: from mta1.viettel.com.vn ([127.0.0.1]) by localhost (mta1.viettel.com.vn [127.0.0.1]) (amavisd-new, port 10026) with ESMTP id SnZ1BsITSQ0P; Fri, 8 Sep 2017 11:41:52 +0700 (ICT) Received: from zimbra.viettel.com.vn (mailbox3.viettel.com.vn [10.30.182.138]) by mta1.viettel.com.vn (Postfix) with ESMTP id A635A622685; Fri, 8 Sep 2017 11:41:52 +0700 (ICT) To: keith wiles Cc: users@dpdk.org, skhare@vmware.com Message-ID: <1102618336.80141455.1504845712695.JavaMail.zimbra@viettel.com.vn> In-Reply-To: References: <142883318.73479844.1504745082183.JavaMail.zimbra@viettel.com.vn> <1551920177.73501197.1504745506408.JavaMail.zimbra@viettel.com.vn> <20170907075808.09fc227a@xeon-e3> <635485224.78963862.1504836430391.JavaMail.zimbra@viettel.com.vn> <44199C9D-EC75-495D-9FA2-EFB2CBC0BB79@intel.com> <1873902514.79394236.1504839292666.JavaMail.zimbra@viettel.com.vn> MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable X-Originating-IP: [27.72.58.160] X-Mailer: Zimbra 8.0.9_GA_6191 (ZimbraWebClient - GC60 (Win)/8.0.9_GA_6191) Thread-Topic: [dpdk-users] [VMXNET3] vmxnet3_dev_stats_get() segfault when called by pktgen Thread-Index: AQHTJ+nL5iLJI4tGv0ydrpmXk5BmVKKqGByAx/WGBme4CxoAAGCS0dVX/Pt7+4Bop0fV1g== MilterAction: FORWARD Date: Fri, 8 Sep 2017 11:41:52 +0700 (ICT) From: longtb5@viettel.com.vn Subject: Re: [dpdk-users] [VMXNET3] vmxnet3_dev_stats_get() segfault when called by pktgen X-BeenThere: users@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK usage discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 08 Sep 2017 04:42:11 -0000 Hi Keith, All of the test runs that I listed gave me the same segfault from the same = function in the driver.=20 The config was not for any purposes other than trying to start pktgen. Mayb= e it is indeed a=20 driver problem then. I was able to run pktgen by replacing cli_start_with_timers(NULL); in pktgen_cli_start() in app/cli-functions.c, with cli_start(NULL); effectively disabled this flag setup=20 this_cli->flags |=3D CLI_USE_TIMERS With that I was able to run pktgen, but segfault still happens when I quit = or want to access the stats page. Do you have any ideas why this might happen? Regards, BL ----- Original Message ----- From: "keith wiles" To: longtb5@viettel.com.vn Sent: Friday, September 8, 2017 10:28:46 AM Subject: Re: [dpdk-users] [VMXNET3] vmxnet3_dev_stats_get() segfault when c= alled by pktgen > On Sep 7, 2017, at 7:54 PM, longtb5@viettel.com.vn wrote: >=20 > I tested these: > $ sudo ./app/x86_64-native-linuxapp-gcc/pktgen > $ sudo ./app/x86_64-native-linuxapp-gcc/pktgen -l 0-3 -n 2 The above will not work as you do not have the =E2=80=94 arguments. > $ sudo ./app/x86_64-native-linuxapp-gcc/pktgen -l 0-3 -n 2 -- -P -m "1.0,= 2.1=E2=80=9D These above look fine except you use 4 cores and only use one for pktgen an= d 1 for each port for a total of three, you could have used -l 0-2 If the above still give the segfault then I do not know what is happening h= ere and it does look like a driver problem?? > $ sudo ./tools/dpdk-run.py default This one seem odd to me. >=20 > This is my default.cfg > # Run command and options > run =3D { > 'exec': ( > 'sudo', > '-E' > ), >=20 > # Application name and use app_path to help locate the app > 'app_name': 'pktgen', >=20 > # using (sdk) or (target) for specific variables > # add (app_name) of the application > # Each path is tested for the application > 'app_path': ( > './app/%(target)s/%(app_name)s', > '%(sdk)s/%(target)s/app/%(app_name)s', > ), >=20 > =09'dpdk': ( > =09=09'-l 0-1', > =09=09'-n 2', > =09=09'--proc-type auto', > =09=09'--log-level 7', > =09=09#'--socket-mem 2048,2048', > =09=09#'--file-prefix pg' > =09=09), > =09 >=20 > =09'app': ( > =09=09'-T', > =09=09'-P', > =09=09'--crc-strip', > =09=09'-m 1.[2/5]=E2=80=99 Only doing one port trying to control port 2 and port 5?? Should this not b= e just 1.0 for one port. > =09=09), > =09 > =09'misc': ( > =09=09#'-f', > =09=09'themes/black-yellow.theme' > =09=09) > =09} >=20 > $ sudo ~/dev/dpdk/usertools/cpu_layout.py > cores =3D [0, 1] > sockets =3D [0, 1] >=20 > Socket 0 Socket 1 =20 > -------- -------- =20 > Core 0 [0] [2] =20 > Core 1 [1] [3] =20 >=20 > $ sudo ./dpdk/usertools/dpdk-devbind.py -s >=20 > Network devices using DPDK-compatible driver > =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D > 0000:03:00.0 'VMXNET3 Ethernet Controller 07b0' drv=3Digb_uio unused=3D > 0000:04:00.0 'VMXNET3 Ethernet Controller 07b0' drv=3Digb_uio unused=3D > 0000:0b:00.0 'VMXNET3 Ethernet Controller 07b0' drv=3Digb_uio unused=3D > 0000:0c:00.0 'VMXNET3 Ethernet Controller 07b0' drv=3Digb_uio unused=3D > 0000:13:00.0 'VMXNET3 Ethernet Controller 07b0' drv=3Digb_uio unused=3D > 0000:14:00.0 'VMXNET3 Ethernet Controller 07b0' drv=3Digb_uio unused=3D > 0000:1b:00.0 'VMXNET3 Ethernet Controller 07b0' drv=3Digb_uio unused=3D > 0000:1c:00.0 'VMXNET3 Ethernet Controller 07b0' drv=3Digb_uio unused=3D >=20 > Network devices using kernel driver > =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D > 0000:05:00.0 'VMXNET3 Ethernet Controller 07b0' if=3Dens162 drv=3Dvmxnet3= unused=3Digb_uio *Active* > 0000:0d:00.0 'VMXNET3 Ethernet Controller 07b0' if=3Dens194 drv=3Dvmxnet3= unused=3Digb_uio *Active* >=20 >=20 >>=20 >> ----- Original Message ----- >> From: "keith wiles" >> To: longtb5@viettel.com.vn >> Cc: "Stephen Hemminger" , users@dpdk.org, sk= hare@vmware.com >> Sent: Friday, September 8, 2017 9:22:25 AM >> Subject: Re: [dpdk-users] [VMXNET3] vmxnet3_dev_stats_get() segfault whe= n called by pktgen >>=20 >>=20 >>> On Sep 7, 2017, at 7:07 PM, longtb5@viettel.com.vn wrote: >>>=20 >>>> ----- Original Message ----- >>>> From: "keith wiles" >>>> To: "Stephen Hemminger" >>>> Cc: longtb5@viettel.com.vn, users@dpdk.org >>>> Sent: Thursday, September 7, 2017 11:49:39 PM >>>> Subject: Re: [dpdk-users] [VMXNET3] vmxnet3_dev_stats_get() segfault w= hen called by pktgen >>>>=20 >>>>=20 >>>> I use the option EXTRA_CFLAGS on the command line to improve debug. >>>>=20 >>>> make install T=3Dx86_64-native-linuxapp-gcc EXTRA_CFLAGS=3D=E2=80=9C-g= -O0=E2=80=9D >>>>=20 >>>> Using -O0 for gdb is good for non-performance testing. >>>>=20 >>>>=20 >>>> Regards, >>>> Keith >>>=20 >>> Hi, >>>=20 >>> Thanks Keith and Stephen for the pointer. I have enabled the debug flag= s and acquired some more infos. >>>=20 >>> (gdb) r >>> Thread 1 "pktgen" received signal SIGSEGV, Segmentation fault. >>> 0x000000000075aa6c in vmxnet3_hw_tx_stats_get (hw=3D0x7fffb55b61c0, q= =3D0, res=3D0x7fffffffdf10) at /home/tester/dpdk/drivers/net/vmxnet3/vmxnet= 3_ethdev.c:905 >>> 905=09=09VMXNET3_UPDATE_TX_STAT(hw, q, ucastPktsTxOK, res); >>> (gdb) backtrace >>> #0 0x000000000075aa6c in vmxnet3_hw_tx_stats_get (hw=3D0x7fffb55b61c0,= q=3D0, res=3D0x7fffffffdf10) at /home/tester/dpdk/drivers/net/vmxnet3/vmxn= et3_ethdev.c:905 >>> #1 0x000000000075b3c8 in vmxnet3_dev_stats_get (dev=3D0xba34c0 , stats=3D0x7fffffffe000) at /home/tester/dpdk/drivers/net/vmxnet= 3/vmxnet3_ethdev.c:1049 >>> #2 0x00000000004a2f76 in rte_eth_stats_get (port_id=3D0 '\000', stats= =3D0x7fffffffe000) at /home/tester/dpdk/lib/librte_ether/rte_ethdev.c:1340 >>> #3 0x0000000000467059 in pktgen_process_stats (tim=3D, = arg=3D) at /home/tester/pktgen-dpdk/app/pktgen-stats.c:468 >>> #4 0x000000000048568a in rte_timer_manage () at /home/tester/dpdk/lib/= librte_timer/rte_timer.c:593 >>> #5 0x00000000007b47dd in cli_start () >>> #6 0x000000000044891b in pktgen_cli_start () at /home/tester/pktgen-dp= dk/app/cli-functions.c:1434 >>> #7 0x00000000004423f6 in main (argc=3D, argv=3D) at /home/tester/pktgen-dpdk/app/pktgen-main.c:470 >>> (gdb) l >>> 900=09{ >>> 901=09#define VMXNET3_UPDATE_TX_STAT(h, i, f, r)=09=09\ >>> 902=09=09=09((r)->f =3D (h)->tqd_start[(i)].stats.f +=09\ >>> 903=09=09=09=09(h)->saved_tx_stats[(i)].f) >>> 904=09 >>> 905=09=09VMXNET3_UPDATE_TX_STAT(hw, q, ucastPktsTxOK, res); >>> 906=09=09VMXNET3_UPDATE_TX_STAT(hw, q, mcastPktsTxOK, res); >>> 907=09=09VMXNET3_UPDATE_TX_STAT(hw, q, bcastPktsTxOK, res); >>> 908=09=09VMXNET3_UPDATE_TX_STAT(hw, q, ucastBytesTxOK, res); >>> 909=09=09VMXNET3_UPDATE_TX_STAT(hw, q, mcastBytesTxOK, res); >>> (gdb) p hw->tqd_start >>> $1 =3D (Vmxnet3_TxQueueDesc *) 0x0 >>>=20 >>> The problem was on line 902. >>> This tqd_start is the "start address of all tx queue desc" according to= comments in source code. Looks like it was not initialized properly. >>> I don't have enough knowledge to fix this. Any advice? >>=20 >> I am not able to fix this problem, but it seems like it could be pktgen = command line configuration issue. Normally these types of problems are beca= use the application tried to access a port/queue that is not >setup. The tq= d_start[] is an internal array to the VMXNET3 driver, so I do not know how = it relates to the DPDK API requests for port/queue.> >>=20 >> Sorry, if you provided this before, but what is the command line you are= using for Pktgen? >>=20 >>>=20 >>> Regards, >>> BL. >>=20 >> Regards, >> Keith >=20 Regards, Keith