From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 24A87A0524; Tue, 1 Jun 2021 15:01:04 +0200 (CEST) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 9653440689; Tue, 1 Jun 2021 15:01:03 +0200 (CEST) Received: from mx0b-0016f401.pphosted.com (mx0b-0016f401.pphosted.com [67.231.156.173]) by mails.dpdk.org (Postfix) with ESMTP id B9FEE40041; Tue, 1 Jun 2021 15:01:01 +0200 (CEST) Received: from pps.filterd (m0045851.ppops.net [127.0.0.1]) by mx0b-0016f401.pphosted.com (8.16.0.43/8.16.0.43) with SMTP id 151D0uec006766; Tue, 1 Jun 2021 06:01:00 -0700 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=marvell.com; h=message-id : date : mime-version : subject : to : cc : references : from : in-reply-to : content-type : content-transfer-encoding; s=pfpt0220; bh=xxz2vwfqRQcY5hL3frivPh2S+IYsifaqe6L5KldOO14=; b=aY8r1M/5v6iiy4YQqd2B1BIZKbM9zC/O6aD3dUp+sh17fToZRwCEvAFelgphpNjWhYt5 9mSfmRlH7InmfXjoiqHDlGVcvbSMK9u5nY7NRmDDcxM51MiJ2+2XoDDmxuPZHkZKlICq 9fdx6Je6zL0Zl1pGYB7xDh+xptLIG7kdLQ7rQtbOUgiaB9scETp2zhrGJnWKgW+a6Tje /wJJtj+I+tYMl2pT7WdpKJlKGUQnwIn2TX6QG2+dK2gAG8w/Z5Q/myLUhM3ihw4MKCZx r52wl70th90sF6U8tATna91mZMhD1deN4xjt6w70dS7T/IbhRlYx0dyrmS+bj4/fcPf0 YQ== Received: from dc5-exch02.marvell.com ([199.233.59.182]) by mx0b-0016f401.pphosted.com with ESMTP id 38vtnjc5ah-6 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-SHA384 bits=256 verify=NOT); Tue, 01 Jun 2021 06:00:59 -0700 Received: from DC5-EXCH02.marvell.com (10.69.176.39) by DC5-EXCH02.marvell.com (10.69.176.39) with Microsoft SMTP Server (TLS) id 15.0.1497.2; Tue, 1 Jun 2021 06:00:54 -0700 Received: from maili.marvell.com (10.69.176.80) by DC5-EXCH02.marvell.com (10.69.176.39) with Microsoft SMTP Server id 15.0.1497.2 via Frontend Transport; Tue, 1 Jun 2021 06:00:54 -0700 Received: from [10.193.32.53] (unknown [10.193.32.53]) by maili.marvell.com (Postfix) with ESMTP id 024873F7040; Tue, 1 Jun 2021 06:00:51 -0700 (PDT) Message-ID: <4233f366-b4ea-fbbb-0fde-d478be4292ac@marvell.com> Date: Tue, 1 Jun 2021 15:00:50 +0200 MIME-Version: 1.0 User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:89.0) Gecko/20100101 Thunderbird/89.0 Content-Language: en-US To: "Varghese, Vipin" , Truring Team , Nishant Verma CC: , USERS , Devendra Singh Rawat , Rasesh Mody References: From: Igor Russkikh In-Reply-To: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: 8bit X-Proofpoint-GUID: kZ3S0ljetIC-P5ycPcHJSzPj9g1xVPE3 X-Proofpoint-ORIG-GUID: kZ3S0ljetIC-P5ycPcHJSzPj9g1xVPE3 X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10434:6.0.391, 18.0.761 definitions=2021-06-01_06:2021-05-31, 2021-06-01 signatures=0 Subject: Re: [dpdk-dev] [EXT] Re: [dpdk-users] DPDK issue with Marvell NIC QLogic Corp. FastLinQ QL41000 X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" Adding my team members for that. Strangely, even if by any reason we would get OOM condition - driver should reject load gracefully. Is there any possibility FW image could be corrupted/of bad size? Regards, Igor On 5/26/2021 11:57 AM, Varghese, Vipin wrote: > External Email > > ---------------------------------------------------------------------- > Based on the logs shared from `Turing Team`, the observation is as follows > > 1. 1GB huge pages are used in the application > > ```logs > 12:46:11.792 §EAL: No free hugepages reported in hugepages-2048kB > 12:46:11.792 §EAL: No available hugepages reported in hugepages-2048kB > ``` > > 2. Issue comes right at `device porbe to initialize` > > ```logs > 12:46:46.382 §EAL: Probe PCI driver: net_qede (1077:8070) device: > 0000:65:00.0 (socket 0) > 12:46:46.403 §[QEDE PMD: ()]ecore_load_mcp_offsets:The address of the MCP > scratch-pad is not configured > 12:46:46.403 §[QEDE PMD: ()]ecore_mcp_cmd_init:MCP is not initialized > 12:46:46.404 §[QEDE PMD: ()]ecore_mcp_cmd_and_union:MFW is not initialized! > 12:46:46.404 §[QEDE PMD: ()]ecore_hw_get_nvm_info:Shared memory not > initialized > 12:46:46.404 §[QEDE PMD: ()]ecore_hw_prepare_single:Failed to get HW > information > 12:46:46.404 §[qed_probe:74(65:00.0:dpdk-port-0)]hw prepare failed > 12:46:46.404 §[qede_common_dev_init:2566(65:00.0:dpdk-port-> 0)]qede probe > failed rc -3 > ``` > > Hence in my opinion this is related to NIC in use and how it is getting > initialized. So please try to debug driver > >> -----Original Message----- >> From: dev On Behalf Of Truring Team >> Sent: Wednesday, May 26, 2021 10:12 AM >> To: Nishant Verma >> Cc: dev@dpdk.org; USERS >> Subject: Re: [dpdk-dev] [dpdk-users] DPDK issue with Marvell NIC QLogic >> Corp. FastLinQ QL41000 >> >> Yes , I have configured it via grub. >> >> Grub: >> "default_hugepagesz=1G hugepagesz=1G hugepages=2" >> grub2-mkconfig -o /boot/grub2/grub.cfg >> >> Manual: >> mkdir /dev/hugepages1G >> mount -t hugetlbfs -o pagesize=1G none /dev/hugepages1G echo 2 > >> /sys/devices/system/node/node1/hugepages/hugepages- >> 1048576kB/nr_hugepages >> >> Regards >> Puneet >> >> On Wed, 26 May 2021 at 10:05, Nishant Verma wrote: >> >>> meminfo states: Hugepagesize: 1048576 kB >>> But in your initial mail you state Hugepage size is 1G. >>> >>> have you configured HugePage via grub.... ? >>> >>> >>> Thanks. >>> >>> Regards, >>> NV >>> >>> On Wed, May 26, 2021 at 12:21 AM Truring Team >> wrote: >>> >>>> Hi Nishant, >>>> >>>> cat /proc/meminfo >>>> 11:59:29.200 §MemTotal: 23101672 kB >>>> 11:59:29.200 §MemFree: 15254764 kB >>>> 11:59:29.200 §MemAvailable: 19243184 kB >>>> 11:59:29.200 §Buffers: 6792 kB >>>> 11:59:29.201 §Cached: 4084844 kB >>>> 11:59:29.201 §SwapCached: 0 kB >>>> 11:59:29.201 §Active: 3609016 kB >>>> 11:59:29.201 §Inactive: 1175160 kB >>>> 11:59:29.201 §Active(anon): 695644 kB >>>> 11:59:29.202 §Inactive(anon): 19716 kB >>>> 11:59:29.202 §Active(file): 2913372 kB >>>> 11:59:29.202 §Inactive(file): 1155444 kB >>>> 11:59:29.202 §Unevictable: 0 kB >>>> 11:59:29.202 §Mlocked: 0 kB >>>> 11:59:29.203 §SwapTotal: 11665404 kB >>>> 11:59:29.204 §SwapFree: 11665404 kB >>>> 11:59:29.204 §Dirty: 32 kB >>>> 11:59:29.204 §Writeback: 0 kB >>>> 11:59:29.204 §AnonPages: 694764 kB >>>> 11:59:29.205 §Mapped: 140220 kB >>>> 11:59:29.205 §Shmem: 22800 kB >>>> 11:59:29.205 §Slab: 445016 kB >>>> 11:59:29.205 §SReclaimable: 289968 kB >>>> 11:59:29.205 §SUnreclaim: 155048 kB >>>> 11:59:29.206 §KernelStack: 16624 kB >>>> 11:59:29.206 §PageTables: 35892 kB >>>> 11:59:29.206 §NFS_Unstable: 0 kB >>>> 11:59:29.206 §Bounce: 0 kB >>>> 11:59:29.206 §WritebackTmp: 0 kB >>>> 11:59:29.207 §CommitLimit: 22167664 kB >>>> 11:59:29.207 §Committed_AS: 4342668 kB >>>> 11:59:29.207 §VmallocTotal: 34359738367 kB >>>> 11:59:29.207 §VmallocUsed: 463188 kB >>>> 11:59:29.207 §VmallocChunk: 34358835196 kB >>>> 11:59:29.208 §HardwareCorrupted: 0 kB >>>> 11:59:29.208 §AnonHugePages: 329728 kB >>>> 11:59:29.208 §CmaTotal: 0 kB >>>> 11:59:29.209 §CmaFree: 0 kB >>>> 11:59:29.209 §HugePages_Total: 1024 >>>> 11:59:29.209 §HugePages_Free: 1019 >>>> 11:59:29.209 §HugePages_Rsvd: 0 >>>> 11:59:29.209 §HugePages_Surp: 0 >>>> 11:59:29.209 §Hugepagesize: 2048 kB >>>> 11:59:29.209 §DirectMap4k: 381760 kB >>>> 11:59:29.209 §DirectMap2M: 13973504 kB >>>> 11:59:29.215 §DirectMap1G: 11534336 kB >>>> >>>> Regards >>>> >>>> >>>> On Wed, 26 May 2021 at 09:43, Nishant Verma >> wrote: >>>> >>>>> Can you paste output of /proc/meminfo ? >>>>> >>>>> >>>>> >>>>> Regards, >>>>> NV >>>>> >>>>> On Tue, May 25, 2021 at 9:52 PM Truring Team >>>>> wrote: >>>>> >>>>>> Hi Team, >>>>>> >>>>>> I am trying to run the dpdk test-pmd application (DPDK 20.11 and >>>>>> DPDK >>>>>> 21.05) with NIC QLogic Corp. FastLinQ QL41000 Series 10/25/40/50GbE >>>>>> Controller but facing following issues: >>>>>> >>>>>> If someone can look into the following logs and help me figure out >>>>>> the issue? >>>>>> >>>>>> >>>>>> >>>>>> 1. DPDK 20.11 ( Using 1 GB hugepages) >>>>>> >>>>>> gdb ./dpdk-testpmd >>>>>> >>>>>> 12:45:59.347 §GNU gdb (GDB) Red Hat Enterprise Linux 7.6.1-114.el7 >>>>>> >>>>>> 12:45:59.347 §Copyright (C) 2013 Free Software Foundation, Inc. >>>>>> >>>>>> 12:45:59.348 §License GPLv3+: GNU GPL version 3 or later >>>>>> https://urldefense.proofpoint.com/v2/url?u=http-3A__gnu.org_licenses_gpl.html&d=DwIGaQ&c=nKjWec2b6R0mOyPaz7xtfQ&r=3kUjVPjrPMvlbd3rzgP63W0eewvCq4D-kzQRqaXHOqU&m=XPT-odmQXtSC6Yu2ao1h1I9z-Lzp3iTY8Er77XRkORc&s=_ge-P-H6ALF17WiCqG3NOsjp15Nbk8ptb-RpqVIpLrE&e= >>>>>> >>>>>> 12:45:59.349 §This is free software: you are free to change and >>>>>> redistribute it. >>>>>> >>>>>> 12:45:59.349 §There is NO WARRANTY, to the extent permitted by law. >>>>>> Type >>>>>> "show copying" >>>>>> >>>>>> 12:45:59.349 §and "show warranty" for details. >>>>>> >>>>>> 12:45:59.350 §This GDB was configured as "x86_64-redhat-linux-gnu". >>>>>> >>>>>> 12:45:59.351 §For bug reporting instructions, please see: >>>>>> >>>>>> 12:45:59.351 >>>>>> §https://urldefense.proofpoint.com/v2/url?u=http-3A__www.gnu.org_software_gdb_bugs_&d=DwIGaQ&c=nKjWec2b6R0mOyPaz7xtfQ&r=3kUjVPjrPMvlbd3rzgP63W0eewvCq4D-kzQRqaXHOqU&m=XPT-odmQXtSC6Yu2ao1h1I9z-Lzp3iTY8Er77XRkORc&s=eGKKc0q8qdyW4CSbWUdq2X6ShOHrSYguXKRC6qpG-RQ&e= >>>>>> ... >>>>>> >>>>>> 12:45:59.415 §Reading symbols from >>>>>> /home/itwsadmin/dpdk-20.11/build/app/dpdk-testpmd...(no >> debugging >>>>>> symbols found)...done. >>>>>> >>>>>> 12:46:11.492 §(gdb) r -l 0-5 -n4 -- -i >>>>>> >>>>>> 12:46:11.493 §Starting program: >>>>>> /home/itwsadmin/dpdk-20.11/build/app/./dpdk-testpmd -l 0-5 -n4 -- >>>>>> -i >>>>>> >>>>>> 12:46:11.717 §[Thread debugging using libthread_db enabled] >>>>>> >>>>>> 12:46:11.719 §Using host libthread_db library >>>>>> "/lib64/libthread_db.so.1". >>>>>> >>>>>> 12:46:11.768 §EAL: Detected 6 lcore(s) >>>>>> >>>>>> 12:46:11.769 §EAL: Detected 1 NUMA nodes >>>>>> >>>>>> 12:46:11.769 §[New Thread 0x7ffff6a71700 (LWP 23184)] >>>>>> >>>>>> 12:46:11.769 §EAL: Multi-process socket /var/run/dpdk/rte/mp_socket >>>>>> >>>>>> 12:46:11.770 §[New Thread 0x7ffff6270700 (LWP 23185)] >>>>>> >>>>>> 12:46:11.792 §EAL: Selected IOVA mode 'PA' >>>>>> >>>>>> 12:46:11.792 §EAL: No free hugepages reported in hugepages-2048kB >>>>>> >>>>>> 12:46:11.792 §EAL: No available hugepages reported in >>>>>> hugepages-2048kB >>>>>> >>>>>> 12:46:11.854 §EAL: Probing VFIO support... >>>>>> >>>>>> 12:46:11.855 §[New Thread 0x7ffff5a6f700 (LWP 23186)] >>>>>> >>>>>> 12:46:11.855 §[New Thread 0x7ffff526e700 (LWP 23187)] >>>>>> >>>>>> 12:46:11.855 §[New Thread 0x7ffff4a6d700 (LWP 23188)] >>>>>> >>>>>> 12:46:11.855 §[New Thread 0x7ffff426c700 (LWP 23189)] >>>>>> >>>>>> 12:46:11.856 §[New Thread 0x7ffff3a6b700 (LWP 23190)] >>>>>> >>>>>> 12:46:11.896 §EAL: Probe PCI driver: net_qede (1077:8070) device: >>>>>> 0000:65:00.0 (socket 0) >>>>>> >>>>>> 12:46:11.915 § >>>>>> >>>>>> 12:46:11.915 §Program received signal SIGSEGV, Segmentation fault. >>>>>> >>>>>> 12:46:11.916 §0x0000000000ec5b20 in ecore_fw_overlay_mem_alloc () >>>>>> >>>>>> 12:46:11.916 §Missing separate debuginfos, use: debuginfo-install >>>>>> glibc-2.17-324.el7_9.x86_64 libgcc-4.8.5-44.el7.x86_64 >>>>>> libpcap-1.5.3-12.el7.x86_64 numactl-libs-2.0.12-5.el7.x86_64 >>>>>> zlib-1.2.7-19.el7_9.x86_64 >>>>>> >>>>>> 12:46:14.542 §(gdb) bt >>>>>> >>>>>> 12:46:14.548 §#0 0x0000000000ec5b20 in >> ecore_fw_overlay_mem_alloc >>>>>> () >>>>>> >>>>>> 12:46:14.548 §#1 0x0000000000eb811e in ecore_hw_init () >>>>>> >>>>>> 12:46:14.549 §#2 0x0000000000ef7397 in qed_slowpath_start () >>>>>> >>>>>> 12:46:14.549 §#3 0x0000000000ef0483 in qede_common_dev_init () >>>>>> >>>>>> 12:46:14.549 §#4 0x0000000000ef1113 in qede_eth_dev_pci_probe () >>>>>> >>>>>> 12:46:14.549 §#5 0x0000000000a4450b in pci_probe () >>>>>> >>>>>> 12:46:14.550 §#6 0x00000000009dbc6c in rte_bus_probe () >>>>>> >>>>>> 12:46:14.550 §#7 0x00000000009fa91c in rte_eal_init () >>>>>> >>>>>> 12:46:14.617 §#8 0x0000000000425db1 in main () >>>>>> >>>>>> 12:46:17.052 §(gdb) quit >>>>>> >>>>>> 12:46:17.056 §A debugging session is active. >>>>>> >>>>>> 12:46:17.056 § >>>>>> >>>>>> 12:46:17.057 § Inferior 1 [process 23180] will be killed. >>>>>> >>>>>> >>>>>> >>>>>> >>>>>> >>>>>> 2. *DPDK 21.05:* >>>>>> >>>>>> ./dpdk-testpmd -l 0-5 -n4 -- -i >>>>>> >>>>>> 12:46:46.257 §EAL: Detected 6 lcore(s) >>>>>> >>>>>> 12:46:46.257 §EAL: Detected 1 NUMA nodes >>>>>> >>>>>> 12:46:46.258 §EAL: Detected static linkage of DPDK >>>>>> >>>>>> 12:46:46.260 §EAL: Multi-process socket /var/run/dpdk/rte/mp_socket >>>>>> >>>>>> 12:46:46.285 §EAL: Selected IOVA mode 'PA' >>>>>> >>>>>> 12:46:46.285 §EAL: No free 2048 kB hugepages reported on node 0 >>>>>> >>>>>> 12:46:46.288 §EAL: No available 2048 kB hugepages reported >>>>>> >>>>>> 12:46:46.382 §EAL: Probe PCI driver: net_qede (1077:8070) device: >>>>>> 0000:65:00.0 (socket 0) >>>>>> >>>>>> 12:46:46.403 §[QEDE PMD: ()]ecore_load_mcp_offsets:The address of >>>>>> the MCP scratch-pad is not configured >>>>>> >>>>>> 12:46:46.403 §[QEDE PMD: ()]ecore_mcp_cmd_init:MCP is not >>>>>> initialized >>>>>> >>>>>> 12:46:46.404 §[QEDE PMD: ()]ecore_mcp_cmd_and_union:MFW is not >>>>>> initialized! >>>>>> >>>>>> 12:46:46.404 §[QEDE PMD: ()]ecore_hw_get_nvm_info:Shared >> memory not >>>>>> initialized >>>>>> >>>>>> 12:46:46.404 §[QEDE PMD: ()]ecore_hw_prepare_single:Failed to get >>>>>> HW information >>>>>> >>>>>> 12:46:46.404 §[qed_probe:74(65:00.0:dpdk-port-0)]hw prepare failed >>>>>> >>>>>> 12:46:46.404 §[qede_common_dev_init:2566(65:00.0:dpdk-port- >> 0)]qede >>>>>> probe failed rc -3 >>>>>> >>>>>> 12:46:46.406 §EAL: Requested device 0000:65:00.0 cannot be used >>>>>> >>>>>> 12:46:46.408 §TELEMETRY: No legacy callbacks, legacy socket not >>>>>> created >>>>>> >>>>>> 12:46:46.423 §testpmd: No probed ethernet devices >>>>>> >>>>>> 12:46:46.461 §Interactive-mode selected >>>>>> >>>>>> 12:46:46.519 §testpmd: create a new mbuf pool : >>>>>> n=187456, size=2176, socket=0 >>>>>> >>>>>> 12:46:46.522 §testpmd: preferred mempool ops selected: ring_mp_mc >>>>>> >>>>>> 12:46:46.540 §Done >>>>>> >>>>>> >>>>>> >>>>>> >>>>>> >>>>>> System Information: >>>>>> >>>>>> *ethtool -i p2p1* >>>>>> driver: qede >>>>>> version: 8.33.0.20 >>>>>> *firmware-version: mfw 8.24.44.0 storm 8.33.11.0* >>>>>> expansion-rom-version: >>>>>> bus-info: 0000:65:00.0 >>>>>> supports-statistics: yes >>>>>> supports-test: yes >>>>>> supports-eeprom-access: no >>>>>> supports-register-dump: yes >>>>>> supports-priv-flags: yes >>>>>> >>>>>> >>>>>> >>>>>> *ethtool -i p2p2 *driver: qede >>>>>> version: 8.33.0.20 >>>>>> firmware-version: mfw 8.24.44.0 storm 8.33.11.0 >>>>>> expansion-rom-version: >>>>>> bus-info: 0000:65:00.1 >>>>>> supports-statistics: yes >>>>>> supports-test: yes >>>>>> supports-eeprom-access: no >>>>>> supports-register-dump: yes >>>>>> supports-priv-flags: yes >>>>>> >>>>>> >>>>>> >>>>>> >>>>>> >>>>>> *lspci -nn | grep Ether* >>>>>> >>>>>> 12:42:50.579 §04:00.0 Ethernet controller [0200]: Broadcom Inc. and >>>>>> subsidiaries NetXtreme BCM5720 Gigabit Ethernet PCIe [14e4:165f] >>>>>> >>>>>> 12:42:50.581 §04:00.1 Ethernet controller [0200]: Broadcom Inc. and >>>>>> subsidiaries NetXtreme BCM5720 Gigabit Ethernet PCIe [14e4:165f] >>>>>> >>>>>> 12:42:50.582 §65:00.0 Ethernet controller [0200]: QLogic Corp. >>>>>> FastLinQ >>>>>> QL41000 Series 10/25/40/50GbE Controller [1077:8070] (rev 02) >>>>>> >>>>>> 12:42:50.583 §65:00.1 Ethernet controller [0200]: QLogic Corp. >>>>>> FastLinQ >>>>>> QL41000 Series 10/25/40/50GbE Controller [1077:8070] (rev 02) >>>>>> >>>>>> >>>>>> >>>>>> cd /lib/firmware/qed/ >>>>>> >>>>>> ls >>>>>> >>>>>> 12:48:27.269 §qed_init_values-8.10.9.0.bin >>>>>> qed_init_values-8.30.12.0.bin qed_init_values_zipped-8.10.5.0.bin >>>>>> qed_init_values_zipped-8.33.11.0.bin >>>>>> qed_init_values_zipped-8.7.3.0.bin >>>>>> >>>>>> 12:48:27.272 §qed_init_values-8.14.6.0.bin >>>>>> qed_init_values-8.33.12.0.bin qed_init_values_zipped-8.15.3.0.bin >>>>>> qed_init_values_zipped-8.37.2.0.bin >>>>>> >>>>>> 12:48:27.273 §qed_init_values-8.18.9.0.bin >>>>>> qed_init_values-8.40.33.0.bin qed_init_values_zipped-8.20.0.0.bin >>>>>> qed_init_values_zipped-8.37.7.0.bin >>>>>> >>>>>> 12:48:27.275 §qed_init_values-8.20.0.0.bin >>>>>> qed_init_values_zipped-8.10.10.0.bin >>>>>> qed_init_values_zipped-8.33.1.0.bin >>>>>> qed_init_values_zipped-8.4.2.0.bin >>>>>> >>>>>> >>>>>> Best Regards >>>>>> >>>>>> Puneet >>>>>> >>>>>