From: Igor Russkikh <irusskikh@marvell.com>
To: "Varghese, Vipin" <vipin.varghese@intel.com>,
Truring Team <truring12@gmail.com>,
Nishant Verma <vnish11@gmail.com>
Cc: <dev@dpdk.org>, USERS <users@dpdk.org>,
Devendra Singh Rawat <dsinghrawat@marvell.com>,
Rasesh Mody <rmody@marvell.com>
Subject: Re: [dpdk-users] [EXT] Re: [dpdk-dev] DPDK issue with Marvell NIC QLogic Corp. FastLinQ QL41000
Date: Tue, 1 Jun 2021 15:00:50 +0200 [thread overview]
Message-ID: <4233f366-b4ea-fbbb-0fde-d478be4292ac@marvell.com> (raw)
In-Reply-To: <PH0PR11MB4872EDC74E2A286A430D4BEC90249@PH0PR11MB4872.namprd11.prod.outlook.com>
Adding my team members for that.
Strangely, even if by any reason we would get OOM condition - driver should reject load gracefully.
Is there any possibility FW image could be corrupted/of bad size?
Regards,
Igor
On 5/26/2021 11:57 AM, Varghese, Vipin wrote:
> External Email
>
> ----------------------------------------------------------------------
> Based on the logs shared from `Turing Team`, the observation is as follows
>
> 1. 1GB huge pages are used in the application
>
> ```logs
> 12:46:11.792 §EAL: No free hugepages reported in hugepages-2048kB
> 12:46:11.792 §EAL: No available hugepages reported in hugepages-2048kB
> ```
>
> 2. Issue comes right at `device porbe to initialize`
>
> ```logs
> 12:46:46.382 §EAL: Probe PCI driver: net_qede (1077:8070) device:
> 0000:65:00.0 (socket 0)
> 12:46:46.403 §[QEDE PMD: ()]ecore_load_mcp_offsets:The address of the MCP
> scratch-pad is not configured
> 12:46:46.403 §[QEDE PMD: ()]ecore_mcp_cmd_init:MCP is not initialized
> 12:46:46.404 §[QEDE PMD: ()]ecore_mcp_cmd_and_union:MFW is not initialized!
> 12:46:46.404 §[QEDE PMD: ()]ecore_hw_get_nvm_info:Shared memory not
> initialized
> 12:46:46.404 §[QEDE PMD: ()]ecore_hw_prepare_single:Failed to get HW
> information
> 12:46:46.404 §[qed_probe:74(65:00.0:dpdk-port-0)]hw prepare failed
> 12:46:46.404 §[qede_common_dev_init:2566(65:00.0:dpdk-port-> 0)]qede probe
> failed rc -3
> ```
>
> Hence in my opinion this is related to NIC in use and how it is getting
> initialized. So please try to debug driver
>
>> -----Original Message-----
>> From: dev <dev-bounces@dpdk.org> On Behalf Of Truring Team
>> Sent: Wednesday, May 26, 2021 10:12 AM
>> To: Nishant Verma <vnish11@gmail.com>
>> Cc: dev@dpdk.org; USERS <users@dpdk.org>
>> Subject: Re: [dpdk-dev] [dpdk-users] DPDK issue with Marvell NIC QLogic
>> Corp. FastLinQ QL41000
>>
>> Yes , I have configured it via grub.
>>
>> Grub:
>> "default_hugepagesz=1G hugepagesz=1G hugepages=2"
>> grub2-mkconfig -o /boot/grub2/grub.cfg
>>
>> Manual:
>> mkdir /dev/hugepages1G
>> mount -t hugetlbfs -o pagesize=1G none /dev/hugepages1G echo 2 >
>> /sys/devices/system/node/node1/hugepages/hugepages-
>> 1048576kB/nr_hugepages
>>
>> Regards
>> Puneet
>>
>> On Wed, 26 May 2021 at 10:05, Nishant Verma <vnish11@gmail.com> wrote:
>>
>>> meminfo states: Hugepagesize: 1048576 kB
>>> But in your initial mail you state Hugepage size is 1G.
>>>
>>> have you configured HugePage via grub.... ?
>>>
>>>
>>> Thanks.
>>>
>>> Regards,
>>> NV
>>>
>>> On Wed, May 26, 2021 at 12:21 AM Truring Team <truring12@gmail.com>
>> wrote:
>>>
>>>> Hi Nishant,
>>>>
>>>> cat /proc/meminfo
>>>> 11:59:29.200 §MemTotal: 23101672 kB
>>>> 11:59:29.200 §MemFree: 15254764 kB
>>>> 11:59:29.200 §MemAvailable: 19243184 kB
>>>> 11:59:29.200 §Buffers: 6792 kB
>>>> 11:59:29.201 §Cached: 4084844 kB
>>>> 11:59:29.201 §SwapCached: 0 kB
>>>> 11:59:29.201 §Active: 3609016 kB
>>>> 11:59:29.201 §Inactive: 1175160 kB
>>>> 11:59:29.201 §Active(anon): 695644 kB
>>>> 11:59:29.202 §Inactive(anon): 19716 kB
>>>> 11:59:29.202 §Active(file): 2913372 kB
>>>> 11:59:29.202 §Inactive(file): 1155444 kB
>>>> 11:59:29.202 §Unevictable: 0 kB
>>>> 11:59:29.202 §Mlocked: 0 kB
>>>> 11:59:29.203 §SwapTotal: 11665404 kB
>>>> 11:59:29.204 §SwapFree: 11665404 kB
>>>> 11:59:29.204 §Dirty: 32 kB
>>>> 11:59:29.204 §Writeback: 0 kB
>>>> 11:59:29.204 §AnonPages: 694764 kB
>>>> 11:59:29.205 §Mapped: 140220 kB
>>>> 11:59:29.205 §Shmem: 22800 kB
>>>> 11:59:29.205 §Slab: 445016 kB
>>>> 11:59:29.205 §SReclaimable: 289968 kB
>>>> 11:59:29.205 §SUnreclaim: 155048 kB
>>>> 11:59:29.206 §KernelStack: 16624 kB
>>>> 11:59:29.206 §PageTables: 35892 kB
>>>> 11:59:29.206 §NFS_Unstable: 0 kB
>>>> 11:59:29.206 §Bounce: 0 kB
>>>> 11:59:29.206 §WritebackTmp: 0 kB
>>>> 11:59:29.207 §CommitLimit: 22167664 kB
>>>> 11:59:29.207 §Committed_AS: 4342668 kB
>>>> 11:59:29.207 §VmallocTotal: 34359738367 kB
>>>> 11:59:29.207 §VmallocUsed: 463188 kB
>>>> 11:59:29.207 §VmallocChunk: 34358835196 kB
>>>> 11:59:29.208 §HardwareCorrupted: 0 kB
>>>> 11:59:29.208 §AnonHugePages: 329728 kB
>>>> 11:59:29.208 §CmaTotal: 0 kB
>>>> 11:59:29.209 §CmaFree: 0 kB
>>>> 11:59:29.209 §HugePages_Total: 1024
>>>> 11:59:29.209 §HugePages_Free: 1019
>>>> 11:59:29.209 §HugePages_Rsvd: 0
>>>> 11:59:29.209 §HugePages_Surp: 0
>>>> 11:59:29.209 §Hugepagesize: 2048 kB
>>>> 11:59:29.209 §DirectMap4k: 381760 kB
>>>> 11:59:29.209 §DirectMap2M: 13973504 kB
>>>> 11:59:29.215 §DirectMap1G: 11534336 kB
>>>>
>>>> Regards
>>>>
>>>>
>>>> On Wed, 26 May 2021 at 09:43, Nishant Verma <vnish11@gmail.com>
>> wrote:
>>>>
>>>>> Can you paste output of /proc/meminfo ?
>>>>>
>>>>>
>>>>>
>>>>> Regards,
>>>>> NV
>>>>>
>>>>> On Tue, May 25, 2021 at 9:52 PM Truring Team <truring12@gmail.com>
>>>>> wrote:
>>>>>
>>>>>> Hi Team,
>>>>>>
>>>>>> I am trying to run the dpdk test-pmd application (DPDK 20.11 and
>>>>>> DPDK
>>>>>> 21.05) with NIC QLogic Corp. FastLinQ QL41000 Series 10/25/40/50GbE
>>>>>> Controller but facing following issues:
>>>>>>
>>>>>> If someone can look into the following logs and help me figure out
>>>>>> the issue?
>>>>>>
>>>>>>
>>>>>>
>>>>>> 1. DPDK 20.11 ( Using 1 GB hugepages)
>>>>>>
>>>>>> gdb ./dpdk-testpmd
>>>>>>
>>>>>> 12:45:59.347 §GNU gdb (GDB) Red Hat Enterprise Linux 7.6.1-114.el7
>>>>>>
>>>>>> 12:45:59.347 §Copyright (C) 2013 Free Software Foundation, Inc.
>>>>>>
>>>>>> 12:45:59.348 §License GPLv3+: GNU GPL version 3 or later
>>>>>> https://urldefense.proofpoint.com/v2/url?u=http-3A__gnu.org_licenses_gpl.html&d=DwIGaQ&c=nKjWec2b6R0mOyPaz7xtfQ&r=3kUjVPjrPMvlbd3rzgP63W0eewvCq4D-kzQRqaXHOqU&m=XPT-odmQXtSC6Yu2ao1h1I9z-Lzp3iTY8Er77XRkORc&s=_ge-P-H6ALF17WiCqG3NOsjp15Nbk8ptb-RpqVIpLrE&e=
>>>>>>
>>>>>> 12:45:59.349 §This is free software: you are free to change and
>>>>>> redistribute it.
>>>>>>
>>>>>> 12:45:59.349 §There is NO WARRANTY, to the extent permitted by law.
>>>>>> Type
>>>>>> "show copying"
>>>>>>
>>>>>> 12:45:59.349 §and "show warranty" for details.
>>>>>>
>>>>>> 12:45:59.350 §This GDB was configured as "x86_64-redhat-linux-gnu".
>>>>>>
>>>>>> 12:45:59.351 §For bug reporting instructions, please see:
>>>>>>
>>>>>> 12:45:59.351
>>>>>> §https://urldefense.proofpoint.com/v2/url?u=http-3A__www.gnu.org_software_gdb_bugs_&d=DwIGaQ&c=nKjWec2b6R0mOyPaz7xtfQ&r=3kUjVPjrPMvlbd3rzgP63W0eewvCq4D-kzQRqaXHOqU&m=XPT-odmQXtSC6Yu2ao1h1I9z-Lzp3iTY8Er77XRkORc&s=eGKKc0q8qdyW4CSbWUdq2X6ShOHrSYguXKRC6qpG-RQ&e=
>>>>>> ...
>>>>>>
>>>>>> 12:45:59.415 §Reading symbols from
>>>>>> /home/itwsadmin/dpdk-20.11/build/app/dpdk-testpmd...(no
>> debugging
>>>>>> symbols found)...done.
>>>>>>
>>>>>> 12:46:11.492 §(gdb) r -l 0-5 -n4 -- -i
>>>>>>
>>>>>> 12:46:11.493 §Starting program:
>>>>>> /home/itwsadmin/dpdk-20.11/build/app/./dpdk-testpmd -l 0-5 -n4 --
>>>>>> -i
>>>>>>
>>>>>> 12:46:11.717 §[Thread debugging using libthread_db enabled]
>>>>>>
>>>>>> 12:46:11.719 §Using host libthread_db library
>>>>>> "/lib64/libthread_db.so.1".
>>>>>>
>>>>>> 12:46:11.768 §EAL: Detected 6 lcore(s)
>>>>>>
>>>>>> 12:46:11.769 §EAL: Detected 1 NUMA nodes
>>>>>>
>>>>>> 12:46:11.769 §[New Thread 0x7ffff6a71700 (LWP 23184)]
>>>>>>
>>>>>> 12:46:11.769 §EAL: Multi-process socket /var/run/dpdk/rte/mp_socket
>>>>>>
>>>>>> 12:46:11.770 §[New Thread 0x7ffff6270700 (LWP 23185)]
>>>>>>
>>>>>> 12:46:11.792 §EAL: Selected IOVA mode 'PA'
>>>>>>
>>>>>> 12:46:11.792 §EAL: No free hugepages reported in hugepages-2048kB
>>>>>>
>>>>>> 12:46:11.792 §EAL: No available hugepages reported in
>>>>>> hugepages-2048kB
>>>>>>
>>>>>> 12:46:11.854 §EAL: Probing VFIO support...
>>>>>>
>>>>>> 12:46:11.855 §[New Thread 0x7ffff5a6f700 (LWP 23186)]
>>>>>>
>>>>>> 12:46:11.855 §[New Thread 0x7ffff526e700 (LWP 23187)]
>>>>>>
>>>>>> 12:46:11.855 §[New Thread 0x7ffff4a6d700 (LWP 23188)]
>>>>>>
>>>>>> 12:46:11.855 §[New Thread 0x7ffff426c700 (LWP 23189)]
>>>>>>
>>>>>> 12:46:11.856 §[New Thread 0x7ffff3a6b700 (LWP 23190)]
>>>>>>
>>>>>> 12:46:11.896 §EAL: Probe PCI driver: net_qede (1077:8070) device:
>>>>>> 0000:65:00.0 (socket 0)
>>>>>>
>>>>>> 12:46:11.915 §
>>>>>>
>>>>>> 12:46:11.915 §Program received signal SIGSEGV, Segmentation fault.
>>>>>>
>>>>>> 12:46:11.916 §0x0000000000ec5b20 in ecore_fw_overlay_mem_alloc ()
>>>>>>
>>>>>> 12:46:11.916 §Missing separate debuginfos, use: debuginfo-install
>>>>>> glibc-2.17-324.el7_9.x86_64 libgcc-4.8.5-44.el7.x86_64
>>>>>> libpcap-1.5.3-12.el7.x86_64 numactl-libs-2.0.12-5.el7.x86_64
>>>>>> zlib-1.2.7-19.el7_9.x86_64
>>>>>>
>>>>>> 12:46:14.542 §(gdb) bt
>>>>>>
>>>>>> 12:46:14.548 §#0 0x0000000000ec5b20 in
>> ecore_fw_overlay_mem_alloc
>>>>>> ()
>>>>>>
>>>>>> 12:46:14.548 §#1 0x0000000000eb811e in ecore_hw_init ()
>>>>>>
>>>>>> 12:46:14.549 §#2 0x0000000000ef7397 in qed_slowpath_start ()
>>>>>>
>>>>>> 12:46:14.549 §#3 0x0000000000ef0483 in qede_common_dev_init ()
>>>>>>
>>>>>> 12:46:14.549 §#4 0x0000000000ef1113 in qede_eth_dev_pci_probe ()
>>>>>>
>>>>>> 12:46:14.549 §#5 0x0000000000a4450b in pci_probe ()
>>>>>>
>>>>>> 12:46:14.550 §#6 0x00000000009dbc6c in rte_bus_probe ()
>>>>>>
>>>>>> 12:46:14.550 §#7 0x00000000009fa91c in rte_eal_init ()
>>>>>>
>>>>>> 12:46:14.617 §#8 0x0000000000425db1 in main ()
>>>>>>
>>>>>> 12:46:17.052 §(gdb) quit
>>>>>>
>>>>>> 12:46:17.056 §A debugging session is active.
>>>>>>
>>>>>> 12:46:17.056 §
>>>>>>
>>>>>> 12:46:17.057 § Inferior 1 [process 23180] will be killed.
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>> 2. *DPDK 21.05:*
>>>>>>
>>>>>> ./dpdk-testpmd -l 0-5 -n4 -- -i
>>>>>>
>>>>>> 12:46:46.257 §EAL: Detected 6 lcore(s)
>>>>>>
>>>>>> 12:46:46.257 §EAL: Detected 1 NUMA nodes
>>>>>>
>>>>>> 12:46:46.258 §EAL: Detected static linkage of DPDK
>>>>>>
>>>>>> 12:46:46.260 §EAL: Multi-process socket /var/run/dpdk/rte/mp_socket
>>>>>>
>>>>>> 12:46:46.285 §EAL: Selected IOVA mode 'PA'
>>>>>>
>>>>>> 12:46:46.285 §EAL: No free 2048 kB hugepages reported on node 0
>>>>>>
>>>>>> 12:46:46.288 §EAL: No available 2048 kB hugepages reported
>>>>>>
>>>>>> 12:46:46.382 §EAL: Probe PCI driver: net_qede (1077:8070) device:
>>>>>> 0000:65:00.0 (socket 0)
>>>>>>
>>>>>> 12:46:46.403 §[QEDE PMD: ()]ecore_load_mcp_offsets:The address of
>>>>>> the MCP scratch-pad is not configured
>>>>>>
>>>>>> 12:46:46.403 §[QEDE PMD: ()]ecore_mcp_cmd_init:MCP is not
>>>>>> initialized
>>>>>>
>>>>>> 12:46:46.404 §[QEDE PMD: ()]ecore_mcp_cmd_and_union:MFW is not
>>>>>> initialized!
>>>>>>
>>>>>> 12:46:46.404 §[QEDE PMD: ()]ecore_hw_get_nvm_info:Shared
>> memory not
>>>>>> initialized
>>>>>>
>>>>>> 12:46:46.404 §[QEDE PMD: ()]ecore_hw_prepare_single:Failed to get
>>>>>> HW information
>>>>>>
>>>>>> 12:46:46.404 §[qed_probe:74(65:00.0:dpdk-port-0)]hw prepare failed
>>>>>>
>>>>>> 12:46:46.404 §[qede_common_dev_init:2566(65:00.0:dpdk-port-
>> 0)]qede
>>>>>> probe failed rc -3
>>>>>>
>>>>>> 12:46:46.406 §EAL: Requested device 0000:65:00.0 cannot be used
>>>>>>
>>>>>> 12:46:46.408 §TELEMETRY: No legacy callbacks, legacy socket not
>>>>>> created
>>>>>>
>>>>>> 12:46:46.423 §testpmd: No probed ethernet devices
>>>>>>
>>>>>> 12:46:46.461 §Interactive-mode selected
>>>>>>
>>>>>> 12:46:46.519 §testpmd: create a new mbuf pool <mb_pool_0>:
>>>>>> n=187456, size=2176, socket=0
>>>>>>
>>>>>> 12:46:46.522 §testpmd: preferred mempool ops selected: ring_mp_mc
>>>>>>
>>>>>> 12:46:46.540 §Done
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>> System Information:
>>>>>>
>>>>>> *ethtool -i p2p1*
>>>>>> driver: qede
>>>>>> version: 8.33.0.20
>>>>>> *firmware-version: mfw 8.24.44.0 storm 8.33.11.0*
>>>>>> expansion-rom-version:
>>>>>> bus-info: 0000:65:00.0
>>>>>> supports-statistics: yes
>>>>>> supports-test: yes
>>>>>> supports-eeprom-access: no
>>>>>> supports-register-dump: yes
>>>>>> supports-priv-flags: yes
>>>>>>
>>>>>>
>>>>>>
>>>>>> *ethtool -i p2p2 *driver: qede
>>>>>> version: 8.33.0.20
>>>>>> firmware-version: mfw 8.24.44.0 storm 8.33.11.0
>>>>>> expansion-rom-version:
>>>>>> bus-info: 0000:65:00.1
>>>>>> supports-statistics: yes
>>>>>> supports-test: yes
>>>>>> supports-eeprom-access: no
>>>>>> supports-register-dump: yes
>>>>>> supports-priv-flags: yes
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>> *lspci -nn | grep Ether*
>>>>>>
>>>>>> 12:42:50.579 §04:00.0 Ethernet controller [0200]: Broadcom Inc. and
>>>>>> subsidiaries NetXtreme BCM5720 Gigabit Ethernet PCIe [14e4:165f]
>>>>>>
>>>>>> 12:42:50.581 §04:00.1 Ethernet controller [0200]: Broadcom Inc. and
>>>>>> subsidiaries NetXtreme BCM5720 Gigabit Ethernet PCIe [14e4:165f]
>>>>>>
>>>>>> 12:42:50.582 §65:00.0 Ethernet controller [0200]: QLogic Corp.
>>>>>> FastLinQ
>>>>>> QL41000 Series 10/25/40/50GbE Controller [1077:8070] (rev 02)
>>>>>>
>>>>>> 12:42:50.583 §65:00.1 Ethernet controller [0200]: QLogic Corp.
>>>>>> FastLinQ
>>>>>> QL41000 Series 10/25/40/50GbE Controller [1077:8070] (rev 02)
>>>>>>
>>>>>>
>>>>>>
>>>>>> cd /lib/firmware/qed/
>>>>>>
>>>>>> ls
>>>>>>
>>>>>> 12:48:27.269 §qed_init_values-8.10.9.0.bin
>>>>>> qed_init_values-8.30.12.0.bin qed_init_values_zipped-8.10.5.0.bin
>>>>>> qed_init_values_zipped-8.33.11.0.bin
>>>>>> qed_init_values_zipped-8.7.3.0.bin
>>>>>>
>>>>>> 12:48:27.272 §qed_init_values-8.14.6.0.bin
>>>>>> qed_init_values-8.33.12.0.bin qed_init_values_zipped-8.15.3.0.bin
>>>>>> qed_init_values_zipped-8.37.2.0.bin
>>>>>>
>>>>>> 12:48:27.273 §qed_init_values-8.18.9.0.bin
>>>>>> qed_init_values-8.40.33.0.bin qed_init_values_zipped-8.20.0.0.bin
>>>>>> qed_init_values_zipped-8.37.7.0.bin
>>>>>>
>>>>>> 12:48:27.275 §qed_init_values-8.20.0.0.bin
>>>>>> qed_init_values_zipped-8.10.10.0.bin
>>>>>> qed_init_values_zipped-8.33.1.0.bin
>>>>>> qed_init_values_zipped-8.4.2.0.bin
>>>>>>
>>>>>>
>>>>>> Best Regards
>>>>>>
>>>>>> Puneet
>>>>>>
>>>>>
prev parent reply other threads:[~2021-06-01 13:01 UTC|newest]
Thread overview: 7+ messages / expand[flat|nested] mbox.gz Atom feed top
2021-05-26 1:51 [dpdk-users] " Truring Team
2021-05-26 4:13 ` Nishant Verma
2021-05-26 4:20 ` Truring Team
2021-05-26 4:35 ` Nishant Verma
2021-05-26 4:41 ` Truring Team
2021-05-26 9:57 ` [dpdk-users] [dpdk-dev] " Varghese, Vipin
2021-06-01 13:00 ` Igor Russkikh [this message]
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=4233f366-b4ea-fbbb-0fde-d478be4292ac@marvell.com \
--to=irusskikh@marvell.com \
--cc=dev@dpdk.org \
--cc=dsinghrawat@marvell.com \
--cc=rmody@marvell.com \
--cc=truring12@gmail.com \
--cc=users@dpdk.org \
--cc=vipin.varghese@intel.com \
--cc=vnish11@gmail.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).