* Re: [dpdk-users] [DPDK-PDUMP] Segmentation fault while Restarting PDUMP
@ 2018-03-21 13:43 RAJESH KUMAR S.R
2018-03-22 14:02 ` RAJESH KUMAR S.R
0 siblings, 1 reply; 2+ messages in thread
From: RAJESH KUMAR S.R @ 2018-03-21 13:43 UTC (permalink / raw)
To: Rosen, Rami, users
Hi Rami,
Got some temperary fix.
My understanding of the issue is as follows.
When the pdump is called for the first time,
rte_eth_devices attach to new ports.
rte_eth_dev_find_free_port() is called that returns 2 for rx_port and 3 for
tx_port.
Corresponding, tx_queues and rx_queues and other data are stored in
rte_eth_device.data[2] & rte_eth_devices[3].data, obtained from
rte_eth_dev_shared_data->data[2] and rte_eth_dev_shared_data->data[3]
During the program exit, these data are not getting cleaned up.
rte_eth_dev_shared_data->data[2].name and
rte_eth_dev_shared_data->data[3].name have the old values ("
net_pcap_rx_0","net_pcap_tx_0")
rte_eth_dev_find_free_port() iterates over all ports and selects the first
port that has no name.
When pdump is restarted, rte_eth_devices are getting attached to new
ports(rx_port at 4, tx_port at 5).
So, rte_eth_dev_find_free_port() returns 4 for rx and 5 for tx instead of 2
and 3 which are free.
In configure_vdev in pdump/main.c, the check (port_id > nb_ports) fails
and returns.
While debugging using gdb, I set ports as rx= 2 and tx= 3 when it was
getting set to rx=4 and tx=5, it was working fine
So, I reverted this DPDK Commit 8ee892a2385c50427c03db5cef1789babceb5999
ethdev: fix port id allocation
for (i = 0; i < RTE_MAX_ETHPORTS; i++) \{
- if (rte_eth_devices\[i\].state == RTE_ETH_DEV_UNUSED)
+ /* Using shared name field to find a free port. */
+ if (rte_eth_dev_data\[i\].name[0] == '\0') {
+ RTE_ASSERT(rte_eth_devices\[i\].state ==
+ RTE_ETH_DEV_UNUSED);
return i;
+ \}
\}
return RTE_MAX_ETHPORTS;
\}
After reverting, it was working fine.
I'm not sure whether this is the right way to do.
Regards,
Rajesh kumar S R
On Tue, Mar 20, 2018 at 2:55 PM, RAJESH KUMAR S.R <rajuuu1992@gmail.com>
wrote:
> Hi Rami,
>
>
> I don't have a proper understanding of the whole flow. I'm not sure what I
> have missed. Need some help
> Got some info while debugging.
>
>
> There is a structure "rte_eth_dev_shared_data". The "data" field in this
> structure has some old data from previous run.
>
> It is getting assigned in rte_eth_dev_shared_data() -->
>
> mz = rte_memzone_lookup(MZ_RTE_ETH_DEV_DATA);
> ....
> rte_eth_dev_shared_data = mz->addr;
>
>
> a.
>
> Function: rte_eth_dev_find_free_port
> backtrace: (create_mp_ring_vdev -->rte_eth_dev_attach((devargs=0x7fffffffe6e0
> "net_pcap_rx_0,tx_pcap=/tmp/rx.pcap", port_id=0x7fffffffe6ca))
> --> rte_eal_dev_attach -->rte_eal_hotplug_add --> vdev_plug--->
> vdev_probe_all_drivers --> pmd_pcap_probe --> eth_from_pcaps -->
> eth_from_pcaps_common -->pmd_init_internals --> rte_eth_vdev_allocate-->
> rte_eth_dev_allocate --> rte_eth_dev_find_free_port)
>
>
> During the first run,
>
> Port_ids that are allocated from rte_eth_dev_find_free_port function are
> 2(rx) and 3(tx).
>
> The data values in rte_eth_dev_shared_data are
>
> rte_eth_dev_shared_data->data[2]
> {name = "net_pcap_rx_0", .....}
>
> p rte_eth_dev_shared_data->data[3]
> {name = "net_pcap_tx_0", ....}
>
> The packet capture is working fine.
>
> b.
>
> During the second run,
> In function, rte_eth_dev_find_free_port
>
> "....
> if (rte_eth_dev_shared_data->data[i].name[0] == '\0') {
> RTE_ASSERT(rte_eth_devices[i].state ==
> RTE_ETH_DEV_UNUSED);
> return i;
> }
> ..."
>
>
> p rte_eth_dev_shared_data->data[2]
> $3 = {name = "net_pcap_rx_0", .....}
>
> p rte_eth_dev_shared_data->data[3]
> $4 = {name = "net_pcap_tx_0", ....}
>
>
> The previous data exists here.
> The port_ids that is returned by rte_eth_dev_find_free_port are 4(rx) &
> 5(tx).
>
> c.
> pdump/main.c -> In configure_vdev() , for port_id=4
> const uint8_t nb_ports = rte_eth_dev_count();
> (= 3)
> This check is failing
> if (port_id > nb_ports)
> return -1;
>
>
> In configure_vdev() , for port_id=5
> const uint8_t nb_ports = rte_eth_dev_count();
> (= 4)
> This check is failing
> if (port_id > nb_ports)
> return -1;
>
> rte_eth_dev_configure, rte_eth_dev_start are not getting called.
> So, In the rte_eth_tx_burst (dump_packets -->pdump_rxtx-->rte_eth_tx_burst)
> , some data in rte_eth_devices[port_id] seems to be NULL.
>
>
> I'll update with more info
>
>
> Regards,
> Rajesh Kumar S R
>
> On Mon, Mar 19, 2018 at 4:54 PM, RAJESH KUMAR S.R <rajuuu1992@gmail.com>
> wrote:
>
>> Hi Rami,
>> Thanks for the response.
>>
>> I used "echo 0 > /proc/sys/kernel/randomize_va_space "
>> for disabling ASLR, but the error still exists.
>> I checked with testpmd as primary process and the same error occurs when
>> I restart the pdump tool.
>>
>> I'm just trying to debug the segmentation fault in rte_eth_tx_burst
>> function for now.
>>
>>
>>
>> Thanks & Regards,
>> Rajesh kumar S R
>>
>> On Mon, Mar 19, 2018 at 12:39 AM, Rosen, Rami <rami.rosen@intel.com>
>> wrote:
>>
>>> Hi Rajesh,
>>>
>>>
>>>
>>> First notice the warning you get:
>>>
>>> >EAL: WARNING: Address Space Layout Randomization (ASLR) is enabled in
>>> the kernel.
>>>
>>> >EAL: This may cause issues with mapping memory into secondary
>>> processes
>>>
>>>
>>>
>>> So first, I would suggest that you will disable ASLR and try again, to
>>> verify it is not the cause. IIRC, in the past I had some issue with
>>> dpdk-pdump when ASLR was not enabled (but IIRC the issue itself was
>>> different – probably not getting packets written to the pcap file or
>>> something else).
>>>
>>> Disabling ASLR is done easily:
>>>
>>> Probably you have : cat /proc/sys/kernel/randomize_va_space
>>>
>>> gives “2”
>>>
>>> To disable it run
>>>
>>> echo 0 > /proc/sys/kernel/randomize_va_space
>>>
>>>
>>>
>>> And also I would suggest to try it with testpmd. I actually ran it today
>>> with
>>>
>>> DPDK 18.02 with testpmd, and it worked for me. I sent L2 traffic with
>>> scapy, and captured only RX packets with rx-dev; any other L2 packet
>>> generator should be OK.
>>>
>>>
>>>
>>> Regards,
>>>
>>> Rami Rosen
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>> *From:* RAJESH KUMAR S.R [mailto:rajuuu1992@gmail.com]
>>> *Sent:* Sunday, March 18, 2018 19:52
>>> *To:* Rosen, Rami <rami.rosen@intel.com>; users@dpdk.org
>>> *Subject:* Re: [dpdk-users] [DPDK-PDUMP] Issue: RING: Cannot reserve
>>> memory
>>>
>>>
>>>
>>>
>>>
>>> Hi Rami,
>>>
>>> Thanks for the response.
>>>
>>> DPDK version I'm using is 18.02.0.
>>>
>>> The primary process I'm using is custom program which uses dpdk.
>>>
>>> I have initialized pdump in the primary process using
>>> "rte_pdump_init(NULL)" similar to that in testpmd example.
>>>
>>> I'm using the pdump tool without making any changes.
>>>
>>>
>>>
>>> I enabled the following flags
>>> 1. CONFIG_RTE_LIBRTE_PDUMP=y
>>> 2. CONFIG_RTE_LIBRTE_PMD_PCAP=y
>>>
>>> I added the libpcap library
>>>
>>> The problem I'm facing is bit different from what I mentioned earlier.
>>>
>>> The first time when I run, I'm able to see the packets getting saved in
>>> the output pcap file, rx.pcap and tx.pcap.
>>>
>>> The second time when I run, I'm getting a segmentation fault as I have
>>> pasted the output in the previous message.
>>>
>>> I'm using gdb for debugging, the functions rte_eal_init(),
>>> create_mp_ring_vdev();
>>> enable_pdump(); are working fine.
>>>
>>>
>>>
>>> The segmentation fault is occuring in rte_eth_tx_burst
>>> function(dump_packets ->pdump_rxtx-->rte_eth_tx_burst)
>>>
>>> The rings tx_ring_0 and rx_ring_0 are not getting freed. I checked that
>>> with rte_memzone_dump(). So, next time when I start the pdump tool I was
>>> getting " memzone <RG_rx_ring_0> already exists . RING: Cannot reserve
>>> memory" error.
>>>
>>> So, issue is in rte_eth_tx_burst function, where the
>>> "dev->data->tx_queues[queue_id]" is NULL.
>>>
>>>
>>> "
>>> EAL: Detected lcore 0 as core 0 on socket 0
>>> EAL: Detected lcore 1 as core 0 on socket 0
>>> EAL: Support maximum 128 logical core(s) by configuration.
>>> EAL: Detected 2 lcore(s)
>>> EAL: Module /sys/module/vfio_pci not found! error 2 (No such file or
>>> directory)
>>> EAL: VFIO PCI modules not loaded
>>> EAL: Multi-process socket /var/run/.rte_unix_1817_14c6748457c5
>>> [New Thread 0x7ffff67d4700 (LWP 1821)]
>>> EAL: Probing VFIO support...
>>> EAL: Module /sys/module/vfio not found! error 2 (No such file or
>>> directory)
>>> EAL: VFIO modules not loaded, skipping VFIO support...
>>> EAL: Module /sys/module/vfio not found! error 2 (No such file or
>>> directory)
>>> EAL: Setting up physically contiguous memory...
>>> EAL: WARNING: Address Space Layout Randomization (ASLR) is enabled in
>>> the kernel.
>>> EAL: This may cause issues with mapping memory into secondary
>>> processes
>>> EAL: Analysing 300 files
>>> EAL: Mapped segment 0 of size 0x400000
>>> EAL: Mapped segment 1 of size 0x200000
>>> EAL: Mapped segment 2 of size 0x200000
>>> EAL: Mapped segment 3 of size 0x200000
>>> EAL: Mapped segment 4 of size 0x24c00000
>>> EAL: Mapped segment 5 of size 0x200000
>>> EAL: TSC frequency is ~2904192 KHz
>>> EAL: WARNING: cpu flags constant_tsc=yes nonstop_tsc=no -> using
>>> unreliable clock cycles !
>>> EAL: Master lcore 0 is ready (tid=f7fec8c0;cpuset=[0])
>>> [New Thread 0x7ffff5fd3700 (LWP 1822)]
>>> EAL: PCI device 0000:00:03.0 on NUMA socket -1
>>> EAL: Invalid NUMA socket, default to 0
>>> EAL: probe driver: 1af4:1000 net_virtio
>>> EAL: PCI Port IO found start=0xc060
>>> EAL: PCI device 0000:00:04.0 on NUMA socket -1
>>> EAL: Invalid NUMA socket, default to 0
>>> EAL: probe driver: 1af4:1000 net_virtio
>>> EAL: PCI Port IO found start=0xc080
>>> EAL: PCI device 0000:00:05.0 on NUMA socket -1
>>> EAL: Invalid NUMA socket, default to 0
>>> EAL: probe driver: 1af4:1000 net_virtio
>>> EAL: Requested device 0000:00:05.0 cannot be used
>>> PMD: Initializing pmd_pcap for net_pcap_rx_0
>>> PMD: Creating pcap-backed ethdev on numa socket -1
>>> PMD: Initializing pmd_pcap for net_pcap_tx_0
>>> PMD: Creating pcap-backed ethdev on numa socket -1
>>>
>>> Thread 1 "dpdk-pdump" received signal SIGSEGV, Segmentation fault.
>>> 0x000000000043ee07 in rte_eth_tx_burst (port_id=4, queue_id=0,
>>> tx_pkts=0x7fffffffe8c0, nb_pkts=1)
>>> "
>>>
>>> I don't know why tx_queues is NULL.
>>>
>>> I'm currently checking where that is getting populated.
>>>
>>>
>>>
>>> Thanks & Regards,
>>>
>>> Rajesh kumar S R
>>>
>>>
>>>
>>>
>>>
>>> On Sat, Mar 17, 2018 at 9:28 PM, Rosen, Rami <rami.rosen@intel.com>
>>> wrote:
>>>
>>> Hi Rajesh,
>>>
>>> Can you please provide more details:
>>>
>>> - Which DPDK version are you using?
>>>
>>> - dpdk-pdump is a secondary process and must be launched along with
>>> any primary process. Which is the primary process you are running ?
>>> is it something that you developed on your own/took something from DPDK
>>> examples (with/without extending it), etc ?
>>>
>>> Regards,
>>> Rami Rosen
>>>
>>>
>>> -----Original Message-----
>>> From: users [mailto:users-bounces@dpdk.org] On Behalf Of RAJESH KUMAR
>>> S.R
>>> Sent: Monday, March 12, 2018 12:25
>>> To: users@dpdk.org
>>> Subject: [dpdk-users] [DPDK-PDUMP] Issue: RING: Cannot reserve memory
>>>
>>> Hi,
>>>
>>> I am new to dpdk.
>>> I'm trying to use dpdk-pdump tool. I'm able to capture packets on dpdk
>>> ports.
>>>
>>> But, I'm facing the following issue
>>> I'm getting a error "RING:Cannot reserve memory" while trying to restart
>>> the pdump tool or running 2 instances of pdump.
>>> I have also used the rte_eal_cleanup while exiting pdump main function.
>>>
>>>
>>> Output:
>>> * sudo /opt/pep/active/bin/dpdk-pdump -- --pdump
>>> 'port=0,queue=*,rx-dev=/tmp/rx.pcap,tx-dev=/tmp/tx.pcap'*
>>>
>>> [sudo] password for admin:
>>> EAL: Detected 2 lcore(s)
>>> EAL: Multi-process socket /var/run/.rte_unix_1061_ce297dfe759
>>> EAL: Probing VFIO support...
>>> EAL: WARNING: Address Space Layout Randomization (ASLR) is enabled in
>>> the kernel.
>>> EAL: This may cause issues with mapping memory into secondary
>>> processes
>>> EAL: WARNING: cpu flags constant_tsc=yes nonstop_tsc=no -> using
>>> unreliable clock cycles !
>>> EAL: PCI device 0000:00:03.0 on NUMA socket -1
>>> EAL: Invalid NUMA socket, default to 0
>>> EAL: probe driver: 1af4:1000 net_virtio
>>> EAL: PCI device 0000:00:04.0 on NUMA socket -1
>>> EAL: Invalid NUMA socket, default to 0
>>> EAL: probe driver: 1af4:1000 net_virtio
>>> EAL: PCI device 0000:00:05.0 on NUMA socket -1
>>> EAL: Invalid NUMA socket, default to 0
>>> EAL: probe driver: 1af4:1000 net_virtio
>>> EAL: Requested device 0000:00:05.0 cannot be used
>>> PMD: Initializing pmd_pcap for net_pcap_rx_0
>>> PMD: Creating pcap-backed ethdev on numa socket -1 Port 2 MAC: 00 00 00
>>> 01 02 03
>>> PMD: Initializing pmd_pcap for net_pcap_tx_0
>>> PMD: Creating pcap-backed ethdev on numa socket -1 Port 3 MAC: 00 00 00
>>> 01 02 03 ^C
>>>
>>> Signal 2 received, preparing to exit...
>>> ##### PDUMP DEBUG STATS #####
>>> -packets dequeued: 4
>>> -packets transmitted to vdev: 4
>>> -packets freed: 0
>>>
>>>
>>> Restarting........
>>>
>>> *> sudo /opt/pep/active/bin/dpdk-pdump -- --pdump
>>> 'port=0,queue=*,rx-dev=/tmp/rx.pcap,tx-dev=/tmp/tx.pcap'*
>>> EAL: Detected 2 lcore(s)
>>> EAL: Multi-process socket /var/run/.rte_unix_1073_cea9cc51241
>>> EAL: Probing VFIO support...
>>> EAL: WARNING: Address Space Layout Randomization (ASLR) is enabled in
>>> the kernel.
>>> EAL: This may cause issues with mapping memory into secondary
>>> processes
>>> EAL: WARNING: cpu flags constant_tsc=yes nonstop_tsc=no -> using
>>> unreliable clock cycles !
>>> EAL: PCI device 0000:00:03.0 on NUMA socket -1
>>> EAL: Invalid NUMA socket, default to 0
>>> EAL: probe driver: 1af4:1000 net_virtio
>>> EAL: PCI device 0000:00:04.0 on NUMA socket -1
>>> EAL: Invalid NUMA socket, default to 0
>>> EAL: probe driver: 1af4:1000 net_virtio
>>> EAL: PCI device 0000:00:05.0 on NUMA socket -1
>>> EAL: Invalid NUMA socket, default to 0
>>> EAL: probe driver: 1af4:1000 net_virtio
>>> EAL: Requested device 0000:00:05.0 cannot be used
>>> PMD: Initializing pmd_pcap for net_pcap_rx_0
>>> PMD: Creating pcap-backed ethdev on numa socket -1
>>> PMD: Initializing pmd_pcap for net_pcap_tx_0
>>> PMD: Creating pcap-backed ethdev on numa socket -1 Segmentation fault
>>>
>>> *> sudo /opt/pep/active/bin/dpdk-pdump -- --pdump
>>> 'port=0,queue=*,rx-dev=/tmp/rx.pcap,tx-dev=/tmp/tx.pcap'*
>>>
>>> EAL: Detected 2 lcore(s)
>>> EAL: Multi-process socket /var/run/.rte_unix_1087_cec3ab006b1
>>> EAL: Probing VFIO support...
>>> EAL: WARNING: Address Space Layout Randomization (ASLR) is enabled in
>>> the kernel.
>>> EAL: This may cause issues with mapping memory into secondary
>>> processes
>>> EAL: WARNING: cpu flags constant_tsc=yes nonstop_tsc=no -> using
>>> unreliable clock cycles !
>>> EAL: PCI device 0000:00:03.0 on NUMA socket -1
>>> EAL: Invalid NUMA socket, default to 0
>>> EAL: probe driver: 1af4:1000 net_virtio
>>> EAL: PCI device 0000:00:04.0 on NUMA socket -1
>>> EAL: Invalid NUMA socket, default to 0
>>> EAL: probe driver: 1af4:1000 net_virtio
>>> EAL: PCI device 0000:00:05.0 on NUMA socket -1
>>> EAL: Invalid NUMA socket, default to 0
>>> EAL: probe driver: 1af4:1000 net_virtio
>>> EAL: Requested device 0000:00:05.0 cannot be used
>>> RING: Cannot reserve memory
>>> EAL: Error - exiting with code: 1
>>> Cause: File exists:create_mp_ring_vdev:634
>>>
>>>
>>>
>>> Hugepage info:
>>> cat /proc/meminfo | grep Huge
>>> AnonHugePages: 6144 kB
>>> HugePages_Total: 300
>>> HugePages_Free: 0
>>> HugePages_Rsvd: 0
>>> HugePages_Surp: 0
>>> Hugepagesize: 2048 kB
>>>
>>>
>>> Can you please help me in finding the issue.
>>>
>>>
>>>
>>> Thanks,
>>> Rajesh kumar S R
>>>
>>>
>>>
>>
>>
>
^ permalink raw reply [flat|nested] 2+ messages in thread
* Re: [dpdk-users] [DPDK-PDUMP] Segmentation fault while Restarting PDUMP
2018-03-21 13:43 [dpdk-users] [DPDK-PDUMP] Segmentation fault while Restarting PDUMP RAJESH KUMAR S.R
@ 2018-03-22 14:02 ` RAJESH KUMAR S.R
0 siblings, 0 replies; 2+ messages in thread
From: RAJESH KUMAR S.R @ 2018-03-22 14:02 UTC (permalink / raw)
To: Rosen, Rami, users
I verified the with testpmd as primary application and pdump as secondary
process( with latest 18.02 version without any changes).
Segmentation fault is occuring when I restart and send packets. Not sure
where I went wrong.
On Wed, Mar 21, 2018 at 7:13 PM, RAJESH KUMAR S.R <rajuuu1992@gmail.com>
wrote:
> Hi Rami,
>
> Got some temporary fix.
>
> My understanding of the issue is as follows.
>
> When the pdump is called for the first time,
>
> rte_eth_devices attach to new ports.
>
> rte_eth_dev_find_free_port() is called that returns 2 for rx_port and 3
> for tx_port.
>
> Corresponding, tx_queues and rx_queues and other data are stored in
> rte_eth_device.data[2] & rte_eth_devices[3].data, obtained from
> rte_eth_dev_shared_data->data[2] and rte_eth_dev_shared_data->data[3]
>
"rte_eth_devices[2].data and rte_eth_devices[3].data are pointed to
rte_eth_dev_shared_data-.data[2] and rte_eth_dev_shared_data->data[3]
respectively and names "net_pcap_tx_0" and "net_pcap_rx_0" are written to
it"
> During the program exit, these data are not getting cleaned up.
> rte_eth_dev_shared_data->data[2].name and rte_eth_dev_shared_data->data[3].name
> have the old values (" net_pcap_rx_0","net_pcap_tx_0")
>
>
> rte_eth_dev_find_free_port() iterates over all ports and selects the first
> port that has no name.
>
> When pdump is restarted, rte_eth_devices are getting attached to new
> ports(rx_port at 4, tx_port at 5).
>
> So, rte_eth_dev_find_free_port() returns 4 for rx and 5 for tx instead of
> 2 and 3 which are free.
>
> In configure_vdev in pdump/main.c, the check (port_id > nb_ports) fails
> and returns.
>
> While debugging using gdb, I set ports as rx= 2 and tx= 3 when it was
> getting set to rx=4 and tx=5, it was working fine
>
> So, I reverted this DPDK Commit 8ee892a2385c50427c03db5cef1789babceb5999
>
> ethdev: fix port id allocation
>
> for (i = 0; i < RTE_MAX_ETHPORTS; i++) \{
> - if (rte_eth_devices\[i\].state == RTE_ETH_DEV_UNUSED)
> + /* Using shared name field to find a free port. */
> + if (rte_eth_dev_data\[i\].name[0] == '\0') {
> + RTE_ASSERT(rte_eth_devices\[i\].state ==
> + RTE_ETH_DEV_UNUSED);
> return i;
> + \}
> \}
> return RTE_MAX_ETHPORTS;
> \}
>
> After reverting, it was working fine.
> I'm not sure whether this is the right way to do.
>
>
>
> Regards,
> Rajesh kumar S R
>
> On Tue, Mar 20, 2018 at 2:55 PM, RAJESH KUMAR S.R <rajuuu1992@gmail.com>
> wrote:
>
>> Hi Rami,
>>
>>
>> I don't have a proper understanding of the whole flow. I'm not sure what
>> I have missed. Need some help
>> Got some info while debugging.
>>
>>
>> There is a structure "rte_eth_dev_shared_data". The "data" field in this
>> structure has some old data from previous run.
>>
>> It is getting assigned in rte_eth_dev_shared_data() -->
>>
>> mz = rte_memzone_lookup(MZ_RTE_ETH_DEV_DATA);
>> ....
>> rte_eth_dev_shared_data = mz->addr;
>>
>>
>> a.
>>
>> Function: rte_eth_dev_find_free_port
>> backtrace: (create_mp_ring_vdev -->rte_eth_dev_attach((devargs=0x7fffffffe6e0
>> "net_pcap_rx_0,tx_pcap=/tmp/rx.pcap", port_id=0x7fffffffe6ca))
>> --> rte_eal_dev_attach -->rte_eal_hotplug_add --> vdev_plug--->
>> vdev_probe_all_drivers --> pmd_pcap_probe --> eth_from_pcaps -->
>> eth_from_pcaps_common -->pmd_init_internals --> rte_eth_vdev_allocate-->
>> rte_eth_dev_allocate --> rte_eth_dev_find_free_port)
>>
>>
>> During the first run,
>>
>> Port_ids that are allocated from rte_eth_dev_find_free_port function are
>> 2(rx) and 3(tx).
>>
>> The data values in rte_eth_dev_shared_data are
>>
>> rte_eth_dev_shared_data->data[2]
>> {name = "net_pcap_rx_0", .....}
>>
>> p rte_eth_dev_shared_data->data[3]
>> {name = "net_pcap_tx_0", ....}
>>
>> The packet capture is working fine.
>>
>> b.
>>
>> During the second run,
>> In function, rte_eth_dev_find_free_port
>>
>> "....
>> if (rte_eth_dev_shared_data->data[i].name[0] == '\0') {
>> RTE_ASSERT(rte_eth_devices[i].state ==
>> RTE_ETH_DEV_UNUSED);
>> return i;
>> }
>> ..."
>>
>>
>> p rte_eth_dev_shared_data->data[2]
>> $3 = {name = "net_pcap_rx_0", .....}
>>
>> p rte_eth_dev_shared_data->data[3]
>> $4 = {name = "net_pcap_tx_0", ....}
>>
>>
>> The previous data exists here.
>> The port_ids that is returned by rte_eth_dev_find_free_port are 4(rx) &
>> 5(tx).
>>
>> c.
>> pdump/main.c -> In configure_vdev() , for port_id=4
>> const uint8_t nb_ports = rte_eth_dev_count();
>> (= 3)
>> This check is failing
>> if (port_id > nb_ports)
>> return -1;
>>
>>
>> In configure_vdev() , for port_id=5
>> const uint8_t nb_ports = rte_eth_dev_count();
>> (= 4)
>> This check is failing
>> if (port_id > nb_ports)
>> return -1;
>>
>> rte_eth_dev_configure, rte_eth_dev_start are not getting called.
>> So, In the rte_eth_tx_burst (dump_packets -->pdump_rxtx-->rte_eth_tx_burst)
>> , some data in rte_eth_devices[port_id] seems to be NULL.
>>
>>
>> I'll update with more info
>>
>>
>> Regards,
>> Rajesh Kumar S R
>>
>> On Mon, Mar 19, 2018 at 4:54 PM, RAJESH KUMAR S.R <rajuuu1992@gmail.com>
>> wrote:
>>
>>> Hi Rami,
>>> Thanks for the response.
>>>
>>> I used "echo 0 > /proc/sys/kernel/randomize_va_space "
>>> for disabling ASLR, but the error still exists.
>>> I checked with testpmd as primary process and the same error occurs when
>>> I restart the pdump tool.
>>>
>>> I'm just trying to debug the segmentation fault in rte_eth_tx_burst
>>> function for now.
>>>
>>>
>>>
>>> Thanks & Regards,
>>> Rajesh kumar S R
>>>
>>> On Mon, Mar 19, 2018 at 12:39 AM, Rosen, Rami <rami.rosen@intel.com>
>>> wrote:
>>>
>>>> Hi Rajesh,
>>>>
>>>>
>>>>
>>>> First notice the warning you get:
>>>>
>>>> >EAL: WARNING: Address Space Layout Randomization (ASLR) is enabled in
>>>> the kernel.
>>>>
>>>> >EAL: This may cause issues with mapping memory into secondary
>>>> processes
>>>>
>>>>
>>>>
>>>> So first, I would suggest that you will disable ASLR and try again, to
>>>> verify it is not the cause. IIRC, in the past I had some issue with
>>>> dpdk-pdump when ASLR was not enabled (but IIRC the issue itself was
>>>> different – probably not getting packets written to the pcap file or
>>>> something else).
>>>>
>>>> Disabling ASLR is done easily:
>>>>
>>>> Probably you have : cat /proc/sys/kernel/randomize_va_space
>>>>
>>>> gives “2”
>>>>
>>>> To disable it run
>>>>
>>>> echo 0 > /proc/sys/kernel/randomize_va_space
>>>>
>>>>
>>>>
>>>> And also I would suggest to try it with testpmd. I actually ran it
>>>> today with
>>>>
>>>> DPDK 18.02 with testpmd, and it worked for me. I sent L2 traffic with
>>>> scapy, and captured only RX packets with rx-dev; any other L2 packet
>>>> generator should be OK.
>>>>
>>>>
>>>>
>>>> Regards,
>>>>
>>>> Rami Rosen
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>> *From:* RAJESH KUMAR S.R [mailto:rajuuu1992@gmail.com]
>>>> *Sent:* Sunday, March 18, 2018 19:52
>>>> *To:* Rosen, Rami <rami.rosen@intel.com>; users@dpdk.org
>>>> *Subject:* Re: [dpdk-users] [DPDK-PDUMP] Issue: RING: Cannot reserve
>>>> memory
>>>>
>>>>
>>>>
>>>>
>>>>
>>>> Hi Rami,
>>>>
>>>> Thanks for the response.
>>>>
>>>> DPDK version I'm using is 18.02.0.
>>>>
>>>> The primary process I'm using is custom program which uses dpdk.
>>>>
>>>> I have initialized pdump in the primary process using
>>>> "rte_pdump_init(NULL)" similar to that in testpmd example.
>>>>
>>>> I'm using the pdump tool without making any changes.
>>>>
>>>>
>>>>
>>>> I enabled the following flags
>>>> 1. CONFIG_RTE_LIBRTE_PDUMP=y
>>>> 2. CONFIG_RTE_LIBRTE_PMD_PCAP=y
>>>>
>>>> I added the libpcap library
>>>>
>>>> The problem I'm facing is bit different from what I mentioned earlier.
>>>>
>>>> The first time when I run, I'm able to see the packets getting saved in
>>>> the output pcap file, rx.pcap and tx.pcap.
>>>>
>>>> The second time when I run, I'm getting a segmentation fault as I have
>>>> pasted the output in the previous message.
>>>>
>>>> I'm using gdb for debugging, the functions rte_eal_init(),
>>>> create_mp_ring_vdev();
>>>> enable_pdump(); are working fine.
>>>>
>>>>
>>>>
>>>> The segmentation fault is occuring in rte_eth_tx_burst
>>>> function(dump_packets ->pdump_rxtx-->rte_eth_tx_burst)
>>>>
>>>> The rings tx_ring_0 and rx_ring_0 are not getting freed. I checked that
>>>> with rte_memzone_dump(). So, next time when I start the pdump tool I was
>>>> getting " memzone <RG_rx_ring_0> already exists . RING: Cannot reserve
>>>> memory" error.
>>>>
>>>> So, issue is in rte_eth_tx_burst function, where the
>>>> "dev->data->tx_queues[queue_id]" is NULL.
>>>>
>>>>
>>>> "
>>>> EAL: Detected lcore 0 as core 0 on socket 0
>>>> EAL: Detected lcore 1 as core 0 on socket 0
>>>> EAL: Support maximum 128 logical core(s) by configuration.
>>>> EAL: Detected 2 lcore(s)
>>>> EAL: Module /sys/module/vfio_pci not found! error 2 (No such file or
>>>> directory)
>>>> EAL: VFIO PCI modules not loaded
>>>> EAL: Multi-process socket /var/run/.rte_unix_1817_14c6748457c5
>>>> [New Thread 0x7ffff67d4700 (LWP 1821)]
>>>> EAL: Probing VFIO support...
>>>> EAL: Module /sys/module/vfio not found! error 2 (No such file or
>>>> directory)
>>>> EAL: VFIO modules not loaded, skipping VFIO support...
>>>> EAL: Module /sys/module/vfio not found! error 2 (No such file or
>>>> directory)
>>>> EAL: Setting up physically contiguous memory...
>>>> EAL: WARNING: Address Space Layout Randomization (ASLR) is enabled in
>>>> the kernel.
>>>> EAL: This may cause issues with mapping memory into secondary
>>>> processes
>>>> EAL: Analysing 300 files
>>>> EAL: Mapped segment 0 of size 0x400000
>>>> EAL: Mapped segment 1 of size 0x200000
>>>> EAL: Mapped segment 2 of size 0x200000
>>>> EAL: Mapped segment 3 of size 0x200000
>>>> EAL: Mapped segment 4 of size 0x24c00000
>>>> EAL: Mapped segment 5 of size 0x200000
>>>> EAL: TSC frequency is ~2904192 KHz
>>>> EAL: WARNING: cpu flags constant_tsc=yes nonstop_tsc=no -> using
>>>> unreliable clock cycles !
>>>> EAL: Master lcore 0 is ready (tid=f7fec8c0;cpuset=[0])
>>>> [New Thread 0x7ffff5fd3700 (LWP 1822)]
>>>> EAL: PCI device 0000:00:03.0 on NUMA socket -1
>>>> EAL: Invalid NUMA socket, default to 0
>>>> EAL: probe driver: 1af4:1000 net_virtio
>>>> EAL: PCI Port IO found start=0xc060
>>>> EAL: PCI device 0000:00:04.0 on NUMA socket -1
>>>> EAL: Invalid NUMA socket, default to 0
>>>> EAL: probe driver: 1af4:1000 net_virtio
>>>> EAL: PCI Port IO found start=0xc080
>>>> EAL: PCI device 0000:00:05.0 on NUMA socket -1
>>>> EAL: Invalid NUMA socket, default to 0
>>>> EAL: probe driver: 1af4:1000 net_virtio
>>>> EAL: Requested device 0000:00:05.0 cannot be used
>>>> PMD: Initializing pmd_pcap for net_pcap_rx_0
>>>> PMD: Creating pcap-backed ethdev on numa socket -1
>>>> PMD: Initializing pmd_pcap for net_pcap_tx_0
>>>> PMD: Creating pcap-backed ethdev on numa socket -1
>>>>
>>>> Thread 1 "dpdk-pdump" received signal SIGSEGV, Segmentation fault.
>>>> 0x000000000043ee07 in rte_eth_tx_burst (port_id=4, queue_id=0,
>>>> tx_pkts=0x7fffffffe8c0, nb_pkts=1)
>>>> "
>>>>
>>>> I don't know why tx_queues is NULL.
>>>>
>>>> I'm currently checking where that is getting populated.
>>>>
>>>>
>>>>
>>>> Thanks & Regards,
>>>>
>>>> Rajesh kumar S R
>>>>
>>>>
>>>>
>>>>
>>>>
>>>> On Sat, Mar 17, 2018 at 9:28 PM, Rosen, Rami <rami.rosen@intel.com>
>>>> wrote:
>>>>
>>>> Hi Rajesh,
>>>>
>>>> Can you please provide more details:
>>>>
>>>> - Which DPDK version are you using?
>>>>
>>>> - dpdk-pdump is a secondary process and must be launched along with
>>>> any primary process. Which is the primary process you are running ?
>>>> is it something that you developed on your own/took something from DPDK
>>>> examples (with/without extending it), etc ?
>>>>
>>>> Regards,
>>>> Rami Rosen
>>>>
>>>>
>>>> -----Original Message-----
>>>> From: users [mailto:users-bounces@dpdk.org] On Behalf Of RAJESH KUMAR
>>>> S.R
>>>> Sent: Monday, March 12, 2018 12:25
>>>> To: users@dpdk.org
>>>> Subject: [dpdk-users] [DPDK-PDUMP] Issue: RING: Cannot reserve memory
>>>>
>>>> Hi,
>>>>
>>>> I am new to dpdk.
>>>> I'm trying to use dpdk-pdump tool. I'm able to capture packets on dpdk
>>>> ports.
>>>>
>>>> But, I'm facing the following issue
>>>> I'm getting a error "RING:Cannot reserve memory" while trying to
>>>> restart the pdump tool or running 2 instances of pdump.
>>>> I have also used the rte_eal_cleanup while exiting pdump main function.
>>>>
>>>>
>>>> Output:
>>>> * sudo /opt/pep/active/bin/dpdk-pdump -- --pdump
>>>> 'port=0,queue=*,rx-dev=/tmp/rx.pcap,tx-dev=/tmp/tx.pcap'*
>>>>
>>>> [sudo] password for admin:
>>>> EAL: Detected 2 lcore(s)
>>>> EAL: Multi-process socket /var/run/.rte_unix_1061_ce297dfe759
>>>> EAL: Probing VFIO support...
>>>> EAL: WARNING: Address Space Layout Randomization (ASLR) is enabled in
>>>> the kernel.
>>>> EAL: This may cause issues with mapping memory into secondary
>>>> processes
>>>> EAL: WARNING: cpu flags constant_tsc=yes nonstop_tsc=no -> using
>>>> unreliable clock cycles !
>>>> EAL: PCI device 0000:00:03.0 on NUMA socket -1
>>>> EAL: Invalid NUMA socket, default to 0
>>>> EAL: probe driver: 1af4:1000 net_virtio
>>>> EAL: PCI device 0000:00:04.0 on NUMA socket -1
>>>> EAL: Invalid NUMA socket, default to 0
>>>> EAL: probe driver: 1af4:1000 net_virtio
>>>> EAL: PCI device 0000:00:05.0 on NUMA socket -1
>>>> EAL: Invalid NUMA socket, default to 0
>>>> EAL: probe driver: 1af4:1000 net_virtio
>>>> EAL: Requested device 0000:00:05.0 cannot be used
>>>> PMD: Initializing pmd_pcap for net_pcap_rx_0
>>>> PMD: Creating pcap-backed ethdev on numa socket -1 Port 2 MAC: 00 00 00
>>>> 01 02 03
>>>> PMD: Initializing pmd_pcap for net_pcap_tx_0
>>>> PMD: Creating pcap-backed ethdev on numa socket -1 Port 3 MAC: 00 00 00
>>>> 01 02 03 ^C
>>>>
>>>> Signal 2 received, preparing to exit...
>>>> ##### PDUMP DEBUG STATS #####
>>>> -packets dequeued: 4
>>>> -packets transmitted to vdev: 4
>>>> -packets freed: 0
>>>>
>>>>
>>>> Restarting........
>>>>
>>>> *> sudo /opt/pep/active/bin/dpdk-pdump -- --pdump
>>>> 'port=0,queue=*,rx-dev=/tmp/rx.pcap,tx-dev=/tmp/tx.pcap'*
>>>> EAL: Detected 2 lcore(s)
>>>> EAL: Multi-process socket /var/run/.rte_unix_1073_cea9cc51241
>>>> EAL: Probing VFIO support...
>>>> EAL: WARNING: Address Space Layout Randomization (ASLR) is enabled in
>>>> the kernel.
>>>> EAL: This may cause issues with mapping memory into secondary
>>>> processes
>>>> EAL: WARNING: cpu flags constant_tsc=yes nonstop_tsc=no -> using
>>>> unreliable clock cycles !
>>>> EAL: PCI device 0000:00:03.0 on NUMA socket -1
>>>> EAL: Invalid NUMA socket, default to 0
>>>> EAL: probe driver: 1af4:1000 net_virtio
>>>> EAL: PCI device 0000:00:04.0 on NUMA socket -1
>>>> EAL: Invalid NUMA socket, default to 0
>>>> EAL: probe driver: 1af4:1000 net_virtio
>>>> EAL: PCI device 0000:00:05.0 on NUMA socket -1
>>>> EAL: Invalid NUMA socket, default to 0
>>>> EAL: probe driver: 1af4:1000 net_virtio
>>>> EAL: Requested device 0000:00:05.0 cannot be used
>>>> PMD: Initializing pmd_pcap for net_pcap_rx_0
>>>> PMD: Creating pcap-backed ethdev on numa socket -1
>>>> PMD: Initializing pmd_pcap for net_pcap_tx_0
>>>> PMD: Creating pcap-backed ethdev on numa socket -1 Segmentation fault
>>>>
>>>> *> sudo /opt/pep/active/bin/dpdk-pdump -- --pdump
>>>> 'port=0,queue=*,rx-dev=/tmp/rx.pcap,tx-dev=/tmp/tx.pcap'*
>>>>
>>>> EAL: Detected 2 lcore(s)
>>>> EAL: Multi-process socket /var/run/.rte_unix_1087_cec3ab006b1
>>>> EAL: Probing VFIO support...
>>>> EAL: WARNING: Address Space Layout Randomization (ASLR) is enabled in
>>>> the kernel.
>>>> EAL: This may cause issues with mapping memory into secondary
>>>> processes
>>>> EAL: WARNING: cpu flags constant_tsc=yes nonstop_tsc=no -> using
>>>> unreliable clock cycles !
>>>> EAL: PCI device 0000:00:03.0 on NUMA socket -1
>>>> EAL: Invalid NUMA socket, default to 0
>>>> EAL: probe driver: 1af4:1000 net_virtio
>>>> EAL: PCI device 0000:00:04.0 on NUMA socket -1
>>>> EAL: Invalid NUMA socket, default to 0
>>>> EAL: probe driver: 1af4:1000 net_virtio
>>>> EAL: PCI device 0000:00:05.0 on NUMA socket -1
>>>> EAL: Invalid NUMA socket, default to 0
>>>> EAL: probe driver: 1af4:1000 net_virtio
>>>> EAL: Requested device 0000:00:05.0 cannot be used
>>>> RING: Cannot reserve memory
>>>> EAL: Error - exiting with code: 1
>>>> Cause: File exists:create_mp_ring_vdev:634
>>>>
>>>>
>>>>
>>>> Hugepage info:
>>>> cat /proc/meminfo | grep Huge
>>>> AnonHugePages: 6144 kB
>>>> HugePages_Total: 300
>>>> HugePages_Free: 0
>>>> HugePages_Rsvd: 0
>>>> HugePages_Surp: 0
>>>> Hugepagesize: 2048 kB
>>>>
>>>>
>>>> Can you please help me in finding the issue.
>>>>
>>>>
>>>>
>>>> Thanks,
>>>> Rajesh kumar S R
>>>>
>>>>
>>>>
>>>
>>>
>>
>
^ permalink raw reply [flat|nested] 2+ messages in thread
end of thread, other threads:[~2018-03-22 14:03 UTC | newest]
Thread overview: 2+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2018-03-21 13:43 [dpdk-users] [DPDK-PDUMP] Segmentation fault while Restarting PDUMP RAJESH KUMAR S.R
2018-03-22 14:02 ` RAJESH KUMAR S.R
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).