* [dpdk-users] Packets not processed when huge pages are not used
@ 2015-10-21 20:29 shesha Sreenivasamurthy (shesha)
2015-10-21 21:08 ` shesha Sreenivasamurthy (shesha)
0 siblings, 1 reply; 5+ messages in thread
From: shesha Sreenivasamurthy (shesha) @ 2015-10-21 20:29 UTC (permalink / raw)
To: users
Hi,
I have a packet processing application running correctly using hugepages with 82599 interface. However, when no-huge is set, I do not see any packets being processed by DPDK. When arp packets are sent to this interface, I see them being received by the device (BPRC counter gets incremented). However, the DD register does not get set by the hardware. Therefore, _recv_raw_pkts_vec() returns thinking that there are no packets (shown below). Therefore, when huge pages are not used, I suspect that some DMA configuration is getting messed up. Any tips on how to go about finding the issue will be helpful.
drivers/net/ixgbe/ixgbe_rxtx_vec.c
303 /* Before we start moving massive data around, check to see if
304 * there is actually a packet available */
305 if (!(rxdp->wb.upper.status_error &
306 rte_cpu_to_le_32(IXGBE_RXDADV_STAT_DD)))
307 return 0;
Observation: In lib/librte_eal/linuxapp/eal/eal_memory.c::rte_eal_hugepage_init() physical memory is set to be same as virtual (mcfg->memseg[0].phys_addr = (phys_addr_t)(uintptr_t)addr;). Is that expected ?
^ permalink raw reply [flat|nested] 5+ messages in thread
* Re: [dpdk-users] Packets not processed when huge pages are not used
2015-10-21 20:29 [dpdk-users] Packets not processed when huge pages are not used shesha Sreenivasamurthy (shesha)
@ 2015-10-21 21:08 ` shesha Sreenivasamurthy (shesha)
2015-10-22 14:37 ` Sergio Gonzalez Monroy
0 siblings, 1 reply; 5+ messages in thread
From: shesha Sreenivasamurthy (shesha) @ 2015-10-21 21:08 UTC (permalink / raw)
To: users
Wanted to add ring addresses information:
PMD: ixgbe_dev_rx_queue_setup(): sw_ring=0x7fffa0979440 sw_sc_ring=0x7fffa0975300 hw_ring=0x7fffa097d580 dma_addr=0x7fffa097d580
--
- Thanks
char * (*shesha) (uint64_t cache, uint8_t F00D)
{ return 0x0000C0DE; }
From: Cisco Employee <shesha@cisco.com<mailto:shesha@cisco.com>>
Date: Wednesday, October 21, 2015 at 1:30 PM
To: "users@dpdk.org<mailto:users@dpdk.org>" <users@dpdk.org<mailto:users@dpdk.org>>
Subject: Packets not processed when huge pages are not used
Hi,
I have a packet processing application running correctly using hugepages with 82599 interface. However, when no-huge is set, I do not see any packets being processed by DPDK. When arp packets are sent to this interface, I see them being received by the device (BPRC counter gets incremented). However, the DD register does not get set by the hardware. Therefore, _recv_raw_pkts_vec() returns thinking that there are no packets (shown below). Therefore, when huge pages are not used, I suspect that some DMA configuration is getting messed up. Any tips on how to go about finding the issue will be helpful.
drivers/net/ixgbe/ixgbe_rxtx_vec.c
303 /* Before we start moving massive data around, check to see if
304 * there is actually a packet available */
305 if (!(rxdp->wb.upper.status_error &
306 rte_cpu_to_le_32(IXGBE_RXDADV_STAT_DD)))
307 return 0;
Observation: In lib/librte_eal/linuxapp/eal/eal_memory.c::rte_eal_hugepage_init() physical memory is set to be same as virtual (mcfg->memseg[0].phys_addr = (phys_addr_t)(uintptr_t)addr;). Is that expected ?
^ permalink raw reply [flat|nested] 5+ messages in thread* Re: [dpdk-users] Packets not processed when huge pages are not used
2015-10-21 21:08 ` shesha Sreenivasamurthy (shesha)
@ 2015-10-22 14:37 ` Sergio Gonzalez Monroy
2015-10-22 16:09 ` shesha Sreenivasamurthy (shesha)
0 siblings, 1 reply; 5+ messages in thread
From: Sergio Gonzalez Monroy @ 2015-10-22 14:37 UTC (permalink / raw)
To: shesha Sreenivasamurthy (shesha); +Cc: users
On 21/10/2015 22:08, shesha Sreenivasamurthy (shesha) wrote:
> Wanted to add ring addresses information:
>
> PMD: ixgbe_dev_rx_queue_setup(): sw_ring=0x7fffa0979440 sw_sc_ring=0x7fffa0975300 hw_ring=0x7fffa097d580 dma_addr=0x7fffa097d580
>
> --
> - Thanks
> char * (*shesha) (uint64_t cache, uint8_t F00D)
> { return 0x0000C0DE; }
>
> From: Cisco Employee <shesha@cisco.com<mailto:shesha@cisco.com>>
> Date: Wednesday, October 21, 2015 at 1:30 PM
> To: "users@dpdk.org<mailto:users@dpdk.org>" <users@dpdk.org<mailto:users@dpdk.org>>
> Subject: Packets not processed when huge pages are not used
>
> Hi,
> I have a packet processing application running correctly using hugepages with 82599 interface. However, when no-huge is set, I do not see any packets being processed by DPDK. When arp packets are sent to this interface, I see them being received by the device (BPRC counter gets incremented). However, the DD register does not get set by the hardware. Therefore, _recv_raw_pkts_vec() returns thinking that there are no packets (shown below). Therefore, when huge pages are not used, I suspect that some DMA configuration is getting messed up. Any tips on how to go about finding the issue will be helpful.
>
> drivers/net/ixgbe/ixgbe_rxtx_vec.c
> 303 /* Before we start moving massive data around, check to see if
> 304 * there is actually a packet available */
> 305 if (!(rxdp->wb.upper.status_error &
> 306 rte_cpu_to_le_32(IXGBE_RXDADV_STAT_DD)))
> 307 return 0;
>
> Observation: In lib/librte_eal/linuxapp/eal/eal_memory.c::rte_eal_hugepage_init() physical memory is set to be same as virtual (mcfg->memseg[0].phys_addr = (phys_addr_t)(uintptr_t)addr;). Is that expected ?
Hi Shesha,
It is a known issue:
http://dpdk.org/doc/guides/rel_notes/known_issues.html#pmd-does-not-work-with-no-huge-eal-command-line-parameter
It should work with IOMMU=on and VFIO driver, but I haven't verified
that myself.
Sergio
^ permalink raw reply [flat|nested] 5+ messages in thread* Re: [dpdk-users] Packets not processed when huge pages are not used
2015-10-22 14:37 ` Sergio Gonzalez Monroy
@ 2015-10-22 16:09 ` shesha Sreenivasamurthy (shesha)
2015-10-23 9:24 ` Sergio Gonzalez Monroy
0 siblings, 1 reply; 5+ messages in thread
From: shesha Sreenivasamurthy (shesha) @ 2015-10-22 16:09 UTC (permalink / raw)
To: Sergio Gonzalez Monroy; +Cc: users
If someone else is not woking on it, I can fix that. What is the right way
to fix it?
Should we have to mmap and perform similar trick (as done for huge pages)
of finding physical pages that are contiguous and discarding the unneeded
mappings ?
--
- Thanks
char * (*shesha) (uint64_t cache, uint8_t F00D)
{ return 0x0000C0DE; }
-----Original Message-----
From: Sergio Gonzalez Monroy <sergio.gonzalez.monroy@intel.com>
Date: Thursday, October 22, 2015 at 7:37 AM
To: Cisco Employee <shesha@cisco.com>
Cc: "users@dpdk.org" <users@dpdk.org>
Subject: Re: [dpdk-users] Packets not processed when huge pages are not
used
On 21/10/2015 22:08, shesha Sreenivasamurthy (shesha) wrote:
> Wanted to add ring addresses information:
>
> PMD: ixgbe_dev_rx_queue_setup(): sw_ring=0x7fffa0979440
>sw_sc_ring=0x7fffa0975300 hw_ring=0x7fffa097d580 dma_addr=0x7fffa097d580
>
> --
> - Thanks
> char * (*shesha) (uint64_t cache, uint8_t F00D)
> { return 0x0000C0DE; }
>
> From: Cisco Employee <shesha@cisco.com<mailto:shesha@cisco.com>>
> Date: Wednesday, October 21, 2015 at 1:30 PM
> To: "users@dpdk.org<mailto:users@dpdk.org>"
><users@dpdk.org<mailto:users@dpdk.org>>
> Subject: Packets not processed when huge pages are not used
>
> Hi,
> I have a packet processing application running correctly using
>hugepages with 82599 interface. However, when no-huge is set, I do not
>see any packets being processed by DPDK. When arp packets are sent to
>this interface, I see them being received by the device (BPRC counter
>gets incremented). However, the DD register does not get set by the
>hardware. Therefore, _recv_raw_pkts_vec() returns thinking that there are
>no packets (shown below). Therefore, when huge pages are not used, I
>suspect that some DMA configuration is getting messed up. Any tips on how
>to go about finding the issue will be helpful.
>
> drivers/net/ixgbe/ixgbe_rxtx_vec.c
> 303 /* Before we start moving massive data around, check to see if
> 304 * there is actually a packet available */
> 305 if (!(rxdp->wb.upper.status_error &
> 306 rte_cpu_to_le_32(IXGBE_RXDADV_STAT_DD)))
> 307 return 0;
>
> Observation: In
>lib/librte_eal/linuxapp/eal/eal_memory.c::rte_eal_hugepage_init()
>physical memory is set to be same as virtual (mcfg->memseg[0].phys_addr =
>(phys_addr_t)(uintptr_t)addr;). Is that expected ?
Hi Shesha,
It is a known issue:
http://dpdk.org/doc/guides/rel_notes/known_issues.html#pmd-does-not-work-wi
th-no-huge-eal-command-line-parameter
It should work with IOMMU=on and VFIO driver, but I haven't verified
that myself.
Sergio
^ permalink raw reply [flat|nested] 5+ messages in thread* Re: [dpdk-users] Packets not processed when huge pages are not used
2015-10-22 16:09 ` shesha Sreenivasamurthy (shesha)
@ 2015-10-23 9:24 ` Sergio Gonzalez Monroy
0 siblings, 0 replies; 5+ messages in thread
From: Sergio Gonzalez Monroy @ 2015-10-23 9:24 UTC (permalink / raw)
To: shesha Sreenivasamurthy (shesha); +Cc: users
On 22/10/2015 17:09, shesha Sreenivasamurthy (shesha) wrote:
> If someone else is not woking on it, I can fix that. What is the right way
> to fix it?
>
> Should we have to mmap and perform similar trick (as done for huge pages)
> of finding physical pages that are contiguous and discarding the unneeded
> mappings ?
So this is an old problem; get DMA memory in user-space.
You would have to to use a kernel module that would give you physically
contiguous memory.
The contigmem module that we use for FreeBSD does something like that.
Sergio
> --
> - Thanks
> char * (*shesha) (uint64_t cache, uint8_t F00D)
> { return 0x0000C0DE; }
>
>
> -----Original Message-----
> From: Sergio Gonzalez Monroy <sergio.gonzalez.monroy@intel.com>
> Date: Thursday, October 22, 2015 at 7:37 AM
> To: Cisco Employee <shesha@cisco.com>
> Cc: "users@dpdk.org" <users@dpdk.org>
> Subject: Re: [dpdk-users] Packets not processed when huge pages are not
> used
>
> On 21/10/2015 22:08, shesha Sreenivasamurthy (shesha) wrote:
>> Wanted to add ring addresses information:
>>
>> PMD: ixgbe_dev_rx_queue_setup(): sw_ring=0x7fffa0979440
>> sw_sc_ring=0x7fffa0975300 hw_ring=0x7fffa097d580 dma_addr=0x7fffa097d580
>>
>> --
>> - Thanks
>> char * (*shesha) (uint64_t cache, uint8_t F00D)
>> { return 0x0000C0DE; }
>>
>> From: Cisco Employee <shesha@cisco.com<mailto:shesha@cisco.com>>
>> Date: Wednesday, October 21, 2015 at 1:30 PM
>> To: "users@dpdk.org<mailto:users@dpdk.org>"
>> <users@dpdk.org<mailto:users@dpdk.org>>
>> Subject: Packets not processed when huge pages are not used
>>
>> Hi,
>> I have a packet processing application running correctly using
>> hugepages with 82599 interface. However, when no-huge is set, I do not
>> see any packets being processed by DPDK. When arp packets are sent to
>> this interface, I see them being received by the device (BPRC counter
>> gets incremented). However, the DD register does not get set by the
>> hardware. Therefore, _recv_raw_pkts_vec() returns thinking that there are
>> no packets (shown below). Therefore, when huge pages are not used, I
>> suspect that some DMA configuration is getting messed up. Any tips on how
>> to go about finding the issue will be helpful.
>>
>> drivers/net/ixgbe/ixgbe_rxtx_vec.c
>> 303 /* Before we start moving massive data around, check to see if
>> 304 * there is actually a packet available */
>> 305 if (!(rxdp->wb.upper.status_error &
>> 306 rte_cpu_to_le_32(IXGBE_RXDADV_STAT_DD)))
>> 307 return 0;
>>
>> Observation: In
>> lib/librte_eal/linuxapp/eal/eal_memory.c::rte_eal_hugepage_init()
>> physical memory is set to be same as virtual (mcfg->memseg[0].phys_addr =
>> (phys_addr_t)(uintptr_t)addr;). Is that expected ?
> Hi Shesha,
>
> It is a known issue:
> http://dpdk.org/doc/guides/rel_notes/known_issues.html#pmd-does-not-work-wi
> th-no-huge-eal-command-line-parameter
>
> It should work with IOMMU=on and VFIO driver, but I haven't verified
> that myself.
>
> Sergio
>
^ permalink raw reply [flat|nested] 5+ messages in thread
end of thread, other threads:[~2015-10-23 9:25 UTC | newest]
Thread overview: 5+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2015-10-21 20:29 [dpdk-users] Packets not processed when huge pages are not used shesha Sreenivasamurthy (shesha)
2015-10-21 21:08 ` shesha Sreenivasamurthy (shesha)
2015-10-22 14:37 ` Sergio Gonzalez Monroy
2015-10-22 16:09 ` shesha Sreenivasamurthy (shesha)
2015-10-23 9:24 ` Sergio Gonzalez Monroy
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).