* [dpdk-users] RX of multi-segment jumbo frames
@ 2019-02-09 11:11 Filip Janiszewski
2019-02-09 13:51 ` Wiles, Keith
0 siblings, 1 reply; 6+ messages in thread
From: Filip Janiszewski @ 2019-02-09 11:11 UTC (permalink / raw)
To: users
Hi,
I'm attempting to receive jumbo frames (~9000 bytes) on a Mellonox card
using DPDK, I've configured the DEV_RX_OFFLOAD_JUMBO_FRAME offload for
rte_eth_conf and rte_eth_rxconf (per RX Queue), but I can capture jumbo
frames only if the mbuf is large enough to contain the whole packet, is
there a way to enable DPDK to chain the incoming data in mbufs smaller
than the actual packet?
We don't have many of those big packets coming in, so would be optimal
to leave the mbuf size to RTE_MBUF_DEFAULT_BUF_SIZE and then configure
the RX device to chain those bufs for larger packets, but can't find a
way to do it, any suggestion?
Thanks
--
BR, Filip
+48 666 369 823
^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: [dpdk-users] RX of multi-segment jumbo frames
2019-02-09 11:11 [dpdk-users] RX of multi-segment jumbo frames Filip Janiszewski
@ 2019-02-09 13:51 ` Wiles, Keith
2019-02-09 15:27 ` Filip Janiszewski
0 siblings, 1 reply; 6+ messages in thread
From: Wiles, Keith @ 2019-02-09 13:51 UTC (permalink / raw)
To: Filip Janiszewski; +Cc: users
> On Feb 9, 2019, at 5:11 AM, Filip Janiszewski <contact@filipjaniszewski.com> wrote:
>
> Hi,
>
> I'm attempting to receive jumbo frames (~9000 bytes) on a Mellonox card
> using DPDK, I've configured the DEV_RX_OFFLOAD_JUMBO_FRAME offload for
> rte_eth_conf and rte_eth_rxconf (per RX Queue), but I can capture jumbo
> frames only if the mbuf is large enough to contain the whole packet, is
> there a way to enable DPDK to chain the incoming data in mbufs smaller
> than the actual packet?
>
> We don't have many of those big packets coming in, so would be optimal
> to leave the mbuf size to RTE_MBUF_DEFAULT_BUF_SIZE and then configure
> the RX device to chain those bufs for larger packets, but can't find a
> way to do it, any suggestion?
>
the best i understand is the nic or pmd needs to be configured to split up packets between mbufs in the rx ring. i look in the docs for the nic and see if it supports splitting up packets or ask the maintainer from the maintainers file.
> Thanks
>
> --
> BR, Filip
> +48 666 369 823
Regards,
Keith
^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: [dpdk-users] RX of multi-segment jumbo frames
2019-02-09 13:51 ` Wiles, Keith
@ 2019-02-09 15:27 ` Filip Janiszewski
2019-02-09 15:36 ` Wiles, Keith
0 siblings, 1 reply; 6+ messages in thread
From: Filip Janiszewski @ 2019-02-09 15:27 UTC (permalink / raw)
To: Wiles, Keith; +Cc: users
Il 09/02/19 14:51, Wiles, Keith ha scritto:
>
>
>> On Feb 9, 2019, at 5:11 AM, Filip Janiszewski <contact@filipjaniszewski.com> wrote:
>>
>> Hi,
>>
>> I'm attempting to receive jumbo frames (~9000 bytes) on a Mellonox card
>> using DPDK, I've configured the DEV_RX_OFFLOAD_JUMBO_FRAME offload for
>> rte_eth_conf and rte_eth_rxconf (per RX Queue), but I can capture jumbo
>> frames only if the mbuf is large enough to contain the whole packet, is
>> there a way to enable DPDK to chain the incoming data in mbufs smaller
>> than the actual packet?
>>
>> We don't have many of those big packets coming in, so would be optimal
>> to leave the mbuf size to RTE_MBUF_DEFAULT_BUF_SIZE and then configure
>> the RX device to chain those bufs for larger packets, but can't find a
>> way to do it, any suggestion?
>>
>
> the best i understand is the nic or pmd needs to be configured to split up packets between mbufs in the rx ring. i look in the docs for the nic and see if it supports splitting up packets or ask the maintainer from the maintainers file.
I can capture jumbo packets with Wireshark on the same card (same port,
same setup), which let me think the problem is purely on my DPDK card
configuration.
According to ethtools, the jumbo packet (from now on JF, Jumbo Frame) is
detected at phy level, the couters rx_packets_phy, rx_bytes_phy,
rx_8192_to_10239_bytes_phy are properly increased.
There was an option to setup manually the support for JF but was remove
from DPDK after version 16.07: CONFIG_RTE_LIBRTE_MLX5_SGE_WR_N.
According to the release note:
.
Improved jumbo frames support, by dynamically setting RX scatter gather
elements according to the MTU and mbuf size, no need for compilation
parameter ``MLX5_PMD_SGE_WR_N``
.
Not quire sure where to look for..
>> Thanks
>>
>> --
>> BR, Filip
>> +48 666 369 823
>
> Regards,
> Keith
>
--
BR, Filip
+48 666 369 823
^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: [dpdk-users] RX of multi-segment jumbo frames
2019-02-09 15:27 ` Filip Janiszewski
@ 2019-02-09 15:36 ` Wiles, Keith
2019-02-15 5:59 ` Filip Janiszewski
0 siblings, 1 reply; 6+ messages in thread
From: Wiles, Keith @ 2019-02-09 15:36 UTC (permalink / raw)
To: Filip Janiszewski; +Cc: users
> On Feb 9, 2019, at 9:27 AM, Filip Janiszewski <contact@filipjaniszewski.com> wrote:
>
>
>
> Il 09/02/19 14:51, Wiles, Keith ha scritto:
>>
>>
>>> On Feb 9, 2019, at 5:11 AM, Filip Janiszewski <contact@filipjaniszewski.com> wrote:
>>>
>>> Hi,
>>>
>>> I'm attempting to receive jumbo frames (~9000 bytes) on a Mellonox card
>>> using DPDK, I've configured the DEV_RX_OFFLOAD_JUMBO_FRAME offload for
>>> rte_eth_conf and rte_eth_rxconf (per RX Queue), but I can capture jumbo
>>> frames only if the mbuf is large enough to contain the whole packet, is
>>> there a way to enable DPDK to chain the incoming data in mbufs smaller
>>> than the actual packet?
>>>
>>> We don't have many of those big packets coming in, so would be optimal
>>> to leave the mbuf size to RTE_MBUF_DEFAULT_BUF_SIZE and then configure
>>> the RX device to chain those bufs for larger packets, but can't find a
>>> way to do it, any suggestion?
>>>
>>
>> the best i understand is the nic or pmd needs to be configured to split up packets between mbufs in the rx ring. i look in the docs for the nic and see if it supports splitting up packets or ask the maintainer from the maintainers file.
>
> I can capture jumbo packets with Wireshark on the same card (same port,
> same setup), which let me think the problem is purely on my DPDK card
> configuration.
>
> According to ethtools, the jumbo packet (from now on JF, Jumbo Frame) is
> detected at phy level, the couters rx_packets_phy, rx_bytes_phy,
> rx_8192_to_10239_bytes_phy are properly increased.
>
> There was an option to setup manually the support for JF but was remove
> from DPDK after version 16.07: CONFIG_RTE_LIBRTE_MLX5_SGE_WR_N.
> According to the release note:
>
> .
> Improved jumbo frames support, by dynamically setting RX scatter gather
> elements according to the MTU and mbuf size, no need for compilation
> parameter ``MLX5_PMD_SGE_WR_N``
> .
>
> Not quire sure where to look for..
>
maintainer is your best bet now.
>>> Thanks
>>>
>>> --
>>> BR, Filip
>>> +48 666 369 823
>>
>> Regards,
>> Keith
>>
>
> --
> BR, Filip
> +48 666 369 823
Regards,
Keith
^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: [dpdk-users] RX of multi-segment jumbo frames
2019-02-09 15:36 ` Wiles, Keith
@ 2019-02-15 5:59 ` Filip Janiszewski
2019-02-15 13:30 ` Wiles, Keith
0 siblings, 1 reply; 6+ messages in thread
From: Filip Janiszewski @ 2019-02-15 5:59 UTC (permalink / raw)
To: users; +Cc: Wiles, Keith
Unfortunately I didn't get much help from the maintainers at Mellanox,
but I discovered that with DPDK 18.05 there's the flag
ignore_offload_bitfield which once toggled to 1 along with the offloads
set to DEV_RX_OFFLOAD_JUMBO_FRAME|DEV_RX_OFFLOAD_SCATTER allows DPDK to
capture Jumbo on Mellanox:
https://doc.dpdk.org/api-18.05/structrte__eth__rxmode.html
In DPDK 19.02 this flag is missing and I can't capture Jumbos with my
current configuration.
Sadly, even if setting ignore_offload_bitfield to 1 fix my problem it
creates a bunch more, the packets coming in are not timestamped for
example (setting hw_timestamp to 1 does not fix the issue as the
timestamp are still EPOCH + some ms.).
Not sure if this can trigger any idea, for me it is not completely clear
what was the purpose of ignore_offload_bitfield (removed later) and how
to enable Jumbos properly.
What I've attempted so far (apart from the ignore_offload_bitfield):
1) Set mtu to 9600 (rte_eth_dev_set_mtu)
2) Configure port with offloads DEV_RX_OFFLOAD_SCATTER |
DEV_RX_OFFLOAD_JUMBO_FRAME, max_rx_pkt_len set to 9600
3) Configure RX queue with default_rxconf (from rte_eth_dev_info) adding
the offloads from the port configuration (DEV_RX_OFFLOAD_SCATTER |
DEV_RX_OFFLOAD_JUMBO_FRAME)
The JF are reported as ierror in rte_eth_stats.
Thanks
Il 09/02/19 16:36, Wiles, Keith ha scritto:
>
>
>> On Feb 9, 2019, at 9:27 AM, Filip Janiszewski <contact@filipjaniszewski.com> wrote:
>>
>>
>>
>> Il 09/02/19 14:51, Wiles, Keith ha scritto:
>>>
>>>
>>>> On Feb 9, 2019, at 5:11 AM, Filip Janiszewski <contact@filipjaniszewski.com> wrote:
>>>>
>>>> Hi,
>>>>
>>>> I'm attempting to receive jumbo frames (~9000 bytes) on a Mellonox card
>>>> using DPDK, I've configured the DEV_RX_OFFLOAD_JUMBO_FRAME offload for
>>>> rte_eth_conf and rte_eth_rxconf (per RX Queue), but I can capture jumbo
>>>> frames only if the mbuf is large enough to contain the whole packet, is
>>>> there a way to enable DPDK to chain the incoming data in mbufs smaller
>>>> than the actual packet?
>>>>
>>>> We don't have many of those big packets coming in, so would be optimal
>>>> to leave the mbuf size to RTE_MBUF_DEFAULT_BUF_SIZE and then configure
>>>> the RX device to chain those bufs for larger packets, but can't find a
>>>> way to do it, any suggestion?
>>>>
>>>
>>> the best i understand is the nic or pmd needs to be configured to split up packets between mbufs in the rx ring. i look in the docs for the nic and see if it supports splitting up packets or ask the maintainer from the maintainers file.
>>
>> I can capture jumbo packets with Wireshark on the same card (same port,
>> same setup), which let me think the problem is purely on my DPDK card
>> configuration.
>>
>> According to ethtools, the jumbo packet (from now on JF, Jumbo Frame) is
>> detected at phy level, the couters rx_packets_phy, rx_bytes_phy,
>> rx_8192_to_10239_bytes_phy are properly increased.
>>
>> There was an option to setup manually the support for JF but was remove
>> from DPDK after version 16.07: CONFIG_RTE_LIBRTE_MLX5_SGE_WR_N.
>> According to the release note:
>>
>> .
>> Improved jumbo frames support, by dynamically setting RX scatter gather
>> elements according to the MTU and mbuf size, no need for compilation
>> parameter ``MLX5_PMD_SGE_WR_N``
>> .
>>
>> Not quire sure where to look for..
>>
>
> maintainer is your best bet now.
>>>> Thanks
>>>>
>>>> --
>>>> BR, Filip
>>>> +48 666 369 823
>>>
>>> Regards,
>>> Keith
>>>
>>
>> --
>> BR, Filip
>> +48 666 369 823
>
> Regards,
> Keith
>
--
BR, Filip
+48 666 369 823
^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: [dpdk-users] RX of multi-segment jumbo frames
2019-02-15 5:59 ` Filip Janiszewski
@ 2019-02-15 13:30 ` Wiles, Keith
0 siblings, 0 replies; 6+ messages in thread
From: Wiles, Keith @ 2019-02-15 13:30 UTC (permalink / raw)
To: Filip Janiszewski; +Cc: users
> On Feb 14, 2019, at 11:59 PM, Filip Janiszewski <contact@filipjaniszewski.com> wrote:
>
> Unfortunately I didn't get much help from the maintainers at Mellanox,
> but I discovered that with DPDK 18.05 there's the flag
> ignore_offload_bitfield which once toggled to 1 along with the offloads
> set to DEV_RX_OFFLOAD_JUMBO_FRAME|DEV_RX_OFFLOAD_SCATTER allows DPDK to
> capture Jumbo on Mellanox:
>
> https://doc.dpdk.org/api-18.05/structrte__eth__rxmode.html
>
> In DPDK 19.02 this flag is missing and I can't capture Jumbos with my
> current configuration.
>
> Sadly, even if setting ignore_offload_bitfield to 1 fix my problem it
> creates a bunch more, the packets coming in are not timestamped for
> example (setting hw_timestamp to 1 does not fix the issue as the
> timestamp are still EPOCH + some ms.).
>
> Not sure if this can trigger any idea, for me it is not completely clear
> what was the purpose of ignore_offload_bitfield (removed later) and how
> to enable Jumbos properly.
>
> What I've attempted so far (apart from the ignore_offload_bitfield):
>
> 1) Set mtu to 9600 (rte_eth_dev_set_mtu)
> 2) Configure port with offloads DEV_RX_OFFLOAD_SCATTER |
> DEV_RX_OFFLOAD_JUMBO_FRAME, max_rx_pkt_len set to 9600
> 3) Configure RX queue with default_rxconf (from rte_eth_dev_info) adding
> the offloads from the port configuration (DEV_RX_OFFLOAD_SCATTER |
> DEV_RX_OFFLOAD_JUMBO_FRAME)
>
> The JF are reported as ierror in rte_eth_stats.
sorry, the last time i had any dealings with mellanox i was not able to get it to work. so not going to be much help here.
>
> Thanks
>
> Il 09/02/19 16:36, Wiles, Keith ha scritto:
>>
>>
>>> On Feb 9, 2019, at 9:27 AM, Filip Janiszewski <contact@filipjaniszewski.com> wrote:
>>>
>>>
>>>
>>> Il 09/02/19 14:51, Wiles, Keith ha scritto:
>>>>
>>>>
>>>>> On Feb 9, 2019, at 5:11 AM, Filip Janiszewski <contact@filipjaniszewski.com> wrote:
>>>>>
>>>>> Hi,
>>>>>
>>>>> I'm attempting to receive jumbo frames (~9000 bytes) on a Mellonox card
>>>>> using DPDK, I've configured the DEV_RX_OFFLOAD_JUMBO_FRAME offload for
>>>>> rte_eth_conf and rte_eth_rxconf (per RX Queue), but I can capture jumbo
>>>>> frames only if the mbuf is large enough to contain the whole packet, is
>>>>> there a way to enable DPDK to chain the incoming data in mbufs smaller
>>>>> than the actual packet?
>>>>>
>>>>> We don't have many of those big packets coming in, so would be optimal
>>>>> to leave the mbuf size to RTE_MBUF_DEFAULT_BUF_SIZE and then configure
>>>>> the RX device to chain those bufs for larger packets, but can't find a
>>>>> way to do it, any suggestion?
>>>>>
>>>>
>>>> the best i understand is the nic or pmd needs to be configured to split up packets between mbufs in the rx ring. i look in the docs for the nic and see if it supports splitting up packets or ask the maintainer from the maintainers file.
>>>
>>> I can capture jumbo packets with Wireshark on the same card (same port,
>>> same setup), which let me think the problem is purely on my DPDK card
>>> configuration.
>>>
>>> According to ethtools, the jumbo packet (from now on JF, Jumbo Frame) is
>>> detected at phy level, the couters rx_packets_phy, rx_bytes_phy,
>>> rx_8192_to_10239_bytes_phy are properly increased.
>>>
>>> There was an option to setup manually the support for JF but was remove
>>> from DPDK after version 16.07: CONFIG_RTE_LIBRTE_MLX5_SGE_WR_N.
>>> According to the release note:
>>>
>>> .
>>> Improved jumbo frames support, by dynamically setting RX scatter gather
>>> elements according to the MTU and mbuf size, no need for compilation
>>> parameter ``MLX5_PMD_SGE_WR_N``
>>> .
>>>
>>> Not quire sure where to look for..
>>>
>>
>> maintainer is your best bet now.
>>>>> Thanks
>>>>>
>>>>> --
>>>>> BR, Filip
>>>>> +48 666 369 823
>>>>
>>>> Regards,
>>>> Keith
>>>>
>>>
>>> --
>>> BR, Filip
>>> +48 666 369 823
>>
>> Regards,
>> Keith
>>
>
> --
> BR, Filip
> +48 666 369 823
Regards,
Keith
^ permalink raw reply [flat|nested] 6+ messages in thread
end of thread, other threads:[~2019-02-15 13:30 UTC | newest]
Thread overview: 6+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2019-02-09 11:11 [dpdk-users] RX of multi-segment jumbo frames Filip Janiszewski
2019-02-09 13:51 ` Wiles, Keith
2019-02-09 15:27 ` Filip Janiszewski
2019-02-09 15:36 ` Wiles, Keith
2019-02-15 5:59 ` Filip Janiszewski
2019-02-15 13:30 ` Wiles, Keith
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).