[dpdk-dev] Question about jumbo frame support on ixgbe

[dpdk-dev] Question about jumbo frame support on ixgbe

[Hideyuki Yamashita]

Hi, 

I have a very basic question about jumbo frame support for ixgbe.

I understand that some drivers support jumbo frame and 
if it receive jumbo packet (greater than 1500 byte), it creates
mbuf chains and pass it to DPDK application through 
e.g. rte_eth_rx_burst.

However it looks that ixgbe driver does not support jumbo frame.

Q1. Is my understanding above correct?
Q2. If A1 equals YES, then are there any future plan to support 
jumbo frame on ixgbe?

BR,
Hideyuki Yamashita
NTT TechnoCross

[dpdk-dev] [PATCH] Doc: add known issue for restricted vdev ethdev ops in secondary process

[Marvin Liu]

Signed-off-by: Marvin Liu <yong.liu@intel.com>

diff --git a/doc/guides/rel_notes/known_issues.rst b/doc/guides/rel_notes/known_issues.rst
index 95e4ce693..617255cd7 100644
--- a/doc/guides/rel_notes/known_issues.rst
+++ b/doc/guides/rel_notes/known_issues.rst
@@ -759,3 +759,21 @@ Netvsc driver and application restart
 
 **Driver/Module**:
    ``uio_hv_generic`` module.
+
+
+Restricted vdev ethdev operations supported in secondary process
+----------------------------------------------------------------
+**Description**
+   In current virtual device sharing model, Ethernet device data structure will be
+   shared between primary and secondary process. Only those Ethernet device operations
+   which based on it are workable in secondary process.
+
+**Implication**
+   Some Ethernet device operations like device start/stop will be failed on virtual
+   device in secondary process.
+
+**Affected Environment/Platform**:
+   ALL.
+
+**Driver/Module**:
+   Virtual Device Poll Mode Driver (PMD).
-- 
2.14.1

Re: [dpdk-dev] Question about jumbo frame support on ixgbe

[Stephen Hemminger]

On Wed, 31 Oct 2018 17:22:02 +0900
Hideyuki Yamashita <yamashita.hideyuki@po.ntt-tx.co.jp> wrote:

> Hi, 
> 
> I have a very basic question about jumbo frame support for ixgbe.
> 
> I understand that some drivers support jumbo frame and 
> if it receive jumbo packet (greater than 1500 byte), it creates
> mbuf chains and pass it to DPDK application through 
> e.g. rte_eth_rx_burst.
> 
> However it looks that ixgbe driver does not support jumbo frame.
> 
> Q1. Is my understanding above correct?
> Q2. If A1 equals YES, then are there any future plan to support 
> jumbo frame on ixgbe?
> 
> BR,
> Hideyuki Yamashita
> NTT TechnoCross
> 

I don't work for Intel, and haven't tried jumbo frames on ixgbe.
The hardware does support jumbo frames.
You might be confusing the overlapping receive offload values.

The setting for using jumbo frames is in current DPDK is shown
as available in device as:
	rx_offload_capa & DEV_RX_OFFLOAD_JUMBO_FRAME
In order to use it, this bit must be setn in rx_conf that
is passed to rte_eth_rx_queue_setup


The feature bit that creates multiple mbuf's if necessary on
receive is:
	rx_offload_capa  & DEV_RX_OFFLOAD_SCATTER
Likewise to enable it, this must be set in rte_eth_rx_queue_setup
rx_conf.

Re: [dpdk-dev] Question about jumbo frame support on ixgbe

[Zhao1, Wei]

Hi,  Hideyuki Yamashita


> -----Original Message-----
> From: dev [mailto:dev-bounces@dpdk.org] On Behalf Of Hideyuki Yamashita
> Sent: Wednesday, October 31, 2018 4:22 PM
> To: dev@dpdk.org
> Subject: [dpdk-dev] Question about jumbo frame support on ixgbe
> 
> Hi,
> 
> I have a very basic question about jumbo frame support for ixgbe.
> 
> I understand that some drivers support jumbo frame and if it receive jumbo
> packet (greater than 1500 byte), it creates mbuf chains and pass it to DPDK
> application through e.g. rte_eth_rx_burst.
> 
> However it looks that ixgbe driver does not support jumbo frame.
> 
> Q1. Is my understanding above correct?
> Q2. If A1 equals YES, then are there any future plan to support jumbo frame
> on ixgbe?

Your understanding above correct, but 82599 and x550 has support jumbo frame receive by now!
In order to use this feature on ixgbe, you need do the following steps:

1. you must set dev_conf.rxmode. max_rx_pkt_len to a big number, eg. 9500, when doing port start, for example when start port in API rte_eth_dev_start().
ixgbe_dev_rx_init() will chose a scatter receive function if the max_rx_pkt_len is larger than the mbuf size, you do not need to set DEV_RX_OFFLOAD_SCATTER bit in 
dev_conf.rxmode.offloads, this is the work of PMD driver when it detect jumbo frame is needed to be supported.

2. set dev_conf.txmode.offloads bit of  DEV_TX_OFFLOAD_MULTI_SEGS to 1 when doing rte_eth_tx_queue_setup() or , this is very important!! 
If you not do this, you may only receive JUMBO frame but not forward out. Because as you say,
the receive packets maybe has a mbuf chains(depending on the size relationship of mbuf size and the max_rx_pkt_len).
But in ixgbe PMD for setting tx function, it is confusing  in ixgbe_set_tx_function(), you need  to take care of it, it is based on queue offloads bit!

3. enable it using CLI "port config mtu <port_id> <value>" if you are using testpmd or using API rte_eth_dev_set_mtu() for your own APP.
The mtu number is just what you need to update for a large number.

And all my discussion is based on pf port, if you are using vf, we can have a further discussion.
Please feel free to contact me if necessary.

> 
> BR,
> Hideyuki Yamashita
> NTT TechnoCross

Re: [dpdk-dev] Question about jumbo frame support on ixgbe

[Zhao1, Wei]

Add Stephen Hemminger <stephen@networkplumber.org> for discussion.

> -----Original Message-----
> From: Zhao1, Wei
> Sent: Thursday, November 1, 2018 11:10 AM
> To: 'Hideyuki Yamashita' <yamashita.hideyuki@po.ntt-tx.co.jp>;
> dev@dpdk.org
> Subject: RE: [dpdk-dev] Question about jumbo frame support on ixgbe
> 
> Hi,  Hideyuki Yamashita
> 
> 
> > -----Original Message-----
> > From: dev [mailto:dev-bounces@dpdk.org] On Behalf Of Hideyuki
> > Yamashita
> > Sent: Wednesday, October 31, 2018 4:22 PM
> > To: dev@dpdk.org
> > Subject: [dpdk-dev] Question about jumbo frame support on ixgbe
> >
> > Hi,
> >
> > I have a very basic question about jumbo frame support for ixgbe.
> >
> > I understand that some drivers support jumbo frame and if it receive
> > jumbo packet (greater than 1500 byte), it creates mbuf chains and pass
> > it to DPDK application through e.g. rte_eth_rx_burst.
> >
> > However it looks that ixgbe driver does not support jumbo frame.
> >
> > Q1. Is my understanding above correct?
> > Q2. If A1 equals YES, then are there any future plan to support jumbo
> > frame on ixgbe?
> 
> Your understanding above correct, but 82599 and x550 has support jumbo
> frame receive by now!
> In order to use this feature on ixgbe, you need do the following steps:
> 
> 1. you must set dev_conf.rxmode. max_rx_pkt_len to a big number, eg. 9500,
> when doing port start, for example when start port in API
> rte_eth_dev_start().
> ixgbe_dev_rx_init() will chose a scatter receive function if the
> max_rx_pkt_len is larger than the mbuf size, you do not need to set
> DEV_RX_OFFLOAD_SCATTER bit in dev_conf.rxmode.offloads, this is the
> work of PMD driver when it detect jumbo frame is needed to be supported.
> 
> 2. set dev_conf.txmode.offloads bit of  DEV_TX_OFFLOAD_MULTI_SEGS to 1
> when doing rte_eth_tx_queue_setup() or , this is very important!!
> If you not do this, you may only receive JUMBO frame but not forward out.
> Because as you say, the receive packets maybe has a mbuf chains(depending
> on the size relationship of mbuf size and the max_rx_pkt_len).
> But in ixgbe PMD for setting tx function, it is confusing  in
> ixgbe_set_tx_function(), you need  to take care of it, it is based on queue
> offloads bit!
> 
> 3. enable it using CLI "port config mtu <port_id> <value>" if you are using
> testpmd or using API rte_eth_dev_set_mtu() for your own APP.
> The mtu number is just what you need to update for a large number.
> 
> And all my discussion is based on pf port, if you are using vf, we can have a
> further discussion.
> Please feel free to contact me if necessary.
> 
> >
> > BR,
> > Hideyuki Yamashita
> > NTT TechnoCross

Re: [dpdk-dev] Question about jumbo frame support on ixgbe

[Zhao1, Wei]

> -----Original Message-----
> From: dev [mailto:dev-bounces@dpdk.org] On Behalf Of Stephen
> Hemminger
> Sent: Wednesday, October 31, 2018 11:49 PM
> To: Hideyuki Yamashita <yamashita.hideyuki@po.ntt-tx.co.jp>
> Cc: dev@dpdk.org
> Subject: Re: [dpdk-dev] Question about jumbo frame support on ixgbe
> 
> On Wed, 31 Oct 2018 17:22:02 +0900
> Hideyuki Yamashita <yamashita.hideyuki@po.ntt-tx.co.jp> wrote:
> 
> > Hi,
> >
> > I have a very basic question about jumbo frame support for ixgbe.
> >
> > I understand that some drivers support jumbo frame and if it receive
> > jumbo packet (greater than 1500 byte), it creates mbuf chains and pass
> > it to DPDK application through e.g. rte_eth_rx_burst.
> >
> > However it looks that ixgbe driver does not support jumbo frame.
> >
> > Q1. Is my understanding above correct?
> > Q2. If A1 equals YES, then are there any future plan to support jumbo
> > frame on ixgbe?
> >
> > BR,
> > Hideyuki Yamashita
> > NTT TechnoCross
> >
> 
> I don't work for Intel, and haven't tried jumbo frames on ixgbe.
> The hardware does support jumbo frames.
> You might be confusing the overlapping receive offload values.
> 
> The setting for using jumbo frames is in current DPDK is shown as available in
> device as:
> 	rx_offload_capa & DEV_RX_OFFLOAD_JUMBO_FRAME In order to
> use it, this bit must be setn in rx_conf that is passed to
> rte_eth_rx_queue_setup
> 

Yes, 
dev_conf.rxmode.offloads bits of DEV_RX_OFFLOAD_JUMBO_FRAME should be set to 1 when start port in API rte_eth_dev_start().
IF you are using testpmd  APP, you should set struct rte_eth_rxmode rx_mode  offloads bits of DEV_RX_OFFLOAD_JUMBO_FRAME.
 but  I think only set this bit of DEV_RX_OFFLOAD_JUMBO_FRAME is not sufficient for enable this jumbo frame feature,
Because in PMD code, this flag do not set bit of  IXGBE_HLREG0_JUMBOEN in register IXGBE_HLREG0, I think this is something
We need update.

> 
> The feature bit that creates multiple mbuf's if necessary on receive is:
> 	rx_offload_capa  & DEV_RX_OFFLOAD_SCATTER Likewise to enable it,
> this must be set in rte_eth_rx_queue_setup rx_conf.
> 
> 

Yes, this step can chose scatter rx function for PMD, but not MUST, details of discussion for this is in other mail.
https://mails.dpdk.org/archives/dev/2018-November/117771.html

 

Re: [dpdk-dev] Question about jumbo frame support on ixgbe

[Zhao1, Wei]

Hi,  Stephen Hemminger

> -----Original Message-----
> From: Zhao1, Wei
> Sent: Thursday, November 1, 2018 11:27 AM
> To: 'Stephen Hemminger' <stephen@networkplumber.org>; Hideyuki
> Yamashita <yamashita.hideyuki@po.ntt-tx.co.jp>
> Cc: dev@dpdk.org
> Subject: RE: [dpdk-dev] Question about jumbo frame support on ixgbe
> 
> 
> 
> > -----Original Message-----
> > From: dev [mailto:dev-bounces@dpdk.org] On Behalf Of Stephen
> Hemminger
> > Sent: Wednesday, October 31, 2018 11:49 PM
> > To: Hideyuki Yamashita <yamashita.hideyuki@po.ntt-tx.co.jp>
> > Cc: dev@dpdk.org
> > Subject: Re: [dpdk-dev] Question about jumbo frame support on ixgbe
> >
> > On Wed, 31 Oct 2018 17:22:02 +0900
> > Hideyuki Yamashita <yamashita.hideyuki@po.ntt-tx.co.jp> wrote:
> >
> > > Hi,
> > >
> > > I have a very basic question about jumbo frame support for ixgbe.
> > >
> > > I understand that some drivers support jumbo frame and if it receive
> > > jumbo packet (greater than 1500 byte), it creates mbuf chains and
> > > pass it to DPDK application through e.g. rte_eth_rx_burst.
> > >
> > > However it looks that ixgbe driver does not support jumbo frame.
> > >
> > > Q1. Is my understanding above correct?
> > > Q2. If A1 equals YES, then are there any future plan to support
> > > jumbo frame on ixgbe?
> > >
> > > BR,
> > > Hideyuki Yamashita
> > > NTT TechnoCross
> > >
> >
> > I don't work for Intel, and haven't tried jumbo frames on ixgbe.
> > The hardware does support jumbo frames.
> > You might be confusing the overlapping receive offload values.
> >
> > The setting for using jumbo frames is in current DPDK is shown as
> > available in device as:
> > 	rx_offload_capa & DEV_RX_OFFLOAD_JUMBO_FRAME In order to
> use it, this
> > bit must be setn in rx_conf that is passed to rte_eth_rx_queue_setup
> >
> 
> Yes,
> dev_conf.rxmode.offloads bits of DEV_RX_OFFLOAD_JUMBO_FRAME
> should be set to 1 when start port in API rte_eth_dev_start().
> IF you are using testpmd  APP, you should set struct rte_eth_rxmode
> rx_mode  offloads bits of DEV_RX_OFFLOAD_JUMBO_FRAME.
>  but  I think only set this bit of DEV_RX_OFFLOAD_JUMBO_FRAME is not
> sufficient for enable this jumbo frame feature, Because in PMD code, this
> flag do not set bit of  IXGBE_HLREG0_JUMBOEN in register IXGBE_HLREG0, I
> think this is something We need update.

Correct some content above, set this bit of DEV_RX_OFFLOAD_JUMBO_FRAME can 
result in set bit of  IXGBE_HLREG0_JUMBOEN in register IXGBE_HLREG0,  but it still need some check of 
packet size boundary for code safe!

> 
> >
> > The feature bit that creates multiple mbuf's if necessary on receive is:
> > 	rx_offload_capa  & DEV_RX_OFFLOAD_SCATTER Likewise to enable it,
> this
> > must be set in rte_eth_rx_queue_setup rx_conf.
> >
> >
> 
> Yes, this step can chose scatter rx function for PMD, but not MUST, details of
> discussion for this is in other mail.
> https://mails.dpdk.org/archives/dev/2018-November/117771.html

After a double check of code, I am sure DEV_RX_OFFLOAD_SCATTER is not a must choice, it is depend on 
the relationship of max_rx_pkt_len and mbuf size, for example you set  max_rx_pkt_len to 9.5k and mbuf size to 2048,
of course we need to using scatter receiving mode.

> 
> 
> 

Re: [dpdk-dev] Question about jumbo frame support on ixgbe

[Hideyuki Yamashita]

Hi 

Thanks for your answering to my question.
Please see inline.
> Hi,  Hideyuki Yamashita
> 
> 
> > -----Original Message-----
> > From: dev [mailto:dev-bounces@dpdk.org] On Behalf Of Hideyuki Yamashita
> > Sent: Wednesday, October 31, 2018 4:22 PM
> > To: dev@dpdk.org
> > Subject: [dpdk-dev] Question about jumbo frame support on ixgbe
> > 
> > Hi,
> > 
> > I have a very basic question about jumbo frame support for ixgbe.
> > 
> > I understand that some drivers support jumbo frame and if it receive jumbo
> > packet (greater than 1500 byte), it creates mbuf chains and pass it to DPDK
> > application through e.g. rte_eth_rx_burst.
> > 
> > However it looks that ixgbe driver does not support jumbo frame.
> > 
> > Q1. Is my understanding above correct?
> > Q2. If A1 equals YES, then are there any future plan to support jumbo frame
> > on ixgbe?
> 
> Your understanding above correct, but 82599 and x550 has support jumbo frame receive by now!
> In order to use this feature on ixgbe, you need do the following steps:
> 
> 1. you must set dev_conf.rxmode. max_rx_pkt_len to a big number, eg. 9500, when doing port start, for example when start port in API rte_eth_dev_start().
> ixgbe_dev_rx_init() will chose a scatter receive function if the max_rx_pkt_len is larger than the mbuf size, you do not need to set DEV_RX_OFFLOAD_SCATTER bit in 
> dev_conf.rxmode.offloads, this is the work of PMD driver when it detect jumbo frame is needed to be supported.
Thanks for your info.

> 2. set dev_conf.txmode.offloads bit of  DEV_TX_OFFLOAD_MULTI_SEGS to 1 when doing rte_eth_tx_queue_setup() or , this is very important!! 
> If you not do this, you may only receive JUMBO frame but not forward out. Because as you say,
> the receive packets maybe has a mbuf chains(depending on the size relationship of mbuf size and the max_rx_pkt_len).
> But in ixgbe PMD for setting tx function, it is confusing  in ixgbe_set_tx_function(), you need  to take care of it, it is based on queue offloads bit!
What will happen if DEV_TX_OFFLOAD_MULTI_SEGS is set to 1.
Packets are sent fragmented or re-built as a Jumbo frame and sent to
network?
(My guess is former(packet will be sent fragmented though))

> 3. enable it using CLI "port config mtu <port_id> <value>" if you are using testpmd or using API rte_eth_dev_set_mtu() for your own APP.
> The mtu number is just what you need to update for a large number.
I want to know the relationship between 1,2 and 3.
Do I have to do all 3 steps to send/receive jumbo frame? 
Or, when I relalize jumbo frame support programatically execute 1 and 2,
if I do not modify program and just change setting via CLI then execute
3?

Thanks and BR,
Hideyuki Yamashita
NTT TechnoCross
 
> And all my discussion is based on pf port, if you are using vf, we can have a further discussion.
> Please feel free to contact me if necessary.
> 
> > 
> > BR,
> > Hideyuki Yamashita
> > NTT TechnoCross
> 

Re: [dpdk-dev] Question about jumbo frame support on ixgbe

[Zhao1, Wei]

Hi，Hideyuki Yamashita

> -----Original Message-----
> From: Hideyuki Yamashita [mailto:yamashita.hideyuki@po.ntt-tx.co.jp]
> Sent: Friday, November 2, 2018 9:38 AM
> To: Zhao1, Wei <wei.zhao1@intel.com>
> Cc: dev@dpdk.org
> Subject: Re: [dpdk-dev] Question about jumbo frame support on ixgbe
> 
> Hi
> 
> Thanks for your answering to my question.
> Please see inline.
> > Hi,  Hideyuki Yamashita
> >
> >
> > > -----Original Message-----
> > > From: dev [mailto:dev-bounces@dpdk.org] On Behalf Of Hideyuki
> > > Yamashita
> > > Sent: Wednesday, October 31, 2018 4:22 PM
> > > To: dev@dpdk.org
> > > Subject: [dpdk-dev] Question about jumbo frame support on ixgbe
> > >
> > > Hi,
> > >
> > > I have a very basic question about jumbo frame support for ixgbe.
> > >
> > > I understand that some drivers support jumbo frame and if it receive
> > > jumbo packet (greater than 1500 byte), it creates mbuf chains and
> > > pass it to DPDK application through e.g. rte_eth_rx_burst.
> > >
> > > However it looks that ixgbe driver does not support jumbo frame.
> > >
> > > Q1. Is my understanding above correct?
> > > Q2. If A1 equals YES, then are there any future plan to support
> > > jumbo frame on ixgbe?
> >
> > Your understanding above correct, but 82599 and x550 has support jumbo
> frame receive by now!
> > In order to use this feature on ixgbe, you need do the following steps:
> >
> > 1. you must set dev_conf.rxmode. max_rx_pkt_len to a big number, eg.
> 9500, when doing port start, for example when start port in API
> rte_eth_dev_start().
> > ixgbe_dev_rx_init() will chose a scatter receive function if the
> > max_rx_pkt_len is larger than the mbuf size, you do not need to set
> DEV_RX_OFFLOAD_SCATTER bit in dev_conf.rxmode.offloads, this is the
> work of PMD driver when it detect jumbo frame is needed to be supported.
> Thanks for your info.
> 
> > 2. set dev_conf.txmode.offloads bit of  DEV_TX_OFFLOAD_MULTI_SEGS to
> 1 when doing rte_eth_tx_queue_setup() or , this is very important!!
> > If you not do this, you may only receive JUMBO frame but not forward
> > out. Because as you say, the receive packets maybe has a mbuf
> chains(depending on the size relationship of mbuf size and the
> max_rx_pkt_len).
> > But in ixgbe PMD for setting tx function, it is confusing  in
> ixgbe_set_tx_function(), you need  to take care of it, it is based on queue
> offloads bit!
> What will happen if DEV_TX_OFFLOAD_MULTI_SEGS is set to 1.
> Packets are sent fragmented or re-built as a Jumbo frame and sent to
> network?
> (My guess is former(packet will be sent fragmented though))

Ixgbe will store the jumbo frame into several segments in mbuf when receiving from network,
It will not be rebuild into one jumbo packets in PMD, so ixgbe will do that work when transmitting it.
PMD need the flag for indication of this packet is Jumbo and do specific work,  DEV_TX_OFFLOAD_MULTI_SEGS
Can tell PMD chose a purpose tx function for doing these work.
 
> 
> > 3. enable it using CLI "port config mtu <port_id> <value>" if you are using
> testpmd or using API rte_eth_dev_set_mtu() for your own APP.
> > The mtu number is just what you need to update for a large number.
> I want to know the relationship between 1,2 and 3.
> Do I have to do all 3 steps to send/receive jumbo frame?
> Or, when I relalize jumbo frame support programatically execute 1 and 2, if I
> do not modify program and just change setting via CLI then execute 3?

If you are use testpmd APP, you can excute (3). If not, you can set dev_conf.rxmode.offloads bits of DEV_RX_OFFLOAD_JUMBO_FRAME 
to 1 when you start ixgbe PMD code. 1-2-3 are all steps for configuration of registers for JUMBO frame enable.


> 
> Thanks and BR,
> Hideyuki Yamashita
> NTT TechnoCross
> 
> > And all my discussion is based on pf port, if you are using vf, we can have a
> further discussion.
> > Please feel free to contact me if necessary.
> >
> > >
> > > BR,
> > > Hideyuki Yamashita
> > > NTT TechnoCross
> >
> 
> 

Re: [dpdk-dev] [PATCH] Doc: add known issue for restricted vdev ethdev ops in secondary process

[Mcnamara, John]

> -----Original Message-----
> From: dev [mailto:dev-bounces@dpdk.org] On Behalf Of Marvin Liu
> Sent: Wednesday, October 31, 2018 2:49 PM
> To: thomas@monjalon.net
> Cc: dev@dpdk.org; Liu, Yong <yong.liu@intel.com>
> Subject: [dpdk-dev] [PATCH] Doc: add known issue for restricted vdev
> ethdev ops in secondary process
> 


Acked-by: John McNamara <john.mcnamara@intel.com>

Re: [dpdk-dev] [PATCH] Doc: add known issue for restricted vdev ethdev ops in secondary process

[Thomas Monjalon]

> > From: Marvin Liu
> 
> Acked-by: John McNamara <john.mcnamara@intel.com>

Applied, thanks