[dpdk-dev] Regarding rte_eth_rx_burst

[dpdk-dev] Regarding rte_eth_rx_burst

[Prashant Upadhyaya]

Hi,

I recently started using X722 NIC which uses i40 PMD of DPDK.
I am on DPDK 20.02.
I am seeing that when I call the rte_eth_rx_burst with last argument
as 1 (polling for 1 mbuf), then I am not receiving data via repeated
calls.
When I go for calls to rte_eth_rx_burst with last argument as 32, the
function does return the mbuf's as received data.

Is this expected ? Or this is a bug in the i40 driver handling this NIC ?

The polls to rte_eth_rx_burst with last argument as 1 works well for
the ixgbe PMD for sure since I have been using X520 successfully with
last argument as 1.

Regards
-Prashant

Re: [dpdk-dev] Regarding rte_eth_rx_burst

[Wang, Haiyue]

> -----Original Message-----
> From: dev <dev-bounces@dpdk.org> On Behalf Of Prashant Upadhyaya
> Sent: Sunday, May 10, 2020 22:06
> To: dev@dpdk.org
> Subject: [dpdk-dev] Regarding rte_eth_rx_burst
> 
> Hi,
> 
> I recently started using X722 NIC which uses i40 PMD of DPDK.
> I am on DPDK 20.02.
> I am seeing that when I call the rte_eth_rx_burst with last argument
> as 1 (polling for 1 mbuf), then I am not receiving data via repeated
> calls.

I saw this kind of issue many times. ;-)

This is *burst* for vector mode:

drivers/net/i40e/i40e_rxtx_vec_sse.c

static inline uint16_t
_recv_raw_pkts_vec(struct i40e_rx_queue *rxq, struct rte_mbuf **rx_pkts,
		   uint16_t nb_pkts, uint8_t *split_packet)
{
      ....

	/* nb_pkts shall be less equal than RTE_I40E_MAX_RX_BURST */
	nb_pkts = RTE_MIN(nb_pkts, RTE_I40E_MAX_RX_BURST);

	/* nb_pkts has to be floor-aligned to RTE_I40E_DESCS_PER_LOOP */
	nb_pkts = RTE_ALIGN_FLOOR(nb_pkts, RTE_I40E_DESCS_PER_LOOP);   <--- nb_pkts = 0, if you passed 1.


> When I go for calls to rte_eth_rx_burst with last argument as 32, the
> function does return the mbuf's as received data.
> 
> Is this expected ? Or this is a bug in the i40 driver handling this NIC ?
> 
> The polls to rte_eth_rx_burst with last argument as 1 works well for
> the ixgbe PMD for sure since I have been using X520 successfully with
> last argument as 1.
> 
> Regards
> -Prashant

Re: [dpdk-dev] Regarding rte_eth_rx_burst

[Prashant Upadhyaya]

Thanks Wang !
...which begs the proverbial question -- is this a bug or a feature ?
I would say it is a bug as the polling for 1 mbuf works for the other
PMD's, worse still the setting to zero is done in a quiet manner
leading to entire rx blockage if the caller keeps calling with polling
for 1 mbuf and keeps wondering why the rx is not working.

Regards
-Prashant


On Sun, May 10, 2020 at 10:46 PM Wang, Haiyue <haiyue.wang@intel.com> wrote:
>
> > -----Original Message-----
> > From: dev <dev-bounces@dpdk.org> On Behalf Of Prashant Upadhyaya
> > Sent: Sunday, May 10, 2020 22:06
> > To: dev@dpdk.org
> > Subject: [dpdk-dev] Regarding rte_eth_rx_burst
> >
> > Hi,
> >
> > I recently started using X722 NIC which uses i40 PMD of DPDK.
> > I am on DPDK 20.02.
> > I am seeing that when I call the rte_eth_rx_burst with last argument
> > as 1 (polling for 1 mbuf), then I am not receiving data via repeated
> > calls.
>
> I saw this kind of issue many times. ;-)
>
> This is *burst* for vector mode:
>
> drivers/net/i40e/i40e_rxtx_vec_sse.c
>
> static inline uint16_t
> _recv_raw_pkts_vec(struct i40e_rx_queue *rxq, struct rte_mbuf **rx_pkts,
>                    uint16_t nb_pkts, uint8_t *split_packet)
> {
>       ....
>
>         /* nb_pkts shall be less equal than RTE_I40E_MAX_RX_BURST */
>         nb_pkts = RTE_MIN(nb_pkts, RTE_I40E_MAX_RX_BURST);
>
>         /* nb_pkts has to be floor-aligned to RTE_I40E_DESCS_PER_LOOP */
>         nb_pkts = RTE_ALIGN_FLOOR(nb_pkts, RTE_I40E_DESCS_PER_LOOP);   <--- nb_pkts = 0, if you passed 1.
>
>
> > When I go for calls to rte_eth_rx_burst with last argument as 32, the
> > function does return the mbuf's as received data.
> >
> > Is this expected ? Or this is a bug in the i40 driver handling this NIC ?
> >
> > The polls to rte_eth_rx_burst with last argument as 1 works well for
> > the ixgbe PMD for sure since I have been using X520 successfully with
> > last argument as 1.
> >
> > Regards
> > -Prashant

Re: [dpdk-dev] Regarding rte_eth_rx_burst

[Wang, Haiyue]

> -----Original Message-----
> From: Prashant Upadhyaya <praupadhyaya@gmail.com>
> Sent: Monday, May 11, 2020 13:58
> To: Wang, Haiyue <haiyue.wang@intel.com>
> Cc: dev@dpdk.org
> Subject: Re: [dpdk-dev] Regarding rte_eth_rx_burst
> 
> Thanks Wang !
> ...which begs the proverbial question -- is this a bug or a feature ?
> I would say it is a bug as the polling for 1 mbuf works for the other
> PMD's, worse still the setting to zero is done in a quiet manner
> leading to entire rx blockage if the caller keeps calling with polling
> for 1 mbuf and keeps wondering why the rx is not working.

Maybe a feature, this is good for vector handling best performance.
A slow Rx ' i40e_recv_pkts' in 'drivers/net/i40e/i40e_rxtx.c' can handle
1 mbuf, detail please refer to 'i40e_set_rx_function'

Why 1 mbuf ? Why not loop one by one each received buffer after
calling rte_eth_rx_burst ?

> 
> Regards
> -Prashant
> 
> 
> On Sun, May 10, 2020 at 10:46 PM Wang, Haiyue <haiyue.wang@intel.com> wrote:
> >
> > > -----Original Message-----
> > > From: dev <dev-bounces@dpdk.org> On Behalf Of Prashant Upadhyaya
> > > Sent: Sunday, May 10, 2020 22:06
> > > To: dev@dpdk.org
> > > Subject: [dpdk-dev] Regarding rte_eth_rx_burst
> > >
> > > Hi,
> > >
> > > I recently started using X722 NIC which uses i40 PMD of DPDK.
> > > I am on DPDK 20.02.
> > > I am seeing that when I call the rte_eth_rx_burst with last argument
> > > as 1 (polling for 1 mbuf), then I am not receiving data via repeated
> > > calls.
> >
> > I saw this kind of issue many times. ;-)
> >
> > This is *burst* for vector mode:
> >
> > drivers/net/i40e/i40e_rxtx_vec_sse.c
> >
> > static inline uint16_t
> > _recv_raw_pkts_vec(struct i40e_rx_queue *rxq, struct rte_mbuf **rx_pkts,
> >                    uint16_t nb_pkts, uint8_t *split_packet)
> > {
> >       ....
> >
> >         /* nb_pkts shall be less equal than RTE_I40E_MAX_RX_BURST */
> >         nb_pkts = RTE_MIN(nb_pkts, RTE_I40E_MAX_RX_BURST);
> >
> >         /* nb_pkts has to be floor-aligned to RTE_I40E_DESCS_PER_LOOP */
> >         nb_pkts = RTE_ALIGN_FLOOR(nb_pkts, RTE_I40E_DESCS_PER_LOOP);   <--- nb_pkts = 0, if you
> passed 1.
> >
> >
> > > When I go for calls to rte_eth_rx_burst with last argument as 32, the
> > > function does return the mbuf's as received data.
> > >
> > > Is this expected ? Or this is a bug in the i40 driver handling this NIC ?
> > >
> > > The polls to rte_eth_rx_burst with last argument as 1 works well for
> > > the ixgbe PMD for sure since I have been using X520 successfully with
> > > last argument as 1.
> > >
> > > Regards
> > > -Prashant

Re: [dpdk-dev] Regarding rte_eth_rx_burst

[Prashant Upadhyaya]

The reason for 1 mbuf is application design, the SDK should not drive
the application design.
Eg. in my case the poll loop is in control of another module which
calls a 'get work' function which expects to be returned a single mbuf
and therefore the simple way was to do poll of 1 mbuf and return it if
it is found or return null to get work function.
Of course there are ways around this.
But I would consider it as a bug because nowhere in the documentation
is it prohibited to do polling of 1 mbuf. And it works for other
PMD's. If it doesn't give the best performance, it should be up to the
application to use the API more efficiently, it should not mean that
the usecase should stop working altogether if somebody polls for 1
mbuf.

Regards
-Prashant


On Mon, May 11, 2020 at 11:58 AM Wang, Haiyue <haiyue.wang@intel.com> wrote:
>
> > -----Original Message-----
> > From: Prashant Upadhyaya <praupadhyaya@gmail.com>
> > Sent: Monday, May 11, 2020 13:58
> > To: Wang, Haiyue <haiyue.wang@intel.com>
> > Cc: dev@dpdk.org
> > Subject: Re: [dpdk-dev] Regarding rte_eth_rx_burst
> >
> > Thanks Wang !
> > ...which begs the proverbial question -- is this a bug or a feature ?
> > I would say it is a bug as the polling for 1 mbuf works for the other
> > PMD's, worse still the setting to zero is done in a quiet manner
> > leading to entire rx blockage if the caller keeps calling with polling
> > for 1 mbuf and keeps wondering why the rx is not working.
>
> Maybe a feature, this is good for vector handling best performance.
> A slow Rx ' i40e_recv_pkts' in 'drivers/net/i40e/i40e_rxtx.c' can handle
> 1 mbuf, detail please refer to 'i40e_set_rx_function'
>
> Why 1 mbuf ? Why not loop one by one each received buffer after
> calling rte_eth_rx_burst ?
>
> >
> > Regards
> > -Prashant
> >
> >
> > On Sun, May 10, 2020 at 10:46 PM Wang, Haiyue <haiyue.wang@intel.com> wrote:
> > >
> > > > -----Original Message-----
> > > > From: dev <dev-bounces@dpdk.org> On Behalf Of Prashant Upadhyaya
> > > > Sent: Sunday, May 10, 2020 22:06
> > > > To: dev@dpdk.org
> > > > Subject: [dpdk-dev] Regarding rte_eth_rx_burst
> > > >
> > > > Hi,
> > > >
> > > > I recently started using X722 NIC which uses i40 PMD of DPDK.
> > > > I am on DPDK 20.02.
> > > > I am seeing that when I call the rte_eth_rx_burst with last argument
> > > > as 1 (polling for 1 mbuf), then I am not receiving data via repeated
> > > > calls.
> > >
> > > I saw this kind of issue many times. ;-)
> > >
> > > This is *burst* for vector mode:
> > >
> > > drivers/net/i40e/i40e_rxtx_vec_sse.c
> > >
> > > static inline uint16_t
> > > _recv_raw_pkts_vec(struct i40e_rx_queue *rxq, struct rte_mbuf **rx_pkts,
> > >                    uint16_t nb_pkts, uint8_t *split_packet)
> > > {
> > >       ....
> > >
> > >         /* nb_pkts shall be less equal than RTE_I40E_MAX_RX_BURST */
> > >         nb_pkts = RTE_MIN(nb_pkts, RTE_I40E_MAX_RX_BURST);
> > >
> > >         /* nb_pkts has to be floor-aligned to RTE_I40E_DESCS_PER_LOOP */
> > >         nb_pkts = RTE_ALIGN_FLOOR(nb_pkts, RTE_I40E_DESCS_PER_LOOP);   <--- nb_pkts = 0, if you
> > passed 1.
> > >
> > >
> > > > When I go for calls to rte_eth_rx_burst with last argument as 32, the
> > > > function does return the mbuf's as received data.
> > > >
> > > > Is this expected ? Or this is a bug in the i40 driver handling this NIC ?
> > > >
> > > > The polls to rte_eth_rx_burst with last argument as 1 works well for
> > > > the ixgbe PMD for sure since I have been using X520 successfully with
> > > > last argument as 1.
> > > >
> > > > Regards
> > > > -Prashant

Re: [dpdk-dev] Regarding rte_eth_rx_burst

[Wang, Haiyue]

+Xiaolong,

> -----Original Message-----
> From: Prashant Upadhyaya <praupadhyaya@gmail.com>
> Sent: Monday, May 11, 2020 16:32
> To: Wang, Haiyue <haiyue.wang@intel.com>
> Cc: dev@dpdk.org
> Subject: Re: [dpdk-dev] Regarding rte_eth_rx_burst
> 
> The reason for 1 mbuf is application design, the SDK should not drive
> the application design.
> Eg. in my case the poll loop is in control of another module which
> calls a 'get work' function which expects to be returned a single mbuf
> and therefore the simple way was to do poll of 1 mbuf and return it if
> it is found or return null to get work function.
> Of course there are ways around this.
> But I would consider it as a bug because nowhere in the documentation
> is it prohibited to do polling of 1 mbuf. 

Yes, a little weird without documentation mentioned. ;-) This is paranoid
for pursuing performance. I added @Xiaolong here to suggest adding some notice
in i40e documentation. This may be good.

Thanks for your kind feedback.


> And it works for other
> PMD's. If it doesn't give the best performance, it should be up to the
> application to use the API more efficiently, it should not mean that
> the usecase should stop working altogether if somebody polls for 1
> mbuf.
> 
> Regards
> -Prashant
> 
> 
> On Mon, May 11, 2020 at 11:58 AM Wang, Haiyue <haiyue.wang@intel.com> wrote:
> >
> > > -----Original Message-----
> > > From: Prashant Upadhyaya <praupadhyaya@gmail.com>
> > > Sent: Monday, May 11, 2020 13:58
> > > To: Wang, Haiyue <haiyue.wang@intel.com>
> > > Cc: dev@dpdk.org
> > > Subject: Re: [dpdk-dev] Regarding rte_eth_rx_burst
> > >
> > > Thanks Wang !
> > > ...which begs the proverbial question -- is this a bug or a feature ?
> > > I would say it is a bug as the polling for 1 mbuf works for the other
> > > PMD's, worse still the setting to zero is done in a quiet manner
> > > leading to entire rx blockage if the caller keeps calling with polling
> > > for 1 mbuf and keeps wondering why the rx is not working.
> >
> > Maybe a feature, this is good for vector handling best performance.
> > A slow Rx ' i40e_recv_pkts' in 'drivers/net/i40e/i40e_rxtx.c' can handle
> > 1 mbuf, detail please refer to 'i40e_set_rx_function'
> >
> > Why 1 mbuf ? Why not loop one by one each received buffer after
> > calling rte_eth_rx_burst ?
> >
> > >
> > > Regards
> > > -Prashant
> > >
> > >
> > > On Sun, May 10, 2020 at 10:46 PM Wang, Haiyue <haiyue.wang@intel.com> wrote:
> > > >
> > > > > -----Original Message-----
> > > > > From: dev <dev-bounces@dpdk.org> On Behalf Of Prashant Upadhyaya
> > > > > Sent: Sunday, May 10, 2020 22:06
> > > > > To: dev@dpdk.org
> > > > > Subject: [dpdk-dev] Regarding rte_eth_rx_burst
> > > > >
> > > > > Hi,
> > > > >
> > > > > I recently started using X722 NIC which uses i40 PMD of DPDK.
> > > > > I am on DPDK 20.02.
> > > > > I am seeing that when I call the rte_eth_rx_burst with last argument
> > > > > as 1 (polling for 1 mbuf), then I am not receiving data via repeated
> > > > > calls.
> > > >
> > > > I saw this kind of issue many times. ;-)
> > > >
> > > > This is *burst* for vector mode:
> > > >
> > > > drivers/net/i40e/i40e_rxtx_vec_sse.c
> > > >
> > > > static inline uint16_t
> > > > _recv_raw_pkts_vec(struct i40e_rx_queue *rxq, struct rte_mbuf **rx_pkts,
> > > >                    uint16_t nb_pkts, uint8_t *split_packet)
> > > > {
> > > >       ....
> > > >
> > > >         /* nb_pkts shall be less equal than RTE_I40E_MAX_RX_BURST */
> > > >         nb_pkts = RTE_MIN(nb_pkts, RTE_I40E_MAX_RX_BURST);
> > > >
> > > >         /* nb_pkts has to be floor-aligned to RTE_I40E_DESCS_PER_LOOP */
> > > >         nb_pkts = RTE_ALIGN_FLOOR(nb_pkts, RTE_I40E_DESCS_PER_LOOP);   <--- nb_pkts = 0, if you
> > > passed 1.
> > > >
> > > >
> > > > > When I go for calls to rte_eth_rx_burst with last argument as 32, the
> > > > > function does return the mbuf's as received data.
> > > > >
> > > > > Is this expected ? Or this is a bug in the i40 driver handling this NIC ?
> > > > >
> > > > > The polls to rte_eth_rx_burst with last argument as 1 works well for
> > > > > the ixgbe PMD for sure since I have been using X520 successfully with
> > > > > last argument as 1.
> > > > >
> > > > > Regards
> > > > > -Prashant