[dpdk-dev] can eth_igb_xmit_pkts called with len 0 affect transmission?

[dpdk-dev] can eth_igb_xmit_pkts called with len 0 affect transmission?

[ciprian.barbu]

Hi,

I'm seeing a strange behavior when calling rte_eth_tx_burst with len == 
0. I'll explain the reason for this situation further bellow. But what 
I'm seeing is that after doing this call my application keeps returning 
from eth_igb_xmit_pkts here, even when len > 0: 
http://dpdk.org/browse/dpdk/tree/lib/librte_pmd_e1000/igb_rxtx.c?h=releases#n476

I can't really explain myself this behavior, I don't understand very 
well what the nic does once it receives buffers in its rings, but to me 
it looks like calling rte_eth_tx_burst with len 0 has this effect.

What I'm using in my tests is the linaro odp-dpdk implementation and the 
odp_l2fwd example. The odp-dpdk implementation makes this call to try 
and make the pmd to flush the tx queue in case there are no more free 
buffers in the pool, but this was only verified for ixgbe 82599 cards, 
for igb the packets are not actually flushed until the tail circles back 
the whole length of the queue. I'm pretty much the only one (that I know 
of) that uses odp with 1G igb i350 cards and see this issue.

Can anyone explain whether I'm getting this right and there could be 
side effects to calling eth_igb_xmit_pkts with len 0?

Thank you,
/Ciprian

Re: [dpdk-dev] can eth_igb_xmit_pkts called with len 0 affect transmission?

[ciprian.barbu]

Anyone?

On 17.07.2015 16:48, ciprian.barbu wrote:
> Hi,
>
> I'm seeing a strange behavior when calling rte_eth_tx_burst with len ==
> 0. I'll explain the reason for this situation further bellow. But what
> I'm seeing is that after doing this call my application keeps returning
> from eth_igb_xmit_pkts here, even when len > 0:
> http://dpdk.org/browse/dpdk/tree/lib/librte_pmd_e1000/igb_rxtx.c?h=releases#n476
>
>
> I can't really explain myself this behavior, I don't understand very
> well what the nic does once it receives buffers in its rings, but to me
> it looks like calling rte_eth_tx_burst with len 0 has this effect.
>
> What I'm using in my tests is the linaro odp-dpdk implementation and the
> odp_l2fwd example. The odp-dpdk implementation makes this call to try
> and make the pmd to flush the tx queue in case there are no more free
> buffers in the pool, but this was only verified for ixgbe 82599 cards,
> for igb the packets are not actually flushed until the tail circles back
> the whole length of the queue. I'm pretty much the only one (that I know
> of) that uses odp with 1G igb i350 cards and see this issue.
>
> Can anyone explain whether I'm getting this right and there could be
> side effects to calling eth_igb_xmit_pkts with len 0?
>
> Thank you,
> /Ciprian

Re: [dpdk-dev] can eth_igb_xmit_pkts called with len 0 affect transmission?

[Lu, Wenzhuo]

Hi Ciprian,

> -----Original Message-----
> From: dev [mailto:dev-bounces@dpdk.org] On Behalf Of ciprian.barbu
> Sent: Thursday, July 23, 2015 11:20 PM
> To: dev@dpdk.org
> Subject: Re: [dpdk-dev] can eth_igb_xmit_pkts called with len 0 affect
> transmission?
> 
> Anyone?
> 
> On 17.07.2015 16:48, ciprian.barbu wrote:
> > Hi,
> >
> > I'm seeing a strange behavior when calling rte_eth_tx_burst with len
> > == 0. I'll explain the reason for this situation further bellow. But
> > what I'm seeing is that after doing this call my application keeps
> > returning from eth_igb_xmit_pkts here, even when len > 0:
> > http://dpdk.org/browse/dpdk/tree/lib/librte_pmd_e1000/igb_rxtx.c?h=rel
> > eases#n476
Don’t understand what the len means? The packet length? 
According to where the code returns, seems the queue is not initialized correctly.
I saw you used odp-dpdk. Sorry, I'm not familiar with it. Why not try dpdk examples,
and check if the behavior is same?

> >
> >
> > I can't really explain myself this behavior, I don't understand very
> > well what the nic does once it receives buffers in its rings, but to
> > me it looks like calling rte_eth_tx_burst with len 0 has this effect.
> >
> > What I'm using in my tests is the linaro odp-dpdk implementation and
> > the odp_l2fwd example. The odp-dpdk implementation makes this call to
> > try and make the pmd to flush the tx queue in case there are no more
> > free buffers in the pool, but this was only verified for ixgbe 82599
> > cards, for igb the packets are not actually flushed until the tail
> > circles back the whole length of the queue. I'm pretty much the only
> > one (that I know
> > of) that uses odp with 1G igb i350 cards and see this issue.
> >
> > Can anyone explain whether I'm getting this right and there could be
> > side effects to calling eth_igb_xmit_pkts with len 0?
> >
> > Thank you,
> > /Ciprian

Re: [dpdk-dev] can eth_igb_xmit_pkts called with len 0 affect transmission?

[ciprian.barbu]

Hi Lu,

Thank you for getting back to me.

On 24.07.2015 05:49, Lu, Wenzhuo wrote:
> Hi Ciprian,
>
>> -----Original Message-----
>> From: dev [mailto:dev-bounces@dpdk.org] On Behalf Of ciprian.barbu
>> Sent: Thursday, July 23, 2015 11:20 PM
>> To: dev@dpdk.org
>> Subject: Re: [dpdk-dev] can eth_igb_xmit_pkts called with len 0 affect
>> transmission?
>>
>> Anyone?
>>
>> On 17.07.2015 16:48, ciprian.barbu wrote:
>>> Hi,
>>>
>>> I'm seeing a strange behavior when calling rte_eth_tx_burst with len
>>> == 0. I'll explain the reason for this situation further bellow. But
>>> what I'm seeing is that after doing this call my application keeps
>>> returning from eth_igb_xmit_pkts here, even when len > 0:
>>> http://dpdk.org/browse/dpdk/tree/lib/librte_pmd_e1000/igb_rxtx.c?h=rel
>>> eases#n476
> Don’t understand what the len means? The packet length?

Sorry about this, failure to communicate :) I meant to say nb_pkts = 0

> According to where the code returns, seems the queue is not initialized correctly.

The queue should have been initialized ok, I'm able to several millions 
of packets before getting stuck. It's only after I make a 
rte_eth_tx_burst call with nb_pkts = 0 that I get stuck in that return. 
Looking at the source code, the function will only do:

         rte_wmb();
         E1000_PCI_REG_WRITE(txq->tdt_reg_addr, tx_id);

The barrier should be harmless, so I'm thinking that writing that 
register without sending any packets causes this behavior.


> I saw you used odp-dpdk. Sorry, I'm not familiar with it. Why not try dpdk examples,
> and check if the behavior is same?

Of course, I tried the DPDK l2fwd example, that's how I managed to find 
the difference between odp_l2fwd and DPDK l2fwd. After I removed the 
piece of code that was calling rte_eth_tx_burst with nb_pkts 0 it 
started behaving the same as DPDK l2fwd.

I suppose I could also modify l2fwd to make this call and see it it gets 
stuck too.

>
>>>
>>>
>>> I can't really explain myself this behavior, I don't understand very
>>> well what the nic does once it receives buffers in its rings, but to
>>> me it looks like calling rte_eth_tx_burst with len 0 has this effect.
>>>
>>> What I'm using in my tests is the linaro odp-dpdk implementation and
>>> the odp_l2fwd example. The odp-dpdk implementation makes this call to
>>> try and make the pmd to flush the tx queue in case there are no more
>>> free buffers in the pool, but this was only verified for ixgbe 82599
>>> cards, for igb the packets are not actually flushed until the tail
>>> circles back the whole length of the queue. I'm pretty much the only
>>> one (that I know
>>> of) that uses odp with 1G igb i350 cards and see this issue.
>>>
>>> Can anyone explain whether I'm getting this right and there could be
>>> side effects to calling eth_igb_xmit_pkts with len 0?
>>>
>>> Thank you,
>>> /Ciprian