[dpdk-dev] rte_eth_bond 8023ad dedicated queues with i40e with vectorized rx does not work

[dpdk-dev] rte_eth_bond 8023ad dedicated queues with i40e with vectorized rx does not work

[Kyle Larose]

Hello,

I've been doing some testing using the 8023ad link bonding driver on a system with 4 10G i40e interfaces in the link bond. It's working fine, except that when any of the links are overloaded, it starts dropping the LACPDUs, which is rather unfortunate for many reasons.

While thinking about that problem, I noticed that the driver provides the ability to allocate dedicated queues for rx and tx of LACPDUs. This is great! Solves my problem (sort of - I'll send another email about that later)... Or so I thought. After enabling the dedicated queues, I noticed  a few things:
	1. The link bond never started distributing
	2. The slave interfaces started dropping frames on their dedicated control queues after some time
	3. The connected interfaces reported both sending and receiving LACP PDUs. 

After digging in to this, I found out that the call to rte_eth_rx_burst was returning 0 packets, despite their being many in the queue. It turns out that the i40e was using one of the vectorized rx_burst functions, which require that the user poll for more than 1 packet at a time. bond_mode_8023ad_periodic_cb was polling for exactly one.

I changed the code to read up to 16 at a time, and everything started working. I'm not sure this is the right fix, though, since the normal behaviour of processing one packet at a time maintains some hold offs/etc that may be nice, and I don't want to discard any packets past the first one.

Does anyone have some thoughts/comments on this? I can submit a patch with my current workaround, if desired.

Thanks,

Kyle 

Re: [dpdk-dev] rte_eth_bond 8023ad dedicated queues with i40e with vectorized rx does not work

[Doherty, Declan]

On 08/11/2017 7:21 PM, Kyle Larose wrote:
> Hello,
> 
> I've been doing some testing using the 8023ad link bonding driver on a system with 4 10G i40e interfaces in the link bond. It's working fine, except that when any of the links are overloaded, it starts dropping the LACPDUs, which is rather unfortunate for many reasons.
> 
> While thinking about that problem, I noticed that the driver provides the ability to allocate dedicated queues for rx and tx of LACPDUs. This is great! Solves my problem (sort of - I'll send another email about that later)... Or so I thought. After enabling the dedicated queues, I noticed  a few things:
> 	1. The link bond never started distributing
> 	2. The slave interfaces started dropping frames on their dedicated control queues after some time
> 	3. The connected interfaces reported both sending and receiving LACP PDUs.
> 
> After digging in to this, I found out that the call to rte_eth_rx_burst was returning 0 packets, despite their being many in the queue. It turns out that the i40e was using one of the vectorized rx_burst functions, which require that the user poll for more than 1 packet at a time. bond_mode_8023ad_periodic_cb was polling for exactly one.
> 
> I changed the code to read up to 16 at a time, and everything started working. I'm not sure this is the right fix, though, since the normal behaviour of processing one packet at a time maintains some hold offs/etc that may be nice, and I don't want to discard any packets past the first one.
> 
> Does anyone have some thoughts/comments on this? I can submit a patch with my current workaround, if desired.
> 
> Thanks,
> 
> Kyle
> 

Hey Kyle,

I think this fix is fine, as is possible that there would be more than 
one packet on the queue, with both control packets and marker packets 
possible.

Please send a patch so we can review and try out.

Thanks
Declan