[dpdk-users] rte_eth_stats counters

[dpdk-users] rte_eth_stats counters

[Gerry Wan]

Hi,

I'm testing out the maximum RX throughput my application can handle and am
using rte_eth_stats_get() to measure the point at which RX packets start
getting dropped. I have a traffic generator directly connected to my RX
port.

I've noticed that imissed is the number of packets that are dropped due to
hardware queue fulls, while ipackets is the number of packets successfully
received and agrees with the total number of packets retrieved from calls
to rte_eth_rx_burst(). I'm not sure exactly what ierrors is supposed to
count, but so far I have not seen this go beyond 0?

I have been interpreting the sum of (ipackets + imissed + ierrors) = itotal
as the total number of packets hitting the port. However, I've noticed that
when throughput gets too high, imissed will remain 0 while itotal is
smaller than the number of packets sent by the traffic generator. I've
ruled out connection issues because increasing the number of RSS queues
seems to fix the problem (up to a certain threshold before itotal again
becomes smaller than the number sent), but I don't understand why. If it is
not dropped in HW because the queues are full (since imissed = 0), where
are the packets being dropped and is there a way I can count these?

I am using DPDK 20.08 with a Mellanox CX-5, RSS queue size = 4096

Thanks

Re: [dpdk-users] rte_eth_stats counters

[David Christensen]

On 9/16/20 8:42 PM, Gerry Wan wrote:
> Hi,
> 
> I'm testing out the maximum RX throughput my application can handle and am
> using rte_eth_stats_get() to measure the point at which RX packets start
> getting dropped. I have a traffic generator directly connected to my RX
> port.
> 
> I've noticed that imissed is the number of packets that are dropped due to
> hardware queue fulls, while ipackets is the number of packets successfully
> received and agrees with the total number of packets retrieved from calls
> to rte_eth_rx_burst(). I'm not sure exactly what ierrors is supposed to
> count, but so far I have not seen this go beyond 0?
> 
> I have been interpreting the sum of (ipackets + imissed + ierrors) = itotal
> as the total number of packets hitting the port. However, I've noticed that
> when throughput gets too high, imissed will remain 0 while itotal is
> smaller than the number of packets sent by the traffic generator. I've
> ruled out connection issues because increasing the number of RSS queues
> seems to fix the problem (up to a certain threshold before itotal again
> becomes smaller than the number sent), but I don't understand why. If it is
> not dropped in HW because the queues are full (since imissed = 0), where
> are the packets being dropped and is there a way I can count these?
> 
> I am using DPDK 20.08 with a Mellanox CX-5, RSS queue size = 4096

When app buffers fill up then the HW buffers start to fill up.  When HW 
buffers are full then the PHY responds by generating flow control frames 
or simply dropping packets.  You could experiment by enabling/disabling 
flow control to verify that the packet counts are correct when flow 
control is enabled.

You could also look at the rx_discards_phy counter and contrast it with 
the rx_out_of_buffer statistic:

https://community.mellanox.com/s/article/understanding-mlx5-ethtool-counters

My read is that rx_out_of_buffer indicates that the HW doesn't have any 
RX descriptors available, possibly because of PCIe congestion or because 
the app's receive queue is empty.  On the other hand, rx_discards_phy 
indicates that the HW buffers are full.  I don't see the rx_discards_phy 
used in any stats, only available as an xstat.

Dave