From: David Christensen <drc@linux.vnet.ibm.com>
To: users@dpdk.org
Subject: Re: [dpdk-users] rte_eth_stats counters
Date: Mon, 21 Sep 2020 10:53:49 -0700 [thread overview]
Message-ID: <8ca52ae9-ec04-a12a-82c6-1b61392c1e9a@linux.vnet.ibm.com> (raw)
In-Reply-To: <CAAcwi3-G-hmv6UnpR1=FLKhkJQr2oDH9xw812JAc+A-0hgmR+A@mail.gmail.com>
On 9/16/20 8:42 PM, Gerry Wan wrote:
> Hi,
>
> I'm testing out the maximum RX throughput my application can handle and am
> using rte_eth_stats_get() to measure the point at which RX packets start
> getting dropped. I have a traffic generator directly connected to my RX
> port.
>
> I've noticed that imissed is the number of packets that are dropped due to
> hardware queue fulls, while ipackets is the number of packets successfully
> received and agrees with the total number of packets retrieved from calls
> to rte_eth_rx_burst(). I'm not sure exactly what ierrors is supposed to
> count, but so far I have not seen this go beyond 0?
>
> I have been interpreting the sum of (ipackets + imissed + ierrors) = itotal
> as the total number of packets hitting the port. However, I've noticed that
> when throughput gets too high, imissed will remain 0 while itotal is
> smaller than the number of packets sent by the traffic generator. I've
> ruled out connection issues because increasing the number of RSS queues
> seems to fix the problem (up to a certain threshold before itotal again
> becomes smaller than the number sent), but I don't understand why. If it is
> not dropped in HW because the queues are full (since imissed = 0), where
> are the packets being dropped and is there a way I can count these?
>
> I am using DPDK 20.08 with a Mellanox CX-5, RSS queue size = 4096
When app buffers fill up then the HW buffers start to fill up. When HW
buffers are full then the PHY responds by generating flow control frames
or simply dropping packets. You could experiment by enabling/disabling
flow control to verify that the packet counts are correct when flow
control is enabled.
You could also look at the rx_discards_phy counter and contrast it with
the rx_out_of_buffer statistic:
https://community.mellanox.com/s/article/understanding-mlx5-ethtool-counters
My read is that rx_out_of_buffer indicates that the HW doesn't have any
RX descriptors available, possibly because of PCIe congestion or because
the app's receive queue is empty. On the other hand, rx_discards_phy
indicates that the HW buffers are full. I don't see the rx_discards_phy
used in any stats, only available as an xstat.
Dave
prev parent reply other threads:[~2020-09-21 17:53 UTC|newest]
Thread overview: 2+ messages / expand[flat|nested] mbox.gz Atom feed top
2020-09-17 3:42 Gerry Wan
2020-09-21 17:53 ` David Christensen [this message]
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=8ca52ae9-ec04-a12a-82c6-1b61392c1e9a@linux.vnet.ibm.com \
--to=drc@linux.vnet.ibm.com \
--cc=users@dpdk.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).