rte_eth_stats_get seems slow

rte_eth_stats_get seems slow

[Morten Brørup]

Bruce,

rte_eth_stats_get() on Intel NICs seems slow to me.

E.g. getting the stats on a single port takes ~132 us (~451,000 CPU cycles) using the igb driver, and ~50 us using the i40e driver.

Referring to the igb driver source code [1], it's 44 calls to E1000_READ_REG(), so the math says that each one takes 3 us (~10,000 CPU cycles).

Is this expected behavior?

It adds up, e.g. it takes a full millisecond to fetch the stats from eight ports using the igb driver.

[1]: https://elixir.bootlin.com/dpdk/v24.11.1/source/drivers/net/e1000/igb_ethdev.c#L1724


Med venlig hilsen / Kind regards,
-Morten Brørup

Re: rte_eth_stats_get seems slow

[Stephen Hemminger]

On Fri, 25 Apr 2025 13:52:55 +0200
Morten Brørup <mb@smartsharesystems.com> wrote:

> Bruce,
> 
> rte_eth_stats_get() on Intel NICs seems slow to me.
> 
> E.g. getting the stats on a single port takes ~132 us (~451,000 CPU cycles) using the igb driver, and ~50 us using the i40e driver.
> 
> Referring to the igb driver source code [1], it's 44 calls to E1000_READ_REG(), so the math says that each one takes 3 us (~10,000 CPU cycles).
> 
> Is this expected behavior?
> 
> It adds up, e.g. it takes a full millisecond to fetch the stats from eight ports using the igb driver.
> 
> [1]: https://elixir.bootlin.com/dpdk/v24.11.1/source/drivers/net/e1000/igb_ethdev.c#L1724
> 
> 
> Med venlig hilsen / Kind regards,
> -Morten Brørup
> 

Well reading each stat requires a PCI access. And PCI accesses are non-cached.

RE: rte_eth_stats_get seems slow

[Morten Brørup]

> From: Stephen Hemminger [mailto:stephen@networkplumber.org]
> Sent: Saturday, 26 April 2025 17.24
> 
> On Fri, 25 Apr 2025 13:52:55 +0200
> Morten Brørup <mb@smartsharesystems.com> wrote:
> 
> > Bruce,
> >
> > rte_eth_stats_get() on Intel NICs seems slow to me.
> >
> > E.g. getting the stats on a single port takes ~132 us (~451,000 CPU
> cycles) using the igb driver, and ~50 us using the i40e driver.
> >
> > Referring to the igb driver source code [1], it's 44 calls to
> E1000_READ_REG(), so the math says that each one takes 3 us (~10,000
> CPU cycles).
> >
> > Is this expected behavior?
> >
> > It adds up, e.g. it takes a full millisecond to fetch the stats from
> eight ports using the igb driver.
> >
> > [1]:
> https://elixir.bootlin.com/dpdk/v24.11.1/source/drivers/net/e1000/igb_e
> thdev.c#L1724
> >
> >
> > Med venlig hilsen / Kind regards,
> > -Morten Brørup
> >
> 
> Well reading each stat requires a PCI access. And PCI accesses are non-
> cached.

You're right, thank you for reminding me. I was caught by surprise that getting 7 counters took so long.
Perhaps reading 44 NIC registers over the PCI bus is required to calculate those summary counters. Or nobody cared to optimize this function to only read the necessary registers.

We periodically poll the ethdev stats in a management thread, and I noticed the duration because of the jitter it caused in that thread.
It's not a real problem. If it was, we could easily move it to a separate thread or poll the counters iteratively port by port, instead of all ports in one go.
A longwinded way of saying: Probably not worth optimizing. ;-)