From: Honnappa Nagarahalli <Honnappa.Nagarahalli@arm.com>
To: Slava Ovsiienko <viacheslavo@nvidia.com>
Cc: Wathsala Wathawana Vithanage <wathsala.vithanage@arm.com>,
"thomas@monjalon.net" <thomas@monjalon.net>,
Dariusz Sosnowski <dsosnowski@nvidia.com>,
Ori Kam <orika@nvidia.com>, Suanming Mou <suanmingm@nvidia.com>,
Matan Azrad <matan@nvidia.com>, "dev@dpdk.org" <dev@dpdk.org>,
nd <nd@arm.com>
Subject: Re: [PATCH] net/mlx5: enable PCI related counters
Date: Wed, 14 Feb 2024 01:50:43 +0000 [thread overview]
Message-ID: <D87657FC-ABD1-4F3C-9E55-9E2A1354290C@arm.com> (raw)
In-Reply-To: <IA1PR12MB807867A54593FB0C310C9A88DF4F2@IA1PR12MB8078.namprd12.prod.outlook.com>
> On Feb 13, 2024, at 7:12 AM, Slava Ovsiienko <viacheslavo@nvidia.com> wrote:
>
> Hi,
>
> Regarding "dev_out_of_buffer" - it is global counter, relates to the whole device port,
> Including queues not managed by DPDK application - Mellanox/Nvidia NICs operate
> In "bifurcated mode" - there might be queues managed by kernel or another DPDK
> application. Not sure it makes a lot of sense, but I have no strong objections.
These are still helpful to debug in lab environment. But, it would be good to document these.
>
> The PCI related counters are also global ones and reflect statistics, impacted by
> PCI activity of the whole physical device, including all the network ports located
> on the same NIC board (and, sometimes, by internal activity in BlueField).
>
> As I said, no objections from my side:
>
> Acked-by: Viacheslav Ovsiienko <viacheslavo@nvidia.com>
>
> With best regards,
> Slava
>
>> -----Original Message-----
>> From: Wathsala Vithanage <wathsala.vithanage@arm.com>
>> Sent: Friday, February 9, 2024 10:42 PM
>> To: NBU-Contact-Thomas Monjalon (EXTERNAL) <thomas@monjalon.net>;
>> Dariusz Sosnowski <dsosnowski@nvidia.com>; Slava Ovsiienko
>> <viacheslavo@nvidia.com>; Ori Kam <orika@nvidia.com>; Suanming Mou
>> <suanmingm@nvidia.com>; Matan Azrad <matan@nvidia.com>
>> Cc: dev@dpdk.org; nd@arm.com; Wathsala Vithanage
>> <wathsala.vithanage@arm.com>; Honnappa Nagarahalli
>> <honnappa.nagarahalli@arm.com>
>> Subject: [PATCH] net/mlx5: enable PCI related counters
>>
>> Versions of Mellanox NICs starting from CX5 have device counters related to PCI.
>> These counters are helpful in debugging IO bottlenecks. For instance, the
>> outbound_pci_stalled_rd and outbound_pci_stalled_wr counters can help with
>> identifying NIC stalls due to insufficient PCI credits, which otherwise would have
>> required a PCI analyzer or a sophisticated PCI root port with a PMU.
>> Currently none of these are available in the MLX5 PMD even though ethtool is
>> capable of reading some of them.
>> Since PMD uses the same ioctl used by ethtool (SIOCETHTOOL) and reads via the
>> kernel driver it is possible to add support with ease.
>> There is one more PCI related counter and a device counter that aren't
>> implemented in the Linux driver at the moment. These two are named
>> outbound_pci_buffer_overflow and dev_out_of_buffer respectively. As per
>> Nvidia's documentation these two counters can tell the number of packets
>> dropped due to pci buffer overflow and the number of times the device owned
>> queue had not enough buffers allocated.
>>
>> Signed-off-by: Wathsala Vithanage <wathsala.vithanage@arm.com>
>> Reviewed-by: Honnappa Nagarahalli <honnappa.nagarahalli@arm.com>
>> ---
>> .mailmap | 1 +
>> drivers/net/mlx5/linux/mlx5_ethdev_os.c | 33
>> +++++++++++++++++++++++++
>> 2 files changed, 34 insertions(+)
>>
>> diff --git a/.mailmap b/.mailmap
>> index aa569ff456..f57415f7a1 100644
>> --- a/.mailmap
>> +++ b/.mailmap
>> @@ -1510,6 +1510,7 @@ Walter Heymans <walter.heymans@corigine.com>
>> Wang Sheng-Hui <shhuiw@gmail.com> Wangyu (Eric)
>> <seven.wangyu@huawei.com> Waterman Cao <waterman.cao@intel.com>
>> +Wathsala Vithanage <wathsala.vithanage@arm.com>
>> Weichun Chen <weichunx.chen@intel.com>
>> Wei Dai <wei.dai@intel.com>
>> Weifeng Li <liweifeng96@126.com>
>> diff --git a/drivers/net/mlx5/linux/mlx5_ethdev_os.c
>> b/drivers/net/mlx5/linux/mlx5_ethdev_os.c
>> index dd5a0c546d..8f1567f6a7 100644
>> --- a/drivers/net/mlx5/linux/mlx5_ethdev_os.c
>> +++ b/drivers/net/mlx5/linux/mlx5_ethdev_os.c
>> @@ -1574,6 +1574,39 @@ static const struct mlx5_counter_ctrl
>> mlx5_counters_init[] = {
>> .dpdk_name = "tx_vport_bytes",
>> .ctr_name = "vport_tx_bytes",
>> },
>> + /* Device counters */
>> + {
>> + .dpdk_name = "rx_pci_signal_integrity",
>> + .ctr_name = "rx_pci_signal_integrity",
>> + },
>> + {
>> + .dpdk_name = "tx_pci_signal_integrity",
>> + .ctr_name = "tx_pci_signal_integrity",
>> + },
>> + {
>> + .dpdk_name = "outbound_pci_buffer_overflow",
>> + .ctr_name = "outbound_pci_buffer_overflow",
>> + },
>> + {
>> + .dpdk_name = "outbound_pci_stalled_rd",
>> + .ctr_name = "outbound_pci_stalled_rd",
>> + },
>> + {
>> + .dpdk_name = "outbound_pci_stalled_wr",
>> + .ctr_name = "outbound_pci_stalled_wr",
>> + },
>> + {
>> + .dpdk_name = "outbound_pci_stalled_rd_events",
>> + .ctr_name = "outbound_pci_stalled_rd_events",
>> + },
>> + {
>> + .dpdk_name = "outbound_pci_stalled_wr_events",
>> + .ctr_name = "outbound_pci_stalled_wr_events",
>> + },
>> + {
>> + .dpdk_name = "dev_out_of_buffer",
>> + .ctr_name = "dev_out_of_buffer",
>> + },
>> };
>>
>> static const unsigned int xstats_n = RTE_DIM(mlx5_counters_init);
>> --
>> 2.25.1
>
next prev parent reply other threads:[~2024-02-14 1:51 UTC|newest]
Thread overview: 11+ messages / expand[flat|nested] mbox.gz Atom feed top
2024-02-09 20:41 Wathsala Vithanage
2024-02-09 21:16 ` Honnappa Nagarahalli
2024-02-10 1:32 ` Stephen Hemminger
2024-02-13 16:13 ` Dariusz Sosnowski
2024-02-14 0:52 ` Honnappa Nagarahalli
2024-02-13 13:12 ` Slava Ovsiienko
2024-02-14 1:50 ` Honnappa Nagarahalli [this message]
2024-02-13 16:17 ` Dariusz Sosnowski
2024-02-14 20:14 ` [PATCH v2] " Wathsala Vithanage
2024-02-15 18:26 ` [PATCH v3] " Wathsala Vithanage
2024-02-27 16:14 ` Raslan Darawsheh
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=D87657FC-ABD1-4F3C-9E55-9E2A1354290C@arm.com \
--to=honnappa.nagarahalli@arm.com \
--cc=dev@dpdk.org \
--cc=dsosnowski@nvidia.com \
--cc=matan@nvidia.com \
--cc=nd@arm.com \
--cc=orika@nvidia.com \
--cc=suanmingm@nvidia.com \
--cc=thomas@monjalon.net \
--cc=viacheslavo@nvidia.com \
--cc=wathsala.vithanage@arm.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).