From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: <users-bounces@dpdk.org> Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 808C7A0547 for <public@inbox.dpdk.org>; Wed, 19 May 2021 18:06:56 +0200 (CEST) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 671CE40041; Wed, 19 May 2021 18:06:56 +0200 (CEST) Received: from mail-pg1-f181.google.com (mail-pg1-f181.google.com [209.85.215.181]) by mails.dpdk.org (Postfix) with ESMTP id 196B74003F for <users@dpdk.org>; Wed, 19 May 2021 18:06:54 +0200 (CEST) Received: by mail-pg1-f181.google.com with SMTP id q15so9779153pgg.12 for <users@dpdk.org>; Wed, 19 May 2021 09:06:54 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=networkplumber-org.20150623.gappssmtp.com; s=20150623; h=date:from:to:cc:subject:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=6CeBM3gThClllH9anw45G0kHet0402febYrdinRoWLc=; b=cXZ1uuOqOUkok3ggh8EBOqbTxVbsXqN8HccnN1GS2uGcw+gJxARmh2hFN8L9pWvndF Iu+hRWojHvZJPbgT2nSX6v9Y2kj0wBdImkUKCaP9wOHjXqoSR7v4pTgcqKeE27cClyxJ 3KLxXnlwOy1mARNaom/5rdkSV4uJgQPFVh7qQ1709lUMuCPjshN+3kb3v8aytNqgQ8fq UsG0a9JKaxvUmazhgFmE/nxXOO8902u3ODGTf2VBLWoMoXjQUbtqimztCuJ6LhPsgubM NcC+N7UrNDPMP0sSH4P8+kT4HMLRg8o3xJsJJXAx6mY/Nz3gzBnzRmd7lA9Q6drnvy7u ESHg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:from:to:cc:subject:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=6CeBM3gThClllH9anw45G0kHet0402febYrdinRoWLc=; b=CqY+tIwZ64zgNMkVW7VqCuOsrEqX8UCtv3GSRndknZK3o0/VozpMSRONS6sPciI+5Z KdAgjYNjwvglbX386YVoZNWeGt7KjzEPcTsyPs17LDo5bewSoxEq6sUzJAkek7oEME8u sZGZb02YIBKlGCl0VyvPC9P6iIzV5xUYBKOtys9PEwr862mLuWFC46YO3oZ9KPMASBXU 7FpV7CzrqUavvSzmM2SnodgZn6CeDcNnTWBnAhkPENqtSy3yZzg8E9nGiZULVOh3qSET gmrDuUXXDA1+ymy844IFE21DWHI87SluQyRBNdV0rnHQUEWzyJFC2emvIQazOH/P3jVs Sa1g== X-Gm-Message-State: AOAM531f7Z/GHGjJU21d1YG/yt+t13YTRDDgI7G92T3/wxZPuxHDDMyj g9wkxkAgfnoHVEscX5Aq+btU+g== X-Google-Smtp-Source: ABdhPJx8aa93THCulIDBlKgmPduLnJnR+PZ3FGYzCCCs/zP8z+EmRTQYmcDNM+jIhPBLpGyLYSAOCA== X-Received: by 2002:a05:6a00:1a90:b029:2dd:dd7a:bd7d with SMTP id e16-20020a056a001a90b02902dddd7abd7dmr11495854pfv.34.1621440413991; Wed, 19 May 2021 09:06:53 -0700 (PDT) Received: from hermes.local (76-14-218-44.or.wavecable.com. [76.14.218.44]) by smtp.gmail.com with ESMTPSA id ev2sm13019259pjb.46.2021.05.19.09.06.52 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 19 May 2021 09:06:53 -0700 (PDT) Date: Wed, 19 May 2021 09:06:50 -0700 From: Stephen Hemminger <stephen@networkplumber.org> To: "Van Haaren, Harry" <harry.van.haaren@intel.com> Cc: Filip Janiszewski <contact@filipjaniszewski.com>, "users@dpdk.org" <users@dpdk.org> Message-ID: <20210519090650.5023ce00@hermes.local> In-Reply-To: <BYAPR11MB3143AB2DBB2D49DAB425924CD72B9@BYAPR11MB3143.namprd11.prod.outlook.com> References: <01e2ccbe-a2c8-be7c-7f9d-af43f609e75f@filipjaniszewski.com> <BYAPR11MB3143AB2DBB2D49DAB425924CD72B9@BYAPR11MB3143.namprd11.prod.outlook.com> MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Subject: Re: [dpdk-users] Performance of rte_eth_stats_get X-BeenThere: users@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK usage discussions <users.dpdk.org> List-Unsubscribe: <https://mails.dpdk.org/options/users>, <mailto:users-request@dpdk.org?subject=unsubscribe> List-Archive: <http://mails.dpdk.org/archives/users/> List-Post: <mailto:users@dpdk.org> List-Help: <mailto:users-request@dpdk.org?subject=help> List-Subscribe: <https://mails.dpdk.org/listinfo/users>, <mailto:users-request@dpdk.org?subject=subscribe> Errors-To: users-bounces@dpdk.org Sender: "users" <users-bounces@dpdk.org> On Wed, 19 May 2021 15:14:38 +0000 "Van Haaren, Harry" <harry.van.haaren@intel.com> wrote: > > -----Original Message----- > > From: users <users-bounces@dpdk.org> On Behalf Of Filip Janiszewski > > Sent: Wednesday, May 19, 2021 2:10 PM > > To: users@dpdk.org > > Subject: [dpdk-users] Performance of rte_eth_stats_get > > > > Hi, > > > > Is it safe to call rte_eth_stats_get while capturing from the port? > > > > I'm mostly concerned about performance, if rte_eth_stats_get will in any > > way impact the port performance, in the application I plan to call the > > function from a thread that is not directly involved in the capture, > > there's another worker responsible for rx bursting, but I wonder if the > > NIC might get upset if I call it too frequently (say 10 times per > > second) and potentially cause some performance issues. > > > > The question is really Nic agnostic, but if the Nic vendor is actually > > relevant then I'm running Intel 700 series nic and Mellanox ConnectX-4/5. > > To understand what really goes on when getting stats, it might help to list the > steps involved in getting statistics from the NIC HW. > > 1) CPU sends an MMIO read (Memory Mapped I/O, aka, sometimes referred > to as a "pci read") to the NIC. > 2) The PCI bus has to handle extra TLPs (pci transactions) to satisfy read > 3) NIC has to send a reply based on accessing its internal counters > 4) CPU gets a result from the PCI read. > > Notice how elegantly this whole process is abstracted from SW? In code, reading > a stat value is just dereferencing a pointer that is mapped to the NIC HW address. > In practice from a CPU performance point of view, doing an MMIO-read is one of > the slowest things you can do. You say the stats-reads are occurring from a thread > that is not handling rx/datapath, so perhaps the CPU cycle cost itself isn't a concern. > > Do note however, that when reading a full set of extended stats from the NIC, there > could be many 10's to 100's of MMIO reads (depending on the statistics requested, > and how the PMD itself is implemented to handle stats updates). > > The PCI bus does become more busy with reads to the NIC HW when doing lots of > statistic updates, so there is some more contention/activity to be expected there. > The PCM tool can be very useful to see MMIO traffic, you could measure how many > extra PCI transactions are occurring due to reading stats every X ms? > https://github.com/opcm/pcm > > I can recommend measuring pkt latency/jitter as a histogram, as then outliers in performance > can be identified. If you specifically want to identify if these are due stats reads, compare > with a "no stats reads" latency/jitter histogram, and graphically see the impact. > In the end if it doesn't affect packet latency/jitter, then it has no impact right? > > Ultimately, I can't give a generic answer - best steps are to measure carefully and find out! > > > Thanks > > Hope the above helps and doesn't add confusion :) Regards, -Harry Many drivers require transactions with the firmware via mailbox. And that transaction needs a spin wait for the shared area.