From mboxrd@z Thu Jan 1 00:00:00 1970
Return-Path: <dev-bounces@dpdk.org>
Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124])
by inbox.dpdk.org (Postfix) with ESMTP id 1416443F31;
Fri, 3 May 2024 17:46:06 +0200 (CEST)
Received: from mails.dpdk.org (localhost [127.0.0.1])
by mails.dpdk.org (Postfix) with ESMTP id C2829402D1;
Fri, 3 May 2024 17:46:05 +0200 (CEST)
Received: from NAM12-BN8-obe.outbound.protection.outlook.com
(mail-bn8nam12on2042.outbound.protection.outlook.com [40.107.237.42])
by mails.dpdk.org (Postfix) with ESMTP id 9FBE3402DA
for <dev@dpdk.org>; Fri, 3 May 2024 17:46:03 +0200 (CEST)
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
b=KIsS5RwVabGao2qz9ylqRNMJaMkHoNtFFaayCcoRjcE39wvIb162eiaVZv7sYAzyRhV+ItEo5LEGoBECD+36G9AOuSXQLanDtHtshzhSKBMsqbiZ4rNcBfcP+vYSyEJKP1WxTI8VU4IhHEndrNMIafrQtW6AZI+HkDHEsMm3c6lv6lr1jaZPnyXRBSlzFSthZaLSAFo0o17GWbRsuLTLJdS8pN+A6bojwyBAPI94vK8J7AjWal5km13bcw04Q6MeLjbu3MyW2Knrii7qWRipD0FmpkzlZzOXJzeMSMVd73VrVVDQnpepjg4capcgo54uq0O7jv0rqvJFuUHzve1u2g==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
s=arcselector9901;
h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1;
bh=OzIM9yipxsOG4gaaYSpVlDiInKB0LiAVGXJ9R6tjLYo=;
b=NFgoWzDuovL20dsfnaj2TvrJsLyy6+mYwUrOX0tdt79jxqjvnWFg1uNGAYgK3r/sI390rINpJu6AcCNSEmIK1ghbWUum5YFnfdilADPRmiAQYFD1z04DXACQMA/akBM1wAK19px/Qj69oYIaNdEDBOicPqIhddCYoNmhKXpQj12svCpxFKhB4TqiVS/qTofvt8KULAkFyqxOnKldF53RmGVAyRxmrjj6Ug0RqFDfUZcIBX1ThDVIFtQWcWwHoiCjT+8LuizGXwSXh0Y3X9XW99rmKfYGRt8JYfHV+ZMelCCN7d5uDL9NW1YIt+N3aBNH1oZsIq+JjZThpNtMEIyUBw==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is
165.204.84.17) smtp.rcpttodomain=tuxdriver.com smtp.mailfrom=amd.com;
dmarc=pass (p=quarantine sp=quarantine pct=100) action=none
header.from=amd.com; dkim=none (message not signed); arc=none (0)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amd.com; s=selector1;
h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
bh=OzIM9yipxsOG4gaaYSpVlDiInKB0LiAVGXJ9R6tjLYo=;
b=L5aBfFdj0lBhPCEyXmKdB8hJRSoLGjVxwGg1Hrk7Ug3I/d76ITlze0UTvfSDcg+o3ULUwpTiHyVablxSLZQdPiC8jPHPtyAwo2hzucPgKlncxQZxikpbZEglcZ/q+v/MX5TojNw3IVYghQ3tLbRXJ7QjN0fLQOJWwSoiStmuAbc=
Received: from MN2PR22CA0008.namprd22.prod.outlook.com (2603:10b6:208:238::13)
by PH0PR12MB5608.namprd12.prod.outlook.com (2603:10b6:510:143::13)
with Microsoft SMTP Server (version=TLS1_2,
cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7544.25; Fri, 3 May
2024 15:45:56 +0000
Received: from MN1PEPF0000F0DF.namprd04.prod.outlook.com
(2603:10b6:208:238:cafe::29) by MN2PR22CA0008.outlook.office365.com
(2603:10b6:208:238::13) with Microsoft SMTP Server (version=TLS1_2,
cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7544.32 via Frontend
Transport; Fri, 3 May 2024 15:45:55 +0000
X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 165.204.84.17)
smtp.mailfrom=amd.com; dkim=none (message not signed)
header.d=none;dmarc=pass action=none header.from=amd.com;
Received-SPF: Pass (protection.outlook.com: domain of amd.com designates
165.204.84.17 as permitted sender) receiver=protection.outlook.com;
client-ip=165.204.84.17; helo=SATLEXMB04.amd.com; pr=C
Received: from SATLEXMB04.amd.com (165.204.84.17) by
MN1PEPF0000F0DF.mail.protection.outlook.com (10.167.242.37) with Microsoft
SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id
15.20.7544.18 via Frontend Transport; Fri, 3 May 2024 15:45:55 +0000
Received: from telcodpdk.amd.com (10.180.168.240) by SATLEXMB04.amd.com
(10.181.40.145) with Microsoft SMTP Server (version=TLS1_2,
cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.35; Fri, 3 May
2024 10:45:54 -0500
From: Ferruh Yigit <ferruh.yigit@amd.com>
To: "John W. Linville" <linville@tuxdriver.com>
CC: Thomas Monjalon <thomas@monjalon.net>, <dev@dpdk.org>,
=?UTF-8?q?Mattias=20R=C3=B6nnblom?= <mattias.ronnblom@ericsson.com>, Stephen
Hemminger <stephen@networkplumber.org>, =?UTF-8?q?Morten=20Br=C3=B8rup?=
<mb@smartsharesystems.com>
Subject: [RFC v3] net/af_packet: make stats reset reliable
Date: Fri, 3 May 2024 16:45:47 +0100
Message-ID: <20240503154547.392069-1-ferruh.yigit@amd.com>
X-Mailer: git-send-email 2.34.1
In-Reply-To: <20240425174617.2126159-1-ferruh.yigit@amd.com>
References: <20240425174617.2126159-1-ferruh.yigit@amd.com>
MIME-Version: 1.0
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 8bit
X-Originating-IP: [10.180.168.240]
X-ClientProxiedBy: SATLEXMB03.amd.com (10.181.40.144) To SATLEXMB04.amd.com
(10.181.40.145)
X-EOPAttributedMessage: 0
X-MS-PublicTrafficType: Email
X-MS-TrafficTypeDiagnostic: MN1PEPF0000F0DF:EE_|PH0PR12MB5608:EE_
X-MS-Office365-Filtering-Correlation-Id: 40460906-dcb5-4bca-07c1-08dc6b881edd
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam: BCL:0;ARA:13230031|36860700004|376005|1800799015;
X-Microsoft-Antispam-Message-Info: =?utf-8?B?c1YrMlRvZVp4dTJIaGc5dEZZL0xaMjhtQ0kwZVZvWnA1ZmJ4WHhBRnhxaDVr?=
=?utf-8?B?Vkh0RldjVUlsODhrbTlzUlhSeUY5Q0p1cmpaU1lPSXBpUFhDVmhjOVdPNjRL?=
=?utf-8?B?VVk0NVRBTm5YRnp1eTByVjB3L1B0T2orSE9hVzZCWEFxTkwrNmR5QXNSWlJ3?=
=?utf-8?B?TFBYRHl5YnFoSDRVZEs5eEZsbVE2MWUyL1JhcGZ0U3hoZllobzRoTURLM3ln?=
=?utf-8?B?OGxLSWxjejRVRTVHTHljSFdyS2lScHF1SDZ5anI4Vngzc0p3TWhEZm9Ec0ht?=
=?utf-8?B?SUJsMG1RQ1FNKzMzNDlXVm1OOUtGYzNxc0FNRjdMTGcwTDAxdUoyaU1yQnZi?=
=?utf-8?B?S2dVZld3dlZGQzdDWjRRQWUwSGt1c1M3S1psWkFWM2lyTWhYckxidkhNc0Fu?=
=?utf-8?B?TnZvczc5blg0UkpPTHJRWTc2THpUMDUyL0FxeVN4eWM5WkZ6eEVCT2pBTjMv?=
=?utf-8?B?NHY4WGhadFZUaE1UZ0Z2QjhoMDlRallFYW5ES1RBZkNHL3ZYck01aFB4WUFC?=
=?utf-8?B?bU9WVFI0WjRIcGZGdTNvV2NRNTVjTWgvVUgyZnlNU0pFN2c2bVBJckpyUlVu?=
=?utf-8?B?T0JDMW9DdzlTYzFKY0hwaG94MmREdW1HTHArMkFaWitnOVdrekZVa1N0T01K?=
=?utf-8?B?aGJzNmxsOURqMHBTZ2ZFb1NDaldkakRBUm1mZ0Y5cmM2YVhZK3pMb0t6dWRJ?=
=?utf-8?B?M1JVWXhyOGlKWC80ZzVoUjUxVEhSZXBmc0M0YUVGdlB1VkNFb0lGQ1R4N2JS?=
=?utf-8?B?blpPWDNsYXNyS0dCRmZCdndmWmdsdTl3UXlFQStYMlVNSTMrdU54SFJzNFV4?=
=?utf-8?B?ektnUk9BbXQvcWFzNHRzYkZpYnJ4RWJlRUR6ekI3Mi94QVdSYkVNVnQ5M2lD?=
=?utf-8?B?MW5mSzk3TFQzeUsyVVlPdFRmelBUc0ZMTzNXOUNVMjBjZVRGTFYrUCtRcWVG?=
=?utf-8?B?VS9ZUWJHL1BnbjlpelFKM3dWMyt0VFFZMUVNQ2c0NDhCLzViV2x1S1BILzJC?=
=?utf-8?B?MEtlWEN4OWRkT01JK0dRUDVSVE5aV28ySVRWdUJUMXY2TE9PcEd5UmpxbkxJ?=
=?utf-8?B?dFZHRzFCVFBVT1Nna3NLdDZqalhCNi9RNzd6bkVjdHREU2NwelJzNTRBUHF5?=
=?utf-8?B?NmpxdThzSHZIdnNHSENacTJWVDF3alJlVmFIRC9yS2FaVG9uZzFWVEZoTk1W?=
=?utf-8?B?T25NUG1yb3VmK1ZkUVo2b0NmT29DV1BGRkVhWENJdTdPRENPTHZDNENlQ3g5?=
=?utf-8?B?bUVsZzFGUnlNK2tEN0kwVFJjeS9kbVRTdUIxVGRFU2h6K0N2YlpGUmR0K0FQ?=
=?utf-8?B?Wm81YitMbEFiVktHVFFNbWMzdXJObmFxRVU5Q3ZoaVc1VUs2Qm4rZWE0ekha?=
=?utf-8?B?RzZhcUpicnA4VnFLaGZUTnRGOUhOSjQxQm0yNFFjM0gxNEx5aURWOUg2cnhp?=
=?utf-8?B?d1lhRDc2aXQ2NVIzS2J6K00xSnJ6QlNzSEQ5akV5dXZ6aWQ0WlF3N0NlQ01z?=
=?utf-8?B?NThsTGt1dUg0N2lXR1gwdGlETVBpL0VsOFJkVXJ0TkRsUEppamo2QlBxZ0tS?=
=?utf-8?B?SGJ2OG5sanM1bUdNclp1Ym51aGpiMGJWZzhlSGIrSXdrT2E5em5Mbzg0eFhS?=
=?utf-8?B?QWQ2aThXczFsU2ZZWXgvd0lPRC85QTNzaExKMFBMOHRyYm5iQWN6cFQyaDhh?=
=?utf-8?B?TURTanhacVRQV1k5SWhIRUdOcE9oMDJhMjBUc05MSmo3TE5pd25CU1l3TCs4?=
=?utf-8?B?alcycjVydzVVbkx6M3FvSTd1VExpUnhvdll5UnlwTHZLYmVkWVZSNEp6WW5K?=
=?utf-8?B?UlgvUlU1bFFkWmZpeXJEbmhtdEExd1hBaUJRSE9Bb3RnaytoV0FYNFpVMzJi?=
=?utf-8?Q?g3YHOPSA3CVz9?=
X-Forefront-Antispam-Report: CIP:165.204.84.17; CTRY:US; LANG:en; SCL:1; SRV:;
IPV:CAL; SFV:NSPM; H:SATLEXMB04.amd.com; PTR:InfoDomainNonexistent; CAT:NONE;
SFS:(13230031)(36860700004)(376005)(1800799015); DIR:OUT; SFP:1101;
X-OriginatorOrg: amd.com
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 03 May 2024 15:45:55.6512 (UTC)
X-MS-Exchange-CrossTenant-Network-Message-Id: 40460906-dcb5-4bca-07c1-08dc6b881edd
X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d
X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=3dd8961f-e488-4e60-8e11-a82d994e183d; Ip=[165.204.84.17];
Helo=[SATLEXMB04.amd.com]
X-MS-Exchange-CrossTenant-AuthSource: MN1PEPF0000F0DF.namprd04.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Anonymous
X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem
X-MS-Exchange-Transport-CrossTenantHeadersStamped: PH0PR12MB5608
X-BeenThere: dev@dpdk.org
X-Mailman-Version: 2.1.29
Precedence: list
List-Id: DPDK patches and discussions <dev.dpdk.org>
List-Unsubscribe: <https://mails.dpdk.org/options/dev>,
<mailto:dev-request@dpdk.org?subject=unsubscribe>
List-Archive: <http://mails.dpdk.org/archives/dev/>
List-Post: <mailto:dev@dpdk.org>
List-Help: <mailto:dev-request@dpdk.org?subject=help>
List-Subscribe: <https://mails.dpdk.org/listinfo/dev>,
<mailto:dev-request@dpdk.org?subject=subscribe>
Errors-To: dev-bounces@dpdk.org
For stats reset, use an offset instead of zeroing out actual stats values,
get_stats() displays diff between stats and offset.
This way stats only updated in datapath and offset only updated in stats
reset function. This makes stats reset function more reliable.
As stats only written by single thread, we can remove 'volatile' qualifier
which should improve the performance in datapath.
While updating around, 'igb_stats' parameter renamed as 'stats'.
Signed-off-by: Ferruh Yigit <ferruh.yigit@amd.com>
---
Cc: Mattias Rönnblom <mattias.ronnblom@ericsson.com>
Cc: Stephen Hemminger <stephen@networkplumber.org>
Cc: Morten Brørup <mb@smartsharesystems.com>
This update triggered by mail list discussion [1].
[1]
https://inbox.dpdk.org/dev/3b2cf48e-2293-4226-b6cd-5f4dd3969f99@lysator.liu.se/
v2:
* Remove wrapping check for stats
v3:
* counter and offset put into same struct per stats
* Use atomic load / store for stats values
---
drivers/net/af_packet/rte_eth_af_packet.c | 98 ++++++++++++++++-------
1 file changed, 68 insertions(+), 30 deletions(-)
diff --git a/drivers/net/af_packet/rte_eth_af_packet.c b/drivers/net/af_packet/rte_eth_af_packet.c
index 6b7b16f3486d..ebef1cb06450 100644
--- a/drivers/net/af_packet/rte_eth_af_packet.c
+++ b/drivers/net/af_packet/rte_eth_af_packet.c
@@ -6,6 +6,7 @@
* All rights reserved.
*/
+#include <rte_atomic.h>
#include <rte_common.h>
#include <rte_string_fns.h>
#include <rte_mbuf.h>
@@ -40,6 +41,11 @@
#define DFLT_FRAME_SIZE (1 << 11)
#define DFLT_FRAME_COUNT (1 << 9)
+struct stats {
+ uint64_t counter;
+ uint64_t offset;
+};
+
struct __rte_cache_aligned pkt_rx_queue {
int sockfd;
@@ -52,8 +58,8 @@ struct __rte_cache_aligned pkt_rx_queue {
uint16_t in_port;
uint8_t vlan_strip;
- volatile unsigned long rx_pkts;
- volatile unsigned long rx_bytes;
+ struct stats rx_pkts;
+ struct stats rx_bytes;
};
struct __rte_cache_aligned pkt_tx_queue {
@@ -65,9 +71,9 @@ struct __rte_cache_aligned pkt_tx_queue {
unsigned int framecount;
unsigned int framenum;
- volatile unsigned long tx_pkts;
- volatile unsigned long err_pkts;
- volatile unsigned long tx_bytes;
+ struct stats tx_pkts;
+ struct stats err_pkts;
+ struct stats tx_bytes;
};
struct pmd_internals {
@@ -111,6 +117,34 @@ RTE_LOG_REGISTER_DEFAULT(af_packet_logtype, NOTICE);
rte_log(RTE_LOG_ ## level, af_packet_logtype, \
"%s(): " fmt ":%s\n", __func__, ##args, strerror(errno))
+static inline uint64_t
+stats_get(struct stats *s)
+{
+ uint64_t counter = rte_atomic_load_explicit(&s->counter,
+ rte_memory_order_relaxed);
+ uint64_t offset = rte_atomic_load_explicit(&s->offset,
+ rte_memory_order_relaxed);
+ return counter - offset;
+}
+
+static inline void
+stats_add(struct stats *s, uint16_t n)
+{
+ uint64_t counter = s->counter;
+ counter += n;
+ rte_atomic_store_explicit(&s->counter, counter,
+ rte_memory_order_relaxed);
+}
+
+static inline void
+stats_reset(struct stats *s)
+{
+ uint64_t counter = rte_atomic_load_explicit(&s->counter,
+ rte_memory_order_relaxed);
+ rte_atomic_store_explicit(&s->offset, counter,
+ rte_memory_order_relaxed);
+}
+
static uint16_t
eth_af_packet_rx(void *queue, struct rte_mbuf **bufs, uint16_t nb_pkts)
{
@@ -169,8 +203,8 @@ eth_af_packet_rx(void *queue, struct rte_mbuf **bufs, uint16_t nb_pkts)
num_rx_bytes += mbuf->pkt_len;
}
pkt_q->framenum = framenum;
- pkt_q->rx_pkts += num_rx;
- pkt_q->rx_bytes += num_rx_bytes;
+ stats_add(&pkt_q->rx_pkts, num_rx);
+ stats_add(&pkt_q->rx_bytes, num_rx_bytes);
return num_rx;
}
@@ -305,9 +339,9 @@ eth_af_packet_tx(void *queue, struct rte_mbuf **bufs, uint16_t nb_pkts)
}
pkt_q->framenum = framenum;
- pkt_q->tx_pkts += num_tx;
- pkt_q->err_pkts += i - num_tx;
- pkt_q->tx_bytes += num_tx_bytes;
+ stats_add(&pkt_q->tx_pkts, num_tx);
+ stats_add(&pkt_q->err_pkts, i - num_tx);
+ stats_add(&pkt_q->tx_bytes, num_tx_bytes);
return i;
}
@@ -387,7 +421,7 @@ eth_dev_info(struct rte_eth_dev *dev, struct rte_eth_dev_info *dev_info)
}
static int
-eth_stats_get(struct rte_eth_dev *dev, struct rte_eth_stats *igb_stats)
+eth_stats_get(struct rte_eth_dev *dev, struct rte_eth_stats *stats)
{
unsigned i, imax;
unsigned long rx_total = 0, tx_total = 0, tx_err_total = 0;
@@ -397,27 +431,29 @@ eth_stats_get(struct rte_eth_dev *dev, struct rte_eth_stats *igb_stats)
imax = (internal->nb_queues < RTE_ETHDEV_QUEUE_STAT_CNTRS ?
internal->nb_queues : RTE_ETHDEV_QUEUE_STAT_CNTRS);
for (i = 0; i < imax; i++) {
- igb_stats->q_ipackets[i] = internal->rx_queue[i].rx_pkts;
- igb_stats->q_ibytes[i] = internal->rx_queue[i].rx_bytes;
- rx_total += igb_stats->q_ipackets[i];
- rx_bytes_total += igb_stats->q_ibytes[i];
+ struct pkt_rx_queue *rxq = &internal->rx_queue[i];
+ stats->q_ipackets[i] = stats_get(&rxq->rx_pkts);
+ stats->q_ibytes[i] = stats_get(&rxq->rx_bytes);
+ rx_total += stats->q_ipackets[i];
+ rx_bytes_total += stats->q_ibytes[i];
}
imax = (internal->nb_queues < RTE_ETHDEV_QUEUE_STAT_CNTRS ?
internal->nb_queues : RTE_ETHDEV_QUEUE_STAT_CNTRS);
for (i = 0; i < imax; i++) {
- igb_stats->q_opackets[i] = internal->tx_queue[i].tx_pkts;
- igb_stats->q_obytes[i] = internal->tx_queue[i].tx_bytes;
- tx_total += igb_stats->q_opackets[i];
- tx_err_total += internal->tx_queue[i].err_pkts;
- tx_bytes_total += igb_stats->q_obytes[i];
+ struct pkt_tx_queue *txq = &internal->tx_queue[i];
+ stats->q_opackets[i] = stats_get(&txq->tx_pkts);
+ stats->q_obytes[i] = stats_get(&txq->tx_bytes);
+ tx_total += stats->q_opackets[i];
+ tx_err_total += stats_get(&txq->err_pkts);
+ tx_bytes_total += stats->q_obytes[i];
}
- igb_stats->ipackets = rx_total;
- igb_stats->ibytes = rx_bytes_total;
- igb_stats->opackets = tx_total;
- igb_stats->oerrors = tx_err_total;
- igb_stats->obytes = tx_bytes_total;
+ stats->ipackets = rx_total;
+ stats->ibytes = rx_bytes_total;
+ stats->opackets = tx_total;
+ stats->oerrors = tx_err_total;
+ stats->obytes = tx_bytes_total;
return 0;
}
@@ -428,14 +464,16 @@ eth_stats_reset(struct rte_eth_dev *dev)
struct pmd_internals *internal = dev->data->dev_private;
for (i = 0; i < internal->nb_queues; i++) {
- internal->rx_queue[i].rx_pkts = 0;
- internal->rx_queue[i].rx_bytes = 0;
+ struct pkt_rx_queue *rxq = &internal->rx_queue[i];
+ stats_reset(&rxq->rx_pkts);
+ stats_reset(&rxq->rx_bytes);
}
for (i = 0; i < internal->nb_queues; i++) {
- internal->tx_queue[i].tx_pkts = 0;
- internal->tx_queue[i].err_pkts = 0;
- internal->tx_queue[i].tx_bytes = 0;
+ struct pkt_tx_queue *txq = &internal->tx_queue[i];
+ stats_reset(&txq->tx_pkts);
+ stats_reset(&txq->err_pkts);
+ stats_reset(&txq->tx_bytes);
}
return 0;
--
2.34.1