DPDK patches and discussions
 help / color / mirror / Atom feed
From: Andrew Rybchenko <andrew.rybchenko@oktetlabs.ru>
To: dev@dpdk.org
Cc: Andy Moreton <amoreton@xilinx.com>, Ivan Malov <ivan.malov@oktetlabs.ru>
Subject: [dpdk-dev] [PATCH 11/20] net/sfc: add NUMA-aware registry of service logical cores
Date: Thu, 27 May 2021 18:25:01 +0300	[thread overview]
Message-ID: <20210527152510.1551026-12-andrew.rybchenko@oktetlabs.ru> (raw)
In-Reply-To: <20210527152510.1551026-1-andrew.rybchenko@oktetlabs.ru>

The driver requires service cores for housekeeping. Share these
cores for many adapters and various purposes to avoid extra CPU
overhead.

Since housekeeping services will talk to NIC, it should be possible
to choose logical core on matching NUMA node.

Signed-off-by: Andrew Rybchenko <andrew.rybchenko@oktetlabs.ru>
Reviewed-by: Andy Moreton <amoreton@xilinx.com>
Reviewed-by: Ivan Malov <ivan.malov@oktetlabs.ru>
---
 drivers/net/sfc/meson.build   |  1 +
 drivers/net/sfc/sfc_service.c | 99 +++++++++++++++++++++++++++++++++++
 drivers/net/sfc/sfc_service.h | 20 +++++++
 3 files changed, 120 insertions(+)
 create mode 100644 drivers/net/sfc/sfc_service.c
 create mode 100644 drivers/net/sfc/sfc_service.h

diff --git a/drivers/net/sfc/meson.build b/drivers/net/sfc/meson.build
index ccf5984d87..4ac97e8d43 100644
--- a/drivers/net/sfc/meson.build
+++ b/drivers/net/sfc/meson.build
@@ -62,4 +62,5 @@ sources = files(
         'sfc_ef10_tx.c',
         'sfc_ef100_rx.c',
         'sfc_ef100_tx.c',
+        'sfc_service.c',
 )
diff --git a/drivers/net/sfc/sfc_service.c b/drivers/net/sfc/sfc_service.c
new file mode 100644
index 0000000000..9c89484406
--- /dev/null
+++ b/drivers/net/sfc/sfc_service.c
@@ -0,0 +1,99 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ *
+ * Copyright(c) 2020-2021 Xilinx, Inc.
+ */
+
+#include <stdint.h>
+
+#include <rte_common.h>
+#include <rte_spinlock.h>
+#include <rte_lcore.h>
+#include <rte_service.h>
+#include <rte_memory.h>
+
+#include "sfc_log.h"
+#include "sfc_service.h"
+#include "sfc_debug.h"
+
+static uint32_t sfc_service_lcore[RTE_MAX_NUMA_NODES];
+static rte_spinlock_t sfc_service_lcore_lock = RTE_SPINLOCK_INITIALIZER;
+
+RTE_INIT(sfc_service_lcore_init)
+{
+	size_t i;
+
+	for (i = 0; i < RTE_DIM(sfc_service_lcore); ++i)
+		sfc_service_lcore[i] = RTE_MAX_LCORE;
+}
+
+static uint32_t
+sfc_find_service_lcore(int *socket_id)
+{
+	uint32_t service_core_list[RTE_MAX_LCORE];
+	uint32_t lcore_id;
+	int num;
+	int i;
+
+	SFC_ASSERT(rte_spinlock_is_locked(&sfc_service_lcore_lock));
+
+	num = rte_service_lcore_list(service_core_list,
+				    RTE_DIM(service_core_list));
+	if (num == 0) {
+		SFC_GENERIC_LOG(WARNING, "No service cores available");
+		return RTE_MAX_LCORE;
+	}
+	if (num < 0) {
+		SFC_GENERIC_LOG(ERR, "Failed to get service core list");
+		return RTE_MAX_LCORE;
+	}
+
+	for (i = 0; i < num; ++i) {
+		lcore_id = service_core_list[i];
+
+		if (*socket_id == SOCKET_ID_ANY) {
+			*socket_id = rte_lcore_to_socket_id(lcore_id);
+			break;
+		} else if (rte_lcore_to_socket_id(lcore_id) ==
+			   (unsigned int)*socket_id) {
+			break;
+		}
+	}
+
+	if (i == num) {
+		SFC_GENERIC_LOG(WARNING,
+			"No service cores reserved at socket %d", *socket_id);
+		return RTE_MAX_LCORE;
+	}
+
+	return lcore_id;
+}
+
+uint32_t
+sfc_get_service_lcore(int socket_id)
+{
+	uint32_t lcore_id = RTE_MAX_LCORE;
+
+	rte_spinlock_lock(&sfc_service_lcore_lock);
+
+	if (socket_id != SOCKET_ID_ANY) {
+		lcore_id = sfc_service_lcore[socket_id];
+	} else {
+		size_t i;
+
+		for (i = 0; i < RTE_DIM(sfc_service_lcore); ++i) {
+			if (sfc_service_lcore[i] != RTE_MAX_LCORE) {
+				lcore_id = sfc_service_lcore[i];
+				break;
+			}
+		}
+	}
+
+	if (lcore_id == RTE_MAX_LCORE) {
+		lcore_id = sfc_find_service_lcore(&socket_id);
+		if (lcore_id != RTE_MAX_LCORE)
+			sfc_service_lcore[socket_id] = lcore_id;
+	}
+
+	rte_spinlock_unlock(&sfc_service_lcore_lock);
+	return lcore_id;
+}
diff --git a/drivers/net/sfc/sfc_service.h b/drivers/net/sfc/sfc_service.h
new file mode 100644
index 0000000000..bbcce28479
--- /dev/null
+++ b/drivers/net/sfc/sfc_service.h
@@ -0,0 +1,20 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ *
+ * Copyright(c) 2020-2021 Xilinx, Inc.
+ */
+
+#ifndef _SFC_SERVICE_H
+#define _SFC_SERVICE_H
+
+#include <stdint.h>
+
+#ifdef __cplusplus
+extern "C" {
+#endif
+
+uint32_t sfc_get_service_lcore(int socket_id);
+
+#ifdef __cplusplus
+}
+#endif
+#endif  /* _SFC_SERVICE_H */
-- 
2.30.2


  parent reply	other threads:[~2021-05-27 15:27 UTC|newest]

Thread overview: 104+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2021-05-27 15:24 [dpdk-dev] [PATCH 00/20] net/sfc: support flow API COUNT action Andrew Rybchenko
2021-05-27 15:24 ` [dpdk-dev] [PATCH 01/20] net/sfc: introduce ethdev Rx queue ID Andrew Rybchenko
2021-05-27 15:24 ` [dpdk-dev] [PATCH 02/20] net/sfc: do not enable interrupts on internal Rx queues Andrew Rybchenko
2021-05-27 15:24 ` [dpdk-dev] [PATCH 03/20] common/sfc_efx/base: separate target EvQ and IRQ config Andrew Rybchenko
2021-05-27 15:24 ` [dpdk-dev] [PATCH 04/20] common/sfc_efx/base: support custom EvQ to IRQ mapping Andrew Rybchenko
2021-05-27 15:24 ` [dpdk-dev] [PATCH 05/20] net/sfc: explicitly control IRQ used for Rx queues Andrew Rybchenko
2021-05-27 15:24 ` [dpdk-dev] [PATCH 06/20] net/sfc: introduce ethdev Tx queue ID Andrew Rybchenko
2021-05-27 15:24 ` [dpdk-dev] [PATCH 07/20] common/sfc_efx/base: add ingress m-port RxQ flag Andrew Rybchenko
2021-05-27 15:24 ` [dpdk-dev] [PATCH 08/20] common/sfc_efx/base: add user mark " Andrew Rybchenko
2021-05-27 15:24 ` [dpdk-dev] [PATCH 09/20] net/sfc: add abstractions for the management EVQ identity Andrew Rybchenko
2021-05-27 15:25 ` [dpdk-dev] [PATCH 10/20] net/sfc: add support for initialising different RxQ types Andrew Rybchenko
2021-05-27 15:25 ` Andrew Rybchenko [this message]
2021-05-27 15:25 ` [dpdk-dev] [PATCH 12/20] net/sfc: reserve RxQ for counters Andrew Rybchenko
2021-05-27 15:25 ` [dpdk-dev] [PATCH 13/20] common/sfc_efx/base: add counter creation MCDI wrappers Andrew Rybchenko
2021-05-27 15:25 ` [dpdk-dev] [PATCH 14/20] common/sfc_efx/base: add counter stream " Andrew Rybchenko
2021-05-27 15:25 ` [dpdk-dev] [PATCH 15/20] common/sfc_efx/base: support counter in action set Andrew Rybchenko
2021-05-27 15:25 ` [dpdk-dev] [PATCH 16/20] net/sfc: add Rx datapath method to get pushed buffers count Andrew Rybchenko
2021-05-27 15:25 ` [dpdk-dev] [PATCH 17/20] common/sfc_efx/base: add max MAE counters to limits Andrew Rybchenko
2021-05-27 15:25 ` [dpdk-dev] [PATCH 18/20] common/sfc_efx/base: add packetiser packet format definition Andrew Rybchenko
2021-05-27 15:25 ` [dpdk-dev] [PATCH 19/20] net/sfc: support flow action COUNT in transfer rules Andrew Rybchenko
2021-05-27 15:25 ` [dpdk-dev] [PATCH 20/20] net/sfc: support flow API query for count actions Andrew Rybchenko
2021-06-04 14:23 ` [dpdk-dev] [PATCH v2 00/20] net/sfc: support flow API COUNT action Andrew Rybchenko
2021-06-04 14:23   ` [dpdk-dev] [PATCH v2 01/20] net/sfc: introduce ethdev Rx queue ID Andrew Rybchenko
2021-06-04 14:23   ` [dpdk-dev] [PATCH v2 02/20] net/sfc: do not enable interrupts on internal Rx queues Andrew Rybchenko
2021-06-04 14:23   ` [dpdk-dev] [PATCH v2 03/20] common/sfc_efx/base: separate target EvQ and IRQ config Andrew Rybchenko
2021-06-04 14:23   ` [dpdk-dev] [PATCH v2 04/20] common/sfc_efx/base: support custom EvQ to IRQ mapping Andrew Rybchenko
2021-06-04 14:23   ` [dpdk-dev] [PATCH v2 05/20] net/sfc: explicitly control IRQ used for Rx queues Andrew Rybchenko
2021-06-04 14:24   ` [dpdk-dev] [PATCH v2 06/20] net/sfc: introduce ethdev Tx queue ID Andrew Rybchenko
2021-06-04 14:24   ` [dpdk-dev] [PATCH v2 07/20] common/sfc_efx/base: add ingress m-port RxQ flag Andrew Rybchenko
2021-06-04 14:24   ` [dpdk-dev] [PATCH v2 08/20] common/sfc_efx/base: add user mark " Andrew Rybchenko
2021-06-04 14:24   ` [dpdk-dev] [PATCH v2 09/20] net/sfc: add abstractions for the management EVQ identity Andrew Rybchenko
2021-06-04 14:24   ` [dpdk-dev] [PATCH v2 10/20] net/sfc: add support for initialising different RxQ types Andrew Rybchenko
2021-06-04 14:24   ` [dpdk-dev] [PATCH v2 11/20] net/sfc: add NUMA-aware registry of service logical cores Andrew Rybchenko
2021-06-04 14:24   ` [dpdk-dev] [PATCH v2 12/20] net/sfc: reserve RxQ for counters Andrew Rybchenko
2021-06-04 14:24   ` [dpdk-dev] [PATCH v2 13/20] common/sfc_efx/base: add counter creation MCDI wrappers Andrew Rybchenko
2021-06-04 14:24   ` [dpdk-dev] [PATCH v2 14/20] common/sfc_efx/base: add counter stream " Andrew Rybchenko
2021-06-04 14:24   ` [dpdk-dev] [PATCH v2 15/20] common/sfc_efx/base: support counter in action set Andrew Rybchenko
2021-06-04 14:24   ` [dpdk-dev] [PATCH v2 16/20] net/sfc: add Rx datapath method to get pushed buffers count Andrew Rybchenko
2021-06-04 14:24   ` [dpdk-dev] [PATCH v2 17/20] common/sfc_efx/base: add max MAE counters to limits Andrew Rybchenko
2021-06-04 14:24   ` [dpdk-dev] [PATCH v2 18/20] common/sfc_efx/base: add packetiser packet format definition Andrew Rybchenko
2021-06-04 14:24   ` [dpdk-dev] [PATCH v2 19/20] net/sfc: support flow action COUNT in transfer rules Andrew Rybchenko
2021-06-04 14:24   ` [dpdk-dev] [PATCH v2 20/20] net/sfc: support flow API query for count actions Andrew Rybchenko
2021-06-17  8:37   ` [dpdk-dev] [PATCH v2 00/20] net/sfc: support flow API COUNT action David Marchand
2021-06-18 13:40     ` Andrew Rybchenko
2021-06-18 13:40 ` [dpdk-dev] [PATCH v3 " Andrew Rybchenko
2021-06-18 13:40   ` [dpdk-dev] [PATCH v3 01/20] net/sfc: introduce ethdev Rx queue ID Andrew Rybchenko
2021-06-18 13:40   ` [dpdk-dev] [PATCH v3 02/20] net/sfc: do not enable interrupts on internal Rx queues Andrew Rybchenko
2021-06-18 13:40   ` [dpdk-dev] [PATCH v3 03/20] common/sfc_efx/base: separate target EvQ and IRQ config Andrew Rybchenko
2021-06-18 13:40   ` [dpdk-dev] [PATCH v3 04/20] common/sfc_efx/base: support custom EvQ to IRQ mapping Andrew Rybchenko
2021-06-18 13:40   ` [dpdk-dev] [PATCH v3 05/20] net/sfc: explicitly control IRQ used for Rx queues Andrew Rybchenko
2021-06-18 13:40   ` [dpdk-dev] [PATCH v3 06/20] net/sfc: introduce ethdev Tx queue ID Andrew Rybchenko
2021-06-18 13:40   ` [dpdk-dev] [PATCH v3 07/20] common/sfc_efx/base: add ingress m-port RxQ flag Andrew Rybchenko
2021-06-18 13:40   ` [dpdk-dev] [PATCH v3 08/20] common/sfc_efx/base: add user mark " Andrew Rybchenko
2021-06-18 13:40   ` [dpdk-dev] [PATCH v3 09/20] net/sfc: add abstractions for the management EVQ identity Andrew Rybchenko
2021-06-18 13:40   ` [dpdk-dev] [PATCH v3 10/20] net/sfc: add support for initialising different RxQ types Andrew Rybchenko
2021-06-18 13:40   ` [dpdk-dev] [PATCH v3 11/20] net/sfc: add NUMA-aware registry of service logical cores Andrew Rybchenko
2021-06-18 13:40   ` [dpdk-dev] [PATCH v3 12/20] net/sfc: reserve RxQ for counters Andrew Rybchenko
2021-06-18 13:40   ` [dpdk-dev] [PATCH v3 13/20] common/sfc_efx/base: add counter creation MCDI wrappers Andrew Rybchenko
2021-06-18 13:40   ` [dpdk-dev] [PATCH v3 14/20] common/sfc_efx/base: add counter stream " Andrew Rybchenko
2021-06-18 13:40   ` [dpdk-dev] [PATCH v3 15/20] common/sfc_efx/base: support counter in action set Andrew Rybchenko
2021-06-18 13:40   ` [dpdk-dev] [PATCH v3 16/20] net/sfc: add Rx datapath method to get pushed buffers count Andrew Rybchenko
2021-06-18 13:40   ` [dpdk-dev] [PATCH v3 17/20] common/sfc_efx/base: add max MAE counters to limits Andrew Rybchenko
2021-06-18 13:40   ` [dpdk-dev] [PATCH v3 18/20] common/sfc_efx/base: add packetiser packet format definition Andrew Rybchenko
2021-06-18 13:40   ` [dpdk-dev] [PATCH v3 19/20] net/sfc: support flow action COUNT in transfer rules Andrew Rybchenko
2021-06-21  8:28     ` David Marchand
2021-06-21  9:30       ` Thomas Monjalon
2021-07-01  9:22         ` Andrew Rybchenko
2021-07-01 12:34           ` David Marchand
2021-07-01 13:05             ` Andrew Rybchenko
2021-07-01 13:35               ` Bruce Richardson
2021-07-02  8:03                 ` Andrew Rybchenko
2021-07-02  8:43               ` Andrew Rybchenko
2021-07-02 12:30                 ` Thomas Monjalon
2021-07-02 12:53                   ` Andrew Rybchenko
2021-07-04 19:45                     ` Thomas Monjalon
2021-07-05  8:41                       ` Andrew Rybchenko
2021-07-02 13:37                 ` David Marchand
2021-07-02 13:39                   ` Andrew Rybchenko
2021-06-18 13:40   ` [dpdk-dev] [PATCH v3 20/20] net/sfc: support flow API query for count actions Andrew Rybchenko
2021-07-02  8:39 ` [dpdk-dev] [PATCH v4 00/20] net/sfc: support flow API COUNT action Andrew Rybchenko
2021-07-02  8:39   ` [dpdk-dev] [PATCH v4 01/20] net/sfc: introduce ethdev Rx queue ID Andrew Rybchenko
2021-07-02  8:39   ` [dpdk-dev] [PATCH v4 02/20] net/sfc: do not enable interrupts on internal Rx queues Andrew Rybchenko
2021-07-02  8:39   ` [dpdk-dev] [PATCH v4 03/20] common/sfc_efx/base: separate target EvQ and IRQ config Andrew Rybchenko
2021-07-02  8:39   ` [dpdk-dev] [PATCH v4 04/20] common/sfc_efx/base: support custom EvQ to IRQ mapping Andrew Rybchenko
2021-07-02  8:39   ` [dpdk-dev] [PATCH v4 05/20] net/sfc: explicitly control IRQ used for Rx queues Andrew Rybchenko
2021-07-02  8:39   ` [dpdk-dev] [PATCH v4 06/20] net/sfc: introduce ethdev Tx queue ID Andrew Rybchenko
2021-07-02  8:39   ` [dpdk-dev] [PATCH v4 07/20] common/sfc_efx/base: add ingress m-port RxQ flag Andrew Rybchenko
2021-07-02  8:39   ` [dpdk-dev] [PATCH v4 08/20] common/sfc_efx/base: add user mark " Andrew Rybchenko
2021-07-02  8:39   ` [dpdk-dev] [PATCH v4 09/20] net/sfc: add abstractions for the management EVQ identity Andrew Rybchenko
2021-07-02  8:39   ` [dpdk-dev] [PATCH v4 10/20] net/sfc: add support for initialising different RxQ types Andrew Rybchenko
2021-07-02  8:39   ` [dpdk-dev] [PATCH v4 11/20] net/sfc: add NUMA-aware registry of service logical cores Andrew Rybchenko
2021-07-02  8:39   ` [dpdk-dev] [PATCH v4 12/20] net/sfc: reserve RxQ for counters Andrew Rybchenko
2021-07-02  8:39   ` [dpdk-dev] [PATCH v4 13/20] common/sfc_efx/base: add counter creation MCDI wrappers Andrew Rybchenko
2021-07-02  8:39   ` [dpdk-dev] [PATCH v4 14/20] common/sfc_efx/base: add counter stream " Andrew Rybchenko
2021-07-02  8:39   ` [dpdk-dev] [PATCH v4 15/20] common/sfc_efx/base: support counter in action set Andrew Rybchenko
2021-07-02  8:39   ` [dpdk-dev] [PATCH v4 16/20] net/sfc: add Rx datapath method to get pushed buffers count Andrew Rybchenko
2021-07-02  8:39   ` [dpdk-dev] [PATCH v4 17/20] common/sfc_efx/base: add max MAE counters to limits Andrew Rybchenko
2021-07-02  8:39   ` [dpdk-dev] [PATCH v4 18/20] common/sfc_efx/base: add packetiser packet format definition Andrew Rybchenko
2021-07-02  8:39   ` [dpdk-dev] [PATCH v4 19/20] net/sfc: support flow action COUNT in transfer rules Andrew Rybchenko
2021-07-15 14:58     ` David Marchand
2021-07-15 18:30       ` Ivan Malov
2021-07-16 12:12     ` David Marchand
2021-07-02  8:39   ` [dpdk-dev] [PATCH v4 20/20] net/sfc: support flow API query for count actions Andrew Rybchenko
2021-07-20 12:19   ` [dpdk-dev] [PATCH v4 00/20] net/sfc: support flow API COUNT action David Marchand

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=20210527152510.1551026-12-andrew.rybchenko@oktetlabs.ru \
    --to=andrew.rybchenko@oktetlabs.ru \
    --cc=amoreton@xilinx.com \
    --cc=dev@dpdk.org \
    --cc=ivan.malov@oktetlabs.ru \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).