* [PATCH 0/4] enhance NFP service framework
@ 2024-01-24 9:25 Chaoyong He
2024-01-24 9:25 ` [PATCH 1/4] net/nfp: add synchronize module Chaoyong He
` (4 more replies)
0 siblings, 5 replies; 14+ messages in thread
From: Chaoyong He @ 2024-01-24 9:25 UTC (permalink / raw)
To: dev; +Cc: oss-drivers, Chaoyong He
Make multiple devices can use single core to run services for
flower firmware.
Also add synchronize module and service module to support it.
Long Wu (4):
net/nfp: add synchronize module
net/nfp: create new service code related files
net/nfp: flower driver uses one service core
net/nfp: fix stop cpp service problem
drivers/net/nfp/flower/nfp_flower.c | 56 +--
drivers/net/nfp/flower/nfp_flower_ctrl.c | 20 +-
drivers/net/nfp/flower/nfp_flower_ctrl.h | 2 +-
.../net/nfp/flower/nfp_flower_representor.c | 7 +
drivers/net/nfp/flower/nfp_flower_service.c | 196 +++++++++
drivers/net/nfp/flower/nfp_flower_service.h | 17 +
drivers/net/nfp/meson.build | 3 +
drivers/net/nfp/nfp_cpp_bridge.c | 91 +----
drivers/net/nfp/nfp_cpp_bridge.h | 1 -
drivers/net/nfp/nfp_ethdev.c | 31 +-
drivers/net/nfp/nfp_net_common.h | 15 +-
drivers/net/nfp/nfp_service.c | 117 ++++++
drivers/net/nfp/nfp_service.h | 20 +
drivers/net/nfp/nfpcore/nfp_sync.c | 382 ++++++++++++++++++
drivers/net/nfp/nfpcore/nfp_sync.h | 31 ++
15 files changed, 854 insertions(+), 135 deletions(-)
create mode 100644 drivers/net/nfp/flower/nfp_flower_service.c
create mode 100644 drivers/net/nfp/flower/nfp_flower_service.h
create mode 100644 drivers/net/nfp/nfp_service.c
create mode 100644 drivers/net/nfp/nfp_service.h
create mode 100644 drivers/net/nfp/nfpcore/nfp_sync.c
create mode 100644 drivers/net/nfp/nfpcore/nfp_sync.h
--
2.39.1
^ permalink raw reply [flat|nested] 14+ messages in thread
* [PATCH 1/4] net/nfp: add synchronize module
2024-01-24 9:25 [PATCH 0/4] enhance NFP service framework Chaoyong He
@ 2024-01-24 9:25 ` Chaoyong He
2024-01-24 9:25 ` [PATCH 2/4] net/nfp: create new service code related files Chaoyong He
` (3 subsequent siblings)
4 siblings, 0 replies; 14+ messages in thread
From: Chaoyong He @ 2024-01-24 9:25 UTC (permalink / raw)
To: dev; +Cc: oss-drivers, Long Wu, Chaoyong He, Peng Zhang
From: Long Wu <long.wu@corigine.com>
PMD has some information that needs synchronized:
1. Between devices in one process.
2. Between multiple processes for one device, before creating
"rte_eth_device".
3. Between multiple processes for one device, after creating
"rte_eth_device".
The final one is already support by framework
(eth_dev->data->dev_private), and this patch aims to support
the first two cases by adding a synchronize module.
Signed-off-by: Long Wu <long.wu@corigine.com>
Reviewed-by: Chaoyong He <chaoyong.he@corigine.com>
Reviewed-by: Peng Zhang <peng.zhang@corigine.com>
---
drivers/net/nfp/meson.build | 1 +
drivers/net/nfp/nfp_ethdev.c | 31 ++-
drivers/net/nfp/nfp_net_common.h | 4 +
drivers/net/nfp/nfpcore/nfp_sync.c | 382 +++++++++++++++++++++++++++++
drivers/net/nfp/nfpcore/nfp_sync.h | 29 +++
5 files changed, 444 insertions(+), 3 deletions(-)
create mode 100644 drivers/net/nfp/nfpcore/nfp_sync.c
create mode 100644 drivers/net/nfp/nfpcore/nfp_sync.h
diff --git a/drivers/net/nfp/meson.build b/drivers/net/nfp/meson.build
index 46be6f60cd..c625355d7a 100644
--- a/drivers/net/nfp/meson.build
+++ b/drivers/net/nfp/meson.build
@@ -26,6 +26,7 @@ sources = files(
'nfpcore/nfp_nsp_eth.c',
'nfpcore/nfp_resource.c',
'nfpcore/nfp_rtsym.c',
+ 'nfpcore/nfp_sync.c',
'nfpcore/nfp_target.c',
'nfpcore/nfp6000_pcie.c',
'nfp_cpp_bridge.c',
diff --git a/drivers/net/nfp/nfp_ethdev.c b/drivers/net/nfp/nfp_ethdev.c
index 07cd32881c..3ece806487 100644
--- a/drivers/net/nfp/nfp_ethdev.c
+++ b/drivers/net/nfp/nfp_ethdev.c
@@ -19,6 +19,7 @@
#include "nfpcore/nfp_nsp.h"
#include "nfpcore/nfp6000_pcie.h"
#include "nfpcore/nfp_resource.h"
+#include "nfpcore/nfp_sync.h"
#include "nfp_cpp_bridge.h"
#include "nfp_ipsec.h"
@@ -551,6 +552,7 @@ nfp_pf_uninit(struct nfp_pf_dev *pf_dev)
free(pf_dev->nfp_eth_table);
free(pf_dev->hwinfo);
nfp_cpp_free(pf_dev->cpp);
+ nfp_sync_free(pf_dev->sync);
rte_free(pf_dev);
}
@@ -559,6 +561,7 @@ nfp_pf_secondary_uninit(struct nfp_pf_dev *pf_dev)
{
free(pf_dev->sym_tbl);
nfp_cpp_free(pf_dev->cpp);
+ nfp_sync_free(pf_dev->sync);
rte_free(pf_dev);
return 0;
@@ -1612,6 +1615,7 @@ nfp_net_speed_capa_get(struct nfp_pf_dev *pf_dev,
static int
nfp_pf_init(struct rte_pci_device *pci_dev)
{
+ void *sync;
uint32_t i;
uint32_t id;
int ret = 0;
@@ -1652,6 +1656,13 @@ nfp_pf_init(struct rte_pci_device *pci_dev)
return -ENOMEM;
}
+ sync = nfp_sync_alloc();
+ if (sync == NULL) {
+ PMD_INIT_LOG(ERR, "Failed to alloc sync zone.");
+ ret = -ENOMEM;
+ goto pf_cleanup;
+ }
+
/*
* When device bound to UIO, the device could be used, by mistake,
* by two DPDK apps, and the UIO driver does not avoid it. This
@@ -1667,7 +1678,7 @@ nfp_pf_init(struct rte_pci_device *pci_dev)
if (cpp == NULL) {
PMD_INIT_LOG(ERR, "A CPP handle can not be obtained");
ret = -EIO;
- goto pf_cleanup;
+ goto sync_free;
}
hwinfo = nfp_hwinfo_read(cpp);
@@ -1734,6 +1745,7 @@ nfp_pf_init(struct rte_pci_device *pci_dev)
pf_dev->sym_tbl = sym_tbl;
pf_dev->pci_dev = pci_dev;
pf_dev->nfp_eth_table = nfp_eth_table;
+ pf_dev->sync = sync;
/* Get the speed capability */
for (i = 0; i < nfp_eth_table->count; i++) {
@@ -1815,6 +1827,8 @@ nfp_pf_init(struct rte_pci_device *pci_dev)
free(hwinfo);
cpp_cleanup:
nfp_cpp_free(cpp);
+sync_free:
+ nfp_sync_free(sync);
pf_cleanup:
rte_free(pf_dev);
@@ -1878,6 +1892,7 @@ nfp_secondary_init_app_fw_nic(struct nfp_pf_dev *pf_dev)
static int
nfp_pf_secondary_init(struct rte_pci_device *pci_dev)
{
+ void *sync;
int ret = 0;
struct nfp_cpp *cpp;
uint8_t function_id;
@@ -1910,6 +1925,13 @@ nfp_pf_secondary_init(struct rte_pci_device *pci_dev)
return -ENOMEM;
}
+ sync = nfp_sync_alloc();
+ if (sync == NULL) {
+ PMD_INIT_LOG(ERR, "Failed to alloc sync zone.");
+ ret = -ENOMEM;
+ goto pf_cleanup;
+ }
+
/*
* When device bound to UIO, the device could be used, by mistake,
* by two DPDK apps, and the UIO driver does not avoid it. This
@@ -1925,7 +1947,7 @@ nfp_pf_secondary_init(struct rte_pci_device *pci_dev)
if (cpp == NULL) {
PMD_INIT_LOG(ERR, "A CPP handle can not be obtained");
ret = -EIO;
- goto pf_cleanup;
+ goto sync_free;
}
/*
@@ -1936,7 +1958,7 @@ nfp_pf_secondary_init(struct rte_pci_device *pci_dev)
if (sym_tbl == NULL) {
PMD_INIT_LOG(ERR, "Something is wrong with the firmware symbol table");
ret = -EIO;
- goto pf_cleanup;
+ goto sync_free;
}
/* Read the app ID of the firmware loaded */
@@ -1954,6 +1976,7 @@ nfp_pf_secondary_init(struct rte_pci_device *pci_dev)
pf_dev->cpp = cpp;
pf_dev->sym_tbl = sym_tbl;
pf_dev->pci_dev = pci_dev;
+ pf_dev->sync = sync;
/* Call app specific init code now */
switch (app_fw_id) {
@@ -1983,6 +2006,8 @@ nfp_pf_secondary_init(struct rte_pci_device *pci_dev)
sym_tbl_cleanup:
free(sym_tbl);
+sync_free:
+ nfp_sync_free(sync);
pf_cleanup:
rte_free(pf_dev);
diff --git a/drivers/net/nfp/nfp_net_common.h b/drivers/net/nfp/nfp_net_common.h
index 66c900e3b8..110ca015c0 100644
--- a/drivers/net/nfp/nfp_net_common.h
+++ b/drivers/net/nfp/nfp_net_common.h
@@ -12,6 +12,7 @@
#include <nfp_dev.h>
#include <rte_spinlock.h>
+#include "nfpcore/nfp_sync.h"
#include "nfp_net_ctrl.h"
/* Interrupt definitions */
@@ -109,6 +110,9 @@ struct nfp_pf_dev {
/** Supported speeds bitmap */
uint32_t speed_capa;
+
+ /** Synchronized info */
+ struct nfp_sync *sync;
};
#define NFP_NET_FLOW_LIMIT 1024
diff --git a/drivers/net/nfp/nfpcore/nfp_sync.c b/drivers/net/nfp/nfpcore/nfp_sync.c
new file mode 100644
index 0000000000..8bc38d6585
--- /dev/null
+++ b/drivers/net/nfp/nfpcore/nfp_sync.c
@@ -0,0 +1,382 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright (c) 2024 Corigine, Inc.
+ * All rights reserved.
+ */
+
+#include "nfp_sync.h"
+
+#include <rte_dev.h>
+#include <rte_malloc.h>
+#include <rte_memzone.h>
+#include <rte_spinlock.h>
+
+#include "nfp_logs.h"
+
+#define NFP_SYNC_ELEMENT_MAX 8
+#define NFP_SYNC_PCI_MAX 32
+
+struct nfp_sync_element {
+ uint16_t count;
+ /** Element ID, use ASCII - SYN<> */
+ uint32_t magic;
+ void *handle;
+};
+
+struct nfp_sync_common {
+ char pci_name[PCI_PRI_STR_SIZE + 1];
+ uint16_t avail;
+ struct nfp_sync_element element[NFP_SYNC_ELEMENT_MAX];
+};
+
+struct nfp_sync {
+ rte_spinlock_t spinlock;
+
+ uint16_t alloc_count;
+
+ struct nfp_sync_common process;
+
+ struct nfp_sync_common pci[NFP_SYNC_PCI_MAX];
+
+ const struct rte_memzone *mz;
+};
+
+struct nfp_sync *
+nfp_sync_alloc(void)
+{
+ uint16_t i;
+ struct nfp_sync *sync;
+ const struct rte_memzone *mz;
+
+ mz = rte_memzone_lookup("nfp_sync");
+ if (mz != NULL) {
+ sync = mz->addr;
+ sync->alloc_count++;
+
+ return sync;
+ }
+
+ mz = rte_memzone_reserve("nfp_sync", sizeof(*sync), SOCKET_ID_ANY,
+ RTE_MEMZONE_SIZE_HINT_ONLY);
+ if (mz == NULL)
+ return NULL;
+
+ sync = mz->addr;
+
+ memset(sync, 0, sizeof(*sync));
+
+ rte_spinlock_init(&sync->spinlock);
+ sync->alloc_count = 1;
+ sync->mz = mz;
+
+ for (i = 0; i < NFP_SYNC_PCI_MAX; i++)
+ sync->pci[i].avail = NFP_SYNC_ELEMENT_MAX;
+
+ sync->process.avail = NFP_SYNC_ELEMENT_MAX;
+
+ return sync;
+}
+
+void
+nfp_sync_free(struct nfp_sync *sync)
+{
+ uint16_t i;
+
+ rte_spinlock_lock(&sync->spinlock);
+
+ sync->alloc_count--;
+ if (sync->alloc_count != 0) {
+ rte_spinlock_unlock(&sync->spinlock);
+ return;
+ }
+
+ if (sync->process.avail != NFP_SYNC_ELEMENT_MAX)
+ PMD_DRV_LOG(ERR, "Sync process handle residue");
+
+ for (i = 0; i < NFP_SYNC_PCI_MAX; i++) {
+ if (sync->pci[i].avail != NFP_SYNC_ELEMENT_MAX)
+ PMD_DRV_LOG(ERR, "Sync %s pci handle residue",
+ sync->pci[i].pci_name);
+ }
+
+ rte_spinlock_unlock(&sync->spinlock);
+
+ rte_memzone_free(sync->mz);
+}
+
+static void *
+nfp_sync_element_alloc(struct nfp_sync_element *element,
+ uint32_t magic,
+ uint32_t size)
+{
+ void *handle;
+
+ handle = rte_zmalloc(NULL, size, 0);
+ if (handle == NULL)
+ return NULL;
+
+ element->handle = handle;
+ element->count = 1;
+ element->magic = magic;
+
+ return handle;
+}
+
+static void
+nfp_sync_element_free(struct nfp_sync_element *element,
+ void *handle)
+{
+ element->count--;
+ if (element->count != 0)
+ return;
+
+ rte_free(handle);
+ element->handle = NULL;
+ element->magic = 0;
+}
+
+static void *
+nfp_sync_common_handle_alloc(struct nfp_sync_common *common,
+ uint32_t magic,
+ uint32_t size)
+{
+ uint16_t i;
+ void *handle = NULL;
+ uint16_t avail_slot = NFP_SYNC_ELEMENT_MAX;
+
+ for (i = 0; i < NFP_SYNC_ELEMENT_MAX; i++) {
+ if (common->element[i].magic != magic)
+ continue;
+
+ common->element[i].count++;
+
+ return common->element[i].handle;
+ }
+
+ if (common->avail == 0)
+ return NULL;
+
+ for (i = 0; i < NFP_SYNC_ELEMENT_MAX; i++) {
+ if (common->element[i].magic == 0) {
+ avail_slot = i;
+ break;
+ }
+ }
+
+ handle = nfp_sync_element_alloc(&common->element[avail_slot], magic, size);
+ if (handle == NULL)
+ return NULL;
+
+ common->avail--;
+
+ return handle;
+}
+
+static void
+nfp_sync_common_handle_free(struct nfp_sync_common *common,
+ void *handle)
+{
+ uint16_t i;
+
+ if (common->avail == NFP_SYNC_ELEMENT_MAX)
+ return;
+
+ for (i = 0; i < NFP_SYNC_ELEMENT_MAX; i++) {
+ if (common->element[i].handle == handle)
+ break;
+ }
+
+ if (i == NFP_SYNC_ELEMENT_MAX)
+ return;
+
+ nfp_sync_element_free(&common->element[i], handle);
+
+ if (common->element[i].count == 0)
+ common->avail++;
+}
+
+static void *
+nfp_sync_process_inner_handle_alloc(struct nfp_sync *sync,
+ uint32_t magic,
+ uint32_t size)
+{
+ void *handle = NULL;
+
+ rte_spinlock_lock(&sync->spinlock);
+
+ handle = nfp_sync_common_handle_alloc(&sync->process, magic, size);
+ if (handle == NULL)
+ PMD_DRV_LOG(ERR, "Process handle alloc failed");
+
+ rte_spinlock_unlock(&sync->spinlock);
+
+ return handle;
+}
+
+static void
+nfp_sync_process_inner_handle_free(struct nfp_sync *sync,
+ void *handle)
+{
+ rte_spinlock_lock(&sync->spinlock);
+
+ nfp_sync_common_handle_free(&sync->process, handle);
+
+ rte_spinlock_unlock(&sync->spinlock);
+}
+
+static uint16_t
+nfp_sync_process_handle_count_get(struct nfp_sync *sync,
+ void *handle)
+{
+ uint16_t i;
+ uint16_t count = 0;
+
+ rte_spinlock_lock(&sync->spinlock);
+
+ for (i = 0; i < NFP_SYNC_ELEMENT_MAX; i++) {
+ if (sync->process.element[i].handle == handle) {
+ count = sync->process.element[i].count;
+ break;
+ }
+ }
+
+ rte_spinlock_unlock(&sync->spinlock);
+
+ return count;
+}
+
+static void *
+nfp_sync_pci_inner_handle_alloc(struct nfp_sync *sync,
+ const char *pci_name,
+ uint32_t magic,
+ uint32_t size)
+{
+ uint16_t i;
+ void *handle = NULL;
+ uint16_t pci_avail_id = NFP_SYNC_PCI_MAX;
+
+ rte_spinlock_lock(&sync->spinlock);
+
+ for (i = 0; i < NFP_SYNC_PCI_MAX; i++) {
+ if (strcmp(pci_name, sync->pci[i].pci_name) == 0) {
+ pci_avail_id = i;
+ goto common_alloc;
+ }
+ }
+
+ for (i = 0; i < NFP_SYNC_PCI_MAX; i++) {
+ if (strlen(sync->pci[i].pci_name) == 0) {
+ pci_avail_id = i;
+ strcpy(sync->pci[pci_avail_id].pci_name, pci_name);
+ goto common_alloc;
+ }
+ }
+
+ rte_spinlock_unlock(&sync->spinlock);
+
+ return NULL;
+
+common_alloc:
+ handle = nfp_sync_common_handle_alloc(&sync->pci[pci_avail_id],
+ magic, size);
+ if (handle == NULL)
+ PMD_DRV_LOG(ERR, "PCI handle alloc failed");
+
+ rte_spinlock_unlock(&sync->spinlock);
+
+ return handle;
+}
+
+static void
+nfp_sync_pci_inner_handle_free(struct nfp_sync *sync,
+ const char *pci_name,
+ void *handle)
+{
+ uint16_t i;
+ char *name_tmp;
+
+ rte_spinlock_lock(&sync->spinlock);
+
+ for (i = 0; i < NFP_SYNC_PCI_MAX; i++) {
+ name_tmp = sync->pci[i].pci_name;
+ if (strlen(name_tmp) != 0 && strcmp(pci_name, name_tmp) == 0) {
+ nfp_sync_common_handle_free(&sync->pci[i], handle);
+ if (sync->pci[i].avail == NFP_SYNC_ELEMENT_MAX)
+ name_tmp[0] = 0;
+ break;
+ }
+ }
+
+ rte_spinlock_unlock(&sync->spinlock);
+}
+
+static uint16_t
+nfp_sync_pci_handle_count_get(struct nfp_sync *sync,
+ const char *pci_name,
+ void *handle)
+{
+ uint16_t i;
+ uint16_t count = 0;
+ struct nfp_sync_common *pci_common;
+
+ rte_spinlock_lock(&sync->spinlock);
+
+ for (i = 0; i < NFP_SYNC_PCI_MAX; i++) {
+ if (strcmp(sync->pci[i].pci_name, pci_name) == 0)
+ break;
+ }
+
+ if (i == NFP_SYNC_PCI_MAX) {
+ rte_spinlock_unlock(&sync->spinlock);
+ return 0;
+ }
+
+ pci_common = &sync->pci[i];
+
+ for (i = 0; i < NFP_SYNC_ELEMENT_MAX; i++) {
+ if (pci_common->element[i].handle == handle) {
+ count = pci_common->element[i].count;
+ break;
+ }
+ }
+
+ rte_spinlock_unlock(&sync->spinlock);
+
+ return count;
+}
+
+void *
+nfp_sync_handle_alloc(struct nfp_sync *sync,
+ struct rte_pci_device *pci_dev,
+ uint32_t magic,
+ uint32_t size)
+{
+ if (pci_dev == NULL)
+ return nfp_sync_process_inner_handle_alloc(sync, magic, size);
+
+ return nfp_sync_pci_inner_handle_alloc(sync, pci_dev->device.name,
+ magic, size);
+}
+
+void
+nfp_sync_handle_free(struct nfp_sync *sync,
+ struct rte_pci_device *pci_dev,
+ void *handle)
+{
+ if (pci_dev == NULL) {
+ nfp_sync_process_inner_handle_free(sync, handle);
+ return;
+ }
+
+ nfp_sync_pci_inner_handle_free(sync, pci_dev->device.name, handle);
+}
+
+uint16_t
+nfp_sync_handle_count_get(struct nfp_sync *sync,
+ struct rte_pci_device *pci_dev,
+ void *handle)
+{
+ if (pci_dev == NULL)
+ return nfp_sync_process_handle_count_get(sync, handle);
+
+ return nfp_sync_pci_handle_count_get(sync, pci_dev->device.name, handle);
+}
diff --git a/drivers/net/nfp/nfpcore/nfp_sync.h b/drivers/net/nfp/nfpcore/nfp_sync.h
new file mode 100644
index 0000000000..82f01e2652
--- /dev/null
+++ b/drivers/net/nfp/nfpcore/nfp_sync.h
@@ -0,0 +1,29 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright (c) 2024 Corigine, Inc.
+ * All rights reserved.
+ */
+
+#ifndef __NFP_SYNC_H__
+#define __NFP_SYNC_H__
+
+#include <stdint.h>
+
+#include <bus_pci_driver.h>
+
+struct nfp_sync;
+
+struct nfp_sync *nfp_sync_alloc(void);
+void nfp_sync_free(struct nfp_sync *sync);
+
+void *nfp_sync_handle_alloc(struct nfp_sync *sync,
+ struct rte_pci_device *pci_dev,
+ uint32_t magic,
+ uint32_t size);
+void nfp_sync_handle_free(struct nfp_sync *sync,
+ struct rte_pci_device *pci_dev,
+ void *handle);
+uint16_t nfp_sync_handle_count_get(struct nfp_sync *sync,
+ struct rte_pci_device *pci_dev,
+ void *handle);
+
+#endif /* __NFP_SYNC_H__ */
--
2.39.1
^ permalink raw reply [flat|nested] 14+ messages in thread
* [PATCH 2/4] net/nfp: create new service code related files
2024-01-24 9:25 [PATCH 0/4] enhance NFP service framework Chaoyong He
2024-01-24 9:25 ` [PATCH 1/4] net/nfp: add synchronize module Chaoyong He
@ 2024-01-24 9:25 ` Chaoyong He
2024-01-24 9:25 ` [PATCH 3/4] net/nfp: flower driver uses one service core Chaoyong He
` (2 subsequent siblings)
4 siblings, 0 replies; 14+ messages in thread
From: Chaoyong He @ 2024-01-24 9:25 UTC (permalink / raw)
To: dev; +Cc: oss-drivers, Long Wu, Chaoyong He, Peng Zhang
From: Long Wu <long.wu@corigine.com>
For clearer service code structure and more convenient future
addition of service code, NFP creates new service code related
files and move service related code into new files.
This commit also adds service disable interface and lets CPP
service run normally if app uses several flower cards.
Signed-off-by: Long Wu <long.wu@corigine.com>
Reviewed-by: Chaoyong He <chaoyong.he@corigine.com>
Reviewed-by: Peng Zhang <peng.zhang@corigine.com>
---
drivers/net/nfp/flower/nfp_flower.c | 19 ++---
drivers/net/nfp/meson.build | 1 +
drivers/net/nfp/nfp_cpp_bridge.c | 91 +++++-----------------
drivers/net/nfp/nfp_cpp_bridge.h | 1 -
drivers/net/nfp/nfp_net_common.h | 5 +-
drivers/net/nfp/nfp_service.c | 117 ++++++++++++++++++++++++++++
drivers/net/nfp/nfp_service.h | 20 +++++
7 files changed, 164 insertions(+), 90 deletions(-)
create mode 100644 drivers/net/nfp/nfp_service.c
create mode 100644 drivers/net/nfp/nfp_service.h
diff --git a/drivers/net/nfp/flower/nfp_flower.c b/drivers/net/nfp/flower/nfp_flower.c
index 88a1b061e1..a29edf8ca2 100644
--- a/drivers/net/nfp/flower/nfp_flower.c
+++ b/drivers/net/nfp/flower/nfp_flower.c
@@ -598,29 +598,20 @@ static int
nfp_flower_enable_services(struct nfp_app_fw_flower *app_fw_flower)
{
int ret;
- uint32_t service_id;
+ struct nfp_service_info info;
const struct rte_service_spec flower_service = {
.name = "flower_ctrl_vnic_service",
.callback = nfp_flower_ctrl_vnic_service,
.callback_userdata = (void *)app_fw_flower,
};
- /* Register the flower services */
- ret = rte_service_component_register(&flower_service, &service_id);
+ ret = nfp_service_enable(&flower_service, &info);
if (ret != 0) {
- PMD_INIT_LOG(ERR, "Could not register %s", flower_service.name);
- return -EINVAL;
+ PMD_INIT_LOG(ERR, "Could not enable service %s", flower_service.name);
+ return ret;
}
- app_fw_flower->ctrl_vnic_id = service_id;
- PMD_INIT_LOG(INFO, "%s registered", flower_service.name);
-
- /* Map them to available service cores */
- ret = nfp_map_service(service_id);
- if (ret != 0) {
- PMD_INIT_LOG(ERR, "Could not map %s", flower_service.name);
- return -EINVAL;
- }
+ app_fw_flower->ctrl_vnic_id = info.id;
return 0;
}
diff --git a/drivers/net/nfp/meson.build b/drivers/net/nfp/meson.build
index c625355d7a..0f4beccbdf 100644
--- a/drivers/net/nfp/meson.build
+++ b/drivers/net/nfp/meson.build
@@ -40,6 +40,7 @@ sources = files(
'nfp_net_ctrl.c',
'nfp_net_flow.c',
'nfp_rxtx.c',
+ 'nfp_service.c',
)
deps += ['hash', 'security', 'common_nfp']
diff --git a/drivers/net/nfp/nfp_cpp_bridge.c b/drivers/net/nfp/nfp_cpp_bridge.c
index 36dcdca9de..441c0d2843 100644
--- a/drivers/net/nfp/nfp_cpp_bridge.c
+++ b/drivers/net/nfp/nfp_cpp_bridge.c
@@ -8,10 +8,9 @@
#include <unistd.h>
#include <sys/ioctl.h>
-#include <rte_service_component.h>
-
#include "nfpcore/nfp_cpp.h"
#include "nfp_logs.h"
+#include "nfp_service.h"
#define NFP_CPP_MEMIO_BOUNDARY (1 << 20)
#define NFP_BRIDGE_OP_READ 20
@@ -24,82 +23,23 @@
/* Prototypes */
static int nfp_cpp_bridge_service_func(void *args);
-int
-nfp_map_service(uint32_t service_id)
-{
- int32_t ret;
- uint32_t slcore = 0;
- int32_t slcore_count;
- uint8_t service_count;
- const char *service_name;
- uint32_t slcore_array[RTE_MAX_LCORE];
- uint8_t min_service_count = UINT8_MAX;
-
- slcore_count = rte_service_lcore_list(slcore_array, RTE_MAX_LCORE);
- if (slcore_count <= 0) {
- PMD_INIT_LOG(DEBUG, "No service cores found");
- return -ENOENT;
- }
-
- /*
- * Find a service core with the least number of services already
- * registered to it.
- */
- while (slcore_count--) {
- service_count = rte_service_lcore_count_services(slcore_array[slcore_count]);
- if (service_count < min_service_count) {
- slcore = slcore_array[slcore_count];
- min_service_count = service_count;
- }
- }
-
- service_name = rte_service_get_name(service_id);
- PMD_INIT_LOG(INFO, "Mapping service %s to core %u", service_name, slcore);
-
- ret = rte_service_map_lcore_set(service_id, slcore, 1);
- if (ret != 0) {
- PMD_INIT_LOG(DEBUG, "Could not map flower service");
- return -ENOENT;
- }
-
- rte_service_runstate_set(service_id, 1);
- rte_service_component_runstate_set(service_id, 1);
- rte_service_lcore_start(slcore);
- if (rte_service_may_be_active(slcore) != 0)
- PMD_INIT_LOG(INFO, "The service %s is running", service_name);
- else
- PMD_INIT_LOG(ERR, "The service %s is not running", service_name);
-
- return 0;
-}
-
int
nfp_enable_cpp_service(struct nfp_pf_dev *pf_dev)
{
int ret;
- uint32_t service_id = 0;
+ const char *pci_name;
struct rte_service_spec cpp_service = {
- .name = "nfp_cpp_service",
- .callback = nfp_cpp_bridge_service_func,
+ .callback = nfp_cpp_bridge_service_func,
+ .callback_userdata = (void *)pf_dev,
};
- cpp_service.callback_userdata = (void *)pf_dev;
+ pci_name = strchr(pf_dev->pci_dev->name, ':') + 1;
+ snprintf(cpp_service.name, sizeof(cpp_service.name), "%s_cpp_service", pci_name);
- /* Register the cpp service */
- ret = rte_service_component_register(&cpp_service, &service_id);
+ ret = nfp_service_enable(&cpp_service, &pf_dev->cpp_service_info);
if (ret != 0) {
- PMD_INIT_LOG(WARNING, "Could not register nfp cpp service");
- return -EINVAL;
- }
-
- pf_dev->cpp_bridge_id = service_id;
- PMD_INIT_LOG(INFO, "NFP cpp service registered");
-
- /* Map it to available service core */
- ret = nfp_map_service(service_id);
- if (ret != 0) {
- PMD_INIT_LOG(DEBUG, "Could not map nfp cpp service");
- return -EINVAL;
+ PMD_INIT_LOG(DEBUG, "Could not enable service %s", cpp_service.name);
+ return ret;
}
return 0;
@@ -387,12 +327,18 @@ nfp_cpp_bridge_service_func(void *args)
int sockfd;
int datafd;
struct nfp_cpp *cpp;
+ const char *pci_name;
+ char socket_handle[14];
struct sockaddr address;
struct nfp_pf_dev *pf_dev;
struct timeval timeout = {1, 0};
- unlink("/tmp/nfp_cpp");
+ pf_dev = args;
+
+ pci_name = strchr(pf_dev->pci_dev->name, ':') + 1;
+ snprintf(socket_handle, sizeof(socket_handle), "/tmp/%s", pci_name);
+ unlink(socket_handle);
sockfd = socket(AF_UNIX, SOCK_STREAM, 0);
if (sockfd < 0) {
PMD_CPP_LOG(ERR, "socket creation error. Service failed");
@@ -404,7 +350,7 @@ nfp_cpp_bridge_service_func(void *args)
memset(&address, 0, sizeof(struct sockaddr));
address.sa_family = AF_UNIX;
- strcpy(address.sa_data, "/tmp/nfp_cpp");
+ strcpy(address.sa_data, socket_handle);
ret = bind(sockfd, (const struct sockaddr *)&address,
sizeof(struct sockaddr));
@@ -421,9 +367,8 @@ nfp_cpp_bridge_service_func(void *args)
return ret;
}
- pf_dev = args;
cpp = pf_dev->cpp;
- while (rte_service_runstate_get(pf_dev->cpp_bridge_id) != 0) {
+ while (rte_service_runstate_get(pf_dev->cpp_service_info.id) != 0) {
datafd = accept(sockfd, NULL, NULL);
if (datafd < 0) {
if (errno == EAGAIN || errno == EWOULDBLOCK)
diff --git a/drivers/net/nfp/nfp_cpp_bridge.h b/drivers/net/nfp/nfp_cpp_bridge.h
index bf975ce7ba..406c961dce 100644
--- a/drivers/net/nfp/nfp_cpp_bridge.h
+++ b/drivers/net/nfp/nfp_cpp_bridge.h
@@ -9,6 +9,5 @@
#include "nfp_net_common.h"
int nfp_enable_cpp_service(struct nfp_pf_dev *pf_dev);
-int nfp_map_service(uint32_t service_id);
#endif /* __NFP_CPP_BRIDGE_H__ */
diff --git a/drivers/net/nfp/nfp_net_common.h b/drivers/net/nfp/nfp_net_common.h
index 110ca015c0..35707ad5bd 100644
--- a/drivers/net/nfp/nfp_net_common.h
+++ b/drivers/net/nfp/nfp_net_common.h
@@ -14,6 +14,7 @@
#include "nfpcore/nfp_sync.h"
#include "nfp_net_ctrl.h"
+#include "nfp_service.h"
/* Interrupt definitions */
#define NFP_NET_IRQ_LSC_IDX 0
@@ -102,8 +103,8 @@ struct nfp_pf_dev {
struct nfp_hwinfo *hwinfo;
struct nfp_rtsym_table *sym_tbl;
- /** Service id of cpp bridge service */
- uint32_t cpp_bridge_id;
+ /** Service info of cpp bridge service */
+ struct nfp_service_info cpp_service_info;
/** Multiple PF configuration */
struct nfp_multi_pf multi_pf;
diff --git a/drivers/net/nfp/nfp_service.c b/drivers/net/nfp/nfp_service.c
new file mode 100644
index 0000000000..f49fa6addf
--- /dev/null
+++ b/drivers/net/nfp/nfp_service.c
@@ -0,0 +1,117 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright (c) 2023 Corigine, Inc.
+ * All rights reserved.
+ */
+
+#include "nfp_service.h"
+
+#include "nfpcore/nfp_cpp.h"
+#include "nfp_logs.h"
+
+/* Disable service and try to get service status */
+#define NFP_SERVICE_DISABLE_WAIT_COUNT 3000
+
+static int
+nfp_map_service(struct nfp_service_info *info)
+{
+ int32_t ret;
+ uint32_t slcore = 0;
+ int32_t slcore_count;
+ uint8_t service_count;
+ const char *service_name;
+ uint32_t slcore_array[RTE_MAX_LCORE];
+ uint8_t min_service_count = UINT8_MAX;
+
+ slcore_count = rte_service_lcore_list(slcore_array, RTE_MAX_LCORE);
+ if (slcore_count <= 0) {
+ PMD_DRV_LOG(DEBUG, "No service cores found");
+ return -ENOENT;
+ }
+
+ /*
+ * Find a service core with the least number of services already
+ * registered to it
+ */
+ while (slcore_count--) {
+ service_count = rte_service_lcore_count_services(slcore_array[slcore_count]);
+ if (service_count < min_service_count) {
+ slcore = slcore_array[slcore_count];
+ min_service_count = service_count;
+ }
+ }
+
+ service_name = rte_service_get_name(info->id);
+ PMD_INIT_LOG(INFO, "Mapping service %s to core %u", service_name, slcore);
+
+ ret = rte_service_map_lcore_set(info->id, slcore, 1);
+ if (ret != 0) {
+ PMD_DRV_LOG(DEBUG, "Could not map flower service");
+ return -ENOENT;
+ }
+
+ rte_service_runstate_set(info->id, 1);
+ rte_service_component_runstate_set(info->id, 1);
+ rte_service_lcore_start(slcore);
+ if (rte_service_may_be_active(slcore))
+ PMD_DRV_LOG(INFO, "The service %s is running", service_name);
+ else
+ PMD_DRV_LOG(ERR, "The service %s is not running", service_name);
+
+ info->lcore = slcore;
+
+ return 0;
+}
+
+int
+nfp_service_enable(const struct rte_service_spec *service_spec,
+ struct nfp_service_info *info)
+{
+ int ret;
+
+ /* Register the service */
+ ret = rte_service_component_register(service_spec, &info->id);
+ if (ret != 0) {
+ PMD_DRV_LOG(DEBUG, "Could not register %s", service_spec->name);
+ return -EINVAL;
+ }
+
+ /* Map it to available service core */
+ ret = nfp_map_service(info);
+ if (ret != 0) {
+ PMD_DRV_LOG(DEBUG, "Could not map %s", service_spec->name);
+ return -EINVAL;
+ }
+
+ PMD_DRV_LOG(DEBUG, "Enable service %s successfully", service_spec->name);
+
+ return 0;
+}
+
+int
+nfp_service_disable(struct nfp_service_info *info)
+{
+ uint32_t i;
+ const char *service_name;
+
+ service_name = rte_service_get_name(info->id);
+ if (service_name == NULL) {
+ PMD_DRV_LOG(ERR, "Could not find service %u", info->id);
+ return -EINVAL;
+ }
+
+ rte_service_runstate_set(info->id, 0);
+ rte_service_component_runstate_set(info->id, 0);
+
+ for (i = 0; i < NFP_SERVICE_DISABLE_WAIT_COUNT; i++) {
+ if (rte_service_may_be_active(info->id) == 0)
+ break;
+ rte_delay_ms(1);
+ }
+ if (i == NFP_SERVICE_DISABLE_WAIT_COUNT)
+ PMD_DRV_LOG(ERR, "Could not stop service %s", service_name);
+
+ rte_service_map_lcore_set(info->id, info->lcore, 0);
+ rte_service_component_unregister(info->id);
+
+ return 0;
+}
diff --git a/drivers/net/nfp/nfp_service.h b/drivers/net/nfp/nfp_service.h
new file mode 100644
index 0000000000..4b99d5b973
--- /dev/null
+++ b/drivers/net/nfp/nfp_service.h
@@ -0,0 +1,20 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright (c) 2023 Corigine, Inc.
+ * All rights reserved.
+ */
+
+#ifndef __NFP_SERVICE_H__
+#define __NFP_SERVICE_H__
+
+#include <rte_service_component.h>
+
+struct nfp_service_info {
+ uint32_t id;
+ uint32_t lcore;
+};
+
+int nfp_service_disable(struct nfp_service_info *info);
+int nfp_service_enable(const struct rte_service_spec *service_spec,
+ struct nfp_service_info *info);
+
+#endif /* __NFP_SERVICE_H__ */
--
2.39.1
^ permalink raw reply [flat|nested] 14+ messages in thread
* [PATCH 3/4] net/nfp: flower driver uses one service core
2024-01-24 9:25 [PATCH 0/4] enhance NFP service framework Chaoyong He
2024-01-24 9:25 ` [PATCH 1/4] net/nfp: add synchronize module Chaoyong He
2024-01-24 9:25 ` [PATCH 2/4] net/nfp: create new service code related files Chaoyong He
@ 2024-01-24 9:25 ` Chaoyong He
2024-01-24 9:25 ` [PATCH 4/4] net/nfp: fix stop cpp service problem Chaoyong He
2024-02-02 3:04 ` [PATCH v2 0/3] enhance NFP service framework Chaoyong He
4 siblings, 0 replies; 14+ messages in thread
From: Chaoyong He @ 2024-01-24 9:25 UTC (permalink / raw)
To: dev; +Cc: oss-drivers, Long Wu, Chaoyong He, Peng Zhang
From: Long Wu <long.wu@corigine.com>
Now app must provide a service core for each NFP pci device that uses
flower driver to run flower service, which will cause huge resource waste.
This patch is to solve this problem and app can only use one core to
run NFP flower service though it uses several NFP NICs.
Signed-off-by: Long Wu <long.wu@corigine.com>
Reviewed-by: Chaoyong He <chaoyong.he@corigine.com>
Reviewed-by: Peng Zhang <peng.zhang@corigine.com>
---
drivers/net/nfp/flower/nfp_flower.c | 47 ++---
drivers/net/nfp/flower/nfp_flower_ctrl.c | 20 +-
drivers/net/nfp/flower/nfp_flower_ctrl.h | 2 +-
.../net/nfp/flower/nfp_flower_representor.c | 4 +
drivers/net/nfp/flower/nfp_flower_service.c | 196 ++++++++++++++++++
drivers/net/nfp/flower/nfp_flower_service.h | 17 ++
drivers/net/nfp/meson.build | 1 +
drivers/net/nfp/nfp_net_common.h | 6 +
drivers/net/nfp/nfpcore/nfp_sync.h | 2 +
9 files changed, 248 insertions(+), 47 deletions(-)
create mode 100644 drivers/net/nfp/flower/nfp_flower_service.c
create mode 100644 drivers/net/nfp/flower/nfp_flower_service.h
diff --git a/drivers/net/nfp/flower/nfp_flower.c b/drivers/net/nfp/flower/nfp_flower.c
index a29edf8ca2..e84e6ebbff 100644
--- a/drivers/net/nfp/flower/nfp_flower.c
+++ b/drivers/net/nfp/flower/nfp_flower.c
@@ -18,6 +18,7 @@
#include "../nfp_mtr.h"
#include "nfp_flower_ctrl.h"
#include "nfp_flower_representor.h"
+#include "nfp_flower_service.h"
#define CTRL_VNIC_NB_DESC 512
@@ -461,6 +462,13 @@ nfp_flower_init_ctrl_vnic(struct nfp_net_hw *hw)
nn_cfg_writeb(&hw->super, NFP_NET_CFG_TXR_SZ(i), rte_log2_u32(CTRL_VNIC_NB_DESC));
}
+ /* Alloc sync memory zone */
+ ret = nfp_flower_service_sync_alloc(app_fw_flower);
+ if (ret != 0) {
+ PMD_INIT_LOG(ERR, "Alloc sync memory zone failed");
+ goto tx_queue_setup_cleanup;
+ }
+
return 0;
tx_queue_setup_cleanup:
@@ -531,6 +539,7 @@ nfp_flower_cleanup_ctrl_vnic(struct nfp_net_hw *hw)
}
}
+ nfp_flower_service_sync_free(app_fw_flower);
rte_free(eth_dev->data->tx_queues);
rte_free(eth_dev->data->rx_queues);
rte_mempool_free(app_fw_flower->ctrl_pktmbuf_pool);
@@ -584,38 +593,6 @@ nfp_flower_start_ctrl_vnic(struct nfp_net_hw *net_hw)
return 0;
}
-static int
-nfp_flower_ctrl_vnic_service(void *arg)
-{
- struct nfp_app_fw_flower *app_fw_flower = arg;
-
- nfp_flower_ctrl_vnic_poll(app_fw_flower);
-
- return 0;
-}
-
-static int
-nfp_flower_enable_services(struct nfp_app_fw_flower *app_fw_flower)
-{
- int ret;
- struct nfp_service_info info;
- const struct rte_service_spec flower_service = {
- .name = "flower_ctrl_vnic_service",
- .callback = nfp_flower_ctrl_vnic_service,
- .callback_userdata = (void *)app_fw_flower,
- };
-
- ret = nfp_service_enable(&flower_service, &info);
- if (ret != 0) {
- PMD_INIT_LOG(ERR, "Could not enable service %s", flower_service.name);
- return ret;
- }
-
- app_fw_flower->ctrl_vnic_id = info.id;
-
- return 0;
-}
-
static void
nfp_flower_pkt_add_metadata_register(struct nfp_app_fw_flower *app_fw_flower)
{
@@ -760,7 +737,7 @@ nfp_init_app_fw_flower(struct nfp_pf_dev *pf_dev,
}
/* Start up flower services */
- ret = nfp_flower_enable_services(app_fw_flower);
+ ret = nfp_flower_service_start(app_fw_flower);
if (ret != 0) {
PMD_INIT_LOG(ERR, "Could not enable flower services");
ret = -ESRCH;
@@ -770,11 +747,13 @@ nfp_init_app_fw_flower(struct nfp_pf_dev *pf_dev,
ret = nfp_flower_repr_create(app_fw_flower);
if (ret != 0) {
PMD_INIT_LOG(ERR, "Could not create representor ports");
- goto ctrl_vnic_cleanup;
+ goto ctrl_vnic_service_stop;
}
return 0;
+ctrl_vnic_service_stop:
+ nfp_flower_service_stop(app_fw_flower);
ctrl_vnic_cleanup:
nfp_flower_cleanup_ctrl_vnic(app_fw_flower->ctrl_hw);
ctrl_cpp_area_cleanup:
diff --git a/drivers/net/nfp/flower/nfp_flower_ctrl.c b/drivers/net/nfp/flower/nfp_flower_ctrl.c
index c25487c277..bcb325d475 100644
--- a/drivers/net/nfp/flower/nfp_flower_ctrl.c
+++ b/drivers/net/nfp/flower/nfp_flower_ctrl.c
@@ -12,6 +12,7 @@
#include "../nfp_logs.h"
#include "nfp_flower_representor.h"
#include "nfp_mtr.h"
+#include "nfp_flower_service.h"
#define MAX_PKT_BURST 32
@@ -502,26 +503,21 @@ nfp_flower_cmsg_rx(struct nfp_app_fw_flower *app_fw_flower,
}
void
-nfp_flower_ctrl_vnic_poll(struct nfp_app_fw_flower *app_fw_flower)
+nfp_flower_ctrl_vnic_process(struct nfp_app_fw_flower *app_fw_flower)
{
uint16_t count;
struct nfp_net_rxq *rxq;
- struct nfp_net_hw *ctrl_hw;
struct rte_eth_dev *ctrl_eth_dev;
struct rte_mbuf *pkts_burst[MAX_PKT_BURST];
- ctrl_hw = app_fw_flower->ctrl_hw;
- ctrl_eth_dev = ctrl_hw->eth_dev;
+ ctrl_eth_dev = app_fw_flower->ctrl_hw->eth_dev;
/* Ctrl vNIC only has a single Rx queue */
rxq = ctrl_eth_dev->data->rx_queues[0];
-
- while (rte_service_runstate_get(app_fw_flower->ctrl_vnic_id) != 0) {
- count = nfp_flower_ctrl_vnic_recv(rxq, pkts_burst, MAX_PKT_BURST);
- if (count != 0) {
- app_fw_flower->ctrl_vnic_rx_count += count;
- /* Process cmsgs here */
- nfp_flower_cmsg_rx(app_fw_flower, pkts_burst, count);
- }
+ count = nfp_flower_ctrl_vnic_recv(rxq, pkts_burst, MAX_PKT_BURST);
+ if (count != 0) {
+ app_fw_flower->ctrl_vnic_rx_count += count;
+ /* Process cmsgs here */
+ nfp_flower_cmsg_rx(app_fw_flower, pkts_burst, count);
}
}
diff --git a/drivers/net/nfp/flower/nfp_flower_ctrl.h b/drivers/net/nfp/flower/nfp_flower_ctrl.h
index 4c94d36847..131d002ac6 100644
--- a/drivers/net/nfp/flower/nfp_flower_ctrl.h
+++ b/drivers/net/nfp/flower/nfp_flower_ctrl.h
@@ -8,7 +8,7 @@
#include "nfp_flower.h"
-void nfp_flower_ctrl_vnic_poll(struct nfp_app_fw_flower *app_fw_flower);
+void nfp_flower_ctrl_vnic_process(struct nfp_app_fw_flower *app_fw_flower);
uint16_t nfp_flower_ctrl_vnic_xmit(struct nfp_app_fw_flower *app_fw_flower,
struct rte_mbuf *mbuf);
void nfp_flower_ctrl_vnic_xmit_register(struct nfp_app_fw_flower *app_fw_flower);
diff --git a/drivers/net/nfp/flower/nfp_flower_representor.c b/drivers/net/nfp/flower/nfp_flower_representor.c
index 5f7c1fa737..bf67dc3ba2 100644
--- a/drivers/net/nfp/flower/nfp_flower_representor.c
+++ b/drivers/net/nfp/flower/nfp_flower_representor.c
@@ -9,6 +9,7 @@
#include "../nfpcore/nfp_nsp.h"
#include "../nfp_logs.h"
#include "../nfp_mtr.h"
+#include "nfp_flower_service.h"
/* Type of representor */
enum nfp_repr_type {
@@ -396,6 +397,9 @@ nfp_flower_repr_dev_close(struct rte_eth_dev *dev)
if (app_fw_flower->pf_repr != NULL)
return 0;
+ /* Stop flower service first */
+ nfp_flower_service_stop(app_fw_flower);
+
/* Now it is safe to free all PF resources */
nfp_uninit_app_fw_flower(pf_dev);
nfp_pf_uninit(pf_dev);
diff --git a/drivers/net/nfp/flower/nfp_flower_service.c b/drivers/net/nfp/flower/nfp_flower_service.c
new file mode 100644
index 0000000000..5e8811fe24
--- /dev/null
+++ b/drivers/net/nfp/flower/nfp_flower_service.c
@@ -0,0 +1,196 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright (c) 2024 Corigine, Inc.
+ * All rights reserved.
+ */
+
+#include "nfp_flower_service.h"
+
+#include <rte_spinlock.h>
+
+#include "nfp_flower_ctrl.h"
+#include "../nfpcore/nfp_cpp.h"
+#include "../nfpcore/nfp_sync.h"
+#include "../nfp_logs.h"
+#include "../nfp_service.h"
+
+/* Driver limitation, PMD can enlarge it if need. */
+#define MAX_FLOWER_SERVICE_SLOT 8
+
+struct nfp_flower_service {
+ /** Flower service is enabled */
+ bool enabled;
+ /** Flower service info */
+ struct nfp_service_info info;
+ /** Store flower cards' information */
+ struct nfp_app_fw_flower *slots[MAX_FLOWER_SERVICE_SLOT];
+ /** Spinlock for sync slots when add/remove card */
+ rte_spinlock_t spinlock;
+};
+
+static struct nfp_flower_service *
+nfp_flower_service_handle_get(struct nfp_app_fw_flower *app)
+{
+ return app->pf_hw->pf_dev->process_share.fl_service;
+}
+
+static int
+nfp_flower_service_loop(void *arg)
+{
+ uint16_t slot;
+ struct nfp_app_fw_flower *app;
+ struct nfp_flower_service *service_handle;
+
+ service_handle = arg;
+ /* Waiting for enabling service */
+ while (!service_handle->enabled)
+ rte_delay_ms(1);
+
+ while (rte_service_runstate_get(service_handle->info.id) != 0) {
+ rte_spinlock_lock(&service_handle->spinlock);
+ for (slot = 0; slot < MAX_FLOWER_SERVICE_SLOT; slot++) {
+ app = service_handle->slots[slot];
+ if (app == NULL)
+ continue;
+
+ nfp_flower_ctrl_vnic_process(app);
+ }
+ rte_spinlock_unlock(&service_handle->spinlock);
+ }
+
+ return 0;
+}
+
+static int
+nfp_flower_service_enable(struct nfp_flower_service *service_handle)
+{
+ int ret;
+
+ const struct rte_service_spec flower_service = {
+ .name = "flower_ctrl_vnic_service",
+ .callback = nfp_flower_service_loop,
+ .callback_userdata = (void *)service_handle,
+ };
+
+ ret = nfp_service_enable(&flower_service, &service_handle->info);
+ if (ret != 0)
+ return ret;
+
+ rte_spinlock_init(&service_handle->spinlock);
+ service_handle->enabled = true;
+
+ return 0;
+}
+
+static uint16_t
+nfp_flower_service_insert(struct nfp_app_fw_flower *app,
+ struct nfp_flower_service *service_handle)
+{
+ uint16_t slot;
+
+ rte_spinlock_lock(&service_handle->spinlock);
+ for (slot = 0; slot < MAX_FLOWER_SERVICE_SLOT; slot++) {
+ if (service_handle->slots[slot] == NULL) {
+ service_handle->slots[slot] = app;
+ break;
+ }
+ }
+ rte_spinlock_unlock(&service_handle->spinlock);
+
+ return slot;
+}
+
+int
+nfp_flower_service_start(void *app_fw_flower)
+{
+ int ret;
+ struct nfp_flower_service *service_handle;
+ struct nfp_app_fw_flower *app = app_fw_flower;
+
+ service_handle = nfp_flower_service_handle_get(app);
+ if (service_handle == NULL) {
+ PMD_DRV_LOG(ERR, "Can not get service handle");
+ return -EINVAL;
+ }
+
+ /* Enable flower service when driver initializes the first NIC */
+ if (!service_handle->enabled) {
+ ret = nfp_flower_service_enable(service_handle);
+ if (ret != 0) {
+ PMD_DRV_LOG(ERR, "Could not enable flower service");
+ return -ESRCH;
+ }
+ }
+
+ /* Insert the NIC to flower service slot */
+ ret = nfp_flower_service_insert(app, service_handle);
+ if (ret == MAX_FLOWER_SERVICE_SLOT) {
+ PMD_DRV_LOG(ERR, "Flower ctrl vnic service slot over %u",
+ MAX_FLOWER_SERVICE_SLOT);
+ return -ENOSPC;
+ }
+
+ return 0;
+}
+
+void
+nfp_flower_service_stop(void *app_fw_flower)
+{
+ uint16_t slot;
+ uint16_t count;
+ struct nfp_flower_service *service_handle;
+ struct nfp_app_fw_flower *app = app_fw_flower;
+
+ service_handle = nfp_flower_service_handle_get(app);
+ if (service_handle == NULL) {
+ PMD_DRV_LOG(ERR, "Can not get service handle");
+ return;
+ }
+
+ rte_spinlock_lock(&service_handle->spinlock);
+ for (slot = 0; slot < MAX_FLOWER_SERVICE_SLOT; slot++) {
+ /* The app only in one slot */
+ if (service_handle->slots[slot] != app)
+ continue;
+
+ service_handle->slots[slot] = NULL;
+ }
+ rte_spinlock_unlock(&service_handle->spinlock);
+
+ /* Determine whether to disable service */
+ count = nfp_sync_handle_count_get(app->pf_hw->pf_dev->sync, NULL,
+ service_handle);
+ if (count > 1)
+ return;
+
+ if (nfp_service_disable(&service_handle->info) != 0)
+ PMD_DRV_LOG(ERR, "Could not disable service");
+}
+
+int
+nfp_flower_service_sync_alloc(void *app_fw_flower)
+{
+ struct nfp_flower_service *service_handle;
+ struct nfp_app_fw_flower *app = app_fw_flower;
+ struct nfp_pf_dev *pf_dev = app->pf_hw->pf_dev;
+
+ service_handle = nfp_sync_handle_alloc(pf_dev->sync, NULL,
+ NFP_SYNC_MAGIC_FL_SERVICE,
+ sizeof(struct nfp_flower_service));
+ if (service_handle == NULL)
+ return -ENOMEM;
+
+ pf_dev->process_share.fl_service = service_handle;
+
+ return 0;
+}
+
+void
+nfp_flower_service_sync_free(void *app_fw_flower)
+{
+ struct nfp_app_fw_flower *app = app_fw_flower;
+ struct nfp_pf_dev *pf_dev = app->pf_hw->pf_dev;
+
+ nfp_sync_handle_free(pf_dev->sync, NULL, pf_dev->process_share.fl_service);
+
+ pf_dev->process_share.fl_service = NULL;
+}
diff --git a/drivers/net/nfp/flower/nfp_flower_service.h b/drivers/net/nfp/flower/nfp_flower_service.h
new file mode 100644
index 0000000000..a75780274f
--- /dev/null
+++ b/drivers/net/nfp/flower/nfp_flower_service.h
@@ -0,0 +1,17 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright (c) 2024 Corigine, Inc.
+ * All rights reserved.
+ */
+
+#ifndef __NFP_FLOWER_SERVICE_H__
+#define __NFP_FLOWER_SERVICE_H__
+
+struct nfp_flower_service;
+
+int nfp_flower_service_start(void *app_fw_flower);
+void nfp_flower_service_stop(void *app_fw_flower);
+
+int nfp_flower_service_sync_alloc(void *app_fw_flower);
+void nfp_flower_service_sync_free(void *app_fw_flower);
+
+#endif /* __NFP_FLOWER_SERVICE_H__ */
diff --git a/drivers/net/nfp/meson.build b/drivers/net/nfp/meson.build
index 0f4beccbdf..e376fd328f 100644
--- a/drivers/net/nfp/meson.build
+++ b/drivers/net/nfp/meson.build
@@ -13,6 +13,7 @@ sources = files(
'flower/nfp_flower_ctrl.c',
'flower/nfp_flower_flow.c',
'flower/nfp_flower_representor.c',
+ 'flower/nfp_flower_service.c',
'nfd3/nfp_nfd3_dp.c',
'nfdk/nfp_nfdk_dp.c',
'nfpcore/nfp_cppcore.c',
diff --git a/drivers/net/nfp/nfp_net_common.h b/drivers/net/nfp/nfp_net_common.h
index 35707ad5bd..5ba3075dd1 100644
--- a/drivers/net/nfp/nfp_net_common.h
+++ b/drivers/net/nfp/nfp_net_common.h
@@ -12,6 +12,7 @@
#include <nfp_dev.h>
#include <rte_spinlock.h>
+#include "flower/nfp_flower_service.h"
#include "nfpcore/nfp_sync.h"
#include "nfp_net_ctrl.h"
#include "nfp_service.h"
@@ -80,6 +81,10 @@ struct nfp_multi_pf {
uint8_t *beat_addr;
};
+struct nfp_process_share {
+ struct nfp_flower_service *fl_service;
+};
+
struct nfp_pf_dev {
/** Backpointer to associated pci device */
struct rte_pci_device *pci_dev;
@@ -114,6 +119,7 @@ struct nfp_pf_dev {
/** Synchronized info */
struct nfp_sync *sync;
+ struct nfp_process_share process_share;
};
#define NFP_NET_FLOW_LIMIT 1024
diff --git a/drivers/net/nfp/nfpcore/nfp_sync.h b/drivers/net/nfp/nfpcore/nfp_sync.h
index 82f01e2652..ff3f60937e 100644
--- a/drivers/net/nfp/nfpcore/nfp_sync.h
+++ b/drivers/net/nfp/nfpcore/nfp_sync.h
@@ -10,6 +10,8 @@
#include <bus_pci_driver.h>
+#define NFP_SYNC_MAGIC_FL_SERVICE 0x53594e41 /**< ASCII - SYNA */
+
struct nfp_sync;
struct nfp_sync *nfp_sync_alloc(void);
--
2.39.1
^ permalink raw reply [flat|nested] 14+ messages in thread
* [PATCH 4/4] net/nfp: fix stop cpp service problem
2024-01-24 9:25 [PATCH 0/4] enhance NFP service framework Chaoyong He
` (2 preceding siblings ...)
2024-01-24 9:25 ` [PATCH 3/4] net/nfp: flower driver uses one service core Chaoyong He
@ 2024-01-24 9:25 ` Chaoyong He
2024-01-30 12:07 ` Ferruh Yigit
2024-02-02 3:04 ` [PATCH v2 0/3] enhance NFP service framework Chaoyong He
4 siblings, 1 reply; 14+ messages in thread
From: Chaoyong He @ 2024-01-24 9:25 UTC (permalink / raw)
To: dev; +Cc: oss-drivers, Long Wu, chaoyong.he, stable, Peng Zhang
From: Long Wu <long.wu@corigine.com>
Stop cpp service if all representors are closed and cpp service is running.
Fixes: bab0e6f48b6d ("net/nfp: fix infinite loop")
Cc: chaoyong.he@corigine.com
Cc: stable@dpdk.org
Signed-off-by: Long Wu <long.wu@corigine.com>
Reviewed-by: Chaoyong He <chaoyong.he@corigine.com>
Reviewed-by: Peng Zhang <peng.zhang@corigine.com>
---
drivers/net/nfp/flower/nfp_flower_representor.c | 3 +++
1 file changed, 3 insertions(+)
diff --git a/drivers/net/nfp/flower/nfp_flower_representor.c b/drivers/net/nfp/flower/nfp_flower_representor.c
index bf67dc3ba2..2e8eb13e4e 100644
--- a/drivers/net/nfp/flower/nfp_flower_representor.c
+++ b/drivers/net/nfp/flower/nfp_flower_representor.c
@@ -400,6 +400,9 @@ nfp_flower_repr_dev_close(struct rte_eth_dev *dev)
/* Stop flower service first */
nfp_flower_service_stop(app_fw_flower);
+ /* Disable cpp service if need */
+ nfp_service_disable(&pf_dev->cpp_service_info);
+
/* Now it is safe to free all PF resources */
nfp_uninit_app_fw_flower(pf_dev);
nfp_pf_uninit(pf_dev);
--
2.39.1
^ permalink raw reply [flat|nested] 14+ messages in thread
* Re: [PATCH 4/4] net/nfp: fix stop cpp service problem
2024-01-24 9:25 ` [PATCH 4/4] net/nfp: fix stop cpp service problem Chaoyong He
@ 2024-01-30 12:07 ` Ferruh Yigit
2024-01-30 12:17 ` Kevin Traynor
0 siblings, 1 reply; 14+ messages in thread
From: Ferruh Yigit @ 2024-01-30 12:07 UTC (permalink / raw)
To: Chaoyong He, dev
Cc: oss-drivers, Long Wu, stable, Peng Zhang, Kevin Traynor, Luca Boccassi
On 1/24/2024 9:25 AM, Chaoyong He wrote:
> From: Long Wu <long.wu@corigine.com>
>
> Stop cpp service if all representors are closed and cpp service is running.
>
> Fixes: bab0e6f48b6d ("net/nfp: fix infinite loop")
> Cc: chaoyong.he@corigine.com
> Cc: stable@dpdk.org
>
This fix is using a new function that is introduced in previous patches
in this set, so this fix can't be backported.
Can you please change the order of the patches in this set, make this
patch first patch in the set and update it to use old version of
functions, so it can be backported?
Later you can update old function with new one in the patch that
introduces the new functions.
Also instead of saying "cpp service problem" in the title, can you
please describe the problem, I guess the problem you are referring is
something like, "CPP service is consuming resources even there is no
device remaining that requires it" ? Or is it something else?
> Signed-off-by: Long Wu <long.wu@corigine.com>
> Reviewed-by: Chaoyong He <chaoyong.he@corigine.com>
> Reviewed-by: Peng Zhang <peng.zhang@corigine.com>
> ---
> drivers/net/nfp/flower/nfp_flower_representor.c | 3 +++
> 1 file changed, 3 insertions(+)
>
> diff --git a/drivers/net/nfp/flower/nfp_flower_representor.c b/drivers/net/nfp/flower/nfp_flower_representor.c
> index bf67dc3ba2..2e8eb13e4e 100644
> --- a/drivers/net/nfp/flower/nfp_flower_representor.c
> +++ b/drivers/net/nfp/flower/nfp_flower_representor.c
> @@ -400,6 +400,9 @@ nfp_flower_repr_dev_close(struct rte_eth_dev *dev)
> /* Stop flower service first */
> nfp_flower_service_stop(app_fw_flower);
>
> + /* Disable cpp service if need */
> + nfp_service_disable(&pf_dev->cpp_service_info);
> +
> /* Now it is safe to free all PF resources */
> nfp_uninit_app_fw_flower(pf_dev);
> nfp_pf_uninit(pf_dev);
^ permalink raw reply [flat|nested] 14+ messages in thread
* Re: [PATCH 4/4] net/nfp: fix stop cpp service problem
2024-01-30 12:07 ` Ferruh Yigit
@ 2024-01-30 12:17 ` Kevin Traynor
0 siblings, 0 replies; 14+ messages in thread
From: Kevin Traynor @ 2024-01-30 12:17 UTC (permalink / raw)
To: Ferruh Yigit, Chaoyong He, dev
Cc: oss-drivers, Long Wu, stable, Peng Zhang, Luca Boccassi
On 30/01/2024 12:07, Ferruh Yigit wrote:
> On 1/24/2024 9:25 AM, Chaoyong He wrote:
>> From: Long Wu <long.wu@corigine.com>
>>
>> Stop cpp service if all representors are closed and cpp service is running.
>>
>> Fixes: bab0e6f48b6d ("net/nfp: fix infinite loop")
>> Cc: chaoyong.he@corigine.com
>> Cc: stable@dpdk.org
>>
>
> This fix is using a new function that is introduced in previous patches
> in this set, so this fix can't be backported.
>
> Can you please change the order of the patches in this set, make this
> patch first patch in the set and update it to use old version of
> functions, so it can be backported?
> Later you can update old function with new one in the patch that
> introduces the new functions.
>
Agree, it makes sense to put fixes at the start of the series to make
the backport easier. If it's not possible for some reason, then you can
also send a rebased revision of the fix to the stable mailing list with
the correct prefix
(http://doc.dpdk.org/guides/contributing/patches.html#backporting-patches-for-stable-releases)
>
> Also instead of saying "cpp service problem" in the title, can you
> please describe the problem, I guess the problem you are referring is
> something like, "CPP service is consuming resources even there is no
> device remaining that requires it" ? Or is it something else?
>
+1
>> Signed-off-by: Long Wu <long.wu@corigine.com>
>> Reviewed-by: Chaoyong He <chaoyong.he@corigine.com>
>> Reviewed-by: Peng Zhang <peng.zhang@corigine.com>
>> ---
>> drivers/net/nfp/flower/nfp_flower_representor.c | 3 +++
>> 1 file changed, 3 insertions(+)
>>
>> diff --git a/drivers/net/nfp/flower/nfp_flower_representor.c b/drivers/net/nfp/flower/nfp_flower_representor.c
>> index bf67dc3ba2..2e8eb13e4e 100644
>> --- a/drivers/net/nfp/flower/nfp_flower_representor.c
>> +++ b/drivers/net/nfp/flower/nfp_flower_representor.c
>> @@ -400,6 +400,9 @@ nfp_flower_repr_dev_close(struct rte_eth_dev *dev)
>> /* Stop flower service first */
>> nfp_flower_service_stop(app_fw_flower);
>>
>> + /* Disable cpp service if need */
>> + nfp_service_disable(&pf_dev->cpp_service_info);
>> +
>> /* Now it is safe to free all PF resources */
>> nfp_uninit_app_fw_flower(pf_dev);
>> nfp_pf_uninit(pf_dev);
>
^ permalink raw reply [flat|nested] 14+ messages in thread
* [PATCH v2 0/3] enhance NFP service framework
2024-01-24 9:25 [PATCH 0/4] enhance NFP service framework Chaoyong He
` (3 preceding siblings ...)
2024-01-24 9:25 ` [PATCH 4/4] net/nfp: fix stop cpp service problem Chaoyong He
@ 2024-02-02 3:04 ` Chaoyong He
2024-02-02 3:04 ` [PATCH v2 1/3] net/nfp: add synchronize module Chaoyong He
` (4 more replies)
4 siblings, 5 replies; 14+ messages in thread
From: Chaoyong He @ 2024-02-02 3:04 UTC (permalink / raw)
To: dev; +Cc: oss-drivers, Chaoyong He
Make multiple devices can use single core to run services for
flower firmware.
Also add synchronize module and service module to support it.
---
v2:
* Merge the last commit of v1 into the second commit.
---
Long Wu (3):
net/nfp: add synchronize module
net/nfp: create new service code related files
net/nfp: flower driver uses one service core
drivers/net/nfp/flower/nfp_flower.c | 56 +--
drivers/net/nfp/flower/nfp_flower_ctrl.c | 20 +-
drivers/net/nfp/flower/nfp_flower_ctrl.h | 2 +-
.../net/nfp/flower/nfp_flower_representor.c | 7 +
drivers/net/nfp/flower/nfp_flower_service.c | 196 +++++++++
drivers/net/nfp/flower/nfp_flower_service.h | 17 +
drivers/net/nfp/meson.build | 3 +
drivers/net/nfp/nfp_cpp_bridge.c | 91 +----
drivers/net/nfp/nfp_cpp_bridge.h | 1 -
drivers/net/nfp/nfp_ethdev.c | 31 +-
drivers/net/nfp/nfp_net_common.h | 15 +-
drivers/net/nfp/nfp_service.c | 117 ++++++
drivers/net/nfp/nfp_service.h | 20 +
drivers/net/nfp/nfpcore/nfp_sync.c | 382 ++++++++++++++++++
drivers/net/nfp/nfpcore/nfp_sync.h | 31 ++
15 files changed, 854 insertions(+), 135 deletions(-)
create mode 100644 drivers/net/nfp/flower/nfp_flower_service.c
create mode 100644 drivers/net/nfp/flower/nfp_flower_service.h
create mode 100644 drivers/net/nfp/nfp_service.c
create mode 100644 drivers/net/nfp/nfp_service.h
create mode 100644 drivers/net/nfp/nfpcore/nfp_sync.c
create mode 100644 drivers/net/nfp/nfpcore/nfp_sync.h
--
2.39.1
^ permalink raw reply [flat|nested] 14+ messages in thread
* [PATCH v2 1/3] net/nfp: add synchronize module
2024-02-02 3:04 ` [PATCH v2 0/3] enhance NFP service framework Chaoyong He
@ 2024-02-02 3:04 ` Chaoyong He
2024-02-02 3:04 ` [PATCH v2 2/3] net/nfp: create new service code related files Chaoyong He
` (3 subsequent siblings)
4 siblings, 0 replies; 14+ messages in thread
From: Chaoyong He @ 2024-02-02 3:04 UTC (permalink / raw)
To: dev; +Cc: oss-drivers, Long Wu, Chaoyong He, Peng Zhang
From: Long Wu <long.wu@corigine.com>
PMD has some information that needs synchronized:
1. Between devices in one process.
2. Between multiple processes for one device, before creating
"rte_eth_device".
3. Between multiple processes for one device, after creating
"rte_eth_device".
The final one is already support by framework
(eth_dev->data->dev_private), and this patch aims to support
the first two cases by adding a synchronize module.
Signed-off-by: Long Wu <long.wu@corigine.com>
Reviewed-by: Chaoyong He <chaoyong.he@corigine.com>
Reviewed-by: Peng Zhang <peng.zhang@corigine.com>
---
drivers/net/nfp/meson.build | 1 +
drivers/net/nfp/nfp_ethdev.c | 31 ++-
drivers/net/nfp/nfp_net_common.h | 4 +
drivers/net/nfp/nfpcore/nfp_sync.c | 382 +++++++++++++++++++++++++++++
drivers/net/nfp/nfpcore/nfp_sync.h | 29 +++
5 files changed, 444 insertions(+), 3 deletions(-)
create mode 100644 drivers/net/nfp/nfpcore/nfp_sync.c
create mode 100644 drivers/net/nfp/nfpcore/nfp_sync.h
diff --git a/drivers/net/nfp/meson.build b/drivers/net/nfp/meson.build
index 46be6f60cd..c625355d7a 100644
--- a/drivers/net/nfp/meson.build
+++ b/drivers/net/nfp/meson.build
@@ -26,6 +26,7 @@ sources = files(
'nfpcore/nfp_nsp_eth.c',
'nfpcore/nfp_resource.c',
'nfpcore/nfp_rtsym.c',
+ 'nfpcore/nfp_sync.c',
'nfpcore/nfp_target.c',
'nfpcore/nfp6000_pcie.c',
'nfp_cpp_bridge.c',
diff --git a/drivers/net/nfp/nfp_ethdev.c b/drivers/net/nfp/nfp_ethdev.c
index 07cd32881c..3ece806487 100644
--- a/drivers/net/nfp/nfp_ethdev.c
+++ b/drivers/net/nfp/nfp_ethdev.c
@@ -19,6 +19,7 @@
#include "nfpcore/nfp_nsp.h"
#include "nfpcore/nfp6000_pcie.h"
#include "nfpcore/nfp_resource.h"
+#include "nfpcore/nfp_sync.h"
#include "nfp_cpp_bridge.h"
#include "nfp_ipsec.h"
@@ -551,6 +552,7 @@ nfp_pf_uninit(struct nfp_pf_dev *pf_dev)
free(pf_dev->nfp_eth_table);
free(pf_dev->hwinfo);
nfp_cpp_free(pf_dev->cpp);
+ nfp_sync_free(pf_dev->sync);
rte_free(pf_dev);
}
@@ -559,6 +561,7 @@ nfp_pf_secondary_uninit(struct nfp_pf_dev *pf_dev)
{
free(pf_dev->sym_tbl);
nfp_cpp_free(pf_dev->cpp);
+ nfp_sync_free(pf_dev->sync);
rte_free(pf_dev);
return 0;
@@ -1612,6 +1615,7 @@ nfp_net_speed_capa_get(struct nfp_pf_dev *pf_dev,
static int
nfp_pf_init(struct rte_pci_device *pci_dev)
{
+ void *sync;
uint32_t i;
uint32_t id;
int ret = 0;
@@ -1652,6 +1656,13 @@ nfp_pf_init(struct rte_pci_device *pci_dev)
return -ENOMEM;
}
+ sync = nfp_sync_alloc();
+ if (sync == NULL) {
+ PMD_INIT_LOG(ERR, "Failed to alloc sync zone.");
+ ret = -ENOMEM;
+ goto pf_cleanup;
+ }
+
/*
* When device bound to UIO, the device could be used, by mistake,
* by two DPDK apps, and the UIO driver does not avoid it. This
@@ -1667,7 +1678,7 @@ nfp_pf_init(struct rte_pci_device *pci_dev)
if (cpp == NULL) {
PMD_INIT_LOG(ERR, "A CPP handle can not be obtained");
ret = -EIO;
- goto pf_cleanup;
+ goto sync_free;
}
hwinfo = nfp_hwinfo_read(cpp);
@@ -1734,6 +1745,7 @@ nfp_pf_init(struct rte_pci_device *pci_dev)
pf_dev->sym_tbl = sym_tbl;
pf_dev->pci_dev = pci_dev;
pf_dev->nfp_eth_table = nfp_eth_table;
+ pf_dev->sync = sync;
/* Get the speed capability */
for (i = 0; i < nfp_eth_table->count; i++) {
@@ -1815,6 +1827,8 @@ nfp_pf_init(struct rte_pci_device *pci_dev)
free(hwinfo);
cpp_cleanup:
nfp_cpp_free(cpp);
+sync_free:
+ nfp_sync_free(sync);
pf_cleanup:
rte_free(pf_dev);
@@ -1878,6 +1892,7 @@ nfp_secondary_init_app_fw_nic(struct nfp_pf_dev *pf_dev)
static int
nfp_pf_secondary_init(struct rte_pci_device *pci_dev)
{
+ void *sync;
int ret = 0;
struct nfp_cpp *cpp;
uint8_t function_id;
@@ -1910,6 +1925,13 @@ nfp_pf_secondary_init(struct rte_pci_device *pci_dev)
return -ENOMEM;
}
+ sync = nfp_sync_alloc();
+ if (sync == NULL) {
+ PMD_INIT_LOG(ERR, "Failed to alloc sync zone.");
+ ret = -ENOMEM;
+ goto pf_cleanup;
+ }
+
/*
* When device bound to UIO, the device could be used, by mistake,
* by two DPDK apps, and the UIO driver does not avoid it. This
@@ -1925,7 +1947,7 @@ nfp_pf_secondary_init(struct rte_pci_device *pci_dev)
if (cpp == NULL) {
PMD_INIT_LOG(ERR, "A CPP handle can not be obtained");
ret = -EIO;
- goto pf_cleanup;
+ goto sync_free;
}
/*
@@ -1936,7 +1958,7 @@ nfp_pf_secondary_init(struct rte_pci_device *pci_dev)
if (sym_tbl == NULL) {
PMD_INIT_LOG(ERR, "Something is wrong with the firmware symbol table");
ret = -EIO;
- goto pf_cleanup;
+ goto sync_free;
}
/* Read the app ID of the firmware loaded */
@@ -1954,6 +1976,7 @@ nfp_pf_secondary_init(struct rte_pci_device *pci_dev)
pf_dev->cpp = cpp;
pf_dev->sym_tbl = sym_tbl;
pf_dev->pci_dev = pci_dev;
+ pf_dev->sync = sync;
/* Call app specific init code now */
switch (app_fw_id) {
@@ -1983,6 +2006,8 @@ nfp_pf_secondary_init(struct rte_pci_device *pci_dev)
sym_tbl_cleanup:
free(sym_tbl);
+sync_free:
+ nfp_sync_free(sync);
pf_cleanup:
rte_free(pf_dev);
diff --git a/drivers/net/nfp/nfp_net_common.h b/drivers/net/nfp/nfp_net_common.h
index 5648ad32d8..3a441f49f6 100644
--- a/drivers/net/nfp/nfp_net_common.h
+++ b/drivers/net/nfp/nfp_net_common.h
@@ -12,6 +12,7 @@
#include <nfp_dev.h>
#include <rte_spinlock.h>
+#include "nfpcore/nfp_sync.h"
#include "nfp_net_ctrl.h"
/* Interrupt definitions */
@@ -109,6 +110,9 @@ struct nfp_pf_dev {
/** Supported speeds bitmap */
uint32_t speed_capa;
+
+ /** Synchronized info */
+ struct nfp_sync *sync;
};
#define NFP_NET_FLOW_LIMIT 1024
diff --git a/drivers/net/nfp/nfpcore/nfp_sync.c b/drivers/net/nfp/nfpcore/nfp_sync.c
new file mode 100644
index 0000000000..8bc38d6585
--- /dev/null
+++ b/drivers/net/nfp/nfpcore/nfp_sync.c
@@ -0,0 +1,382 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright (c) 2024 Corigine, Inc.
+ * All rights reserved.
+ */
+
+#include "nfp_sync.h"
+
+#include <rte_dev.h>
+#include <rte_malloc.h>
+#include <rte_memzone.h>
+#include <rte_spinlock.h>
+
+#include "nfp_logs.h"
+
+#define NFP_SYNC_ELEMENT_MAX 8
+#define NFP_SYNC_PCI_MAX 32
+
+struct nfp_sync_element {
+ uint16_t count;
+ /** Element ID, use ASCII - SYN<> */
+ uint32_t magic;
+ void *handle;
+};
+
+struct nfp_sync_common {
+ char pci_name[PCI_PRI_STR_SIZE + 1];
+ uint16_t avail;
+ struct nfp_sync_element element[NFP_SYNC_ELEMENT_MAX];
+};
+
+struct nfp_sync {
+ rte_spinlock_t spinlock;
+
+ uint16_t alloc_count;
+
+ struct nfp_sync_common process;
+
+ struct nfp_sync_common pci[NFP_SYNC_PCI_MAX];
+
+ const struct rte_memzone *mz;
+};
+
+struct nfp_sync *
+nfp_sync_alloc(void)
+{
+ uint16_t i;
+ struct nfp_sync *sync;
+ const struct rte_memzone *mz;
+
+ mz = rte_memzone_lookup("nfp_sync");
+ if (mz != NULL) {
+ sync = mz->addr;
+ sync->alloc_count++;
+
+ return sync;
+ }
+
+ mz = rte_memzone_reserve("nfp_sync", sizeof(*sync), SOCKET_ID_ANY,
+ RTE_MEMZONE_SIZE_HINT_ONLY);
+ if (mz == NULL)
+ return NULL;
+
+ sync = mz->addr;
+
+ memset(sync, 0, sizeof(*sync));
+
+ rte_spinlock_init(&sync->spinlock);
+ sync->alloc_count = 1;
+ sync->mz = mz;
+
+ for (i = 0; i < NFP_SYNC_PCI_MAX; i++)
+ sync->pci[i].avail = NFP_SYNC_ELEMENT_MAX;
+
+ sync->process.avail = NFP_SYNC_ELEMENT_MAX;
+
+ return sync;
+}
+
+void
+nfp_sync_free(struct nfp_sync *sync)
+{
+ uint16_t i;
+
+ rte_spinlock_lock(&sync->spinlock);
+
+ sync->alloc_count--;
+ if (sync->alloc_count != 0) {
+ rte_spinlock_unlock(&sync->spinlock);
+ return;
+ }
+
+ if (sync->process.avail != NFP_SYNC_ELEMENT_MAX)
+ PMD_DRV_LOG(ERR, "Sync process handle residue");
+
+ for (i = 0; i < NFP_SYNC_PCI_MAX; i++) {
+ if (sync->pci[i].avail != NFP_SYNC_ELEMENT_MAX)
+ PMD_DRV_LOG(ERR, "Sync %s pci handle residue",
+ sync->pci[i].pci_name);
+ }
+
+ rte_spinlock_unlock(&sync->spinlock);
+
+ rte_memzone_free(sync->mz);
+}
+
+static void *
+nfp_sync_element_alloc(struct nfp_sync_element *element,
+ uint32_t magic,
+ uint32_t size)
+{
+ void *handle;
+
+ handle = rte_zmalloc(NULL, size, 0);
+ if (handle == NULL)
+ return NULL;
+
+ element->handle = handle;
+ element->count = 1;
+ element->magic = magic;
+
+ return handle;
+}
+
+static void
+nfp_sync_element_free(struct nfp_sync_element *element,
+ void *handle)
+{
+ element->count--;
+ if (element->count != 0)
+ return;
+
+ rte_free(handle);
+ element->handle = NULL;
+ element->magic = 0;
+}
+
+static void *
+nfp_sync_common_handle_alloc(struct nfp_sync_common *common,
+ uint32_t magic,
+ uint32_t size)
+{
+ uint16_t i;
+ void *handle = NULL;
+ uint16_t avail_slot = NFP_SYNC_ELEMENT_MAX;
+
+ for (i = 0; i < NFP_SYNC_ELEMENT_MAX; i++) {
+ if (common->element[i].magic != magic)
+ continue;
+
+ common->element[i].count++;
+
+ return common->element[i].handle;
+ }
+
+ if (common->avail == 0)
+ return NULL;
+
+ for (i = 0; i < NFP_SYNC_ELEMENT_MAX; i++) {
+ if (common->element[i].magic == 0) {
+ avail_slot = i;
+ break;
+ }
+ }
+
+ handle = nfp_sync_element_alloc(&common->element[avail_slot], magic, size);
+ if (handle == NULL)
+ return NULL;
+
+ common->avail--;
+
+ return handle;
+}
+
+static void
+nfp_sync_common_handle_free(struct nfp_sync_common *common,
+ void *handle)
+{
+ uint16_t i;
+
+ if (common->avail == NFP_SYNC_ELEMENT_MAX)
+ return;
+
+ for (i = 0; i < NFP_SYNC_ELEMENT_MAX; i++) {
+ if (common->element[i].handle == handle)
+ break;
+ }
+
+ if (i == NFP_SYNC_ELEMENT_MAX)
+ return;
+
+ nfp_sync_element_free(&common->element[i], handle);
+
+ if (common->element[i].count == 0)
+ common->avail++;
+}
+
+static void *
+nfp_sync_process_inner_handle_alloc(struct nfp_sync *sync,
+ uint32_t magic,
+ uint32_t size)
+{
+ void *handle = NULL;
+
+ rte_spinlock_lock(&sync->spinlock);
+
+ handle = nfp_sync_common_handle_alloc(&sync->process, magic, size);
+ if (handle == NULL)
+ PMD_DRV_LOG(ERR, "Process handle alloc failed");
+
+ rte_spinlock_unlock(&sync->spinlock);
+
+ return handle;
+}
+
+static void
+nfp_sync_process_inner_handle_free(struct nfp_sync *sync,
+ void *handle)
+{
+ rte_spinlock_lock(&sync->spinlock);
+
+ nfp_sync_common_handle_free(&sync->process, handle);
+
+ rte_spinlock_unlock(&sync->spinlock);
+}
+
+static uint16_t
+nfp_sync_process_handle_count_get(struct nfp_sync *sync,
+ void *handle)
+{
+ uint16_t i;
+ uint16_t count = 0;
+
+ rte_spinlock_lock(&sync->spinlock);
+
+ for (i = 0; i < NFP_SYNC_ELEMENT_MAX; i++) {
+ if (sync->process.element[i].handle == handle) {
+ count = sync->process.element[i].count;
+ break;
+ }
+ }
+
+ rte_spinlock_unlock(&sync->spinlock);
+
+ return count;
+}
+
+static void *
+nfp_sync_pci_inner_handle_alloc(struct nfp_sync *sync,
+ const char *pci_name,
+ uint32_t magic,
+ uint32_t size)
+{
+ uint16_t i;
+ void *handle = NULL;
+ uint16_t pci_avail_id = NFP_SYNC_PCI_MAX;
+
+ rte_spinlock_lock(&sync->spinlock);
+
+ for (i = 0; i < NFP_SYNC_PCI_MAX; i++) {
+ if (strcmp(pci_name, sync->pci[i].pci_name) == 0) {
+ pci_avail_id = i;
+ goto common_alloc;
+ }
+ }
+
+ for (i = 0; i < NFP_SYNC_PCI_MAX; i++) {
+ if (strlen(sync->pci[i].pci_name) == 0) {
+ pci_avail_id = i;
+ strcpy(sync->pci[pci_avail_id].pci_name, pci_name);
+ goto common_alloc;
+ }
+ }
+
+ rte_spinlock_unlock(&sync->spinlock);
+
+ return NULL;
+
+common_alloc:
+ handle = nfp_sync_common_handle_alloc(&sync->pci[pci_avail_id],
+ magic, size);
+ if (handle == NULL)
+ PMD_DRV_LOG(ERR, "PCI handle alloc failed");
+
+ rte_spinlock_unlock(&sync->spinlock);
+
+ return handle;
+}
+
+static void
+nfp_sync_pci_inner_handle_free(struct nfp_sync *sync,
+ const char *pci_name,
+ void *handle)
+{
+ uint16_t i;
+ char *name_tmp;
+
+ rte_spinlock_lock(&sync->spinlock);
+
+ for (i = 0; i < NFP_SYNC_PCI_MAX; i++) {
+ name_tmp = sync->pci[i].pci_name;
+ if (strlen(name_tmp) != 0 && strcmp(pci_name, name_tmp) == 0) {
+ nfp_sync_common_handle_free(&sync->pci[i], handle);
+ if (sync->pci[i].avail == NFP_SYNC_ELEMENT_MAX)
+ name_tmp[0] = 0;
+ break;
+ }
+ }
+
+ rte_spinlock_unlock(&sync->spinlock);
+}
+
+static uint16_t
+nfp_sync_pci_handle_count_get(struct nfp_sync *sync,
+ const char *pci_name,
+ void *handle)
+{
+ uint16_t i;
+ uint16_t count = 0;
+ struct nfp_sync_common *pci_common;
+
+ rte_spinlock_lock(&sync->spinlock);
+
+ for (i = 0; i < NFP_SYNC_PCI_MAX; i++) {
+ if (strcmp(sync->pci[i].pci_name, pci_name) == 0)
+ break;
+ }
+
+ if (i == NFP_SYNC_PCI_MAX) {
+ rte_spinlock_unlock(&sync->spinlock);
+ return 0;
+ }
+
+ pci_common = &sync->pci[i];
+
+ for (i = 0; i < NFP_SYNC_ELEMENT_MAX; i++) {
+ if (pci_common->element[i].handle == handle) {
+ count = pci_common->element[i].count;
+ break;
+ }
+ }
+
+ rte_spinlock_unlock(&sync->spinlock);
+
+ return count;
+}
+
+void *
+nfp_sync_handle_alloc(struct nfp_sync *sync,
+ struct rte_pci_device *pci_dev,
+ uint32_t magic,
+ uint32_t size)
+{
+ if (pci_dev == NULL)
+ return nfp_sync_process_inner_handle_alloc(sync, magic, size);
+
+ return nfp_sync_pci_inner_handle_alloc(sync, pci_dev->device.name,
+ magic, size);
+}
+
+void
+nfp_sync_handle_free(struct nfp_sync *sync,
+ struct rte_pci_device *pci_dev,
+ void *handle)
+{
+ if (pci_dev == NULL) {
+ nfp_sync_process_inner_handle_free(sync, handle);
+ return;
+ }
+
+ nfp_sync_pci_inner_handle_free(sync, pci_dev->device.name, handle);
+}
+
+uint16_t
+nfp_sync_handle_count_get(struct nfp_sync *sync,
+ struct rte_pci_device *pci_dev,
+ void *handle)
+{
+ if (pci_dev == NULL)
+ return nfp_sync_process_handle_count_get(sync, handle);
+
+ return nfp_sync_pci_handle_count_get(sync, pci_dev->device.name, handle);
+}
diff --git a/drivers/net/nfp/nfpcore/nfp_sync.h b/drivers/net/nfp/nfpcore/nfp_sync.h
new file mode 100644
index 0000000000..82f01e2652
--- /dev/null
+++ b/drivers/net/nfp/nfpcore/nfp_sync.h
@@ -0,0 +1,29 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright (c) 2024 Corigine, Inc.
+ * All rights reserved.
+ */
+
+#ifndef __NFP_SYNC_H__
+#define __NFP_SYNC_H__
+
+#include <stdint.h>
+
+#include <bus_pci_driver.h>
+
+struct nfp_sync;
+
+struct nfp_sync *nfp_sync_alloc(void);
+void nfp_sync_free(struct nfp_sync *sync);
+
+void *nfp_sync_handle_alloc(struct nfp_sync *sync,
+ struct rte_pci_device *pci_dev,
+ uint32_t magic,
+ uint32_t size);
+void nfp_sync_handle_free(struct nfp_sync *sync,
+ struct rte_pci_device *pci_dev,
+ void *handle);
+uint16_t nfp_sync_handle_count_get(struct nfp_sync *sync,
+ struct rte_pci_device *pci_dev,
+ void *handle);
+
+#endif /* __NFP_SYNC_H__ */
--
2.39.1
^ permalink raw reply [flat|nested] 14+ messages in thread
* [PATCH v2 2/3] net/nfp: create new service code related files
2024-02-02 3:04 ` [PATCH v2 0/3] enhance NFP service framework Chaoyong He
2024-02-02 3:04 ` [PATCH v2 1/3] net/nfp: add synchronize module Chaoyong He
@ 2024-02-02 3:04 ` Chaoyong He
2024-02-02 3:04 ` [PATCH v2 3/3] net/nfp: flower driver uses one service core Chaoyong He
` (2 subsequent siblings)
4 siblings, 0 replies; 14+ messages in thread
From: Chaoyong He @ 2024-02-02 3:04 UTC (permalink / raw)
To: dev; +Cc: oss-drivers, Long Wu, Chaoyong He, Peng Zhang
From: Long Wu <long.wu@corigine.com>
For clearer service code structure and more convenient future
addition of service code, NFP creates new service code related
files and move service related code into new files.
This commit also adds service disable interface and lets CPP
service run normally if app uses several flower cards.
Signed-off-by: Long Wu <long.wu@corigine.com>
Reviewed-by: Chaoyong He <chaoyong.he@corigine.com>
Reviewed-by: Peng Zhang <peng.zhang@corigine.com>
---
drivers/net/nfp/flower/nfp_flower.c | 19 +--
.../net/nfp/flower/nfp_flower_representor.c | 3 +
drivers/net/nfp/meson.build | 1 +
drivers/net/nfp/nfp_cpp_bridge.c | 91 +++-----------
drivers/net/nfp/nfp_cpp_bridge.h | 1 -
drivers/net/nfp/nfp_net_common.h | 5 +-
drivers/net/nfp/nfp_service.c | 117 ++++++++++++++++++
drivers/net/nfp/nfp_service.h | 20 +++
8 files changed, 167 insertions(+), 90 deletions(-)
create mode 100644 drivers/net/nfp/nfp_service.c
create mode 100644 drivers/net/nfp/nfp_service.h
diff --git a/drivers/net/nfp/flower/nfp_flower.c b/drivers/net/nfp/flower/nfp_flower.c
index 88a1b061e1..a29edf8ca2 100644
--- a/drivers/net/nfp/flower/nfp_flower.c
+++ b/drivers/net/nfp/flower/nfp_flower.c
@@ -598,29 +598,20 @@ static int
nfp_flower_enable_services(struct nfp_app_fw_flower *app_fw_flower)
{
int ret;
- uint32_t service_id;
+ struct nfp_service_info info;
const struct rte_service_spec flower_service = {
.name = "flower_ctrl_vnic_service",
.callback = nfp_flower_ctrl_vnic_service,
.callback_userdata = (void *)app_fw_flower,
};
- /* Register the flower services */
- ret = rte_service_component_register(&flower_service, &service_id);
+ ret = nfp_service_enable(&flower_service, &info);
if (ret != 0) {
- PMD_INIT_LOG(ERR, "Could not register %s", flower_service.name);
- return -EINVAL;
+ PMD_INIT_LOG(ERR, "Could not enable service %s", flower_service.name);
+ return ret;
}
- app_fw_flower->ctrl_vnic_id = service_id;
- PMD_INIT_LOG(INFO, "%s registered", flower_service.name);
-
- /* Map them to available service cores */
- ret = nfp_map_service(service_id);
- if (ret != 0) {
- PMD_INIT_LOG(ERR, "Could not map %s", flower_service.name);
- return -EINVAL;
- }
+ app_fw_flower->ctrl_vnic_id = info.id;
return 0;
}
diff --git a/drivers/net/nfp/flower/nfp_flower_representor.c b/drivers/net/nfp/flower/nfp_flower_representor.c
index 63fe37c8d7..12e8af54fb 100644
--- a/drivers/net/nfp/flower/nfp_flower_representor.c
+++ b/drivers/net/nfp/flower/nfp_flower_representor.c
@@ -396,6 +396,9 @@ nfp_flower_repr_dev_close(struct rte_eth_dev *dev)
if (app_fw_flower->pf_repr != NULL)
return 0;
+ /* Disable cpp service */
+ nfp_service_disable(&pf_dev->cpp_service_info);
+
/* Now it is safe to free all PF resources */
nfp_uninit_app_fw_flower(pf_dev);
nfp_pf_uninit(pf_dev);
diff --git a/drivers/net/nfp/meson.build b/drivers/net/nfp/meson.build
index c625355d7a..0f4beccbdf 100644
--- a/drivers/net/nfp/meson.build
+++ b/drivers/net/nfp/meson.build
@@ -40,6 +40,7 @@ sources = files(
'nfp_net_ctrl.c',
'nfp_net_flow.c',
'nfp_rxtx.c',
+ 'nfp_service.c',
)
deps += ['hash', 'security', 'common_nfp']
diff --git a/drivers/net/nfp/nfp_cpp_bridge.c b/drivers/net/nfp/nfp_cpp_bridge.c
index 36dcdca9de..441c0d2843 100644
--- a/drivers/net/nfp/nfp_cpp_bridge.c
+++ b/drivers/net/nfp/nfp_cpp_bridge.c
@@ -8,10 +8,9 @@
#include <unistd.h>
#include <sys/ioctl.h>
-#include <rte_service_component.h>
-
#include "nfpcore/nfp_cpp.h"
#include "nfp_logs.h"
+#include "nfp_service.h"
#define NFP_CPP_MEMIO_BOUNDARY (1 << 20)
#define NFP_BRIDGE_OP_READ 20
@@ -24,82 +23,23 @@
/* Prototypes */
static int nfp_cpp_bridge_service_func(void *args);
-int
-nfp_map_service(uint32_t service_id)
-{
- int32_t ret;
- uint32_t slcore = 0;
- int32_t slcore_count;
- uint8_t service_count;
- const char *service_name;
- uint32_t slcore_array[RTE_MAX_LCORE];
- uint8_t min_service_count = UINT8_MAX;
-
- slcore_count = rte_service_lcore_list(slcore_array, RTE_MAX_LCORE);
- if (slcore_count <= 0) {
- PMD_INIT_LOG(DEBUG, "No service cores found");
- return -ENOENT;
- }
-
- /*
- * Find a service core with the least number of services already
- * registered to it.
- */
- while (slcore_count--) {
- service_count = rte_service_lcore_count_services(slcore_array[slcore_count]);
- if (service_count < min_service_count) {
- slcore = slcore_array[slcore_count];
- min_service_count = service_count;
- }
- }
-
- service_name = rte_service_get_name(service_id);
- PMD_INIT_LOG(INFO, "Mapping service %s to core %u", service_name, slcore);
-
- ret = rte_service_map_lcore_set(service_id, slcore, 1);
- if (ret != 0) {
- PMD_INIT_LOG(DEBUG, "Could not map flower service");
- return -ENOENT;
- }
-
- rte_service_runstate_set(service_id, 1);
- rte_service_component_runstate_set(service_id, 1);
- rte_service_lcore_start(slcore);
- if (rte_service_may_be_active(slcore) != 0)
- PMD_INIT_LOG(INFO, "The service %s is running", service_name);
- else
- PMD_INIT_LOG(ERR, "The service %s is not running", service_name);
-
- return 0;
-}
-
int
nfp_enable_cpp_service(struct nfp_pf_dev *pf_dev)
{
int ret;
- uint32_t service_id = 0;
+ const char *pci_name;
struct rte_service_spec cpp_service = {
- .name = "nfp_cpp_service",
- .callback = nfp_cpp_bridge_service_func,
+ .callback = nfp_cpp_bridge_service_func,
+ .callback_userdata = (void *)pf_dev,
};
- cpp_service.callback_userdata = (void *)pf_dev;
+ pci_name = strchr(pf_dev->pci_dev->name, ':') + 1;
+ snprintf(cpp_service.name, sizeof(cpp_service.name), "%s_cpp_service", pci_name);
- /* Register the cpp service */
- ret = rte_service_component_register(&cpp_service, &service_id);
+ ret = nfp_service_enable(&cpp_service, &pf_dev->cpp_service_info);
if (ret != 0) {
- PMD_INIT_LOG(WARNING, "Could not register nfp cpp service");
- return -EINVAL;
- }
-
- pf_dev->cpp_bridge_id = service_id;
- PMD_INIT_LOG(INFO, "NFP cpp service registered");
-
- /* Map it to available service core */
- ret = nfp_map_service(service_id);
- if (ret != 0) {
- PMD_INIT_LOG(DEBUG, "Could not map nfp cpp service");
- return -EINVAL;
+ PMD_INIT_LOG(DEBUG, "Could not enable service %s", cpp_service.name);
+ return ret;
}
return 0;
@@ -387,12 +327,18 @@ nfp_cpp_bridge_service_func(void *args)
int sockfd;
int datafd;
struct nfp_cpp *cpp;
+ const char *pci_name;
+ char socket_handle[14];
struct sockaddr address;
struct nfp_pf_dev *pf_dev;
struct timeval timeout = {1, 0};
- unlink("/tmp/nfp_cpp");
+ pf_dev = args;
+
+ pci_name = strchr(pf_dev->pci_dev->name, ':') + 1;
+ snprintf(socket_handle, sizeof(socket_handle), "/tmp/%s", pci_name);
+ unlink(socket_handle);
sockfd = socket(AF_UNIX, SOCK_STREAM, 0);
if (sockfd < 0) {
PMD_CPP_LOG(ERR, "socket creation error. Service failed");
@@ -404,7 +350,7 @@ nfp_cpp_bridge_service_func(void *args)
memset(&address, 0, sizeof(struct sockaddr));
address.sa_family = AF_UNIX;
- strcpy(address.sa_data, "/tmp/nfp_cpp");
+ strcpy(address.sa_data, socket_handle);
ret = bind(sockfd, (const struct sockaddr *)&address,
sizeof(struct sockaddr));
@@ -421,9 +367,8 @@ nfp_cpp_bridge_service_func(void *args)
return ret;
}
- pf_dev = args;
cpp = pf_dev->cpp;
- while (rte_service_runstate_get(pf_dev->cpp_bridge_id) != 0) {
+ while (rte_service_runstate_get(pf_dev->cpp_service_info.id) != 0) {
datafd = accept(sockfd, NULL, NULL);
if (datafd < 0) {
if (errno == EAGAIN || errno == EWOULDBLOCK)
diff --git a/drivers/net/nfp/nfp_cpp_bridge.h b/drivers/net/nfp/nfp_cpp_bridge.h
index bf975ce7ba..406c961dce 100644
--- a/drivers/net/nfp/nfp_cpp_bridge.h
+++ b/drivers/net/nfp/nfp_cpp_bridge.h
@@ -9,6 +9,5 @@
#include "nfp_net_common.h"
int nfp_enable_cpp_service(struct nfp_pf_dev *pf_dev);
-int nfp_map_service(uint32_t service_id);
#endif /* __NFP_CPP_BRIDGE_H__ */
diff --git a/drivers/net/nfp/nfp_net_common.h b/drivers/net/nfp/nfp_net_common.h
index 3a441f49f6..eaa7501e8e 100644
--- a/drivers/net/nfp/nfp_net_common.h
+++ b/drivers/net/nfp/nfp_net_common.h
@@ -14,6 +14,7 @@
#include "nfpcore/nfp_sync.h"
#include "nfp_net_ctrl.h"
+#include "nfp_service.h"
/* Interrupt definitions */
#define NFP_NET_IRQ_LSC_IDX 0
@@ -102,8 +103,8 @@ struct nfp_pf_dev {
struct nfp_hwinfo *hwinfo;
struct nfp_rtsym_table *sym_tbl;
- /** Service id of cpp bridge service */
- uint32_t cpp_bridge_id;
+ /** Service info of cpp bridge service */
+ struct nfp_service_info cpp_service_info;
/** Multiple PF configuration */
struct nfp_multi_pf multi_pf;
diff --git a/drivers/net/nfp/nfp_service.c b/drivers/net/nfp/nfp_service.c
new file mode 100644
index 0000000000..f49fa6addf
--- /dev/null
+++ b/drivers/net/nfp/nfp_service.c
@@ -0,0 +1,117 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright (c) 2023 Corigine, Inc.
+ * All rights reserved.
+ */
+
+#include "nfp_service.h"
+
+#include "nfpcore/nfp_cpp.h"
+#include "nfp_logs.h"
+
+/* Disable service and try to get service status */
+#define NFP_SERVICE_DISABLE_WAIT_COUNT 3000
+
+static int
+nfp_map_service(struct nfp_service_info *info)
+{
+ int32_t ret;
+ uint32_t slcore = 0;
+ int32_t slcore_count;
+ uint8_t service_count;
+ const char *service_name;
+ uint32_t slcore_array[RTE_MAX_LCORE];
+ uint8_t min_service_count = UINT8_MAX;
+
+ slcore_count = rte_service_lcore_list(slcore_array, RTE_MAX_LCORE);
+ if (slcore_count <= 0) {
+ PMD_DRV_LOG(DEBUG, "No service cores found");
+ return -ENOENT;
+ }
+
+ /*
+ * Find a service core with the least number of services already
+ * registered to it
+ */
+ while (slcore_count--) {
+ service_count = rte_service_lcore_count_services(slcore_array[slcore_count]);
+ if (service_count < min_service_count) {
+ slcore = slcore_array[slcore_count];
+ min_service_count = service_count;
+ }
+ }
+
+ service_name = rte_service_get_name(info->id);
+ PMD_INIT_LOG(INFO, "Mapping service %s to core %u", service_name, slcore);
+
+ ret = rte_service_map_lcore_set(info->id, slcore, 1);
+ if (ret != 0) {
+ PMD_DRV_LOG(DEBUG, "Could not map flower service");
+ return -ENOENT;
+ }
+
+ rte_service_runstate_set(info->id, 1);
+ rte_service_component_runstate_set(info->id, 1);
+ rte_service_lcore_start(slcore);
+ if (rte_service_may_be_active(slcore))
+ PMD_DRV_LOG(INFO, "The service %s is running", service_name);
+ else
+ PMD_DRV_LOG(ERR, "The service %s is not running", service_name);
+
+ info->lcore = slcore;
+
+ return 0;
+}
+
+int
+nfp_service_enable(const struct rte_service_spec *service_spec,
+ struct nfp_service_info *info)
+{
+ int ret;
+
+ /* Register the service */
+ ret = rte_service_component_register(service_spec, &info->id);
+ if (ret != 0) {
+ PMD_DRV_LOG(DEBUG, "Could not register %s", service_spec->name);
+ return -EINVAL;
+ }
+
+ /* Map it to available service core */
+ ret = nfp_map_service(info);
+ if (ret != 0) {
+ PMD_DRV_LOG(DEBUG, "Could not map %s", service_spec->name);
+ return -EINVAL;
+ }
+
+ PMD_DRV_LOG(DEBUG, "Enable service %s successfully", service_spec->name);
+
+ return 0;
+}
+
+int
+nfp_service_disable(struct nfp_service_info *info)
+{
+ uint32_t i;
+ const char *service_name;
+
+ service_name = rte_service_get_name(info->id);
+ if (service_name == NULL) {
+ PMD_DRV_LOG(ERR, "Could not find service %u", info->id);
+ return -EINVAL;
+ }
+
+ rte_service_runstate_set(info->id, 0);
+ rte_service_component_runstate_set(info->id, 0);
+
+ for (i = 0; i < NFP_SERVICE_DISABLE_WAIT_COUNT; i++) {
+ if (rte_service_may_be_active(info->id) == 0)
+ break;
+ rte_delay_ms(1);
+ }
+ if (i == NFP_SERVICE_DISABLE_WAIT_COUNT)
+ PMD_DRV_LOG(ERR, "Could not stop service %s", service_name);
+
+ rte_service_map_lcore_set(info->id, info->lcore, 0);
+ rte_service_component_unregister(info->id);
+
+ return 0;
+}
diff --git a/drivers/net/nfp/nfp_service.h b/drivers/net/nfp/nfp_service.h
new file mode 100644
index 0000000000..4b99d5b973
--- /dev/null
+++ b/drivers/net/nfp/nfp_service.h
@@ -0,0 +1,20 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright (c) 2023 Corigine, Inc.
+ * All rights reserved.
+ */
+
+#ifndef __NFP_SERVICE_H__
+#define __NFP_SERVICE_H__
+
+#include <rte_service_component.h>
+
+struct nfp_service_info {
+ uint32_t id;
+ uint32_t lcore;
+};
+
+int nfp_service_disable(struct nfp_service_info *info);
+int nfp_service_enable(const struct rte_service_spec *service_spec,
+ struct nfp_service_info *info);
+
+#endif /* __NFP_SERVICE_H__ */
--
2.39.1
^ permalink raw reply [flat|nested] 14+ messages in thread
* [PATCH v2 3/3] net/nfp: flower driver uses one service core
2024-02-02 3:04 ` [PATCH v2 0/3] enhance NFP service framework Chaoyong He
2024-02-02 3:04 ` [PATCH v2 1/3] net/nfp: add synchronize module Chaoyong He
2024-02-02 3:04 ` [PATCH v2 2/3] net/nfp: create new service code related files Chaoyong He
@ 2024-02-02 3:04 ` Chaoyong He
2024-02-06 15:17 ` [PATCH v2 0/3] enhance NFP service framework Ferruh Yigit
2024-02-08 13:58 ` Ferruh Yigit
4 siblings, 0 replies; 14+ messages in thread
From: Chaoyong He @ 2024-02-02 3:04 UTC (permalink / raw)
To: dev; +Cc: oss-drivers, Long Wu, Chaoyong He, Peng Zhang
From: Long Wu <long.wu@corigine.com>
Now app must provide a service core for each NFP pci device that uses
flower driver to run flower service, which will cause huge resource waste.
This patch is to solve this problem and app can only use one core to
run NFP flower service though it uses several NFP NICs.
Signed-off-by: Long Wu <long.wu@corigine.com>
Reviewed-by: Chaoyong He <chaoyong.he@corigine.com>
Reviewed-by: Peng Zhang <peng.zhang@corigine.com>
---
drivers/net/nfp/flower/nfp_flower.c | 47 ++---
drivers/net/nfp/flower/nfp_flower_ctrl.c | 20 +-
drivers/net/nfp/flower/nfp_flower_ctrl.h | 2 +-
.../net/nfp/flower/nfp_flower_representor.c | 4 +
drivers/net/nfp/flower/nfp_flower_service.c | 196 ++++++++++++++++++
drivers/net/nfp/flower/nfp_flower_service.h | 17 ++
drivers/net/nfp/meson.build | 1 +
drivers/net/nfp/nfp_net_common.h | 6 +
drivers/net/nfp/nfpcore/nfp_sync.h | 2 +
9 files changed, 248 insertions(+), 47 deletions(-)
create mode 100644 drivers/net/nfp/flower/nfp_flower_service.c
create mode 100644 drivers/net/nfp/flower/nfp_flower_service.h
diff --git a/drivers/net/nfp/flower/nfp_flower.c b/drivers/net/nfp/flower/nfp_flower.c
index a29edf8ca2..e84e6ebbff 100644
--- a/drivers/net/nfp/flower/nfp_flower.c
+++ b/drivers/net/nfp/flower/nfp_flower.c
@@ -18,6 +18,7 @@
#include "../nfp_mtr.h"
#include "nfp_flower_ctrl.h"
#include "nfp_flower_representor.h"
+#include "nfp_flower_service.h"
#define CTRL_VNIC_NB_DESC 512
@@ -461,6 +462,13 @@ nfp_flower_init_ctrl_vnic(struct nfp_net_hw *hw)
nn_cfg_writeb(&hw->super, NFP_NET_CFG_TXR_SZ(i), rte_log2_u32(CTRL_VNIC_NB_DESC));
}
+ /* Alloc sync memory zone */
+ ret = nfp_flower_service_sync_alloc(app_fw_flower);
+ if (ret != 0) {
+ PMD_INIT_LOG(ERR, "Alloc sync memory zone failed");
+ goto tx_queue_setup_cleanup;
+ }
+
return 0;
tx_queue_setup_cleanup:
@@ -531,6 +539,7 @@ nfp_flower_cleanup_ctrl_vnic(struct nfp_net_hw *hw)
}
}
+ nfp_flower_service_sync_free(app_fw_flower);
rte_free(eth_dev->data->tx_queues);
rte_free(eth_dev->data->rx_queues);
rte_mempool_free(app_fw_flower->ctrl_pktmbuf_pool);
@@ -584,38 +593,6 @@ nfp_flower_start_ctrl_vnic(struct nfp_net_hw *net_hw)
return 0;
}
-static int
-nfp_flower_ctrl_vnic_service(void *arg)
-{
- struct nfp_app_fw_flower *app_fw_flower = arg;
-
- nfp_flower_ctrl_vnic_poll(app_fw_flower);
-
- return 0;
-}
-
-static int
-nfp_flower_enable_services(struct nfp_app_fw_flower *app_fw_flower)
-{
- int ret;
- struct nfp_service_info info;
- const struct rte_service_spec flower_service = {
- .name = "flower_ctrl_vnic_service",
- .callback = nfp_flower_ctrl_vnic_service,
- .callback_userdata = (void *)app_fw_flower,
- };
-
- ret = nfp_service_enable(&flower_service, &info);
- if (ret != 0) {
- PMD_INIT_LOG(ERR, "Could not enable service %s", flower_service.name);
- return ret;
- }
-
- app_fw_flower->ctrl_vnic_id = info.id;
-
- return 0;
-}
-
static void
nfp_flower_pkt_add_metadata_register(struct nfp_app_fw_flower *app_fw_flower)
{
@@ -760,7 +737,7 @@ nfp_init_app_fw_flower(struct nfp_pf_dev *pf_dev,
}
/* Start up flower services */
- ret = nfp_flower_enable_services(app_fw_flower);
+ ret = nfp_flower_service_start(app_fw_flower);
if (ret != 0) {
PMD_INIT_LOG(ERR, "Could not enable flower services");
ret = -ESRCH;
@@ -770,11 +747,13 @@ nfp_init_app_fw_flower(struct nfp_pf_dev *pf_dev,
ret = nfp_flower_repr_create(app_fw_flower);
if (ret != 0) {
PMD_INIT_LOG(ERR, "Could not create representor ports");
- goto ctrl_vnic_cleanup;
+ goto ctrl_vnic_service_stop;
}
return 0;
+ctrl_vnic_service_stop:
+ nfp_flower_service_stop(app_fw_flower);
ctrl_vnic_cleanup:
nfp_flower_cleanup_ctrl_vnic(app_fw_flower->ctrl_hw);
ctrl_cpp_area_cleanup:
diff --git a/drivers/net/nfp/flower/nfp_flower_ctrl.c b/drivers/net/nfp/flower/nfp_flower_ctrl.c
index c25487c277..bcb325d475 100644
--- a/drivers/net/nfp/flower/nfp_flower_ctrl.c
+++ b/drivers/net/nfp/flower/nfp_flower_ctrl.c
@@ -12,6 +12,7 @@
#include "../nfp_logs.h"
#include "nfp_flower_representor.h"
#include "nfp_mtr.h"
+#include "nfp_flower_service.h"
#define MAX_PKT_BURST 32
@@ -502,26 +503,21 @@ nfp_flower_cmsg_rx(struct nfp_app_fw_flower *app_fw_flower,
}
void
-nfp_flower_ctrl_vnic_poll(struct nfp_app_fw_flower *app_fw_flower)
+nfp_flower_ctrl_vnic_process(struct nfp_app_fw_flower *app_fw_flower)
{
uint16_t count;
struct nfp_net_rxq *rxq;
- struct nfp_net_hw *ctrl_hw;
struct rte_eth_dev *ctrl_eth_dev;
struct rte_mbuf *pkts_burst[MAX_PKT_BURST];
- ctrl_hw = app_fw_flower->ctrl_hw;
- ctrl_eth_dev = ctrl_hw->eth_dev;
+ ctrl_eth_dev = app_fw_flower->ctrl_hw->eth_dev;
/* Ctrl vNIC only has a single Rx queue */
rxq = ctrl_eth_dev->data->rx_queues[0];
-
- while (rte_service_runstate_get(app_fw_flower->ctrl_vnic_id) != 0) {
- count = nfp_flower_ctrl_vnic_recv(rxq, pkts_burst, MAX_PKT_BURST);
- if (count != 0) {
- app_fw_flower->ctrl_vnic_rx_count += count;
- /* Process cmsgs here */
- nfp_flower_cmsg_rx(app_fw_flower, pkts_burst, count);
- }
+ count = nfp_flower_ctrl_vnic_recv(rxq, pkts_burst, MAX_PKT_BURST);
+ if (count != 0) {
+ app_fw_flower->ctrl_vnic_rx_count += count;
+ /* Process cmsgs here */
+ nfp_flower_cmsg_rx(app_fw_flower, pkts_burst, count);
}
}
diff --git a/drivers/net/nfp/flower/nfp_flower_ctrl.h b/drivers/net/nfp/flower/nfp_flower_ctrl.h
index 4c94d36847..131d002ac6 100644
--- a/drivers/net/nfp/flower/nfp_flower_ctrl.h
+++ b/drivers/net/nfp/flower/nfp_flower_ctrl.h
@@ -8,7 +8,7 @@
#include "nfp_flower.h"
-void nfp_flower_ctrl_vnic_poll(struct nfp_app_fw_flower *app_fw_flower);
+void nfp_flower_ctrl_vnic_process(struct nfp_app_fw_flower *app_fw_flower);
uint16_t nfp_flower_ctrl_vnic_xmit(struct nfp_app_fw_flower *app_fw_flower,
struct rte_mbuf *mbuf);
void nfp_flower_ctrl_vnic_xmit_register(struct nfp_app_fw_flower *app_fw_flower);
diff --git a/drivers/net/nfp/flower/nfp_flower_representor.c b/drivers/net/nfp/flower/nfp_flower_representor.c
index 12e8af54fb..4f4df0cd2e 100644
--- a/drivers/net/nfp/flower/nfp_flower_representor.c
+++ b/drivers/net/nfp/flower/nfp_flower_representor.c
@@ -9,6 +9,7 @@
#include "../nfpcore/nfp_nsp.h"
#include "../nfp_logs.h"
#include "../nfp_mtr.h"
+#include "nfp_flower_service.h"
/* Type of representor */
enum nfp_repr_type {
@@ -396,6 +397,9 @@ nfp_flower_repr_dev_close(struct rte_eth_dev *dev)
if (app_fw_flower->pf_repr != NULL)
return 0;
+ /* Stop flower service first */
+ nfp_flower_service_stop(app_fw_flower);
+
/* Disable cpp service */
nfp_service_disable(&pf_dev->cpp_service_info);
diff --git a/drivers/net/nfp/flower/nfp_flower_service.c b/drivers/net/nfp/flower/nfp_flower_service.c
new file mode 100644
index 0000000000..5e8811fe24
--- /dev/null
+++ b/drivers/net/nfp/flower/nfp_flower_service.c
@@ -0,0 +1,196 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright (c) 2024 Corigine, Inc.
+ * All rights reserved.
+ */
+
+#include "nfp_flower_service.h"
+
+#include <rte_spinlock.h>
+
+#include "nfp_flower_ctrl.h"
+#include "../nfpcore/nfp_cpp.h"
+#include "../nfpcore/nfp_sync.h"
+#include "../nfp_logs.h"
+#include "../nfp_service.h"
+
+/* Driver limitation, PMD can enlarge it if need. */
+#define MAX_FLOWER_SERVICE_SLOT 8
+
+struct nfp_flower_service {
+ /** Flower service is enabled */
+ bool enabled;
+ /** Flower service info */
+ struct nfp_service_info info;
+ /** Store flower cards' information */
+ struct nfp_app_fw_flower *slots[MAX_FLOWER_SERVICE_SLOT];
+ /** Spinlock for sync slots when add/remove card */
+ rte_spinlock_t spinlock;
+};
+
+static struct nfp_flower_service *
+nfp_flower_service_handle_get(struct nfp_app_fw_flower *app)
+{
+ return app->pf_hw->pf_dev->process_share.fl_service;
+}
+
+static int
+nfp_flower_service_loop(void *arg)
+{
+ uint16_t slot;
+ struct nfp_app_fw_flower *app;
+ struct nfp_flower_service *service_handle;
+
+ service_handle = arg;
+ /* Waiting for enabling service */
+ while (!service_handle->enabled)
+ rte_delay_ms(1);
+
+ while (rte_service_runstate_get(service_handle->info.id) != 0) {
+ rte_spinlock_lock(&service_handle->spinlock);
+ for (slot = 0; slot < MAX_FLOWER_SERVICE_SLOT; slot++) {
+ app = service_handle->slots[slot];
+ if (app == NULL)
+ continue;
+
+ nfp_flower_ctrl_vnic_process(app);
+ }
+ rte_spinlock_unlock(&service_handle->spinlock);
+ }
+
+ return 0;
+}
+
+static int
+nfp_flower_service_enable(struct nfp_flower_service *service_handle)
+{
+ int ret;
+
+ const struct rte_service_spec flower_service = {
+ .name = "flower_ctrl_vnic_service",
+ .callback = nfp_flower_service_loop,
+ .callback_userdata = (void *)service_handle,
+ };
+
+ ret = nfp_service_enable(&flower_service, &service_handle->info);
+ if (ret != 0)
+ return ret;
+
+ rte_spinlock_init(&service_handle->spinlock);
+ service_handle->enabled = true;
+
+ return 0;
+}
+
+static uint16_t
+nfp_flower_service_insert(struct nfp_app_fw_flower *app,
+ struct nfp_flower_service *service_handle)
+{
+ uint16_t slot;
+
+ rte_spinlock_lock(&service_handle->spinlock);
+ for (slot = 0; slot < MAX_FLOWER_SERVICE_SLOT; slot++) {
+ if (service_handle->slots[slot] == NULL) {
+ service_handle->slots[slot] = app;
+ break;
+ }
+ }
+ rte_spinlock_unlock(&service_handle->spinlock);
+
+ return slot;
+}
+
+int
+nfp_flower_service_start(void *app_fw_flower)
+{
+ int ret;
+ struct nfp_flower_service *service_handle;
+ struct nfp_app_fw_flower *app = app_fw_flower;
+
+ service_handle = nfp_flower_service_handle_get(app);
+ if (service_handle == NULL) {
+ PMD_DRV_LOG(ERR, "Can not get service handle");
+ return -EINVAL;
+ }
+
+ /* Enable flower service when driver initializes the first NIC */
+ if (!service_handle->enabled) {
+ ret = nfp_flower_service_enable(service_handle);
+ if (ret != 0) {
+ PMD_DRV_LOG(ERR, "Could not enable flower service");
+ return -ESRCH;
+ }
+ }
+
+ /* Insert the NIC to flower service slot */
+ ret = nfp_flower_service_insert(app, service_handle);
+ if (ret == MAX_FLOWER_SERVICE_SLOT) {
+ PMD_DRV_LOG(ERR, "Flower ctrl vnic service slot over %u",
+ MAX_FLOWER_SERVICE_SLOT);
+ return -ENOSPC;
+ }
+
+ return 0;
+}
+
+void
+nfp_flower_service_stop(void *app_fw_flower)
+{
+ uint16_t slot;
+ uint16_t count;
+ struct nfp_flower_service *service_handle;
+ struct nfp_app_fw_flower *app = app_fw_flower;
+
+ service_handle = nfp_flower_service_handle_get(app);
+ if (service_handle == NULL) {
+ PMD_DRV_LOG(ERR, "Can not get service handle");
+ return;
+ }
+
+ rte_spinlock_lock(&service_handle->spinlock);
+ for (slot = 0; slot < MAX_FLOWER_SERVICE_SLOT; slot++) {
+ /* The app only in one slot */
+ if (service_handle->slots[slot] != app)
+ continue;
+
+ service_handle->slots[slot] = NULL;
+ }
+ rte_spinlock_unlock(&service_handle->spinlock);
+
+ /* Determine whether to disable service */
+ count = nfp_sync_handle_count_get(app->pf_hw->pf_dev->sync, NULL,
+ service_handle);
+ if (count > 1)
+ return;
+
+ if (nfp_service_disable(&service_handle->info) != 0)
+ PMD_DRV_LOG(ERR, "Could not disable service");
+}
+
+int
+nfp_flower_service_sync_alloc(void *app_fw_flower)
+{
+ struct nfp_flower_service *service_handle;
+ struct nfp_app_fw_flower *app = app_fw_flower;
+ struct nfp_pf_dev *pf_dev = app->pf_hw->pf_dev;
+
+ service_handle = nfp_sync_handle_alloc(pf_dev->sync, NULL,
+ NFP_SYNC_MAGIC_FL_SERVICE,
+ sizeof(struct nfp_flower_service));
+ if (service_handle == NULL)
+ return -ENOMEM;
+
+ pf_dev->process_share.fl_service = service_handle;
+
+ return 0;
+}
+
+void
+nfp_flower_service_sync_free(void *app_fw_flower)
+{
+ struct nfp_app_fw_flower *app = app_fw_flower;
+ struct nfp_pf_dev *pf_dev = app->pf_hw->pf_dev;
+
+ nfp_sync_handle_free(pf_dev->sync, NULL, pf_dev->process_share.fl_service);
+
+ pf_dev->process_share.fl_service = NULL;
+}
diff --git a/drivers/net/nfp/flower/nfp_flower_service.h b/drivers/net/nfp/flower/nfp_flower_service.h
new file mode 100644
index 0000000000..a75780274f
--- /dev/null
+++ b/drivers/net/nfp/flower/nfp_flower_service.h
@@ -0,0 +1,17 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright (c) 2024 Corigine, Inc.
+ * All rights reserved.
+ */
+
+#ifndef __NFP_FLOWER_SERVICE_H__
+#define __NFP_FLOWER_SERVICE_H__
+
+struct nfp_flower_service;
+
+int nfp_flower_service_start(void *app_fw_flower);
+void nfp_flower_service_stop(void *app_fw_flower);
+
+int nfp_flower_service_sync_alloc(void *app_fw_flower);
+void nfp_flower_service_sync_free(void *app_fw_flower);
+
+#endif /* __NFP_FLOWER_SERVICE_H__ */
diff --git a/drivers/net/nfp/meson.build b/drivers/net/nfp/meson.build
index 0f4beccbdf..e376fd328f 100644
--- a/drivers/net/nfp/meson.build
+++ b/drivers/net/nfp/meson.build
@@ -13,6 +13,7 @@ sources = files(
'flower/nfp_flower_ctrl.c',
'flower/nfp_flower_flow.c',
'flower/nfp_flower_representor.c',
+ 'flower/nfp_flower_service.c',
'nfd3/nfp_nfd3_dp.c',
'nfdk/nfp_nfdk_dp.c',
'nfpcore/nfp_cppcore.c',
diff --git a/drivers/net/nfp/nfp_net_common.h b/drivers/net/nfp/nfp_net_common.h
index eaa7501e8e..e374739022 100644
--- a/drivers/net/nfp/nfp_net_common.h
+++ b/drivers/net/nfp/nfp_net_common.h
@@ -12,6 +12,7 @@
#include <nfp_dev.h>
#include <rte_spinlock.h>
+#include "flower/nfp_flower_service.h"
#include "nfpcore/nfp_sync.h"
#include "nfp_net_ctrl.h"
#include "nfp_service.h"
@@ -80,6 +81,10 @@ struct nfp_multi_pf {
uint8_t *beat_addr;
};
+struct nfp_process_share {
+ struct nfp_flower_service *fl_service;
+};
+
struct nfp_pf_dev {
/** Backpointer to associated pci device */
struct rte_pci_device *pci_dev;
@@ -114,6 +119,7 @@ struct nfp_pf_dev {
/** Synchronized info */
struct nfp_sync *sync;
+ struct nfp_process_share process_share;
};
#define NFP_NET_FLOW_LIMIT 1024
diff --git a/drivers/net/nfp/nfpcore/nfp_sync.h b/drivers/net/nfp/nfpcore/nfp_sync.h
index 82f01e2652..ff3f60937e 100644
--- a/drivers/net/nfp/nfpcore/nfp_sync.h
+++ b/drivers/net/nfp/nfpcore/nfp_sync.h
@@ -10,6 +10,8 @@
#include <bus_pci_driver.h>
+#define NFP_SYNC_MAGIC_FL_SERVICE 0x53594e41 /**< ASCII - SYNA */
+
struct nfp_sync;
struct nfp_sync *nfp_sync_alloc(void);
--
2.39.1
^ permalink raw reply [flat|nested] 14+ messages in thread
* Re: [PATCH v2 0/3] enhance NFP service framework
2024-02-02 3:04 ` [PATCH v2 0/3] enhance NFP service framework Chaoyong He
` (2 preceding siblings ...)
2024-02-02 3:04 ` [PATCH v2 3/3] net/nfp: flower driver uses one service core Chaoyong He
@ 2024-02-06 15:17 ` Ferruh Yigit
2024-02-08 9:52 ` Chaoyong He
2024-02-08 13:58 ` Ferruh Yigit
4 siblings, 1 reply; 14+ messages in thread
From: Ferruh Yigit @ 2024-02-06 15:17 UTC (permalink / raw)
To: Chaoyong He, dev; +Cc: oss-drivers
On 2/2/2024 3:04 AM, Chaoyong He wrote:
> Make multiple devices can use single core to run services for
> flower firmware.
> Also add synchronize module and service module to support it.
>
> ---
> v2:
> * Merge the last commit of v1 into the second commit.
>
Ask was to move the fix as first patch to make is backportable.
But by merging last (fix) commit into another one, you are completely
loosing the capability to backport it to LTS releases.
I just want to double check if this is intentionally and you don't want
to backport a fix?
> ---
>
> Long Wu (3):
> net/nfp: add synchronize module
> net/nfp: create new service code related files
> net/nfp: flower driver uses one service core
>
> drivers/net/nfp/flower/nfp_flower.c | 56 +--
> drivers/net/nfp/flower/nfp_flower_ctrl.c | 20 +-
> drivers/net/nfp/flower/nfp_flower_ctrl.h | 2 +-
> .../net/nfp/flower/nfp_flower_representor.c | 7 +
> drivers/net/nfp/flower/nfp_flower_service.c | 196 +++++++++
> drivers/net/nfp/flower/nfp_flower_service.h | 17 +
> drivers/net/nfp/meson.build | 3 +
> drivers/net/nfp/nfp_cpp_bridge.c | 91 +----
> drivers/net/nfp/nfp_cpp_bridge.h | 1 -
> drivers/net/nfp/nfp_ethdev.c | 31 +-
> drivers/net/nfp/nfp_net_common.h | 15 +-
> drivers/net/nfp/nfp_service.c | 117 ++++++
> drivers/net/nfp/nfp_service.h | 20 +
> drivers/net/nfp/nfpcore/nfp_sync.c | 382 ++++++++++++++++++
> drivers/net/nfp/nfpcore/nfp_sync.h | 31 ++
> 15 files changed, 854 insertions(+), 135 deletions(-)
> create mode 100644 drivers/net/nfp/flower/nfp_flower_service.c
> create mode 100644 drivers/net/nfp/flower/nfp_flower_service.h
> create mode 100644 drivers/net/nfp/nfp_service.c
> create mode 100644 drivers/net/nfp/nfp_service.h
> create mode 100644 drivers/net/nfp/nfpcore/nfp_sync.c
> create mode 100644 drivers/net/nfp/nfpcore/nfp_sync.h
>
^ permalink raw reply [flat|nested] 14+ messages in thread
* RE: [PATCH v2 0/3] enhance NFP service framework
2024-02-06 15:17 ` [PATCH v2 0/3] enhance NFP service framework Ferruh Yigit
@ 2024-02-08 9:52 ` Chaoyong He
0 siblings, 0 replies; 14+ messages in thread
From: Chaoyong He @ 2024-02-08 9:52 UTC (permalink / raw)
To: Ferruh Yigit, dev; +Cc: oss-drivers
> On 2/2/2024 3:04 AM, Chaoyong He wrote:
> > Make multiple devices can use single core to run services for flower
> > firmware.
> > Also add synchronize module and service module to support it.
> >
> > ---
> > v2:
> > * Merge the last commit of v1 into the second commit.
> >
>
> Ask was to move the fix as first patch to make is backportable.
> But by merging last (fix) commit into another one, you are completely loosing the
> capability to backport it to LTS releases.
>
> I just want to double check if this is intentionally and you don't want to backport
> a fix?
>
Yes, it's intentionally and we don't want to backport it, thanks.
>
> > ---
> >
> > Long Wu (3):
> > net/nfp: add synchronize module
> > net/nfp: create new service code related files
> > net/nfp: flower driver uses one service core
> >
> > drivers/net/nfp/flower/nfp_flower.c | 56 +--
> > drivers/net/nfp/flower/nfp_flower_ctrl.c | 20 +-
> > drivers/net/nfp/flower/nfp_flower_ctrl.h | 2 +-
> > .../net/nfp/flower/nfp_flower_representor.c | 7 +
> > drivers/net/nfp/flower/nfp_flower_service.c | 196 +++++++++
> > drivers/net/nfp/flower/nfp_flower_service.h | 17 +
> > drivers/net/nfp/meson.build | 3 +
> > drivers/net/nfp/nfp_cpp_bridge.c | 91 +----
> > drivers/net/nfp/nfp_cpp_bridge.h | 1 -
> > drivers/net/nfp/nfp_ethdev.c | 31 +-
> > drivers/net/nfp/nfp_net_common.h | 15 +-
> > drivers/net/nfp/nfp_service.c | 117 ++++++
> > drivers/net/nfp/nfp_service.h | 20 +
> > drivers/net/nfp/nfpcore/nfp_sync.c | 382 ++++++++++++++++++
> > drivers/net/nfp/nfpcore/nfp_sync.h | 31 ++
> > 15 files changed, 854 insertions(+), 135 deletions(-) create mode
> > 100644 drivers/net/nfp/flower/nfp_flower_service.c
> > create mode 100644 drivers/net/nfp/flower/nfp_flower_service.h
> > create mode 100644 drivers/net/nfp/nfp_service.c create mode 100644
> > drivers/net/nfp/nfp_service.h create mode 100644
> > drivers/net/nfp/nfpcore/nfp_sync.c
> > create mode 100644 drivers/net/nfp/nfpcore/nfp_sync.h
> >
^ permalink raw reply [flat|nested] 14+ messages in thread
* Re: [PATCH v2 0/3] enhance NFP service framework
2024-02-02 3:04 ` [PATCH v2 0/3] enhance NFP service framework Chaoyong He
` (3 preceding siblings ...)
2024-02-06 15:17 ` [PATCH v2 0/3] enhance NFP service framework Ferruh Yigit
@ 2024-02-08 13:58 ` Ferruh Yigit
4 siblings, 0 replies; 14+ messages in thread
From: Ferruh Yigit @ 2024-02-08 13:58 UTC (permalink / raw)
To: Chaoyong He, dev; +Cc: oss-drivers
On 2/2/2024 3:04 AM, Chaoyong He wrote:
> Make multiple devices can use single core to run services for
> flower firmware.
> Also add synchronize module and service module to support it.
>
> ---
> v2:
> * Merge the last commit of v1 into the second commit.
> ---
>
> Long Wu (3):
> net/nfp: add synchronize module
> net/nfp: create new service code related files
> net/nfp: flower driver uses one service core
>
Series applied to dpdk-next-net/main, thanks.
^ permalink raw reply [flat|nested] 14+ messages in thread
end of thread, other threads:[~2024-02-08 13:58 UTC | newest]
Thread overview: 14+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2024-01-24 9:25 [PATCH 0/4] enhance NFP service framework Chaoyong He
2024-01-24 9:25 ` [PATCH 1/4] net/nfp: add synchronize module Chaoyong He
2024-01-24 9:25 ` [PATCH 2/4] net/nfp: create new service code related files Chaoyong He
2024-01-24 9:25 ` [PATCH 3/4] net/nfp: flower driver uses one service core Chaoyong He
2024-01-24 9:25 ` [PATCH 4/4] net/nfp: fix stop cpp service problem Chaoyong He
2024-01-30 12:07 ` Ferruh Yigit
2024-01-30 12:17 ` Kevin Traynor
2024-02-02 3:04 ` [PATCH v2 0/3] enhance NFP service framework Chaoyong He
2024-02-02 3:04 ` [PATCH v2 1/3] net/nfp: add synchronize module Chaoyong He
2024-02-02 3:04 ` [PATCH v2 2/3] net/nfp: create new service code related files Chaoyong He
2024-02-02 3:04 ` [PATCH v2 3/3] net/nfp: flower driver uses one service core Chaoyong He
2024-02-06 15:17 ` [PATCH v2 0/3] enhance NFP service framework Ferruh Yigit
2024-02-08 9:52 ` Chaoyong He
2024-02-08 13:58 ` Ferruh Yigit
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).