* [PATCH v2 1/8] net/nfp: fix resource leak of secondary process
2024-04-19 5:23 ` [PATCH 0/8] refactor logic to support secondary process Chaoyong He
@ 2024-04-19 5:23 ` Chaoyong He
2024-04-19 5:23 ` [PATCH v2 2/8] net/nfp: fix configuration BAR problem Chaoyong He
` (8 subsequent siblings)
9 siblings, 0 replies; 24+ messages in thread
From: Chaoyong He @ 2024-04-19 5:23 UTC (permalink / raw)
To: dev; +Cc: oss-drivers, Chaoyong He, stable, Long Wu, Peng Zhang
Fix one resource leak problem in the abnormal logic of secondary process.
Fixes: 016141b18b3a ("net/nfp: refactor secondary process probe")
Cc: stable@dpdk.org
Signed-off-by: Chaoyong He <chaoyong.he@corigine.com>
Reviewed-by: Long Wu <long.wu@corigine.com>
Reviewed-by: Peng Zhang <peng.zhang@corigine.com>
---
drivers/net/nfp/nfp_ethdev.c | 4 +++-
1 file changed, 3 insertions(+), 1 deletion(-)
diff --git a/drivers/net/nfp/nfp_ethdev.c b/drivers/net/nfp/nfp_ethdev.c
index b711e15b9f..98d8e87028 100644
--- a/drivers/net/nfp/nfp_ethdev.c
+++ b/drivers/net/nfp/nfp_ethdev.c
@@ -2123,7 +2123,7 @@ nfp_pf_secondary_init(struct rte_pci_device *pci_dev)
if (sym_tbl == NULL) {
PMD_INIT_LOG(ERR, "Something is wrong with the firmware symbol table");
ret = -EIO;
- goto sync_free;
+ goto cpp_cleanup;
}
/* Read the app ID of the firmware loaded */
@@ -2171,6 +2171,8 @@ nfp_pf_secondary_init(struct rte_pci_device *pci_dev)
sym_tbl_cleanup:
free(sym_tbl);
+cpp_cleanup:
+ nfp_cpp_free(cpp);
sync_free:
nfp_sync_free(sync);
pf_cleanup:
--
2.39.1
^ permalink raw reply [flat|nested] 24+ messages in thread* [PATCH v2 2/8] net/nfp: fix configuration BAR problem
2024-04-19 5:23 ` [PATCH 0/8] refactor logic to support secondary process Chaoyong He
2024-04-19 5:23 ` [PATCH v2 1/8] net/nfp: fix resource leak of " Chaoyong He
@ 2024-04-19 5:23 ` Chaoyong He
2024-04-19 5:23 ` [PATCH v2 3/8] net/nfp: adjust the data field of Rx/Tx queue Chaoyong He
` (7 subsequent siblings)
9 siblings, 0 replies; 24+ messages in thread
From: Chaoyong He @ 2024-04-19 5:23 UTC (permalink / raw)
To: dev; +Cc: oss-drivers, Chaoyong He, stable, Long Wu, Peng Zhang
All the configuration bars are sit in an array, and the initial logic
sort all the configuration BAR from small to large based on size and index,
which result all valid bars are behind the invalid bars.
But the BAR alloc logic search this array from the very beginning and
only try limited times (equal to the valid bars number).
It's ok for primary process because which has enough valid bars, and
finally it can find one to use.
But for secondary process and run with igb_uio driver, the valid bars
are very limit, and it can not find one, and the logic will fail.
Fix this by drop the sort logic, and search the bar array from the end
to begin.
Fixes: 1fbe51cd9c3a ("net/nfp: extend usage of BAR from 8 to 24")
Cc: stable@dpdk.org
Signed-off-by: Chaoyong He <chaoyong.he@corigine.com>
Reviewed-by: Long Wu <long.wu@corigine.com>
Reviewed-by: Peng Zhang <peng.zhang@corigine.com>
---
drivers/net/nfp/nfpcore/nfp6000_pcie.c | 34 +++++++++-----------------
1 file changed, 11 insertions(+), 23 deletions(-)
diff --git a/drivers/net/nfp/nfpcore/nfp6000_pcie.c b/drivers/net/nfp/nfpcore/nfp6000_pcie.c
index a6fd89b6c8..ef1ffd6d01 100644
--- a/drivers/net/nfp/nfpcore/nfp6000_pcie.c
+++ b/drivers/net/nfp/nfpcore/nfp6000_pcie.c
@@ -263,19 +263,6 @@ nfp_bitsize_calc(uint64_t mask)
return bit_size;
}
-static int
-nfp_cmp_bars(const void *ptr_a,
- const void *ptr_b)
-{
- const struct nfp_bar *a = ptr_a;
- const struct nfp_bar *b = ptr_b;
-
- if (a->bitsize == b->bitsize)
- return a->index - b->index;
- else
- return a->bitsize - b->bitsize;
-}
-
static bool
nfp_bars_for_secondary(uint32_t index)
{
@@ -383,9 +370,6 @@ nfp_enable_bars(struct nfp_pcie_user *nfp)
if (nfp_bar_write(nfp, bar, barcfg_msix_general) < 0)
return -EIO;
- /* Sort bars by bit size - use the smallest possible first. */
- qsort(&nfp->bar[0], nfp->bars, sizeof(nfp->bar[0]), nfp_cmp_bars);
-
return 0;
}
@@ -466,16 +450,18 @@ find_matching_bar(struct nfp_pcie_user *nfp,
int width)
{
uint32_t n;
+ uint32_t index;
- for (n = 0; n < nfp->bars; n++) {
- struct nfp_bar *bar = &nfp->bar[n];
+ for (n = RTE_DIM(nfp->bar) ; n > 0; n--) {
+ index = n - 1;
+ struct nfp_bar *bar = &nfp->bar[index];
if (bar->lock)
continue;
if (matching_bar_exist(bar, target, action, token,
offset, size, width))
- return n;
+ return index;
}
return -1;
@@ -493,10 +479,12 @@ find_unused_bar_noblock(struct nfp_pcie_user *nfp,
{
int ret;
uint32_t n;
+ uint32_t index;
const struct nfp_bar *bar;
- for (n = 0; n < nfp->bars; n++) {
- bar = &nfp->bar[n];
+ for (n = RTE_DIM(nfp->bar); n > 0; n--) {
+ index = n - 1;
+ bar = &nfp->bar[index];
if (bar->bitsize == 0)
continue;
@@ -508,7 +496,7 @@ find_unused_bar_noblock(struct nfp_pcie_user *nfp,
continue;
if (!bar->lock)
- return n;
+ return index;
}
return -EAGAIN;
@@ -561,7 +549,7 @@ nfp_disable_bars(struct nfp_pcie_user *nfp)
uint32_t i;
struct nfp_bar *bar;
- for (i = 0; i < nfp->bars; i++) {
+ for (i = 0; i < RTE_DIM(nfp->bar); i++) {
bar = &nfp->bar[i];
if (bar->iomem != NULL) {
bar->iomem = NULL;
--
2.39.1
^ permalink raw reply [flat|nested] 24+ messages in thread* [PATCH v2 3/8] net/nfp: adjust the data field of Rx/Tx queue
2024-04-19 5:23 ` [PATCH 0/8] refactor logic to support secondary process Chaoyong He
2024-04-19 5:23 ` [PATCH v2 1/8] net/nfp: fix resource leak of " Chaoyong He
2024-04-19 5:23 ` [PATCH v2 2/8] net/nfp: fix configuration BAR problem Chaoyong He
@ 2024-04-19 5:23 ` Chaoyong He
2024-04-19 5:23 ` [PATCH v2 4/8] net/nfp: add the process private structure Chaoyong He
` (6 subsequent siblings)
9 siblings, 0 replies; 24+ messages in thread
From: Chaoyong He @ 2024-04-19 5:23 UTC (permalink / raw)
To: dev; +Cc: oss-drivers, Chaoyong He, Long Wu, Peng Zhang
Adjust the data field of Rx/Tx queue structure, move the cold data back
and hold position for hot data.
Signed-off-by: Chaoyong He <chaoyong.he@corigine.com>
Reviewed-by: Long Wu <long.wu@corigine.com>
Reviewed-by: Peng Zhang <peng.zhang@corigine.com>
---
drivers/net/nfp/nfp_rxtx.h | 71 +++++++++++++++++++-------------------
1 file changed, 36 insertions(+), 35 deletions(-)
diff --git a/drivers/net/nfp/nfp_rxtx.h b/drivers/net/nfp/nfp_rxtx.h
index f463b9cf75..0aa73a6432 100644
--- a/drivers/net/nfp/nfp_rxtx.h
+++ b/drivers/net/nfp/nfp_rxtx.h
@@ -31,21 +31,6 @@ struct nfp_net_txq {
/** Backpointer to nfp_net structure */
struct nfp_net_hw *hw;
- /** Point to the base of the queue structure on the NFP. */
- uint8_t *qcp_q;
-
- /**
- * Host side read and write pointer, they are free running and
- * have little relation to the QCP pointers.
- */
- uint32_t wr_p;
- uint32_t rd_p;
-
- /** The size of the queue in number of descriptors. */
- uint32_t tx_count;
-
- uint32_t tx_free_thresh;
-
/**
* For each descriptor keep a reference to the mbuf and
* DMA address used until completion is signalled.
@@ -61,6 +46,18 @@ struct nfp_net_txq {
struct nfp_net_nfdk_tx_desc *ktxds;
};
+ /**
+ * Host side read and write pointer, they are free running and
+ * have little relation to the QCP pointers.
+ */
+ uint32_t wr_p;
+ uint32_t rd_p;
+
+ /** The size of the queue in number of descriptors. */
+ uint32_t tx_count;
+
+ uint32_t tx_free_thresh;
+
/** The index of the QCP queue relative to the TX queue BAR. */
uint32_t tx_qcidx;
@@ -72,8 +69,8 @@ struct nfp_net_txq {
uint16_t data_pending;
/**
- * At this point 58 bytes have been used for all the fields in the
- * TX critical path. We have room for 6 bytes and still all placed
+ * At this point 50 bytes have been used for all the fields in the
+ * TX critical path. We have room for 14 bytes and still all placed
* in a cache line.
*/
uint64_t dma;
@@ -83,6 +80,9 @@ struct nfp_net_txq {
/** TX pointer ring write back area DMA address */
uint64_t txrwb_dma;
+
+ /** Point to the base of the queue structure on the NFP. */
+ uint8_t *qcp_q;
} __rte_aligned(64);
/* RX and freelist descriptor format */
@@ -145,19 +145,6 @@ struct nfp_net_rxq {
*/
uint8_t *qcp_fl;
- /**
- * Host side read pointer, free running and have little relation
- * to the QCP pointers. It is where the driver start reading
- * descriptors for newly arrive packets from.
- */
- uint32_t rd_p;
-
- /**
- * The index of the QCP queue relative to the RX queue BAR
- * used for the freelist.
- */
- uint32_t fl_qcidx;
-
/**
* For each buffer placed on the freelist, record the
* associated mbuf.
@@ -177,6 +164,14 @@ struct nfp_net_rxq {
* safely copied to the mbuf using the NFP_NET_RX_OFFSET.
*/
struct rte_mempool *mem_pool;
+
+ /**
+ * Host side read pointer, free running and have little relation
+ * to the QCP pointers. It is where the driver start reading
+ * descriptors for newly arrive packets from.
+ */
+ uint32_t rd_p;
+
uint16_t mbuf_size;
/**
@@ -186,9 +181,6 @@ struct nfp_net_rxq {
uint16_t rx_free_thresh;
uint16_t nb_rx_hold;
- /** The size of the queue in number of descriptors */
- uint16_t rx_count;
-
/** Referencing dev->data->port_id */
uint16_t port_id;
@@ -196,11 +188,20 @@ struct nfp_net_rxq {
uint16_t qidx;
/**
- * At this point 60 bytes have been used for all the fields in the
- * RX critical path. We have room for 4 bytes and still all placed
+ * At this point 54 bytes have been used for all the fields in the
+ * RX critical path. We have room for 10 bytes and still all placed
* in a cache line.
*/
+ /** The size of the queue in number of descriptors */
+ uint16_t rx_count;
+
+ /**
+ * The index of the QCP queue relative to the RX queue BAR
+ * used for the freelist.
+ */
+ uint32_t fl_qcidx;
+
/** DMA address of the queue */
uint64_t dma;
} __rte_aligned(64);
--
2.39.1
^ permalink raw reply [flat|nested] 24+ messages in thread* [PATCH v2 4/8] net/nfp: add the process private structure
2024-04-19 5:23 ` [PATCH 0/8] refactor logic to support secondary process Chaoyong He
` (2 preceding siblings ...)
2024-04-19 5:23 ` [PATCH v2 3/8] net/nfp: adjust the data field of Rx/Tx queue Chaoyong He
@ 2024-04-19 5:23 ` Chaoyong He
2024-04-19 5:23 ` [PATCH v2 5/8] net/nfp: move device info data field Chaoyong He
` (5 subsequent siblings)
9 siblings, 0 replies; 24+ messages in thread
From: Chaoyong He @ 2024-04-19 5:23 UTC (permalink / raw)
To: dev; +Cc: oss-drivers, Chaoyong He, Long Wu, Peng Zhang
Add a new data structure to hold the process private data, and modify
logic to make sure only get 'pf_dev' from the process private data.
Signed-off-by: Chaoyong He <chaoyong.he@corigine.com>
Reviewed-by: Long Wu <long.wu@corigine.com>
Reviewed-by: Peng Zhang <peng.zhang@corigine.com>
---
drivers/net/nfp/flower/nfp_flower.c | 71 ++++++------
drivers/net/nfp/flower/nfp_flower.h | 8 +-
drivers/net/nfp/flower/nfp_flower_cmsg.c | 5 +-
drivers/net/nfp/flower/nfp_flower_cmsg.h | 3 +-
.../net/nfp/flower/nfp_flower_representor.c | 85 ++++++++++----
.../net/nfp/flower/nfp_flower_representor.h | 3 +-
drivers/net/nfp/flower/nfp_flower_service.c | 32 +++---
drivers/net/nfp/flower/nfp_flower_service.h | 10 +-
drivers/net/nfp/nfd3/nfp_nfd3_dp.c | 1 +
drivers/net/nfp/nfdk/nfp_nfdk_dp.c | 1 +
drivers/net/nfp/nfp_ethdev.c | 107 +++++++++++++-----
drivers/net/nfp/nfp_net_common.c | 67 +++++++----
drivers/net/nfp/nfp_net_common.h | 15 ++-
drivers/net/nfp/nfp_net_flow.c | 20 +++-
drivers/net/nfp/nfp_rxtx.c | 5 +-
drivers/net/nfp/nfp_rxtx.h | 10 +-
16 files changed, 284 insertions(+), 159 deletions(-)
diff --git a/drivers/net/nfp/flower/nfp_flower.c b/drivers/net/nfp/flower/nfp_flower.c
index 303f6bd3f6..8dd9e23c78 100644
--- a/drivers/net/nfp/flower/nfp_flower.c
+++ b/drivers/net/nfp/flower/nfp_flower.c
@@ -102,14 +102,14 @@ static const struct eth_dev_ops nfp_flower_pf_vnic_ops = {
};
static inline struct nfp_flower_representor *
-nfp_flower_get_repr(struct nfp_net_hw *hw,
+nfp_flower_get_repr(struct nfp_net_hw_priv *hw_priv,
uint32_t port_id)
{
uint8_t port;
struct nfp_app_fw_flower *app_fw_flower;
/* Obtain handle to app_fw_flower here */
- app_fw_flower = NFP_PRIV_TO_APP_FW_FLOWER(hw->pf_dev->app_fw_priv);
+ app_fw_flower = NFP_PRIV_TO_APP_FW_FLOWER(hw_priv->pf_dev->app_fw_priv);
switch (NFP_FLOWER_CMSG_PORT_TYPE(port_id)) {
case NFP_FLOWER_CMSG_PORT_TYPE_PHYS_PORT:
@@ -126,15 +126,15 @@ nfp_flower_get_repr(struct nfp_net_hw *hw,
}
bool
-nfp_flower_pf_dispatch_pkts(struct nfp_net_hw *hw,
+nfp_flower_pf_dispatch_pkts(struct nfp_net_hw_priv *hw_priv,
struct rte_mbuf *mbuf,
uint32_t port_id)
{
struct nfp_flower_representor *repr;
- repr = nfp_flower_get_repr(hw, port_id);
+ repr = nfp_flower_get_repr(hw_priv, port_id);
if (repr == NULL) {
- PMD_RX_LOG(ERR, "Can not get repr for port %u", hw->idx);
+ PMD_RX_LOG(ERR, "Can not get repr for port %u", port_id);
return false;
}
@@ -189,13 +189,14 @@ nfp_flower_pf_xmit_pkts(void *tx_queue,
struct nfp_app_fw_flower *app_fw_flower;
txq = tx_queue;
- app_fw_flower = txq->hw->pf_dev->app_fw_priv;
+ app_fw_flower = txq->hw_priv->pf_dev->app_fw_priv;
return app_fw_flower->nfd_func.pf_xmit_t(tx_queue, tx_pkts, nb_pkts);
}
static int
-nfp_flower_init_vnic_common(struct nfp_net_hw *hw,
+nfp_flower_init_vnic_common(struct nfp_net_hw_priv *hw_priv,
+ struct nfp_net_hw *hw,
const char *vnic_type)
{
int err;
@@ -205,8 +206,8 @@ nfp_flower_init_vnic_common(struct nfp_net_hw *hw,
struct nfp_pf_dev *pf_dev;
struct rte_pci_device *pci_dev;
- pf_dev = hw->pf_dev;
- pci_dev = hw->pf_dev->pci_dev;
+ pf_dev = hw_priv->pf_dev;
+ pci_dev = pf_dev->pci_dev;
PMD_INIT_LOG(DEBUG, "%s vNIC ctrl bar: %p", vnic_type, hw->super.ctrl_bar);
@@ -239,13 +240,15 @@ nfp_flower_init_vnic_common(struct nfp_net_hw *hw,
}
static int
-nfp_flower_init_ctrl_vnic(struct nfp_net_hw *hw)
+nfp_flower_init_ctrl_vnic(struct nfp_app_fw_flower *app_fw_flower,
+ struct nfp_net_hw_priv *hw_priv)
{
uint16_t i;
int ret = 0;
uint16_t n_txq;
uint16_t n_rxq;
const char *pci_name;
+ struct nfp_net_hw *hw;
unsigned int numa_node;
struct rte_mempool *mp;
struct nfp_net_rxq *rxq;
@@ -253,16 +256,15 @@ nfp_flower_init_ctrl_vnic(struct nfp_net_hw *hw)
struct nfp_pf_dev *pf_dev;
struct rte_eth_dev *eth_dev;
const struct rte_memzone *tz;
- struct nfp_app_fw_flower *app_fw_flower;
char ctrl_rxring_name[RTE_MEMZONE_NAMESIZE];
char ctrl_txring_name[RTE_MEMZONE_NAMESIZE];
char ctrl_pktmbuf_pool_name[RTE_MEMZONE_NAMESIZE];
/* Set up some pointers here for ease of use */
- pf_dev = hw->pf_dev;
- app_fw_flower = NFP_PRIV_TO_APP_FW_FLOWER(pf_dev->app_fw_priv);
+ pf_dev = hw_priv->pf_dev;
+ hw = app_fw_flower->ctrl_hw;
- ret = nfp_flower_init_vnic_common(hw, "ctrl_vnic");
+ ret = nfp_flower_init_vnic_common(hw_priv, hw, "ctrl_vnic");
if (ret != 0) {
PMD_INIT_LOG(ERR, "Could not init pf vnic");
return -EINVAL;
@@ -397,6 +399,7 @@ nfp_flower_init_ctrl_vnic(struct nfp_net_hw *hw)
nfp_net_reset_rx_queue(rxq);
rxq->hw = hw;
+ rxq->hw_priv = hw_priv;
/*
* Telling the HW about the physical address of the RX ring and number
@@ -462,6 +465,7 @@ nfp_flower_init_ctrl_vnic(struct nfp_net_hw *hw)
nfp_net_reset_tx_queue(txq);
txq->hw = hw;
+ txq->hw_priv = hw_priv;
/*
* Telling the HW about the physical address of the TX ring and number
@@ -472,7 +476,7 @@ nfp_flower_init_ctrl_vnic(struct nfp_net_hw *hw)
}
/* Alloc sync memory zone */
- ret = nfp_flower_service_sync_alloc(app_fw_flower);
+ ret = nfp_flower_service_sync_alloc(hw_priv);
if (ret != 0) {
PMD_INIT_LOG(ERR, "Alloc sync memory zone failed");
goto tx_queue_setup_cleanup;
@@ -512,21 +516,22 @@ nfp_flower_init_ctrl_vnic(struct nfp_net_hw *hw)
}
static void
-nfp_flower_cleanup_ctrl_vnic(struct nfp_net_hw *hw)
+nfp_flower_cleanup_ctrl_vnic(struct nfp_app_fw_flower *app_fw_flower,
+ struct nfp_net_hw_priv *hw_priv)
{
uint32_t i;
const char *pci_name;
+ struct nfp_net_hw *hw;
struct nfp_net_rxq *rxq;
struct nfp_net_txq *txq;
struct rte_eth_dev *eth_dev;
- struct nfp_app_fw_flower *app_fw_flower;
char ctrl_txring_name[RTE_MEMZONE_NAMESIZE];
char ctrl_rxring_name[RTE_MEMZONE_NAMESIZE];
+ hw = app_fw_flower->ctrl_hw;
eth_dev = hw->eth_dev;
- app_fw_flower = NFP_PRIV_TO_APP_FW_FLOWER(hw->pf_dev->app_fw_priv);
- pci_name = strchr(app_fw_flower->pf_hw->pf_dev->pci_dev->name, ':') + 1;
+ pci_name = strchr(hw_priv->pf_dev->pci_dev->name, ':') + 1;
snprintf(ctrl_txring_name, sizeof(ctrl_txring_name), "%s_cttx_ring", pci_name);
for (i = 0; i < hw->max_tx_queues; i++) {
@@ -548,7 +553,7 @@ nfp_flower_cleanup_ctrl_vnic(struct nfp_net_hw *hw)
}
}
- nfp_flower_service_sync_free(app_fw_flower);
+ nfp_flower_service_sync_free(hw_priv);
rte_free(eth_dev->data->tx_queues);
rte_free(eth_dev->data->rx_queues);
rte_mempool_free(app_fw_flower->ctrl_pktmbuf_pool);
@@ -634,7 +639,7 @@ nfp_flower_nfd_func_register(struct nfp_app_fw_flower *app_fw_flower)
}
int
-nfp_init_app_fw_flower(struct nfp_pf_dev *pf_dev,
+nfp_init_app_fw_flower(struct nfp_net_hw_priv *hw_priv,
const struct nfp_dev_info *dev_info)
{
int ret;
@@ -644,6 +649,7 @@ nfp_init_app_fw_flower(struct nfp_pf_dev *pf_dev,
struct nfp_net_hw *pf_hw;
struct nfp_net_hw *ctrl_hw;
struct nfp_app_fw_flower *app_fw_flower;
+ struct nfp_pf_dev *pf_dev = hw_priv->pf_dev;
numa_node = rte_socket_id();
@@ -702,11 +708,10 @@ nfp_init_app_fw_flower(struct nfp_pf_dev *pf_dev,
/* Fill in the PF vNIC and populate app struct */
app_fw_flower->pf_hw = pf_hw;
pf_hw->super.ctrl_bar = pf_dev->ctrl_bar;
- pf_hw->pf_dev = pf_dev;
pf_hw->cpp = pf_dev->cpp;
pf_hw->dev_info = dev_info;
- ret = nfp_flower_init_vnic_common(app_fw_flower->pf_hw, "pf_vnic");
+ ret = nfp_flower_init_vnic_common(hw_priv, pf_hw, "pf_vnic");
if (ret != 0) {
PMD_INIT_LOG(ERR, "Could not initialize flower PF vNIC");
goto pf_cpp_area_cleanup;
@@ -728,11 +733,10 @@ nfp_init_app_fw_flower(struct nfp_pf_dev *pf_dev,
}
/* Now populate the ctrl vNIC */
- ctrl_hw->pf_dev = pf_dev;
ctrl_hw->cpp = pf_dev->cpp;
ctrl_hw->dev_info = dev_info;
- ret = nfp_flower_init_ctrl_vnic(app_fw_flower->ctrl_hw);
+ ret = nfp_flower_init_ctrl_vnic(app_fw_flower, hw_priv);
if (ret != 0) {
PMD_INIT_LOG(ERR, "Could not initialize flower ctrl vNIC");
goto ctrl_cpp_area_cleanup;
@@ -746,14 +750,14 @@ nfp_init_app_fw_flower(struct nfp_pf_dev *pf_dev,
}
/* Start up flower services */
- ret = nfp_flower_service_start(app_fw_flower);
+ ret = nfp_flower_service_start(app_fw_flower, hw_priv);
if (ret != 0) {
PMD_INIT_LOG(ERR, "Could not enable flower services");
ret = -ESRCH;
goto ctrl_vnic_cleanup;
}
- ret = nfp_flower_repr_create(app_fw_flower);
+ ret = nfp_flower_repr_create(app_fw_flower, hw_priv);
if (ret != 0) {
PMD_INIT_LOG(ERR, "Could not create representor ports");
goto ctrl_vnic_service_stop;
@@ -762,9 +766,9 @@ nfp_init_app_fw_flower(struct nfp_pf_dev *pf_dev,
return 0;
ctrl_vnic_service_stop:
- nfp_flower_service_stop(app_fw_flower);
+ nfp_flower_service_stop(app_fw_flower, hw_priv);
ctrl_vnic_cleanup:
- nfp_flower_cleanup_ctrl_vnic(app_fw_flower->ctrl_hw);
+ nfp_flower_cleanup_ctrl_vnic(app_fw_flower, hw_priv);
ctrl_cpp_area_cleanup:
nfp_cpp_area_free(ctrl_hw->ctrl_area);
pf_cpp_area_cleanup:
@@ -782,12 +786,13 @@ nfp_init_app_fw_flower(struct nfp_pf_dev *pf_dev,
}
void
-nfp_uninit_app_fw_flower(struct nfp_pf_dev *pf_dev)
+nfp_uninit_app_fw_flower(struct nfp_net_hw_priv *hw_priv)
{
struct nfp_app_fw_flower *app_fw_flower;
+ struct nfp_pf_dev *pf_dev = hw_priv->pf_dev;
app_fw_flower = pf_dev->app_fw_priv;
- nfp_flower_cleanup_ctrl_vnic(app_fw_flower->ctrl_hw);
+ nfp_flower_cleanup_ctrl_vnic(app_fw_flower, hw_priv);
nfp_cpp_area_free(app_fw_flower->ctrl_hw->ctrl_area);
nfp_cpp_area_free(pf_dev->ctrl_area);
rte_free(app_fw_flower->pf_hw);
@@ -799,7 +804,7 @@ nfp_uninit_app_fw_flower(struct nfp_pf_dev *pf_dev)
}
int
-nfp_secondary_init_app_fw_flower(struct nfp_pf_dev *pf_dev)
+nfp_secondary_init_app_fw_flower(struct nfp_net_hw_priv *hw_priv)
{
struct rte_eth_dev *eth_dev;
const char *port_name = "pf_vnic_eth_dev";
@@ -812,7 +817,7 @@ nfp_secondary_init_app_fw_flower(struct nfp_pf_dev *pf_dev)
return -ENODEV;
}
- eth_dev->process_private = pf_dev;
+ eth_dev->process_private = hw_priv;
eth_dev->dev_ops = &nfp_flower_pf_vnic_ops;
eth_dev->rx_pkt_burst = nfp_net_recv_pkts;
eth_dev->tx_pkt_burst = nfp_flower_pf_xmit_pkts;
diff --git a/drivers/net/nfp/flower/nfp_flower.h b/drivers/net/nfp/flower/nfp_flower.h
index 8393de66c5..5cc29dcc87 100644
--- a/drivers/net/nfp/flower/nfp_flower.h
+++ b/drivers/net/nfp/flower/nfp_flower.h
@@ -104,11 +104,11 @@ nfp_flower_support_decap_v2(const struct nfp_app_fw_flower *app_fw_flower)
return app_fw_flower->ext_features & NFP_FL_FEATS_DECAP_V2;
}
-int nfp_init_app_fw_flower(struct nfp_pf_dev *pf_dev,
+int nfp_init_app_fw_flower(struct nfp_net_hw_priv *hw_priv,
const struct nfp_dev_info *dev_info);
-void nfp_uninit_app_fw_flower(struct nfp_pf_dev *pf_dev);
-int nfp_secondary_init_app_fw_flower(struct nfp_pf_dev *pf_dev);
-bool nfp_flower_pf_dispatch_pkts(struct nfp_net_hw *hw,
+void nfp_uninit_app_fw_flower(struct nfp_net_hw_priv *hw_priv);
+int nfp_secondary_init_app_fw_flower(struct nfp_net_hw_priv *hw_priv);
+bool nfp_flower_pf_dispatch_pkts(struct nfp_net_hw_priv *hw_priv,
struct rte_mbuf *mbuf,
uint32_t port_id);
uint16_t nfp_flower_pf_xmit_pkts(void *tx_queue, struct rte_mbuf **tx_pkts,
diff --git a/drivers/net/nfp/flower/nfp_flower_cmsg.c b/drivers/net/nfp/flower/nfp_flower_cmsg.c
index eefeb0d887..2a0e9afe90 100644
--- a/drivers/net/nfp/flower/nfp_flower_cmsg.c
+++ b/drivers/net/nfp/flower/nfp_flower_cmsg.c
@@ -81,7 +81,8 @@ nfp_flower_cmsg_mac_repr_fill(struct rte_mbuf *m,
}
int
-nfp_flower_cmsg_mac_repr(struct nfp_app_fw_flower *app_fw_flower)
+nfp_flower_cmsg_mac_repr(struct nfp_app_fw_flower *app_fw_flower,
+ struct nfp_eth_table *nfp_eth_table)
{
uint8_t i;
uint16_t cnt;
@@ -89,7 +90,6 @@ nfp_flower_cmsg_mac_repr(struct nfp_app_fw_flower *app_fw_flower)
uint32_t nbi_port;
uint32_t phys_port;
struct rte_mbuf *mbuf;
- struct nfp_eth_table *nfp_eth_table;
mbuf = rte_pktmbuf_alloc(app_fw_flower->ctrl_pktmbuf_pool);
if (mbuf == NULL) {
@@ -100,7 +100,6 @@ nfp_flower_cmsg_mac_repr(struct nfp_app_fw_flower *app_fw_flower)
nfp_flower_cmsg_mac_repr_init(mbuf, app_fw_flower);
/* Fill in the mac repr cmsg */
- nfp_eth_table = app_fw_flower->pf_hw->pf_dev->nfp_eth_table;
for (i = 0; i < app_fw_flower->num_phyport_reprs; i++) {
nbi = nfp_eth_table->ports[i].nbi;
nbi_port = nfp_eth_table->ports[i].base;
diff --git a/drivers/net/nfp/flower/nfp_flower_cmsg.h b/drivers/net/nfp/flower/nfp_flower_cmsg.h
index c94ea706bb..afaf733ef7 100644
--- a/drivers/net/nfp/flower/nfp_flower_cmsg.h
+++ b/drivers/net/nfp/flower/nfp_flower_cmsg.h
@@ -975,7 +975,8 @@ struct nfp_fl_act_mark {
rte_be32_t mark;
};
-int nfp_flower_cmsg_mac_repr(struct nfp_app_fw_flower *app_fw_flower);
+int nfp_flower_cmsg_mac_repr(struct nfp_app_fw_flower *app_fw_flower,
+ struct nfp_eth_table *nfp_eth_table);
int nfp_flower_cmsg_repr_reify(struct nfp_app_fw_flower *app_fw_flower,
struct nfp_flower_representor *repr);
int nfp_flower_cmsg_port_mod(struct nfp_app_fw_flower *app_fw_flower,
diff --git a/drivers/net/nfp/flower/nfp_flower_representor.c b/drivers/net/nfp/flower/nfp_flower_representor.c
index d4c3c30682..934f078dca 100644
--- a/drivers/net/nfp/flower/nfp_flower_representor.c
+++ b/drivers/net/nfp/flower/nfp_flower_representor.c
@@ -83,17 +83,17 @@ nfp_flower_repr_dev_infos_get(__rte_unused struct rte_eth_dev *dev,
static int
nfp_flower_repr_dev_start(struct rte_eth_dev *dev)
{
+ uint16_t i;
+ struct nfp_net_hw_priv *hw_priv;
struct nfp_flower_representor *repr;
struct nfp_app_fw_flower *app_fw_flower;
- uint16_t i;
repr = dev->data->dev_private;
+ hw_priv = dev->process_private;
app_fw_flower = repr->app_fw_flower;
- if (repr->repr_type == NFP_REPR_TYPE_PHYS_PORT) {
- nfp_eth_set_configured(app_fw_flower->pf_hw->pf_dev->cpp,
- repr->nfp_idx, 1);
- }
+ if (repr->repr_type == NFP_REPR_TYPE_PHYS_PORT)
+ nfp_eth_set_configured(hw_priv->pf_dev->cpp, repr->nfp_idx, 1);
nfp_flower_cmsg_port_mod(app_fw_flower, repr->port_id, true);
@@ -108,19 +108,19 @@ nfp_flower_repr_dev_start(struct rte_eth_dev *dev)
static int
nfp_flower_repr_dev_stop(struct rte_eth_dev *dev)
{
+ uint16_t i;
+ struct nfp_net_hw_priv *hw_priv;
struct nfp_flower_representor *repr;
struct nfp_app_fw_flower *app_fw_flower;
- uint16_t i;
repr = dev->data->dev_private;
+ hw_priv = dev->process_private;
app_fw_flower = repr->app_fw_flower;
nfp_flower_cmsg_port_mod(app_fw_flower, repr->port_id, false);
- if (repr->repr_type == NFP_REPR_TYPE_PHYS_PORT) {
- nfp_eth_set_configured(app_fw_flower->pf_hw->pf_dev->cpp,
- repr->nfp_idx, 0);
- }
+ if (repr->repr_type == NFP_REPR_TYPE_PHYS_PORT)
+ nfp_eth_set_configured(hw_priv->pf_dev->cpp, repr->nfp_idx, 0);
for (i = 0; i < dev->data->nb_rx_queues; i++)
dev->data->rx_queue_state[i] = RTE_ETH_QUEUE_STATE_STOPPED;
@@ -408,8 +408,8 @@ static int
nfp_flower_repr_dev_close(struct rte_eth_dev *dev)
{
uint16_t i;
- struct nfp_net_hw *hw;
struct nfp_pf_dev *pf_dev;
+ struct nfp_net_hw_priv *hw_priv;
struct nfp_flower_representor *repr;
struct nfp_app_fw_flower *app_fw_flower;
@@ -417,9 +417,10 @@ nfp_flower_repr_dev_close(struct rte_eth_dev *dev)
return 0;
repr = dev->data->dev_private;
+ hw_priv = dev->process_private;
+
app_fw_flower = repr->app_fw_flower;
- hw = app_fw_flower->pf_hw;
- pf_dev = hw->pf_dev;
+ pf_dev = hw_priv->pf_dev;
if (pf_dev->app_fw_id != NFP_APP_FW_FLOWER_NIC)
return -EINVAL;
@@ -442,14 +443,14 @@ nfp_flower_repr_dev_close(struct rte_eth_dev *dev)
return 0;
/* Stop flower service first */
- nfp_flower_service_stop(app_fw_flower);
+ nfp_flower_service_stop(app_fw_flower, hw_priv);
/* Disable cpp service */
nfp_service_disable(&pf_dev->cpp_service_info);
/* Now it is safe to free all PF resources */
- nfp_uninit_app_fw_flower(pf_dev);
- nfp_pf_uninit(pf_dev);
+ nfp_uninit_app_fw_flower(hw_priv);
+ nfp_pf_uninit(hw_priv);
return 0;
}
@@ -708,8 +709,43 @@ nfp_flower_repr_free_all(struct nfp_app_fw_flower *app_fw_flower)
}
}
+static void
+nfp_flower_repr_priv_init(struct nfp_app_fw_flower *app_fw_flower,
+ struct nfp_net_hw_priv *hw_priv)
+{
+ uint32_t i;
+ struct rte_eth_dev *eth_dev;
+ struct nfp_flower_representor *repr;
+
+ repr = app_fw_flower->pf_repr;
+ if (repr != NULL) {
+ eth_dev = repr->eth_dev;
+ if (eth_dev != NULL)
+ eth_dev->process_private = hw_priv;
+ }
+
+ for (i = 0; i < NFP_MAX_PHYPORTS; i++) {
+ repr = app_fw_flower->phy_reprs[i];
+ if (repr != NULL) {
+ eth_dev = repr->eth_dev;
+ if (eth_dev != NULL)
+ eth_dev->process_private = hw_priv;
+ }
+ }
+
+ for (i = 0; i < MAX_FLOWER_VFS; i++) {
+ repr = app_fw_flower->vf_reprs[i];
+ if (repr != NULL) {
+ eth_dev = repr->eth_dev;
+ if (eth_dev != NULL)
+ eth_dev->process_private = hw_priv;
+ }
+ }
+}
+
static int
-nfp_flower_repr_alloc(struct nfp_app_fw_flower *app_fw_flower)
+nfp_flower_repr_alloc(struct nfp_app_fw_flower *app_fw_flower,
+ struct nfp_net_hw_priv *hw_priv)
{
int i;
int ret;
@@ -723,11 +759,11 @@ nfp_flower_repr_alloc(struct nfp_app_fw_flower *app_fw_flower)
.app_fw_flower = app_fw_flower,
};
- nfp_eth_table = app_fw_flower->pf_hw->pf_dev->nfp_eth_table;
+ nfp_eth_table = hw_priv->pf_dev->nfp_eth_table;
eth_dev = app_fw_flower->ctrl_hw->eth_dev;
/* Send a NFP_FLOWER_CMSG_TYPE_MAC_REPR cmsg to hardware */
- ret = nfp_flower_cmsg_mac_repr(app_fw_flower);
+ ret = nfp_flower_cmsg_mac_repr(app_fw_flower, nfp_eth_table);
if (ret != 0) {
PMD_INIT_LOG(ERR, "Cloud not send mac repr cmsgs");
return ret;
@@ -739,7 +775,7 @@ nfp_flower_repr_alloc(struct nfp_app_fw_flower *app_fw_flower)
/* PF vNIC reprs get a random MAC address */
rte_eth_random_addr(flower_repr.mac_addr.addr_bytes);
- pci_dev = app_fw_flower->pf_hw->pf_dev->pci_dev;
+ pci_dev = hw_priv->pf_dev->pci_dev;
pci_name = strchr(pci_dev->name, ':') + 1;
@@ -813,6 +849,8 @@ nfp_flower_repr_alloc(struct nfp_app_fw_flower *app_fw_flower)
if (i < app_fw_flower->num_vf_reprs)
goto repr_free;
+ nfp_flower_repr_priv_init(app_fw_flower, hw_priv);
+
return 0;
repr_free:
@@ -822,7 +860,8 @@ nfp_flower_repr_alloc(struct nfp_app_fw_flower *app_fw_flower)
}
int
-nfp_flower_repr_create(struct nfp_app_fw_flower *app_fw_flower)
+nfp_flower_repr_create(struct nfp_app_fw_flower *app_fw_flower,
+ struct nfp_net_hw_priv *hw_priv)
{
int ret;
struct nfp_pf_dev *pf_dev;
@@ -832,7 +871,7 @@ nfp_flower_repr_create(struct nfp_app_fw_flower *app_fw_flower)
.nb_representor_ports = 0
};
- pf_dev = app_fw_flower->pf_hw->pf_dev;
+ pf_dev = hw_priv->pf_dev;
pci_dev = pf_dev->pci_dev;
/* Allocate a switch domain for the flower app */
@@ -876,7 +915,7 @@ nfp_flower_repr_create(struct nfp_app_fw_flower *app_fw_flower)
PMD_INIT_LOG(INFO, "%d number of VF reprs", app_fw_flower->num_vf_reprs);
PMD_INIT_LOG(INFO, "%d number of phyport reprs", app_fw_flower->num_phyport_reprs);
- ret = nfp_flower_repr_alloc(app_fw_flower);
+ ret = nfp_flower_repr_alloc(app_fw_flower, hw_priv);
if (ret != 0) {
PMD_INIT_LOG(ERR, "representors allocation failed");
ret = -EINVAL;
diff --git a/drivers/net/nfp/flower/nfp_flower_representor.h b/drivers/net/nfp/flower/nfp_flower_representor.h
index 7a4e7ecfc1..41965f6e0e 100644
--- a/drivers/net/nfp/flower/nfp_flower_representor.h
+++ b/drivers/net/nfp/flower/nfp_flower_representor.h
@@ -23,7 +23,8 @@ struct nfp_flower_representor {
struct rte_eth_dev *eth_dev;
};
-int nfp_flower_repr_create(struct nfp_app_fw_flower *app_fw_flower);
+int nfp_flower_repr_create(struct nfp_app_fw_flower *app_fw_flower,
+ struct nfp_net_hw_priv *hw_priv);
bool nfp_flower_repr_is_vf(struct nfp_flower_representor *repr);
#endif /* __NFP_FLOWER_REPRESENTOR_H__ */
diff --git a/drivers/net/nfp/flower/nfp_flower_service.c b/drivers/net/nfp/flower/nfp_flower_service.c
index 5e8811fe24..515b4abef0 100644
--- a/drivers/net/nfp/flower/nfp_flower_service.c
+++ b/drivers/net/nfp/flower/nfp_flower_service.c
@@ -28,9 +28,9 @@ struct nfp_flower_service {
};
static struct nfp_flower_service *
-nfp_flower_service_handle_get(struct nfp_app_fw_flower *app)
+nfp_flower_service_handle_get(struct nfp_net_hw_priv *hw_priv)
{
- return app->pf_hw->pf_dev->process_share.fl_service;
+ return hw_priv->pf_dev->process_share.fl_service;
}
static int
@@ -100,13 +100,13 @@ nfp_flower_service_insert(struct nfp_app_fw_flower *app,
}
int
-nfp_flower_service_start(void *app_fw_flower)
+nfp_flower_service_start(void *app_fw_flower,
+ struct nfp_net_hw_priv *hw_priv)
{
int ret;
struct nfp_flower_service *service_handle;
- struct nfp_app_fw_flower *app = app_fw_flower;
- service_handle = nfp_flower_service_handle_get(app);
+ service_handle = nfp_flower_service_handle_get(hw_priv);
if (service_handle == NULL) {
PMD_DRV_LOG(ERR, "Can not get service handle");
return -EINVAL;
@@ -122,7 +122,7 @@ nfp_flower_service_start(void *app_fw_flower)
}
/* Insert the NIC to flower service slot */
- ret = nfp_flower_service_insert(app, service_handle);
+ ret = nfp_flower_service_insert(app_fw_flower, service_handle);
if (ret == MAX_FLOWER_SERVICE_SLOT) {
PMD_DRV_LOG(ERR, "Flower ctrl vnic service slot over %u",
MAX_FLOWER_SERVICE_SLOT);
@@ -133,14 +133,14 @@ nfp_flower_service_start(void *app_fw_flower)
}
void
-nfp_flower_service_stop(void *app_fw_flower)
+nfp_flower_service_stop(void *app_fw_flower,
+ struct nfp_net_hw_priv *hw_priv)
{
uint16_t slot;
uint16_t count;
struct nfp_flower_service *service_handle;
- struct nfp_app_fw_flower *app = app_fw_flower;
- service_handle = nfp_flower_service_handle_get(app);
+ service_handle = nfp_flower_service_handle_get(hw_priv);
if (service_handle == NULL) {
PMD_DRV_LOG(ERR, "Can not get service handle");
return;
@@ -149,7 +149,7 @@ nfp_flower_service_stop(void *app_fw_flower)
rte_spinlock_lock(&service_handle->spinlock);
for (slot = 0; slot < MAX_FLOWER_SERVICE_SLOT; slot++) {
/* The app only in one slot */
- if (service_handle->slots[slot] != app)
+ if (service_handle->slots[slot] != app_fw_flower)
continue;
service_handle->slots[slot] = NULL;
@@ -157,7 +157,7 @@ nfp_flower_service_stop(void *app_fw_flower)
rte_spinlock_unlock(&service_handle->spinlock);
/* Determine whether to disable service */
- count = nfp_sync_handle_count_get(app->pf_hw->pf_dev->sync, NULL,
+ count = nfp_sync_handle_count_get(hw_priv->pf_dev->sync, NULL,
service_handle);
if (count > 1)
return;
@@ -167,11 +167,10 @@ nfp_flower_service_stop(void *app_fw_flower)
}
int
-nfp_flower_service_sync_alloc(void *app_fw_flower)
+nfp_flower_service_sync_alloc(struct nfp_net_hw_priv *hw_priv)
{
struct nfp_flower_service *service_handle;
- struct nfp_app_fw_flower *app = app_fw_flower;
- struct nfp_pf_dev *pf_dev = app->pf_hw->pf_dev;
+ struct nfp_pf_dev *pf_dev = hw_priv->pf_dev;
service_handle = nfp_sync_handle_alloc(pf_dev->sync, NULL,
NFP_SYNC_MAGIC_FL_SERVICE,
@@ -185,10 +184,9 @@ nfp_flower_service_sync_alloc(void *app_fw_flower)
}
void
-nfp_flower_service_sync_free(void *app_fw_flower)
+nfp_flower_service_sync_free(struct nfp_net_hw_priv *hw_priv)
{
- struct nfp_app_fw_flower *app = app_fw_flower;
- struct nfp_pf_dev *pf_dev = app->pf_hw->pf_dev;
+ struct nfp_pf_dev *pf_dev = hw_priv->pf_dev;
nfp_sync_handle_free(pf_dev->sync, NULL, pf_dev->process_share.fl_service);
diff --git a/drivers/net/nfp/flower/nfp_flower_service.h b/drivers/net/nfp/flower/nfp_flower_service.h
index a75780274f..2fbe4fa601 100644
--- a/drivers/net/nfp/flower/nfp_flower_service.h
+++ b/drivers/net/nfp/flower/nfp_flower_service.h
@@ -6,12 +6,12 @@
#ifndef __NFP_FLOWER_SERVICE_H__
#define __NFP_FLOWER_SERVICE_H__
-struct nfp_flower_service;
+#include "../nfp_net_common.h"
-int nfp_flower_service_start(void *app_fw_flower);
-void nfp_flower_service_stop(void *app_fw_flower);
+int nfp_flower_service_start(void *app_fw_flower, struct nfp_net_hw_priv *hw_priv);
+void nfp_flower_service_stop(void *app_fw_flower, struct nfp_net_hw_priv *hw_priv);
-int nfp_flower_service_sync_alloc(void *app_fw_flower);
-void nfp_flower_service_sync_free(void *app_fw_flower);
+int nfp_flower_service_sync_alloc(struct nfp_net_hw_priv *hw_priv);
+void nfp_flower_service_sync_free(struct nfp_net_hw_priv *hw_priv);
#endif /* __NFP_FLOWER_SERVICE_H__ */
diff --git a/drivers/net/nfp/nfd3/nfp_nfd3_dp.c b/drivers/net/nfp/nfd3/nfp_nfd3_dp.c
index 253872f4a1..681a63eacd 100644
--- a/drivers/net/nfp/nfd3/nfp_nfd3_dp.c
+++ b/drivers/net/nfp/nfd3/nfp_nfd3_dp.c
@@ -469,6 +469,7 @@ nfp_net_nfd3_tx_queue_setup(struct rte_eth_dev *dev,
nfp_net_reset_tx_queue(txq);
txq->hw = hw;
+ txq->hw_priv = dev->process_private;
/*
* Telling the HW about the physical address of the TX ring and number
diff --git a/drivers/net/nfp/nfdk/nfp_nfdk_dp.c b/drivers/net/nfp/nfdk/nfp_nfdk_dp.c
index 41cdfd3a40..83eaec9c24 100644
--- a/drivers/net/nfp/nfdk/nfp_nfdk_dp.c
+++ b/drivers/net/nfp/nfdk/nfp_nfdk_dp.c
@@ -555,6 +555,7 @@ nfp_net_nfdk_tx_queue_setup(struct rte_eth_dev *dev,
dev->data->tx_queues[queue_idx] = txq;
txq->hw = hw;
+ txq->hw_priv = dev->process_private;
/*
* Telling the HW about the physical address of the TX ring and number
diff --git a/drivers/net/nfp/nfp_ethdev.c b/drivers/net/nfp/nfp_ethdev.c
index 98d8e87028..58e155e92b 100644
--- a/drivers/net/nfp/nfp_ethdev.c
+++ b/drivers/net/nfp/nfp_ethdev.c
@@ -103,12 +103,14 @@ nfp_net_pf_read_mac(struct nfp_app_fw_nic *app_fw_nic,
uint16_t port)
{
struct nfp_net_hw *hw;
+ struct nfp_net_hw_priv *hw_priv;
struct nfp_eth_table *nfp_eth_table;
/* Grab a pointer to the correct physical port */
hw = app_fw_nic->ports[port];
+ hw_priv = hw->eth_dev->process_private;
- nfp_eth_table = app_fw_nic->pf_dev->nfp_eth_table;
+ nfp_eth_table = hw_priv->pf_dev->nfp_eth_table;
rte_ether_addr_copy(&nfp_eth_table->ports[port].mac_addr, &hw->super.mac_addr);
}
@@ -183,8 +185,7 @@ nfp_net_nfp4000_speed_configure_check(uint16_t port_id,
}
static int
-nfp_net_speed_configure(struct rte_eth_dev *dev,
- struct nfp_net_hw *net_hw)
+nfp_net_speed_configure(struct rte_eth_dev *dev)
{
int ret;
uint32_t speed_capa;
@@ -193,11 +194,13 @@ nfp_net_speed_configure(struct rte_eth_dev *dev,
uint32_t configure_speed;
struct nfp_eth_table_port *eth_port;
struct nfp_eth_table *nfp_eth_table;
+ struct nfp_net_hw *net_hw = dev->data->dev_private;
+ struct nfp_net_hw_priv *hw_priv = dev->process_private;
- nfp_eth_table = net_hw->pf_dev->nfp_eth_table;
+ nfp_eth_table = hw_priv->pf_dev->nfp_eth_table;
eth_port = &nfp_eth_table->ports[net_hw->idx];
- speed_capa = net_hw->pf_dev->speed_capa;
+ speed_capa = hw_priv->pf_dev->speed_capa;
if (speed_capa == 0) {
PMD_DRV_LOG(ERR, "Speed_capa is invalid.");
return -EINVAL;
@@ -272,12 +275,14 @@ nfp_net_start(struct rte_eth_dev *dev)
struct nfp_net_hw *net_hw;
struct nfp_pf_dev *pf_dev;
struct rte_eth_rxmode *rxmode;
+ struct nfp_net_hw_priv *hw_priv;
struct nfp_app_fw_nic *app_fw_nic;
struct rte_pci_device *pci_dev = RTE_ETH_DEV_TO_PCI(dev);
struct rte_intr_handle *intr_handle = pci_dev->intr_handle;
net_hw = dev->data->dev_private;
- pf_dev = net_hw->pf_dev;
+ hw_priv = dev->process_private;
+ pf_dev = hw_priv->pf_dev;
app_fw_nic = NFP_PRIV_TO_APP_FW_NIC(pf_dev->app_fw_priv);
hw = &net_hw->super;
@@ -288,7 +293,7 @@ nfp_net_start(struct rte_eth_dev *dev)
nfp_net_enable_queues(dev);
/* Configure the port speed and the auto-negotiation mode. */
- ret = nfp_net_speed_configure(dev, net_hw);
+ ret = nfp_net_speed_configure(dev);
if (ret < 0) {
PMD_DRV_LOG(ERR, "Failed to set the speed and auto-negotiation mode.");
return ret;
@@ -400,7 +405,7 @@ nfp_net_start(struct rte_eth_dev *dev)
if (rte_eal_process_type() == RTE_PROC_PRIMARY)
cpp = net_hw->cpp;
else
- cpp = ((struct nfp_pf_dev *)(dev->process_private))->cpp;
+ cpp = pf_dev->cpp;
/* Configure the physical port up */
nfp_eth_set_configured(cpp, net_hw->nfp_idx, 1);
@@ -437,13 +442,15 @@ nfp_net_set_link_up(struct rte_eth_dev *dev)
{
struct nfp_cpp *cpp;
struct nfp_net_hw *hw;
+ struct nfp_net_hw_priv *hw_priv;
hw = dev->data->dev_private;
+ hw_priv = dev->process_private;
if (rte_eal_process_type() == RTE_PROC_PRIMARY)
cpp = hw->cpp;
else
- cpp = ((struct nfp_pf_dev *)(dev->process_private))->cpp;
+ cpp = hw_priv->pf_dev->cpp;
return nfp_eth_set_configured(cpp, hw->nfp_idx, 1);
}
@@ -454,13 +461,15 @@ nfp_net_set_link_down(struct rte_eth_dev *dev)
{
struct nfp_cpp *cpp;
struct nfp_net_hw *hw;
+ struct nfp_net_hw_priv *hw_priv;
hw = dev->data->dev_private;
+ hw_priv = dev->process_private;
if (rte_eal_process_type() == RTE_PROC_PRIMARY)
cpp = hw->cpp;
else
- cpp = ((struct nfp_pf_dev *)(dev->process_private))->cpp;
+ cpp = hw_priv->pf_dev->cpp;
return nfp_eth_set_configured(cpp, hw->nfp_idx, 0);
}
@@ -573,11 +582,13 @@ static void
nfp_net_uninit(struct rte_eth_dev *eth_dev)
{
struct nfp_net_hw *net_hw;
+ struct nfp_net_hw_priv *hw_priv;
net_hw = eth_dev->data->dev_private;
+ hw_priv = eth_dev->process_private;
if ((net_hw->super.cap_ext & NFP_NET_CFG_CTRL_FLOW_STEER) != 0)
- nfp_net_flow_priv_uninit(net_hw->pf_dev, net_hw->idx);
+ nfp_net_flow_priv_uninit(hw_priv->pf_dev, net_hw->idx);
rte_free(net_hw->eth_xstats_base);
if ((net_hw->super.cap & NFP_NET_CFG_CTRL_TXRWB) != 0)
@@ -612,8 +623,10 @@ nfp_uninit_app_fw_nic(struct nfp_pf_dev *pf_dev)
}
void
-nfp_pf_uninit(struct nfp_pf_dev *pf_dev)
+nfp_pf_uninit(struct nfp_net_hw_priv *hw_priv)
{
+ struct nfp_pf_dev *pf_dev = hw_priv->pf_dev;
+
nfp_cpp_area_release_free(pf_dev->qc_area);
free(pf_dev->sym_tbl);
if (pf_dev->multi_pf.enabled) {
@@ -626,15 +639,19 @@ nfp_pf_uninit(struct nfp_pf_dev *pf_dev)
nfp_cpp_free(pf_dev->cpp);
nfp_sync_free(pf_dev->sync);
rte_free(pf_dev);
+ rte_free(hw_priv);
}
static int
-nfp_pf_secondary_uninit(struct nfp_pf_dev *pf_dev)
+nfp_pf_secondary_uninit(struct nfp_net_hw_priv *hw_priv)
{
+ struct nfp_pf_dev *pf_dev = hw_priv->pf_dev;
+
free(pf_dev->sym_tbl);
nfp_cpp_free(pf_dev->cpp);
nfp_sync_free(pf_dev->sync);
rte_free(pf_dev);
+ rte_free(hw_priv);
return 0;
}
@@ -648,8 +665,11 @@ nfp_net_close(struct rte_eth_dev *dev)
struct nfp_net_hw *hw;
struct nfp_pf_dev *pf_dev;
struct rte_pci_device *pci_dev;
+ struct nfp_net_hw_priv *hw_priv;
struct nfp_app_fw_nic *app_fw_nic;
+ hw_priv = dev->process_private;
+
/*
* In secondary process, a released eth device can be found by its name
* in shared memory.
@@ -660,12 +680,12 @@ nfp_net_close(struct rte_eth_dev *dev)
if (dev->state == RTE_ETH_DEV_UNUSED)
return 0;
- nfp_pf_secondary_uninit(dev->process_private);
+ nfp_pf_secondary_uninit(hw_priv);
return 0;
}
hw = dev->data->dev_private;
- pf_dev = hw->pf_dev;
+ pf_dev = hw_priv->pf_dev;
pci_dev = RTE_ETH_DEV_TO_PCI(dev);
app_fw_nic = NFP_PRIV_TO_APP_FW_NIC(pf_dev->app_fw_priv);
@@ -707,7 +727,7 @@ nfp_net_close(struct rte_eth_dev *dev)
nfp_net_dev_interrupt_handler, (void *)dev);
nfp_uninit_app_fw_nic(pf_dev);
- nfp_pf_uninit(pf_dev);
+ nfp_pf_uninit(hw_priv);
return 0;
}
@@ -887,13 +907,15 @@ nfp_net_init(struct rte_eth_dev *eth_dev)
struct nfp_net_hw *net_hw;
struct nfp_pf_dev *pf_dev;
struct rte_pci_device *pci_dev;
+ struct nfp_net_hw_priv *hw_priv;
struct nfp_app_fw_nic *app_fw_nic;
pci_dev = RTE_ETH_DEV_TO_PCI(eth_dev);
net_hw = eth_dev->data->dev_private;
/* Use backpointer here to the PF of this eth_dev */
- pf_dev = net_hw->pf_dev;
+ hw_priv = eth_dev->process_private;
+ pf_dev = hw_priv->pf_dev;
/* Use backpointer to the CoreNIC app struct */
app_fw_nic = NFP_PRIV_TO_APP_FW_NIC(pf_dev->app_fw_priv);
@@ -915,8 +937,8 @@ nfp_net_init(struct rte_eth_dev *eth_dev)
uint32_t min_size;
hw->ctrl_bar = pf_dev->ctrl_bar;
- min_size = NFP_MAC_STATS_SIZE * net_hw->pf_dev->nfp_eth_table->max_index;
- net_hw->mac_stats_bar = nfp_rtsym_map(net_hw->pf_dev->sym_tbl, "_mac_stats",
+ min_size = NFP_MAC_STATS_SIZE * pf_dev->nfp_eth_table->max_index;
+ net_hw->mac_stats_bar = nfp_rtsym_map(pf_dev->sym_tbl, "_mac_stats",
min_size, &net_hw->mac_stats_area);
if (net_hw->mac_stats_bar == NULL) {
PMD_INIT_LOG(ERR, "nfp_rtsym_map fails for _mac_stats_bar");
@@ -1472,7 +1494,7 @@ nfp_enable_multi_pf(struct nfp_pf_dev *pf_dev)
}
static int
-nfp_init_app_fw_nic(struct nfp_pf_dev *pf_dev,
+nfp_init_app_fw_nic(struct nfp_net_hw_priv *hw_priv,
const struct nfp_dev_info *dev_info)
{
uint8_t i;
@@ -1487,6 +1509,7 @@ nfp_init_app_fw_nic(struct nfp_pf_dev *pf_dev,
char bar_name[RTE_ETH_NAME_MAX_LEN];
char port_name[RTE_ETH_NAME_MAX_LEN];
char vnic_name[RTE_ETH_NAME_MAX_LEN];
+ struct nfp_pf_dev *pf_dev = hw_priv->pf_dev;
nfp_eth_table = pf_dev->nfp_eth_table;
PMD_INIT_LOG(INFO, "Total physical ports: %d", nfp_eth_table->count);
@@ -1529,7 +1552,6 @@ nfp_init_app_fw_nic(struct nfp_pf_dev *pf_dev,
/* Populate coreNIC app properties */
app_fw_nic->total_phyports = total_vnics;
- app_fw_nic->pf_dev = pf_dev;
if (total_vnics > 1)
app_fw_nic->multiport = true;
@@ -1580,13 +1602,13 @@ nfp_init_app_fw_nic(struct nfp_pf_dev *pf_dev,
app_fw_nic->ports[id] = hw;
hw->dev_info = dev_info;
- hw->pf_dev = pf_dev;
hw->cpp = pf_dev->cpp;
hw->eth_dev = eth_dev;
hw->idx = id;
hw->nfp_idx = nfp_eth_table->ports[id].index;
eth_dev->device = &pf_dev->pci_dev->device;
+ eth_dev->process_private = hw_priv;
/*
* Ctrl/tx/rx BAR mappings and remaining init happens in
@@ -1792,6 +1814,7 @@ nfp_pf_init(struct rte_pci_device *pci_dev)
enum nfp_app_fw_id app_fw_id;
char name[RTE_ETH_NAME_MAX_LEN];
struct nfp_rtsym_table *sym_tbl;
+ struct nfp_net_hw_priv *hw_priv;
char app_name[RTE_ETH_NAME_MAX_LEN];
struct nfp_eth_table *nfp_eth_table;
const struct nfp_dev_info *dev_info;
@@ -1810,13 +1833,20 @@ nfp_pf_init(struct rte_pci_device *pci_dev)
return -ENODEV;
}
+ hw_priv = rte_zmalloc(NULL, sizeof(*hw_priv), 0);
+ if (hw_priv == NULL) {
+ PMD_INIT_LOG(ERR, "Can not alloc memory for hw priv data");
+ return -ENOMEM;
+ }
+
/* Allocate memory for the PF "device" */
function_id = (pci_dev->addr.function) & 0x07;
snprintf(name, sizeof(name), "nfp_pf%u", function_id);
pf_dev = rte_zmalloc(name, sizeof(*pf_dev), 0);
if (pf_dev == NULL) {
PMD_INIT_LOG(ERR, "Can't allocate memory for the PF device");
- return -ENOMEM;
+ ret = -ENOMEM;
+ goto hw_priv_free;
}
sync = nfp_sync_alloc();
@@ -1937,6 +1967,8 @@ nfp_pf_init(struct rte_pci_device *pci_dev)
PMD_INIT_LOG(DEBUG, "qc_bar address: %p", pf_dev->qc_bar);
+ hw_priv->pf_dev = pf_dev;
+
/*
* PF initialization has been done at this point. Call app specific
* init code now.
@@ -1950,7 +1982,7 @@ nfp_pf_init(struct rte_pci_device *pci_dev)
}
PMD_INIT_LOG(INFO, "Initializing coreNIC");
- ret = nfp_init_app_fw_nic(pf_dev, dev_info);
+ ret = nfp_init_app_fw_nic(hw_priv, dev_info);
if (ret != 0) {
PMD_INIT_LOG(ERR, "Could not initialize coreNIC!");
goto hwqueues_cleanup;
@@ -1958,7 +1990,7 @@ nfp_pf_init(struct rte_pci_device *pci_dev)
break;
case NFP_APP_FW_FLOWER_NIC:
PMD_INIT_LOG(INFO, "Initializing Flower");
- ret = nfp_init_app_fw_flower(pf_dev, dev_info);
+ ret = nfp_init_app_fw_flower(hw_priv, dev_info);
if (ret != 0) {
PMD_INIT_LOG(ERR, "Could not initialize Flower!");
goto hwqueues_cleanup;
@@ -1996,12 +2028,14 @@ nfp_pf_init(struct rte_pci_device *pci_dev)
nfp_sync_free(sync);
pf_cleanup:
rte_free(pf_dev);
+hw_priv_free:
+ rte_free(hw_priv);
return ret;
}
static int
-nfp_secondary_init_app_fw_nic(struct nfp_pf_dev *pf_dev)
+nfp_secondary_init_app_fw_nic(struct nfp_net_hw_priv *hw_priv)
{
uint32_t i;
int err = 0;
@@ -2010,6 +2044,7 @@ nfp_secondary_init_app_fw_nic(struct nfp_pf_dev *pf_dev)
uint32_t total_vnics;
struct nfp_net_hw *hw;
char pf_name[RTE_ETH_NAME_MAX_LEN];
+ struct nfp_pf_dev *pf_dev = hw_priv->pf_dev;
/* Read the number of vNIC's created for the PF */
function_id = (pf_dev->pci_dev->addr.function) & 0x07;
@@ -2039,7 +2074,7 @@ nfp_secondary_init_app_fw_nic(struct nfp_pf_dev *pf_dev)
break;
}
- eth_dev->process_private = pf_dev;
+ eth_dev->process_private = hw_priv;
hw = eth_dev->data->dev_private;
nfp_net_ethdev_ops_mount(hw, eth_dev);
@@ -2065,6 +2100,7 @@ nfp_pf_secondary_init(struct rte_pci_device *pci_dev)
enum nfp_app_fw_id app_fw_id;
char name[RTE_ETH_NAME_MAX_LEN];
struct nfp_rtsym_table *sym_tbl;
+ struct nfp_net_hw_priv *hw_priv;
const struct nfp_dev_info *dev_info;
char app_name[RTE_ETH_NAME_MAX_LEN];
@@ -2082,12 +2118,19 @@ nfp_pf_secondary_init(struct rte_pci_device *pci_dev)
return -ENODEV;
}
+ hw_priv = rte_zmalloc(NULL, sizeof(*hw_priv), 0);
+ if (hw_priv == NULL) {
+ PMD_INIT_LOG(ERR, "Can not alloc memory for hw priv data");
+ return -ENOMEM;
+ }
+
/* Allocate memory for the PF "device" */
snprintf(name, sizeof(name), "nfp_pf%d", 0);
pf_dev = rte_zmalloc(name, sizeof(*pf_dev), 0);
if (pf_dev == NULL) {
PMD_INIT_LOG(ERR, "Can't allocate memory for the PF device");
- return -ENOMEM;
+ ret = -ENOMEM;
+ goto hw_priv_free;
}
sync = nfp_sync_alloc();
@@ -2143,11 +2186,13 @@ nfp_pf_secondary_init(struct rte_pci_device *pci_dev)
pf_dev->pci_dev = pci_dev;
pf_dev->sync = sync;
+ hw_priv->pf_dev = pf_dev;
+
/* Call app specific init code now */
switch (app_fw_id) {
case NFP_APP_FW_CORE_NIC:
PMD_INIT_LOG(INFO, "Initializing coreNIC");
- ret = nfp_secondary_init_app_fw_nic(pf_dev);
+ ret = nfp_secondary_init_app_fw_nic(hw_priv);
if (ret != 0) {
PMD_INIT_LOG(ERR, "Could not initialize coreNIC!");
goto sym_tbl_cleanup;
@@ -2155,7 +2200,7 @@ nfp_pf_secondary_init(struct rte_pci_device *pci_dev)
break;
case NFP_APP_FW_FLOWER_NIC:
PMD_INIT_LOG(INFO, "Initializing Flower");
- ret = nfp_secondary_init_app_fw_flower(pf_dev);
+ ret = nfp_secondary_init_app_fw_flower(hw_priv);
if (ret != 0) {
PMD_INIT_LOG(ERR, "Could not initialize Flower!");
goto sym_tbl_cleanup;
@@ -2177,6 +2222,8 @@ nfp_pf_secondary_init(struct rte_pci_device *pci_dev)
nfp_sync_free(sync);
pf_cleanup:
rte_free(pf_dev);
+hw_priv_free:
+ rte_free(hw_priv);
return ret;
}
diff --git a/drivers/net/nfp/nfp_net_common.c b/drivers/net/nfp/nfp_net_common.c
index da1a7e7be1..60d77b6178 100644
--- a/drivers/net/nfp/nfp_net_common.c
+++ b/drivers/net/nfp/nfp_net_common.c
@@ -685,6 +685,7 @@ nfp_net_allmulticast_disable(struct rte_eth_dev *dev)
static int
nfp_net_speed_aneg_update(struct rte_eth_dev *dev,
struct nfp_net_hw *hw,
+ struct nfp_net_hw_priv *hw_priv,
struct rte_eth_link *link)
{
uint32_t i;
@@ -700,11 +701,11 @@ nfp_net_speed_aneg_update(struct rte_eth_dev *dev,
return -EIO;
}
- hw->pf_dev->nfp_eth_table->ports[hw->idx] = nfp_eth_table->ports[hw->idx];
+ hw_priv->pf_dev->nfp_eth_table->ports[hw->idx] = nfp_eth_table->ports[hw->idx];
free(nfp_eth_table);
}
- nfp_eth_table = hw->pf_dev->nfp_eth_table;
+ nfp_eth_table = hw_priv->pf_dev->nfp_eth_table;
eth_port = &nfp_eth_table->ports[hw->idx];
speed = eth_port->speed;
@@ -730,10 +731,12 @@ nfp_net_link_update_common(struct rte_eth_dev *dev,
{
int ret;
uint32_t nn_link_status;
+ struct nfp_net_hw_priv *hw_priv;
+ hw_priv = dev->process_private;
if (link->link_status == RTE_ETH_LINK_UP) {
- if (hw->pf_dev != NULL) {
- ret = nfp_net_speed_aneg_update(dev, hw, link);
+ if (hw_priv->pf_dev != NULL) {
+ ret = nfp_net_speed_aneg_update(dev, hw, hw_priv, link);
if (ret != 0) {
PMD_DRV_LOG(DEBUG, "Failed to update speed and aneg.");
return ret;
@@ -775,8 +778,10 @@ nfp_net_link_update(struct rte_eth_dev *dev,
struct nfp_net_hw *hw;
uint32_t nn_link_status;
struct rte_eth_link link;
+ struct nfp_net_hw_priv *hw_priv;
hw = nfp_net_get_hw(dev);
+ hw_priv = dev->process_private;
memset(&link, 0, sizeof(struct rte_eth_link));
@@ -796,7 +801,7 @@ nfp_net_link_update(struct rte_eth_dev *dev,
* Not applicable for VFs as the associated PF is still attached to the
* kernel driver.
*/
- if (hw->pf_dev != NULL)
+ if (hw_priv != NULL && hw_priv->pf_dev != NULL)
nfp_net_notify_port_speed(hw, &link);
return ret;
@@ -1173,8 +1178,10 @@ nfp_net_infos_get(struct rte_eth_dev *dev, struct rte_eth_dev_info *dev_info)
uint16_t min_tx_desc;
uint16_t max_tx_desc;
struct nfp_net_hw *hw;
+ struct nfp_net_hw_priv *hw_priv;
hw = nfp_net_get_hw(dev);
+ hw_priv = dev->process_private;
nfp_net_rx_desc_limits(hw, &min_rx_desc, &max_rx_desc);
nfp_net_tx_desc_limits(hw, &min_tx_desc, &max_tx_desc);
@@ -1274,8 +1281,8 @@ nfp_net_infos_get(struct rte_eth_dev *dev, struct rte_eth_dev_info *dev_info)
}
/* Only PF supports getting speed capability. */
- if (hw->pf_dev != NULL)
- dev_info->speed_capa = hw->pf_dev->speed_capa;
+ if (hw_priv != NULL && hw_priv->pf_dev != NULL)
+ dev_info->speed_capa = hw_priv->pf_dev->speed_capa;
return 0;
}
@@ -2135,10 +2142,10 @@ nfp_net_get_mip_name(struct nfp_net_hw *hw,
}
static void
-nfp_net_get_app_name(struct nfp_net_hw *hw,
+nfp_net_get_app_name(struct nfp_net_hw_priv *hw_priv,
char *app_name)
{
- switch (hw->pf_dev->app_fw_id) {
+ switch (hw_priv->pf_dev->app_fw_id) {
case NFP_APP_FW_CORE_NIC:
snprintf(app_name, FW_VER_LEN, "%s", "nic");
break;
@@ -2161,11 +2168,13 @@ nfp_net_firmware_version_get(struct rte_eth_dev *dev,
char app_name[FW_VER_LEN];
char nsp_version[FW_VER_LEN];
char vnic_version[FW_VER_LEN];
+ struct nfp_net_hw_priv *hw_priv;
if (fw_size < FW_VER_LEN)
return FW_VER_LEN;
hw = nfp_net_get_hw(dev);
+ hw_priv = dev->process_private;
if (!rte_eth_dev_is_repr(dev)) {
snprintf(vnic_version, FW_VER_LEN, "%d.%d.%d.%d",
@@ -2177,7 +2186,7 @@ nfp_net_firmware_version_get(struct rte_eth_dev *dev,
nfp_net_get_nsp_info(hw, nsp_version);
nfp_net_get_mip_name(hw, mip_name);
- nfp_net_get_app_name(hw, app_name);
+ nfp_net_get_app_name(hw_priv, app_name);
snprintf(fw_version, FW_VER_LEN, "%s %s %s %s",
vnic_version, nsp_version, mip_name, app_name);
@@ -2212,8 +2221,10 @@ nfp_net_stop(struct rte_eth_dev *dev)
{
struct nfp_cpp *cpp;
struct nfp_net_hw *hw;
+ struct nfp_net_hw_priv *hw_priv;
hw = nfp_net_get_hw(dev);
+ hw_priv = dev->process_private;
nfp_net_disable_queues(dev);
@@ -2224,7 +2235,7 @@ nfp_net_stop(struct rte_eth_dev *dev)
if (rte_eal_process_type() == RTE_PROC_PRIMARY)
cpp = hw->cpp;
else
- cpp = ((struct nfp_pf_dev *)(dev->process_private))->cpp;
+ cpp = hw_priv->pf_dev->cpp;
nfp_eth_set_configured(cpp, hw->nfp_idx, 0);
@@ -2255,15 +2266,15 @@ int
nfp_net_flow_ctrl_get(struct rte_eth_dev *dev,
struct rte_eth_fc_conf *fc_conf)
{
- struct nfp_net_hw *net_hw;
+ struct nfp_net_hw_priv *hw_priv;
struct nfp_eth_table *nfp_eth_table;
struct nfp_eth_table_port *eth_port;
- net_hw = nfp_net_get_hw(dev);
- if (net_hw->pf_dev == NULL)
+ hw_priv = dev->process_private;
+ if (hw_priv == NULL || hw_priv->pf_dev == NULL)
return -EINVAL;
- nfp_eth_table = net_hw->pf_dev->nfp_eth_table;
+ nfp_eth_table = hw_priv->pf_dev->nfp_eth_table;
eth_port = &nfp_eth_table->ports[dev->data->port_id];
/* Currently only RX/TX switch are supported */
@@ -2319,15 +2330,17 @@ nfp_net_flow_ctrl_set(struct rte_eth_dev *dev,
int ret;
struct nfp_net_hw *net_hw;
enum rte_eth_fc_mode set_mode;
+ struct nfp_net_hw_priv *hw_priv;
enum rte_eth_fc_mode original_mode;
struct nfp_eth_table *nfp_eth_table;
struct nfp_eth_table_port *eth_port;
net_hw = nfp_net_get_hw(dev);
- if (net_hw->pf_dev == NULL)
+ hw_priv = dev->process_private;
+ if (hw_priv == NULL || hw_priv->pf_dev == NULL)
return -EINVAL;
- nfp_eth_table = net_hw->pf_dev->nfp_eth_table;
+ nfp_eth_table = hw_priv->pf_dev->nfp_eth_table;
eth_port = &nfp_eth_table->ports[net_hw->idx];
original_mode = nfp_net_get_pause_mode(eth_port);
@@ -2355,14 +2368,16 @@ nfp_net_fec_get_capability(struct rte_eth_dev *dev,
uint16_t speed;
struct nfp_net_hw *hw;
uint32_t supported_fec;
+ struct nfp_net_hw_priv *hw_priv;
struct nfp_eth_table *nfp_eth_table;
struct nfp_eth_table_port *eth_port;
hw = nfp_net_get_hw(dev);
- if (hw->pf_dev == NULL)
+ hw_priv = dev->process_private;
+ if (hw_priv == NULL || hw_priv->pf_dev == NULL)
return -EINVAL;
- nfp_eth_table = hw->pf_dev->nfp_eth_table;
+ nfp_eth_table = hw_priv->pf_dev->nfp_eth_table;
eth_port = &nfp_eth_table->ports[hw->idx];
speed = eth_port->speed;
@@ -2412,20 +2427,22 @@ nfp_net_fec_get(struct rte_eth_dev *dev,
uint32_t *fec_capa)
{
struct nfp_net_hw *hw;
+ struct nfp_net_hw_priv *hw_priv;
struct nfp_eth_table *nfp_eth_table;
struct nfp_eth_table_port *eth_port;
hw = nfp_net_get_hw(dev);
- if (hw->pf_dev == NULL)
+ hw_priv = dev->process_private;
+ if (hw_priv == NULL || hw_priv->pf_dev == NULL)
return -EINVAL;
if (dev->data->dev_link.link_status == RTE_ETH_LINK_DOWN) {
nfp_eth_table = nfp_eth_read_ports(hw->cpp);
- hw->pf_dev->nfp_eth_table->ports[hw->idx] = nfp_eth_table->ports[hw->idx];
+ hw_priv->pf_dev->nfp_eth_table->ports[hw->idx] = nfp_eth_table->ports[hw->idx];
free(nfp_eth_table);
}
- nfp_eth_table = hw->pf_dev->nfp_eth_table;
+ nfp_eth_table = hw_priv->pf_dev->nfp_eth_table;
eth_port = &nfp_eth_table->ports[hw->idx];
if (!nfp_eth_can_support_fec(eth_port)) {
@@ -2473,14 +2490,16 @@ nfp_net_fec_set(struct rte_eth_dev *dev,
enum nfp_eth_fec fec;
struct nfp_net_hw *hw;
uint32_t supported_fec;
+ struct nfp_net_hw_priv *hw_priv;
struct nfp_eth_table *nfp_eth_table;
struct nfp_eth_table_port *eth_port;
hw = nfp_net_get_hw(dev);
- if (hw->pf_dev == NULL)
+ hw_priv = dev->process_private;
+ if (hw_priv == NULL || hw_priv->pf_dev == NULL)
return -EINVAL;
- nfp_eth_table = hw->pf_dev->nfp_eth_table;
+ nfp_eth_table = hw_priv->pf_dev->nfp_eth_table;
eth_port = &nfp_eth_table->ports[hw->idx];
supported_fec = nfp_eth_supported_fec_modes(eth_port);
diff --git a/drivers/net/nfp/nfp_net_common.h b/drivers/net/nfp/nfp_net_common.h
index 8066e77e6f..8dbe2fb973 100644
--- a/drivers/net/nfp/nfp_net_common.h
+++ b/drivers/net/nfp/nfp_net_common.h
@@ -12,7 +12,6 @@
#include <nfp_dev.h>
#include <rte_spinlock.h>
-#include "flower/nfp_flower_service.h"
#include "nfpcore/nfp_sync.h"
#include "nfp_net_ctrl.h"
#include "nfp_service.h"
@@ -86,6 +85,8 @@ struct nfp_multi_pf {
uint8_t *beat_addr;
};
+struct nfp_flower_service;
+
struct nfp_process_share {
struct nfp_flower_service *fl_service;
};
@@ -145,9 +146,6 @@ struct nfp_net_priv {
};
struct nfp_app_fw_nic {
- /** Backpointer to the PF device */
- struct nfp_pf_dev *pf_dev;
-
/**
* Array of physical ports belonging to this CoreNIC app.
* This is really a list of vNIC's, one for each physical port.
@@ -158,13 +156,14 @@ struct nfp_app_fw_nic {
uint8_t total_phyports;
};
+struct nfp_net_hw_priv {
+ struct nfp_pf_dev *pf_dev;
+};
+
struct nfp_net_hw {
/** The parent class */
struct nfp_hw super;
- /** Backpointer to the PF this port belongs to */
- struct nfp_pf_dev *pf_dev;
-
/** Backpointer to the eth_dev of this port */
struct rte_eth_dev *eth_dev;
@@ -314,7 +313,7 @@ int nfp_net_flow_ctrl_get(struct rte_eth_dev *dev,
struct rte_eth_fc_conf *fc_conf);
int nfp_net_flow_ctrl_set(struct rte_eth_dev *dev,
struct rte_eth_fc_conf *fc_conf);
-void nfp_pf_uninit(struct nfp_pf_dev *pf_dev);
+void nfp_pf_uninit(struct nfp_net_hw_priv *hw_priv);
int nfp_net_fec_get_capability(struct rte_eth_dev *dev,
struct rte_eth_fec_capa *speed_fec_capa,
unsigned int num);
diff --git a/drivers/net/nfp/nfp_net_flow.c b/drivers/net/nfp/nfp_net_flow.c
index ea743eed73..b0d1a57d99 100644
--- a/drivers/net/nfp/nfp_net_flow.c
+++ b/drivers/net/nfp/nfp_net_flow.c
@@ -658,10 +658,12 @@ nfp_net_flow_setup(struct rte_eth_dev *dev,
struct nfp_net_priv *priv;
struct rte_flow *nfp_flow;
struct rte_flow *flow_find;
+ struct nfp_net_hw_priv *hw_priv;
struct nfp_app_fw_nic *app_fw_nic;
hw = dev->data->dev_private;
- app_fw_nic = NFP_PRIV_TO_APP_FW_NIC(hw->pf_dev->app_fw_priv);
+ hw_priv = dev->process_private;
+ app_fw_nic = NFP_PRIV_TO_APP_FW_NIC(hw_priv->pf_dev->app_fw_priv);
priv = app_fw_nic->ports[hw->idx]->priv;
ret = nfp_net_flow_calculate_items(items, &match_len);
@@ -775,10 +777,12 @@ nfp_net_flow_validate(struct rte_eth_dev *dev,
struct nfp_net_hw *hw;
struct rte_flow *nfp_flow;
struct nfp_net_priv *priv;
+ struct nfp_net_hw_priv *hw_priv;
struct nfp_app_fw_nic *app_fw_nic;
hw = dev->data->dev_private;
- app_fw_nic = NFP_PRIV_TO_APP_FW_NIC(hw->pf_dev->app_fw_priv);
+ hw_priv = dev->process_private;
+ app_fw_nic = NFP_PRIV_TO_APP_FW_NIC(hw_priv->pf_dev->app_fw_priv);
priv = app_fw_nic->ports[hw->idx]->priv;
nfp_flow = nfp_net_flow_setup(dev, attr, items, actions);
@@ -811,10 +815,12 @@ nfp_net_flow_create(struct rte_eth_dev *dev,
struct nfp_net_hw *hw;
struct rte_flow *nfp_flow;
struct nfp_net_priv *priv;
+ struct nfp_net_hw_priv *hw_priv;
struct nfp_app_fw_nic *app_fw_nic;
hw = dev->data->dev_private;
- app_fw_nic = NFP_PRIV_TO_APP_FW_NIC(hw->pf_dev->app_fw_priv);
+ hw_priv = dev->process_private;
+ app_fw_nic = NFP_PRIV_TO_APP_FW_NIC(hw_priv->pf_dev->app_fw_priv);
priv = app_fw_nic->ports[hw->idx]->priv;
nfp_flow = nfp_net_flow_setup(dev, attr, items, actions);
@@ -861,10 +867,12 @@ nfp_net_flow_destroy(struct rte_eth_dev *dev,
struct nfp_net_hw *hw;
struct nfp_net_priv *priv;
struct rte_flow *flow_find;
+ struct nfp_net_hw_priv *hw_priv;
struct nfp_app_fw_nic *app_fw_nic;
hw = dev->data->dev_private;
- app_fw_nic = NFP_PRIV_TO_APP_FW_NIC(hw->pf_dev->app_fw_priv);
+ hw_priv = dev->process_private;
+ app_fw_nic = NFP_PRIV_TO_APP_FW_NIC(hw_priv->pf_dev->app_fw_priv);
priv = app_fw_nic->ports[hw->idx]->priv;
/* Find the flow in flow hash table */
@@ -920,10 +928,12 @@ nfp_net_flow_flush(struct rte_eth_dev *dev,
struct nfp_net_hw *hw;
struct rte_flow *nfp_flow;
struct rte_hash *flow_table;
+ struct nfp_net_hw_priv *hw_priv;
struct nfp_app_fw_nic *app_fw_nic;
hw = dev->data->dev_private;
- app_fw_nic = NFP_PRIV_TO_APP_FW_NIC(hw->pf_dev->app_fw_priv);
+ hw_priv = dev->process_private;
+ app_fw_nic = NFP_PRIV_TO_APP_FW_NIC(hw_priv->pf_dev->app_fw_priv);
flow_table = app_fw_nic->ports[hw->idx]->priv->flow_table;
while (rte_hash_iterate(flow_table, &next_key, &next_data, &iter) >= 0) {
diff --git a/drivers/net/nfp/nfp_rxtx.c b/drivers/net/nfp/nfp_rxtx.c
index f9c4636688..12f362a4f4 100644
--- a/drivers/net/nfp/nfp_rxtx.c
+++ b/drivers/net/nfp/nfp_rxtx.c
@@ -418,6 +418,7 @@ nfp_net_recv_pkts(void *rx_queue,
struct nfp_net_dp_buf *rxb;
struct nfp_net_rx_desc *rxds;
uint16_t avail_multiplexed = 0;
+ struct nfp_net_hw_priv *hw_priv;
rxq = rx_queue;
if (unlikely(rxq == NULL)) {
@@ -430,6 +431,7 @@ nfp_net_recv_pkts(void *rx_queue,
}
hw = rxq->hw;
+ hw_priv = rxq->hw_priv;
while (avail + avail_multiplexed < nb_pkts) {
rxb = &rxq->rxbufs[rxq->rd_p];
@@ -520,7 +522,7 @@ nfp_net_recv_pkts(void *rx_queue,
if (((meta.flags >> NFP_NET_META_PORTID) & 0x1) == 0) {
rx_pkts[avail++] = mb;
- } else if (nfp_flower_pf_dispatch_pkts(hw, mb, meta.port_id)) {
+ } else if (nfp_flower_pf_dispatch_pkts(hw_priv, mb, meta.port_id)) {
avail_multiplexed++;
} else {
rte_pktmbuf_free(mb);
@@ -684,6 +686,7 @@ nfp_net_rx_queue_setup(struct rte_eth_dev *dev,
nfp_net_reset_rx_queue(rxq);
rxq->hw = hw;
+ rxq->hw_priv = dev->process_private;
/*
* Telling the HW about the physical address of the RX ring and number
diff --git a/drivers/net/nfp/nfp_rxtx.h b/drivers/net/nfp/nfp_rxtx.h
index 0aa73a6432..8cd3649882 100644
--- a/drivers/net/nfp/nfp_rxtx.h
+++ b/drivers/net/nfp/nfp_rxtx.h
@@ -30,6 +30,7 @@ struct nfp_tx_ipsec_desc_msg {
struct nfp_net_txq {
/** Backpointer to nfp_net structure */
struct nfp_net_hw *hw;
+ struct nfp_net_hw_priv *hw_priv;
/**
* For each descriptor keep a reference to the mbuf and
@@ -69,8 +70,8 @@ struct nfp_net_txq {
uint16_t data_pending;
/**
- * At this point 50 bytes have been used for all the fields in the
- * TX critical path. We have room for 14 bytes and still all placed
+ * At this point 58 bytes have been used for all the fields in the
+ * TX critical path. We have room for 6 bytes and still all placed
* in a cache line.
*/
uint64_t dma;
@@ -138,6 +139,7 @@ struct nfp_net_rx_desc {
struct nfp_net_rxq {
/** Backpointer to nfp_net structure */
struct nfp_net_hw *hw;
+ struct nfp_net_hw_priv *hw_priv;
/**
* Point to the base addresses of the freelist queue
@@ -188,8 +190,8 @@ struct nfp_net_rxq {
uint16_t qidx;
/**
- * At this point 54 bytes have been used for all the fields in the
- * RX critical path. We have room for 10 bytes and still all placed
+ * At this point 62 bytes have been used for all the fields in the
+ * RX critical path. We have room for 2 bytes and still all placed
* in a cache line.
*/
--
2.39.1
^ permalink raw reply [flat|nested] 24+ messages in thread* [PATCH v2 5/8] net/nfp: move device info data field
2024-04-19 5:23 ` [PATCH 0/8] refactor logic to support secondary process Chaoyong He
` (3 preceding siblings ...)
2024-04-19 5:23 ` [PATCH v2 4/8] net/nfp: add the process private structure Chaoyong He
@ 2024-04-19 5:23 ` Chaoyong He
2024-04-19 5:23 ` [PATCH v2 6/8] net/nfp: unify CPP acquire method Chaoyong He
` (4 subsequent siblings)
9 siblings, 0 replies; 24+ messages in thread
From: Chaoyong He @ 2024-04-19 5:23 UTC (permalink / raw)
To: dev; +Cc: oss-drivers, Chaoyong He, Long Wu, Peng Zhang
Move 'dev_info' data field from process shared data structure into
process private data structure.
Signed-off-by: Chaoyong He <chaoyong.he@corigine.com>
Reviewed-by: Long Wu <long.wu@corigine.com>
Reviewed-by: Peng Zhang <peng.zhang@corigine.com>
---
drivers/net/nfp/flower/nfp_flower.c | 9 +++------
drivers/net/nfp/flower/nfp_flower.h | 3 +--
drivers/net/nfp/nfd3/nfp_nfd3_dp.c | 4 +++-
drivers/net/nfp/nfdk/nfp_nfdk_dp.c | 4 +++-
drivers/net/nfp/nfp_ethdev.c | 10 +++++-----
drivers/net/nfp/nfp_ethdev_vf.c | 18 +++++++++++++++++-
drivers/net/nfp/nfp_net_common.c | 15 ++++++++-------
drivers/net/nfp/nfp_net_common.h | 9 +++++----
drivers/net/nfp/nfp_rxtx.c | 4 +++-
9 files changed, 48 insertions(+), 28 deletions(-)
diff --git a/drivers/net/nfp/flower/nfp_flower.c b/drivers/net/nfp/flower/nfp_flower.c
index 8dd9e23c78..80874e23de 100644
--- a/drivers/net/nfp/flower/nfp_flower.c
+++ b/drivers/net/nfp/flower/nfp_flower.c
@@ -372,7 +372,7 @@ nfp_flower_init_ctrl_vnic(struct nfp_app_fw_flower *app_fw_flower,
*/
tz = rte_eth_dma_zone_reserve(eth_dev, ctrl_rxring_name, i,
sizeof(struct nfp_net_rx_desc) *
- hw->dev_info->max_qc_size,
+ hw_priv->dev_info->max_qc_size,
NFP_MEMZONE_ALIGN, numa_node);
if (tz == NULL) {
PMD_DRV_LOG(ERR, "Error allocating rx dma");
@@ -430,7 +430,7 @@ nfp_flower_init_ctrl_vnic(struct nfp_app_fw_flower *app_fw_flower,
*/
tz = rte_eth_dma_zone_reserve(eth_dev, ctrl_txring_name, i,
sizeof(struct nfp_net_nfd3_tx_desc) *
- hw->dev_info->max_qc_size,
+ hw_priv->dev_info->max_qc_size,
NFP_MEMZONE_ALIGN, numa_node);
if (tz == NULL) {
PMD_DRV_LOG(ERR, "Error allocating tx dma");
@@ -639,8 +639,7 @@ nfp_flower_nfd_func_register(struct nfp_app_fw_flower *app_fw_flower)
}
int
-nfp_init_app_fw_flower(struct nfp_net_hw_priv *hw_priv,
- const struct nfp_dev_info *dev_info)
+nfp_init_app_fw_flower(struct nfp_net_hw_priv *hw_priv)
{
int ret;
int err;
@@ -709,7 +708,6 @@ nfp_init_app_fw_flower(struct nfp_net_hw_priv *hw_priv,
app_fw_flower->pf_hw = pf_hw;
pf_hw->super.ctrl_bar = pf_dev->ctrl_bar;
pf_hw->cpp = pf_dev->cpp;
- pf_hw->dev_info = dev_info;
ret = nfp_flower_init_vnic_common(hw_priv, pf_hw, "pf_vnic");
if (ret != 0) {
@@ -734,7 +732,6 @@ nfp_init_app_fw_flower(struct nfp_net_hw_priv *hw_priv,
/* Now populate the ctrl vNIC */
ctrl_hw->cpp = pf_dev->cpp;
- ctrl_hw->dev_info = dev_info;
ret = nfp_flower_init_ctrl_vnic(app_fw_flower, hw_priv);
if (ret != 0) {
diff --git a/drivers/net/nfp/flower/nfp_flower.h b/drivers/net/nfp/flower/nfp_flower.h
index 5cc29dcc87..5c2f338a20 100644
--- a/drivers/net/nfp/flower/nfp_flower.h
+++ b/drivers/net/nfp/flower/nfp_flower.h
@@ -104,8 +104,7 @@ nfp_flower_support_decap_v2(const struct nfp_app_fw_flower *app_fw_flower)
return app_fw_flower->ext_features & NFP_FL_FEATS_DECAP_V2;
}
-int nfp_init_app_fw_flower(struct nfp_net_hw_priv *hw_priv,
- const struct nfp_dev_info *dev_info);
+int nfp_init_app_fw_flower(struct nfp_net_hw_priv *hw_priv);
void nfp_uninit_app_fw_flower(struct nfp_net_hw_priv *hw_priv);
int nfp_secondary_init_app_fw_flower(struct nfp_net_hw_priv *hw_priv);
bool nfp_flower_pf_dispatch_pkts(struct nfp_net_hw_priv *hw_priv,
diff --git a/drivers/net/nfp/nfd3/nfp_nfd3_dp.c b/drivers/net/nfp/nfd3/nfp_nfd3_dp.c
index 681a63eacd..ee96cd8e46 100644
--- a/drivers/net/nfp/nfd3/nfp_nfd3_dp.c
+++ b/drivers/net/nfp/nfd3/nfp_nfd3_dp.c
@@ -385,10 +385,12 @@ nfp_net_nfd3_tx_queue_setup(struct rte_eth_dev *dev,
struct nfp_net_txq *txq;
uint16_t tx_free_thresh;
const struct rte_memzone *tz;
+ struct nfp_net_hw_priv *hw_priv;
hw = nfp_net_get_hw(dev);
+ hw_priv = dev->process_private;
- nfp_net_tx_desc_limits(hw, &min_tx_desc, &max_tx_desc);
+ nfp_net_tx_desc_limits(hw, hw_priv, &min_tx_desc, &max_tx_desc);
/* Validating number of descriptors */
tx_desc_sz = nb_desc * sizeof(struct nfp_net_nfd3_tx_desc);
diff --git a/drivers/net/nfp/nfdk/nfp_nfdk_dp.c b/drivers/net/nfp/nfdk/nfp_nfdk_dp.c
index 83eaec9c24..3ba97e23a9 100644
--- a/drivers/net/nfp/nfdk/nfp_nfdk_dp.c
+++ b/drivers/net/nfp/nfdk/nfp_nfdk_dp.c
@@ -464,10 +464,12 @@ nfp_net_nfdk_tx_queue_setup(struct rte_eth_dev *dev,
uint16_t tx_free_thresh;
struct nfp_net_txq *txq;
const struct rte_memzone *tz;
+ struct nfp_net_hw_priv *hw_priv;
hw = nfp_net_get_hw(dev);
+ hw_priv = dev->process_private;
- nfp_net_tx_desc_limits(hw, &min_tx_desc, &max_tx_desc);
+ nfp_net_tx_desc_limits(hw, hw_priv, &min_tx_desc, &max_tx_desc);
/* Validating number of descriptors */
tx_desc_sz = nb_desc * sizeof(struct nfp_net_nfdk_tx_desc);
diff --git a/drivers/net/nfp/nfp_ethdev.c b/drivers/net/nfp/nfp_ethdev.c
index 58e155e92b..38fe153ab4 100644
--- a/drivers/net/nfp/nfp_ethdev.c
+++ b/drivers/net/nfp/nfp_ethdev.c
@@ -1494,8 +1494,7 @@ nfp_enable_multi_pf(struct nfp_pf_dev *pf_dev)
}
static int
-nfp_init_app_fw_nic(struct nfp_net_hw_priv *hw_priv,
- const struct nfp_dev_info *dev_info)
+nfp_init_app_fw_nic(struct nfp_net_hw_priv *hw_priv)
{
uint8_t i;
uint8_t id;
@@ -1601,7 +1600,6 @@ nfp_init_app_fw_nic(struct nfp_net_hw_priv *hw_priv,
/* Add this device to the PF's array of physical ports */
app_fw_nic->ports[id] = hw;
- hw->dev_info = dev_info;
hw->cpp = pf_dev->cpp;
hw->eth_dev = eth_dev;
hw->idx = id;
@@ -1968,6 +1966,7 @@ nfp_pf_init(struct rte_pci_device *pci_dev)
PMD_INIT_LOG(DEBUG, "qc_bar address: %p", pf_dev->qc_bar);
hw_priv->pf_dev = pf_dev;
+ hw_priv->dev_info = dev_info;
/*
* PF initialization has been done at this point. Call app specific
@@ -1982,7 +1981,7 @@ nfp_pf_init(struct rte_pci_device *pci_dev)
}
PMD_INIT_LOG(INFO, "Initializing coreNIC");
- ret = nfp_init_app_fw_nic(hw_priv, dev_info);
+ ret = nfp_init_app_fw_nic(hw_priv);
if (ret != 0) {
PMD_INIT_LOG(ERR, "Could not initialize coreNIC!");
goto hwqueues_cleanup;
@@ -1990,7 +1989,7 @@ nfp_pf_init(struct rte_pci_device *pci_dev)
break;
case NFP_APP_FW_FLOWER_NIC:
PMD_INIT_LOG(INFO, "Initializing Flower");
- ret = nfp_init_app_fw_flower(hw_priv, dev_info);
+ ret = nfp_init_app_fw_flower(hw_priv);
if (ret != 0) {
PMD_INIT_LOG(ERR, "Could not initialize Flower!");
goto hwqueues_cleanup;
@@ -2187,6 +2186,7 @@ nfp_pf_secondary_init(struct rte_pci_device *pci_dev)
pf_dev->sync = sync;
hw_priv->pf_dev = pf_dev;
+ hw_priv->dev_info = dev_info;
/* Call app specific init code now */
switch (app_fw_id) {
diff --git a/drivers/net/nfp/nfp_ethdev_vf.c b/drivers/net/nfp/nfp_ethdev_vf.c
index 88da593190..bd3745a5c5 100644
--- a/drivers/net/nfp/nfp_ethdev_vf.c
+++ b/drivers/net/nfp/nfp_ethdev_vf.c
@@ -162,14 +162,17 @@ nfp_netvf_close(struct rte_eth_dev *dev)
{
struct nfp_net_hw *net_hw;
struct rte_pci_device *pci_dev;
+ struct nfp_net_hw_priv *hw_priv;
if (rte_eal_process_type() != RTE_PROC_PRIMARY)
return 0;
net_hw = dev->data->dev_private;
pci_dev = RTE_ETH_DEV_TO_PCI(dev);
+ hw_priv = dev->process_private;
rte_free(net_hw->eth_xstats_base);
+ rte_free(hw_priv);
/*
* We assume that the DPDK application is stopping all the
@@ -255,6 +258,7 @@ nfp_netvf_init(struct rte_eth_dev *eth_dev)
uint64_t tx_bar_off = 0;
uint64_t rx_bar_off = 0;
struct rte_pci_device *pci_dev;
+ struct nfp_net_hw_priv *hw_priv;
const struct nfp_dev_info *dev_info;
port = eth_dev->data->port_id;
@@ -267,7 +271,6 @@ nfp_netvf_init(struct rte_eth_dev *eth_dev)
}
net_hw = eth_dev->data->dev_private;
- net_hw->dev_info = dev_info;
hw = &net_hw->super;
hw->ctrl_bar = pci_dev->mem_resource[0].addr;
@@ -284,6 +287,17 @@ nfp_netvf_init(struct rte_eth_dev *eth_dev)
nfp_netvf_ethdev_ops_mount(net_hw, eth_dev);
+ hw_priv = rte_zmalloc(NULL, sizeof(*hw_priv), 0);
+ if (hw_priv == NULL) {
+ PMD_INIT_LOG(ERR, "Can not alloc memory for hw priv data");
+ err = -ENOMEM;
+ goto hw_priv_free;
+ }
+
+ hw_priv->dev_info = dev_info;
+
+ eth_dev->process_private = hw_priv;
+
/* For secondary processes, the primary has done all the work */
if (rte_eal_process_type() != RTE_PROC_PRIMARY)
return 0;
@@ -366,6 +380,8 @@ nfp_netvf_init(struct rte_eth_dev *eth_dev)
free_xstats:
rte_free(net_hw->eth_xstats_base);
+hw_priv_free:
+ rte_free(hw_priv);
return err;
}
diff --git a/drivers/net/nfp/nfp_net_common.c b/drivers/net/nfp/nfp_net_common.c
index 60d77b6178..8cc8b77318 100644
--- a/drivers/net/nfp/nfp_net_common.c
+++ b/drivers/net/nfp/nfp_net_common.c
@@ -1144,16 +1144,17 @@ nfp_net_xstats_reset(struct rte_eth_dev *dev)
}
void
-nfp_net_rx_desc_limits(struct nfp_net_hw *hw,
+nfp_net_rx_desc_limits(struct nfp_net_hw_priv *hw_priv,
uint16_t *min_rx_desc,
uint16_t *max_rx_desc)
{
- *max_rx_desc = hw->dev_info->max_qc_size;
- *min_rx_desc = hw->dev_info->min_qc_size;
+ *max_rx_desc = hw_priv->dev_info->max_qc_size;
+ *min_rx_desc = hw_priv->dev_info->min_qc_size;
}
void
nfp_net_tx_desc_limits(struct nfp_net_hw *hw,
+ struct nfp_net_hw_priv *hw_priv,
uint16_t *min_tx_desc,
uint16_t *max_tx_desc)
{
@@ -1164,8 +1165,8 @@ nfp_net_tx_desc_limits(struct nfp_net_hw *hw,
else
tx_dpp = NFDK_TX_DESC_PER_SIMPLE_PKT;
- *max_tx_desc = hw->dev_info->max_qc_size / tx_dpp;
- *min_tx_desc = hw->dev_info->min_qc_size / tx_dpp;
+ *max_tx_desc = hw_priv->dev_info->max_qc_size / tx_dpp;
+ *min_tx_desc = hw_priv->dev_info->min_qc_size / tx_dpp;
}
int
@@ -1183,8 +1184,8 @@ nfp_net_infos_get(struct rte_eth_dev *dev, struct rte_eth_dev_info *dev_info)
hw = nfp_net_get_hw(dev);
hw_priv = dev->process_private;
- nfp_net_rx_desc_limits(hw, &min_rx_desc, &max_rx_desc);
- nfp_net_tx_desc_limits(hw, &min_tx_desc, &max_tx_desc);
+ nfp_net_rx_desc_limits(hw_priv, &min_rx_desc, &max_rx_desc);
+ nfp_net_tx_desc_limits(hw, hw_priv, &min_tx_desc, &max_tx_desc);
dev_info->max_rx_queues = (uint16_t)hw->max_rx_queues;
dev_info->max_tx_queues = (uint16_t)hw->max_tx_queues;
diff --git a/drivers/net/nfp/nfp_net_common.h b/drivers/net/nfp/nfp_net_common.h
index 8dbe2fb973..31deeb43ce 100644
--- a/drivers/net/nfp/nfp_net_common.h
+++ b/drivers/net/nfp/nfp_net_common.h
@@ -158,6 +158,9 @@ struct nfp_app_fw_nic {
struct nfp_net_hw_priv {
struct nfp_pf_dev *pf_dev;
+
+ /** NFP ASIC params */
+ const struct nfp_dev_info *dev_info;
};
struct nfp_net_hw {
@@ -177,9 +180,6 @@ struct nfp_net_hw {
uint32_t rx_offset;
enum nfp_net_meta_format meta_format;
- /** NFP ASIC params */
- const struct nfp_dev_info *dev_info;
-
uint8_t *tx_bar;
uint8_t *rx_bar;
@@ -297,10 +297,11 @@ void nfp_net_close_rx_queue(struct rte_eth_dev *dev);
void nfp_net_stop_tx_queue(struct rte_eth_dev *dev);
void nfp_net_close_tx_queue(struct rte_eth_dev *dev);
int nfp_net_set_vxlan_port(struct nfp_net_hw *hw, size_t idx, uint16_t port);
-void nfp_net_rx_desc_limits(struct nfp_net_hw *hw,
+void nfp_net_rx_desc_limits(struct nfp_net_hw_priv *hw_priv,
uint16_t *min_rx_desc,
uint16_t *max_rx_desc);
void nfp_net_tx_desc_limits(struct nfp_net_hw *hw,
+ struct nfp_net_hw_priv *hw_priv,
uint16_t *min_tx_desc,
uint16_t *max_tx_desc);
int nfp_net_check_dma_mask(struct nfp_net_hw *hw, char *name);
diff --git a/drivers/net/nfp/nfp_rxtx.c b/drivers/net/nfp/nfp_rxtx.c
index 12f362a4f4..8697c5d711 100644
--- a/drivers/net/nfp/nfp_rxtx.c
+++ b/drivers/net/nfp/nfp_rxtx.c
@@ -606,10 +606,12 @@ nfp_net_rx_queue_setup(struct rte_eth_dev *dev,
struct nfp_net_hw *hw;
struct nfp_net_rxq *rxq;
const struct rte_memzone *tz;
+ struct nfp_net_hw_priv *hw_priv;
hw = nfp_net_get_hw(dev);
+ hw_priv = dev->process_private;
- nfp_net_rx_desc_limits(hw, &min_rx_desc, &max_rx_desc);
+ nfp_net_rx_desc_limits(hw_priv, &min_rx_desc, &max_rx_desc);
/* Validating number of descriptors */
rx_desc_sz = nb_desc * sizeof(struct nfp_net_rx_desc);
--
2.39.1
^ permalink raw reply [flat|nested] 24+ messages in thread* [PATCH v2 6/8] net/nfp: unify CPP acquire method
2024-04-19 5:23 ` [PATCH 0/8] refactor logic to support secondary process Chaoyong He
` (4 preceding siblings ...)
2024-04-19 5:23 ` [PATCH v2 5/8] net/nfp: move device info data field Chaoyong He
@ 2024-04-19 5:23 ` Chaoyong He
2024-04-19 5:23 ` [PATCH v2 7/8] net/nfp: remove ethernet device data field Chaoyong He
` (3 subsequent siblings)
9 siblings, 0 replies; 24+ messages in thread
From: Chaoyong He @ 2024-04-19 5:23 UTC (permalink / raw)
To: dev; +Cc: oss-drivers, Chaoyong He, Long Wu, Peng Zhang
Getting CPP from 'struct nfp_net_hw' is not right in multiple process
user case. Modify the data structure and related logic to get it from
process private data structure.
Signed-off-by: Chaoyong He <chaoyong.he@corigine.com>
Signed-off-by: Long Wu <long.wu@corigine.com>
Reviewed-by: Peng Zhang <peng.zhang@corigine.com>
---
drivers/net/nfp/flower/nfp_flower.c | 4 ---
.../net/nfp/flower/nfp_flower_representor.c | 2 +-
drivers/net/nfp/nfp_ethdev.c | 31 +++-------------
drivers/net/nfp/nfp_net_common.c | 36 ++++++++-----------
drivers/net/nfp/nfp_net_common.h | 3 +-
5 files changed, 22 insertions(+), 54 deletions(-)
diff --git a/drivers/net/nfp/flower/nfp_flower.c b/drivers/net/nfp/flower/nfp_flower.c
index 80874e23de..83249feedb 100644
--- a/drivers/net/nfp/flower/nfp_flower.c
+++ b/drivers/net/nfp/flower/nfp_flower.c
@@ -707,7 +707,6 @@ nfp_init_app_fw_flower(struct nfp_net_hw_priv *hw_priv)
/* Fill in the PF vNIC and populate app struct */
app_fw_flower->pf_hw = pf_hw;
pf_hw->super.ctrl_bar = pf_dev->ctrl_bar;
- pf_hw->cpp = pf_dev->cpp;
ret = nfp_flower_init_vnic_common(hw_priv, pf_hw, "pf_vnic");
if (ret != 0) {
@@ -730,9 +729,6 @@ nfp_init_app_fw_flower(struct nfp_net_hw_priv *hw_priv)
goto pf_cpp_area_cleanup;
}
- /* Now populate the ctrl vNIC */
- ctrl_hw->cpp = pf_dev->cpp;
-
ret = nfp_flower_init_ctrl_vnic(app_fw_flower, hw_priv);
if (ret != 0) {
PMD_INIT_LOG(ERR, "Could not initialize flower ctrl vNIC");
diff --git a/drivers/net/nfp/flower/nfp_flower_representor.c b/drivers/net/nfp/flower/nfp_flower_representor.c
index 934f078dca..0e19723f57 100644
--- a/drivers/net/nfp/flower/nfp_flower_representor.c
+++ b/drivers/net/nfp/flower/nfp_flower_representor.c
@@ -826,7 +826,7 @@ nfp_flower_repr_alloc(struct nfp_app_fw_flower *app_fw_flower,
*/
for (i = 0; i < app_fw_flower->num_vf_reprs; i++) {
flower_repr.repr_type = NFP_REPR_TYPE_VF;
- flower_repr.port_id = nfp_get_pcie_port_id(app_fw_flower->pf_hw->cpp,
+ flower_repr.port_id = nfp_get_pcie_port_id(hw_priv->pf_dev->cpp,
NFP_FLOWER_CMSG_PORT_VNIC_TYPE_VF, i, 0);
flower_repr.nfp_idx = 0;
flower_repr.vf_id = i;
diff --git a/drivers/net/nfp/nfp_ethdev.c b/drivers/net/nfp/nfp_ethdev.c
index 38fe153ab4..258b617eb2 100644
--- a/drivers/net/nfp/nfp_ethdev.c
+++ b/drivers/net/nfp/nfp_ethdev.c
@@ -224,7 +224,7 @@ nfp_net_speed_configure(struct rte_eth_dev *dev)
}
}
- nsp = nfp_eth_config_start(net_hw->cpp, eth_port->index);
+ nsp = nfp_eth_config_start(hw_priv->pf_dev->cpp, eth_port->index);
if (nsp == NULL) {
PMD_DRV_LOG(ERR, "Couldn't get NSP.");
return -EIO;
@@ -267,7 +267,6 @@ nfp_net_start(struct rte_eth_dev *dev)
uint16_t i;
struct nfp_hw *hw;
uint32_t new_ctrl;
- struct nfp_cpp *cpp;
uint32_t update = 0;
uint32_t cap_extend;
uint32_t intr_vector;
@@ -402,13 +401,8 @@ nfp_net_start(struct rte_eth_dev *dev)
goto error;
}
- if (rte_eal_process_type() == RTE_PROC_PRIMARY)
- cpp = net_hw->cpp;
- else
- cpp = pf_dev->cpp;
-
/* Configure the physical port up */
- nfp_eth_set_configured(cpp, net_hw->nfp_idx, 1);
+ nfp_eth_set_configured(pf_dev->cpp, net_hw->nfp_idx, 1);
for (i = 0; i < dev->data->nb_rx_queues; i++)
dev->data->rx_queue_state[i] = RTE_ETH_QUEUE_STATE_STARTED;
@@ -440,38 +434,26 @@ nfp_net_start(struct rte_eth_dev *dev)
static int
nfp_net_set_link_up(struct rte_eth_dev *dev)
{
- struct nfp_cpp *cpp;
struct nfp_net_hw *hw;
struct nfp_net_hw_priv *hw_priv;
hw = dev->data->dev_private;
hw_priv = dev->process_private;
- if (rte_eal_process_type() == RTE_PROC_PRIMARY)
- cpp = hw->cpp;
- else
- cpp = hw_priv->pf_dev->cpp;
-
- return nfp_eth_set_configured(cpp, hw->nfp_idx, 1);
+ return nfp_eth_set_configured(hw_priv->pf_dev->cpp, hw->nfp_idx, 1);
}
/* Set the link down. */
static int
nfp_net_set_link_down(struct rte_eth_dev *dev)
{
- struct nfp_cpp *cpp;
struct nfp_net_hw *hw;
struct nfp_net_hw_priv *hw_priv;
hw = dev->data->dev_private;
hw_priv = dev->process_private;
- if (rte_eal_process_type() == RTE_PROC_PRIMARY)
- cpp = hw->cpp;
- else
- cpp = hw_priv->pf_dev->cpp;
-
- return nfp_eth_set_configured(cpp, hw->nfp_idx, 0);
+ return nfp_eth_set_configured(hw_priv->pf_dev->cpp, hw->nfp_idx, 0);
}
static uint8_t
@@ -1178,7 +1160,6 @@ nfp_fw_check_change(struct nfp_cpp *cpp,
bool *fw_changed)
{
int ret;
- struct nfp_net_hw hw;
uint32_t new_version = 0;
uint32_t old_version = 0;
@@ -1186,8 +1167,7 @@ nfp_fw_check_change(struct nfp_cpp *cpp,
if (ret != 0)
return ret;
- hw.cpp = cpp;
- nfp_net_get_fw_version(&hw, &old_version);
+ nfp_net_get_fw_version(cpp, &old_version);
if (new_version != old_version) {
PMD_DRV_LOG(INFO, "FW version is changed, new %u, old %u",
@@ -1600,7 +1580,6 @@ nfp_init_app_fw_nic(struct nfp_net_hw_priv *hw_priv)
/* Add this device to the PF's array of physical ports */
app_fw_nic->ports[id] = hw;
- hw->cpp = pf_dev->cpp;
hw->eth_dev = eth_dev;
hw->idx = id;
hw->nfp_idx = nfp_eth_table->ports[id].index;
diff --git a/drivers/net/nfp/nfp_net_common.c b/drivers/net/nfp/nfp_net_common.c
index 8cc8b77318..7541afa235 100644
--- a/drivers/net/nfp/nfp_net_common.c
+++ b/drivers/net/nfp/nfp_net_common.c
@@ -695,7 +695,7 @@ nfp_net_speed_aneg_update(struct rte_eth_dev *dev,
/* Compare whether the current status has changed. */
if (dev->data->dev_link.link_status != link->link_status) {
- nfp_eth_table = nfp_eth_read_ports(hw->cpp);
+ nfp_eth_table = nfp_eth_read_ports(hw_priv->pf_dev->cpp);
if (nfp_eth_table == NULL) {
PMD_DRV_LOG(DEBUG, "Error reading NFP ethernet table.");
return -EIO;
@@ -2094,12 +2094,12 @@ nfp_net_cfg_read_version(struct nfp_net_hw *hw)
}
static void
-nfp_net_get_nsp_info(struct nfp_net_hw *hw,
+nfp_net_get_nsp_info(struct nfp_net_hw_priv *hw_priv,
char *nsp_version)
{
struct nfp_nsp *nsp;
- nsp = nfp_nsp_open(hw->cpp);
+ nsp = nfp_nsp_open(hw_priv->pf_dev->cpp);
if (nsp == NULL)
return;
@@ -2111,12 +2111,12 @@ nfp_net_get_nsp_info(struct nfp_net_hw *hw,
}
void
-nfp_net_get_fw_version(struct nfp_net_hw *hw,
+nfp_net_get_fw_version(struct nfp_cpp *cpp,
uint32_t *mip_version)
{
struct nfp_mip *mip;
- mip = nfp_mip_open(hw->cpp);
+ mip = nfp_mip_open(cpp);
if (mip == NULL) {
*mip_version = 0;
return;
@@ -2128,12 +2128,12 @@ nfp_net_get_fw_version(struct nfp_net_hw *hw,
}
static void
-nfp_net_get_mip_name(struct nfp_net_hw *hw,
+nfp_net_get_mip_name(struct nfp_net_hw_priv *hw_priv,
char *mip_name)
{
struct nfp_mip *mip;
- mip = nfp_mip_open(hw->cpp);
+ mip = nfp_mip_open(hw_priv->pf_dev->cpp);
if (mip == NULL)
return;
@@ -2185,8 +2185,8 @@ nfp_net_firmware_version_get(struct rte_eth_dev *dev,
snprintf(vnic_version, FW_VER_LEN, "*");
}
- nfp_net_get_nsp_info(hw, nsp_version);
- nfp_net_get_mip_name(hw, mip_name);
+ nfp_net_get_nsp_info(hw_priv, nsp_version);
+ nfp_net_get_mip_name(hw_priv, mip_name);
nfp_net_get_app_name(hw_priv, app_name);
snprintf(fw_version, FW_VER_LEN, "%s %s %s %s",
@@ -2220,7 +2220,6 @@ nfp_net_is_valid_nfd_version(struct nfp_net_fw_ver version)
int
nfp_net_stop(struct rte_eth_dev *dev)
{
- struct nfp_cpp *cpp;
struct nfp_net_hw *hw;
struct nfp_net_hw_priv *hw_priv;
@@ -2233,12 +2232,7 @@ nfp_net_stop(struct rte_eth_dev *dev)
nfp_net_stop_tx_queue(dev);
nfp_net_stop_rx_queue(dev);
- if (rte_eal_process_type() == RTE_PROC_PRIMARY)
- cpp = hw->cpp;
- else
- cpp = hw_priv->pf_dev->cpp;
-
- nfp_eth_set_configured(cpp, hw->nfp_idx, 0);
+ nfp_eth_set_configured(hw_priv->pf_dev->cpp, hw->nfp_idx, 0);
return 0;
}
@@ -2285,7 +2279,7 @@ nfp_net_flow_ctrl_get(struct rte_eth_dev *dev,
}
static int
-nfp_net_pause_frame_set(struct nfp_net_hw *net_hw,
+nfp_net_pause_frame_set(struct nfp_net_hw_priv *hw_priv,
struct nfp_eth_table_port *eth_port,
enum rte_eth_fc_mode mode)
{
@@ -2293,7 +2287,7 @@ nfp_net_pause_frame_set(struct nfp_net_hw *net_hw,
bool flag;
struct nfp_nsp *nsp;
- nsp = nfp_eth_config_start(net_hw->cpp, eth_port->index);
+ nsp = nfp_eth_config_start(hw_priv->pf_dev->cpp, eth_port->index);
if (nsp == NULL) {
PMD_DRV_LOG(ERR, "NFP error when obtaining NSP handle.");
return -EIO;
@@ -2350,7 +2344,7 @@ nfp_net_flow_ctrl_set(struct rte_eth_dev *dev,
if (set_mode == original_mode)
return 0;
- ret = nfp_net_pause_frame_set(net_hw, eth_port, set_mode);
+ ret = nfp_net_pause_frame_set(hw_priv, eth_port, set_mode);
if (ret != 0)
return ret;
@@ -2438,7 +2432,7 @@ nfp_net_fec_get(struct rte_eth_dev *dev,
return -EINVAL;
if (dev->data->dev_link.link_status == RTE_ETH_LINK_DOWN) {
- nfp_eth_table = nfp_eth_read_ports(hw->cpp);
+ nfp_eth_table = nfp_eth_read_ports(hw_priv->pf_dev->cpp);
hw_priv->pf_dev->nfp_eth_table->ports[hw->idx] = nfp_eth_table->ports[hw->idx];
free(nfp_eth_table);
}
@@ -2520,5 +2514,5 @@ nfp_net_fec_set(struct rte_eth_dev *dev,
return -EIO;
}
- return nfp_eth_set_fec(hw->cpp, eth_port->index, fec);
+ return nfp_eth_set_fec(hw_priv->pf_dev->cpp, eth_port->index, fec);
}
diff --git a/drivers/net/nfp/nfp_net_common.h b/drivers/net/nfp/nfp_net_common.h
index 31deeb43ce..40f314af0a 100644
--- a/drivers/net/nfp/nfp_net_common.h
+++ b/drivers/net/nfp/nfp_net_common.h
@@ -201,7 +201,6 @@ struct nfp_net_hw {
struct rte_eth_stats eth_stats_base;
struct rte_eth_xstat *eth_xstats_base;
- struct nfp_cpp *cpp;
struct nfp_cpp_area *ctrl_area;
struct nfp_cpp_area *mac_stats_area;
uint8_t *mac_stats_bar;
@@ -322,7 +321,7 @@ int nfp_net_fec_get(struct rte_eth_dev *dev,
uint32_t *fec_capa);
int nfp_net_fec_set(struct rte_eth_dev *dev,
uint32_t fec_capa);
-void nfp_net_get_fw_version(struct nfp_net_hw *hw,
+void nfp_net_get_fw_version(struct nfp_cpp *cpp,
uint32_t *fw_version);
int nfp_net_txrwb_alloc(struct rte_eth_dev *eth_dev);
void nfp_net_txrwb_free(struct rte_eth_dev *eth_dev);
--
2.39.1
^ permalink raw reply [flat|nested] 24+ messages in thread* [PATCH v2 7/8] net/nfp: remove ethernet device data field
2024-04-19 5:23 ` [PATCH 0/8] refactor logic to support secondary process Chaoyong He
` (5 preceding siblings ...)
2024-04-19 5:23 ` [PATCH v2 6/8] net/nfp: unify CPP acquire method Chaoyong He
@ 2024-04-19 5:23 ` Chaoyong He
2024-04-19 5:23 ` [PATCH v2 8/8] net/nfp: unify port create and destroy interface Chaoyong He
` (2 subsequent siblings)
9 siblings, 0 replies; 24+ messages in thread
From: Chaoyong He @ 2024-04-19 5:23 UTC (permalink / raw)
To: dev; +Cc: oss-drivers, Chaoyong He, Long Wu, Peng Zhang
Remove the 'eth_dev' field in process shared data 'struct
nfp_net_hw', and modify the related logic.
Signed-off-by: Chaoyong He <chaoyong.he@corigine.com>
Reviewed-by: Long Wu <long.wu@corigine.com>
Reviewed-by: Peng Zhang <peng.zhang@corigine.com>
---
drivers/net/nfp/flower/nfp_flower.c | 16 ++--
drivers/net/nfp/flower/nfp_flower.h | 2 +
drivers/net/nfp/flower/nfp_flower_ctrl.c | 8 +-
drivers/net/nfp/flower/nfp_flower_flow.c | 16 ++--
.../net/nfp/flower/nfp_flower_representor.c | 76 +++++++++----------
.../net/nfp/flower/nfp_flower_representor.h | 1 -
drivers/net/nfp/nfp_ethdev.c | 36 ++++-----
drivers/net/nfp/nfp_net_common.h | 3 -
8 files changed, 78 insertions(+), 80 deletions(-)
diff --git a/drivers/net/nfp/flower/nfp_flower.c b/drivers/net/nfp/flower/nfp_flower.c
index 83249feedb..76e5d674f1 100644
--- a/drivers/net/nfp/flower/nfp_flower.c
+++ b/drivers/net/nfp/flower/nfp_flower.c
@@ -271,15 +271,15 @@ nfp_flower_init_ctrl_vnic(struct nfp_app_fw_flower *app_fw_flower,
}
/* Allocate memory for the eth_dev of the vNIC */
- hw->eth_dev = rte_zmalloc("nfp_ctrl_vnic",
+ app_fw_flower->ctrl_ethdev = rte_zmalloc("nfp_ctrl_vnic",
sizeof(struct rte_eth_dev), RTE_CACHE_LINE_SIZE);
- if (hw->eth_dev == NULL) {
+ if (app_fw_flower->ctrl_ethdev == NULL) {
PMD_INIT_LOG(ERR, "Could not allocate ctrl vnic");
return -ENOMEM;
}
/* Grab the pointer to the newly created rte_eth_dev here */
- eth_dev = hw->eth_dev;
+ eth_dev = app_fw_flower->ctrl_ethdev;
/* Also allocate memory for the data part of the eth_dev */
eth_dev->data = rte_zmalloc("nfp_ctrl_vnic_data",
@@ -529,7 +529,7 @@ nfp_flower_cleanup_ctrl_vnic(struct nfp_app_fw_flower *app_fw_flower,
char ctrl_rxring_name[RTE_MEMZONE_NAMESIZE];
hw = app_fw_flower->ctrl_hw;
- eth_dev = hw->eth_dev;
+ eth_dev = app_fw_flower->ctrl_ethdev;
pci_name = strchr(hw_priv->pf_dev->pci_dev->name, ':') + 1;
@@ -562,15 +562,17 @@ nfp_flower_cleanup_ctrl_vnic(struct nfp_app_fw_flower *app_fw_flower,
}
static int
-nfp_flower_start_ctrl_vnic(struct nfp_net_hw *net_hw)
+nfp_flower_start_ctrl_vnic(struct nfp_app_fw_flower *app_fw_flower)
{
int ret;
uint32_t update;
uint32_t new_ctrl;
struct nfp_hw *hw;
struct rte_eth_dev *dev;
+ struct nfp_net_hw *net_hw;
- dev = net_hw->eth_dev;
+ net_hw = app_fw_flower->ctrl_hw;
+ dev = app_fw_flower->ctrl_ethdev;
hw = &net_hw->super;
/* Disabling queues just in case... */
@@ -736,7 +738,7 @@ nfp_init_app_fw_flower(struct nfp_net_hw_priv *hw_priv)
}
/* Start the ctrl vNIC */
- ret = nfp_flower_start_ctrl_vnic(app_fw_flower->ctrl_hw);
+ ret = nfp_flower_start_ctrl_vnic(app_fw_flower);
if (ret != 0) {
PMD_INIT_LOG(ERR, "Could not start flower ctrl vNIC");
goto ctrl_vnic_cleanup;
diff --git a/drivers/net/nfp/flower/nfp_flower.h b/drivers/net/nfp/flower/nfp_flower.h
index 5c2f338a20..ae80782df4 100644
--- a/drivers/net/nfp/flower/nfp_flower.h
+++ b/drivers/net/nfp/flower/nfp_flower.h
@@ -63,12 +63,14 @@ struct nfp_app_fw_flower {
/** Pointer to the PF vNIC */
struct nfp_net_hw *pf_hw;
+ struct rte_eth_dev *pf_ethdev;
/** Pointer to a mempool for the Ctrl vNIC */
struct rte_mempool *ctrl_pktmbuf_pool;
/** Pointer to the ctrl vNIC */
struct nfp_net_hw *ctrl_hw;
+ struct rte_eth_dev *ctrl_ethdev;
/** Ctrl vNIC Rx counter */
uint64_t ctrl_vnic_rx_count;
diff --git a/drivers/net/nfp/flower/nfp_flower_ctrl.c b/drivers/net/nfp/flower/nfp_flower_ctrl.c
index 4c2aeb9849..de6e419cac 100644
--- a/drivers/net/nfp/flower/nfp_flower_ctrl.c
+++ b/drivers/net/nfp/flower/nfp_flower_ctrl.c
@@ -153,12 +153,10 @@ nfp_flower_ctrl_vnic_nfd3_xmit(struct nfp_app_fw_flower *app_fw_flower,
uint32_t free_descs;
struct rte_mbuf **lmbuf;
struct nfp_net_txq *txq;
- struct nfp_net_hw *ctrl_hw;
struct rte_eth_dev *ctrl_dev;
struct nfp_net_nfd3_tx_desc *txds;
- ctrl_hw = app_fw_flower->ctrl_hw;
- ctrl_dev = ctrl_hw->eth_dev;
+ ctrl_dev = app_fw_flower->ctrl_ethdev;
/* Flower ctrl vNIC only has a single tx queue */
txq = ctrl_dev->data->tx_queues[0];
@@ -232,7 +230,7 @@ nfp_flower_ctrl_vnic_nfdk_xmit(struct nfp_app_fw_flower *app_fw_flower,
struct rte_eth_dev *ctrl_dev;
struct nfp_net_nfdk_tx_desc *ktxds;
- ctrl_dev = app_fw_flower->ctrl_hw->eth_dev;
+ ctrl_dev = app_fw_flower->ctrl_ethdev;
/* Flower ctrl vNIC only has a single tx queue */
txq = ctrl_dev->data->tx_queues[0];
@@ -511,7 +509,7 @@ nfp_flower_ctrl_vnic_process(struct nfp_app_fw_flower *app_fw_flower)
struct rte_eth_dev *ctrl_eth_dev;
struct rte_mbuf *pkts_burst[MAX_PKT_BURST];
- ctrl_eth_dev = app_fw_flower->ctrl_hw->eth_dev;
+ ctrl_eth_dev = app_fw_flower->ctrl_ethdev;
/* Ctrl vNIC only has a single Rx queue */
rxq = ctrl_eth_dev->data->rx_queues[0];
diff --git a/drivers/net/nfp/flower/nfp_flower_flow.c b/drivers/net/nfp/flower/nfp_flower_flow.c
index 086cc8079a..8ff13923e5 100644
--- a/drivers/net/nfp/flower/nfp_flower_flow.c
+++ b/drivers/net/nfp/flower/nfp_flower_flow.c
@@ -3518,7 +3518,7 @@ nfp_flow_action_rss_add(struct nfp_flower_representor *representor,
struct nfp_fl_rss **rss_store)
{
int ret;
- struct nfp_net_hw *pf_hw;
+ struct rte_eth_dev *eth_dev;
struct rte_eth_rss_conf rss_conf;
struct nfp_fl_rss *rss_store_tmp;
const struct rte_flow_action_rss *rss;
@@ -3536,8 +3536,8 @@ nfp_flow_action_rss_add(struct nfp_flower_representor *representor,
rss_conf.rss_hf = 0;
rss_conf.rss_key = rss_key;
- pf_hw = representor->app_fw_flower->pf_hw;
- ret = nfp_net_rss_hash_conf_get(pf_hw->eth_dev, &rss_conf);
+ eth_dev = representor->app_fw_flower->pf_ethdev;
+ ret = nfp_net_rss_hash_conf_get(eth_dev, &rss_conf);
if (ret != 0) {
PMD_DRV_LOG(ERR, "Get RSS conf failed.");
return ret;
@@ -3563,7 +3563,7 @@ nfp_flow_action_rss_add(struct nfp_flower_representor *representor,
rss_store_tmp->key_len = rss->key_len;
}
- ret = nfp_net_rss_hash_update(pf_hw->eth_dev, &rss_conf);
+ ret = nfp_net_rss_hash_update(eth_dev, &rss_conf);
if (ret != 0) {
PMD_DRV_LOG(ERR, "Update RSS conf failed.");
free(rss_store_tmp);
@@ -3580,7 +3580,7 @@ nfp_flow_action_rss_del(struct nfp_flower_representor *representor,
struct rte_flow *nfp_flow)
{
int ret;
- struct nfp_net_hw *pf_hw;
+ struct rte_eth_dev *eth_dev;
struct nfp_fl_rss *rss_store;
struct rte_eth_rss_conf rss_conf;
uint8_t rss_key[NFP_NET_CFG_RSS_KEY_SZ];
@@ -3590,8 +3590,8 @@ nfp_flow_action_rss_del(struct nfp_flower_representor *representor,
rss_conf.rss_hf = 0;
rss_conf.rss_key = rss_key;
- pf_hw = representor->app_fw_flower->pf_hw;
- ret = nfp_net_rss_hash_conf_get(pf_hw->eth_dev, &rss_conf);
+ eth_dev = representor->app_fw_flower->pf_ethdev;
+ ret = nfp_net_rss_hash_conf_get(eth_dev, &rss_conf);
if (ret != 0) {
PMD_DRV_LOG(ERR, "Get RSS conf failed.");
goto exit;
@@ -3612,7 +3612,7 @@ nfp_flow_action_rss_del(struct nfp_flower_representor *representor,
rss_conf.rss_key_len = 0;
}
- ret = nfp_net_rss_hash_update(pf_hw->eth_dev, &rss_conf);
+ ret = nfp_net_rss_hash_update(eth_dev, &rss_conf);
if (ret != 0)
PMD_DRV_LOG(ERR, "Update RSS conf failed.");
diff --git a/drivers/net/nfp/flower/nfp_flower_representor.c b/drivers/net/nfp/flower/nfp_flower_representor.c
index 0e19723f57..60f02ad919 100644
--- a/drivers/net/nfp/flower/nfp_flower_representor.c
+++ b/drivers/net/nfp/flower/nfp_flower_representor.c
@@ -270,7 +270,6 @@ nfp_flower_repr_tx_burst(void *tx_queue,
uint16_t sent;
void *pf_tx_queue;
struct nfp_net_txq *txq;
- struct nfp_net_hw *pf_hw;
struct rte_eth_dev *dev;
struct rte_eth_dev *repr_dev;
struct nfp_flower_representor *repr;
@@ -290,8 +289,7 @@ nfp_flower_repr_tx_burst(void *tx_queue,
tx_pkts[i], repr->port_id);
/* This points to the PF vNIC that owns this representor */
- pf_hw = txq->hw;
- dev = pf_hw->eth_dev;
+ dev = repr->app_fw_flower->pf_ethdev;
/* Only using Tx queue 0 for now. */
pf_tx_queue = dev->data->tx_queues[0];
@@ -306,10 +304,9 @@ nfp_flower_repr_tx_burst(void *tx_queue,
}
static void
-nfp_flower_repr_free_queue(struct nfp_flower_representor *repr)
+nfp_flower_repr_free_queue(struct rte_eth_dev *eth_dev)
{
uint16_t i;
- struct rte_eth_dev *eth_dev = repr->eth_dev;
for (i = 0; i < eth_dev->data->nb_tx_queues; i++)
rte_free(eth_dev->data->tx_queues[i]);
@@ -319,10 +316,8 @@ nfp_flower_repr_free_queue(struct nfp_flower_representor *repr)
}
static void
-nfp_flower_pf_repr_close_queue(struct nfp_flower_representor *repr)
+nfp_flower_pf_repr_close_queue(struct rte_eth_dev *eth_dev)
{
- struct rte_eth_dev *eth_dev = repr->eth_dev;
-
/*
* We assume that the DPDK application is stopping all the
* threads/queues before calling the device close function.
@@ -335,17 +330,18 @@ nfp_flower_pf_repr_close_queue(struct nfp_flower_representor *repr)
}
static void
-nfp_flower_repr_close_queue(struct nfp_flower_representor *repr)
+nfp_flower_repr_close_queue(struct rte_eth_dev *eth_dev,
+ enum nfp_repr_type repr_type)
{
- switch (repr->repr_type) {
+ switch (repr_type) {
case NFP_REPR_TYPE_PHYS_PORT:
- nfp_flower_repr_free_queue(repr);
+ nfp_flower_repr_free_queue(eth_dev);
break;
case NFP_REPR_TYPE_PF:
- nfp_flower_pf_repr_close_queue(repr);
+ nfp_flower_pf_repr_close_queue(eth_dev);
break;
case NFP_REPR_TYPE_VF:
- nfp_flower_repr_free_queue(repr);
+ nfp_flower_repr_free_queue(eth_dev);
break;
default:
PMD_DRV_LOG(ERR, "Unsupported repr port type.");
@@ -384,18 +380,18 @@ nfp_flower_pf_repr_uninit(struct rte_eth_dev *eth_dev)
}
static void
-nfp_flower_repr_free(struct nfp_flower_representor *repr,
+nfp_flower_repr_free(struct rte_eth_dev *eth_dev,
enum nfp_repr_type repr_type)
{
switch (repr_type) {
case NFP_REPR_TYPE_PHYS_PORT:
- nfp_flower_repr_uninit(repr->eth_dev);
+ nfp_flower_repr_uninit(eth_dev);
break;
case NFP_REPR_TYPE_PF:
- nfp_flower_pf_repr_uninit(repr->eth_dev);
+ nfp_flower_pf_repr_uninit(eth_dev);
break;
case NFP_REPR_TYPE_VF:
- nfp_flower_repr_uninit(repr->eth_dev);
+ nfp_flower_repr_uninit(eth_dev);
break;
default:
PMD_DRV_LOG(ERR, "Unsupported repr port type.");
@@ -425,9 +421,8 @@ nfp_flower_repr_dev_close(struct rte_eth_dev *dev)
if (pf_dev->app_fw_id != NFP_APP_FW_FLOWER_NIC)
return -EINVAL;
- nfp_flower_repr_close_queue(repr);
-
- nfp_flower_repr_free(repr, repr->repr_type);
+ nfp_flower_repr_close_queue(dev, repr->repr_type);
+ nfp_flower_repr_free(dev, repr->repr_type);
for (i = 0; i < MAX_FLOWER_VFS; i++) {
if (app_fw_flower->vf_reprs[i] != NULL)
@@ -575,8 +570,7 @@ nfp_flower_pf_repr_init(struct rte_eth_dev *eth_dev,
rte_ether_addr_copy(&init_repr_data->mac_addr, eth_dev->data->mac_addrs);
repr->app_fw_flower->pf_repr = repr;
- repr->app_fw_flower->pf_hw->eth_dev = eth_dev;
- repr->eth_dev = eth_dev;
+ repr->app_fw_flower->pf_ethdev = eth_dev;
return 0;
}
@@ -668,8 +662,6 @@ nfp_flower_repr_init(struct rte_eth_dev *eth_dev,
app_fw_flower->vf_reprs[index] = repr;
}
- repr->eth_dev = eth_dev;
-
return 0;
mac_cleanup:
@@ -684,28 +676,38 @@ static void
nfp_flower_repr_free_all(struct nfp_app_fw_flower *app_fw_flower)
{
uint32_t i;
+ struct rte_eth_dev *eth_dev;
struct nfp_flower_representor *repr;
for (i = 0; i < MAX_FLOWER_VFS; i++) {
repr = app_fw_flower->vf_reprs[i];
if (repr != NULL) {
- nfp_flower_repr_free(repr, NFP_REPR_TYPE_VF);
- app_fw_flower->vf_reprs[i] = NULL;
+ eth_dev = rte_eth_dev_get_by_name(repr->name);
+ if (eth_dev != NULL) {
+ nfp_flower_repr_free(eth_dev, NFP_REPR_TYPE_VF);
+ app_fw_flower->vf_reprs[i] = NULL;
+ }
}
}
for (i = 0; i < NFP_MAX_PHYPORTS; i++) {
repr = app_fw_flower->phy_reprs[i];
if (repr != NULL) {
- nfp_flower_repr_free(repr, NFP_REPR_TYPE_PHYS_PORT);
- app_fw_flower->phy_reprs[i] = NULL;
+ eth_dev = rte_eth_dev_get_by_name(repr->name);
+ if (eth_dev != NULL) {
+ nfp_flower_repr_free(eth_dev, NFP_REPR_TYPE_PHYS_PORT);
+ app_fw_flower->phy_reprs[i] = NULL;
+ }
}
}
repr = app_fw_flower->pf_repr;
if (repr != NULL) {
- nfp_flower_repr_free(repr, NFP_REPR_TYPE_PF);
- app_fw_flower->pf_repr = NULL;
+ eth_dev = rte_eth_dev_get_by_name(repr->name);
+ if (eth_dev != NULL) {
+ nfp_flower_repr_free(eth_dev, NFP_REPR_TYPE_PF);
+ app_fw_flower->pf_repr = NULL;
+ }
}
}
@@ -719,7 +721,7 @@ nfp_flower_repr_priv_init(struct nfp_app_fw_flower *app_fw_flower,
repr = app_fw_flower->pf_repr;
if (repr != NULL) {
- eth_dev = repr->eth_dev;
+ eth_dev = rte_eth_dev_get_by_name(repr->name);
if (eth_dev != NULL)
eth_dev->process_private = hw_priv;
}
@@ -727,7 +729,7 @@ nfp_flower_repr_priv_init(struct nfp_app_fw_flower *app_fw_flower,
for (i = 0; i < NFP_MAX_PHYPORTS; i++) {
repr = app_fw_flower->phy_reprs[i];
if (repr != NULL) {
- eth_dev = repr->eth_dev;
+ eth_dev = rte_eth_dev_get_by_name(repr->name);
if (eth_dev != NULL)
eth_dev->process_private = hw_priv;
}
@@ -736,7 +738,7 @@ nfp_flower_repr_priv_init(struct nfp_app_fw_flower *app_fw_flower,
for (i = 0; i < MAX_FLOWER_VFS; i++) {
repr = app_fw_flower->vf_reprs[i];
if (repr != NULL) {
- eth_dev = repr->eth_dev;
+ eth_dev = rte_eth_dev_get_by_name(repr->name);
if (eth_dev != NULL)
eth_dev->process_private = hw_priv;
}
@@ -750,7 +752,6 @@ nfp_flower_repr_alloc(struct nfp_app_fw_flower *app_fw_flower,
int i;
int ret;
const char *pci_name;
- struct rte_eth_dev *eth_dev;
struct rte_pci_device *pci_dev;
struct nfp_eth_table *nfp_eth_table;
struct nfp_eth_table_port *eth_port;
@@ -760,7 +761,6 @@ nfp_flower_repr_alloc(struct nfp_app_fw_flower *app_fw_flower,
};
nfp_eth_table = hw_priv->pf_dev->nfp_eth_table;
- eth_dev = app_fw_flower->ctrl_hw->eth_dev;
/* Send a NFP_FLOWER_CMSG_TYPE_MAC_REPR cmsg to hardware */
ret = nfp_flower_cmsg_mac_repr(app_fw_flower, nfp_eth_table);
@@ -783,7 +783,7 @@ nfp_flower_repr_alloc(struct nfp_app_fw_flower *app_fw_flower,
"%s_repr_pf", pci_name);
/* Create a eth_dev for this representor */
- ret = rte_eth_dev_create(eth_dev->device, flower_repr.name,
+ ret = rte_eth_dev_create(&pci_dev->device, flower_repr.name,
sizeof(struct nfp_flower_representor),
NULL, NULL, nfp_flower_pf_repr_init, &flower_repr);
if (ret != 0) {
@@ -808,7 +808,7 @@ nfp_flower_repr_alloc(struct nfp_app_fw_flower *app_fw_flower,
* Create a eth_dev for this representor.
* This will also allocate private memory for the device.
*/
- ret = rte_eth_dev_create(eth_dev->device, flower_repr.name,
+ ret = rte_eth_dev_create(&pci_dev->device, flower_repr.name,
sizeof(struct nfp_flower_representor),
NULL, NULL, nfp_flower_repr_init, &flower_repr);
if (ret != 0) {
@@ -837,7 +837,7 @@ nfp_flower_repr_alloc(struct nfp_app_fw_flower *app_fw_flower,
"%s_repr_vf%d", pci_name, i);
/* This will also allocate private memory for the device */
- ret = rte_eth_dev_create(eth_dev->device, flower_repr.name,
+ ret = rte_eth_dev_create(&pci_dev->device, flower_repr.name,
sizeof(struct nfp_flower_representor),
NULL, NULL, nfp_flower_repr_init, &flower_repr);
if (ret != 0) {
diff --git a/drivers/net/nfp/flower/nfp_flower_representor.h b/drivers/net/nfp/flower/nfp_flower_representor.h
index 41965f6e0e..d539e53b23 100644
--- a/drivers/net/nfp/flower/nfp_flower_representor.h
+++ b/drivers/net/nfp/flower/nfp_flower_representor.h
@@ -20,7 +20,6 @@ struct nfp_flower_representor {
struct rte_ring *ring;
struct rte_eth_link link;
struct rte_eth_stats repr_stats;
- struct rte_eth_dev *eth_dev;
};
int nfp_flower_repr_create(struct nfp_app_fw_flower *app_fw_flower,
diff --git a/drivers/net/nfp/nfp_ethdev.c b/drivers/net/nfp/nfp_ethdev.c
index 258b617eb2..d450c9472e 100644
--- a/drivers/net/nfp/nfp_ethdev.c
+++ b/drivers/net/nfp/nfp_ethdev.c
@@ -100,15 +100,14 @@ nfp_devargs_parse(struct nfp_devargs *nfp_devargs_param,
static void
nfp_net_pf_read_mac(struct nfp_app_fw_nic *app_fw_nic,
- uint16_t port)
+ uint16_t port,
+ struct nfp_net_hw_priv *hw_priv)
{
struct nfp_net_hw *hw;
- struct nfp_net_hw_priv *hw_priv;
struct nfp_eth_table *nfp_eth_table;
/* Grab a pointer to the correct physical port */
hw = app_fw_nic->ports[port];
- hw_priv = hw->eth_dev->process_private;
nfp_eth_table = hw_priv->pf_dev->nfp_eth_table;
@@ -582,17 +581,14 @@ nfp_net_uninit(struct rte_eth_dev *eth_dev)
static void
nfp_cleanup_port_app_fw_nic(struct nfp_pf_dev *pf_dev,
- uint8_t id)
+ uint8_t id,
+ struct rte_eth_dev *eth_dev)
{
- struct rte_eth_dev *eth_dev;
struct nfp_app_fw_nic *app_fw_nic;
app_fw_nic = pf_dev->app_fw_priv;
if (app_fw_nic->ports[id] != NULL) {
- eth_dev = app_fw_nic->ports[id]->eth_dev;
- if (eth_dev != NULL)
- nfp_net_uninit(eth_dev);
-
+ nfp_net_uninit(eth_dev);
app_fw_nic->ports[id] = NULL;
}
}
@@ -691,7 +687,7 @@ nfp_net_close(struct rte_eth_dev *dev)
if (pf_dev->app_fw_id != NFP_APP_FW_CORE_NIC)
return -EINVAL;
- nfp_cleanup_port_app_fw_nic(pf_dev, hw->idx);
+ nfp_cleanup_port_app_fw_nic(pf_dev, hw->idx, dev);
for (i = 0; i < app_fw_nic->total_phyports; i++) {
id = nfp_function_id_get(pf_dev, i);
@@ -971,7 +967,6 @@ nfp_net_init(struct rte_eth_dev *eth_dev)
net_hw->tx_bar = pf_dev->qc_bar + tx_base * NFP_QCP_QUEUE_ADDR_SZ;
net_hw->rx_bar = pf_dev->qc_bar + rx_base * NFP_QCP_QUEUE_ADDR_SZ;
- eth_dev->data->dev_private = net_hw;
PMD_INIT_LOG(DEBUG, "ctrl_bar: %p, tx_bar: %p, rx_bar: %p",
hw->ctrl_bar, net_hw->tx_bar, net_hw->rx_bar);
@@ -1002,7 +997,7 @@ nfp_net_init(struct rte_eth_dev *eth_dev)
goto xstats_free;
}
- nfp_net_pf_read_mac(app_fw_nic, port);
+ nfp_net_pf_read_mac(app_fw_nic, port, hw_priv);
nfp_write_mac(hw, &hw->mac_addr.addr_bytes[0]);
if (rte_is_valid_assigned_ether_addr(&hw->mac_addr) == 0) {
@@ -1580,7 +1575,6 @@ nfp_init_app_fw_nic(struct nfp_net_hw_priv *hw_priv)
/* Add this device to the PF's array of physical ports */
app_fw_nic->ports[id] = hw;
- hw->eth_dev = eth_dev;
hw->idx = id;
hw->nfp_idx = nfp_eth_table->ports[id].index;
@@ -1605,12 +1599,18 @@ nfp_init_app_fw_nic(struct nfp_net_hw_priv *hw_priv)
port_cleanup:
for (i = 0; i < app_fw_nic->total_phyports; i++) {
- id = nfp_function_id_get(pf_dev, i);
- hw = app_fw_nic->ports[id];
+ struct rte_eth_dev *eth_dev;
- if (hw != NULL && hw->eth_dev != NULL) {
- nfp_net_uninit(hw->eth_dev);
- rte_eth_dev_release_port(hw->eth_dev);
+ if (pf_dev->multi_pf.enabled)
+ snprintf(port_name, sizeof(port_name), "%s",
+ pf_dev->pci_dev->device.name);
+ else
+ snprintf(port_name, sizeof(port_name), "%s_port%u",
+ pf_dev->pci_dev->device.name, i);
+ eth_dev = rte_eth_dev_get_by_name(port_name);
+ if (eth_dev != NULL) {
+ nfp_net_uninit(eth_dev);
+ rte_eth_dev_release_port(eth_dev);
}
}
nfp_cpp_area_release_free(pf_dev->ctrl_area);
diff --git a/drivers/net/nfp/nfp_net_common.h b/drivers/net/nfp/nfp_net_common.h
index 40f314af0a..a4eaed6433 100644
--- a/drivers/net/nfp/nfp_net_common.h
+++ b/drivers/net/nfp/nfp_net_common.h
@@ -167,9 +167,6 @@ struct nfp_net_hw {
/** The parent class */
struct nfp_hw super;
- /** Backpointer to the eth_dev of this port */
- struct rte_eth_dev *eth_dev;
-
/** TX pointer ring write back memzone */
const struct rte_memzone *txrwb_mz;
--
2.39.1
^ permalink raw reply [flat|nested] 24+ messages in thread* [PATCH v2 8/8] net/nfp: unify port create and destroy interface
2024-04-19 5:23 ` [PATCH 0/8] refactor logic to support secondary process Chaoyong He
` (6 preceding siblings ...)
2024-04-19 5:23 ` [PATCH v2 7/8] net/nfp: remove ethernet device data field Chaoyong He
@ 2024-04-19 5:23 ` Chaoyong He
2024-05-20 22:19 ` [PATCH 0/8] refactor logic to support secondary process Ferruh Yigit
2024-05-21 15:08 ` Ferruh Yigit
9 siblings, 0 replies; 24+ messages in thread
From: Chaoyong He @ 2024-04-19 5:23 UTC (permalink / raw)
To: dev; +Cc: oss-drivers, Chaoyong He, Long Wu, Peng Zhang
The interface 'rte_eth_dev_create()' and 'rte_eth_dev_destroy()' are
available for both primary and secondary process.
Signed-off-by: Chaoyong He <chaoyong.he@corigine.com>
Signed-off-by: Long Wu <long.wu@corigine.com>
Reviewed-by: Peng Zhang <peng.zhang@corigine.com>
---
drivers/net/nfp/flower/nfp_flower.c | 31 +++--
.../net/nfp/flower/nfp_flower_representor.c | 18 +--
drivers/net/nfp/nfp_ethdev.c | 112 +++++++++---------
3 files changed, 80 insertions(+), 81 deletions(-)
diff --git a/drivers/net/nfp/flower/nfp_flower.c b/drivers/net/nfp/flower/nfp_flower.c
index 76e5d674f1..0edebd574a 100644
--- a/drivers/net/nfp/flower/nfp_flower.c
+++ b/drivers/net/nfp/flower/nfp_flower.c
@@ -798,25 +798,36 @@ nfp_uninit_app_fw_flower(struct nfp_net_hw_priv *hw_priv)
rte_free(app_fw_flower);
}
+static int
+nfp_secondary_flower_init(struct rte_eth_dev *eth_dev,
+ void *para)
+{
+ eth_dev->process_private = para;
+ eth_dev->dev_ops = &nfp_flower_pf_vnic_ops;
+ eth_dev->rx_pkt_burst = nfp_net_recv_pkts;
+ eth_dev->tx_pkt_burst = nfp_flower_pf_xmit_pkts;
+
+ return 0;
+}
+
int
nfp_secondary_init_app_fw_flower(struct nfp_net_hw_priv *hw_priv)
{
- struct rte_eth_dev *eth_dev;
- const char *port_name = "pf_vnic_eth_dev";
+ int ret;
+ const char *pci_name;
+ char port_name[RTE_ETH_NAME_MAX_LEN];
+
+ pci_name = strchr(hw_priv->pf_dev->pci_dev->name, ':') + 1;
+ snprintf(port_name, RTE_ETH_NAME_MAX_LEN, "%s_repr_pf", pci_name);
PMD_INIT_LOG(DEBUG, "Secondary attaching to port %s", port_name);
- eth_dev = rte_eth_dev_attach_secondary(port_name);
- if (eth_dev == NULL) {
+ ret = rte_eth_dev_create(&hw_priv->pf_dev->pci_dev->device, port_name, 0, NULL,
+ NULL, nfp_secondary_flower_init, hw_priv);
+ if (ret != 0) {
PMD_INIT_LOG(ERR, "Secondary process attach to port %s failed", port_name);
return -ENODEV;
}
- eth_dev->process_private = hw_priv;
- eth_dev->dev_ops = &nfp_flower_pf_vnic_ops;
- eth_dev->rx_pkt_burst = nfp_net_recv_pkts;
- eth_dev->tx_pkt_burst = nfp_flower_pf_xmit_pkts;
- rte_eth_dev_probing_finish(eth_dev);
-
return 0;
}
diff --git a/drivers/net/nfp/flower/nfp_flower_representor.c b/drivers/net/nfp/flower/nfp_flower_representor.c
index 60f02ad919..f9001ab1f0 100644
--- a/drivers/net/nfp/flower/nfp_flower_representor.c
+++ b/drivers/net/nfp/flower/nfp_flower_representor.c
@@ -683,10 +683,8 @@ nfp_flower_repr_free_all(struct nfp_app_fw_flower *app_fw_flower)
repr = app_fw_flower->vf_reprs[i];
if (repr != NULL) {
eth_dev = rte_eth_dev_get_by_name(repr->name);
- if (eth_dev != NULL) {
- nfp_flower_repr_free(eth_dev, NFP_REPR_TYPE_VF);
- app_fw_flower->vf_reprs[i] = NULL;
- }
+ if (eth_dev != NULL)
+ rte_eth_dev_destroy(eth_dev, nfp_flower_repr_uninit);
}
}
@@ -694,20 +692,16 @@ nfp_flower_repr_free_all(struct nfp_app_fw_flower *app_fw_flower)
repr = app_fw_flower->phy_reprs[i];
if (repr != NULL) {
eth_dev = rte_eth_dev_get_by_name(repr->name);
- if (eth_dev != NULL) {
- nfp_flower_repr_free(eth_dev, NFP_REPR_TYPE_PHYS_PORT);
- app_fw_flower->phy_reprs[i] = NULL;
- }
+ if (eth_dev != NULL)
+ rte_eth_dev_destroy(eth_dev, nfp_flower_repr_uninit);
}
}
repr = app_fw_flower->pf_repr;
if (repr != NULL) {
eth_dev = rte_eth_dev_get_by_name(repr->name);
- if (eth_dev != NULL) {
- nfp_flower_repr_free(eth_dev, NFP_REPR_TYPE_PF);
- app_fw_flower->pf_repr = NULL;
- }
+ if (eth_dev != NULL)
+ rte_eth_dev_destroy(eth_dev, nfp_flower_pf_repr_uninit);
}
}
diff --git a/drivers/net/nfp/nfp_ethdev.c b/drivers/net/nfp/nfp_ethdev.c
index d450c9472e..cdc946faff 100644
--- a/drivers/net/nfp/nfp_ethdev.c
+++ b/drivers/net/nfp/nfp_ethdev.c
@@ -34,6 +34,16 @@
#define NFP_PF_DRIVER_NAME net_nfp_pf
#define NFP_PF_FORCE_RELOAD_FW "force_reload_fw"
+struct nfp_net_init {
+ /** Sequential physical port number, only valid for CoreNIC firmware */
+ uint8_t idx;
+
+ /** Internal port number as seen from NFP */
+ uint8_t nfp_idx;
+
+ struct nfp_net_hw_priv *hw_priv;
+};
+
static int
nfp_devarg_handle_int(const char *key,
const char *value,
@@ -559,7 +569,7 @@ nfp_net_keepalive_stop(struct nfp_multi_pf *multi_pf)
rte_eal_alarm_cancel(nfp_net_beat_timer, (void *)multi_pf);
}
-static void
+static int
nfp_net_uninit(struct rte_eth_dev *eth_dev)
{
struct nfp_net_hw *net_hw;
@@ -577,6 +587,8 @@ nfp_net_uninit(struct rte_eth_dev *eth_dev)
nfp_ipsec_uninit(eth_dev);
if (net_hw->mac_stats_area != NULL)
nfp_cpp_area_release_free(net_hw->mac_stats_area);
+
+ return 0;
}
static void
@@ -875,7 +887,8 @@ nfp_net_ethdev_ops_mount(struct nfp_net_hw *hw,
}
static int
-nfp_net_init(struct rte_eth_dev *eth_dev)
+nfp_net_init(struct rte_eth_dev *eth_dev,
+ void *para)
{
int err;
uint16_t port;
@@ -884,6 +897,7 @@ nfp_net_init(struct rte_eth_dev *eth_dev)
struct nfp_hw *hw;
struct nfp_net_hw *net_hw;
struct nfp_pf_dev *pf_dev;
+ struct nfp_net_init *hw_init;
struct rte_pci_device *pci_dev;
struct nfp_net_hw_priv *hw_priv;
struct nfp_app_fw_nic *app_fw_nic;
@@ -891,6 +905,11 @@ nfp_net_init(struct rte_eth_dev *eth_dev)
pci_dev = RTE_ETH_DEV_TO_PCI(eth_dev);
net_hw = eth_dev->data->dev_private;
+ hw_init = para;
+ net_hw->idx = hw_init->idx;
+ net_hw->nfp_idx = hw_init->nfp_idx;
+ eth_dev->process_private = hw_init->hw_priv;
+
/* Use backpointer here to the PF of this eth_dev */
hw_priv = eth_dev->process_private;
pf_dev = hw_priv->pf_dev;
@@ -898,7 +917,10 @@ nfp_net_init(struct rte_eth_dev *eth_dev)
/* Use backpointer to the CoreNIC app struct */
app_fw_nic = NFP_PRIV_TO_APP_FW_NIC(pf_dev->app_fw_priv);
- port = ((struct nfp_net_hw *)eth_dev->data->dev_private)->idx;
+ /* Add this device to the PF's array of physical ports */
+ app_fw_nic->ports[net_hw->idx] = net_hw;
+
+ port = net_hw->idx;
if (port > 7) {
PMD_DRV_LOG(ERR, "Port value is wrong");
return -ENODEV;
@@ -1475,15 +1497,15 @@ nfp_init_app_fw_nic(struct nfp_net_hw_priv *hw_priv)
uint8_t id;
int ret = 0;
uint32_t total_vnics;
- struct nfp_net_hw *hw;
- unsigned int numa_node;
- struct rte_eth_dev *eth_dev;
struct nfp_app_fw_nic *app_fw_nic;
struct nfp_eth_table *nfp_eth_table;
char bar_name[RTE_ETH_NAME_MAX_LEN];
char port_name[RTE_ETH_NAME_MAX_LEN];
char vnic_name[RTE_ETH_NAME_MAX_LEN];
struct nfp_pf_dev *pf_dev = hw_priv->pf_dev;
+ struct nfp_net_init hw_init = {
+ .hw_priv = hw_priv,
+ };
nfp_eth_table = pf_dev->nfp_eth_table;
PMD_INIT_LOG(INFO, "Total physical ports: %d", nfp_eth_table->count);
@@ -1543,7 +1565,6 @@ nfp_init_app_fw_nic(struct nfp_net_hw_priv *hw_priv)
PMD_INIT_LOG(DEBUG, "ctrl bar: %p", pf_dev->ctrl_bar);
/* Loop through all physical ports on PF */
- numa_node = rte_socket_id();
for (i = 0; i < app_fw_nic->total_phyports; i++) {
if (pf_dev->multi_pf.enabled)
snprintf(port_name, sizeof(port_name), "%s",
@@ -1552,46 +1573,14 @@ nfp_init_app_fw_nic(struct nfp_net_hw_priv *hw_priv)
snprintf(port_name, sizeof(port_name), "%s_port%u",
pf_dev->pci_dev->device.name, i);
- /* Allocate a eth_dev for this phyport */
- eth_dev = rte_eth_dev_allocate(port_name);
- if (eth_dev == NULL) {
- ret = -ENODEV;
- goto port_cleanup;
- }
-
- /* Allocate memory for this phyport */
- eth_dev->data->dev_private = rte_zmalloc_socket(port_name,
- sizeof(struct nfp_net_hw),
- RTE_CACHE_LINE_SIZE, numa_node);
- if (eth_dev->data->dev_private == NULL) {
- ret = -ENOMEM;
- rte_eth_dev_release_port(eth_dev);
- goto port_cleanup;
- }
-
- hw = eth_dev->data->dev_private;
id = nfp_function_id_get(pf_dev, i);
-
- /* Add this device to the PF's array of physical ports */
- app_fw_nic->ports[id] = hw;
-
- hw->idx = id;
- hw->nfp_idx = nfp_eth_table->ports[id].index;
-
- eth_dev->device = &pf_dev->pci_dev->device;
- eth_dev->process_private = hw_priv;
-
- /*
- * Ctrl/tx/rx BAR mappings and remaining init happens in
- * @nfp_net_init()
- */
- ret = nfp_net_init(eth_dev);
- if (ret != 0) {
- ret = -ENODEV;
+ hw_init.idx = id;
+ hw_init.nfp_idx = nfp_eth_table->ports[id].index;
+ ret = rte_eth_dev_create(&pf_dev->pci_dev->device, port_name,
+ sizeof(struct nfp_net_hw), NULL, NULL,
+ nfp_net_init, &hw_init);
+ if (ret != 0)
goto port_cleanup;
- }
-
- rte_eth_dev_probing_finish(eth_dev);
} /* End loop, all ports on this PF */
@@ -1608,10 +1597,8 @@ nfp_init_app_fw_nic(struct nfp_net_hw_priv *hw_priv)
snprintf(port_name, sizeof(port_name), "%s_port%u",
pf_dev->pci_dev->device.name, i);
eth_dev = rte_eth_dev_get_by_name(port_name);
- if (eth_dev != NULL) {
- nfp_net_uninit(eth_dev);
- rte_eth_dev_release_port(eth_dev);
- }
+ if (eth_dev != NULL)
+ rte_eth_dev_destroy(eth_dev, nfp_net_uninit);
}
nfp_cpp_area_release_free(pf_dev->ctrl_area);
app_cleanup:
@@ -2012,6 +1999,20 @@ nfp_pf_init(struct rte_pci_device *pci_dev)
return ret;
}
+static int
+nfp_secondary_net_init(struct rte_eth_dev *eth_dev,
+ void *para)
+{
+ struct nfp_net_hw *net_hw;
+
+ net_hw = eth_dev->data->dev_private;
+ nfp_net_ethdev_ops_mount(net_hw, eth_dev);
+
+ eth_dev->process_private = para;
+
+ return 0;
+}
+
static int
nfp_secondary_init_app_fw_nic(struct nfp_net_hw_priv *hw_priv)
{
@@ -2020,7 +2021,6 @@ nfp_secondary_init_app_fw_nic(struct nfp_net_hw_priv *hw_priv)
int ret = 0;
uint8_t function_id;
uint32_t total_vnics;
- struct nfp_net_hw *hw;
char pf_name[RTE_ETH_NAME_MAX_LEN];
struct nfp_pf_dev *pf_dev = hw_priv->pf_dev;
@@ -2034,7 +2034,6 @@ nfp_secondary_init_app_fw_nic(struct nfp_net_hw_priv *hw_priv)
}
for (i = 0; i < total_vnics; i++) {
- struct rte_eth_dev *eth_dev;
char port_name[RTE_ETH_NAME_MAX_LEN];
if (nfp_check_multi_pf_from_fw(total_vnics))
@@ -2045,18 +2044,13 @@ nfp_secondary_init_app_fw_nic(struct nfp_net_hw_priv *hw_priv)
pf_dev->pci_dev->device.name, i);
PMD_INIT_LOG(DEBUG, "Secondary attaching to port %s", port_name);
- eth_dev = rte_eth_dev_attach_secondary(port_name);
- if (eth_dev == NULL) {
+ ret = rte_eth_dev_create(&pf_dev->pci_dev->device, port_name, 0,
+ NULL, NULL, nfp_secondary_net_init, hw_priv);
+ if (ret != 0) {
PMD_INIT_LOG(ERR, "Secondary process attach to port %s failed", port_name);
ret = -ENODEV;
break;
}
-
- eth_dev->process_private = hw_priv;
- hw = eth_dev->data->dev_private;
- nfp_net_ethdev_ops_mount(hw, eth_dev);
-
- rte_eth_dev_probing_finish(eth_dev);
}
return ret;
--
2.39.1
^ permalink raw reply [flat|nested] 24+ messages in thread* Re: [PATCH 0/8] refactor logic to support secondary process
2024-04-19 5:23 ` [PATCH 0/8] refactor logic to support secondary process Chaoyong He
` (7 preceding siblings ...)
2024-04-19 5:23 ` [PATCH v2 8/8] net/nfp: unify port create and destroy interface Chaoyong He
@ 2024-05-20 22:19 ` Ferruh Yigit
2024-05-21 2:17 ` Chaoyong He
2024-05-21 15:08 ` Ferruh Yigit
9 siblings, 1 reply; 24+ messages in thread
From: Ferruh Yigit @ 2024-05-20 22:19 UTC (permalink / raw)
To: Chaoyong He, dev; +Cc: oss-drivers
On 4/19/2024 6:23 AM, Chaoyong He wrote:
> Refactor data structure and related logic to make the secondary process
> can work as expect.
>
Hi Chaoyong,
Patchset looks good, but I have a question related to the motivation of
moving so many structs to process private data?
Normally ethdev is process private, and ethdev->data is shared. Primary
configures the device and secondary learns shared data and uses it for
datapath.
There are cases, like file descriptors for same file can be different
for different process and process private structure is used.
In below patches, device private data level structs seems moved to the
process private structure, is the intention both primary process and
secondary process do the control path and configuration?
If so, just a reminder that this is not intended usage of the
multi-process support and many control APIs are not designed as thread-safe.
Would you mind describing a little more about your use case that
requires below process private data changes?
> ---
> v2:
> * Solve the build problem.
> ---
>
> Chaoyong He (8):
> net/nfp: fix resource leak of secondary process
> net/nfp: fix configuration BAR problem
> net/nfp: adjust the data field of Rx/Tx queue
> net/nfp: add the process private structure
> net/nfp: move device info data field
> net/nfp: unify CPP acquire method
> net/nfp: remove ethernet device data field
> net/nfp: unify port create and destroy interface
>
> drivers/net/nfp/flower/nfp_flower.c | 127 ++++----
> drivers/net/nfp/flower/nfp_flower.h | 11 +-
> drivers/net/nfp/flower/nfp_flower_cmsg.c | 5 +-
> drivers/net/nfp/flower/nfp_flower_cmsg.h | 3 +-
> drivers/net/nfp/flower/nfp_flower_ctrl.c | 8 +-
> drivers/net/nfp/flower/nfp_flower_flow.c | 16 +-
> .../net/nfp/flower/nfp_flower_representor.c | 151 ++++++----
> .../net/nfp/flower/nfp_flower_representor.h | 4 +-
> drivers/net/nfp/flower/nfp_flower_service.c | 32 +-
> drivers/net/nfp/flower/nfp_flower_service.h | 10 +-
> drivers/net/nfp/nfd3/nfp_nfd3_dp.c | 5 +-
> drivers/net/nfp/nfdk/nfp_nfdk_dp.c | 5 +-
> drivers/net/nfp/nfp_ethdev.c | 274 ++++++++++--------
> drivers/net/nfp/nfp_ethdev_vf.c | 18 +-
> drivers/net/nfp/nfp_net_common.c | 116 ++++----
> drivers/net/nfp/nfp_net_common.h | 30 +-
> drivers/net/nfp/nfp_net_flow.c | 20 +-
> drivers/net/nfp/nfp_rxtx.c | 9 +-
> drivers/net/nfp/nfp_rxtx.h | 69 ++---
> drivers/net/nfp/nfpcore/nfp6000_pcie.c | 34 +--
> 20 files changed, 524 insertions(+), 423 deletions(-)
>
^ permalink raw reply [flat|nested] 24+ messages in thread* RE: [PATCH 0/8] refactor logic to support secondary process
2024-05-20 22:19 ` [PATCH 0/8] refactor logic to support secondary process Ferruh Yigit
@ 2024-05-21 2:17 ` Chaoyong He
2024-05-21 11:09 ` Ferruh Yigit
0 siblings, 1 reply; 24+ messages in thread
From: Chaoyong He @ 2024-05-21 2:17 UTC (permalink / raw)
To: Ferruh Yigit, dev; +Cc: oss-drivers
> On 4/19/2024 6:23 AM, Chaoyong He wrote:
> > Refactor data structure and related logic to make the secondary
> > process can work as expect.
> >
>
> Hi Chaoyong,
>
> Patchset looks good, but I have a question related to the motivation of moving
> so many structs to process private data?
>
> Normally ethdev is process private, and ethdev->data is shared. Primary
> configures the device and secondary learns shared data and uses it for
> datapath.
> There are cases, like file descriptors for same file can be different for different
> process and process private structure is used.
>
> In below patches, device private data level structs seems moved to the process
> private structure, is the intention both primary process and secondary process
> do the control path and configuration?
> If so, just a reminder that this is not intended usage of the multi-process
> support and many control APIs are not designed as thread-safe.
>
> Would you mind describing a little more about your use case that requires
> below process private data changes?
The main motivation of these changes is to solve the problems when customers using the
secondary process (they use primary process for monitor and secondary process for rx/tx/configuration ...).
The NFP card support different firmware applications, this means we need read message from firmware to
decide to run which logic.
And with single-pf firmware (this means one PCI BDF address for multi physical ports), we also need
read message (how many ports totally) from firmware before attach to the primary process.
All this relates with CPP and symbol table, and they are different for primary process and secondary process.
If still put them in the process shared section (ethdev->data), the assignment logic in secondary process will overwrite it
and cause the primary process crash.
So we move them into the process private section (ethdev->process_private).
>
>
> > ---
> > v2:
> > * Solve the build problem.
> > ---
> >
> > Chaoyong He (8):
> > net/nfp: fix resource leak of secondary process
> > net/nfp: fix configuration BAR problem
> > net/nfp: adjust the data field of Rx/Tx queue
> > net/nfp: add the process private structure
> > net/nfp: move device info data field
> > net/nfp: unify CPP acquire method
> > net/nfp: remove ethernet device data field
> > net/nfp: unify port create and destroy interface
> >
> > drivers/net/nfp/flower/nfp_flower.c | 127 ++++----
> > drivers/net/nfp/flower/nfp_flower.h | 11 +-
> > drivers/net/nfp/flower/nfp_flower_cmsg.c | 5 +-
> > drivers/net/nfp/flower/nfp_flower_cmsg.h | 3 +-
> > drivers/net/nfp/flower/nfp_flower_ctrl.c | 8 +-
> > drivers/net/nfp/flower/nfp_flower_flow.c | 16 +-
> > .../net/nfp/flower/nfp_flower_representor.c | 151 ++++++----
> > .../net/nfp/flower/nfp_flower_representor.h | 4 +-
> > drivers/net/nfp/flower/nfp_flower_service.c | 32 +-
> > drivers/net/nfp/flower/nfp_flower_service.h | 10 +-
> > drivers/net/nfp/nfd3/nfp_nfd3_dp.c | 5 +-
> > drivers/net/nfp/nfdk/nfp_nfdk_dp.c | 5 +-
> > drivers/net/nfp/nfp_ethdev.c | 274 ++++++++++--------
> > drivers/net/nfp/nfp_ethdev_vf.c | 18 +-
> > drivers/net/nfp/nfp_net_common.c | 116 ++++----
> > drivers/net/nfp/nfp_net_common.h | 30 +-
> > drivers/net/nfp/nfp_net_flow.c | 20 +-
> > drivers/net/nfp/nfp_rxtx.c | 9 +-
> > drivers/net/nfp/nfp_rxtx.h | 69 ++---
> > drivers/net/nfp/nfpcore/nfp6000_pcie.c | 34 +--
> > 20 files changed, 524 insertions(+), 423 deletions(-)
> >
^ permalink raw reply [flat|nested] 24+ messages in thread
* Re: [PATCH 0/8] refactor logic to support secondary process
2024-05-21 2:17 ` Chaoyong He
@ 2024-05-21 11:09 ` Ferruh Yigit
2024-05-21 11:24 ` Chaoyong He
0 siblings, 1 reply; 24+ messages in thread
From: Ferruh Yigit @ 2024-05-21 11:09 UTC (permalink / raw)
To: Chaoyong He, dev; +Cc: oss-drivers
On 5/21/2024 3:17 AM, Chaoyong He wrote:
>> On 4/19/2024 6:23 AM, Chaoyong He wrote:
>>> Refactor data structure and related logic to make the secondary
>>> process can work as expect.
>>>
>>
>> Hi Chaoyong,
>>
>> Patchset looks good, but I have a question related to the motivation of moving
>> so many structs to process private data?
>>
>> Normally ethdev is process private, and ethdev->data is shared. Primary
>> configures the device and secondary learns shared data and uses it for
>> datapath.
>> There are cases, like file descriptors for same file can be different for different
>> process and process private structure is used.
>>
>> In below patches, device private data level structs seems moved to the process
>> private structure, is the intention both primary process and secondary process
>> do the control path and configuration?
>> If so, just a reminder that this is not intended usage of the multi-process
>> support and many control APIs are not designed as thread-safe.
>>
>> Would you mind describing a little more about your use case that requires
>> below process private data changes?
>
> The main motivation of these changes is to solve the problems when customers using the
> secondary process (they use primary process for monitor and secondary process for rx/tx/configuration ...).
>
Got it, if you want to do 'configuration' on the secondary, it requires
more information, and as this is not shared you need to move them to
process private.
This approach requires synchronizing secondaries for this control path.
So more work for the application layer.
I understand you are trying to enable your customer, and this is a
solution but I am not sure this is the best approach. And this solution
won't be portable, because many PMDs won't support configuring from
secondary.
Can you guide your customer that configuring on primary and limit
secondaries for the datapath?
This way only some limited information is required to shared with
secondaries (lets say like firmware application version) and these can
be passed via shared data or even MP communication.
Although this is not a hard requirement, I believe this can make both
your and your customer's life easier in long run.
> The NFP card support different firmware applications, this means we need read message from firmware to
> decide to run which logic.
>
> And with single-pf firmware (this means one PCI BDF address for multi physical ports), we also need
> read message (how many ports totally) from firmware before attach to the primary process.
>
> All this relates with CPP and symbol table, and they are different for primary process and secondary process.
> If still put them in the process shared section (ethdev->data), the assignment logic in secondary process will overwrite it
> and cause the primary process crash.
>
> So we move them into the process private section (ethdev->process_private).
>
^ permalink raw reply [flat|nested] 24+ messages in thread
* RE: [PATCH 0/8] refactor logic to support secondary process
2024-05-21 11:09 ` Ferruh Yigit
@ 2024-05-21 11:24 ` Chaoyong He
2024-05-21 12:54 ` Ferruh Yigit
0 siblings, 1 reply; 24+ messages in thread
From: Chaoyong He @ 2024-05-21 11:24 UTC (permalink / raw)
To: Ferruh Yigit, dev; +Cc: oss-drivers
> On 5/21/2024 3:17 AM, Chaoyong He wrote:
> >> On 4/19/2024 6:23 AM, Chaoyong He wrote:
> >>> Refactor data structure and related logic to make the secondary
> >>> process can work as expect.
> >>>
> >>
> >> Hi Chaoyong,
> >>
> >> Patchset looks good, but I have a question related to the motivation
> >> of moving so many structs to process private data?
> >>
> >> Normally ethdev is process private, and ethdev->data is shared.
> >> Primary configures the device and secondary learns shared data and
> >> uses it for datapath.
> >> There are cases, like file descriptors for same file can be different
> >> for different process and process private structure is used.
> >>
> >> In below patches, device private data level structs seems moved to
> >> the process private structure, is the intention both primary process
> >> and secondary process do the control path and configuration?
> >> If so, just a reminder that this is not intended usage of the
> >> multi-process support and many control APIs are not designed as thread-
> safe.
> >>
> >> Would you mind describing a little more about your use case that
> >> requires below process private data changes?
> >
> > The main motivation of these changes is to solve the problems when
> > customers using the secondary process (they use primary process for
> monitor and secondary process for rx/tx/configuration ...).
> >
>
> Got it, if you want to do 'configuration' on the secondary, it requires more
> information, and as this is not shared you need to move them to process
> private.
>
> This approach requires synchronizing secondaries for this control path.
> So more work for the application layer.
>
> I understand you are trying to enable your customer, and this is a solution but
> I am not sure this is the best approach. And this solution won't be portable,
> because many PMDs won't support configuring from secondary.
>
> Can you guide your customer that configuring on primary and limit
> secondaries for the datapath?
Sorry, but it is almost certain that it is impossible to make customer modify their architecture.
Because they have complex software stack, and they themselves are only user of the software stack.
They are just responsible for the basic NIC adaptation work and the upper software architecture are
design and maintain by some other department.
For this generation NFP hardware and firmware architecture, seems this is the only best solution
we can achieve.
But for the next generation NIC and firmware (we already in development, and will publish in a near future),
we will have one PCI BDF address for one physical port and use a unified firmware.
Then the problems we meet for now will not exist anymore, and we can make the logic just as what you said.
> This way only some limited information is required to shared with secondaries
> (lets say like firmware application version) and these can be passed via shared
> data or even MP communication.
>
> Although this is not a hard requirement, I believe this can make both your and
> your customer's life easier in long run.
>
>
> > The NFP card support different firmware applications, this means we
> > need read message from firmware to decide to run which logic.
> >
> > And with single-pf firmware (this means one PCI BDF address for multi
> > physical ports), we also need read message (how many ports totally) from
> firmware before attach to the primary process.
> >
> > All this relates with CPP and symbol table, and they are different for primary
> process and secondary process.
> > If still put them in the process shared section (ethdev->data), the
> > assignment logic in secondary process will overwrite it and cause the primary
> process crash.
> >
> > So we move them into the process private section (ethdev-
> >process_private).
> >
>
>
^ permalink raw reply [flat|nested] 24+ messages in thread
* Re: [PATCH 0/8] refactor logic to support secondary process
2024-05-21 11:24 ` Chaoyong He
@ 2024-05-21 12:54 ` Ferruh Yigit
0 siblings, 0 replies; 24+ messages in thread
From: Ferruh Yigit @ 2024-05-21 12:54 UTC (permalink / raw)
To: Chaoyong He, dev; +Cc: oss-drivers
On 5/21/2024 12:24 PM, Chaoyong He wrote:
>> On 5/21/2024 3:17 AM, Chaoyong He wrote:
>>>> On 4/19/2024 6:23 AM, Chaoyong He wrote:
>>>>> Refactor data structure and related logic to make the secondary
>>>>> process can work as expect.
>>>>>
>>>>
>>>> Hi Chaoyong,
>>>>
>>>> Patchset looks good, but I have a question related to the motivation
>>>> of moving so many structs to process private data?
>>>>
>>>> Normally ethdev is process private, and ethdev->data is shared.
>>>> Primary configures the device and secondary learns shared data and
>>>> uses it for datapath.
>>>> There are cases, like file descriptors for same file can be different
>>>> for different process and process private structure is used.
>>>>
>>>> In below patches, device private data level structs seems moved to
>>>> the process private structure, is the intention both primary process
>>>> and secondary process do the control path and configuration?
>>>> If so, just a reminder that this is not intended usage of the
>>>> multi-process support and many control APIs are not designed as thread-
>> safe.
>>>>
>>>> Would you mind describing a little more about your use case that
>>>> requires below process private data changes?
>>>
>>> The main motivation of these changes is to solve the problems when
>>> customers using the secondary process (they use primary process for
>> monitor and secondary process for rx/tx/configuration ...).
>>>
>>
>> Got it, if you want to do 'configuration' on the secondary, it requires more
>> information, and as this is not shared you need to move them to process
>> private.
>>
>> This approach requires synchronizing secondaries for this control path.
>> So more work for the application layer.
>>
>> I understand you are trying to enable your customer, and this is a solution but
>> I am not sure this is the best approach. And this solution won't be portable,
>> because many PMDs won't support configuring from secondary.
>>
>> Can you guide your customer that configuring on primary and limit
>> secondaries for the datapath?
>
> Sorry, but it is almost certain that it is impossible to make customer modify their architecture.
> Because they have complex software stack, and they themselves are only user of the software stack.
> They are just responsible for the basic NIC adaptation work and the upper software architecture are
> design and maintain by some other department.
>
> For this generation NFP hardware and firmware architecture, seems this is the only best solution
> we can achieve.
>
> But for the next generation NIC and firmware (we already in development, and will publish in a near future),
> we will have one PCI BDF address for one physical port and use a unified firmware.
> Then the problems we meet for now will not exist anymore, and we can make the logic just as what you said.
>
Ack, it looks like you are making an informed decision, although it is
not ideal, I will proceed with the set.
>> This way only some limited information is required to shared with secondaries
>> (lets say like firmware application version) and these can be passed via shared
>> data or even MP communication.
>>
>> Although this is not a hard requirement, I believe this can make both your and
>> your customer's life easier in long run.
>>
>>
>>> The NFP card support different firmware applications, this means we
>>> need read message from firmware to decide to run which logic.
>>>
>>> And with single-pf firmware (this means one PCI BDF address for multi
>>> physical ports), we also need read message (how many ports totally) from
>> firmware before attach to the primary process.
>>>
>>> All this relates with CPP and symbol table, and they are different for primary
>> process and secondary process.
>>> If still put them in the process shared section (ethdev->data), the
>>> assignment logic in secondary process will overwrite it and cause the primary
>> process crash.
>>>
>>> So we move them into the process private section (ethdev-
>>> process_private).
>>>
>>
>>
>
^ permalink raw reply [flat|nested] 24+ messages in thread
* Re: [PATCH 0/8] refactor logic to support secondary process
2024-04-19 5:23 ` [PATCH 0/8] refactor logic to support secondary process Chaoyong He
` (8 preceding siblings ...)
2024-05-20 22:19 ` [PATCH 0/8] refactor logic to support secondary process Ferruh Yigit
@ 2024-05-21 15:08 ` Ferruh Yigit
9 siblings, 0 replies; 24+ messages in thread
From: Ferruh Yigit @ 2024-05-21 15:08 UTC (permalink / raw)
To: Chaoyong He, dev; +Cc: oss-drivers
On 4/19/2024 6:23 AM, Chaoyong He wrote:
> Refactor data structure and related logic to make the secondary process
> can work as expect.
>
> ---
> v2:
> * Solve the build problem.
> ---
>
> Chaoyong He (8):
> net/nfp: fix resource leak of secondary process
> net/nfp: fix configuration BAR problem
> net/nfp: adjust the data field of Rx/Tx queue
> net/nfp: add the process private structure
> net/nfp: move device info data field
> net/nfp: unify CPP acquire method
> net/nfp: remove ethernet device data field
> net/nfp: unify port create and destroy interface
>
Series applied to dpdk-next-net/main, thanks.
^ permalink raw reply [flat|nested] 24+ messages in thread