* [PATCH 00/13] net/ionic: miscellaneous fixes and improvements
@ 2024-02-02 19:32 Andrew Boyer
2024-02-02 19:32 ` [PATCH 01/13] net/ionic: add stat for completion queue entries processed Andrew Boyer
` (28 more replies)
0 siblings, 29 replies; 52+ messages in thread
From: Andrew Boyer @ 2024-02-02 19:32 UTC (permalink / raw)
To: dev; +Cc: Andrew Boyer
This patchset provides miscellaneous fixes and improvements for
the net/ionic driver used by AMD Pensando devices.
Akshay Dorwat (1):
net/ionic: fix RSS query routine
Andrew Boyer (8):
net/ionic: add stat for completion queue entries processed
net/ionic: increase max supported MTU to 9750 bytes
net/ionic: don't auto-enable Rx scatter-gather a second time
net/ionic: replace non-standard type in structure definition
net/ionic: fix device close sequence to avoid crash
net/ionic: optimize device close operation
net/ionic: optimize device stop operation
net/ionic: optimize device start operation
Brad Larson (1):
net/ionic: add flexible firmware xstat counters
Neel Patel (2):
net/ionic: fix missing volatile type for cqe pointers
net/ionic: memcpy descriptors when using Q-in-CMB
Vamsi Krishna Atluri (1):
net/ionic: report 1G and 200G link speeds when applicable
drivers/net/ionic/ionic.h | 3 +
drivers/net/ionic/ionic_dev.c | 9 +-
drivers/net/ionic/ionic_dev.h | 8 +-
drivers/net/ionic/ionic_dev_pci.c | 2 +-
drivers/net/ionic/ionic_ethdev.c | 81 ++++++--
drivers/net/ionic/ionic_if.h | 70 +++----
drivers/net/ionic/ionic_lif.c | 288 +++++++++++++++++---------
drivers/net/ionic/ionic_lif.h | 19 +-
drivers/net/ionic/ionic_main.c | 17 +-
drivers/net/ionic/ionic_rxtx.c | 160 ++++++++++----
drivers/net/ionic/ionic_rxtx.h | 80 ++++++-
drivers/net/ionic/ionic_rxtx_sg.c | 28 +--
drivers/net/ionic/ionic_rxtx_simple.c | 28 +--
13 files changed, 550 insertions(+), 243 deletions(-)
--
2.17.1
^ permalink raw reply [flat|nested] 52+ messages in thread
* [PATCH 01/13] net/ionic: add stat for completion queue entries processed
2024-02-02 19:32 [PATCH 00/13] net/ionic: miscellaneous fixes and improvements Andrew Boyer
@ 2024-02-02 19:32 ` Andrew Boyer
2024-02-03 4:24 ` Stephen Hemminger
2024-02-02 19:32 ` [PATCH 02/13] net/ionic: increase max supported MTU to 9750 bytes Andrew Boyer
` (27 subsequent siblings)
28 siblings, 1 reply; 52+ messages in thread
From: Andrew Boyer @ 2024-02-02 19:32 UTC (permalink / raw)
To: dev; +Cc: Andrew Boyer
When completion coalescing is turned on in the FW, there will be
fewer CQE than Tx packets. Expose the stat through debug logging.
Signed-off-by: Andrew Boyer <andrew.boyer@amd.com>
---
drivers/net/ionic/ionic_lif.h | 1 +
drivers/net/ionic/ionic_rxtx.c | 3 +++
drivers/net/ionic/ionic_rxtx_sg.c | 2 ++
drivers/net/ionic/ionic_rxtx_simple.c | 2 ++
4 files changed, 8 insertions(+)
diff --git a/drivers/net/ionic/ionic_lif.h b/drivers/net/ionic/ionic_lif.h
index 36b3bcc5a9..cac7a4583b 100644
--- a/drivers/net/ionic/ionic_lif.h
+++ b/drivers/net/ionic/ionic_lif.h
@@ -32,6 +32,7 @@
struct ionic_tx_stats {
uint64_t packets;
uint64_t bytes;
+ uint64_t comps;
uint64_t drop;
uint64_t stop;
uint64_t no_csum;
diff --git a/drivers/net/ionic/ionic_rxtx.c b/drivers/net/ionic/ionic_rxtx.c
index b9e73b4871..d92b231f8f 100644
--- a/drivers/net/ionic/ionic_rxtx.c
+++ b/drivers/net/ionic/ionic_rxtx.c
@@ -117,6 +117,9 @@ ionic_dev_tx_queue_stop(struct rte_eth_dev *eth_dev, uint16_t tx_queue_id)
stats = &txq->stats;
IONIC_PRINT(DEBUG, "TX queue %u pkts %ju tso %ju",
txq->qcq.q.index, stats->packets, stats->tso);
+ IONIC_PRINT(DEBUG, "TX queue %u comps %ju (%ju per)",
+ txq->qcq.q.index, stats->comps,
+ stats->comps ? stats->packets / stats->comps : 0);
return 0;
}
diff --git a/drivers/net/ionic/ionic_rxtx_sg.c b/drivers/net/ionic/ionic_rxtx_sg.c
index ab8e56e91c..6c028a698c 100644
--- a/drivers/net/ionic/ionic_rxtx_sg.c
+++ b/drivers/net/ionic/ionic_rxtx_sg.c
@@ -26,6 +26,7 @@ ionic_tx_flush_sg(struct ionic_tx_qcq *txq)
{
struct ionic_cq *cq = &txq->qcq.cq;
struct ionic_queue *q = &txq->qcq.q;
+ struct ionic_tx_stats *stats = &txq->stats;
struct rte_mbuf *txm;
struct ionic_txq_comp *cq_desc, *cq_desc_base = cq->base;
void **info;
@@ -72,6 +73,7 @@ ionic_tx_flush_sg(struct ionic_tx_qcq *txq)
}
cq_desc = &cq_desc_base[cq->tail_idx];
+ stats->comps++;
}
}
diff --git a/drivers/net/ionic/ionic_rxtx_simple.c b/drivers/net/ionic/ionic_rxtx_simple.c
index 5f81856256..5969287b66 100644
--- a/drivers/net/ionic/ionic_rxtx_simple.c
+++ b/drivers/net/ionic/ionic_rxtx_simple.c
@@ -26,6 +26,7 @@ ionic_tx_flush(struct ionic_tx_qcq *txq)
{
struct ionic_cq *cq = &txq->qcq.cq;
struct ionic_queue *q = &txq->qcq.q;
+ struct ionic_tx_stats *stats = &txq->stats;
struct rte_mbuf *txm;
struct ionic_txq_comp *cq_desc, *cq_desc_base = cq->base;
void **info;
@@ -67,6 +68,7 @@ ionic_tx_flush(struct ionic_tx_qcq *txq)
}
cq_desc = &cq_desc_base[cq->tail_idx];
+ stats->comps++;
}
}
--
2.17.1
^ permalink raw reply [flat|nested] 52+ messages in thread
* [PATCH 02/13] net/ionic: increase max supported MTU to 9750 bytes
2024-02-02 19:32 [PATCH 00/13] net/ionic: miscellaneous fixes and improvements Andrew Boyer
2024-02-02 19:32 ` [PATCH 01/13] net/ionic: add stat for completion queue entries processed Andrew Boyer
@ 2024-02-02 19:32 ` Andrew Boyer
2024-02-02 19:32 ` [PATCH 03/13] net/ionic: don't auto-enable Rx scatter-gather a second time Andrew Boyer
` (26 subsequent siblings)
28 siblings, 0 replies; 52+ messages in thread
From: Andrew Boyer @ 2024-02-02 19:32 UTC (permalink / raw)
To: dev; +Cc: Andrew Boyer, Bhuvan Mital
Some configurations want to use values this high internally.
Allow them to do so without modifying the code.
Signed-off-by: Andrew Boyer <andrew.boyer@amd.com>
Signed-off-by: Bhuvan Mital <bhuvan.mital@amd.com>
---
drivers/net/ionic/ionic_dev.h | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/drivers/net/ionic/ionic_dev.h b/drivers/net/ionic/ionic_dev.h
index b1e74fbd89..971c261b27 100644
--- a/drivers/net/ionic/ionic_dev.h
+++ b/drivers/net/ionic/ionic_dev.h
@@ -14,7 +14,7 @@
#define VLAN_TAG_SIZE 4
#define IONIC_MIN_MTU RTE_ETHER_MIN_MTU
-#define IONIC_MAX_MTU 9378
+#define IONIC_MAX_MTU 9750
#define IONIC_ETH_OVERHEAD (RTE_ETHER_HDR_LEN + VLAN_TAG_SIZE)
#define IONIC_MAX_RING_DESC 32768
--
2.17.1
^ permalink raw reply [flat|nested] 52+ messages in thread
* [PATCH 03/13] net/ionic: don't auto-enable Rx scatter-gather a second time
2024-02-02 19:32 [PATCH 00/13] net/ionic: miscellaneous fixes and improvements Andrew Boyer
2024-02-02 19:32 ` [PATCH 01/13] net/ionic: add stat for completion queue entries processed Andrew Boyer
2024-02-02 19:32 ` [PATCH 02/13] net/ionic: increase max supported MTU to 9750 bytes Andrew Boyer
@ 2024-02-02 19:32 ` Andrew Boyer
2024-02-02 19:32 ` [PATCH 04/13] net/ionic: fix missing volatile type for cqe pointers Andrew Boyer
` (25 subsequent siblings)
28 siblings, 0 replies; 52+ messages in thread
From: Andrew Boyer @ 2024-02-02 19:32 UTC (permalink / raw)
To: dev; +Cc: Andrew Boyer
The receive side will enable scatter-gather if required based on the
mbuf size. If the client already enabled it in the config, it does
not need to be enabled again. This reduces log output.
Signed-off-by: Andrew Boyer <andrew.boyer@amd.com>
---
drivers/net/ionic/ionic_lif.c | 3 ++-
1 file changed, 2 insertions(+), 1 deletion(-)
diff --git a/drivers/net/ionic/ionic_lif.c b/drivers/net/ionic/ionic_lif.c
index 25b490deb6..fe2112c057 100644
--- a/drivers/net/ionic/ionic_lif.c
+++ b/drivers/net/ionic/ionic_lif.c
@@ -768,7 +768,8 @@ ionic_rx_qcq_alloc(struct ionic_lif *lif, uint32_t socket_id, uint32_t index,
max_mtu = rte_le_to_cpu_32(lif->adapter->ident.lif.eth.max_mtu);
/* If mbufs are too small to hold received packets, enable SG */
- if (max_mtu > hdr_seg_size) {
+ if (max_mtu > hdr_seg_size &&
+ !(lif->features & IONIC_ETH_HW_RX_SG)) {
IONIC_PRINT(NOTICE, "Enabling RX_OFFLOAD_SCATTER");
lif->eth_dev->data->dev_conf.rxmode.offloads |=
RTE_ETH_RX_OFFLOAD_SCATTER;
--
2.17.1
^ permalink raw reply [flat|nested] 52+ messages in thread
* [PATCH 04/13] net/ionic: fix missing volatile type for cqe pointers
2024-02-02 19:32 [PATCH 00/13] net/ionic: miscellaneous fixes and improvements Andrew Boyer
` (2 preceding siblings ...)
2024-02-02 19:32 ` [PATCH 03/13] net/ionic: don't auto-enable Rx scatter-gather a second time Andrew Boyer
@ 2024-02-02 19:32 ` Andrew Boyer
2024-02-03 4:26 ` Stephen Hemminger
2024-02-02 19:32 ` [PATCH 05/13] net/ionic: replace non-standard type in structure definition Andrew Boyer
` (24 subsequent siblings)
28 siblings, 1 reply; 52+ messages in thread
From: Andrew Boyer @ 2024-02-02 19:32 UTC (permalink / raw)
To: dev; +Cc: Neel Patel, stable, Andrew Boyer
From: Neel Patel <neel.patel@amd.com>
This memory may be changed by the hardware, so the volatile
keyword is required for correctness.
Fixes: e86a6fcc7cf3 ("net/ionic: add optimized non-scattered Rx/Tx")
cc: stable@dpdk.org
Signed-off-by: Andrew Boyer <andrew.boyer@amd.com>
Signed-off-by: Neel Patel <neel.patel@amd.com>
---
drivers/net/ionic/ionic_rxtx.c | 4 ++--
drivers/net/ionic/ionic_rxtx_sg.c | 8 +++++---
drivers/net/ionic/ionic_rxtx_simple.c | 8 +++++---
3 files changed, 12 insertions(+), 8 deletions(-)
diff --git a/drivers/net/ionic/ionic_rxtx.c b/drivers/net/ionic/ionic_rxtx.c
index d92b231f8f..d92fa1cca7 100644
--- a/drivers/net/ionic/ionic_rxtx.c
+++ b/drivers/net/ionic/ionic_rxtx.c
@@ -755,7 +755,7 @@ ionic_dev_rx_descriptor_status(void *rx_queue, uint16_t offset)
{
struct ionic_rx_qcq *rxq = rx_queue;
struct ionic_qcq *qcq = &rxq->qcq;
- struct ionic_rxq_comp *cq_desc;
+ volatile struct ionic_rxq_comp *cq_desc;
uint16_t mask, head, tail, pos;
bool done_color;
@@ -794,7 +794,7 @@ ionic_dev_tx_descriptor_status(void *tx_queue, uint16_t offset)
{
struct ionic_tx_qcq *txq = tx_queue;
struct ionic_qcq *qcq = &txq->qcq;
- struct ionic_txq_comp *cq_desc;
+ volatile struct ionic_txq_comp *cq_desc;
uint16_t mask, head, tail, pos, cq_pos;
bool done_color;
diff --git a/drivers/net/ionic/ionic_rxtx_sg.c b/drivers/net/ionic/ionic_rxtx_sg.c
index 6c028a698c..1392342463 100644
--- a/drivers/net/ionic/ionic_rxtx_sg.c
+++ b/drivers/net/ionic/ionic_rxtx_sg.c
@@ -28,7 +28,8 @@ ionic_tx_flush_sg(struct ionic_tx_qcq *txq)
struct ionic_queue *q = &txq->qcq.q;
struct ionic_tx_stats *stats = &txq->stats;
struct rte_mbuf *txm;
- struct ionic_txq_comp *cq_desc, *cq_desc_base = cq->base;
+ struct ionic_txq_comp *cq_desc_base = cq->base;
+ volatile struct ionic_txq_comp *cq_desc;
void **info;
uint32_t i;
@@ -254,7 +255,7 @@ ionic_xmit_pkts_sg(void *tx_queue, struct rte_mbuf **tx_pkts,
*/
static __rte_always_inline void
ionic_rx_clean_one_sg(struct ionic_rx_qcq *rxq,
- struct ionic_rxq_comp *cq_desc,
+ volatile struct ionic_rxq_comp *cq_desc,
struct ionic_rx_service *rx_svc)
{
struct ionic_queue *q = &rxq->qcq.q;
@@ -440,7 +441,8 @@ ionic_rxq_service_sg(struct ionic_rx_qcq *rxq, uint32_t work_to_do,
struct ionic_cq *cq = &rxq->qcq.cq;
struct ionic_queue *q = &rxq->qcq.q;
struct ionic_rxq_desc *q_desc_base = q->base;
- struct ionic_rxq_comp *cq_desc, *cq_desc_base = cq->base;
+ struct ionic_rxq_comp *cq_desc_base = cq->base;
+ volatile struct ionic_rxq_comp *cq_desc;
uint32_t work_done = 0;
uint64_t then, now, hz, delta;
diff --git a/drivers/net/ionic/ionic_rxtx_simple.c b/drivers/net/ionic/ionic_rxtx_simple.c
index 5969287b66..00152c885a 100644
--- a/drivers/net/ionic/ionic_rxtx_simple.c
+++ b/drivers/net/ionic/ionic_rxtx_simple.c
@@ -28,7 +28,8 @@ ionic_tx_flush(struct ionic_tx_qcq *txq)
struct ionic_queue *q = &txq->qcq.q;
struct ionic_tx_stats *stats = &txq->stats;
struct rte_mbuf *txm;
- struct ionic_txq_comp *cq_desc, *cq_desc_base = cq->base;
+ struct ionic_txq_comp *cq_desc_base = cq->base;
+ volatile struct ionic_txq_comp *cq_desc;
void **info;
cq_desc = &cq_desc_base[cq->tail_idx];
@@ -227,7 +228,7 @@ ionic_xmit_pkts(void *tx_queue, struct rte_mbuf **tx_pkts,
*/
static __rte_always_inline void
ionic_rx_clean_one(struct ionic_rx_qcq *rxq,
- struct ionic_rxq_comp *cq_desc,
+ volatile struct ionic_rxq_comp *cq_desc,
struct ionic_rx_service *rx_svc)
{
struct ionic_queue *q = &rxq->qcq.q;
@@ -361,7 +362,8 @@ ionic_rxq_service(struct ionic_rx_qcq *rxq, uint32_t work_to_do,
struct ionic_cq *cq = &rxq->qcq.cq;
struct ionic_queue *q = &rxq->qcq.q;
struct ionic_rxq_desc *q_desc_base = q->base;
- struct ionic_rxq_comp *cq_desc, *cq_desc_base = cq->base;
+ struct ionic_rxq_comp *cq_desc_base = cq->base;
+ volatile struct ionic_rxq_comp *cq_desc;
uint32_t work_done = 0;
uint64_t then, now, hz, delta;
--
2.17.1
^ permalink raw reply [flat|nested] 52+ messages in thread
* [PATCH 05/13] net/ionic: replace non-standard type in structure definition
2024-02-02 19:32 [PATCH 00/13] net/ionic: miscellaneous fixes and improvements Andrew Boyer
` (3 preceding siblings ...)
2024-02-02 19:32 ` [PATCH 04/13] net/ionic: fix missing volatile type for cqe pointers Andrew Boyer
@ 2024-02-02 19:32 ` Andrew Boyer
2024-02-02 19:32 ` [PATCH 06/13] net/ionic: memcpy descriptors when using Q-in-CMB Andrew Boyer
` (23 subsequent siblings)
28 siblings, 0 replies; 52+ messages in thread
From: Andrew Boyer @ 2024-02-02 19:32 UTC (permalink / raw)
To: dev; +Cc: Andrew Boyer
Use uint8_t instead of u_char. This simplifies the code.
Signed-off-by: Andrew Boyer <andrew.boyer@amd.com>
---
drivers/net/ionic/ionic_dev_pci.c | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/drivers/net/ionic/ionic_dev_pci.c b/drivers/net/ionic/ionic_dev_pci.c
index 5e74a6da71..cbaac2c5bc 100644
--- a/drivers/net/ionic/ionic_dev_pci.c
+++ b/drivers/net/ionic/ionic_dev_pci.c
@@ -38,7 +38,7 @@ ionic_pci_setup(struct ionic_adapter *adapter)
struct ionic_dev *idev = &adapter->idev;
struct rte_pci_device *bus_dev = adapter->bus_dev;
uint32_t sig;
- u_char *bar0_base;
+ uint8_t *bar0_base;
unsigned int i;
/* BAR0: dev_cmd and interrupts */
--
2.17.1
^ permalink raw reply [flat|nested] 52+ messages in thread
* [PATCH 06/13] net/ionic: memcpy descriptors when using Q-in-CMB
2024-02-02 19:32 [PATCH 00/13] net/ionic: miscellaneous fixes and improvements Andrew Boyer
` (4 preceding siblings ...)
2024-02-02 19:32 ` [PATCH 05/13] net/ionic: replace non-standard type in structure definition Andrew Boyer
@ 2024-02-02 19:32 ` Andrew Boyer
2024-02-02 19:32 ` [PATCH 07/13] net/ionic: fix RSS query routine Andrew Boyer
` (22 subsequent siblings)
28 siblings, 0 replies; 52+ messages in thread
From: Andrew Boyer @ 2024-02-02 19:32 UTC (permalink / raw)
To: dev; +Cc: Neel Patel, Andrew Boyer
From: Neel Patel <neel.patel@amd.com>
They can be batched together this way, reducing the number
of PCIe transactions. This improves transmit PPS by up to 50% in
some configurations.
Signed-off-by: Andrew Boyer <andrew.boyer@amd.com>
Signed-off-by: Neel Patel <neel.patel@amd.com>
---
drivers/net/ionic/ionic_dev.c | 9 +++--
drivers/net/ionic/ionic_dev.h | 6 ++-
drivers/net/ionic/ionic_lif.c | 26 +++++++++----
drivers/net/ionic/ionic_rxtx.h | 56 +++++++++++++++++++++++++++
drivers/net/ionic/ionic_rxtx_sg.c | 18 ++++-----
drivers/net/ionic/ionic_rxtx_simple.c | 18 ++++-----
6 files changed, 101 insertions(+), 32 deletions(-)
diff --git a/drivers/net/ionic/ionic_dev.c b/drivers/net/ionic/ionic_dev.c
index 70c14882ed..7f15914f74 100644
--- a/drivers/net/ionic/ionic_dev.c
+++ b/drivers/net/ionic/ionic_dev.c
@@ -369,17 +369,19 @@ ionic_q_init(struct ionic_queue *q, uint32_t index, uint16_t num_descs)
q->index = index;
q->num_descs = num_descs;
q->size_mask = num_descs - 1;
- q->head_idx = 0;
- q->tail_idx = 0;
+ ionic_q_reset(q);
return 0;
}
void
-ionic_q_map(struct ionic_queue *q, void *base, rte_iova_t base_pa)
+ionic_q_map(struct ionic_queue *q, void *base, rte_iova_t base_pa,
+ void *cmb_base, rte_iova_t cmb_base_pa)
{
q->base = base;
q->base_pa = base_pa;
+ q->cmb_base = cmb_base;
+ q->cmb_base_pa = cmb_base_pa;
}
void
@@ -393,5 +395,6 @@ void
ionic_q_reset(struct ionic_queue *q)
{
q->head_idx = 0;
+ q->cmb_head_idx = 0;
q->tail_idx = 0;
}
diff --git a/drivers/net/ionic/ionic_dev.h b/drivers/net/ionic/ionic_dev.h
index 971c261b27..3a366247f1 100644
--- a/drivers/net/ionic/ionic_dev.h
+++ b/drivers/net/ionic/ionic_dev.h
@@ -145,11 +145,13 @@ struct ionic_queue {
uint16_t num_descs;
uint16_t num_segs;
uint16_t head_idx;
+ uint16_t cmb_head_idx;
uint16_t tail_idx;
uint16_t size_mask;
uint8_t type;
uint8_t hw_type;
void *base;
+ void *cmb_base;
void *sg_base;
struct ionic_doorbell __iomem *db;
void **info;
@@ -158,6 +160,7 @@ struct ionic_queue {
uint32_t hw_index;
rte_iova_t base_pa;
rte_iova_t sg_base_pa;
+ rte_iova_t cmb_base_pa;
};
#define IONIC_INTR_NONE (-1)
@@ -244,7 +247,8 @@ uint32_t ionic_cq_service(struct ionic_cq *cq, uint32_t work_to_do,
int ionic_q_init(struct ionic_queue *q, uint32_t index, uint16_t num_descs);
void ionic_q_reset(struct ionic_queue *q);
-void ionic_q_map(struct ionic_queue *q, void *base, rte_iova_t base_pa);
+void ionic_q_map(struct ionic_queue *q, void *base, rte_iova_t base_pa,
+ void *cmb_base, rte_iova_t cmb_base_pa);
void ionic_q_sg_map(struct ionic_queue *q, void *base, rte_iova_t base_pa);
static inline uint16_t
diff --git a/drivers/net/ionic/ionic_lif.c b/drivers/net/ionic/ionic_lif.c
index fe2112c057..2713f8aa24 100644
--- a/drivers/net/ionic/ionic_lif.c
+++ b/drivers/net/ionic/ionic_lif.c
@@ -572,10 +572,11 @@ ionic_qcq_alloc(struct ionic_lif *lif,
{
struct ionic_qcq *new;
uint32_t q_size, cq_size, sg_size, total_size;
- void *q_base, *cq_base, *sg_base;
+ void *q_base, *cmb_q_base, *cq_base, *sg_base;
rte_iova_t q_base_pa = 0;
rte_iova_t cq_base_pa = 0;
rte_iova_t sg_base_pa = 0;
+ rte_iova_t cmb_q_base_pa = 0;
size_t page_size = rte_mem_page_size();
int err;
@@ -666,19 +667,22 @@ ionic_qcq_alloc(struct ionic_lif *lif,
IONIC_PRINT(ERR, "Cannot reserve queue from NIC mem");
return -ENOMEM;
}
- q_base = (void *)
+ cmb_q_base = (void *)
((uintptr_t)lif->adapter->bars.bar[2].vaddr +
(uintptr_t)lif->adapter->cmb_offset);
/* CMB PA is a relative address */
- q_base_pa = lif->adapter->cmb_offset;
+ cmb_q_base_pa = lif->adapter->cmb_offset;
lif->adapter->cmb_offset += q_size;
+ } else {
+ cmb_q_base = NULL;
+ cmb_q_base_pa = 0;
}
IONIC_PRINT(DEBUG, "Q-Base-PA = %#jx CQ-Base-PA = %#jx "
"SG-base-PA = %#jx",
q_base_pa, cq_base_pa, sg_base_pa);
- ionic_q_map(&new->q, q_base, q_base_pa);
+ ionic_q_map(&new->q, q_base, q_base_pa, cmb_q_base, cmb_q_base_pa);
ionic_cq_map(&new->cq, cq_base, cq_base_pa);
*qcq = new;
@@ -1583,7 +1587,6 @@ ionic_lif_txq_init(struct ionic_tx_qcq *txq)
.flags = rte_cpu_to_le_16(IONIC_QINIT_F_ENA),
.intr_index = rte_cpu_to_le_16(IONIC_INTR_NONE),
.ring_size = rte_log2_u32(q->num_descs),
- .ring_base = rte_cpu_to_le_64(q->base_pa),
.cq_ring_base = rte_cpu_to_le_64(cq->base_pa),
.sg_ring_base = rte_cpu_to_le_64(q->sg_base_pa),
},
@@ -1592,8 +1595,12 @@ ionic_lif_txq_init(struct ionic_tx_qcq *txq)
if (txq->flags & IONIC_QCQ_F_SG)
ctx.cmd.q_init.flags |= rte_cpu_to_le_16(IONIC_QINIT_F_SG);
- if (txq->flags & IONIC_QCQ_F_CMB)
+ if (txq->flags & IONIC_QCQ_F_CMB) {
ctx.cmd.q_init.flags |= rte_cpu_to_le_16(IONIC_QINIT_F_CMB);
+ ctx.cmd.q_init.ring_base = rte_cpu_to_le_64(q->cmb_base_pa);
+ } else {
+ ctx.cmd.q_init.ring_base = rte_cpu_to_le_64(q->base_pa);
+ }
IONIC_PRINT(DEBUG, "txq_init.index %d", q->index);
IONIC_PRINT(DEBUG, "txq_init.ring_base 0x%" PRIx64 "", q->base_pa);
@@ -1638,7 +1645,6 @@ ionic_lif_rxq_init(struct ionic_rx_qcq *rxq)
.flags = rte_cpu_to_le_16(IONIC_QINIT_F_ENA),
.intr_index = rte_cpu_to_le_16(IONIC_INTR_NONE),
.ring_size = rte_log2_u32(q->num_descs),
- .ring_base = rte_cpu_to_le_64(q->base_pa),
.cq_ring_base = rte_cpu_to_le_64(cq->base_pa),
.sg_ring_base = rte_cpu_to_le_64(q->sg_base_pa),
},
@@ -1647,8 +1653,12 @@ ionic_lif_rxq_init(struct ionic_rx_qcq *rxq)
if (rxq->flags & IONIC_QCQ_F_SG)
ctx.cmd.q_init.flags |= rte_cpu_to_le_16(IONIC_QINIT_F_SG);
- if (rxq->flags & IONIC_QCQ_F_CMB)
+ if (rxq->flags & IONIC_QCQ_F_CMB) {
ctx.cmd.q_init.flags |= rte_cpu_to_le_16(IONIC_QINIT_F_CMB);
+ ctx.cmd.q_init.ring_base = rte_cpu_to_le_64(q->cmb_base_pa);
+ } else {
+ ctx.cmd.q_init.ring_base = rte_cpu_to_le_64(q->base_pa);
+ }
IONIC_PRINT(DEBUG, "rxq_init.index %d", q->index);
IONIC_PRINT(DEBUG, "rxq_init.ring_base 0x%" PRIx64 "", q->base_pa);
diff --git a/drivers/net/ionic/ionic_rxtx.h b/drivers/net/ionic/ionic_rxtx.h
index 8537141597..5348395956 100644
--- a/drivers/net/ionic/ionic_rxtx.h
+++ b/drivers/net/ionic/ionic_rxtx.h
@@ -77,4 +77,60 @@ uint16_t ionic_xmit_pkts_sg(void *tx_queue, struct rte_mbuf **tx_pkts,
int ionic_rx_fill_sg(struct ionic_rx_qcq *rxq);
+static inline void
+ionic_rxq_flush(struct ionic_queue *q)
+{
+ struct ionic_rxq_desc *desc_base = q->base;
+ struct ionic_rxq_desc *cmb_desc_base = q->cmb_base;
+
+ if (q->cmb_base) {
+ if (q->head_idx < q->cmb_head_idx) {
+ /* copy [cmb_head, num_descs) */
+ rte_memcpy((void *)&cmb_desc_base[q->cmb_head_idx],
+ (void *)&desc_base[q->cmb_head_idx],
+ (q->num_descs - q->cmb_head_idx) * sizeof(*desc_base));
+ /* copy [0, head) */
+ rte_memcpy((void *)&cmb_desc_base[0],
+ (void *)&desc_base[0],
+ q->head_idx * sizeof(*desc_base));
+ } else {
+ /* copy [cmb_head, head) */
+ rte_memcpy((void *)&cmb_desc_base[q->cmb_head_idx],
+ (void *)&desc_base[q->cmb_head_idx],
+ (q->head_idx - q->cmb_head_idx) * sizeof(*desc_base));
+ }
+ q->cmb_head_idx = q->head_idx;
+ }
+
+ ionic_q_flush(q);
+}
+
+static inline void
+ionic_txq_flush(struct ionic_queue *q)
+{
+ struct ionic_txq_desc *desc_base = q->base;
+ struct ionic_txq_desc *cmb_desc_base = q->cmb_base;
+
+ if (q->cmb_base) {
+ if (q->head_idx < q->cmb_head_idx) {
+ /* copy [cmb_head, num_descs) */
+ rte_memcpy((void *)&cmb_desc_base[q->cmb_head_idx],
+ (void *)&desc_base[q->cmb_head_idx],
+ (q->num_descs - q->cmb_head_idx) * sizeof(*desc_base));
+ /* copy [0, head) */
+ rte_memcpy((void *)&cmb_desc_base[0],
+ (void *)&desc_base[0],
+ q->head_idx * sizeof(*desc_base));
+ } else {
+ /* copy [cmb_head, head) */
+ rte_memcpy((void *)&cmb_desc_base[q->cmb_head_idx],
+ (void *)&desc_base[q->cmb_head_idx],
+ (q->head_idx - q->cmb_head_idx) * sizeof(*desc_base));
+ }
+ q->cmb_head_idx = q->head_idx;
+ }
+
+ ionic_q_flush(q);
+}
+
#endif /* _IONIC_RXTX_H_ */
diff --git a/drivers/net/ionic/ionic_rxtx_sg.c b/drivers/net/ionic/ionic_rxtx_sg.c
index 1392342463..92e1d6e259 100644
--- a/drivers/net/ionic/ionic_rxtx_sg.c
+++ b/drivers/net/ionic/ionic_rxtx_sg.c
@@ -166,6 +166,7 @@ ionic_xmit_pkts_sg(void *tx_queue, struct rte_mbuf **tx_pkts,
{
struct ionic_tx_qcq *txq = tx_queue;
struct ionic_queue *q = &txq->qcq.q;
+ struct ionic_txq_desc *desc_base = q->base;
struct ionic_tx_stats *stats = &txq->stats;
struct rte_mbuf *mbuf;
uint32_t bytes_tx = 0;
@@ -173,9 +174,7 @@ ionic_xmit_pkts_sg(void *tx_queue, struct rte_mbuf **tx_pkts,
uint64_t then, now, hz, delta;
int err;
- struct ionic_txq_desc *desc_base = q->base;
- if (!(txq->flags & IONIC_QCQ_F_CMB))
- rte_prefetch0(&desc_base[q->head_idx]);
+ rte_prefetch0(&desc_base[q->head_idx]);
rte_prefetch0(IONIC_INFO_PTR(q, q->head_idx));
if (nb_pkts) {
@@ -196,8 +195,7 @@ ionic_xmit_pkts_sg(void *tx_queue, struct rte_mbuf **tx_pkts,
while (nb_tx < nb_pkts) {
uint16_t next_idx = Q_NEXT_TO_POST(q, 1);
- if (!(txq->flags & IONIC_QCQ_F_CMB))
- rte_prefetch0(&desc_base[next_idx]);
+ rte_prefetch0(&desc_base[next_idx]);
rte_prefetch0(IONIC_INFO_PTR(q, next_idx));
if (nb_tx + 1 < nb_pkts) {
@@ -222,7 +220,7 @@ ionic_xmit_pkts_sg(void *tx_queue, struct rte_mbuf **tx_pkts,
if (nb_tx > 0) {
rte_wmb();
- ionic_q_flush(q);
+ ionic_txq_flush(q);
txq->last_wdog_cycles = rte_get_timer_cycles();
@@ -458,8 +456,7 @@ ionic_rxq_service_sg(struct ionic_rx_qcq *rxq, uint32_t work_to_do,
/* Prefetch 4 x 16B comp */
rte_prefetch0(&cq_desc_base[Q_NEXT_TO_SRVC(cq, 4)]);
/* Prefetch 4 x 16B descriptors */
- if (!(rxq->flags & IONIC_QCQ_F_CMB))
- rte_prefetch0(&q_desc_base[Q_NEXT_TO_POST(q, 4)]);
+ rte_prefetch0(&q_desc_base[Q_NEXT_TO_POST(q, 4)]);
/* Clean one descriptor */
ionic_rx_clean_one_sg(rxq, cq_desc, rx_svc);
@@ -478,7 +475,8 @@ ionic_rxq_service_sg(struct ionic_rx_qcq *rxq, uint32_t work_to_do,
/* Update the queue indices and ring the doorbell */
if (work_done) {
- ionic_q_flush(q);
+ ionic_rxq_flush(q);
+
rxq->last_wdog_cycles = rte_get_timer_cycles();
rxq->wdog_ms = IONIC_Q_WDOG_MS;
} else {
@@ -542,7 +540,7 @@ ionic_rx_fill_sg(struct ionic_rx_qcq *rxq)
q->head_idx = Q_NEXT_TO_POST(q, 1);
}
- ionic_q_flush(q);
+ ionic_rxq_flush(q);
return err;
}
diff --git a/drivers/net/ionic/ionic_rxtx_simple.c b/drivers/net/ionic/ionic_rxtx_simple.c
index 00152c885a..f12f66f40c 100644
--- a/drivers/net/ionic/ionic_rxtx_simple.c
+++ b/drivers/net/ionic/ionic_rxtx_simple.c
@@ -139,6 +139,7 @@ ionic_xmit_pkts(void *tx_queue, struct rte_mbuf **tx_pkts,
{
struct ionic_tx_qcq *txq = tx_queue;
struct ionic_queue *q = &txq->qcq.q;
+ struct ionic_txq_desc *desc_base = q->base;
struct ionic_tx_stats *stats = &txq->stats;
struct rte_mbuf *mbuf;
uint32_t bytes_tx = 0;
@@ -146,9 +147,7 @@ ionic_xmit_pkts(void *tx_queue, struct rte_mbuf **tx_pkts,
uint64_t then, now, hz, delta;
int err;
- struct ionic_txq_desc *desc_base = q->base;
- if (!(txq->flags & IONIC_QCQ_F_CMB))
- rte_prefetch0(&desc_base[q->head_idx]);
+ rte_prefetch0(&desc_base[q->head_idx]);
rte_prefetch0(&q->info[q->head_idx]);
if (nb_pkts) {
@@ -169,8 +168,7 @@ ionic_xmit_pkts(void *tx_queue, struct rte_mbuf **tx_pkts,
while (nb_tx < nb_pkts) {
uint16_t next_idx = Q_NEXT_TO_POST(q, 1);
- if (!(txq->flags & IONIC_QCQ_F_CMB))
- rte_prefetch0(&desc_base[next_idx]);
+ rte_prefetch0(&desc_base[next_idx]);
rte_prefetch0(&q->info[next_idx]);
if (nb_tx + 1 < nb_pkts) {
@@ -195,7 +193,7 @@ ionic_xmit_pkts(void *tx_queue, struct rte_mbuf **tx_pkts,
if (nb_tx > 0) {
rte_wmb();
- ionic_q_flush(q);
+ ionic_txq_flush(q);
txq->last_wdog_cycles = rte_get_timer_cycles();
@@ -379,8 +377,7 @@ ionic_rxq_service(struct ionic_rx_qcq *rxq, uint32_t work_to_do,
/* Prefetch 4 x 16B comp */
rte_prefetch0(&cq_desc_base[Q_NEXT_TO_SRVC(cq, 4)]);
/* Prefetch 4 x 16B descriptors */
- if (!(rxq->flags & IONIC_QCQ_F_CMB))
- rte_prefetch0(&q_desc_base[Q_NEXT_TO_POST(q, 4)]);
+ rte_prefetch0(&q_desc_base[Q_NEXT_TO_POST(q, 4)]);
/* Clean one descriptor */
ionic_rx_clean_one(rxq, cq_desc, rx_svc);
@@ -399,7 +396,8 @@ ionic_rxq_service(struct ionic_rx_qcq *rxq, uint32_t work_to_do,
/* Update the queue indices and ring the doorbell */
if (work_done) {
- ionic_q_flush(q);
+ ionic_rxq_flush(q);
+
rxq->last_wdog_cycles = rte_get_timer_cycles();
rxq->wdog_ms = IONIC_Q_WDOG_MS;
} else {
@@ -463,7 +461,7 @@ ionic_rx_fill(struct ionic_rx_qcq *rxq)
q->head_idx = Q_NEXT_TO_POST(q, 1);
}
- ionic_q_flush(q);
+ ionic_rxq_flush(q);
return err;
}
--
2.17.1
^ permalink raw reply [flat|nested] 52+ messages in thread
* [PATCH 07/13] net/ionic: fix RSS query routine
2024-02-02 19:32 [PATCH 00/13] net/ionic: miscellaneous fixes and improvements Andrew Boyer
` (5 preceding siblings ...)
2024-02-02 19:32 ` [PATCH 06/13] net/ionic: memcpy descriptors when using Q-in-CMB Andrew Boyer
@ 2024-02-02 19:32 ` Andrew Boyer
2024-02-02 19:32 ` [PATCH 08/13] net/ionic: report 1G and 200G link speeds when applicable Andrew Boyer
` (21 subsequent siblings)
28 siblings, 0 replies; 52+ messages in thread
From: Andrew Boyer @ 2024-02-02 19:32 UTC (permalink / raw)
To: dev; +Cc: Akshay Dorwat, cardigliano, stable, Andrew Boyer
From: Akshay Dorwat <akshay.dorwat@amd.com>
The routine that copies out the RSS config can't use memcpy() because
'reta_conf->reta' is an array of uint16_t while 'lif->rss_ind_tbl' is
an array of uint8_t. Instead, copy the values individually.
Fixes: 22e7171bc63b ("net/ionic: support RSS")
Cc: cardigliano@ntop.org
Cc: stable@dpdk.org
Signed-off-by: Akshay Dorwat <akshay.dorwat@amd.com>
Signed-off-by: Andrew Boyer <andrew.boyer@amd.com>
---
drivers/net/ionic/ionic_ethdev.c | 9 +++++----
1 file changed, 5 insertions(+), 4 deletions(-)
diff --git a/drivers/net/ionic/ionic_ethdev.c b/drivers/net/ionic/ionic_ethdev.c
index 340fd0cd59..008e50e0b9 100644
--- a/drivers/net/ionic/ionic_ethdev.c
+++ b/drivers/net/ionic/ionic_ethdev.c
@@ -561,7 +561,7 @@ ionic_dev_rss_reta_query(struct rte_eth_dev *eth_dev,
struct ionic_lif *lif = IONIC_ETH_DEV_TO_LIF(eth_dev);
struct ionic_adapter *adapter = lif->adapter;
struct ionic_identity *ident = &adapter->ident;
- int i, num;
+ int i, j, num;
uint16_t tbl_sz = rte_le_to_cpu_16(ident->lif.eth.rss_ind_tbl_sz);
IONIC_PRINT_CALL();
@@ -582,9 +582,10 @@ ionic_dev_rss_reta_query(struct rte_eth_dev *eth_dev,
num = reta_size / RTE_ETH_RETA_GROUP_SIZE;
for (i = 0; i < num; i++) {
- memcpy(reta_conf->reta,
- &lif->rss_ind_tbl[i * RTE_ETH_RETA_GROUP_SIZE],
- RTE_ETH_RETA_GROUP_SIZE);
+ for (j = 0; j < RTE_ETH_RETA_GROUP_SIZE; j++) {
+ reta_conf->reta[j] =
+ lif->rss_ind_tbl[(i * RTE_ETH_RETA_GROUP_SIZE) + j];
+ }
reta_conf++;
}
--
2.17.1
^ permalink raw reply [flat|nested] 52+ messages in thread
* [PATCH 08/13] net/ionic: report 1G and 200G link speeds when applicable
2024-02-02 19:32 [PATCH 00/13] net/ionic: miscellaneous fixes and improvements Andrew Boyer
` (6 preceding siblings ...)
2024-02-02 19:32 ` [PATCH 07/13] net/ionic: fix RSS query routine Andrew Boyer
@ 2024-02-02 19:32 ` Andrew Boyer
2024-02-02 19:32 ` [PATCH 09/13] net/ionic: add flexible firmware xstat counters Andrew Boyer
` (20 subsequent siblings)
28 siblings, 0 replies; 52+ messages in thread
From: Andrew Boyer @ 2024-02-02 19:32 UTC (permalink / raw)
To: dev; +Cc: Vamsi Krishna Atluri, Andrew Boyer
From: Vamsi Krishna Atluri <vamsi.atluri@amd.com>
The hardware supports these speeds, so we should report them
correctly.
Signed-off-by: Andrew Boyer <andrew.boyer@amd.com>
Signed-off-by: Vamsi Krishna Atluri <vamsi.atluri@amd.com>
---
drivers/net/ionic/ionic_ethdev.c | 6 ++++++
1 file changed, 6 insertions(+)
diff --git a/drivers/net/ionic/ionic_ethdev.c b/drivers/net/ionic/ionic_ethdev.c
index 008e50e0b9..327f6b9de5 100644
--- a/drivers/net/ionic/ionic_ethdev.c
+++ b/drivers/net/ionic/ionic_ethdev.c
@@ -285,6 +285,9 @@ ionic_dev_link_update(struct rte_eth_dev *eth_dev,
link.link_status = RTE_ETH_LINK_UP;
link.link_duplex = RTE_ETH_LINK_FULL_DUPLEX;
switch (adapter->link_speed) {
+ case 1000:
+ link.link_speed = RTE_ETH_SPEED_NUM_1G;
+ break;
case 10000:
link.link_speed = RTE_ETH_SPEED_NUM_10G;
break;
@@ -300,6 +303,9 @@ ionic_dev_link_update(struct rte_eth_dev *eth_dev,
case 100000:
link.link_speed = RTE_ETH_SPEED_NUM_100G;
break;
+ case 200000:
+ link.link_speed = RTE_ETH_SPEED_NUM_200G;
+ break;
default:
link.link_speed = RTE_ETH_SPEED_NUM_NONE;
break;
--
2.17.1
^ permalink raw reply [flat|nested] 52+ messages in thread
* [PATCH 09/13] net/ionic: add flexible firmware xstat counters
2024-02-02 19:32 [PATCH 00/13] net/ionic: miscellaneous fixes and improvements Andrew Boyer
` (7 preceding siblings ...)
2024-02-02 19:32 ` [PATCH 08/13] net/ionic: report 1G and 200G link speeds when applicable Andrew Boyer
@ 2024-02-02 19:32 ` Andrew Boyer
2024-02-02 19:32 ` [PATCH 10/13] net/ionic: fix device close sequence to avoid crash Andrew Boyer
` (19 subsequent siblings)
28 siblings, 0 replies; 52+ messages in thread
From: Andrew Boyer @ 2024-02-02 19:32 UTC (permalink / raw)
To: dev; +Cc: Brad Larson, Andrew Boyer
From: Brad Larson <bradley.larson@amd.com>
Assign 32 counters for flexible firmware events. These can be used as
per-port or per-queue counters in certain firmware configurations.
They are displayed as fw_flex_eventX in xstats.
Signed-off-by: Andrew Boyer <andrew.boyer@amd.com>
Signed-off-by: Brad Larson <bradley.larson@amd.com>
---
drivers/net/ionic/ionic_ethdev.c | 33 +++++++++++++++
drivers/net/ionic/ionic_if.h | 70 ++++++++++++++++----------------
2 files changed, 68 insertions(+), 35 deletions(-)
diff --git a/drivers/net/ionic/ionic_ethdev.c b/drivers/net/ionic/ionic_ethdev.c
index 327f6b9de5..7c55a26956 100644
--- a/drivers/net/ionic/ionic_ethdev.c
+++ b/drivers/net/ionic/ionic_ethdev.c
@@ -196,6 +196,39 @@ static const struct rte_ionic_xstats_name_off rte_ionic_xstats_strings[] = {
tx_desc_fetch_error)},
{"tx_desc_data_error", offsetof(struct ionic_lif_stats,
tx_desc_data_error)},
+ /* Flexible firmware events */
+ {"fw_flex_event1", offsetof(struct ionic_lif_stats, flex1)},
+ {"fw_flex_event2", offsetof(struct ionic_lif_stats, flex2)},
+ {"fw_flex_event3", offsetof(struct ionic_lif_stats, flex3)},
+ {"fw_flex_event4", offsetof(struct ionic_lif_stats, flex4)},
+ {"fw_flex_event5", offsetof(struct ionic_lif_stats, flex5)},
+ {"fw_flex_event6", offsetof(struct ionic_lif_stats, flex6)},
+ {"fw_flex_event7", offsetof(struct ionic_lif_stats, flex7)},
+ {"fw_flex_event8", offsetof(struct ionic_lif_stats, flex8)},
+ {"fw_flex_event9", offsetof(struct ionic_lif_stats, flex9)},
+ {"fw_flex_event10", offsetof(struct ionic_lif_stats, flex10)},
+ {"fw_flex_event11", offsetof(struct ionic_lif_stats, flex11)},
+ {"fw_flex_event12", offsetof(struct ionic_lif_stats, flex12)},
+ {"fw_flex_event13", offsetof(struct ionic_lif_stats, flex13)},
+ {"fw_flex_event14", offsetof(struct ionic_lif_stats, flex14)},
+ {"fw_flex_event15", offsetof(struct ionic_lif_stats, flex15)},
+ {"fw_flex_event16", offsetof(struct ionic_lif_stats, flex16)},
+ {"fw_flex_event17", offsetof(struct ionic_lif_stats, flex17)},
+ {"fw_flex_event18", offsetof(struct ionic_lif_stats, flex18)},
+ {"fw_flex_event19", offsetof(struct ionic_lif_stats, flex19)},
+ {"fw_flex_event20", offsetof(struct ionic_lif_stats, flex20)},
+ {"fw_flex_event21", offsetof(struct ionic_lif_stats, flex21)},
+ {"fw_flex_event22", offsetof(struct ionic_lif_stats, flex22)},
+ {"fw_flex_event23", offsetof(struct ionic_lif_stats, flex23)},
+ {"fw_flex_event24", offsetof(struct ionic_lif_stats, flex24)},
+ {"fw_flex_event25", offsetof(struct ionic_lif_stats, flex25)},
+ {"fw_flex_event26", offsetof(struct ionic_lif_stats, flex26)},
+ {"fw_flex_event27", offsetof(struct ionic_lif_stats, flex27)},
+ {"fw_flex_event28", offsetof(struct ionic_lif_stats, flex28)},
+ {"fw_flex_event29", offsetof(struct ionic_lif_stats, flex29)},
+ {"fw_flex_event30", offsetof(struct ionic_lif_stats, flex30)},
+ {"fw_flex_event31", offsetof(struct ionic_lif_stats, flex31)},
+ {"fw_flex_event32", offsetof(struct ionic_lif_stats, flex32)},
};
#define IONIC_NB_HW_STATS RTE_DIM(rte_ionic_xstats_strings)
diff --git a/drivers/net/ionic/ionic_if.h b/drivers/net/ionic/ionic_if.h
index 79aa196345..7ca604a7bb 100644
--- a/drivers/net/ionic/ionic_if.h
+++ b/drivers/net/ionic/ionic_if.h
@@ -2592,41 +2592,41 @@ struct ionic_lif_stats {
__le64 rsvd16;
__le64 rsvd17;
- __le64 rsvd18;
- __le64 rsvd19;
- __le64 rsvd20;
- __le64 rsvd21;
- __le64 rsvd22;
- __le64 rsvd23;
- __le64 rsvd24;
- __le64 rsvd25;
-
- __le64 rsvd26;
- __le64 rsvd27;
- __le64 rsvd28;
- __le64 rsvd29;
- __le64 rsvd30;
- __le64 rsvd31;
- __le64 rsvd32;
- __le64 rsvd33;
-
- __le64 rsvd34;
- __le64 rsvd35;
- __le64 rsvd36;
- __le64 rsvd37;
- __le64 rsvd38;
- __le64 rsvd39;
- __le64 rsvd40;
- __le64 rsvd41;
-
- __le64 rsvd42;
- __le64 rsvd43;
- __le64 rsvd44;
- __le64 rsvd45;
- __le64 rsvd46;
- __le64 rsvd47;
- __le64 rsvd48;
- __le64 rsvd49;
+ __le64 flex1;
+ __le64 flex2;
+ __le64 flex3;
+ __le64 flex4;
+ __le64 flex5;
+ __le64 flex6;
+ __le64 flex7;
+ __le64 flex8;
+
+ __le64 flex9;
+ __le64 flex10;
+ __le64 flex11;
+ __le64 flex12;
+ __le64 flex13;
+ __le64 flex14;
+ __le64 flex15;
+ __le64 flex16;
+
+ __le64 flex17;
+ __le64 flex18;
+ __le64 flex19;
+ __le64 flex20;
+ __le64 flex21;
+ __le64 flex22;
+ __le64 flex23;
+ __le64 flex24;
+
+ __le64 flex25;
+ __le64 flex26;
+ __le64 flex27;
+ __le64 flex28;
+ __le64 flex29;
+ __le64 flex30;
+ __le64 flex31;
+ __le64 flex32;
/* RDMA/ROCE REQ Error/Debugs (768 - 895) */
__le64 rdma_req_rx_pkt_seq_err;
--
2.17.1
^ permalink raw reply [flat|nested] 52+ messages in thread
* [PATCH 10/13] net/ionic: fix device close sequence to avoid crash
2024-02-02 19:32 [PATCH 00/13] net/ionic: miscellaneous fixes and improvements Andrew Boyer
` (8 preceding siblings ...)
2024-02-02 19:32 ` [PATCH 09/13] net/ionic: add flexible firmware xstat counters Andrew Boyer
@ 2024-02-02 19:32 ` Andrew Boyer
2024-02-02 19:32 ` [PATCH 11/13] net/ionic: optimize device close operation Andrew Boyer
` (18 subsequent siblings)
28 siblings, 0 replies; 52+ messages in thread
From: Andrew Boyer @ 2024-02-02 19:32 UTC (permalink / raw)
To: dev; +Cc: Andrew Boyer, stable
The close routine should release all resources, but not
call rte_eth_dev_destroy(). As written this code will call
rte_eth_dev_release_port() twice and segfault.
Instead, move rte_eth_dev_destroy() to the remove routine.
eth_ionic_dev_uninit() will call close if necessary.
Fixes: 175e4e7ed760 ("net/ionic: complete release on close")
Cc: stable@dpdk.org
Signed-off-by: Andrew Boyer <andrew.boyer@amd.com>
---
drivers/net/ionic/ionic_ethdev.c | 30 +++++++++++++++---------------
1 file changed, 15 insertions(+), 15 deletions(-)
diff --git a/drivers/net/ionic/ionic_ethdev.c b/drivers/net/ionic/ionic_ethdev.c
index 7c55a26956..bedcf958e2 100644
--- a/drivers/net/ionic/ionic_ethdev.c
+++ b/drivers/net/ionic/ionic_ethdev.c
@@ -1009,19 +1009,21 @@ ionic_dev_close(struct rte_eth_dev *eth_dev)
ionic_lif_stop(lif);
- ionic_lif_free_queues(lif);
-
IONIC_PRINT(NOTICE, "Removing device %s", eth_dev->device->name);
if (adapter->intf->unconfigure_intr)
(*adapter->intf->unconfigure_intr)(adapter);
- rte_eth_dev_destroy(eth_dev, eth_ionic_dev_uninit);
-
ionic_port_reset(adapter);
ionic_reset(adapter);
+
+ ionic_lif_free_queues(lif);
+ ionic_lif_deinit(lif);
+ ionic_lif_free(lif); /* Does not free LIF object */
+
if (adapter->intf->unmap_bars)
(*adapter->intf->unmap_bars)(adapter);
+ lif->adapter = NULL;
rte_free(adapter);
return 0;
@@ -1098,21 +1100,18 @@ eth_ionic_dev_init(struct rte_eth_dev *eth_dev, void *init_params)
static int
eth_ionic_dev_uninit(struct rte_eth_dev *eth_dev)
{
- struct ionic_lif *lif = IONIC_ETH_DEV_TO_LIF(eth_dev);
- struct ionic_adapter *adapter = lif->adapter;
-
IONIC_PRINT_CALL();
if (rte_eal_process_type() != RTE_PROC_PRIMARY)
return 0;
- adapter->lif = NULL;
-
- ionic_lif_deinit(lif);
- ionic_lif_free(lif);
+ if (eth_dev->state != RTE_ETH_DEV_UNUSED)
+ ionic_dev_close(eth_dev);
- if (!(lif->state & IONIC_LIF_F_FW_RESET))
- ionic_lif_reset(lif);
+ eth_dev->dev_ops = NULL;
+ eth_dev->rx_pkt_burst = NULL;
+ eth_dev->tx_pkt_burst = NULL;
+ eth_dev->tx_pkt_prepare = NULL;
return 0;
}
@@ -1267,17 +1266,18 @@ eth_ionic_dev_remove(struct rte_device *rte_dev)
{
char name[RTE_ETH_NAME_MAX_LEN];
struct rte_eth_dev *eth_dev;
+ int ret = 0;
/* Adapter lookup is using the eth_dev name */
snprintf(name, sizeof(name), "%s_lif", rte_dev->name);
eth_dev = rte_eth_dev_allocated(name);
if (eth_dev)
- ionic_dev_close(eth_dev);
+ ret = rte_eth_dev_destroy(eth_dev, eth_ionic_dev_uninit);
else
IONIC_PRINT(DEBUG, "Cannot find device %s", rte_dev->name);
- return 0;
+ return ret;
}
RTE_LOG_REGISTER_DEFAULT(ionic_logtype, NOTICE);
--
2.17.1
^ permalink raw reply [flat|nested] 52+ messages in thread
* [PATCH 11/13] net/ionic: optimize device close operation
2024-02-02 19:32 [PATCH 00/13] net/ionic: miscellaneous fixes and improvements Andrew Boyer
` (9 preceding siblings ...)
2024-02-02 19:32 ` [PATCH 10/13] net/ionic: fix device close sequence to avoid crash Andrew Boyer
@ 2024-02-02 19:32 ` Andrew Boyer
2024-02-02 19:32 ` [PATCH 12/13] net/ionic: optimize device stop operation Andrew Boyer
` (17 subsequent siblings)
28 siblings, 0 replies; 52+ messages in thread
From: Andrew Boyer @ 2024-02-02 19:32 UTC (permalink / raw)
To: dev; +Cc: Andrew Boyer
Use a single device reset command to speed up dev_close(). The LIF stop
and port reset commands are not needed.
This reduces the outage window when restarting the process by about 2ms
plus another 1ms per queue.
Signed-off-by: Andrew Boyer <andrew.boyer@amd.com>
---
drivers/net/ionic/ionic_ethdev.c | 3 ---
drivers/net/ionic/ionic_lif.c | 8 +-------
2 files changed, 1 insertion(+), 10 deletions(-)
diff --git a/drivers/net/ionic/ionic_ethdev.c b/drivers/net/ionic/ionic_ethdev.c
index bedcf958e2..7e80751846 100644
--- a/drivers/net/ionic/ionic_ethdev.c
+++ b/drivers/net/ionic/ionic_ethdev.c
@@ -1007,13 +1007,10 @@ ionic_dev_close(struct rte_eth_dev *eth_dev)
if (rte_eal_process_type() != RTE_PROC_PRIMARY)
return 0;
- ionic_lif_stop(lif);
-
IONIC_PRINT(NOTICE, "Removing device %s", eth_dev->device->name);
if (adapter->intf->unconfigure_intr)
(*adapter->intf->unconfigure_intr)(adapter);
- ionic_port_reset(adapter);
ionic_reset(adapter);
ionic_lif_free_queues(lif);
diff --git a/drivers/net/ionic/ionic_lif.c b/drivers/net/ionic/ionic_lif.c
index 2713f8aa24..90efcc8cbb 100644
--- a/drivers/net/ionic/ionic_lif.c
+++ b/drivers/net/ionic/ionic_lif.c
@@ -1231,13 +1231,7 @@ ionic_lif_rss_setup(struct ionic_lif *lif)
static void
ionic_lif_rss_teardown(struct ionic_lif *lif)
{
- if (!lif->rss_ind_tbl)
- return;
-
- if (lif->rss_ind_tbl_z) {
- /* Disable RSS on the NIC */
- ionic_lif_rss_config(lif, 0x0, NULL, NULL);
-
+ if (lif->rss_ind_tbl) {
lif->rss_ind_tbl = NULL;
lif->rss_ind_tbl_pa = 0;
rte_memzone_free(lif->rss_ind_tbl_z);
--
2.17.1
^ permalink raw reply [flat|nested] 52+ messages in thread
* [PATCH 12/13] net/ionic: optimize device stop operation
2024-02-02 19:32 [PATCH 00/13] net/ionic: miscellaneous fixes and improvements Andrew Boyer
` (10 preceding siblings ...)
2024-02-02 19:32 ` [PATCH 11/13] net/ionic: optimize device close operation Andrew Boyer
@ 2024-02-02 19:32 ` Andrew Boyer
2024-02-02 19:32 ` [PATCH 13/13] net/ionic: optimize device start operation Andrew Boyer
` (16 subsequent siblings)
28 siblings, 0 replies; 52+ messages in thread
From: Andrew Boyer @ 2024-02-02 19:32 UTC (permalink / raw)
To: dev; +Cc: Andrew Boyer
Split the queue_stop operation into first-half and second-half helpers.
Move the command context from the stack into each Rx/Tx queue struct.
Expose some needed adminq interfaces.
This allows us to batch up the queue commands during dev_stop(), reducing
the outage window when restarting the process by about 1ms per queue.
Signed-off-by: Andrew Boyer <andrew.boyer@amd.com>
---
drivers/net/ionic/ionic.h | 3 ++
drivers/net/ionic/ionic_lif.c | 81 ++++++++++++++++++++++++----------
drivers/net/ionic/ionic_lif.h | 12 +++--
drivers/net/ionic/ionic_main.c | 17 ++++++-
drivers/net/ionic/ionic_rxtx.c | 78 +++++++++++++++++++-------------
drivers/net/ionic/ionic_rxtx.h | 14 +++++-
6 files changed, 143 insertions(+), 62 deletions(-)
diff --git a/drivers/net/ionic/ionic.h b/drivers/net/ionic/ionic.h
index c479eaba74..cb4ea450a9 100644
--- a/drivers/net/ionic/ionic.h
+++ b/drivers/net/ionic/ionic.h
@@ -83,7 +83,10 @@ struct ionic_admin_ctx {
union ionic_adminq_comp comp;
};
+int ionic_adminq_post(struct ionic_lif *lif, struct ionic_admin_ctx *ctx);
int ionic_adminq_post_wait(struct ionic_lif *lif, struct ionic_admin_ctx *ctx);
+int ionic_adminq_wait(struct ionic_lif *lif, struct ionic_admin_ctx *ctx);
+uint16_t ionic_adminq_space_avail(struct ionic_lif *lif);
int ionic_dev_cmd_wait_check(struct ionic_dev *idev, unsigned long max_wait);
int ionic_setup(struct ionic_adapter *adapter);
diff --git a/drivers/net/ionic/ionic_lif.c b/drivers/net/ionic/ionic_lif.c
index 90efcc8cbb..8ffdbc4df7 100644
--- a/drivers/net/ionic/ionic_lif.c
+++ b/drivers/net/ionic/ionic_lif.c
@@ -31,42 +31,54 @@ static int ionic_lif_addr_add(struct ionic_lif *lif, const uint8_t *addr);
static int ionic_lif_addr_del(struct ionic_lif *lif, const uint8_t *addr);
static int
-ionic_qcq_disable(struct ionic_qcq *qcq)
+ionic_qcq_disable_nowait(struct ionic_qcq *qcq,
+ struct ionic_admin_ctx *ctx)
{
+ int err;
+
struct ionic_queue *q = &qcq->q;
struct ionic_lif *lif = qcq->lif;
- struct ionic_admin_ctx ctx = {
- .pending_work = true,
- .cmd.q_control = {
- .opcode = IONIC_CMD_Q_CONTROL,
- .type = q->type,
- .index = rte_cpu_to_le_32(q->index),
- .oper = IONIC_Q_DISABLE,
- },
- };
- return ionic_adminq_post_wait(lif, &ctx);
+ memset(ctx, 0, sizeof(*ctx));
+ ctx->pending_work = true;
+ ctx->cmd.q_control.opcode = IONIC_CMD_Q_CONTROL;
+ ctx->cmd.q_control.type = q->type;
+ ctx->cmd.q_control.index = rte_cpu_to_le_32(q->index);
+ ctx->cmd.q_control.oper = IONIC_Q_DISABLE;
+
+ /* Does not wait for command completion */
+ err = ionic_adminq_post(lif, ctx);
+ if (err)
+ ctx->pending_work = false;
+ return err;
}
void
ionic_lif_stop(struct ionic_lif *lif)
{
- uint32_t i;
+ struct rte_eth_dev *dev = lif->eth_dev;
+ uint32_t i, j, chunk;
IONIC_PRINT_CALL();
lif->state &= ~IONIC_LIF_F_UP;
- for (i = 0; i < lif->nrxqcqs; i++) {
- struct ionic_rx_qcq *rxq = lif->rxqcqs[i];
- if (rxq->flags & IONIC_QCQ_F_INITED)
- (void)ionic_dev_rx_queue_stop(lif->eth_dev, i);
+ chunk = ionic_adminq_space_avail(lif);
+
+ for (i = 0; i < lif->nrxqcqs; i += chunk) {
+ for (j = 0; j < chunk && i + j < lif->nrxqcqs; j++)
+ ionic_dev_rx_queue_stop_firsthalf(dev, i + j);
+
+ for (j = 0; j < chunk && i + j < lif->nrxqcqs; j++)
+ ionic_dev_rx_queue_stop_secondhalf(dev, i + j);
}
- for (i = 0; i < lif->ntxqcqs; i++) {
- struct ionic_tx_qcq *txq = lif->txqcqs[i];
- if (txq->flags & IONIC_QCQ_F_INITED)
- (void)ionic_dev_tx_queue_stop(lif->eth_dev, i);
+ for (i = 0; i < lif->ntxqcqs; i += chunk) {
+ for (j = 0; j < chunk && i + j < lif->ntxqcqs; j++)
+ ionic_dev_tx_queue_stop_firsthalf(dev, i + j);
+
+ for (j = 0; j < chunk && i + j < lif->ntxqcqs; j++)
+ ionic_dev_tx_queue_stop_secondhalf(dev, i + j);
}
}
@@ -1240,21 +1252,42 @@ ionic_lif_rss_teardown(struct ionic_lif *lif)
}
void
-ionic_lif_txq_deinit(struct ionic_tx_qcq *txq)
+ionic_lif_txq_deinit_nowait(struct ionic_tx_qcq *txq)
{
- ionic_qcq_disable(&txq->qcq);
+ ionic_qcq_disable_nowait(&txq->qcq, &txq->admin_ctx);
txq->flags &= ~IONIC_QCQ_F_INITED;
}
void
-ionic_lif_rxq_deinit(struct ionic_rx_qcq *rxq)
+ionic_lif_txq_stats(struct ionic_tx_qcq *txq)
{
- ionic_qcq_disable(&rxq->qcq);
+ struct ionic_tx_stats *stats = &txq->stats;
+
+ IONIC_PRINT(DEBUG, "TX queue %u pkts %ju tso %ju",
+ txq->qcq.q.index, stats->packets, stats->tso);
+ IONIC_PRINT(DEBUG, "TX queue %u comps %ju (%ju per)",
+ txq->qcq.q.index, stats->comps,
+ stats->comps ? stats->packets / stats->comps : 0);
+}
+
+void
+ionic_lif_rxq_deinit_nowait(struct ionic_rx_qcq *rxq)
+{
+ ionic_qcq_disable_nowait(&rxq->qcq, &rxq->admin_ctx);
rxq->flags &= ~IONIC_QCQ_F_INITED;
}
+void
+ionic_lif_rxq_stats(struct ionic_rx_qcq *rxq)
+{
+ struct ionic_rx_stats *stats = &rxq->stats;
+
+ IONIC_PRINT(DEBUG, "RX queue %u pkts %ju mtod %ju",
+ rxq->qcq.q.index, stats->packets, stats->mtods);
+}
+
static void
ionic_lif_adminq_deinit(struct ionic_lif *lif)
{
diff --git a/drivers/net/ionic/ionic_lif.h b/drivers/net/ionic/ionic_lif.h
index cac7a4583b..ee13f5b7c8 100644
--- a/drivers/net/ionic/ionic_lif.h
+++ b/drivers/net/ionic/ionic_lif.h
@@ -10,7 +10,7 @@
#include <rte_ethdev.h>
#include <rte_ether.h>
-#include "ionic_osdep.h"
+#include "ionic.h"
#include "ionic_dev.h"
#include "ionic_rx_filter.h"
@@ -99,6 +99,8 @@ struct ionic_rx_qcq {
/* cacheline4+ */
struct rte_mbuf *mbs[IONIC_MBUF_BULK_ALLOC] __rte_cache_aligned;
+
+ struct ionic_admin_ctx admin_ctx;
};
struct ionic_tx_qcq {
@@ -112,6 +114,8 @@ struct ionic_tx_qcq {
uint16_t flags;
struct ionic_tx_stats stats;
+
+ struct ionic_admin_ctx admin_ctx;
};
#define IONIC_Q_TO_QCQ(_q) container_of(_q, struct ionic_qcq, q)
@@ -225,10 +229,12 @@ int ionic_tx_qcq_alloc(struct ionic_lif *lif, uint32_t socket_id,
void ionic_qcq_free(struct ionic_qcq *qcq);
int ionic_lif_rxq_init(struct ionic_rx_qcq *rxq);
-void ionic_lif_rxq_deinit(struct ionic_rx_qcq *rxq);
+void ionic_lif_rxq_deinit_nowait(struct ionic_rx_qcq *rxq);
+void ionic_lif_rxq_stats(struct ionic_rx_qcq *rxq);
int ionic_lif_txq_init(struct ionic_tx_qcq *txq);
-void ionic_lif_txq_deinit(struct ionic_tx_qcq *txq);
+void ionic_lif_txq_deinit_nowait(struct ionic_tx_qcq *txq);
+void ionic_lif_txq_stats(struct ionic_tx_qcq *txq);
int ionic_lif_rss_config(struct ionic_lif *lif, const uint16_t types,
const uint8_t *key, const uint32_t *indir);
diff --git a/drivers/net/ionic/ionic_main.c b/drivers/net/ionic/ionic_main.c
index c957d55bf9..1f24f64a33 100644
--- a/drivers/net/ionic/ionic_main.c
+++ b/drivers/net/ionic/ionic_main.c
@@ -180,6 +180,12 @@ ionic_adminq_service(struct ionic_cq *cq, uint16_t cq_desc_index,
return true;
}
+uint16_t
+ionic_adminq_space_avail(struct ionic_lif *lif)
+{
+ return ionic_q_space_avail(&lif->adminqcq->qcq.q);
+}
+
/** ionic_adminq_post - Post an admin command.
* @lif: Handle to lif.
* @cmd_ctx: Api admin command context.
@@ -191,7 +197,7 @@ ionic_adminq_service(struct ionic_cq *cq, uint16_t cq_desc_index,
*
* Return: zero or negative error status.
*/
-static int
+int
ionic_adminq_post(struct ionic_lif *lif, struct ionic_admin_ctx *ctx)
{
struct ionic_queue *q = &lif->adminqcq->qcq.q;
@@ -279,7 +285,6 @@ ionic_adminq_wait_for_completion(struct ionic_lif *lif,
int
ionic_adminq_post_wait(struct ionic_lif *lif, struct ionic_admin_ctx *ctx)
{
- bool done;
int err;
IONIC_PRINT(DEBUG, "Sending %s (%d) via the admin queue",
@@ -292,6 +297,14 @@ ionic_adminq_post_wait(struct ionic_lif *lif, struct ionic_admin_ctx *ctx)
return err;
}
+ return ionic_adminq_wait(lif, ctx);
+}
+
+int
+ionic_adminq_wait(struct ionic_lif *lif, struct ionic_admin_ctx *ctx)
+{
+ bool done;
+
done = ionic_adminq_wait_for_completion(lif, ctx,
IONIC_DEVCMD_TIMEOUT);
diff --git a/drivers/net/ionic/ionic_rxtx.c b/drivers/net/ionic/ionic_rxtx.c
index d92fa1cca7..774dc596c0 100644
--- a/drivers/net/ionic/ionic_rxtx.c
+++ b/drivers/net/ionic/ionic_rxtx.c
@@ -92,36 +92,40 @@ ionic_dev_tx_queue_release(struct rte_eth_dev *dev, uint16_t qid)
}
int __rte_cold
-ionic_dev_tx_queue_stop(struct rte_eth_dev *eth_dev, uint16_t tx_queue_id)
+ionic_dev_tx_queue_stop(struct rte_eth_dev *dev, uint16_t tx_queue_id)
{
- struct ionic_tx_stats *stats;
- struct ionic_tx_qcq *txq;
+ ionic_dev_tx_queue_stop_firsthalf(dev, tx_queue_id);
+ ionic_dev_tx_queue_stop_secondhalf(dev, tx_queue_id);
+
+ return 0;
+}
+
+void __rte_cold
+ionic_dev_tx_queue_stop_firsthalf(struct rte_eth_dev *dev,
+ uint16_t tx_queue_id)
+{
+ struct ionic_tx_qcq *txq = dev->data->tx_queues[tx_queue_id];
IONIC_PRINT(DEBUG, "Stopping TX queue %u", tx_queue_id);
- txq = eth_dev->data->tx_queues[tx_queue_id];
+ dev->data->tx_queue_state[tx_queue_id] = RTE_ETH_QUEUE_STATE_STOPPED;
- eth_dev->data->tx_queue_state[tx_queue_id] =
- RTE_ETH_QUEUE_STATE_STOPPED;
+ ionic_lif_txq_deinit_nowait(txq);
+}
- /*
- * Note: we should better post NOP Tx desc and wait for its completion
- * before disabling Tx queue
- */
+void __rte_cold
+ionic_dev_tx_queue_stop_secondhalf(struct rte_eth_dev *dev,
+ uint16_t tx_queue_id)
+{
+ struct ionic_lif *lif = IONIC_ETH_DEV_TO_LIF(dev);
+ struct ionic_tx_qcq *txq = dev->data->tx_queues[tx_queue_id];
- ionic_lif_txq_deinit(txq);
+ ionic_adminq_wait(lif, &txq->admin_ctx);
/* Free all buffers from descriptor ring */
ionic_tx_empty(txq);
- stats = &txq->stats;
- IONIC_PRINT(DEBUG, "TX queue %u pkts %ju tso %ju",
- txq->qcq.q.index, stats->packets, stats->tso);
- IONIC_PRINT(DEBUG, "TX queue %u comps %ju (%ju per)",
- txq->qcq.q.index, stats->comps,
- stats->comps ? stats->packets / stats->comps : 0);
-
- return 0;
+ ionic_lif_txq_stats(txq);
}
int __rte_cold
@@ -726,28 +730,40 @@ ionic_dev_rx_queue_start(struct rte_eth_dev *eth_dev, uint16_t rx_queue_id)
* Stop Receive Units for specified queue.
*/
int __rte_cold
-ionic_dev_rx_queue_stop(struct rte_eth_dev *eth_dev, uint16_t rx_queue_id)
+ionic_dev_rx_queue_stop(struct rte_eth_dev *dev, uint16_t rx_queue_id)
{
- uint8_t *rx_queue_state = eth_dev->data->rx_queue_state;
- struct ionic_rx_stats *stats;
- struct ionic_rx_qcq *rxq;
+ ionic_dev_rx_queue_stop_firsthalf(dev, rx_queue_id);
+ ionic_dev_rx_queue_stop_secondhalf(dev, rx_queue_id);
+
+ return 0;
+}
+
+void __rte_cold
+ionic_dev_rx_queue_stop_firsthalf(struct rte_eth_dev *dev,
+ uint16_t rx_queue_id)
+{
+ struct ionic_rx_qcq *rxq = dev->data->rx_queues[rx_queue_id];
IONIC_PRINT(DEBUG, "Stopping RX queue %u", rx_queue_id);
- rxq = eth_dev->data->rx_queues[rx_queue_id];
+ dev->data->rx_queue_state[rx_queue_id] = RTE_ETH_QUEUE_STATE_STOPPED;
+
+ ionic_lif_rxq_deinit_nowait(rxq);
+}
- rx_queue_state[rx_queue_id] = RTE_ETH_QUEUE_STATE_STOPPED;
+void __rte_cold
+ionic_dev_rx_queue_stop_secondhalf(struct rte_eth_dev *dev,
+ uint16_t rx_queue_id)
+{
+ struct ionic_lif *lif = IONIC_ETH_DEV_TO_LIF(dev);
+ struct ionic_rx_qcq *rxq = dev->data->rx_queues[rx_queue_id];
- ionic_lif_rxq_deinit(rxq);
+ ionic_adminq_wait(lif, &rxq->admin_ctx);
/* Free all buffers from descriptor ring */
ionic_rx_empty(rxq);
- stats = &rxq->stats;
- IONIC_PRINT(DEBUG, "RX queue %u pkts %ju mtod %ju",
- rxq->qcq.q.index, stats->packets, stats->mtods);
-
- return 0;
+ ionic_lif_rxq_stats(rxq);
}
int
diff --git a/drivers/net/ionic/ionic_rxtx.h b/drivers/net/ionic/ionic_rxtx.h
index 5348395956..7ca23178cc 100644
--- a/drivers/net/ionic/ionic_rxtx.h
+++ b/drivers/net/ionic/ionic_rxtx.h
@@ -37,14 +37,24 @@ int ionic_dev_rx_queue_setup(struct rte_eth_dev *dev, uint16_t rx_queue_id,
const struct rte_eth_rxconf *rx_conf, struct rte_mempool *mp);
void ionic_dev_rx_queue_release(struct rte_eth_dev *dev, uint16_t qid);
int ionic_dev_rx_queue_start(struct rte_eth_dev *dev, uint16_t rx_queue_id);
-int ionic_dev_rx_queue_stop(struct rte_eth_dev *eth_dev, uint16_t rx_queue_id);
+int ionic_dev_rx_queue_stop(struct rte_eth_dev *dev, uint16_t rx_queue_id);
int ionic_dev_tx_queue_setup(struct rte_eth_dev *dev, uint16_t tx_queue_id,
uint16_t nb_desc, uint32_t socket_id,
const struct rte_eth_txconf *tx_conf);
void ionic_dev_tx_queue_release(struct rte_eth_dev *dev, uint16_t qid);
-int ionic_dev_tx_queue_stop(struct rte_eth_dev *eth_dev, uint16_t tx_queue_id);
int ionic_dev_tx_queue_start(struct rte_eth_dev *dev, uint16_t tx_queue_id);
+int ionic_dev_tx_queue_stop(struct rte_eth_dev *dev, uint16_t tx_queue_id);
+
+/* Helpers for optimized dev_stop() */
+void ionic_dev_rx_queue_stop_firsthalf(struct rte_eth_dev *dev,
+ uint16_t rx_queue_id);
+void ionic_dev_rx_queue_stop_secondhalf(struct rte_eth_dev *dev,
+ uint16_t rx_queue_id);
+void ionic_dev_tx_queue_stop_firsthalf(struct rte_eth_dev *dev,
+ uint16_t tx_queue_id);
+void ionic_dev_tx_queue_stop_secondhalf(struct rte_eth_dev *dev,
+ uint16_t tx_queue_id);
void ionic_rxq_info_get(struct rte_eth_dev *dev, uint16_t queue_id,
struct rte_eth_rxq_info *qinfo);
--
2.17.1
^ permalink raw reply [flat|nested] 52+ messages in thread
* [PATCH 13/13] net/ionic: optimize device start operation
2024-02-02 19:32 [PATCH 00/13] net/ionic: miscellaneous fixes and improvements Andrew Boyer
` (11 preceding siblings ...)
2024-02-02 19:32 ` [PATCH 12/13] net/ionic: optimize device stop operation Andrew Boyer
@ 2024-02-02 19:32 ` Andrew Boyer
2024-02-03 4:28 ` Stephen Hemminger
2024-02-05 18:44 ` [PATCH 00/13] net/ionic: miscellaneous fixes and improvements Patrick Robb
` (15 subsequent siblings)
28 siblings, 1 reply; 52+ messages in thread
From: Andrew Boyer @ 2024-02-02 19:32 UTC (permalink / raw)
To: dev; +Cc: Andrew Boyer
Split the queue_start operation into first-half and second-half helpers.
This allows us to batch up the queue commands during dev_start(), reducing
the outage window when restarting the process by about 1ms per queue.
Signed-off-by: Andrew Boyer <andrew.boyer@amd.com>
---
drivers/net/ionic/ionic_lif.c | 178 +++++++++++++++++++++------------
drivers/net/ionic/ionic_lif.h | 6 +-
drivers/net/ionic/ionic_rxtx.c | 81 ++++++++++++---
drivers/net/ionic/ionic_rxtx.h | 10 ++
4 files changed, 194 insertions(+), 81 deletions(-)
diff --git a/drivers/net/ionic/ionic_lif.c b/drivers/net/ionic/ionic_lif.c
index 8ffdbc4df7..1937e48d9b 100644
--- a/drivers/net/ionic/ionic_lif.c
+++ b/drivers/net/ionic/ionic_lif.c
@@ -1598,52 +1598,61 @@ ionic_lif_set_features(struct ionic_lif *lif)
}
int
-ionic_lif_txq_init(struct ionic_tx_qcq *txq)
+ionic_lif_txq_init_nowait(struct ionic_tx_qcq *txq)
{
struct ionic_qcq *qcq = &txq->qcq;
struct ionic_queue *q = &qcq->q;
struct ionic_lif *lif = qcq->lif;
struct ionic_cq *cq = &qcq->cq;
- struct ionic_admin_ctx ctx = {
- .pending_work = true,
- .cmd.q_init = {
- .opcode = IONIC_CMD_Q_INIT,
- .type = q->type,
- .ver = lif->qtype_info[q->type].version,
- .index = rte_cpu_to_le_32(q->index),
- .flags = rte_cpu_to_le_16(IONIC_QINIT_F_ENA),
- .intr_index = rte_cpu_to_le_16(IONIC_INTR_NONE),
- .ring_size = rte_log2_u32(q->num_descs),
- .cq_ring_base = rte_cpu_to_le_64(cq->base_pa),
- .sg_ring_base = rte_cpu_to_le_64(q->sg_base_pa),
- },
- };
+ struct ionic_admin_ctx *ctx = &txq->admin_ctx;
int err;
+ memset(ctx, 0, sizeof(*ctx));
+ ctx->pending_work = true;
+ ctx->cmd.q_init.opcode = IONIC_CMD_Q_INIT;
+ ctx->cmd.q_init.type = q->type;
+ ctx->cmd.q_init.ver = lif->qtype_info[q->type].version;
+ ctx->cmd.q_init.index = rte_cpu_to_le_32(q->index);
+ ctx->cmd.q_init.flags = rte_cpu_to_le_16(IONIC_QINIT_F_ENA);
+ ctx->cmd.q_init.intr_index = rte_cpu_to_le_16(IONIC_INTR_NONE);
+ ctx->cmd.q_init.ring_size = rte_log2_u32(q->num_descs);
+ ctx->cmd.q_init.cq_ring_base = rte_cpu_to_le_64(cq->base_pa);
+ ctx->cmd.q_init.sg_ring_base = rte_cpu_to_le_64(q->sg_base_pa);
+
if (txq->flags & IONIC_QCQ_F_SG)
- ctx.cmd.q_init.flags |= rte_cpu_to_le_16(IONIC_QINIT_F_SG);
+ ctx->cmd.q_init.flags |= rte_cpu_to_le_16(IONIC_QINIT_F_SG);
if (txq->flags & IONIC_QCQ_F_CMB) {
- ctx.cmd.q_init.flags |= rte_cpu_to_le_16(IONIC_QINIT_F_CMB);
- ctx.cmd.q_init.ring_base = rte_cpu_to_le_64(q->cmb_base_pa);
+ ctx->cmd.q_init.flags |= rte_cpu_to_le_16(IONIC_QINIT_F_CMB);
+ ctx->cmd.q_init.ring_base = rte_cpu_to_le_64(q->cmb_base_pa);
} else {
- ctx.cmd.q_init.ring_base = rte_cpu_to_le_64(q->base_pa);
+ ctx->cmd.q_init.ring_base = rte_cpu_to_le_64(q->base_pa);
}
IONIC_PRINT(DEBUG, "txq_init.index %d", q->index);
IONIC_PRINT(DEBUG, "txq_init.ring_base 0x%" PRIx64 "", q->base_pa);
IONIC_PRINT(DEBUG, "txq_init.ring_size %d",
- ctx.cmd.q_init.ring_size);
- IONIC_PRINT(DEBUG, "txq_init.ver %u", ctx.cmd.q_init.ver);
+ ctx->cmd.q_init.ring_size);
+ IONIC_PRINT(DEBUG, "txq_init.ver %u", ctx->cmd.q_init.ver);
ionic_q_reset(q);
ionic_cq_reset(cq);
- err = ionic_adminq_post_wait(lif, &ctx);
+ /* Caller responsible for calling ionic_lif_txq_init_done() */
+ err = ionic_adminq_post(lif, ctx);
if (err)
- return err;
+ ctx->pending_work = false;
+ return err;
+}
- q->hw_type = ctx.comp.q_init.hw_type;
- q->hw_index = rte_le_to_cpu_32(ctx.comp.q_init.hw_index);
+void
+ionic_lif_txq_init_done(struct ionic_tx_qcq *txq)
+{
+ struct ionic_lif *lif = txq->qcq.lif;
+ struct ionic_queue *q = &txq->qcq.q;
+ struct ionic_admin_ctx *ctx = &txq->admin_ctx;
+
+ q->hw_type = ctx->comp.q_init.hw_type;
+ q->hw_index = rte_le_to_cpu_32(ctx->comp.q_init.hw_index);
q->db = ionic_db_map(lif, q);
IONIC_PRINT(DEBUG, "txq->hw_type %d", q->hw_type);
@@ -1651,57 +1660,64 @@ ionic_lif_txq_init(struct ionic_tx_qcq *txq)
IONIC_PRINT(DEBUG, "txq->db %p", q->db);
txq->flags |= IONIC_QCQ_F_INITED;
-
- return 0;
}
int
-ionic_lif_rxq_init(struct ionic_rx_qcq *rxq)
+ionic_lif_rxq_init_nowait(struct ionic_rx_qcq *rxq)
{
struct ionic_qcq *qcq = &rxq->qcq;
struct ionic_queue *q = &qcq->q;
struct ionic_lif *lif = qcq->lif;
struct ionic_cq *cq = &qcq->cq;
- struct ionic_admin_ctx ctx = {
- .pending_work = true,
- .cmd.q_init = {
- .opcode = IONIC_CMD_Q_INIT,
- .type = q->type,
- .ver = lif->qtype_info[q->type].version,
- .index = rte_cpu_to_le_32(q->index),
- .flags = rte_cpu_to_le_16(IONIC_QINIT_F_ENA),
- .intr_index = rte_cpu_to_le_16(IONIC_INTR_NONE),
- .ring_size = rte_log2_u32(q->num_descs),
- .cq_ring_base = rte_cpu_to_le_64(cq->base_pa),
- .sg_ring_base = rte_cpu_to_le_64(q->sg_base_pa),
- },
- };
+ struct ionic_admin_ctx *ctx = &rxq->admin_ctx;
int err;
+ memset(ctx, 0, sizeof(*ctx));
+ ctx->pending_work = true;
+ ctx->cmd.q_init.opcode = IONIC_CMD_Q_INIT;
+ ctx->cmd.q_init.type = q->type;
+ ctx->cmd.q_init.ver = lif->qtype_info[q->type].version;
+ ctx->cmd.q_init.index = rte_cpu_to_le_32(q->index);
+ ctx->cmd.q_init.flags = rte_cpu_to_le_16(IONIC_QINIT_F_ENA);
+ ctx->cmd.q_init.intr_index = rte_cpu_to_le_16(IONIC_INTR_NONE);
+ ctx->cmd.q_init.ring_size = rte_log2_u32(q->num_descs);
+ ctx->cmd.q_init.cq_ring_base = rte_cpu_to_le_64(cq->base_pa);
+ ctx->cmd.q_init.sg_ring_base = rte_cpu_to_le_64(q->sg_base_pa);
+
if (rxq->flags & IONIC_QCQ_F_SG)
- ctx.cmd.q_init.flags |= rte_cpu_to_le_16(IONIC_QINIT_F_SG);
+ ctx->cmd.q_init.flags |= rte_cpu_to_le_16(IONIC_QINIT_F_SG);
if (rxq->flags & IONIC_QCQ_F_CMB) {
- ctx.cmd.q_init.flags |= rte_cpu_to_le_16(IONIC_QINIT_F_CMB);
- ctx.cmd.q_init.ring_base = rte_cpu_to_le_64(q->cmb_base_pa);
+ ctx->cmd.q_init.flags |= rte_cpu_to_le_16(IONIC_QINIT_F_CMB);
+ ctx->cmd.q_init.ring_base = rte_cpu_to_le_64(q->cmb_base_pa);
} else {
- ctx.cmd.q_init.ring_base = rte_cpu_to_le_64(q->base_pa);
+ ctx->cmd.q_init.ring_base = rte_cpu_to_le_64(q->base_pa);
}
IONIC_PRINT(DEBUG, "rxq_init.index %d", q->index);
IONIC_PRINT(DEBUG, "rxq_init.ring_base 0x%" PRIx64 "", q->base_pa);
IONIC_PRINT(DEBUG, "rxq_init.ring_size %d",
- ctx.cmd.q_init.ring_size);
- IONIC_PRINT(DEBUG, "rxq_init.ver %u", ctx.cmd.q_init.ver);
+ ctx->cmd.q_init.ring_size);
+ IONIC_PRINT(DEBUG, "rxq_init.ver %u", ctx->cmd.q_init.ver);
ionic_q_reset(q);
ionic_cq_reset(cq);
- err = ionic_adminq_post_wait(lif, &ctx);
+ /* Caller responsible for calling ionic_lif_rxq_init_done() */
+ err = ionic_adminq_post(lif, ctx);
if (err)
- return err;
+ ctx->pending_work = false;
+ return err;
+}
- q->hw_type = ctx.comp.q_init.hw_type;
- q->hw_index = rte_le_to_cpu_32(ctx.comp.q_init.hw_index);
+void
+ionic_lif_rxq_init_done(struct ionic_rx_qcq *rxq)
+{
+ struct ionic_lif *lif = rxq->qcq.lif;
+ struct ionic_queue *q = &rxq->qcq.q;
+ struct ionic_admin_ctx *ctx = &rxq->admin_ctx;
+
+ q->hw_type = ctx->comp.q_init.hw_type;
+ q->hw_index = rte_le_to_cpu_32(ctx->comp.q_init.hw_index);
q->db = ionic_db_map(lif, q);
rxq->flags |= IONIC_QCQ_F_INITED;
@@ -1709,8 +1725,6 @@ ionic_lif_rxq_init(struct ionic_rx_qcq *rxq)
IONIC_PRINT(DEBUG, "rxq->hw_type %d", q->hw_type);
IONIC_PRINT(DEBUG, "rxq->hw_index %d", q->hw_index);
IONIC_PRINT(DEBUG, "rxq->db %p", q->db);
-
- return 0;
}
static int
@@ -1959,9 +1973,11 @@ ionic_lif_configure(struct ionic_lif *lif)
int
ionic_lif_start(struct ionic_lif *lif)
{
+ struct rte_eth_dev *dev = lif->eth_dev;
uint32_t rx_mode;
- uint32_t i;
+ uint32_t i, j, chunk;
int err;
+ bool fatal = false;
err = ionic_lif_rss_setup(lif);
if (err)
@@ -1982,25 +1998,57 @@ ionic_lif_start(struct ionic_lif *lif)
"on port %u",
lif->nrxqcqs, lif->ntxqcqs, lif->port_id);
- for (i = 0; i < lif->nrxqcqs; i++) {
- struct ionic_rx_qcq *rxq = lif->rxqcqs[i];
- if (!(rxq->flags & IONIC_QCQ_F_DEFERRED)) {
- err = ionic_dev_rx_queue_start(lif->eth_dev, i);
+ chunk = ionic_adminq_space_avail(lif);
+
+ for (i = 0; i < lif->nrxqcqs; i += chunk) {
+ if (lif->rxqcqs[0]->flags & IONIC_QCQ_F_DEFERRED) {
+ IONIC_PRINT(DEBUG, "Rx queue start deferred");
+ break;
+ }
+
+ for (j = 0; j < chunk && i + j < lif->nrxqcqs; j++) {
+ err = ionic_dev_rx_queue_start_firsthalf(dev, i + j);
+ if (err) {
+ fatal = true;
+ break;
+ }
+ }
+ for (j = 0; j < chunk && i + j < lif->nrxqcqs; j++) {
+ /* Commands that failed to post return immediately */
+ err = ionic_dev_rx_queue_start_secondhalf(dev, i + j);
if (err)
- return err;
+ /* Don't break */
+ fatal = true;
}
}
+ if (fatal)
+ return -EIO;
- for (i = 0; i < lif->ntxqcqs; i++) {
- struct ionic_tx_qcq *txq = lif->txqcqs[i];
- if (!(txq->flags & IONIC_QCQ_F_DEFERRED)) {
- err = ionic_dev_tx_queue_start(lif->eth_dev, i);
+ for (i = 0; i < lif->ntxqcqs; i += chunk) {
+ if (lif->txqcqs[0]->flags & IONIC_QCQ_F_DEFERRED) {
+ IONIC_PRINT(DEBUG, "Tx queue start deferred");
+ break;
+ }
+
+ for (j = 0; j < chunk && i + j < lif->ntxqcqs; j++) {
+ err = ionic_dev_tx_queue_start_firsthalf(dev, i + j);
+ if (err) {
+ fatal = true;
+ break;
+ }
+ }
+ for (j = 0; j < chunk && i + j < lif->ntxqcqs; j++) {
+ /* Commands that failed to post return immediately */
+ err = ionic_dev_tx_queue_start_secondhalf(dev, i + j);
if (err)
- return err;
+ /* Don't break */
+ fatal = true;
}
}
+ if (fatal)
+ return -EIO;
/* Carrier ON here */
lif->state |= IONIC_LIF_F_UP;
diff --git a/drivers/net/ionic/ionic_lif.h b/drivers/net/ionic/ionic_lif.h
index ee13f5b7c8..591cf1a2ff 100644
--- a/drivers/net/ionic/ionic_lif.h
+++ b/drivers/net/ionic/ionic_lif.h
@@ -228,11 +228,13 @@ int ionic_tx_qcq_alloc(struct ionic_lif *lif, uint32_t socket_id,
struct ionic_tx_qcq **qcq_out);
void ionic_qcq_free(struct ionic_qcq *qcq);
-int ionic_lif_rxq_init(struct ionic_rx_qcq *rxq);
+int ionic_lif_rxq_init_nowait(struct ionic_rx_qcq *rxq);
+void ionic_lif_rxq_init_done(struct ionic_rx_qcq *rxq);
void ionic_lif_rxq_deinit_nowait(struct ionic_rx_qcq *rxq);
void ionic_lif_rxq_stats(struct ionic_rx_qcq *rxq);
-int ionic_lif_txq_init(struct ionic_tx_qcq *txq);
+int ionic_lif_txq_init_nowait(struct ionic_tx_qcq *txq);
+void ionic_lif_txq_init_done(struct ionic_tx_qcq *txq);
void ionic_lif_txq_deinit_nowait(struct ionic_tx_qcq *txq);
void ionic_lif_txq_stats(struct ionic_tx_qcq *txq);
diff --git a/drivers/net/ionic/ionic_rxtx.c b/drivers/net/ionic/ionic_rxtx.c
index 774dc596c0..ad04e987eb 100644
--- a/drivers/net/ionic/ionic_rxtx.c
+++ b/drivers/net/ionic/ionic_rxtx.c
@@ -203,27 +203,54 @@ ionic_dev_tx_queue_setup(struct rte_eth_dev *eth_dev, uint16_t tx_queue_id,
* Start Transmit Units for specified queue.
*/
int __rte_cold
-ionic_dev_tx_queue_start(struct rte_eth_dev *eth_dev, uint16_t tx_queue_id)
+ionic_dev_tx_queue_start(struct rte_eth_dev *dev, uint16_t tx_queue_id)
{
- uint8_t *tx_queue_state = eth_dev->data->tx_queue_state;
- struct ionic_tx_qcq *txq;
int err;
+ err = ionic_dev_tx_queue_start_firsthalf(dev, tx_queue_id);
+ if (err)
+ return err;
+
+ return ionic_dev_tx_queue_start_secondhalf(dev, tx_queue_id);
+}
+
+int __rte_cold
+ionic_dev_tx_queue_start_firsthalf(struct rte_eth_dev *dev,
+ uint16_t tx_queue_id)
+{
+ uint8_t *tx_queue_state = dev->data->tx_queue_state;
+ struct ionic_tx_qcq *txq = dev->data->tx_queues[tx_queue_id];
+
if (tx_queue_state[tx_queue_id] == RTE_ETH_QUEUE_STATE_STARTED) {
IONIC_PRINT(DEBUG, "TX queue %u already started",
tx_queue_id);
return 0;
}
- txq = eth_dev->data->tx_queues[tx_queue_id];
-
IONIC_PRINT(DEBUG, "Starting TX queue %u, %u descs",
tx_queue_id, txq->qcq.q.num_descs);
- err = ionic_lif_txq_init(txq);
+ return ionic_lif_txq_init_nowait(txq);
+}
+
+int __rte_cold
+ionic_dev_tx_queue_start_secondhalf(struct rte_eth_dev *dev,
+ uint16_t tx_queue_id)
+{
+ uint8_t *tx_queue_state = dev->data->tx_queue_state;
+ struct ionic_lif *lif = IONIC_ETH_DEV_TO_LIF(dev);
+ struct ionic_tx_qcq *txq = dev->data->tx_queues[tx_queue_id];
+ int err;
+
+ if (tx_queue_state[tx_queue_id] == RTE_ETH_QUEUE_STATE_STARTED)
+ return 0;
+
+ err = ionic_adminq_wait(lif, &txq->admin_ctx);
if (err)
return err;
+ ionic_lif_txq_init_done(txq);
+
tx_queue_state[tx_queue_id] = RTE_ETH_QUEUE_STATE_STARTED;
return 0;
@@ -680,22 +707,31 @@ ionic_rx_init_descriptors(struct ionic_rx_qcq *rxq)
* Start Receive Units for specified queue.
*/
int __rte_cold
-ionic_dev_rx_queue_start(struct rte_eth_dev *eth_dev, uint16_t rx_queue_id)
+ionic_dev_rx_queue_start(struct rte_eth_dev *dev, uint16_t rx_queue_id)
{
- uint8_t *rx_queue_state = eth_dev->data->rx_queue_state;
- struct ionic_rx_qcq *rxq;
- struct ionic_queue *q;
int err;
+ err = ionic_dev_rx_queue_start_firsthalf(dev, rx_queue_id);
+ if (err)
+ return err;
+
+ return ionic_dev_rx_queue_start_secondhalf(dev, rx_queue_id);
+}
+
+int __rte_cold
+ionic_dev_rx_queue_start_firsthalf(struct rte_eth_dev *dev,
+ uint16_t rx_queue_id)
+{
+ uint8_t *rx_queue_state = dev->data->rx_queue_state;
+ struct ionic_rx_qcq *rxq = dev->data->rx_queues[rx_queue_id];
+ struct ionic_queue *q = &rxq->qcq.q;
+
if (rx_queue_state[rx_queue_id] == RTE_ETH_QUEUE_STATE_STARTED) {
IONIC_PRINT(DEBUG, "RX queue %u already started",
rx_queue_id);
return 0;
}
- rxq = eth_dev->data->rx_queues[rx_queue_id];
- q = &rxq->qcq.q;
-
rxq->frame_size = rxq->qcq.lif->frame_size - RTE_ETHER_CRC_LEN;
/* Recalculate segment count based on MTU */
@@ -707,10 +743,27 @@ ionic_dev_rx_queue_start(struct rte_eth_dev *eth_dev, uint16_t rx_queue_id)
ionic_rx_init_descriptors(rxq);
- err = ionic_lif_rxq_init(rxq);
+ return ionic_lif_rxq_init_nowait(rxq);
+}
+
+int __rte_cold
+ionic_dev_rx_queue_start_secondhalf(struct rte_eth_dev *dev,
+ uint16_t rx_queue_id)
+{
+ uint8_t *rx_queue_state = dev->data->rx_queue_state;
+ struct ionic_lif *lif = IONIC_ETH_DEV_TO_LIF(dev);
+ struct ionic_rx_qcq *rxq = dev->data->rx_queues[rx_queue_id];
+ int err;
+
+ if (rx_queue_state[rx_queue_id] == RTE_ETH_QUEUE_STATE_STARTED)
+ return 0;
+
+ err = ionic_adminq_wait(lif, &rxq->admin_ctx);
if (err)
return err;
+ ionic_lif_rxq_init_done(rxq);
+
/* Allocate buffers for descriptor ring */
if (rxq->flags & IONIC_QCQ_F_SG)
err = ionic_rx_fill_sg(rxq);
diff --git a/drivers/net/ionic/ionic_rxtx.h b/drivers/net/ionic/ionic_rxtx.h
index 7ca23178cc..a342afec54 100644
--- a/drivers/net/ionic/ionic_rxtx.h
+++ b/drivers/net/ionic/ionic_rxtx.h
@@ -46,6 +46,16 @@ void ionic_dev_tx_queue_release(struct rte_eth_dev *dev, uint16_t qid);
int ionic_dev_tx_queue_start(struct rte_eth_dev *dev, uint16_t tx_queue_id);
int ionic_dev_tx_queue_stop(struct rte_eth_dev *dev, uint16_t tx_queue_id);
+/* Helpers for optimized dev_start() */
+int ionic_dev_rx_queue_start_firsthalf(struct rte_eth_dev *dev,
+ uint16_t rx_queue_id);
+int ionic_dev_rx_queue_start_secondhalf(struct rte_eth_dev *dev,
+ uint16_t rx_queue_id);
+int ionic_dev_tx_queue_start_firsthalf(struct rte_eth_dev *dev,
+ uint16_t tx_queue_id);
+int ionic_dev_tx_queue_start_secondhalf(struct rte_eth_dev *dev,
+ uint16_t tx_queue_id);
+
/* Helpers for optimized dev_stop() */
void ionic_dev_rx_queue_stop_firsthalf(struct rte_eth_dev *dev,
uint16_t rx_queue_id);
--
2.17.1
^ permalink raw reply [flat|nested] 52+ messages in thread
* Re: [PATCH 01/13] net/ionic: add stat for completion queue entries processed
2024-02-02 19:32 ` [PATCH 01/13] net/ionic: add stat for completion queue entries processed Andrew Boyer
@ 2024-02-03 4:24 ` Stephen Hemminger
2024-02-05 12:49 ` Boyer, Andrew
0 siblings, 1 reply; 52+ messages in thread
From: Stephen Hemminger @ 2024-02-03 4:24 UTC (permalink / raw)
To: Andrew Boyer; +Cc: dev
On Fri, 2 Feb 2024 11:32:26 -0800
Andrew Boyer <andrew.boyer@amd.com> wrote:
> When completion coalescing is turned on in the FW, there will be
> fewer CQE than Tx packets. Expose the stat through debug logging.
>
> Signed-off-by: Andrew Boyer <andrew.boyer@amd.com>
If you care about the stat it should be in xstats.
^ permalink raw reply [flat|nested] 52+ messages in thread
* Re: [PATCH 04/13] net/ionic: fix missing volatile type for cqe pointers
2024-02-02 19:32 ` [PATCH 04/13] net/ionic: fix missing volatile type for cqe pointers Andrew Boyer
@ 2024-02-03 4:26 ` Stephen Hemminger
2024-02-05 13:33 ` Boyer, Andrew
0 siblings, 1 reply; 52+ messages in thread
From: Stephen Hemminger @ 2024-02-03 4:26 UTC (permalink / raw)
To: Andrew Boyer; +Cc: dev, Neel Patel, stable
On Fri, 2 Feb 2024 11:32:29 -0800
Andrew Boyer <andrew.boyer@amd.com> wrote:
> From: Neel Patel <neel.patel@amd.com>
>
> This memory may be changed by the hardware, so the volatile
> keyword is required for correctness.
>
> Fixes: e86a6fcc7cf3 ("net/ionic: add optimized non-scattered Rx/Tx")
> cc: stable@dpdk.org
>
> Signed-off-by: Andrew Boyer <andrew.boyer@amd.com>
> Signed-off-by: Neel Patel <neel.patel@amd.com>
In general barriers are better than volatile.
Volatile is a compiler not hardware construct really.
https://www.kernel.org/doc/html/latest/process/volatile-considered-harmful.html
^ permalink raw reply [flat|nested] 52+ messages in thread
* Re: [PATCH 13/13] net/ionic: optimize device start operation
2024-02-02 19:32 ` [PATCH 13/13] net/ionic: optimize device start operation Andrew Boyer
@ 2024-02-03 4:28 ` Stephen Hemminger
2024-02-05 13:21 ` Boyer, Andrew
0 siblings, 1 reply; 52+ messages in thread
From: Stephen Hemminger @ 2024-02-03 4:28 UTC (permalink / raw)
To: Andrew Boyer; +Cc: dev
On Fri, 2 Feb 2024 11:32:38 -0800
Andrew Boyer <andrew.boyer@amd.com> wrote:
> + memset(ctx, 0, sizeof(*ctx));
> + ctx->pending_work = true;
> + ctx->cmd.q_init.opcode = IONIC_CMD_Q_INIT;
> + ctx->cmd.q_init.type = q->type;
> + ctx->cmd.q_init.ver = lif->qtype_info[q->type].version;
> + ctx->cmd.q_init.index = rte_cpu_to_le_32(q->index);
> + ctx->cmd.q_init.flags = rte_cpu_to_le_16(IONIC_QINIT_F_ENA);
> + ctx->cmd.q_init.intr_index = rte_cpu_to_le_16(IONIC_INTR_NONE);
> + ctx->cmd.q_init.ring_size = rte_log2_u32(q->num_descs);
> + ctx->cmd.q_init.cq_ring_base = rte_cpu_to_le_64(cq->base_pa);
> + ctx->cmd.q_init.sg_ring_base = rte_cpu_to_le_64(q->sg_base_pa);
> +
memset followed by assignment is technically slower than structure
initialization because it requires two writes to the data.
But the optimizer may in some cases figure that out.
^ permalink raw reply [flat|nested] 52+ messages in thread
* Re: [PATCH 01/13] net/ionic: add stat for completion queue entries processed
2024-02-03 4:24 ` Stephen Hemminger
@ 2024-02-05 12:49 ` Boyer, Andrew
0 siblings, 0 replies; 52+ messages in thread
From: Boyer, Andrew @ 2024-02-05 12:49 UTC (permalink / raw)
To: Stephen Hemminger; +Cc: Boyer, Andrew, dev
> On Feb 2, 2024, at 11:24 PM, Stephen Hemminger <stephen@networkplumber.org> wrote:
>
> Caution: This message originated from an External Source. Use proper caution when opening attachments, clicking links, or responding.
>
>
> On Fri, 2 Feb 2024 11:32:26 -0800
> Andrew Boyer <andrew.boyer@amd.com> wrote:
>
>> When completion coalescing is turned on in the FW, there will be
>> fewer CQE than Tx packets. Expose the stat through debug logging.
>>
>> Signed-off-by: Andrew Boyer <andrew.boyer@amd.com>
>
> If you care about the stat it should be in xstats.
Thanks for looking. Moving all of our per-queue stats to xstats is on the todo list.
-Andrew
^ permalink raw reply [flat|nested] 52+ messages in thread
* Re: [PATCH 13/13] net/ionic: optimize device start operation
2024-02-03 4:28 ` Stephen Hemminger
@ 2024-02-05 13:21 ` Boyer, Andrew
0 siblings, 0 replies; 52+ messages in thread
From: Boyer, Andrew @ 2024-02-05 13:21 UTC (permalink / raw)
To: Stephen Hemminger; +Cc: Boyer, Andrew, dev
> On Feb 2, 2024, at 11:28 PM, Stephen Hemminger <stephen@networkplumber.org> wrote:
>
> Caution: This message originated from an External Source. Use proper caution when opening attachments, clicking links, or responding.
>
>
> On Fri, 2 Feb 2024 11:32:38 -0800
> Andrew Boyer <andrew.boyer@amd.com> wrote:
>
>> + memset(ctx, 0, sizeof(*ctx));
>> + ctx->pending_work = true;
>> + ctx->cmd.q_init.opcode = IONIC_CMD_Q_INIT;
>> + ctx->cmd.q_init.type = q->type;
>> + ctx->cmd.q_init.ver = lif->qtype_info[q->type].version;
>> + ctx->cmd.q_init.index = rte_cpu_to_le_32(q->index);
>> + ctx->cmd.q_init.flags = rte_cpu_to_le_16(IONIC_QINIT_F_ENA);
>> + ctx->cmd.q_init.intr_index = rte_cpu_to_le_16(IONIC_INTR_NONE);
>> + ctx->cmd.q_init.ring_size = rte_log2_u32(q->num_descs);
>> + ctx->cmd.q_init.cq_ring_base = rte_cpu_to_le_64(cq->base_pa);
>> + ctx->cmd.q_init.sg_ring_base = rte_cpu_to_le_64(q->sg_base_pa);
>> +
>
> memset followed by assignment is technically slower than structure
> initialization because it requires two writes to the data.
> But the optimizer may in some cases figure that out.
Thanks for looking. Since DPDK requires C11 now, and I see other users of compound literals in lib/, I'll respin this to use that instead.
(The internet says unset fields are implicitly initialized to zero, hopefully that's correct.)
It makes the diff a lot cleaner, too.
Thanks,
Andrew
^ permalink raw reply [flat|nested] 52+ messages in thread
* Re: [PATCH 04/13] net/ionic: fix missing volatile type for cqe pointers
2024-02-03 4:26 ` Stephen Hemminger
@ 2024-02-05 13:33 ` Boyer, Andrew
0 siblings, 0 replies; 52+ messages in thread
From: Boyer, Andrew @ 2024-02-05 13:33 UTC (permalink / raw)
To: Stephen Hemminger; +Cc: Boyer, Andrew, dev, Patel, Neel, stable
> On Feb 2, 2024, at 11:26 PM, Stephen Hemminger <stephen@networkplumber.org> wrote:
>
> Caution: This message originated from an External Source. Use proper caution when opening attachments, clicking links, or responding.
>
>
> On Fri, 2 Feb 2024 11:32:29 -0800
> Andrew Boyer <andrew.boyer@amd.com> wrote:
>
>> From: Neel Patel <neel.patel@amd.com>
>>
>> This memory may be changed by the hardware, so the volatile
>> keyword is required for correctness.
>>
>> Fixes: e86a6fcc7cf3 ("net/ionic: add optimized non-scattered Rx/Tx")
>> cc: stable@dpdk.org
>>
>> Signed-off-by: Andrew Boyer <andrew.boyer@amd.com>
>> Signed-off-by: Neel Patel <neel.patel@amd.com>
>
> In general barriers are better than volatile.
> Volatile is a compiler not hardware construct really.
>
> https://www.kernel.org/doc/html/latest/process/volatile-considered-harmful.html
Thanks for looking. The goal here is to prevent inappropriate compiler optimization.
From the link you posted:
> Pointers to data structures in coherent memory which might be modified by I/O devices can, sometimes, legitimately be volatile. A ring buffer used by a network adapter, where that adapter changes pointers to indicate which descriptors have been processed, is an example of this type of situation.
Doesn't that sound like what we are doing?
Thanks,
Andrew
^ permalink raw reply [flat|nested] 52+ messages in thread
* Re: [PATCH 00/13] net/ionic: miscellaneous fixes and improvements
2024-02-02 19:32 [PATCH 00/13] net/ionic: miscellaneous fixes and improvements Andrew Boyer
` (12 preceding siblings ...)
2024-02-02 19:32 ` [PATCH 13/13] net/ionic: optimize device start operation Andrew Boyer
@ 2024-02-05 18:44 ` Patrick Robb
2024-02-05 19:41 ` Boyer, Andrew
2024-02-07 2:18 ` [PATCH v2 " Andrew Boyer
` (14 subsequent siblings)
28 siblings, 1 reply; 52+ messages in thread
From: Patrick Robb @ 2024-02-05 18:44 UTC (permalink / raw)
To: Andrew Boyer; +Cc: dev
[-- Attachment #1: Type: text/plain, Size: 226 bytes --]
Recheck-request: iol-intel-Functional
I saw this failed in CI testing for the DTS MTU_Update test but suspect it
was an infra failure as it happened across 3 distinct patches in a short
span of time. So, triggering a retest.
[-- Attachment #2: Type: text/html, Size: 572 bytes --]
^ permalink raw reply [flat|nested] 52+ messages in thread
* Re: [PATCH 00/13] net/ionic: miscellaneous fixes and improvements
2024-02-05 18:44 ` [PATCH 00/13] net/ionic: miscellaneous fixes and improvements Patrick Robb
@ 2024-02-05 19:41 ` Boyer, Andrew
0 siblings, 0 replies; 52+ messages in thread
From: Boyer, Andrew @ 2024-02-05 19:41 UTC (permalink / raw)
To: Patrick Robb; +Cc: Boyer, Andrew, dev
[-- Attachment #1: Type: text/plain, Size: 519 bytes --]
On Feb 5, 2024, at 1:44 PM, Patrick Robb <probb@iol.unh.edu> wrote:
Caution: This message originated from an External Source. Use proper caution when opening attachments, clicking links, or responding.
Recheck-request: iol-intel-Functional
I saw this failed in CI testing for the DTS MTU_Update test but suspect it was an infra failure as it happened across 3 distinct patches in a short span of time. So, triggering a retest.
I thought I saw another patchset hit the same failure that day.
-Andrew
[-- Attachment #2: Type: text/html, Size: 1814 bytes --]
^ permalink raw reply [flat|nested] 52+ messages in thread
* [PATCH v2 00/13] net/ionic: miscellaneous fixes and improvements
2024-02-02 19:32 [PATCH 00/13] net/ionic: miscellaneous fixes and improvements Andrew Boyer
` (13 preceding siblings ...)
2024-02-05 18:44 ` [PATCH 00/13] net/ionic: miscellaneous fixes and improvements Patrick Robb
@ 2024-02-07 2:18 ` Andrew Boyer
2024-02-07 2:18 ` [PATCH v2 01/13] net/ionic: add stat for completion queue entries processed Andrew Boyer
` (13 subsequent siblings)
28 siblings, 0 replies; 52+ messages in thread
From: Andrew Boyer @ 2024-02-07 2:18 UTC (permalink / raw)
To: dev; +Cc: Andrew Boyer
This patchset provides miscellaneous fixes and improvements for
the net/ionic driver used by AMD Pensando devices.
V2:
- Update device stop and device start patches to use compound literals
as suggested by review.
Akshay Dorwat (1):
net/ionic: fix RSS query routine
Andrew Boyer (8):
net/ionic: add stat for completion queue entries processed
net/ionic: increase max supported MTU to 9750 bytes
net/ionic: don't auto-enable Rx scatter-gather a second time
net/ionic: replace non-standard type in structure definition
net/ionic: fix device close sequence to avoid crash
net/ionic: optimize device close operation
net/ionic: optimize device stop operation
net/ionic: optimize device start operation
Brad Larson (1):
net/ionic: add flexible firmware xstat counters
Neel Patel (2):
net/ionic: fix missing volatile type for cqe pointers
net/ionic: memcpy descriptors when using Q-in-CMB
Vamsi Krishna Atluri (1):
net/ionic: report 1G and 200G link speeds when applicable
drivers/net/ionic/ionic.h | 3 +
drivers/net/ionic/ionic_dev.c | 9 +-
drivers/net/ionic/ionic_dev.h | 8 +-
drivers/net/ionic/ionic_dev_pci.c | 2 +-
drivers/net/ionic/ionic_ethdev.c | 81 ++++++---
drivers/net/ionic/ionic_if.h | 70 ++++----
drivers/net/ionic/ionic_lif.c | 233 ++++++++++++++++++--------
drivers/net/ionic/ionic_lif.h | 19 ++-
drivers/net/ionic/ionic_main.c | 17 +-
drivers/net/ionic/ionic_rxtx.c | 160 +++++++++++++-----
drivers/net/ionic/ionic_rxtx.h | 80 ++++++++-
drivers/net/ionic/ionic_rxtx_sg.c | 28 ++--
drivers/net/ionic/ionic_rxtx_simple.c | 28 ++--
13 files changed, 527 insertions(+), 211 deletions(-)
--
2.17.1
^ permalink raw reply [flat|nested] 52+ messages in thread
* [PATCH v2 01/13] net/ionic: add stat for completion queue entries processed
2024-02-02 19:32 [PATCH 00/13] net/ionic: miscellaneous fixes and improvements Andrew Boyer
` (14 preceding siblings ...)
2024-02-07 2:18 ` [PATCH v2 " Andrew Boyer
@ 2024-02-07 2:18 ` Andrew Boyer
2024-02-07 2:18 ` [PATCH v2 02/13] net/ionic: increase max supported MTU to 9750 bytes Andrew Boyer
` (12 subsequent siblings)
28 siblings, 0 replies; 52+ messages in thread
From: Andrew Boyer @ 2024-02-07 2:18 UTC (permalink / raw)
To: dev; +Cc: Andrew Boyer
When completion coalescing is turned on in the FW, there will be
fewer CQE than Tx packets. Expose the stat through debug logging.
Signed-off-by: Andrew Boyer <andrew.boyer@amd.com>
---
drivers/net/ionic/ionic_lif.h | 1 +
drivers/net/ionic/ionic_rxtx.c | 3 +++
drivers/net/ionic/ionic_rxtx_sg.c | 2 ++
drivers/net/ionic/ionic_rxtx_simple.c | 2 ++
4 files changed, 8 insertions(+)
diff --git a/drivers/net/ionic/ionic_lif.h b/drivers/net/ionic/ionic_lif.h
index 36b3bcc5a9..cac7a4583b 100644
--- a/drivers/net/ionic/ionic_lif.h
+++ b/drivers/net/ionic/ionic_lif.h
@@ -32,6 +32,7 @@
struct ionic_tx_stats {
uint64_t packets;
uint64_t bytes;
+ uint64_t comps;
uint64_t drop;
uint64_t stop;
uint64_t no_csum;
diff --git a/drivers/net/ionic/ionic_rxtx.c b/drivers/net/ionic/ionic_rxtx.c
index b9e73b4871..d92b231f8f 100644
--- a/drivers/net/ionic/ionic_rxtx.c
+++ b/drivers/net/ionic/ionic_rxtx.c
@@ -117,6 +117,9 @@ ionic_dev_tx_queue_stop(struct rte_eth_dev *eth_dev, uint16_t tx_queue_id)
stats = &txq->stats;
IONIC_PRINT(DEBUG, "TX queue %u pkts %ju tso %ju",
txq->qcq.q.index, stats->packets, stats->tso);
+ IONIC_PRINT(DEBUG, "TX queue %u comps %ju (%ju per)",
+ txq->qcq.q.index, stats->comps,
+ stats->comps ? stats->packets / stats->comps : 0);
return 0;
}
diff --git a/drivers/net/ionic/ionic_rxtx_sg.c b/drivers/net/ionic/ionic_rxtx_sg.c
index ab8e56e91c..6c028a698c 100644
--- a/drivers/net/ionic/ionic_rxtx_sg.c
+++ b/drivers/net/ionic/ionic_rxtx_sg.c
@@ -26,6 +26,7 @@ ionic_tx_flush_sg(struct ionic_tx_qcq *txq)
{
struct ionic_cq *cq = &txq->qcq.cq;
struct ionic_queue *q = &txq->qcq.q;
+ struct ionic_tx_stats *stats = &txq->stats;
struct rte_mbuf *txm;
struct ionic_txq_comp *cq_desc, *cq_desc_base = cq->base;
void **info;
@@ -72,6 +73,7 @@ ionic_tx_flush_sg(struct ionic_tx_qcq *txq)
}
cq_desc = &cq_desc_base[cq->tail_idx];
+ stats->comps++;
}
}
diff --git a/drivers/net/ionic/ionic_rxtx_simple.c b/drivers/net/ionic/ionic_rxtx_simple.c
index 5f81856256..5969287b66 100644
--- a/drivers/net/ionic/ionic_rxtx_simple.c
+++ b/drivers/net/ionic/ionic_rxtx_simple.c
@@ -26,6 +26,7 @@ ionic_tx_flush(struct ionic_tx_qcq *txq)
{
struct ionic_cq *cq = &txq->qcq.cq;
struct ionic_queue *q = &txq->qcq.q;
+ struct ionic_tx_stats *stats = &txq->stats;
struct rte_mbuf *txm;
struct ionic_txq_comp *cq_desc, *cq_desc_base = cq->base;
void **info;
@@ -67,6 +68,7 @@ ionic_tx_flush(struct ionic_tx_qcq *txq)
}
cq_desc = &cq_desc_base[cq->tail_idx];
+ stats->comps++;
}
}
--
2.17.1
^ permalink raw reply [flat|nested] 52+ messages in thread
* [PATCH v2 02/13] net/ionic: increase max supported MTU to 9750 bytes
2024-02-02 19:32 [PATCH 00/13] net/ionic: miscellaneous fixes and improvements Andrew Boyer
` (15 preceding siblings ...)
2024-02-07 2:18 ` [PATCH v2 01/13] net/ionic: add stat for completion queue entries processed Andrew Boyer
@ 2024-02-07 2:18 ` Andrew Boyer
2024-02-07 2:18 ` [PATCH v2 03/13] net/ionic: don't auto-enable Rx scatter-gather a second time Andrew Boyer
` (11 subsequent siblings)
28 siblings, 0 replies; 52+ messages in thread
From: Andrew Boyer @ 2024-02-07 2:18 UTC (permalink / raw)
To: dev; +Cc: Andrew Boyer, Bhuvan Mital
Some configurations want to use values this high internally.
Allow them to do so without modifying the code.
Signed-off-by: Andrew Boyer <andrew.boyer@amd.com>
Signed-off-by: Bhuvan Mital <bhuvan.mital@amd.com>
---
drivers/net/ionic/ionic_dev.h | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/drivers/net/ionic/ionic_dev.h b/drivers/net/ionic/ionic_dev.h
index b1e74fbd89..971c261b27 100644
--- a/drivers/net/ionic/ionic_dev.h
+++ b/drivers/net/ionic/ionic_dev.h
@@ -14,7 +14,7 @@
#define VLAN_TAG_SIZE 4
#define IONIC_MIN_MTU RTE_ETHER_MIN_MTU
-#define IONIC_MAX_MTU 9378
+#define IONIC_MAX_MTU 9750
#define IONIC_ETH_OVERHEAD (RTE_ETHER_HDR_LEN + VLAN_TAG_SIZE)
#define IONIC_MAX_RING_DESC 32768
--
2.17.1
^ permalink raw reply [flat|nested] 52+ messages in thread
* [PATCH v2 03/13] net/ionic: don't auto-enable Rx scatter-gather a second time
2024-02-02 19:32 [PATCH 00/13] net/ionic: miscellaneous fixes and improvements Andrew Boyer
` (16 preceding siblings ...)
2024-02-07 2:18 ` [PATCH v2 02/13] net/ionic: increase max supported MTU to 9750 bytes Andrew Boyer
@ 2024-02-07 2:18 ` Andrew Boyer
2024-02-07 2:18 ` [PATCH v2 04/13] net/ionic: fix missing volatile type for cqe pointers Andrew Boyer
` (10 subsequent siblings)
28 siblings, 0 replies; 52+ messages in thread
From: Andrew Boyer @ 2024-02-07 2:18 UTC (permalink / raw)
To: dev; +Cc: Andrew Boyer
The receive side will enable scatter-gather if required based on the
mbuf size. If the client already enabled it in the config, it does
not need to be enabled again. This reduces log output.
Signed-off-by: Andrew Boyer <andrew.boyer@amd.com>
---
drivers/net/ionic/ionic_lif.c | 3 ++-
1 file changed, 2 insertions(+), 1 deletion(-)
diff --git a/drivers/net/ionic/ionic_lif.c b/drivers/net/ionic/ionic_lif.c
index 25b490deb6..fe2112c057 100644
--- a/drivers/net/ionic/ionic_lif.c
+++ b/drivers/net/ionic/ionic_lif.c
@@ -768,7 +768,8 @@ ionic_rx_qcq_alloc(struct ionic_lif *lif, uint32_t socket_id, uint32_t index,
max_mtu = rte_le_to_cpu_32(lif->adapter->ident.lif.eth.max_mtu);
/* If mbufs are too small to hold received packets, enable SG */
- if (max_mtu > hdr_seg_size) {
+ if (max_mtu > hdr_seg_size &&
+ !(lif->features & IONIC_ETH_HW_RX_SG)) {
IONIC_PRINT(NOTICE, "Enabling RX_OFFLOAD_SCATTER");
lif->eth_dev->data->dev_conf.rxmode.offloads |=
RTE_ETH_RX_OFFLOAD_SCATTER;
--
2.17.1
^ permalink raw reply [flat|nested] 52+ messages in thread
* [PATCH v2 04/13] net/ionic: fix missing volatile type for cqe pointers
2024-02-02 19:32 [PATCH 00/13] net/ionic: miscellaneous fixes and improvements Andrew Boyer
` (17 preceding siblings ...)
2024-02-07 2:18 ` [PATCH v2 03/13] net/ionic: don't auto-enable Rx scatter-gather a second time Andrew Boyer
@ 2024-02-07 2:18 ` Andrew Boyer
2024-02-07 2:18 ` [PATCH v2 05/13] net/ionic: replace non-standard type in structure definition Andrew Boyer
` (9 subsequent siblings)
28 siblings, 0 replies; 52+ messages in thread
From: Andrew Boyer @ 2024-02-07 2:18 UTC (permalink / raw)
To: dev; +Cc: Neel Patel, stable, Andrew Boyer
From: Neel Patel <neel.patel@amd.com>
This memory may be changed by the hardware, so the volatile
keyword is required for correctness.
Fixes: e86a6fcc7cf3 ("net/ionic: add optimized non-scattered Rx/Tx")
cc: stable@dpdk.org
Signed-off-by: Andrew Boyer <andrew.boyer@amd.com>
Signed-off-by: Neel Patel <neel.patel@amd.com>
---
drivers/net/ionic/ionic_rxtx.c | 4 ++--
drivers/net/ionic/ionic_rxtx_sg.c | 8 +++++---
drivers/net/ionic/ionic_rxtx_simple.c | 8 +++++---
3 files changed, 12 insertions(+), 8 deletions(-)
diff --git a/drivers/net/ionic/ionic_rxtx.c b/drivers/net/ionic/ionic_rxtx.c
index d92b231f8f..d92fa1cca7 100644
--- a/drivers/net/ionic/ionic_rxtx.c
+++ b/drivers/net/ionic/ionic_rxtx.c
@@ -755,7 +755,7 @@ ionic_dev_rx_descriptor_status(void *rx_queue, uint16_t offset)
{
struct ionic_rx_qcq *rxq = rx_queue;
struct ionic_qcq *qcq = &rxq->qcq;
- struct ionic_rxq_comp *cq_desc;
+ volatile struct ionic_rxq_comp *cq_desc;
uint16_t mask, head, tail, pos;
bool done_color;
@@ -794,7 +794,7 @@ ionic_dev_tx_descriptor_status(void *tx_queue, uint16_t offset)
{
struct ionic_tx_qcq *txq = tx_queue;
struct ionic_qcq *qcq = &txq->qcq;
- struct ionic_txq_comp *cq_desc;
+ volatile struct ionic_txq_comp *cq_desc;
uint16_t mask, head, tail, pos, cq_pos;
bool done_color;
diff --git a/drivers/net/ionic/ionic_rxtx_sg.c b/drivers/net/ionic/ionic_rxtx_sg.c
index 6c028a698c..1392342463 100644
--- a/drivers/net/ionic/ionic_rxtx_sg.c
+++ b/drivers/net/ionic/ionic_rxtx_sg.c
@@ -28,7 +28,8 @@ ionic_tx_flush_sg(struct ionic_tx_qcq *txq)
struct ionic_queue *q = &txq->qcq.q;
struct ionic_tx_stats *stats = &txq->stats;
struct rte_mbuf *txm;
- struct ionic_txq_comp *cq_desc, *cq_desc_base = cq->base;
+ struct ionic_txq_comp *cq_desc_base = cq->base;
+ volatile struct ionic_txq_comp *cq_desc;
void **info;
uint32_t i;
@@ -254,7 +255,7 @@ ionic_xmit_pkts_sg(void *tx_queue, struct rte_mbuf **tx_pkts,
*/
static __rte_always_inline void
ionic_rx_clean_one_sg(struct ionic_rx_qcq *rxq,
- struct ionic_rxq_comp *cq_desc,
+ volatile struct ionic_rxq_comp *cq_desc,
struct ionic_rx_service *rx_svc)
{
struct ionic_queue *q = &rxq->qcq.q;
@@ -440,7 +441,8 @@ ionic_rxq_service_sg(struct ionic_rx_qcq *rxq, uint32_t work_to_do,
struct ionic_cq *cq = &rxq->qcq.cq;
struct ionic_queue *q = &rxq->qcq.q;
struct ionic_rxq_desc *q_desc_base = q->base;
- struct ionic_rxq_comp *cq_desc, *cq_desc_base = cq->base;
+ struct ionic_rxq_comp *cq_desc_base = cq->base;
+ volatile struct ionic_rxq_comp *cq_desc;
uint32_t work_done = 0;
uint64_t then, now, hz, delta;
diff --git a/drivers/net/ionic/ionic_rxtx_simple.c b/drivers/net/ionic/ionic_rxtx_simple.c
index 5969287b66..00152c885a 100644
--- a/drivers/net/ionic/ionic_rxtx_simple.c
+++ b/drivers/net/ionic/ionic_rxtx_simple.c
@@ -28,7 +28,8 @@ ionic_tx_flush(struct ionic_tx_qcq *txq)
struct ionic_queue *q = &txq->qcq.q;
struct ionic_tx_stats *stats = &txq->stats;
struct rte_mbuf *txm;
- struct ionic_txq_comp *cq_desc, *cq_desc_base = cq->base;
+ struct ionic_txq_comp *cq_desc_base = cq->base;
+ volatile struct ionic_txq_comp *cq_desc;
void **info;
cq_desc = &cq_desc_base[cq->tail_idx];
@@ -227,7 +228,7 @@ ionic_xmit_pkts(void *tx_queue, struct rte_mbuf **tx_pkts,
*/
static __rte_always_inline void
ionic_rx_clean_one(struct ionic_rx_qcq *rxq,
- struct ionic_rxq_comp *cq_desc,
+ volatile struct ionic_rxq_comp *cq_desc,
struct ionic_rx_service *rx_svc)
{
struct ionic_queue *q = &rxq->qcq.q;
@@ -361,7 +362,8 @@ ionic_rxq_service(struct ionic_rx_qcq *rxq, uint32_t work_to_do,
struct ionic_cq *cq = &rxq->qcq.cq;
struct ionic_queue *q = &rxq->qcq.q;
struct ionic_rxq_desc *q_desc_base = q->base;
- struct ionic_rxq_comp *cq_desc, *cq_desc_base = cq->base;
+ struct ionic_rxq_comp *cq_desc_base = cq->base;
+ volatile struct ionic_rxq_comp *cq_desc;
uint32_t work_done = 0;
uint64_t then, now, hz, delta;
--
2.17.1
^ permalink raw reply [flat|nested] 52+ messages in thread
* [PATCH v2 05/13] net/ionic: replace non-standard type in structure definition
2024-02-02 19:32 [PATCH 00/13] net/ionic: miscellaneous fixes and improvements Andrew Boyer
` (18 preceding siblings ...)
2024-02-07 2:18 ` [PATCH v2 04/13] net/ionic: fix missing volatile type for cqe pointers Andrew Boyer
@ 2024-02-07 2:18 ` Andrew Boyer
2024-02-07 2:18 ` [PATCH v2 06/13] net/ionic: memcpy descriptors when using Q-in-CMB Andrew Boyer
` (8 subsequent siblings)
28 siblings, 0 replies; 52+ messages in thread
From: Andrew Boyer @ 2024-02-07 2:18 UTC (permalink / raw)
To: dev; +Cc: Andrew Boyer
Use uint8_t instead of u_char. This simplifies the code.
Signed-off-by: Andrew Boyer <andrew.boyer@amd.com>
---
drivers/net/ionic/ionic_dev_pci.c | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/drivers/net/ionic/ionic_dev_pci.c b/drivers/net/ionic/ionic_dev_pci.c
index 5e74a6da71..cbaac2c5bc 100644
--- a/drivers/net/ionic/ionic_dev_pci.c
+++ b/drivers/net/ionic/ionic_dev_pci.c
@@ -38,7 +38,7 @@ ionic_pci_setup(struct ionic_adapter *adapter)
struct ionic_dev *idev = &adapter->idev;
struct rte_pci_device *bus_dev = adapter->bus_dev;
uint32_t sig;
- u_char *bar0_base;
+ uint8_t *bar0_base;
unsigned int i;
/* BAR0: dev_cmd and interrupts */
--
2.17.1
^ permalink raw reply [flat|nested] 52+ messages in thread
* [PATCH v2 06/13] net/ionic: memcpy descriptors when using Q-in-CMB
2024-02-02 19:32 [PATCH 00/13] net/ionic: miscellaneous fixes and improvements Andrew Boyer
` (19 preceding siblings ...)
2024-02-07 2:18 ` [PATCH v2 05/13] net/ionic: replace non-standard type in structure definition Andrew Boyer
@ 2024-02-07 2:18 ` Andrew Boyer
2024-02-07 2:18 ` [PATCH v2 07/13] net/ionic: fix RSS query routine Andrew Boyer
` (7 subsequent siblings)
28 siblings, 0 replies; 52+ messages in thread
From: Andrew Boyer @ 2024-02-07 2:18 UTC (permalink / raw)
To: dev; +Cc: Neel Patel, Andrew Boyer
From: Neel Patel <neel.patel@amd.com>
They can be batched together this way, reducing the number
of PCIe transactions. This improves transmit PPS by up to 50% in
some configurations.
Signed-off-by: Andrew Boyer <andrew.boyer@amd.com>
Signed-off-by: Neel Patel <neel.patel@amd.com>
---
drivers/net/ionic/ionic_dev.c | 9 +++--
drivers/net/ionic/ionic_dev.h | 6 ++-
drivers/net/ionic/ionic_lif.c | 26 +++++++++----
drivers/net/ionic/ionic_rxtx.h | 56 +++++++++++++++++++++++++++
drivers/net/ionic/ionic_rxtx_sg.c | 18 ++++-----
drivers/net/ionic/ionic_rxtx_simple.c | 18 ++++-----
6 files changed, 101 insertions(+), 32 deletions(-)
diff --git a/drivers/net/ionic/ionic_dev.c b/drivers/net/ionic/ionic_dev.c
index 70c14882ed..7f15914f74 100644
--- a/drivers/net/ionic/ionic_dev.c
+++ b/drivers/net/ionic/ionic_dev.c
@@ -369,17 +369,19 @@ ionic_q_init(struct ionic_queue *q, uint32_t index, uint16_t num_descs)
q->index = index;
q->num_descs = num_descs;
q->size_mask = num_descs - 1;
- q->head_idx = 0;
- q->tail_idx = 0;
+ ionic_q_reset(q);
return 0;
}
void
-ionic_q_map(struct ionic_queue *q, void *base, rte_iova_t base_pa)
+ionic_q_map(struct ionic_queue *q, void *base, rte_iova_t base_pa,
+ void *cmb_base, rte_iova_t cmb_base_pa)
{
q->base = base;
q->base_pa = base_pa;
+ q->cmb_base = cmb_base;
+ q->cmb_base_pa = cmb_base_pa;
}
void
@@ -393,5 +395,6 @@ void
ionic_q_reset(struct ionic_queue *q)
{
q->head_idx = 0;
+ q->cmb_head_idx = 0;
q->tail_idx = 0;
}
diff --git a/drivers/net/ionic/ionic_dev.h b/drivers/net/ionic/ionic_dev.h
index 971c261b27..3a366247f1 100644
--- a/drivers/net/ionic/ionic_dev.h
+++ b/drivers/net/ionic/ionic_dev.h
@@ -145,11 +145,13 @@ struct ionic_queue {
uint16_t num_descs;
uint16_t num_segs;
uint16_t head_idx;
+ uint16_t cmb_head_idx;
uint16_t tail_idx;
uint16_t size_mask;
uint8_t type;
uint8_t hw_type;
void *base;
+ void *cmb_base;
void *sg_base;
struct ionic_doorbell __iomem *db;
void **info;
@@ -158,6 +160,7 @@ struct ionic_queue {
uint32_t hw_index;
rte_iova_t base_pa;
rte_iova_t sg_base_pa;
+ rte_iova_t cmb_base_pa;
};
#define IONIC_INTR_NONE (-1)
@@ -244,7 +247,8 @@ uint32_t ionic_cq_service(struct ionic_cq *cq, uint32_t work_to_do,
int ionic_q_init(struct ionic_queue *q, uint32_t index, uint16_t num_descs);
void ionic_q_reset(struct ionic_queue *q);
-void ionic_q_map(struct ionic_queue *q, void *base, rte_iova_t base_pa);
+void ionic_q_map(struct ionic_queue *q, void *base, rte_iova_t base_pa,
+ void *cmb_base, rte_iova_t cmb_base_pa);
void ionic_q_sg_map(struct ionic_queue *q, void *base, rte_iova_t base_pa);
static inline uint16_t
diff --git a/drivers/net/ionic/ionic_lif.c b/drivers/net/ionic/ionic_lif.c
index fe2112c057..2713f8aa24 100644
--- a/drivers/net/ionic/ionic_lif.c
+++ b/drivers/net/ionic/ionic_lif.c
@@ -572,10 +572,11 @@ ionic_qcq_alloc(struct ionic_lif *lif,
{
struct ionic_qcq *new;
uint32_t q_size, cq_size, sg_size, total_size;
- void *q_base, *cq_base, *sg_base;
+ void *q_base, *cmb_q_base, *cq_base, *sg_base;
rte_iova_t q_base_pa = 0;
rte_iova_t cq_base_pa = 0;
rte_iova_t sg_base_pa = 0;
+ rte_iova_t cmb_q_base_pa = 0;
size_t page_size = rte_mem_page_size();
int err;
@@ -666,19 +667,22 @@ ionic_qcq_alloc(struct ionic_lif *lif,
IONIC_PRINT(ERR, "Cannot reserve queue from NIC mem");
return -ENOMEM;
}
- q_base = (void *)
+ cmb_q_base = (void *)
((uintptr_t)lif->adapter->bars.bar[2].vaddr +
(uintptr_t)lif->adapter->cmb_offset);
/* CMB PA is a relative address */
- q_base_pa = lif->adapter->cmb_offset;
+ cmb_q_base_pa = lif->adapter->cmb_offset;
lif->adapter->cmb_offset += q_size;
+ } else {
+ cmb_q_base = NULL;
+ cmb_q_base_pa = 0;
}
IONIC_PRINT(DEBUG, "Q-Base-PA = %#jx CQ-Base-PA = %#jx "
"SG-base-PA = %#jx",
q_base_pa, cq_base_pa, sg_base_pa);
- ionic_q_map(&new->q, q_base, q_base_pa);
+ ionic_q_map(&new->q, q_base, q_base_pa, cmb_q_base, cmb_q_base_pa);
ionic_cq_map(&new->cq, cq_base, cq_base_pa);
*qcq = new;
@@ -1583,7 +1587,6 @@ ionic_lif_txq_init(struct ionic_tx_qcq *txq)
.flags = rte_cpu_to_le_16(IONIC_QINIT_F_ENA),
.intr_index = rte_cpu_to_le_16(IONIC_INTR_NONE),
.ring_size = rte_log2_u32(q->num_descs),
- .ring_base = rte_cpu_to_le_64(q->base_pa),
.cq_ring_base = rte_cpu_to_le_64(cq->base_pa),
.sg_ring_base = rte_cpu_to_le_64(q->sg_base_pa),
},
@@ -1592,8 +1595,12 @@ ionic_lif_txq_init(struct ionic_tx_qcq *txq)
if (txq->flags & IONIC_QCQ_F_SG)
ctx.cmd.q_init.flags |= rte_cpu_to_le_16(IONIC_QINIT_F_SG);
- if (txq->flags & IONIC_QCQ_F_CMB)
+ if (txq->flags & IONIC_QCQ_F_CMB) {
ctx.cmd.q_init.flags |= rte_cpu_to_le_16(IONIC_QINIT_F_CMB);
+ ctx.cmd.q_init.ring_base = rte_cpu_to_le_64(q->cmb_base_pa);
+ } else {
+ ctx.cmd.q_init.ring_base = rte_cpu_to_le_64(q->base_pa);
+ }
IONIC_PRINT(DEBUG, "txq_init.index %d", q->index);
IONIC_PRINT(DEBUG, "txq_init.ring_base 0x%" PRIx64 "", q->base_pa);
@@ -1638,7 +1645,6 @@ ionic_lif_rxq_init(struct ionic_rx_qcq *rxq)
.flags = rte_cpu_to_le_16(IONIC_QINIT_F_ENA),
.intr_index = rte_cpu_to_le_16(IONIC_INTR_NONE),
.ring_size = rte_log2_u32(q->num_descs),
- .ring_base = rte_cpu_to_le_64(q->base_pa),
.cq_ring_base = rte_cpu_to_le_64(cq->base_pa),
.sg_ring_base = rte_cpu_to_le_64(q->sg_base_pa),
},
@@ -1647,8 +1653,12 @@ ionic_lif_rxq_init(struct ionic_rx_qcq *rxq)
if (rxq->flags & IONIC_QCQ_F_SG)
ctx.cmd.q_init.flags |= rte_cpu_to_le_16(IONIC_QINIT_F_SG);
- if (rxq->flags & IONIC_QCQ_F_CMB)
+ if (rxq->flags & IONIC_QCQ_F_CMB) {
ctx.cmd.q_init.flags |= rte_cpu_to_le_16(IONIC_QINIT_F_CMB);
+ ctx.cmd.q_init.ring_base = rte_cpu_to_le_64(q->cmb_base_pa);
+ } else {
+ ctx.cmd.q_init.ring_base = rte_cpu_to_le_64(q->base_pa);
+ }
IONIC_PRINT(DEBUG, "rxq_init.index %d", q->index);
IONIC_PRINT(DEBUG, "rxq_init.ring_base 0x%" PRIx64 "", q->base_pa);
diff --git a/drivers/net/ionic/ionic_rxtx.h b/drivers/net/ionic/ionic_rxtx.h
index 8537141597..5348395956 100644
--- a/drivers/net/ionic/ionic_rxtx.h
+++ b/drivers/net/ionic/ionic_rxtx.h
@@ -77,4 +77,60 @@ uint16_t ionic_xmit_pkts_sg(void *tx_queue, struct rte_mbuf **tx_pkts,
int ionic_rx_fill_sg(struct ionic_rx_qcq *rxq);
+static inline void
+ionic_rxq_flush(struct ionic_queue *q)
+{
+ struct ionic_rxq_desc *desc_base = q->base;
+ struct ionic_rxq_desc *cmb_desc_base = q->cmb_base;
+
+ if (q->cmb_base) {
+ if (q->head_idx < q->cmb_head_idx) {
+ /* copy [cmb_head, num_descs) */
+ rte_memcpy((void *)&cmb_desc_base[q->cmb_head_idx],
+ (void *)&desc_base[q->cmb_head_idx],
+ (q->num_descs - q->cmb_head_idx) * sizeof(*desc_base));
+ /* copy [0, head) */
+ rte_memcpy((void *)&cmb_desc_base[0],
+ (void *)&desc_base[0],
+ q->head_idx * sizeof(*desc_base));
+ } else {
+ /* copy [cmb_head, head) */
+ rte_memcpy((void *)&cmb_desc_base[q->cmb_head_idx],
+ (void *)&desc_base[q->cmb_head_idx],
+ (q->head_idx - q->cmb_head_idx) * sizeof(*desc_base));
+ }
+ q->cmb_head_idx = q->head_idx;
+ }
+
+ ionic_q_flush(q);
+}
+
+static inline void
+ionic_txq_flush(struct ionic_queue *q)
+{
+ struct ionic_txq_desc *desc_base = q->base;
+ struct ionic_txq_desc *cmb_desc_base = q->cmb_base;
+
+ if (q->cmb_base) {
+ if (q->head_idx < q->cmb_head_idx) {
+ /* copy [cmb_head, num_descs) */
+ rte_memcpy((void *)&cmb_desc_base[q->cmb_head_idx],
+ (void *)&desc_base[q->cmb_head_idx],
+ (q->num_descs - q->cmb_head_idx) * sizeof(*desc_base));
+ /* copy [0, head) */
+ rte_memcpy((void *)&cmb_desc_base[0],
+ (void *)&desc_base[0],
+ q->head_idx * sizeof(*desc_base));
+ } else {
+ /* copy [cmb_head, head) */
+ rte_memcpy((void *)&cmb_desc_base[q->cmb_head_idx],
+ (void *)&desc_base[q->cmb_head_idx],
+ (q->head_idx - q->cmb_head_idx) * sizeof(*desc_base));
+ }
+ q->cmb_head_idx = q->head_idx;
+ }
+
+ ionic_q_flush(q);
+}
+
#endif /* _IONIC_RXTX_H_ */
diff --git a/drivers/net/ionic/ionic_rxtx_sg.c b/drivers/net/ionic/ionic_rxtx_sg.c
index 1392342463..92e1d6e259 100644
--- a/drivers/net/ionic/ionic_rxtx_sg.c
+++ b/drivers/net/ionic/ionic_rxtx_sg.c
@@ -166,6 +166,7 @@ ionic_xmit_pkts_sg(void *tx_queue, struct rte_mbuf **tx_pkts,
{
struct ionic_tx_qcq *txq = tx_queue;
struct ionic_queue *q = &txq->qcq.q;
+ struct ionic_txq_desc *desc_base = q->base;
struct ionic_tx_stats *stats = &txq->stats;
struct rte_mbuf *mbuf;
uint32_t bytes_tx = 0;
@@ -173,9 +174,7 @@ ionic_xmit_pkts_sg(void *tx_queue, struct rte_mbuf **tx_pkts,
uint64_t then, now, hz, delta;
int err;
- struct ionic_txq_desc *desc_base = q->base;
- if (!(txq->flags & IONIC_QCQ_F_CMB))
- rte_prefetch0(&desc_base[q->head_idx]);
+ rte_prefetch0(&desc_base[q->head_idx]);
rte_prefetch0(IONIC_INFO_PTR(q, q->head_idx));
if (nb_pkts) {
@@ -196,8 +195,7 @@ ionic_xmit_pkts_sg(void *tx_queue, struct rte_mbuf **tx_pkts,
while (nb_tx < nb_pkts) {
uint16_t next_idx = Q_NEXT_TO_POST(q, 1);
- if (!(txq->flags & IONIC_QCQ_F_CMB))
- rte_prefetch0(&desc_base[next_idx]);
+ rte_prefetch0(&desc_base[next_idx]);
rte_prefetch0(IONIC_INFO_PTR(q, next_idx));
if (nb_tx + 1 < nb_pkts) {
@@ -222,7 +220,7 @@ ionic_xmit_pkts_sg(void *tx_queue, struct rte_mbuf **tx_pkts,
if (nb_tx > 0) {
rte_wmb();
- ionic_q_flush(q);
+ ionic_txq_flush(q);
txq->last_wdog_cycles = rte_get_timer_cycles();
@@ -458,8 +456,7 @@ ionic_rxq_service_sg(struct ionic_rx_qcq *rxq, uint32_t work_to_do,
/* Prefetch 4 x 16B comp */
rte_prefetch0(&cq_desc_base[Q_NEXT_TO_SRVC(cq, 4)]);
/* Prefetch 4 x 16B descriptors */
- if (!(rxq->flags & IONIC_QCQ_F_CMB))
- rte_prefetch0(&q_desc_base[Q_NEXT_TO_POST(q, 4)]);
+ rte_prefetch0(&q_desc_base[Q_NEXT_TO_POST(q, 4)]);
/* Clean one descriptor */
ionic_rx_clean_one_sg(rxq, cq_desc, rx_svc);
@@ -478,7 +475,8 @@ ionic_rxq_service_sg(struct ionic_rx_qcq *rxq, uint32_t work_to_do,
/* Update the queue indices and ring the doorbell */
if (work_done) {
- ionic_q_flush(q);
+ ionic_rxq_flush(q);
+
rxq->last_wdog_cycles = rte_get_timer_cycles();
rxq->wdog_ms = IONIC_Q_WDOG_MS;
} else {
@@ -542,7 +540,7 @@ ionic_rx_fill_sg(struct ionic_rx_qcq *rxq)
q->head_idx = Q_NEXT_TO_POST(q, 1);
}
- ionic_q_flush(q);
+ ionic_rxq_flush(q);
return err;
}
diff --git a/drivers/net/ionic/ionic_rxtx_simple.c b/drivers/net/ionic/ionic_rxtx_simple.c
index 00152c885a..f12f66f40c 100644
--- a/drivers/net/ionic/ionic_rxtx_simple.c
+++ b/drivers/net/ionic/ionic_rxtx_simple.c
@@ -139,6 +139,7 @@ ionic_xmit_pkts(void *tx_queue, struct rte_mbuf **tx_pkts,
{
struct ionic_tx_qcq *txq = tx_queue;
struct ionic_queue *q = &txq->qcq.q;
+ struct ionic_txq_desc *desc_base = q->base;
struct ionic_tx_stats *stats = &txq->stats;
struct rte_mbuf *mbuf;
uint32_t bytes_tx = 0;
@@ -146,9 +147,7 @@ ionic_xmit_pkts(void *tx_queue, struct rte_mbuf **tx_pkts,
uint64_t then, now, hz, delta;
int err;
- struct ionic_txq_desc *desc_base = q->base;
- if (!(txq->flags & IONIC_QCQ_F_CMB))
- rte_prefetch0(&desc_base[q->head_idx]);
+ rte_prefetch0(&desc_base[q->head_idx]);
rte_prefetch0(&q->info[q->head_idx]);
if (nb_pkts) {
@@ -169,8 +168,7 @@ ionic_xmit_pkts(void *tx_queue, struct rte_mbuf **tx_pkts,
while (nb_tx < nb_pkts) {
uint16_t next_idx = Q_NEXT_TO_POST(q, 1);
- if (!(txq->flags & IONIC_QCQ_F_CMB))
- rte_prefetch0(&desc_base[next_idx]);
+ rte_prefetch0(&desc_base[next_idx]);
rte_prefetch0(&q->info[next_idx]);
if (nb_tx + 1 < nb_pkts) {
@@ -195,7 +193,7 @@ ionic_xmit_pkts(void *tx_queue, struct rte_mbuf **tx_pkts,
if (nb_tx > 0) {
rte_wmb();
- ionic_q_flush(q);
+ ionic_txq_flush(q);
txq->last_wdog_cycles = rte_get_timer_cycles();
@@ -379,8 +377,7 @@ ionic_rxq_service(struct ionic_rx_qcq *rxq, uint32_t work_to_do,
/* Prefetch 4 x 16B comp */
rte_prefetch0(&cq_desc_base[Q_NEXT_TO_SRVC(cq, 4)]);
/* Prefetch 4 x 16B descriptors */
- if (!(rxq->flags & IONIC_QCQ_F_CMB))
- rte_prefetch0(&q_desc_base[Q_NEXT_TO_POST(q, 4)]);
+ rte_prefetch0(&q_desc_base[Q_NEXT_TO_POST(q, 4)]);
/* Clean one descriptor */
ionic_rx_clean_one(rxq, cq_desc, rx_svc);
@@ -399,7 +396,8 @@ ionic_rxq_service(struct ionic_rx_qcq *rxq, uint32_t work_to_do,
/* Update the queue indices and ring the doorbell */
if (work_done) {
- ionic_q_flush(q);
+ ionic_rxq_flush(q);
+
rxq->last_wdog_cycles = rte_get_timer_cycles();
rxq->wdog_ms = IONIC_Q_WDOG_MS;
} else {
@@ -463,7 +461,7 @@ ionic_rx_fill(struct ionic_rx_qcq *rxq)
q->head_idx = Q_NEXT_TO_POST(q, 1);
}
- ionic_q_flush(q);
+ ionic_rxq_flush(q);
return err;
}
--
2.17.1
^ permalink raw reply [flat|nested] 52+ messages in thread
* [PATCH v2 07/13] net/ionic: fix RSS query routine
2024-02-02 19:32 [PATCH 00/13] net/ionic: miscellaneous fixes and improvements Andrew Boyer
` (20 preceding siblings ...)
2024-02-07 2:18 ` [PATCH v2 06/13] net/ionic: memcpy descriptors when using Q-in-CMB Andrew Boyer
@ 2024-02-07 2:18 ` Andrew Boyer
2024-02-07 2:18 ` [PATCH v2 08/13] net/ionic: report 1G and 200G link speeds when applicable Andrew Boyer
` (6 subsequent siblings)
28 siblings, 0 replies; 52+ messages in thread
From: Andrew Boyer @ 2024-02-07 2:18 UTC (permalink / raw)
To: dev; +Cc: Akshay Dorwat, cardigliano, stable, Andrew Boyer
From: Akshay Dorwat <akshay.dorwat@amd.com>
The routine that copies out the RSS config can't use memcpy() because
'reta_conf->reta' is an array of uint16_t while 'lif->rss_ind_tbl' is
an array of uint8_t. Instead, copy the values individually.
Fixes: 22e7171bc63b ("net/ionic: support RSS")
Cc: cardigliano@ntop.org
Cc: stable@dpdk.org
Signed-off-by: Akshay Dorwat <akshay.dorwat@amd.com>
Signed-off-by: Andrew Boyer <andrew.boyer@amd.com>
---
drivers/net/ionic/ionic_ethdev.c | 9 +++++----
1 file changed, 5 insertions(+), 4 deletions(-)
diff --git a/drivers/net/ionic/ionic_ethdev.c b/drivers/net/ionic/ionic_ethdev.c
index 340fd0cd59..008e50e0b9 100644
--- a/drivers/net/ionic/ionic_ethdev.c
+++ b/drivers/net/ionic/ionic_ethdev.c
@@ -561,7 +561,7 @@ ionic_dev_rss_reta_query(struct rte_eth_dev *eth_dev,
struct ionic_lif *lif = IONIC_ETH_DEV_TO_LIF(eth_dev);
struct ionic_adapter *adapter = lif->adapter;
struct ionic_identity *ident = &adapter->ident;
- int i, num;
+ int i, j, num;
uint16_t tbl_sz = rte_le_to_cpu_16(ident->lif.eth.rss_ind_tbl_sz);
IONIC_PRINT_CALL();
@@ -582,9 +582,10 @@ ionic_dev_rss_reta_query(struct rte_eth_dev *eth_dev,
num = reta_size / RTE_ETH_RETA_GROUP_SIZE;
for (i = 0; i < num; i++) {
- memcpy(reta_conf->reta,
- &lif->rss_ind_tbl[i * RTE_ETH_RETA_GROUP_SIZE],
- RTE_ETH_RETA_GROUP_SIZE);
+ for (j = 0; j < RTE_ETH_RETA_GROUP_SIZE; j++) {
+ reta_conf->reta[j] =
+ lif->rss_ind_tbl[(i * RTE_ETH_RETA_GROUP_SIZE) + j];
+ }
reta_conf++;
}
--
2.17.1
^ permalink raw reply [flat|nested] 52+ messages in thread
* [PATCH v2 08/13] net/ionic: report 1G and 200G link speeds when applicable
2024-02-02 19:32 [PATCH 00/13] net/ionic: miscellaneous fixes and improvements Andrew Boyer
` (21 preceding siblings ...)
2024-02-07 2:18 ` [PATCH v2 07/13] net/ionic: fix RSS query routine Andrew Boyer
@ 2024-02-07 2:18 ` Andrew Boyer
2024-02-07 2:18 ` [PATCH v2 09/13] net/ionic: add flexible firmware xstat counters Andrew Boyer
` (5 subsequent siblings)
28 siblings, 0 replies; 52+ messages in thread
From: Andrew Boyer @ 2024-02-07 2:18 UTC (permalink / raw)
To: dev; +Cc: Vamsi Krishna Atluri, Andrew Boyer
From: Vamsi Krishna Atluri <vamsi.atluri@amd.com>
The hardware supports these speeds, so we should report them
correctly.
Signed-off-by: Andrew Boyer <andrew.boyer@amd.com>
Signed-off-by: Vamsi Krishna Atluri <vamsi.atluri@amd.com>
---
drivers/net/ionic/ionic_ethdev.c | 6 ++++++
1 file changed, 6 insertions(+)
diff --git a/drivers/net/ionic/ionic_ethdev.c b/drivers/net/ionic/ionic_ethdev.c
index 008e50e0b9..327f6b9de5 100644
--- a/drivers/net/ionic/ionic_ethdev.c
+++ b/drivers/net/ionic/ionic_ethdev.c
@@ -285,6 +285,9 @@ ionic_dev_link_update(struct rte_eth_dev *eth_dev,
link.link_status = RTE_ETH_LINK_UP;
link.link_duplex = RTE_ETH_LINK_FULL_DUPLEX;
switch (adapter->link_speed) {
+ case 1000:
+ link.link_speed = RTE_ETH_SPEED_NUM_1G;
+ break;
case 10000:
link.link_speed = RTE_ETH_SPEED_NUM_10G;
break;
@@ -300,6 +303,9 @@ ionic_dev_link_update(struct rte_eth_dev *eth_dev,
case 100000:
link.link_speed = RTE_ETH_SPEED_NUM_100G;
break;
+ case 200000:
+ link.link_speed = RTE_ETH_SPEED_NUM_200G;
+ break;
default:
link.link_speed = RTE_ETH_SPEED_NUM_NONE;
break;
--
2.17.1
^ permalink raw reply [flat|nested] 52+ messages in thread
* [PATCH v2 09/13] net/ionic: add flexible firmware xstat counters
2024-02-02 19:32 [PATCH 00/13] net/ionic: miscellaneous fixes and improvements Andrew Boyer
` (22 preceding siblings ...)
2024-02-07 2:18 ` [PATCH v2 08/13] net/ionic: report 1G and 200G link speeds when applicable Andrew Boyer
@ 2024-02-07 2:18 ` Andrew Boyer
2024-02-07 2:18 ` [PATCH v2 10/13] net/ionic: fix device close sequence to avoid crash Andrew Boyer
` (4 subsequent siblings)
28 siblings, 0 replies; 52+ messages in thread
From: Andrew Boyer @ 2024-02-07 2:18 UTC (permalink / raw)
To: dev; +Cc: Brad Larson, Andrew Boyer
From: Brad Larson <bradley.larson@amd.com>
Assign 32 counters for flexible firmware events. These can be used as
per-port or per-queue counters in certain firmware configurations.
They are displayed as fw_flex_eventX in xstats.
Signed-off-by: Andrew Boyer <andrew.boyer@amd.com>
Signed-off-by: Brad Larson <bradley.larson@amd.com>
---
drivers/net/ionic/ionic_ethdev.c | 33 +++++++++++++++
drivers/net/ionic/ionic_if.h | 70 ++++++++++++++++----------------
2 files changed, 68 insertions(+), 35 deletions(-)
diff --git a/drivers/net/ionic/ionic_ethdev.c b/drivers/net/ionic/ionic_ethdev.c
index 327f6b9de5..7c55a26956 100644
--- a/drivers/net/ionic/ionic_ethdev.c
+++ b/drivers/net/ionic/ionic_ethdev.c
@@ -196,6 +196,39 @@ static const struct rte_ionic_xstats_name_off rte_ionic_xstats_strings[] = {
tx_desc_fetch_error)},
{"tx_desc_data_error", offsetof(struct ionic_lif_stats,
tx_desc_data_error)},
+ /* Flexible firmware events */
+ {"fw_flex_event1", offsetof(struct ionic_lif_stats, flex1)},
+ {"fw_flex_event2", offsetof(struct ionic_lif_stats, flex2)},
+ {"fw_flex_event3", offsetof(struct ionic_lif_stats, flex3)},
+ {"fw_flex_event4", offsetof(struct ionic_lif_stats, flex4)},
+ {"fw_flex_event5", offsetof(struct ionic_lif_stats, flex5)},
+ {"fw_flex_event6", offsetof(struct ionic_lif_stats, flex6)},
+ {"fw_flex_event7", offsetof(struct ionic_lif_stats, flex7)},
+ {"fw_flex_event8", offsetof(struct ionic_lif_stats, flex8)},
+ {"fw_flex_event9", offsetof(struct ionic_lif_stats, flex9)},
+ {"fw_flex_event10", offsetof(struct ionic_lif_stats, flex10)},
+ {"fw_flex_event11", offsetof(struct ionic_lif_stats, flex11)},
+ {"fw_flex_event12", offsetof(struct ionic_lif_stats, flex12)},
+ {"fw_flex_event13", offsetof(struct ionic_lif_stats, flex13)},
+ {"fw_flex_event14", offsetof(struct ionic_lif_stats, flex14)},
+ {"fw_flex_event15", offsetof(struct ionic_lif_stats, flex15)},
+ {"fw_flex_event16", offsetof(struct ionic_lif_stats, flex16)},
+ {"fw_flex_event17", offsetof(struct ionic_lif_stats, flex17)},
+ {"fw_flex_event18", offsetof(struct ionic_lif_stats, flex18)},
+ {"fw_flex_event19", offsetof(struct ionic_lif_stats, flex19)},
+ {"fw_flex_event20", offsetof(struct ionic_lif_stats, flex20)},
+ {"fw_flex_event21", offsetof(struct ionic_lif_stats, flex21)},
+ {"fw_flex_event22", offsetof(struct ionic_lif_stats, flex22)},
+ {"fw_flex_event23", offsetof(struct ionic_lif_stats, flex23)},
+ {"fw_flex_event24", offsetof(struct ionic_lif_stats, flex24)},
+ {"fw_flex_event25", offsetof(struct ionic_lif_stats, flex25)},
+ {"fw_flex_event26", offsetof(struct ionic_lif_stats, flex26)},
+ {"fw_flex_event27", offsetof(struct ionic_lif_stats, flex27)},
+ {"fw_flex_event28", offsetof(struct ionic_lif_stats, flex28)},
+ {"fw_flex_event29", offsetof(struct ionic_lif_stats, flex29)},
+ {"fw_flex_event30", offsetof(struct ionic_lif_stats, flex30)},
+ {"fw_flex_event31", offsetof(struct ionic_lif_stats, flex31)},
+ {"fw_flex_event32", offsetof(struct ionic_lif_stats, flex32)},
};
#define IONIC_NB_HW_STATS RTE_DIM(rte_ionic_xstats_strings)
diff --git a/drivers/net/ionic/ionic_if.h b/drivers/net/ionic/ionic_if.h
index 79aa196345..7ca604a7bb 100644
--- a/drivers/net/ionic/ionic_if.h
+++ b/drivers/net/ionic/ionic_if.h
@@ -2592,41 +2592,41 @@ struct ionic_lif_stats {
__le64 rsvd16;
__le64 rsvd17;
- __le64 rsvd18;
- __le64 rsvd19;
- __le64 rsvd20;
- __le64 rsvd21;
- __le64 rsvd22;
- __le64 rsvd23;
- __le64 rsvd24;
- __le64 rsvd25;
-
- __le64 rsvd26;
- __le64 rsvd27;
- __le64 rsvd28;
- __le64 rsvd29;
- __le64 rsvd30;
- __le64 rsvd31;
- __le64 rsvd32;
- __le64 rsvd33;
-
- __le64 rsvd34;
- __le64 rsvd35;
- __le64 rsvd36;
- __le64 rsvd37;
- __le64 rsvd38;
- __le64 rsvd39;
- __le64 rsvd40;
- __le64 rsvd41;
-
- __le64 rsvd42;
- __le64 rsvd43;
- __le64 rsvd44;
- __le64 rsvd45;
- __le64 rsvd46;
- __le64 rsvd47;
- __le64 rsvd48;
- __le64 rsvd49;
+ __le64 flex1;
+ __le64 flex2;
+ __le64 flex3;
+ __le64 flex4;
+ __le64 flex5;
+ __le64 flex6;
+ __le64 flex7;
+ __le64 flex8;
+
+ __le64 flex9;
+ __le64 flex10;
+ __le64 flex11;
+ __le64 flex12;
+ __le64 flex13;
+ __le64 flex14;
+ __le64 flex15;
+ __le64 flex16;
+
+ __le64 flex17;
+ __le64 flex18;
+ __le64 flex19;
+ __le64 flex20;
+ __le64 flex21;
+ __le64 flex22;
+ __le64 flex23;
+ __le64 flex24;
+
+ __le64 flex25;
+ __le64 flex26;
+ __le64 flex27;
+ __le64 flex28;
+ __le64 flex29;
+ __le64 flex30;
+ __le64 flex31;
+ __le64 flex32;
/* RDMA/ROCE REQ Error/Debugs (768 - 895) */
__le64 rdma_req_rx_pkt_seq_err;
--
2.17.1
^ permalink raw reply [flat|nested] 52+ messages in thread
* [PATCH v2 10/13] net/ionic: fix device close sequence to avoid crash
2024-02-02 19:32 [PATCH 00/13] net/ionic: miscellaneous fixes and improvements Andrew Boyer
` (23 preceding siblings ...)
2024-02-07 2:18 ` [PATCH v2 09/13] net/ionic: add flexible firmware xstat counters Andrew Boyer
@ 2024-02-07 2:18 ` Andrew Boyer
2024-02-07 2:18 ` [PATCH v2 11/13] net/ionic: optimize device close operation Andrew Boyer
` (3 subsequent siblings)
28 siblings, 0 replies; 52+ messages in thread
From: Andrew Boyer @ 2024-02-07 2:18 UTC (permalink / raw)
To: dev; +Cc: Andrew Boyer, stable
The close routine should release all resources, but not
call rte_eth_dev_destroy(). As written this code will call
rte_eth_dev_release_port() twice and segfault.
Instead, move rte_eth_dev_destroy() to the remove routine.
eth_ionic_dev_uninit() will call close if necessary.
Fixes: 175e4e7ed760 ("net/ionic: complete release on close")
Cc: stable@dpdk.org
Signed-off-by: Andrew Boyer <andrew.boyer@amd.com>
---
drivers/net/ionic/ionic_ethdev.c | 30 +++++++++++++++---------------
1 file changed, 15 insertions(+), 15 deletions(-)
diff --git a/drivers/net/ionic/ionic_ethdev.c b/drivers/net/ionic/ionic_ethdev.c
index 7c55a26956..bedcf958e2 100644
--- a/drivers/net/ionic/ionic_ethdev.c
+++ b/drivers/net/ionic/ionic_ethdev.c
@@ -1009,19 +1009,21 @@ ionic_dev_close(struct rte_eth_dev *eth_dev)
ionic_lif_stop(lif);
- ionic_lif_free_queues(lif);
-
IONIC_PRINT(NOTICE, "Removing device %s", eth_dev->device->name);
if (adapter->intf->unconfigure_intr)
(*adapter->intf->unconfigure_intr)(adapter);
- rte_eth_dev_destroy(eth_dev, eth_ionic_dev_uninit);
-
ionic_port_reset(adapter);
ionic_reset(adapter);
+
+ ionic_lif_free_queues(lif);
+ ionic_lif_deinit(lif);
+ ionic_lif_free(lif); /* Does not free LIF object */
+
if (adapter->intf->unmap_bars)
(*adapter->intf->unmap_bars)(adapter);
+ lif->adapter = NULL;
rte_free(adapter);
return 0;
@@ -1098,21 +1100,18 @@ eth_ionic_dev_init(struct rte_eth_dev *eth_dev, void *init_params)
static int
eth_ionic_dev_uninit(struct rte_eth_dev *eth_dev)
{
- struct ionic_lif *lif = IONIC_ETH_DEV_TO_LIF(eth_dev);
- struct ionic_adapter *adapter = lif->adapter;
-
IONIC_PRINT_CALL();
if (rte_eal_process_type() != RTE_PROC_PRIMARY)
return 0;
- adapter->lif = NULL;
-
- ionic_lif_deinit(lif);
- ionic_lif_free(lif);
+ if (eth_dev->state != RTE_ETH_DEV_UNUSED)
+ ionic_dev_close(eth_dev);
- if (!(lif->state & IONIC_LIF_F_FW_RESET))
- ionic_lif_reset(lif);
+ eth_dev->dev_ops = NULL;
+ eth_dev->rx_pkt_burst = NULL;
+ eth_dev->tx_pkt_burst = NULL;
+ eth_dev->tx_pkt_prepare = NULL;
return 0;
}
@@ -1267,17 +1266,18 @@ eth_ionic_dev_remove(struct rte_device *rte_dev)
{
char name[RTE_ETH_NAME_MAX_LEN];
struct rte_eth_dev *eth_dev;
+ int ret = 0;
/* Adapter lookup is using the eth_dev name */
snprintf(name, sizeof(name), "%s_lif", rte_dev->name);
eth_dev = rte_eth_dev_allocated(name);
if (eth_dev)
- ionic_dev_close(eth_dev);
+ ret = rte_eth_dev_destroy(eth_dev, eth_ionic_dev_uninit);
else
IONIC_PRINT(DEBUG, "Cannot find device %s", rte_dev->name);
- return 0;
+ return ret;
}
RTE_LOG_REGISTER_DEFAULT(ionic_logtype, NOTICE);
--
2.17.1
^ permalink raw reply [flat|nested] 52+ messages in thread
* [PATCH v2 11/13] net/ionic: optimize device close operation
2024-02-02 19:32 [PATCH 00/13] net/ionic: miscellaneous fixes and improvements Andrew Boyer
` (24 preceding siblings ...)
2024-02-07 2:18 ` [PATCH v2 10/13] net/ionic: fix device close sequence to avoid crash Andrew Boyer
@ 2024-02-07 2:18 ` Andrew Boyer
2024-02-07 2:18 ` [PATCH v2 12/13] net/ionic: optimize device stop operation Andrew Boyer
` (2 subsequent siblings)
28 siblings, 0 replies; 52+ messages in thread
From: Andrew Boyer @ 2024-02-07 2:18 UTC (permalink / raw)
To: dev; +Cc: Andrew Boyer
Use a single device reset command to speed up dev_close(). The LIF stop
and port reset commands are not needed.
This reduces the outage window when restarting the process by about 2ms
plus another 1ms per queue.
Signed-off-by: Andrew Boyer <andrew.boyer@amd.com>
---
drivers/net/ionic/ionic_ethdev.c | 3 ---
drivers/net/ionic/ionic_lif.c | 8 +-------
2 files changed, 1 insertion(+), 10 deletions(-)
diff --git a/drivers/net/ionic/ionic_ethdev.c b/drivers/net/ionic/ionic_ethdev.c
index bedcf958e2..7e80751846 100644
--- a/drivers/net/ionic/ionic_ethdev.c
+++ b/drivers/net/ionic/ionic_ethdev.c
@@ -1007,13 +1007,10 @@ ionic_dev_close(struct rte_eth_dev *eth_dev)
if (rte_eal_process_type() != RTE_PROC_PRIMARY)
return 0;
- ionic_lif_stop(lif);
-
IONIC_PRINT(NOTICE, "Removing device %s", eth_dev->device->name);
if (adapter->intf->unconfigure_intr)
(*adapter->intf->unconfigure_intr)(adapter);
- ionic_port_reset(adapter);
ionic_reset(adapter);
ionic_lif_free_queues(lif);
diff --git a/drivers/net/ionic/ionic_lif.c b/drivers/net/ionic/ionic_lif.c
index 2713f8aa24..90efcc8cbb 100644
--- a/drivers/net/ionic/ionic_lif.c
+++ b/drivers/net/ionic/ionic_lif.c
@@ -1231,13 +1231,7 @@ ionic_lif_rss_setup(struct ionic_lif *lif)
static void
ionic_lif_rss_teardown(struct ionic_lif *lif)
{
- if (!lif->rss_ind_tbl)
- return;
-
- if (lif->rss_ind_tbl_z) {
- /* Disable RSS on the NIC */
- ionic_lif_rss_config(lif, 0x0, NULL, NULL);
-
+ if (lif->rss_ind_tbl) {
lif->rss_ind_tbl = NULL;
lif->rss_ind_tbl_pa = 0;
rte_memzone_free(lif->rss_ind_tbl_z);
--
2.17.1
^ permalink raw reply [flat|nested] 52+ messages in thread
* [PATCH v2 12/13] net/ionic: optimize device stop operation
2024-02-02 19:32 [PATCH 00/13] net/ionic: miscellaneous fixes and improvements Andrew Boyer
` (25 preceding siblings ...)
2024-02-07 2:18 ` [PATCH v2 11/13] net/ionic: optimize device close operation Andrew Boyer
@ 2024-02-07 2:18 ` Andrew Boyer
2024-02-07 2:18 ` [PATCH v2 13/13] net/ionic: optimize device start operation Andrew Boyer
2024-02-07 3:13 ` [PATCH v3 00/13] net/ionic: miscellaneous fixes and improvements Andrew Boyer
28 siblings, 0 replies; 52+ messages in thread
From: Andrew Boyer @ 2024-02-07 2:18 UTC (permalink / raw)
To: dev; +Cc: Andrew Boyer
Split the queue_stop operation into first-half and second-half helpers.
Move the command context from the stack into each Rx/Tx queue struct.
Expose some needed adminq interfaces.
This allows us to batch up the queue commands during dev_stop(), reducing
the outage window when restarting the process by about 1ms per queue.
Signed-off-by: Andrew Boyer <andrew.boyer@amd.com>
---
drivers/net/ionic/ionic.h | 3 ++
drivers/net/ionic/ionic_lif.c | 68 ++++++++++++++++++++++-------
drivers/net/ionic/ionic_lif.h | 12 ++++--
drivers/net/ionic/ionic_main.c | 17 +++++++-
drivers/net/ionic/ionic_rxtx.c | 78 ++++++++++++++++++++--------------
drivers/net/ionic/ionic_rxtx.h | 14 +++++-
6 files changed, 138 insertions(+), 54 deletions(-)
diff --git a/drivers/net/ionic/ionic.h b/drivers/net/ionic/ionic.h
index c479eaba74..cb4ea450a9 100644
--- a/drivers/net/ionic/ionic.h
+++ b/drivers/net/ionic/ionic.h
@@ -83,7 +83,10 @@ struct ionic_admin_ctx {
union ionic_adminq_comp comp;
};
+int ionic_adminq_post(struct ionic_lif *lif, struct ionic_admin_ctx *ctx);
int ionic_adminq_post_wait(struct ionic_lif *lif, struct ionic_admin_ctx *ctx);
+int ionic_adminq_wait(struct ionic_lif *lif, struct ionic_admin_ctx *ctx);
+uint16_t ionic_adminq_space_avail(struct ionic_lif *lif);
int ionic_dev_cmd_wait_check(struct ionic_dev *idev, unsigned long max_wait);
int ionic_setup(struct ionic_adapter *adapter);
diff --git a/drivers/net/ionic/ionic_lif.c b/drivers/net/ionic/ionic_lif.c
index 90efcc8cbb..45317590fa 100644
--- a/drivers/net/ionic/ionic_lif.c
+++ b/drivers/net/ionic/ionic_lif.c
@@ -31,11 +31,15 @@ static int ionic_lif_addr_add(struct ionic_lif *lif, const uint8_t *addr);
static int ionic_lif_addr_del(struct ionic_lif *lif, const uint8_t *addr);
static int
-ionic_qcq_disable(struct ionic_qcq *qcq)
+ionic_qcq_disable_nowait(struct ionic_qcq *qcq,
+ struct ionic_admin_ctx *ctx)
{
+ int err;
+
struct ionic_queue *q = &qcq->q;
struct ionic_lif *lif = qcq->lif;
- struct ionic_admin_ctx ctx = {
+
+ *ctx = (struct ionic_admin_ctx) {
.pending_work = true,
.cmd.q_control = {
.opcode = IONIC_CMD_Q_CONTROL,
@@ -45,28 +49,39 @@ ionic_qcq_disable(struct ionic_qcq *qcq)
},
};
- return ionic_adminq_post_wait(lif, &ctx);
+ /* Does not wait for command completion */
+ err = ionic_adminq_post(lif, ctx);
+ if (err)
+ ctx->pending_work = false;
+ return err;
}
void
ionic_lif_stop(struct ionic_lif *lif)
{
- uint32_t i;
+ struct rte_eth_dev *dev = lif->eth_dev;
+ uint32_t i, j, chunk;
IONIC_PRINT_CALL();
lif->state &= ~IONIC_LIF_F_UP;
- for (i = 0; i < lif->nrxqcqs; i++) {
- struct ionic_rx_qcq *rxq = lif->rxqcqs[i];
- if (rxq->flags & IONIC_QCQ_F_INITED)
- (void)ionic_dev_rx_queue_stop(lif->eth_dev, i);
+ chunk = ionic_adminq_space_avail(lif);
+
+ for (i = 0; i < lif->nrxqcqs; i += chunk) {
+ for (j = 0; j < chunk && i + j < lif->nrxqcqs; j++)
+ ionic_dev_rx_queue_stop_firsthalf(dev, i + j);
+
+ for (j = 0; j < chunk && i + j < lif->nrxqcqs; j++)
+ ionic_dev_rx_queue_stop_secondhalf(dev, i + j);
}
- for (i = 0; i < lif->ntxqcqs; i++) {
- struct ionic_tx_qcq *txq = lif->txqcqs[i];
- if (txq->flags & IONIC_QCQ_F_INITED)
- (void)ionic_dev_tx_queue_stop(lif->eth_dev, i);
+ for (i = 0; i < lif->ntxqcqs; i += chunk) {
+ for (j = 0; j < chunk && i + j < lif->ntxqcqs; j++)
+ ionic_dev_tx_queue_stop_firsthalf(dev, i + j);
+
+ for (j = 0; j < chunk && i + j < lif->ntxqcqs; j++)
+ ionic_dev_tx_queue_stop_secondhalf(dev, i + j);
}
}
@@ -1240,21 +1255,42 @@ ionic_lif_rss_teardown(struct ionic_lif *lif)
}
void
-ionic_lif_txq_deinit(struct ionic_tx_qcq *txq)
+ionic_lif_txq_deinit_nowait(struct ionic_tx_qcq *txq)
{
- ionic_qcq_disable(&txq->qcq);
+ ionic_qcq_disable_nowait(&txq->qcq, &txq->admin_ctx);
txq->flags &= ~IONIC_QCQ_F_INITED;
}
void
-ionic_lif_rxq_deinit(struct ionic_rx_qcq *rxq)
+ionic_lif_txq_stats(struct ionic_tx_qcq *txq)
+{
+ struct ionic_tx_stats *stats = &txq->stats;
+
+ IONIC_PRINT(DEBUG, "TX queue %u pkts %ju tso %ju",
+ txq->qcq.q.index, stats->packets, stats->tso);
+ IONIC_PRINT(DEBUG, "TX queue %u comps %ju (%ju per)",
+ txq->qcq.q.index, stats->comps,
+ stats->comps ? stats->packets / stats->comps : 0);
+}
+
+void
+ionic_lif_rxq_deinit_nowait(struct ionic_rx_qcq *rxq)
{
- ionic_qcq_disable(&rxq->qcq);
+ ionic_qcq_disable_nowait(&rxq->qcq, &rxq->admin_ctx);
rxq->flags &= ~IONIC_QCQ_F_INITED;
}
+void
+ionic_lif_rxq_stats(struct ionic_rx_qcq *rxq)
+{
+ struct ionic_rx_stats *stats = &rxq->stats;
+
+ IONIC_PRINT(DEBUG, "RX queue %u pkts %ju mtod %ju",
+ rxq->qcq.q.index, stats->packets, stats->mtods);
+}
+
static void
ionic_lif_adminq_deinit(struct ionic_lif *lif)
{
diff --git a/drivers/net/ionic/ionic_lif.h b/drivers/net/ionic/ionic_lif.h
index cac7a4583b..ee13f5b7c8 100644
--- a/drivers/net/ionic/ionic_lif.h
+++ b/drivers/net/ionic/ionic_lif.h
@@ -10,7 +10,7 @@
#include <rte_ethdev.h>
#include <rte_ether.h>
-#include "ionic_osdep.h"
+#include "ionic.h"
#include "ionic_dev.h"
#include "ionic_rx_filter.h"
@@ -99,6 +99,8 @@ struct ionic_rx_qcq {
/* cacheline4+ */
struct rte_mbuf *mbs[IONIC_MBUF_BULK_ALLOC] __rte_cache_aligned;
+
+ struct ionic_admin_ctx admin_ctx;
};
struct ionic_tx_qcq {
@@ -112,6 +114,8 @@ struct ionic_tx_qcq {
uint16_t flags;
struct ionic_tx_stats stats;
+
+ struct ionic_admin_ctx admin_ctx;
};
#define IONIC_Q_TO_QCQ(_q) container_of(_q, struct ionic_qcq, q)
@@ -225,10 +229,12 @@ int ionic_tx_qcq_alloc(struct ionic_lif *lif, uint32_t socket_id,
void ionic_qcq_free(struct ionic_qcq *qcq);
int ionic_lif_rxq_init(struct ionic_rx_qcq *rxq);
-void ionic_lif_rxq_deinit(struct ionic_rx_qcq *rxq);
+void ionic_lif_rxq_deinit_nowait(struct ionic_rx_qcq *rxq);
+void ionic_lif_rxq_stats(struct ionic_rx_qcq *rxq);
int ionic_lif_txq_init(struct ionic_tx_qcq *txq);
-void ionic_lif_txq_deinit(struct ionic_tx_qcq *txq);
+void ionic_lif_txq_deinit_nowait(struct ionic_tx_qcq *txq);
+void ionic_lif_txq_stats(struct ionic_tx_qcq *txq);
int ionic_lif_rss_config(struct ionic_lif *lif, const uint16_t types,
const uint8_t *key, const uint32_t *indir);
diff --git a/drivers/net/ionic/ionic_main.c b/drivers/net/ionic/ionic_main.c
index c957d55bf9..1f24f64a33 100644
--- a/drivers/net/ionic/ionic_main.c
+++ b/drivers/net/ionic/ionic_main.c
@@ -180,6 +180,12 @@ ionic_adminq_service(struct ionic_cq *cq, uint16_t cq_desc_index,
return true;
}
+uint16_t
+ionic_adminq_space_avail(struct ionic_lif *lif)
+{
+ return ionic_q_space_avail(&lif->adminqcq->qcq.q);
+}
+
/** ionic_adminq_post - Post an admin command.
* @lif: Handle to lif.
* @cmd_ctx: Api admin command context.
@@ -191,7 +197,7 @@ ionic_adminq_service(struct ionic_cq *cq, uint16_t cq_desc_index,
*
* Return: zero or negative error status.
*/
-static int
+int
ionic_adminq_post(struct ionic_lif *lif, struct ionic_admin_ctx *ctx)
{
struct ionic_queue *q = &lif->adminqcq->qcq.q;
@@ -279,7 +285,6 @@ ionic_adminq_wait_for_completion(struct ionic_lif *lif,
int
ionic_adminq_post_wait(struct ionic_lif *lif, struct ionic_admin_ctx *ctx)
{
- bool done;
int err;
IONIC_PRINT(DEBUG, "Sending %s (%d) via the admin queue",
@@ -292,6 +297,14 @@ ionic_adminq_post_wait(struct ionic_lif *lif, struct ionic_admin_ctx *ctx)
return err;
}
+ return ionic_adminq_wait(lif, ctx);
+}
+
+int
+ionic_adminq_wait(struct ionic_lif *lif, struct ionic_admin_ctx *ctx)
+{
+ bool done;
+
done = ionic_adminq_wait_for_completion(lif, ctx,
IONIC_DEVCMD_TIMEOUT);
diff --git a/drivers/net/ionic/ionic_rxtx.c b/drivers/net/ionic/ionic_rxtx.c
index d92fa1cca7..774dc596c0 100644
--- a/drivers/net/ionic/ionic_rxtx.c
+++ b/drivers/net/ionic/ionic_rxtx.c
@@ -92,36 +92,40 @@ ionic_dev_tx_queue_release(struct rte_eth_dev *dev, uint16_t qid)
}
int __rte_cold
-ionic_dev_tx_queue_stop(struct rte_eth_dev *eth_dev, uint16_t tx_queue_id)
+ionic_dev_tx_queue_stop(struct rte_eth_dev *dev, uint16_t tx_queue_id)
{
- struct ionic_tx_stats *stats;
- struct ionic_tx_qcq *txq;
+ ionic_dev_tx_queue_stop_firsthalf(dev, tx_queue_id);
+ ionic_dev_tx_queue_stop_secondhalf(dev, tx_queue_id);
+
+ return 0;
+}
+
+void __rte_cold
+ionic_dev_tx_queue_stop_firsthalf(struct rte_eth_dev *dev,
+ uint16_t tx_queue_id)
+{
+ struct ionic_tx_qcq *txq = dev->data->tx_queues[tx_queue_id];
IONIC_PRINT(DEBUG, "Stopping TX queue %u", tx_queue_id);
- txq = eth_dev->data->tx_queues[tx_queue_id];
+ dev->data->tx_queue_state[tx_queue_id] = RTE_ETH_QUEUE_STATE_STOPPED;
- eth_dev->data->tx_queue_state[tx_queue_id] =
- RTE_ETH_QUEUE_STATE_STOPPED;
+ ionic_lif_txq_deinit_nowait(txq);
+}
- /*
- * Note: we should better post NOP Tx desc and wait for its completion
- * before disabling Tx queue
- */
+void __rte_cold
+ionic_dev_tx_queue_stop_secondhalf(struct rte_eth_dev *dev,
+ uint16_t tx_queue_id)
+{
+ struct ionic_lif *lif = IONIC_ETH_DEV_TO_LIF(dev);
+ struct ionic_tx_qcq *txq = dev->data->tx_queues[tx_queue_id];
- ionic_lif_txq_deinit(txq);
+ ionic_adminq_wait(lif, &txq->admin_ctx);
/* Free all buffers from descriptor ring */
ionic_tx_empty(txq);
- stats = &txq->stats;
- IONIC_PRINT(DEBUG, "TX queue %u pkts %ju tso %ju",
- txq->qcq.q.index, stats->packets, stats->tso);
- IONIC_PRINT(DEBUG, "TX queue %u comps %ju (%ju per)",
- txq->qcq.q.index, stats->comps,
- stats->comps ? stats->packets / stats->comps : 0);
-
- return 0;
+ ionic_lif_txq_stats(txq);
}
int __rte_cold
@@ -726,28 +730,40 @@ ionic_dev_rx_queue_start(struct rte_eth_dev *eth_dev, uint16_t rx_queue_id)
* Stop Receive Units for specified queue.
*/
int __rte_cold
-ionic_dev_rx_queue_stop(struct rte_eth_dev *eth_dev, uint16_t rx_queue_id)
+ionic_dev_rx_queue_stop(struct rte_eth_dev *dev, uint16_t rx_queue_id)
{
- uint8_t *rx_queue_state = eth_dev->data->rx_queue_state;
- struct ionic_rx_stats *stats;
- struct ionic_rx_qcq *rxq;
+ ionic_dev_rx_queue_stop_firsthalf(dev, rx_queue_id);
+ ionic_dev_rx_queue_stop_secondhalf(dev, rx_queue_id);
+
+ return 0;
+}
+
+void __rte_cold
+ionic_dev_rx_queue_stop_firsthalf(struct rte_eth_dev *dev,
+ uint16_t rx_queue_id)
+{
+ struct ionic_rx_qcq *rxq = dev->data->rx_queues[rx_queue_id];
IONIC_PRINT(DEBUG, "Stopping RX queue %u", rx_queue_id);
- rxq = eth_dev->data->rx_queues[rx_queue_id];
+ dev->data->rx_queue_state[rx_queue_id] = RTE_ETH_QUEUE_STATE_STOPPED;
+
+ ionic_lif_rxq_deinit_nowait(rxq);
+}
- rx_queue_state[rx_queue_id] = RTE_ETH_QUEUE_STATE_STOPPED;
+void __rte_cold
+ionic_dev_rx_queue_stop_secondhalf(struct rte_eth_dev *dev,
+ uint16_t rx_queue_id)
+{
+ struct ionic_lif *lif = IONIC_ETH_DEV_TO_LIF(dev);
+ struct ionic_rx_qcq *rxq = dev->data->rx_queues[rx_queue_id];
- ionic_lif_rxq_deinit(rxq);
+ ionic_adminq_wait(lif, &rxq->admin_ctx);
/* Free all buffers from descriptor ring */
ionic_rx_empty(rxq);
- stats = &rxq->stats;
- IONIC_PRINT(DEBUG, "RX queue %u pkts %ju mtod %ju",
- rxq->qcq.q.index, stats->packets, stats->mtods);
-
- return 0;
+ ionic_lif_rxq_stats(rxq);
}
int
diff --git a/drivers/net/ionic/ionic_rxtx.h b/drivers/net/ionic/ionic_rxtx.h
index 5348395956..7ca23178cc 100644
--- a/drivers/net/ionic/ionic_rxtx.h
+++ b/drivers/net/ionic/ionic_rxtx.h
@@ -37,14 +37,24 @@ int ionic_dev_rx_queue_setup(struct rte_eth_dev *dev, uint16_t rx_queue_id,
const struct rte_eth_rxconf *rx_conf, struct rte_mempool *mp);
void ionic_dev_rx_queue_release(struct rte_eth_dev *dev, uint16_t qid);
int ionic_dev_rx_queue_start(struct rte_eth_dev *dev, uint16_t rx_queue_id);
-int ionic_dev_rx_queue_stop(struct rte_eth_dev *eth_dev, uint16_t rx_queue_id);
+int ionic_dev_rx_queue_stop(struct rte_eth_dev *dev, uint16_t rx_queue_id);
int ionic_dev_tx_queue_setup(struct rte_eth_dev *dev, uint16_t tx_queue_id,
uint16_t nb_desc, uint32_t socket_id,
const struct rte_eth_txconf *tx_conf);
void ionic_dev_tx_queue_release(struct rte_eth_dev *dev, uint16_t qid);
-int ionic_dev_tx_queue_stop(struct rte_eth_dev *eth_dev, uint16_t tx_queue_id);
int ionic_dev_tx_queue_start(struct rte_eth_dev *dev, uint16_t tx_queue_id);
+int ionic_dev_tx_queue_stop(struct rte_eth_dev *dev, uint16_t tx_queue_id);
+
+/* Helpers for optimized dev_stop() */
+void ionic_dev_rx_queue_stop_firsthalf(struct rte_eth_dev *dev,
+ uint16_t rx_queue_id);
+void ionic_dev_rx_queue_stop_secondhalf(struct rte_eth_dev *dev,
+ uint16_t rx_queue_id);
+void ionic_dev_tx_queue_stop_firsthalf(struct rte_eth_dev *dev,
+ uint16_t tx_queue_id);
+void ionic_dev_tx_queue_stop_secondhalf(struct rte_eth_dev *dev,
+ uint16_t tx_queue_id);
void ionic_rxq_info_get(struct rte_eth_dev *dev, uint16_t queue_id,
struct rte_eth_rxq_info *qinfo);
--
2.17.1
^ permalink raw reply [flat|nested] 52+ messages in thread
* [PATCH v2 13/13] net/ionic: optimize device start operation
2024-02-02 19:32 [PATCH 00/13] net/ionic: miscellaneous fixes and improvements Andrew Boyer
` (26 preceding siblings ...)
2024-02-07 2:18 ` [PATCH v2 12/13] net/ionic: optimize device stop operation Andrew Boyer
@ 2024-02-07 2:18 ` Andrew Boyer
2024-02-07 3:13 ` [PATCH v3 00/13] net/ionic: miscellaneous fixes and improvements Andrew Boyer
28 siblings, 0 replies; 52+ messages in thread
From: Andrew Boyer @ 2024-02-07 2:18 UTC (permalink / raw)
To: dev; +Cc: Andrew Boyer
Split the queue_start operation into first-half and second-half helpers.
This allows us to batch up the queue commands during dev_start(), reducing
the outage window when restarting the process by about 1ms per queue.
Signed-off-by: Andrew Boyer <andrew.boyer@amd.com>
---
drivers/net/ionic/ionic_lif.c | 136 +++++++++++++++++++++++----------
drivers/net/ionic/ionic_lif.h | 6 +-
drivers/net/ionic/ionic_rxtx.c | 81 ++++++++++++++++----
drivers/net/ionic/ionic_rxtx.h | 10 +++
4 files changed, 176 insertions(+), 57 deletions(-)
diff --git a/drivers/net/ionic/ionic_lif.c b/drivers/net/ionic/ionic_lif.c
index 45317590fa..93a1011772 100644
--- a/drivers/net/ionic/ionic_lif.c
+++ b/drivers/net/ionic/ionic_lif.c
@@ -1601,13 +1601,16 @@ ionic_lif_set_features(struct ionic_lif *lif)
}
int
-ionic_lif_txq_init(struct ionic_tx_qcq *txq)
+ionic_lif_txq_init_nowait(struct ionic_tx_qcq *txq)
{
struct ionic_qcq *qcq = &txq->qcq;
struct ionic_queue *q = &qcq->q;
struct ionic_lif *lif = qcq->lif;
struct ionic_cq *cq = &qcq->cq;
- struct ionic_admin_ctx ctx = {
+ struct ionic_admin_ctx *ctx = &txq->admin_ctx;
+ int err;
+
+ *ctx = (struct ionic_admin_ctx) {
.pending_work = true,
.cmd.q_init = {
.opcode = IONIC_CMD_Q_INIT,
@@ -1621,32 +1624,41 @@ ionic_lif_txq_init(struct ionic_tx_qcq *txq)
.sg_ring_base = rte_cpu_to_le_64(q->sg_base_pa),
},
};
- int err;
if (txq->flags & IONIC_QCQ_F_SG)
- ctx.cmd.q_init.flags |= rte_cpu_to_le_16(IONIC_QINIT_F_SG);
+ ctx->cmd.q_init.flags |= rte_cpu_to_le_16(IONIC_QINIT_F_SG);
if (txq->flags & IONIC_QCQ_F_CMB) {
- ctx.cmd.q_init.flags |= rte_cpu_to_le_16(IONIC_QINIT_F_CMB);
- ctx.cmd.q_init.ring_base = rte_cpu_to_le_64(q->cmb_base_pa);
+ ctx->cmd.q_init.flags |= rte_cpu_to_le_16(IONIC_QINIT_F_CMB);
+ ctx->cmd.q_init.ring_base = rte_cpu_to_le_64(q->cmb_base_pa);
} else {
- ctx.cmd.q_init.ring_base = rte_cpu_to_le_64(q->base_pa);
+ ctx->cmd.q_init.ring_base = rte_cpu_to_le_64(q->base_pa);
}
IONIC_PRINT(DEBUG, "txq_init.index %d", q->index);
IONIC_PRINT(DEBUG, "txq_init.ring_base 0x%" PRIx64 "", q->base_pa);
IONIC_PRINT(DEBUG, "txq_init.ring_size %d",
- ctx.cmd.q_init.ring_size);
- IONIC_PRINT(DEBUG, "txq_init.ver %u", ctx.cmd.q_init.ver);
+ ctx->cmd.q_init.ring_size);
+ IONIC_PRINT(DEBUG, "txq_init.ver %u", ctx->cmd.q_init.ver);
ionic_q_reset(q);
ionic_cq_reset(cq);
- err = ionic_adminq_post_wait(lif, &ctx);
+ /* Caller responsible for calling ionic_lif_txq_init_done() */
+ err = ionic_adminq_post(lif, ctx);
if (err)
- return err;
+ ctx->pending_work = false;
+ return err;
+}
- q->hw_type = ctx.comp.q_init.hw_type;
- q->hw_index = rte_le_to_cpu_32(ctx.comp.q_init.hw_index);
+void
+ionic_lif_txq_init_done(struct ionic_tx_qcq *txq)
+{
+ struct ionic_lif *lif = txq->qcq.lif;
+ struct ionic_queue *q = &txq->qcq.q;
+ struct ionic_admin_ctx *ctx = &txq->admin_ctx;
+
+ q->hw_type = ctx->comp.q_init.hw_type;
+ q->hw_index = rte_le_to_cpu_32(ctx->comp.q_init.hw_index);
q->db = ionic_db_map(lif, q);
IONIC_PRINT(DEBUG, "txq->hw_type %d", q->hw_type);
@@ -1654,18 +1666,19 @@ ionic_lif_txq_init(struct ionic_tx_qcq *txq)
IONIC_PRINT(DEBUG, "txq->db %p", q->db);
txq->flags |= IONIC_QCQ_F_INITED;
-
- return 0;
}
int
-ionic_lif_rxq_init(struct ionic_rx_qcq *rxq)
+ionic_lif_rxq_init_nowait(struct ionic_rx_qcq *rxq)
{
struct ionic_qcq *qcq = &rxq->qcq;
struct ionic_queue *q = &qcq->q;
struct ionic_lif *lif = qcq->lif;
struct ionic_cq *cq = &qcq->cq;
- struct ionic_admin_ctx ctx = {
+ struct ionic_admin_ctx *ctx = &rxq->admin_ctx;
+ int err;
+
+ *ctx = (struct ionic_admin_ctx) {
.pending_work = true,
.cmd.q_init = {
.opcode = IONIC_CMD_Q_INIT,
@@ -1679,32 +1692,41 @@ ionic_lif_rxq_init(struct ionic_rx_qcq *rxq)
.sg_ring_base = rte_cpu_to_le_64(q->sg_base_pa),
},
};
- int err;
if (rxq->flags & IONIC_QCQ_F_SG)
- ctx.cmd.q_init.flags |= rte_cpu_to_le_16(IONIC_QINIT_F_SG);
+ ctx->cmd.q_init.flags |= rte_cpu_to_le_16(IONIC_QINIT_F_SG);
if (rxq->flags & IONIC_QCQ_F_CMB) {
- ctx.cmd.q_init.flags |= rte_cpu_to_le_16(IONIC_QINIT_F_CMB);
- ctx.cmd.q_init.ring_base = rte_cpu_to_le_64(q->cmb_base_pa);
+ ctx->cmd.q_init.flags |= rte_cpu_to_le_16(IONIC_QINIT_F_CMB);
+ ctx->cmd.q_init.ring_base = rte_cpu_to_le_64(q->cmb_base_pa);
} else {
- ctx.cmd.q_init.ring_base = rte_cpu_to_le_64(q->base_pa);
+ ctx->cmd.q_init.ring_base = rte_cpu_to_le_64(q->base_pa);
}
IONIC_PRINT(DEBUG, "rxq_init.index %d", q->index);
IONIC_PRINT(DEBUG, "rxq_init.ring_base 0x%" PRIx64 "", q->base_pa);
IONIC_PRINT(DEBUG, "rxq_init.ring_size %d",
- ctx.cmd.q_init.ring_size);
- IONIC_PRINT(DEBUG, "rxq_init.ver %u", ctx.cmd.q_init.ver);
+ ctx->cmd.q_init.ring_size);
+ IONIC_PRINT(DEBUG, "rxq_init.ver %u", ctx->cmd.q_init.ver);
ionic_q_reset(q);
ionic_cq_reset(cq);
- err = ionic_adminq_post_wait(lif, &ctx);
+ /* Caller responsible for calling ionic_lif_rxq_init_done() */
+ err = ionic_adminq_post(lif, ctx);
if (err)
- return err;
+ ctx->pending_work = false;
+ return err;
+}
- q->hw_type = ctx.comp.q_init.hw_type;
- q->hw_index = rte_le_to_cpu_32(ctx.comp.q_init.hw_index);
+void
+ionic_lif_rxq_init_done(struct ionic_rx_qcq *rxq)
+{
+ struct ionic_lif *lif = rxq->qcq.lif;
+ struct ionic_queue *q = &rxq->qcq.q;
+ struct ionic_admin_ctx *ctx = &rxq->admin_ctx;
+
+ q->hw_type = ctx->comp.q_init.hw_type;
+ q->hw_index = rte_le_to_cpu_32(ctx->comp.q_init.hw_index);
q->db = ionic_db_map(lif, q);
rxq->flags |= IONIC_QCQ_F_INITED;
@@ -1712,8 +1734,6 @@ ionic_lif_rxq_init(struct ionic_rx_qcq *rxq)
IONIC_PRINT(DEBUG, "rxq->hw_type %d", q->hw_type);
IONIC_PRINT(DEBUG, "rxq->hw_index %d", q->hw_index);
IONIC_PRINT(DEBUG, "rxq->db %p", q->db);
-
- return 0;
}
static int
@@ -1962,9 +1982,11 @@ ionic_lif_configure(struct ionic_lif *lif)
int
ionic_lif_start(struct ionic_lif *lif)
{
+ struct rte_eth_dev *dev = lif->eth_dev;
uint32_t rx_mode;
- uint32_t i;
+ uint32_t i, j, chunk;
int err;
+ bool fatal = false;
err = ionic_lif_rss_setup(lif);
if (err)
@@ -1985,25 +2007,57 @@ ionic_lif_start(struct ionic_lif *lif)
"on port %u",
lif->nrxqcqs, lif->ntxqcqs, lif->port_id);
- for (i = 0; i < lif->nrxqcqs; i++) {
- struct ionic_rx_qcq *rxq = lif->rxqcqs[i];
- if (!(rxq->flags & IONIC_QCQ_F_DEFERRED)) {
- err = ionic_dev_rx_queue_start(lif->eth_dev, i);
+ chunk = ionic_adminq_space_avail(lif);
+
+ for (i = 0; i < lif->nrxqcqs; i += chunk) {
+ if (lif->rxqcqs[0]->flags & IONIC_QCQ_F_DEFERRED) {
+ IONIC_PRINT(DEBUG, "Rx queue start deferred");
+ break;
+ }
+
+ for (j = 0; j < chunk && i + j < lif->nrxqcqs; j++) {
+ err = ionic_dev_rx_queue_start_firsthalf(dev, i + j);
+ if (err) {
+ fatal = true;
+ break;
+ }
+ }
+ for (j = 0; j < chunk && i + j < lif->nrxqcqs; j++) {
+ /* Commands that failed to post return immediately */
+ err = ionic_dev_rx_queue_start_secondhalf(dev, i + j);
if (err)
- return err;
+ /* Don't break */
+ fatal = true;
}
}
+ if (fatal)
+ return -EIO;
- for (i = 0; i < lif->ntxqcqs; i++) {
- struct ionic_tx_qcq *txq = lif->txqcqs[i];
- if (!(txq->flags & IONIC_QCQ_F_DEFERRED)) {
- err = ionic_dev_tx_queue_start(lif->eth_dev, i);
+ for (i = 0; i < lif->ntxqcqs; i += chunk) {
+ if (lif->txqcqs[0]->flags & IONIC_QCQ_F_DEFERRED) {
+ IONIC_PRINT(DEBUG, "Tx queue start deferred");
+ break;
+ }
+
+ for (j = 0; j < chunk && i + j < lif->ntxqcqs; j++) {
+ err = ionic_dev_tx_queue_start_firsthalf(dev, i + j);
+ if (err) {
+ fatal = true;
+ break;
+ }
+ }
+ for (j = 0; j < chunk && i + j < lif->ntxqcqs; j++) {
+ /* Commands that failed to post return immediately */
+ err = ionic_dev_tx_queue_start_secondhalf(dev, i + j);
if (err)
- return err;
+ /* Don't break */
+ fatal = true;
}
}
+ if (fatal)
+ return -EIO;
/* Carrier ON here */
lif->state |= IONIC_LIF_F_UP;
diff --git a/drivers/net/ionic/ionic_lif.h b/drivers/net/ionic/ionic_lif.h
index ee13f5b7c8..591cf1a2ff 100644
--- a/drivers/net/ionic/ionic_lif.h
+++ b/drivers/net/ionic/ionic_lif.h
@@ -228,11 +228,13 @@ int ionic_tx_qcq_alloc(struct ionic_lif *lif, uint32_t socket_id,
struct ionic_tx_qcq **qcq_out);
void ionic_qcq_free(struct ionic_qcq *qcq);
-int ionic_lif_rxq_init(struct ionic_rx_qcq *rxq);
+int ionic_lif_rxq_init_nowait(struct ionic_rx_qcq *rxq);
+void ionic_lif_rxq_init_done(struct ionic_rx_qcq *rxq);
void ionic_lif_rxq_deinit_nowait(struct ionic_rx_qcq *rxq);
void ionic_lif_rxq_stats(struct ionic_rx_qcq *rxq);
-int ionic_lif_txq_init(struct ionic_tx_qcq *txq);
+int ionic_lif_txq_init_nowait(struct ionic_tx_qcq *txq);
+void ionic_lif_txq_init_done(struct ionic_tx_qcq *txq);
void ionic_lif_txq_deinit_nowait(struct ionic_tx_qcq *txq);
void ionic_lif_txq_stats(struct ionic_tx_qcq *txq);
diff --git a/drivers/net/ionic/ionic_rxtx.c b/drivers/net/ionic/ionic_rxtx.c
index 774dc596c0..ad04e987eb 100644
--- a/drivers/net/ionic/ionic_rxtx.c
+++ b/drivers/net/ionic/ionic_rxtx.c
@@ -203,27 +203,54 @@ ionic_dev_tx_queue_setup(struct rte_eth_dev *eth_dev, uint16_t tx_queue_id,
* Start Transmit Units for specified queue.
*/
int __rte_cold
-ionic_dev_tx_queue_start(struct rte_eth_dev *eth_dev, uint16_t tx_queue_id)
+ionic_dev_tx_queue_start(struct rte_eth_dev *dev, uint16_t tx_queue_id)
{
- uint8_t *tx_queue_state = eth_dev->data->tx_queue_state;
- struct ionic_tx_qcq *txq;
int err;
+ err = ionic_dev_tx_queue_start_firsthalf(dev, tx_queue_id);
+ if (err)
+ return err;
+
+ return ionic_dev_tx_queue_start_secondhalf(dev, tx_queue_id);
+}
+
+int __rte_cold
+ionic_dev_tx_queue_start_firsthalf(struct rte_eth_dev *dev,
+ uint16_t tx_queue_id)
+{
+ uint8_t *tx_queue_state = dev->data->tx_queue_state;
+ struct ionic_tx_qcq *txq = dev->data->tx_queues[tx_queue_id];
+
if (tx_queue_state[tx_queue_id] == RTE_ETH_QUEUE_STATE_STARTED) {
IONIC_PRINT(DEBUG, "TX queue %u already started",
tx_queue_id);
return 0;
}
- txq = eth_dev->data->tx_queues[tx_queue_id];
-
IONIC_PRINT(DEBUG, "Starting TX queue %u, %u descs",
tx_queue_id, txq->qcq.q.num_descs);
- err = ionic_lif_txq_init(txq);
+ return ionic_lif_txq_init_nowait(txq);
+}
+
+int __rte_cold
+ionic_dev_tx_queue_start_secondhalf(struct rte_eth_dev *dev,
+ uint16_t tx_queue_id)
+{
+ uint8_t *tx_queue_state = dev->data->tx_queue_state;
+ struct ionic_lif *lif = IONIC_ETH_DEV_TO_LIF(dev);
+ struct ionic_tx_qcq *txq = dev->data->tx_queues[tx_queue_id];
+ int err;
+
+ if (tx_queue_state[tx_queue_id] == RTE_ETH_QUEUE_STATE_STARTED)
+ return 0;
+
+ err = ionic_adminq_wait(lif, &txq->admin_ctx);
if (err)
return err;
+ ionic_lif_txq_init_done(txq);
+
tx_queue_state[tx_queue_id] = RTE_ETH_QUEUE_STATE_STARTED;
return 0;
@@ -680,22 +707,31 @@ ionic_rx_init_descriptors(struct ionic_rx_qcq *rxq)
* Start Receive Units for specified queue.
*/
int __rte_cold
-ionic_dev_rx_queue_start(struct rte_eth_dev *eth_dev, uint16_t rx_queue_id)
+ionic_dev_rx_queue_start(struct rte_eth_dev *dev, uint16_t rx_queue_id)
{
- uint8_t *rx_queue_state = eth_dev->data->rx_queue_state;
- struct ionic_rx_qcq *rxq;
- struct ionic_queue *q;
int err;
+ err = ionic_dev_rx_queue_start_firsthalf(dev, rx_queue_id);
+ if (err)
+ return err;
+
+ return ionic_dev_rx_queue_start_secondhalf(dev, rx_queue_id);
+}
+
+int __rte_cold
+ionic_dev_rx_queue_start_firsthalf(struct rte_eth_dev *dev,
+ uint16_t rx_queue_id)
+{
+ uint8_t *rx_queue_state = dev->data->rx_queue_state;
+ struct ionic_rx_qcq *rxq = dev->data->rx_queues[rx_queue_id];
+ struct ionic_queue *q = &rxq->qcq.q;
+
if (rx_queue_state[rx_queue_id] == RTE_ETH_QUEUE_STATE_STARTED) {
IONIC_PRINT(DEBUG, "RX queue %u already started",
rx_queue_id);
return 0;
}
- rxq = eth_dev->data->rx_queues[rx_queue_id];
- q = &rxq->qcq.q;
-
rxq->frame_size = rxq->qcq.lif->frame_size - RTE_ETHER_CRC_LEN;
/* Recalculate segment count based on MTU */
@@ -707,10 +743,27 @@ ionic_dev_rx_queue_start(struct rte_eth_dev *eth_dev, uint16_t rx_queue_id)
ionic_rx_init_descriptors(rxq);
- err = ionic_lif_rxq_init(rxq);
+ return ionic_lif_rxq_init_nowait(rxq);
+}
+
+int __rte_cold
+ionic_dev_rx_queue_start_secondhalf(struct rte_eth_dev *dev,
+ uint16_t rx_queue_id)
+{
+ uint8_t *rx_queue_state = dev->data->rx_queue_state;
+ struct ionic_lif *lif = IONIC_ETH_DEV_TO_LIF(dev);
+ struct ionic_rx_qcq *rxq = dev->data->rx_queues[rx_queue_id];
+ int err;
+
+ if (rx_queue_state[rx_queue_id] == RTE_ETH_QUEUE_STATE_STARTED)
+ return 0;
+
+ err = ionic_adminq_wait(lif, &rxq->admin_ctx);
if (err)
return err;
+ ionic_lif_rxq_init_done(rxq);
+
/* Allocate buffers for descriptor ring */
if (rxq->flags & IONIC_QCQ_F_SG)
err = ionic_rx_fill_sg(rxq);
diff --git a/drivers/net/ionic/ionic_rxtx.h b/drivers/net/ionic/ionic_rxtx.h
index 7ca23178cc..a342afec54 100644
--- a/drivers/net/ionic/ionic_rxtx.h
+++ b/drivers/net/ionic/ionic_rxtx.h
@@ -46,6 +46,16 @@ void ionic_dev_tx_queue_release(struct rte_eth_dev *dev, uint16_t qid);
int ionic_dev_tx_queue_start(struct rte_eth_dev *dev, uint16_t tx_queue_id);
int ionic_dev_tx_queue_stop(struct rte_eth_dev *dev, uint16_t tx_queue_id);
+/* Helpers for optimized dev_start() */
+int ionic_dev_rx_queue_start_firsthalf(struct rte_eth_dev *dev,
+ uint16_t rx_queue_id);
+int ionic_dev_rx_queue_start_secondhalf(struct rte_eth_dev *dev,
+ uint16_t rx_queue_id);
+int ionic_dev_tx_queue_start_firsthalf(struct rte_eth_dev *dev,
+ uint16_t tx_queue_id);
+int ionic_dev_tx_queue_start_secondhalf(struct rte_eth_dev *dev,
+ uint16_t tx_queue_id);
+
/* Helpers for optimized dev_stop() */
void ionic_dev_rx_queue_stop_firsthalf(struct rte_eth_dev *dev,
uint16_t rx_queue_id);
--
2.17.1
^ permalink raw reply [flat|nested] 52+ messages in thread
* [PATCH v3 00/13] net/ionic: miscellaneous fixes and improvements
2024-02-02 19:32 [PATCH 00/13] net/ionic: miscellaneous fixes and improvements Andrew Boyer
` (27 preceding siblings ...)
2024-02-07 2:18 ` [PATCH v2 13/13] net/ionic: optimize device start operation Andrew Boyer
@ 2024-02-07 3:13 ` Andrew Boyer
2024-02-07 3:13 ` [PATCH v3 01/13] net/ionic: add stat for completion queue entries processed Andrew Boyer
` (13 more replies)
28 siblings, 14 replies; 52+ messages in thread
From: Andrew Boyer @ 2024-02-07 3:13 UTC (permalink / raw)
To: dev; +Cc: Andrew Boyer
This patchset provides miscellaneous fixes and improvements for
the net/ionic driver used by AMD Pensando devices.
V3:
- Resend to fix patchwork threading.
V2:
- Update device stop and device start patches to use compound literals
as suggested by review.
Akshay Dorwat (1):
net/ionic: fix RSS query routine
Andrew Boyer (8):
net/ionic: add stat for completion queue entries processed
net/ionic: increase max supported MTU to 9750 bytes
net/ionic: don't auto-enable Rx scatter-gather a second time
net/ionic: replace non-standard type in structure definition
net/ionic: fix device close sequence to avoid crash
net/ionic: optimize device close operation
net/ionic: optimize device stop operation
net/ionic: optimize device start operation
Brad Larson (1):
net/ionic: add flexible firmware xstat counters
Neel Patel (2):
net/ionic: fix missing volatile type for cqe pointers
net/ionic: memcpy descriptors when using Q-in-CMB
Vamsi Krishna Atluri (1):
net/ionic: report 1G and 200G link speeds when applicable
drivers/net/ionic/ionic.h | 3 +
drivers/net/ionic/ionic_dev.c | 9 +-
drivers/net/ionic/ionic_dev.h | 8 +-
drivers/net/ionic/ionic_dev_pci.c | 2 +-
drivers/net/ionic/ionic_ethdev.c | 81 ++++++---
drivers/net/ionic/ionic_if.h | 70 ++++----
drivers/net/ionic/ionic_lif.c | 233 ++++++++++++++++++--------
drivers/net/ionic/ionic_lif.h | 19 ++-
drivers/net/ionic/ionic_main.c | 17 +-
drivers/net/ionic/ionic_rxtx.c | 160 +++++++++++++-----
drivers/net/ionic/ionic_rxtx.h | 80 ++++++++-
drivers/net/ionic/ionic_rxtx_sg.c | 28 ++--
drivers/net/ionic/ionic_rxtx_simple.c | 28 ++--
13 files changed, 527 insertions(+), 211 deletions(-)
--
2.17.1
^ permalink raw reply [flat|nested] 52+ messages in thread
* [PATCH v3 01/13] net/ionic: add stat for completion queue entries processed
2024-02-07 3:13 ` [PATCH v3 00/13] net/ionic: miscellaneous fixes and improvements Andrew Boyer
@ 2024-02-07 3:13 ` Andrew Boyer
2024-02-07 3:13 ` [PATCH v3 02/13] net/ionic: increase max supported MTU to 9750 bytes Andrew Boyer
` (12 subsequent siblings)
13 siblings, 0 replies; 52+ messages in thread
From: Andrew Boyer @ 2024-02-07 3:13 UTC (permalink / raw)
To: dev; +Cc: Andrew Boyer
When completion coalescing is turned on in the FW, there will be
fewer CQE than Tx packets. Expose the stat through debug logging.
Signed-off-by: Andrew Boyer <andrew.boyer@amd.com>
---
drivers/net/ionic/ionic_lif.h | 1 +
drivers/net/ionic/ionic_rxtx.c | 3 +++
drivers/net/ionic/ionic_rxtx_sg.c | 2 ++
drivers/net/ionic/ionic_rxtx_simple.c | 2 ++
4 files changed, 8 insertions(+)
diff --git a/drivers/net/ionic/ionic_lif.h b/drivers/net/ionic/ionic_lif.h
index 36b3bcc5a9..cac7a4583b 100644
--- a/drivers/net/ionic/ionic_lif.h
+++ b/drivers/net/ionic/ionic_lif.h
@@ -32,6 +32,7 @@
struct ionic_tx_stats {
uint64_t packets;
uint64_t bytes;
+ uint64_t comps;
uint64_t drop;
uint64_t stop;
uint64_t no_csum;
diff --git a/drivers/net/ionic/ionic_rxtx.c b/drivers/net/ionic/ionic_rxtx.c
index b9e73b4871..d92b231f8f 100644
--- a/drivers/net/ionic/ionic_rxtx.c
+++ b/drivers/net/ionic/ionic_rxtx.c
@@ -117,6 +117,9 @@ ionic_dev_tx_queue_stop(struct rte_eth_dev *eth_dev, uint16_t tx_queue_id)
stats = &txq->stats;
IONIC_PRINT(DEBUG, "TX queue %u pkts %ju tso %ju",
txq->qcq.q.index, stats->packets, stats->tso);
+ IONIC_PRINT(DEBUG, "TX queue %u comps %ju (%ju per)",
+ txq->qcq.q.index, stats->comps,
+ stats->comps ? stats->packets / stats->comps : 0);
return 0;
}
diff --git a/drivers/net/ionic/ionic_rxtx_sg.c b/drivers/net/ionic/ionic_rxtx_sg.c
index ab8e56e91c..6c028a698c 100644
--- a/drivers/net/ionic/ionic_rxtx_sg.c
+++ b/drivers/net/ionic/ionic_rxtx_sg.c
@@ -26,6 +26,7 @@ ionic_tx_flush_sg(struct ionic_tx_qcq *txq)
{
struct ionic_cq *cq = &txq->qcq.cq;
struct ionic_queue *q = &txq->qcq.q;
+ struct ionic_tx_stats *stats = &txq->stats;
struct rte_mbuf *txm;
struct ionic_txq_comp *cq_desc, *cq_desc_base = cq->base;
void **info;
@@ -72,6 +73,7 @@ ionic_tx_flush_sg(struct ionic_tx_qcq *txq)
}
cq_desc = &cq_desc_base[cq->tail_idx];
+ stats->comps++;
}
}
diff --git a/drivers/net/ionic/ionic_rxtx_simple.c b/drivers/net/ionic/ionic_rxtx_simple.c
index 5f81856256..5969287b66 100644
--- a/drivers/net/ionic/ionic_rxtx_simple.c
+++ b/drivers/net/ionic/ionic_rxtx_simple.c
@@ -26,6 +26,7 @@ ionic_tx_flush(struct ionic_tx_qcq *txq)
{
struct ionic_cq *cq = &txq->qcq.cq;
struct ionic_queue *q = &txq->qcq.q;
+ struct ionic_tx_stats *stats = &txq->stats;
struct rte_mbuf *txm;
struct ionic_txq_comp *cq_desc, *cq_desc_base = cq->base;
void **info;
@@ -67,6 +68,7 @@ ionic_tx_flush(struct ionic_tx_qcq *txq)
}
cq_desc = &cq_desc_base[cq->tail_idx];
+ stats->comps++;
}
}
--
2.17.1
^ permalink raw reply [flat|nested] 52+ messages in thread
* [PATCH v3 02/13] net/ionic: increase max supported MTU to 9750 bytes
2024-02-07 3:13 ` [PATCH v3 00/13] net/ionic: miscellaneous fixes and improvements Andrew Boyer
2024-02-07 3:13 ` [PATCH v3 01/13] net/ionic: add stat for completion queue entries processed Andrew Boyer
@ 2024-02-07 3:13 ` Andrew Boyer
2024-02-07 3:13 ` [PATCH v3 03/13] net/ionic: don't auto-enable Rx scatter-gather a second time Andrew Boyer
` (11 subsequent siblings)
13 siblings, 0 replies; 52+ messages in thread
From: Andrew Boyer @ 2024-02-07 3:13 UTC (permalink / raw)
To: dev; +Cc: Andrew Boyer, Bhuvan Mital
Some configurations want to use values this high internally.
Allow them to do so without modifying the code.
Signed-off-by: Andrew Boyer <andrew.boyer@amd.com>
Signed-off-by: Bhuvan Mital <bhuvan.mital@amd.com>
---
drivers/net/ionic/ionic_dev.h | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/drivers/net/ionic/ionic_dev.h b/drivers/net/ionic/ionic_dev.h
index b1e74fbd89..971c261b27 100644
--- a/drivers/net/ionic/ionic_dev.h
+++ b/drivers/net/ionic/ionic_dev.h
@@ -14,7 +14,7 @@
#define VLAN_TAG_SIZE 4
#define IONIC_MIN_MTU RTE_ETHER_MIN_MTU
-#define IONIC_MAX_MTU 9378
+#define IONIC_MAX_MTU 9750
#define IONIC_ETH_OVERHEAD (RTE_ETHER_HDR_LEN + VLAN_TAG_SIZE)
#define IONIC_MAX_RING_DESC 32768
--
2.17.1
^ permalink raw reply [flat|nested] 52+ messages in thread
* [PATCH v3 03/13] net/ionic: don't auto-enable Rx scatter-gather a second time
2024-02-07 3:13 ` [PATCH v3 00/13] net/ionic: miscellaneous fixes and improvements Andrew Boyer
2024-02-07 3:13 ` [PATCH v3 01/13] net/ionic: add stat for completion queue entries processed Andrew Boyer
2024-02-07 3:13 ` [PATCH v3 02/13] net/ionic: increase max supported MTU to 9750 bytes Andrew Boyer
@ 2024-02-07 3:13 ` Andrew Boyer
2024-02-07 3:13 ` [PATCH v3 04/13] net/ionic: fix missing volatile type for cqe pointers Andrew Boyer
` (10 subsequent siblings)
13 siblings, 0 replies; 52+ messages in thread
From: Andrew Boyer @ 2024-02-07 3:13 UTC (permalink / raw)
To: dev; +Cc: Andrew Boyer
The receive side will enable scatter-gather if required based on the
mbuf size. If the client already enabled it in the config, it does
not need to be enabled again. This reduces log output.
Signed-off-by: Andrew Boyer <andrew.boyer@amd.com>
---
drivers/net/ionic/ionic_lif.c | 3 ++-
1 file changed, 2 insertions(+), 1 deletion(-)
diff --git a/drivers/net/ionic/ionic_lif.c b/drivers/net/ionic/ionic_lif.c
index 25b490deb6..fe2112c057 100644
--- a/drivers/net/ionic/ionic_lif.c
+++ b/drivers/net/ionic/ionic_lif.c
@@ -768,7 +768,8 @@ ionic_rx_qcq_alloc(struct ionic_lif *lif, uint32_t socket_id, uint32_t index,
max_mtu = rte_le_to_cpu_32(lif->adapter->ident.lif.eth.max_mtu);
/* If mbufs are too small to hold received packets, enable SG */
- if (max_mtu > hdr_seg_size) {
+ if (max_mtu > hdr_seg_size &&
+ !(lif->features & IONIC_ETH_HW_RX_SG)) {
IONIC_PRINT(NOTICE, "Enabling RX_OFFLOAD_SCATTER");
lif->eth_dev->data->dev_conf.rxmode.offloads |=
RTE_ETH_RX_OFFLOAD_SCATTER;
--
2.17.1
^ permalink raw reply [flat|nested] 52+ messages in thread
* [PATCH v3 04/13] net/ionic: fix missing volatile type for cqe pointers
2024-02-07 3:13 ` [PATCH v3 00/13] net/ionic: miscellaneous fixes and improvements Andrew Boyer
` (2 preceding siblings ...)
2024-02-07 3:13 ` [PATCH v3 03/13] net/ionic: don't auto-enable Rx scatter-gather a second time Andrew Boyer
@ 2024-02-07 3:13 ` Andrew Boyer
2024-02-07 3:13 ` [PATCH v3 05/13] net/ionic: replace non-standard type in structure definition Andrew Boyer
` (9 subsequent siblings)
13 siblings, 0 replies; 52+ messages in thread
From: Andrew Boyer @ 2024-02-07 3:13 UTC (permalink / raw)
To: dev; +Cc: Neel Patel, stable, Andrew Boyer
From: Neel Patel <neel.patel@amd.com>
This memory may be changed by the hardware, so the volatile
keyword is required for correctness.
Fixes: e86a6fcc7cf3 ("net/ionic: add optimized non-scattered Rx/Tx")
cc: stable@dpdk.org
Signed-off-by: Andrew Boyer <andrew.boyer@amd.com>
Signed-off-by: Neel Patel <neel.patel@amd.com>
---
drivers/net/ionic/ionic_rxtx.c | 4 ++--
drivers/net/ionic/ionic_rxtx_sg.c | 8 +++++---
drivers/net/ionic/ionic_rxtx_simple.c | 8 +++++---
3 files changed, 12 insertions(+), 8 deletions(-)
diff --git a/drivers/net/ionic/ionic_rxtx.c b/drivers/net/ionic/ionic_rxtx.c
index d92b231f8f..d92fa1cca7 100644
--- a/drivers/net/ionic/ionic_rxtx.c
+++ b/drivers/net/ionic/ionic_rxtx.c
@@ -755,7 +755,7 @@ ionic_dev_rx_descriptor_status(void *rx_queue, uint16_t offset)
{
struct ionic_rx_qcq *rxq = rx_queue;
struct ionic_qcq *qcq = &rxq->qcq;
- struct ionic_rxq_comp *cq_desc;
+ volatile struct ionic_rxq_comp *cq_desc;
uint16_t mask, head, tail, pos;
bool done_color;
@@ -794,7 +794,7 @@ ionic_dev_tx_descriptor_status(void *tx_queue, uint16_t offset)
{
struct ionic_tx_qcq *txq = tx_queue;
struct ionic_qcq *qcq = &txq->qcq;
- struct ionic_txq_comp *cq_desc;
+ volatile struct ionic_txq_comp *cq_desc;
uint16_t mask, head, tail, pos, cq_pos;
bool done_color;
diff --git a/drivers/net/ionic/ionic_rxtx_sg.c b/drivers/net/ionic/ionic_rxtx_sg.c
index 6c028a698c..1392342463 100644
--- a/drivers/net/ionic/ionic_rxtx_sg.c
+++ b/drivers/net/ionic/ionic_rxtx_sg.c
@@ -28,7 +28,8 @@ ionic_tx_flush_sg(struct ionic_tx_qcq *txq)
struct ionic_queue *q = &txq->qcq.q;
struct ionic_tx_stats *stats = &txq->stats;
struct rte_mbuf *txm;
- struct ionic_txq_comp *cq_desc, *cq_desc_base = cq->base;
+ struct ionic_txq_comp *cq_desc_base = cq->base;
+ volatile struct ionic_txq_comp *cq_desc;
void **info;
uint32_t i;
@@ -254,7 +255,7 @@ ionic_xmit_pkts_sg(void *tx_queue, struct rte_mbuf **tx_pkts,
*/
static __rte_always_inline void
ionic_rx_clean_one_sg(struct ionic_rx_qcq *rxq,
- struct ionic_rxq_comp *cq_desc,
+ volatile struct ionic_rxq_comp *cq_desc,
struct ionic_rx_service *rx_svc)
{
struct ionic_queue *q = &rxq->qcq.q;
@@ -440,7 +441,8 @@ ionic_rxq_service_sg(struct ionic_rx_qcq *rxq, uint32_t work_to_do,
struct ionic_cq *cq = &rxq->qcq.cq;
struct ionic_queue *q = &rxq->qcq.q;
struct ionic_rxq_desc *q_desc_base = q->base;
- struct ionic_rxq_comp *cq_desc, *cq_desc_base = cq->base;
+ struct ionic_rxq_comp *cq_desc_base = cq->base;
+ volatile struct ionic_rxq_comp *cq_desc;
uint32_t work_done = 0;
uint64_t then, now, hz, delta;
diff --git a/drivers/net/ionic/ionic_rxtx_simple.c b/drivers/net/ionic/ionic_rxtx_simple.c
index 5969287b66..00152c885a 100644
--- a/drivers/net/ionic/ionic_rxtx_simple.c
+++ b/drivers/net/ionic/ionic_rxtx_simple.c
@@ -28,7 +28,8 @@ ionic_tx_flush(struct ionic_tx_qcq *txq)
struct ionic_queue *q = &txq->qcq.q;
struct ionic_tx_stats *stats = &txq->stats;
struct rte_mbuf *txm;
- struct ionic_txq_comp *cq_desc, *cq_desc_base = cq->base;
+ struct ionic_txq_comp *cq_desc_base = cq->base;
+ volatile struct ionic_txq_comp *cq_desc;
void **info;
cq_desc = &cq_desc_base[cq->tail_idx];
@@ -227,7 +228,7 @@ ionic_xmit_pkts(void *tx_queue, struct rte_mbuf **tx_pkts,
*/
static __rte_always_inline void
ionic_rx_clean_one(struct ionic_rx_qcq *rxq,
- struct ionic_rxq_comp *cq_desc,
+ volatile struct ionic_rxq_comp *cq_desc,
struct ionic_rx_service *rx_svc)
{
struct ionic_queue *q = &rxq->qcq.q;
@@ -361,7 +362,8 @@ ionic_rxq_service(struct ionic_rx_qcq *rxq, uint32_t work_to_do,
struct ionic_cq *cq = &rxq->qcq.cq;
struct ionic_queue *q = &rxq->qcq.q;
struct ionic_rxq_desc *q_desc_base = q->base;
- struct ionic_rxq_comp *cq_desc, *cq_desc_base = cq->base;
+ struct ionic_rxq_comp *cq_desc_base = cq->base;
+ volatile struct ionic_rxq_comp *cq_desc;
uint32_t work_done = 0;
uint64_t then, now, hz, delta;
--
2.17.1
^ permalink raw reply [flat|nested] 52+ messages in thread
* [PATCH v3 05/13] net/ionic: replace non-standard type in structure definition
2024-02-07 3:13 ` [PATCH v3 00/13] net/ionic: miscellaneous fixes and improvements Andrew Boyer
` (3 preceding siblings ...)
2024-02-07 3:13 ` [PATCH v3 04/13] net/ionic: fix missing volatile type for cqe pointers Andrew Boyer
@ 2024-02-07 3:13 ` Andrew Boyer
2024-02-07 3:13 ` [PATCH v3 06/13] net/ionic: memcpy descriptors when using Q-in-CMB Andrew Boyer
` (8 subsequent siblings)
13 siblings, 0 replies; 52+ messages in thread
From: Andrew Boyer @ 2024-02-07 3:13 UTC (permalink / raw)
To: dev; +Cc: Andrew Boyer
Use uint8_t instead of u_char. This simplifies the code.
Signed-off-by: Andrew Boyer <andrew.boyer@amd.com>
---
drivers/net/ionic/ionic_dev_pci.c | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/drivers/net/ionic/ionic_dev_pci.c b/drivers/net/ionic/ionic_dev_pci.c
index 5e74a6da71..cbaac2c5bc 100644
--- a/drivers/net/ionic/ionic_dev_pci.c
+++ b/drivers/net/ionic/ionic_dev_pci.c
@@ -38,7 +38,7 @@ ionic_pci_setup(struct ionic_adapter *adapter)
struct ionic_dev *idev = &adapter->idev;
struct rte_pci_device *bus_dev = adapter->bus_dev;
uint32_t sig;
- u_char *bar0_base;
+ uint8_t *bar0_base;
unsigned int i;
/* BAR0: dev_cmd and interrupts */
--
2.17.1
^ permalink raw reply [flat|nested] 52+ messages in thread
* [PATCH v3 06/13] net/ionic: memcpy descriptors when using Q-in-CMB
2024-02-07 3:13 ` [PATCH v3 00/13] net/ionic: miscellaneous fixes and improvements Andrew Boyer
` (4 preceding siblings ...)
2024-02-07 3:13 ` [PATCH v3 05/13] net/ionic: replace non-standard type in structure definition Andrew Boyer
@ 2024-02-07 3:13 ` Andrew Boyer
2024-02-07 3:13 ` [PATCH v3 07/13] net/ionic: fix RSS query routine Andrew Boyer
` (7 subsequent siblings)
13 siblings, 0 replies; 52+ messages in thread
From: Andrew Boyer @ 2024-02-07 3:13 UTC (permalink / raw)
To: dev; +Cc: Neel Patel, Andrew Boyer
From: Neel Patel <neel.patel@amd.com>
They can be batched together this way, reducing the number
of PCIe transactions. This improves transmit PPS by up to 50% in
some configurations.
Signed-off-by: Andrew Boyer <andrew.boyer@amd.com>
Signed-off-by: Neel Patel <neel.patel@amd.com>
---
drivers/net/ionic/ionic_dev.c | 9 +++--
drivers/net/ionic/ionic_dev.h | 6 ++-
drivers/net/ionic/ionic_lif.c | 26 +++++++++----
drivers/net/ionic/ionic_rxtx.h | 56 +++++++++++++++++++++++++++
drivers/net/ionic/ionic_rxtx_sg.c | 18 ++++-----
drivers/net/ionic/ionic_rxtx_simple.c | 18 ++++-----
6 files changed, 101 insertions(+), 32 deletions(-)
diff --git a/drivers/net/ionic/ionic_dev.c b/drivers/net/ionic/ionic_dev.c
index 70c14882ed..7f15914f74 100644
--- a/drivers/net/ionic/ionic_dev.c
+++ b/drivers/net/ionic/ionic_dev.c
@@ -369,17 +369,19 @@ ionic_q_init(struct ionic_queue *q, uint32_t index, uint16_t num_descs)
q->index = index;
q->num_descs = num_descs;
q->size_mask = num_descs - 1;
- q->head_idx = 0;
- q->tail_idx = 0;
+ ionic_q_reset(q);
return 0;
}
void
-ionic_q_map(struct ionic_queue *q, void *base, rte_iova_t base_pa)
+ionic_q_map(struct ionic_queue *q, void *base, rte_iova_t base_pa,
+ void *cmb_base, rte_iova_t cmb_base_pa)
{
q->base = base;
q->base_pa = base_pa;
+ q->cmb_base = cmb_base;
+ q->cmb_base_pa = cmb_base_pa;
}
void
@@ -393,5 +395,6 @@ void
ionic_q_reset(struct ionic_queue *q)
{
q->head_idx = 0;
+ q->cmb_head_idx = 0;
q->tail_idx = 0;
}
diff --git a/drivers/net/ionic/ionic_dev.h b/drivers/net/ionic/ionic_dev.h
index 971c261b27..3a366247f1 100644
--- a/drivers/net/ionic/ionic_dev.h
+++ b/drivers/net/ionic/ionic_dev.h
@@ -145,11 +145,13 @@ struct ionic_queue {
uint16_t num_descs;
uint16_t num_segs;
uint16_t head_idx;
+ uint16_t cmb_head_idx;
uint16_t tail_idx;
uint16_t size_mask;
uint8_t type;
uint8_t hw_type;
void *base;
+ void *cmb_base;
void *sg_base;
struct ionic_doorbell __iomem *db;
void **info;
@@ -158,6 +160,7 @@ struct ionic_queue {
uint32_t hw_index;
rte_iova_t base_pa;
rte_iova_t sg_base_pa;
+ rte_iova_t cmb_base_pa;
};
#define IONIC_INTR_NONE (-1)
@@ -244,7 +247,8 @@ uint32_t ionic_cq_service(struct ionic_cq *cq, uint32_t work_to_do,
int ionic_q_init(struct ionic_queue *q, uint32_t index, uint16_t num_descs);
void ionic_q_reset(struct ionic_queue *q);
-void ionic_q_map(struct ionic_queue *q, void *base, rte_iova_t base_pa);
+void ionic_q_map(struct ionic_queue *q, void *base, rte_iova_t base_pa,
+ void *cmb_base, rte_iova_t cmb_base_pa);
void ionic_q_sg_map(struct ionic_queue *q, void *base, rte_iova_t base_pa);
static inline uint16_t
diff --git a/drivers/net/ionic/ionic_lif.c b/drivers/net/ionic/ionic_lif.c
index fe2112c057..2713f8aa24 100644
--- a/drivers/net/ionic/ionic_lif.c
+++ b/drivers/net/ionic/ionic_lif.c
@@ -572,10 +572,11 @@ ionic_qcq_alloc(struct ionic_lif *lif,
{
struct ionic_qcq *new;
uint32_t q_size, cq_size, sg_size, total_size;
- void *q_base, *cq_base, *sg_base;
+ void *q_base, *cmb_q_base, *cq_base, *sg_base;
rte_iova_t q_base_pa = 0;
rte_iova_t cq_base_pa = 0;
rte_iova_t sg_base_pa = 0;
+ rte_iova_t cmb_q_base_pa = 0;
size_t page_size = rte_mem_page_size();
int err;
@@ -666,19 +667,22 @@ ionic_qcq_alloc(struct ionic_lif *lif,
IONIC_PRINT(ERR, "Cannot reserve queue from NIC mem");
return -ENOMEM;
}
- q_base = (void *)
+ cmb_q_base = (void *)
((uintptr_t)lif->adapter->bars.bar[2].vaddr +
(uintptr_t)lif->adapter->cmb_offset);
/* CMB PA is a relative address */
- q_base_pa = lif->adapter->cmb_offset;
+ cmb_q_base_pa = lif->adapter->cmb_offset;
lif->adapter->cmb_offset += q_size;
+ } else {
+ cmb_q_base = NULL;
+ cmb_q_base_pa = 0;
}
IONIC_PRINT(DEBUG, "Q-Base-PA = %#jx CQ-Base-PA = %#jx "
"SG-base-PA = %#jx",
q_base_pa, cq_base_pa, sg_base_pa);
- ionic_q_map(&new->q, q_base, q_base_pa);
+ ionic_q_map(&new->q, q_base, q_base_pa, cmb_q_base, cmb_q_base_pa);
ionic_cq_map(&new->cq, cq_base, cq_base_pa);
*qcq = new;
@@ -1583,7 +1587,6 @@ ionic_lif_txq_init(struct ionic_tx_qcq *txq)
.flags = rte_cpu_to_le_16(IONIC_QINIT_F_ENA),
.intr_index = rte_cpu_to_le_16(IONIC_INTR_NONE),
.ring_size = rte_log2_u32(q->num_descs),
- .ring_base = rte_cpu_to_le_64(q->base_pa),
.cq_ring_base = rte_cpu_to_le_64(cq->base_pa),
.sg_ring_base = rte_cpu_to_le_64(q->sg_base_pa),
},
@@ -1592,8 +1595,12 @@ ionic_lif_txq_init(struct ionic_tx_qcq *txq)
if (txq->flags & IONIC_QCQ_F_SG)
ctx.cmd.q_init.flags |= rte_cpu_to_le_16(IONIC_QINIT_F_SG);
- if (txq->flags & IONIC_QCQ_F_CMB)
+ if (txq->flags & IONIC_QCQ_F_CMB) {
ctx.cmd.q_init.flags |= rte_cpu_to_le_16(IONIC_QINIT_F_CMB);
+ ctx.cmd.q_init.ring_base = rte_cpu_to_le_64(q->cmb_base_pa);
+ } else {
+ ctx.cmd.q_init.ring_base = rte_cpu_to_le_64(q->base_pa);
+ }
IONIC_PRINT(DEBUG, "txq_init.index %d", q->index);
IONIC_PRINT(DEBUG, "txq_init.ring_base 0x%" PRIx64 "", q->base_pa);
@@ -1638,7 +1645,6 @@ ionic_lif_rxq_init(struct ionic_rx_qcq *rxq)
.flags = rte_cpu_to_le_16(IONIC_QINIT_F_ENA),
.intr_index = rte_cpu_to_le_16(IONIC_INTR_NONE),
.ring_size = rte_log2_u32(q->num_descs),
- .ring_base = rte_cpu_to_le_64(q->base_pa),
.cq_ring_base = rte_cpu_to_le_64(cq->base_pa),
.sg_ring_base = rte_cpu_to_le_64(q->sg_base_pa),
},
@@ -1647,8 +1653,12 @@ ionic_lif_rxq_init(struct ionic_rx_qcq *rxq)
if (rxq->flags & IONIC_QCQ_F_SG)
ctx.cmd.q_init.flags |= rte_cpu_to_le_16(IONIC_QINIT_F_SG);
- if (rxq->flags & IONIC_QCQ_F_CMB)
+ if (rxq->flags & IONIC_QCQ_F_CMB) {
ctx.cmd.q_init.flags |= rte_cpu_to_le_16(IONIC_QINIT_F_CMB);
+ ctx.cmd.q_init.ring_base = rte_cpu_to_le_64(q->cmb_base_pa);
+ } else {
+ ctx.cmd.q_init.ring_base = rte_cpu_to_le_64(q->base_pa);
+ }
IONIC_PRINT(DEBUG, "rxq_init.index %d", q->index);
IONIC_PRINT(DEBUG, "rxq_init.ring_base 0x%" PRIx64 "", q->base_pa);
diff --git a/drivers/net/ionic/ionic_rxtx.h b/drivers/net/ionic/ionic_rxtx.h
index 8537141597..5348395956 100644
--- a/drivers/net/ionic/ionic_rxtx.h
+++ b/drivers/net/ionic/ionic_rxtx.h
@@ -77,4 +77,60 @@ uint16_t ionic_xmit_pkts_sg(void *tx_queue, struct rte_mbuf **tx_pkts,
int ionic_rx_fill_sg(struct ionic_rx_qcq *rxq);
+static inline void
+ionic_rxq_flush(struct ionic_queue *q)
+{
+ struct ionic_rxq_desc *desc_base = q->base;
+ struct ionic_rxq_desc *cmb_desc_base = q->cmb_base;
+
+ if (q->cmb_base) {
+ if (q->head_idx < q->cmb_head_idx) {
+ /* copy [cmb_head, num_descs) */
+ rte_memcpy((void *)&cmb_desc_base[q->cmb_head_idx],
+ (void *)&desc_base[q->cmb_head_idx],
+ (q->num_descs - q->cmb_head_idx) * sizeof(*desc_base));
+ /* copy [0, head) */
+ rte_memcpy((void *)&cmb_desc_base[0],
+ (void *)&desc_base[0],
+ q->head_idx * sizeof(*desc_base));
+ } else {
+ /* copy [cmb_head, head) */
+ rte_memcpy((void *)&cmb_desc_base[q->cmb_head_idx],
+ (void *)&desc_base[q->cmb_head_idx],
+ (q->head_idx - q->cmb_head_idx) * sizeof(*desc_base));
+ }
+ q->cmb_head_idx = q->head_idx;
+ }
+
+ ionic_q_flush(q);
+}
+
+static inline void
+ionic_txq_flush(struct ionic_queue *q)
+{
+ struct ionic_txq_desc *desc_base = q->base;
+ struct ionic_txq_desc *cmb_desc_base = q->cmb_base;
+
+ if (q->cmb_base) {
+ if (q->head_idx < q->cmb_head_idx) {
+ /* copy [cmb_head, num_descs) */
+ rte_memcpy((void *)&cmb_desc_base[q->cmb_head_idx],
+ (void *)&desc_base[q->cmb_head_idx],
+ (q->num_descs - q->cmb_head_idx) * sizeof(*desc_base));
+ /* copy [0, head) */
+ rte_memcpy((void *)&cmb_desc_base[0],
+ (void *)&desc_base[0],
+ q->head_idx * sizeof(*desc_base));
+ } else {
+ /* copy [cmb_head, head) */
+ rte_memcpy((void *)&cmb_desc_base[q->cmb_head_idx],
+ (void *)&desc_base[q->cmb_head_idx],
+ (q->head_idx - q->cmb_head_idx) * sizeof(*desc_base));
+ }
+ q->cmb_head_idx = q->head_idx;
+ }
+
+ ionic_q_flush(q);
+}
+
#endif /* _IONIC_RXTX_H_ */
diff --git a/drivers/net/ionic/ionic_rxtx_sg.c b/drivers/net/ionic/ionic_rxtx_sg.c
index 1392342463..92e1d6e259 100644
--- a/drivers/net/ionic/ionic_rxtx_sg.c
+++ b/drivers/net/ionic/ionic_rxtx_sg.c
@@ -166,6 +166,7 @@ ionic_xmit_pkts_sg(void *tx_queue, struct rte_mbuf **tx_pkts,
{
struct ionic_tx_qcq *txq = tx_queue;
struct ionic_queue *q = &txq->qcq.q;
+ struct ionic_txq_desc *desc_base = q->base;
struct ionic_tx_stats *stats = &txq->stats;
struct rte_mbuf *mbuf;
uint32_t bytes_tx = 0;
@@ -173,9 +174,7 @@ ionic_xmit_pkts_sg(void *tx_queue, struct rte_mbuf **tx_pkts,
uint64_t then, now, hz, delta;
int err;
- struct ionic_txq_desc *desc_base = q->base;
- if (!(txq->flags & IONIC_QCQ_F_CMB))
- rte_prefetch0(&desc_base[q->head_idx]);
+ rte_prefetch0(&desc_base[q->head_idx]);
rte_prefetch0(IONIC_INFO_PTR(q, q->head_idx));
if (nb_pkts) {
@@ -196,8 +195,7 @@ ionic_xmit_pkts_sg(void *tx_queue, struct rte_mbuf **tx_pkts,
while (nb_tx < nb_pkts) {
uint16_t next_idx = Q_NEXT_TO_POST(q, 1);
- if (!(txq->flags & IONIC_QCQ_F_CMB))
- rte_prefetch0(&desc_base[next_idx]);
+ rte_prefetch0(&desc_base[next_idx]);
rte_prefetch0(IONIC_INFO_PTR(q, next_idx));
if (nb_tx + 1 < nb_pkts) {
@@ -222,7 +220,7 @@ ionic_xmit_pkts_sg(void *tx_queue, struct rte_mbuf **tx_pkts,
if (nb_tx > 0) {
rte_wmb();
- ionic_q_flush(q);
+ ionic_txq_flush(q);
txq->last_wdog_cycles = rte_get_timer_cycles();
@@ -458,8 +456,7 @@ ionic_rxq_service_sg(struct ionic_rx_qcq *rxq, uint32_t work_to_do,
/* Prefetch 4 x 16B comp */
rte_prefetch0(&cq_desc_base[Q_NEXT_TO_SRVC(cq, 4)]);
/* Prefetch 4 x 16B descriptors */
- if (!(rxq->flags & IONIC_QCQ_F_CMB))
- rte_prefetch0(&q_desc_base[Q_NEXT_TO_POST(q, 4)]);
+ rte_prefetch0(&q_desc_base[Q_NEXT_TO_POST(q, 4)]);
/* Clean one descriptor */
ionic_rx_clean_one_sg(rxq, cq_desc, rx_svc);
@@ -478,7 +475,8 @@ ionic_rxq_service_sg(struct ionic_rx_qcq *rxq, uint32_t work_to_do,
/* Update the queue indices and ring the doorbell */
if (work_done) {
- ionic_q_flush(q);
+ ionic_rxq_flush(q);
+
rxq->last_wdog_cycles = rte_get_timer_cycles();
rxq->wdog_ms = IONIC_Q_WDOG_MS;
} else {
@@ -542,7 +540,7 @@ ionic_rx_fill_sg(struct ionic_rx_qcq *rxq)
q->head_idx = Q_NEXT_TO_POST(q, 1);
}
- ionic_q_flush(q);
+ ionic_rxq_flush(q);
return err;
}
diff --git a/drivers/net/ionic/ionic_rxtx_simple.c b/drivers/net/ionic/ionic_rxtx_simple.c
index 00152c885a..f12f66f40c 100644
--- a/drivers/net/ionic/ionic_rxtx_simple.c
+++ b/drivers/net/ionic/ionic_rxtx_simple.c
@@ -139,6 +139,7 @@ ionic_xmit_pkts(void *tx_queue, struct rte_mbuf **tx_pkts,
{
struct ionic_tx_qcq *txq = tx_queue;
struct ionic_queue *q = &txq->qcq.q;
+ struct ionic_txq_desc *desc_base = q->base;
struct ionic_tx_stats *stats = &txq->stats;
struct rte_mbuf *mbuf;
uint32_t bytes_tx = 0;
@@ -146,9 +147,7 @@ ionic_xmit_pkts(void *tx_queue, struct rte_mbuf **tx_pkts,
uint64_t then, now, hz, delta;
int err;
- struct ionic_txq_desc *desc_base = q->base;
- if (!(txq->flags & IONIC_QCQ_F_CMB))
- rte_prefetch0(&desc_base[q->head_idx]);
+ rte_prefetch0(&desc_base[q->head_idx]);
rte_prefetch0(&q->info[q->head_idx]);
if (nb_pkts) {
@@ -169,8 +168,7 @@ ionic_xmit_pkts(void *tx_queue, struct rte_mbuf **tx_pkts,
while (nb_tx < nb_pkts) {
uint16_t next_idx = Q_NEXT_TO_POST(q, 1);
- if (!(txq->flags & IONIC_QCQ_F_CMB))
- rte_prefetch0(&desc_base[next_idx]);
+ rte_prefetch0(&desc_base[next_idx]);
rte_prefetch0(&q->info[next_idx]);
if (nb_tx + 1 < nb_pkts) {
@@ -195,7 +193,7 @@ ionic_xmit_pkts(void *tx_queue, struct rte_mbuf **tx_pkts,
if (nb_tx > 0) {
rte_wmb();
- ionic_q_flush(q);
+ ionic_txq_flush(q);
txq->last_wdog_cycles = rte_get_timer_cycles();
@@ -379,8 +377,7 @@ ionic_rxq_service(struct ionic_rx_qcq *rxq, uint32_t work_to_do,
/* Prefetch 4 x 16B comp */
rte_prefetch0(&cq_desc_base[Q_NEXT_TO_SRVC(cq, 4)]);
/* Prefetch 4 x 16B descriptors */
- if (!(rxq->flags & IONIC_QCQ_F_CMB))
- rte_prefetch0(&q_desc_base[Q_NEXT_TO_POST(q, 4)]);
+ rte_prefetch0(&q_desc_base[Q_NEXT_TO_POST(q, 4)]);
/* Clean one descriptor */
ionic_rx_clean_one(rxq, cq_desc, rx_svc);
@@ -399,7 +396,8 @@ ionic_rxq_service(struct ionic_rx_qcq *rxq, uint32_t work_to_do,
/* Update the queue indices and ring the doorbell */
if (work_done) {
- ionic_q_flush(q);
+ ionic_rxq_flush(q);
+
rxq->last_wdog_cycles = rte_get_timer_cycles();
rxq->wdog_ms = IONIC_Q_WDOG_MS;
} else {
@@ -463,7 +461,7 @@ ionic_rx_fill(struct ionic_rx_qcq *rxq)
q->head_idx = Q_NEXT_TO_POST(q, 1);
}
- ionic_q_flush(q);
+ ionic_rxq_flush(q);
return err;
}
--
2.17.1
^ permalink raw reply [flat|nested] 52+ messages in thread
* [PATCH v3 07/13] net/ionic: fix RSS query routine
2024-02-07 3:13 ` [PATCH v3 00/13] net/ionic: miscellaneous fixes and improvements Andrew Boyer
` (5 preceding siblings ...)
2024-02-07 3:13 ` [PATCH v3 06/13] net/ionic: memcpy descriptors when using Q-in-CMB Andrew Boyer
@ 2024-02-07 3:13 ` Andrew Boyer
2024-02-07 3:13 ` [PATCH v3 08/13] net/ionic: report 1G and 200G link speeds when applicable Andrew Boyer
` (6 subsequent siblings)
13 siblings, 0 replies; 52+ messages in thread
From: Andrew Boyer @ 2024-02-07 3:13 UTC (permalink / raw)
To: dev; +Cc: Akshay Dorwat, cardigliano, stable, Andrew Boyer
From: Akshay Dorwat <akshay.dorwat@amd.com>
The routine that copies out the RSS config can't use memcpy() because
'reta_conf->reta' is an array of uint16_t while 'lif->rss_ind_tbl' is
an array of uint8_t. Instead, copy the values individually.
Fixes: 22e7171bc63b ("net/ionic: support RSS")
Cc: cardigliano@ntop.org
Cc: stable@dpdk.org
Signed-off-by: Akshay Dorwat <akshay.dorwat@amd.com>
Signed-off-by: Andrew Boyer <andrew.boyer@amd.com>
---
drivers/net/ionic/ionic_ethdev.c | 9 +++++----
1 file changed, 5 insertions(+), 4 deletions(-)
diff --git a/drivers/net/ionic/ionic_ethdev.c b/drivers/net/ionic/ionic_ethdev.c
index 340fd0cd59..008e50e0b9 100644
--- a/drivers/net/ionic/ionic_ethdev.c
+++ b/drivers/net/ionic/ionic_ethdev.c
@@ -561,7 +561,7 @@ ionic_dev_rss_reta_query(struct rte_eth_dev *eth_dev,
struct ionic_lif *lif = IONIC_ETH_DEV_TO_LIF(eth_dev);
struct ionic_adapter *adapter = lif->adapter;
struct ionic_identity *ident = &adapter->ident;
- int i, num;
+ int i, j, num;
uint16_t tbl_sz = rte_le_to_cpu_16(ident->lif.eth.rss_ind_tbl_sz);
IONIC_PRINT_CALL();
@@ -582,9 +582,10 @@ ionic_dev_rss_reta_query(struct rte_eth_dev *eth_dev,
num = reta_size / RTE_ETH_RETA_GROUP_SIZE;
for (i = 0; i < num; i++) {
- memcpy(reta_conf->reta,
- &lif->rss_ind_tbl[i * RTE_ETH_RETA_GROUP_SIZE],
- RTE_ETH_RETA_GROUP_SIZE);
+ for (j = 0; j < RTE_ETH_RETA_GROUP_SIZE; j++) {
+ reta_conf->reta[j] =
+ lif->rss_ind_tbl[(i * RTE_ETH_RETA_GROUP_SIZE) + j];
+ }
reta_conf++;
}
--
2.17.1
^ permalink raw reply [flat|nested] 52+ messages in thread
* [PATCH v3 08/13] net/ionic: report 1G and 200G link speeds when applicable
2024-02-07 3:13 ` [PATCH v3 00/13] net/ionic: miscellaneous fixes and improvements Andrew Boyer
` (6 preceding siblings ...)
2024-02-07 3:13 ` [PATCH v3 07/13] net/ionic: fix RSS query routine Andrew Boyer
@ 2024-02-07 3:13 ` Andrew Boyer
2024-02-07 3:13 ` [PATCH v3 09/13] net/ionic: add flexible firmware xstat counters Andrew Boyer
` (5 subsequent siblings)
13 siblings, 0 replies; 52+ messages in thread
From: Andrew Boyer @ 2024-02-07 3:13 UTC (permalink / raw)
To: dev; +Cc: Vamsi Krishna Atluri, Andrew Boyer
From: Vamsi Krishna Atluri <vamsi.atluri@amd.com>
The hardware supports these speeds, so we should report them
correctly.
Signed-off-by: Andrew Boyer <andrew.boyer@amd.com>
Signed-off-by: Vamsi Krishna Atluri <vamsi.atluri@amd.com>
---
drivers/net/ionic/ionic_ethdev.c | 6 ++++++
1 file changed, 6 insertions(+)
diff --git a/drivers/net/ionic/ionic_ethdev.c b/drivers/net/ionic/ionic_ethdev.c
index 008e50e0b9..327f6b9de5 100644
--- a/drivers/net/ionic/ionic_ethdev.c
+++ b/drivers/net/ionic/ionic_ethdev.c
@@ -285,6 +285,9 @@ ionic_dev_link_update(struct rte_eth_dev *eth_dev,
link.link_status = RTE_ETH_LINK_UP;
link.link_duplex = RTE_ETH_LINK_FULL_DUPLEX;
switch (adapter->link_speed) {
+ case 1000:
+ link.link_speed = RTE_ETH_SPEED_NUM_1G;
+ break;
case 10000:
link.link_speed = RTE_ETH_SPEED_NUM_10G;
break;
@@ -300,6 +303,9 @@ ionic_dev_link_update(struct rte_eth_dev *eth_dev,
case 100000:
link.link_speed = RTE_ETH_SPEED_NUM_100G;
break;
+ case 200000:
+ link.link_speed = RTE_ETH_SPEED_NUM_200G;
+ break;
default:
link.link_speed = RTE_ETH_SPEED_NUM_NONE;
break;
--
2.17.1
^ permalink raw reply [flat|nested] 52+ messages in thread
* [PATCH v3 09/13] net/ionic: add flexible firmware xstat counters
2024-02-07 3:13 ` [PATCH v3 00/13] net/ionic: miscellaneous fixes and improvements Andrew Boyer
` (7 preceding siblings ...)
2024-02-07 3:13 ` [PATCH v3 08/13] net/ionic: report 1G and 200G link speeds when applicable Andrew Boyer
@ 2024-02-07 3:13 ` Andrew Boyer
2024-02-07 3:13 ` [PATCH v3 10/13] net/ionic: fix device close sequence to avoid crash Andrew Boyer
` (4 subsequent siblings)
13 siblings, 0 replies; 52+ messages in thread
From: Andrew Boyer @ 2024-02-07 3:13 UTC (permalink / raw)
To: dev; +Cc: Brad Larson, Andrew Boyer
From: Brad Larson <bradley.larson@amd.com>
Assign 32 counters for flexible firmware events. These can be used as
per-port or per-queue counters in certain firmware configurations.
They are displayed as fw_flex_eventX in xstats.
Signed-off-by: Andrew Boyer <andrew.boyer@amd.com>
Signed-off-by: Brad Larson <bradley.larson@amd.com>
---
drivers/net/ionic/ionic_ethdev.c | 33 +++++++++++++++
drivers/net/ionic/ionic_if.h | 70 ++++++++++++++++----------------
2 files changed, 68 insertions(+), 35 deletions(-)
diff --git a/drivers/net/ionic/ionic_ethdev.c b/drivers/net/ionic/ionic_ethdev.c
index 327f6b9de5..7c55a26956 100644
--- a/drivers/net/ionic/ionic_ethdev.c
+++ b/drivers/net/ionic/ionic_ethdev.c
@@ -196,6 +196,39 @@ static const struct rte_ionic_xstats_name_off rte_ionic_xstats_strings[] = {
tx_desc_fetch_error)},
{"tx_desc_data_error", offsetof(struct ionic_lif_stats,
tx_desc_data_error)},
+ /* Flexible firmware events */
+ {"fw_flex_event1", offsetof(struct ionic_lif_stats, flex1)},
+ {"fw_flex_event2", offsetof(struct ionic_lif_stats, flex2)},
+ {"fw_flex_event3", offsetof(struct ionic_lif_stats, flex3)},
+ {"fw_flex_event4", offsetof(struct ionic_lif_stats, flex4)},
+ {"fw_flex_event5", offsetof(struct ionic_lif_stats, flex5)},
+ {"fw_flex_event6", offsetof(struct ionic_lif_stats, flex6)},
+ {"fw_flex_event7", offsetof(struct ionic_lif_stats, flex7)},
+ {"fw_flex_event8", offsetof(struct ionic_lif_stats, flex8)},
+ {"fw_flex_event9", offsetof(struct ionic_lif_stats, flex9)},
+ {"fw_flex_event10", offsetof(struct ionic_lif_stats, flex10)},
+ {"fw_flex_event11", offsetof(struct ionic_lif_stats, flex11)},
+ {"fw_flex_event12", offsetof(struct ionic_lif_stats, flex12)},
+ {"fw_flex_event13", offsetof(struct ionic_lif_stats, flex13)},
+ {"fw_flex_event14", offsetof(struct ionic_lif_stats, flex14)},
+ {"fw_flex_event15", offsetof(struct ionic_lif_stats, flex15)},
+ {"fw_flex_event16", offsetof(struct ionic_lif_stats, flex16)},
+ {"fw_flex_event17", offsetof(struct ionic_lif_stats, flex17)},
+ {"fw_flex_event18", offsetof(struct ionic_lif_stats, flex18)},
+ {"fw_flex_event19", offsetof(struct ionic_lif_stats, flex19)},
+ {"fw_flex_event20", offsetof(struct ionic_lif_stats, flex20)},
+ {"fw_flex_event21", offsetof(struct ionic_lif_stats, flex21)},
+ {"fw_flex_event22", offsetof(struct ionic_lif_stats, flex22)},
+ {"fw_flex_event23", offsetof(struct ionic_lif_stats, flex23)},
+ {"fw_flex_event24", offsetof(struct ionic_lif_stats, flex24)},
+ {"fw_flex_event25", offsetof(struct ionic_lif_stats, flex25)},
+ {"fw_flex_event26", offsetof(struct ionic_lif_stats, flex26)},
+ {"fw_flex_event27", offsetof(struct ionic_lif_stats, flex27)},
+ {"fw_flex_event28", offsetof(struct ionic_lif_stats, flex28)},
+ {"fw_flex_event29", offsetof(struct ionic_lif_stats, flex29)},
+ {"fw_flex_event30", offsetof(struct ionic_lif_stats, flex30)},
+ {"fw_flex_event31", offsetof(struct ionic_lif_stats, flex31)},
+ {"fw_flex_event32", offsetof(struct ionic_lif_stats, flex32)},
};
#define IONIC_NB_HW_STATS RTE_DIM(rte_ionic_xstats_strings)
diff --git a/drivers/net/ionic/ionic_if.h b/drivers/net/ionic/ionic_if.h
index 79aa196345..7ca604a7bb 100644
--- a/drivers/net/ionic/ionic_if.h
+++ b/drivers/net/ionic/ionic_if.h
@@ -2592,41 +2592,41 @@ struct ionic_lif_stats {
__le64 rsvd16;
__le64 rsvd17;
- __le64 rsvd18;
- __le64 rsvd19;
- __le64 rsvd20;
- __le64 rsvd21;
- __le64 rsvd22;
- __le64 rsvd23;
- __le64 rsvd24;
- __le64 rsvd25;
-
- __le64 rsvd26;
- __le64 rsvd27;
- __le64 rsvd28;
- __le64 rsvd29;
- __le64 rsvd30;
- __le64 rsvd31;
- __le64 rsvd32;
- __le64 rsvd33;
-
- __le64 rsvd34;
- __le64 rsvd35;
- __le64 rsvd36;
- __le64 rsvd37;
- __le64 rsvd38;
- __le64 rsvd39;
- __le64 rsvd40;
- __le64 rsvd41;
-
- __le64 rsvd42;
- __le64 rsvd43;
- __le64 rsvd44;
- __le64 rsvd45;
- __le64 rsvd46;
- __le64 rsvd47;
- __le64 rsvd48;
- __le64 rsvd49;
+ __le64 flex1;
+ __le64 flex2;
+ __le64 flex3;
+ __le64 flex4;
+ __le64 flex5;
+ __le64 flex6;
+ __le64 flex7;
+ __le64 flex8;
+
+ __le64 flex9;
+ __le64 flex10;
+ __le64 flex11;
+ __le64 flex12;
+ __le64 flex13;
+ __le64 flex14;
+ __le64 flex15;
+ __le64 flex16;
+
+ __le64 flex17;
+ __le64 flex18;
+ __le64 flex19;
+ __le64 flex20;
+ __le64 flex21;
+ __le64 flex22;
+ __le64 flex23;
+ __le64 flex24;
+
+ __le64 flex25;
+ __le64 flex26;
+ __le64 flex27;
+ __le64 flex28;
+ __le64 flex29;
+ __le64 flex30;
+ __le64 flex31;
+ __le64 flex32;
/* RDMA/ROCE REQ Error/Debugs (768 - 895) */
__le64 rdma_req_rx_pkt_seq_err;
--
2.17.1
^ permalink raw reply [flat|nested] 52+ messages in thread
* [PATCH v3 10/13] net/ionic: fix device close sequence to avoid crash
2024-02-07 3:13 ` [PATCH v3 00/13] net/ionic: miscellaneous fixes and improvements Andrew Boyer
` (8 preceding siblings ...)
2024-02-07 3:13 ` [PATCH v3 09/13] net/ionic: add flexible firmware xstat counters Andrew Boyer
@ 2024-02-07 3:13 ` Andrew Boyer
2024-02-07 3:13 ` [PATCH v3 11/13] net/ionic: optimize device close operation Andrew Boyer
` (3 subsequent siblings)
13 siblings, 0 replies; 52+ messages in thread
From: Andrew Boyer @ 2024-02-07 3:13 UTC (permalink / raw)
To: dev; +Cc: Andrew Boyer, stable
The close routine should release all resources, but not
call rte_eth_dev_destroy(). As written this code will call
rte_eth_dev_release_port() twice and segfault.
Instead, move rte_eth_dev_destroy() to the remove routine.
eth_ionic_dev_uninit() will call close if necessary.
Fixes: 175e4e7ed760 ("net/ionic: complete release on close")
Cc: stable@dpdk.org
Signed-off-by: Andrew Boyer <andrew.boyer@amd.com>
---
drivers/net/ionic/ionic_ethdev.c | 30 +++++++++++++++---------------
1 file changed, 15 insertions(+), 15 deletions(-)
diff --git a/drivers/net/ionic/ionic_ethdev.c b/drivers/net/ionic/ionic_ethdev.c
index 7c55a26956..bedcf958e2 100644
--- a/drivers/net/ionic/ionic_ethdev.c
+++ b/drivers/net/ionic/ionic_ethdev.c
@@ -1009,19 +1009,21 @@ ionic_dev_close(struct rte_eth_dev *eth_dev)
ionic_lif_stop(lif);
- ionic_lif_free_queues(lif);
-
IONIC_PRINT(NOTICE, "Removing device %s", eth_dev->device->name);
if (adapter->intf->unconfigure_intr)
(*adapter->intf->unconfigure_intr)(adapter);
- rte_eth_dev_destroy(eth_dev, eth_ionic_dev_uninit);
-
ionic_port_reset(adapter);
ionic_reset(adapter);
+
+ ionic_lif_free_queues(lif);
+ ionic_lif_deinit(lif);
+ ionic_lif_free(lif); /* Does not free LIF object */
+
if (adapter->intf->unmap_bars)
(*adapter->intf->unmap_bars)(adapter);
+ lif->adapter = NULL;
rte_free(adapter);
return 0;
@@ -1098,21 +1100,18 @@ eth_ionic_dev_init(struct rte_eth_dev *eth_dev, void *init_params)
static int
eth_ionic_dev_uninit(struct rte_eth_dev *eth_dev)
{
- struct ionic_lif *lif = IONIC_ETH_DEV_TO_LIF(eth_dev);
- struct ionic_adapter *adapter = lif->adapter;
-
IONIC_PRINT_CALL();
if (rte_eal_process_type() != RTE_PROC_PRIMARY)
return 0;
- adapter->lif = NULL;
-
- ionic_lif_deinit(lif);
- ionic_lif_free(lif);
+ if (eth_dev->state != RTE_ETH_DEV_UNUSED)
+ ionic_dev_close(eth_dev);
- if (!(lif->state & IONIC_LIF_F_FW_RESET))
- ionic_lif_reset(lif);
+ eth_dev->dev_ops = NULL;
+ eth_dev->rx_pkt_burst = NULL;
+ eth_dev->tx_pkt_burst = NULL;
+ eth_dev->tx_pkt_prepare = NULL;
return 0;
}
@@ -1267,17 +1266,18 @@ eth_ionic_dev_remove(struct rte_device *rte_dev)
{
char name[RTE_ETH_NAME_MAX_LEN];
struct rte_eth_dev *eth_dev;
+ int ret = 0;
/* Adapter lookup is using the eth_dev name */
snprintf(name, sizeof(name), "%s_lif", rte_dev->name);
eth_dev = rte_eth_dev_allocated(name);
if (eth_dev)
- ionic_dev_close(eth_dev);
+ ret = rte_eth_dev_destroy(eth_dev, eth_ionic_dev_uninit);
else
IONIC_PRINT(DEBUG, "Cannot find device %s", rte_dev->name);
- return 0;
+ return ret;
}
RTE_LOG_REGISTER_DEFAULT(ionic_logtype, NOTICE);
--
2.17.1
^ permalink raw reply [flat|nested] 52+ messages in thread
* [PATCH v3 11/13] net/ionic: optimize device close operation
2024-02-07 3:13 ` [PATCH v3 00/13] net/ionic: miscellaneous fixes and improvements Andrew Boyer
` (9 preceding siblings ...)
2024-02-07 3:13 ` [PATCH v3 10/13] net/ionic: fix device close sequence to avoid crash Andrew Boyer
@ 2024-02-07 3:13 ` Andrew Boyer
2024-02-07 3:13 ` [PATCH v3 12/13] net/ionic: optimize device stop operation Andrew Boyer
` (2 subsequent siblings)
13 siblings, 0 replies; 52+ messages in thread
From: Andrew Boyer @ 2024-02-07 3:13 UTC (permalink / raw)
To: dev; +Cc: Andrew Boyer
Use a single device reset command to speed up dev_close(). The LIF stop
and port reset commands are not needed.
This reduces the outage window when restarting the process by about 2ms
plus another 1ms per queue.
Signed-off-by: Andrew Boyer <andrew.boyer@amd.com>
---
drivers/net/ionic/ionic_ethdev.c | 3 ---
drivers/net/ionic/ionic_lif.c | 8 +-------
2 files changed, 1 insertion(+), 10 deletions(-)
diff --git a/drivers/net/ionic/ionic_ethdev.c b/drivers/net/ionic/ionic_ethdev.c
index bedcf958e2..7e80751846 100644
--- a/drivers/net/ionic/ionic_ethdev.c
+++ b/drivers/net/ionic/ionic_ethdev.c
@@ -1007,13 +1007,10 @@ ionic_dev_close(struct rte_eth_dev *eth_dev)
if (rte_eal_process_type() != RTE_PROC_PRIMARY)
return 0;
- ionic_lif_stop(lif);
-
IONIC_PRINT(NOTICE, "Removing device %s", eth_dev->device->name);
if (adapter->intf->unconfigure_intr)
(*adapter->intf->unconfigure_intr)(adapter);
- ionic_port_reset(adapter);
ionic_reset(adapter);
ionic_lif_free_queues(lif);
diff --git a/drivers/net/ionic/ionic_lif.c b/drivers/net/ionic/ionic_lif.c
index 2713f8aa24..90efcc8cbb 100644
--- a/drivers/net/ionic/ionic_lif.c
+++ b/drivers/net/ionic/ionic_lif.c
@@ -1231,13 +1231,7 @@ ionic_lif_rss_setup(struct ionic_lif *lif)
static void
ionic_lif_rss_teardown(struct ionic_lif *lif)
{
- if (!lif->rss_ind_tbl)
- return;
-
- if (lif->rss_ind_tbl_z) {
- /* Disable RSS on the NIC */
- ionic_lif_rss_config(lif, 0x0, NULL, NULL);
-
+ if (lif->rss_ind_tbl) {
lif->rss_ind_tbl = NULL;
lif->rss_ind_tbl_pa = 0;
rte_memzone_free(lif->rss_ind_tbl_z);
--
2.17.1
^ permalink raw reply [flat|nested] 52+ messages in thread
* [PATCH v3 12/13] net/ionic: optimize device stop operation
2024-02-07 3:13 ` [PATCH v3 00/13] net/ionic: miscellaneous fixes and improvements Andrew Boyer
` (10 preceding siblings ...)
2024-02-07 3:13 ` [PATCH v3 11/13] net/ionic: optimize device close operation Andrew Boyer
@ 2024-02-07 3:13 ` Andrew Boyer
2024-02-07 3:13 ` [PATCH v3 13/13] net/ionic: optimize device start operation Andrew Boyer
2024-02-07 15:45 ` [PATCH v3 00/13] net/ionic: miscellaneous fixes and improvements Ferruh Yigit
13 siblings, 0 replies; 52+ messages in thread
From: Andrew Boyer @ 2024-02-07 3:13 UTC (permalink / raw)
To: dev; +Cc: Andrew Boyer
Split the queue_stop operation into first-half and second-half helpers.
Move the command context from the stack into each Rx/Tx queue struct.
Expose some needed adminq interfaces.
This allows us to batch up the queue commands during dev_stop(), reducing
the outage window when restarting the process by about 1ms per queue.
Signed-off-by: Andrew Boyer <andrew.boyer@amd.com>
---
drivers/net/ionic/ionic.h | 3 ++
drivers/net/ionic/ionic_lif.c | 68 ++++++++++++++++++++++-------
drivers/net/ionic/ionic_lif.h | 12 ++++--
drivers/net/ionic/ionic_main.c | 17 +++++++-
drivers/net/ionic/ionic_rxtx.c | 78 ++++++++++++++++++++--------------
drivers/net/ionic/ionic_rxtx.h | 14 +++++-
6 files changed, 138 insertions(+), 54 deletions(-)
diff --git a/drivers/net/ionic/ionic.h b/drivers/net/ionic/ionic.h
index c479eaba74..cb4ea450a9 100644
--- a/drivers/net/ionic/ionic.h
+++ b/drivers/net/ionic/ionic.h
@@ -83,7 +83,10 @@ struct ionic_admin_ctx {
union ionic_adminq_comp comp;
};
+int ionic_adminq_post(struct ionic_lif *lif, struct ionic_admin_ctx *ctx);
int ionic_adminq_post_wait(struct ionic_lif *lif, struct ionic_admin_ctx *ctx);
+int ionic_adminq_wait(struct ionic_lif *lif, struct ionic_admin_ctx *ctx);
+uint16_t ionic_adminq_space_avail(struct ionic_lif *lif);
int ionic_dev_cmd_wait_check(struct ionic_dev *idev, unsigned long max_wait);
int ionic_setup(struct ionic_adapter *adapter);
diff --git a/drivers/net/ionic/ionic_lif.c b/drivers/net/ionic/ionic_lif.c
index 90efcc8cbb..45317590fa 100644
--- a/drivers/net/ionic/ionic_lif.c
+++ b/drivers/net/ionic/ionic_lif.c
@@ -31,11 +31,15 @@ static int ionic_lif_addr_add(struct ionic_lif *lif, const uint8_t *addr);
static int ionic_lif_addr_del(struct ionic_lif *lif, const uint8_t *addr);
static int
-ionic_qcq_disable(struct ionic_qcq *qcq)
+ionic_qcq_disable_nowait(struct ionic_qcq *qcq,
+ struct ionic_admin_ctx *ctx)
{
+ int err;
+
struct ionic_queue *q = &qcq->q;
struct ionic_lif *lif = qcq->lif;
- struct ionic_admin_ctx ctx = {
+
+ *ctx = (struct ionic_admin_ctx) {
.pending_work = true,
.cmd.q_control = {
.opcode = IONIC_CMD_Q_CONTROL,
@@ -45,28 +49,39 @@ ionic_qcq_disable(struct ionic_qcq *qcq)
},
};
- return ionic_adminq_post_wait(lif, &ctx);
+ /* Does not wait for command completion */
+ err = ionic_adminq_post(lif, ctx);
+ if (err)
+ ctx->pending_work = false;
+ return err;
}
void
ionic_lif_stop(struct ionic_lif *lif)
{
- uint32_t i;
+ struct rte_eth_dev *dev = lif->eth_dev;
+ uint32_t i, j, chunk;
IONIC_PRINT_CALL();
lif->state &= ~IONIC_LIF_F_UP;
- for (i = 0; i < lif->nrxqcqs; i++) {
- struct ionic_rx_qcq *rxq = lif->rxqcqs[i];
- if (rxq->flags & IONIC_QCQ_F_INITED)
- (void)ionic_dev_rx_queue_stop(lif->eth_dev, i);
+ chunk = ionic_adminq_space_avail(lif);
+
+ for (i = 0; i < lif->nrxqcqs; i += chunk) {
+ for (j = 0; j < chunk && i + j < lif->nrxqcqs; j++)
+ ionic_dev_rx_queue_stop_firsthalf(dev, i + j);
+
+ for (j = 0; j < chunk && i + j < lif->nrxqcqs; j++)
+ ionic_dev_rx_queue_stop_secondhalf(dev, i + j);
}
- for (i = 0; i < lif->ntxqcqs; i++) {
- struct ionic_tx_qcq *txq = lif->txqcqs[i];
- if (txq->flags & IONIC_QCQ_F_INITED)
- (void)ionic_dev_tx_queue_stop(lif->eth_dev, i);
+ for (i = 0; i < lif->ntxqcqs; i += chunk) {
+ for (j = 0; j < chunk && i + j < lif->ntxqcqs; j++)
+ ionic_dev_tx_queue_stop_firsthalf(dev, i + j);
+
+ for (j = 0; j < chunk && i + j < lif->ntxqcqs; j++)
+ ionic_dev_tx_queue_stop_secondhalf(dev, i + j);
}
}
@@ -1240,21 +1255,42 @@ ionic_lif_rss_teardown(struct ionic_lif *lif)
}
void
-ionic_lif_txq_deinit(struct ionic_tx_qcq *txq)
+ionic_lif_txq_deinit_nowait(struct ionic_tx_qcq *txq)
{
- ionic_qcq_disable(&txq->qcq);
+ ionic_qcq_disable_nowait(&txq->qcq, &txq->admin_ctx);
txq->flags &= ~IONIC_QCQ_F_INITED;
}
void
-ionic_lif_rxq_deinit(struct ionic_rx_qcq *rxq)
+ionic_lif_txq_stats(struct ionic_tx_qcq *txq)
+{
+ struct ionic_tx_stats *stats = &txq->stats;
+
+ IONIC_PRINT(DEBUG, "TX queue %u pkts %ju tso %ju",
+ txq->qcq.q.index, stats->packets, stats->tso);
+ IONIC_PRINT(DEBUG, "TX queue %u comps %ju (%ju per)",
+ txq->qcq.q.index, stats->comps,
+ stats->comps ? stats->packets / stats->comps : 0);
+}
+
+void
+ionic_lif_rxq_deinit_nowait(struct ionic_rx_qcq *rxq)
{
- ionic_qcq_disable(&rxq->qcq);
+ ionic_qcq_disable_nowait(&rxq->qcq, &rxq->admin_ctx);
rxq->flags &= ~IONIC_QCQ_F_INITED;
}
+void
+ionic_lif_rxq_stats(struct ionic_rx_qcq *rxq)
+{
+ struct ionic_rx_stats *stats = &rxq->stats;
+
+ IONIC_PRINT(DEBUG, "RX queue %u pkts %ju mtod %ju",
+ rxq->qcq.q.index, stats->packets, stats->mtods);
+}
+
static void
ionic_lif_adminq_deinit(struct ionic_lif *lif)
{
diff --git a/drivers/net/ionic/ionic_lif.h b/drivers/net/ionic/ionic_lif.h
index cac7a4583b..ee13f5b7c8 100644
--- a/drivers/net/ionic/ionic_lif.h
+++ b/drivers/net/ionic/ionic_lif.h
@@ -10,7 +10,7 @@
#include <rte_ethdev.h>
#include <rte_ether.h>
-#include "ionic_osdep.h"
+#include "ionic.h"
#include "ionic_dev.h"
#include "ionic_rx_filter.h"
@@ -99,6 +99,8 @@ struct ionic_rx_qcq {
/* cacheline4+ */
struct rte_mbuf *mbs[IONIC_MBUF_BULK_ALLOC] __rte_cache_aligned;
+
+ struct ionic_admin_ctx admin_ctx;
};
struct ionic_tx_qcq {
@@ -112,6 +114,8 @@ struct ionic_tx_qcq {
uint16_t flags;
struct ionic_tx_stats stats;
+
+ struct ionic_admin_ctx admin_ctx;
};
#define IONIC_Q_TO_QCQ(_q) container_of(_q, struct ionic_qcq, q)
@@ -225,10 +229,12 @@ int ionic_tx_qcq_alloc(struct ionic_lif *lif, uint32_t socket_id,
void ionic_qcq_free(struct ionic_qcq *qcq);
int ionic_lif_rxq_init(struct ionic_rx_qcq *rxq);
-void ionic_lif_rxq_deinit(struct ionic_rx_qcq *rxq);
+void ionic_lif_rxq_deinit_nowait(struct ionic_rx_qcq *rxq);
+void ionic_lif_rxq_stats(struct ionic_rx_qcq *rxq);
int ionic_lif_txq_init(struct ionic_tx_qcq *txq);
-void ionic_lif_txq_deinit(struct ionic_tx_qcq *txq);
+void ionic_lif_txq_deinit_nowait(struct ionic_tx_qcq *txq);
+void ionic_lif_txq_stats(struct ionic_tx_qcq *txq);
int ionic_lif_rss_config(struct ionic_lif *lif, const uint16_t types,
const uint8_t *key, const uint32_t *indir);
diff --git a/drivers/net/ionic/ionic_main.c b/drivers/net/ionic/ionic_main.c
index c957d55bf9..1f24f64a33 100644
--- a/drivers/net/ionic/ionic_main.c
+++ b/drivers/net/ionic/ionic_main.c
@@ -180,6 +180,12 @@ ionic_adminq_service(struct ionic_cq *cq, uint16_t cq_desc_index,
return true;
}
+uint16_t
+ionic_adminq_space_avail(struct ionic_lif *lif)
+{
+ return ionic_q_space_avail(&lif->adminqcq->qcq.q);
+}
+
/** ionic_adminq_post - Post an admin command.
* @lif: Handle to lif.
* @cmd_ctx: Api admin command context.
@@ -191,7 +197,7 @@ ionic_adminq_service(struct ionic_cq *cq, uint16_t cq_desc_index,
*
* Return: zero or negative error status.
*/
-static int
+int
ionic_adminq_post(struct ionic_lif *lif, struct ionic_admin_ctx *ctx)
{
struct ionic_queue *q = &lif->adminqcq->qcq.q;
@@ -279,7 +285,6 @@ ionic_adminq_wait_for_completion(struct ionic_lif *lif,
int
ionic_adminq_post_wait(struct ionic_lif *lif, struct ionic_admin_ctx *ctx)
{
- bool done;
int err;
IONIC_PRINT(DEBUG, "Sending %s (%d) via the admin queue",
@@ -292,6 +297,14 @@ ionic_adminq_post_wait(struct ionic_lif *lif, struct ionic_admin_ctx *ctx)
return err;
}
+ return ionic_adminq_wait(lif, ctx);
+}
+
+int
+ionic_adminq_wait(struct ionic_lif *lif, struct ionic_admin_ctx *ctx)
+{
+ bool done;
+
done = ionic_adminq_wait_for_completion(lif, ctx,
IONIC_DEVCMD_TIMEOUT);
diff --git a/drivers/net/ionic/ionic_rxtx.c b/drivers/net/ionic/ionic_rxtx.c
index d92fa1cca7..774dc596c0 100644
--- a/drivers/net/ionic/ionic_rxtx.c
+++ b/drivers/net/ionic/ionic_rxtx.c
@@ -92,36 +92,40 @@ ionic_dev_tx_queue_release(struct rte_eth_dev *dev, uint16_t qid)
}
int __rte_cold
-ionic_dev_tx_queue_stop(struct rte_eth_dev *eth_dev, uint16_t tx_queue_id)
+ionic_dev_tx_queue_stop(struct rte_eth_dev *dev, uint16_t tx_queue_id)
{
- struct ionic_tx_stats *stats;
- struct ionic_tx_qcq *txq;
+ ionic_dev_tx_queue_stop_firsthalf(dev, tx_queue_id);
+ ionic_dev_tx_queue_stop_secondhalf(dev, tx_queue_id);
+
+ return 0;
+}
+
+void __rte_cold
+ionic_dev_tx_queue_stop_firsthalf(struct rte_eth_dev *dev,
+ uint16_t tx_queue_id)
+{
+ struct ionic_tx_qcq *txq = dev->data->tx_queues[tx_queue_id];
IONIC_PRINT(DEBUG, "Stopping TX queue %u", tx_queue_id);
- txq = eth_dev->data->tx_queues[tx_queue_id];
+ dev->data->tx_queue_state[tx_queue_id] = RTE_ETH_QUEUE_STATE_STOPPED;
- eth_dev->data->tx_queue_state[tx_queue_id] =
- RTE_ETH_QUEUE_STATE_STOPPED;
+ ionic_lif_txq_deinit_nowait(txq);
+}
- /*
- * Note: we should better post NOP Tx desc and wait for its completion
- * before disabling Tx queue
- */
+void __rte_cold
+ionic_dev_tx_queue_stop_secondhalf(struct rte_eth_dev *dev,
+ uint16_t tx_queue_id)
+{
+ struct ionic_lif *lif = IONIC_ETH_DEV_TO_LIF(dev);
+ struct ionic_tx_qcq *txq = dev->data->tx_queues[tx_queue_id];
- ionic_lif_txq_deinit(txq);
+ ionic_adminq_wait(lif, &txq->admin_ctx);
/* Free all buffers from descriptor ring */
ionic_tx_empty(txq);
- stats = &txq->stats;
- IONIC_PRINT(DEBUG, "TX queue %u pkts %ju tso %ju",
- txq->qcq.q.index, stats->packets, stats->tso);
- IONIC_PRINT(DEBUG, "TX queue %u comps %ju (%ju per)",
- txq->qcq.q.index, stats->comps,
- stats->comps ? stats->packets / stats->comps : 0);
-
- return 0;
+ ionic_lif_txq_stats(txq);
}
int __rte_cold
@@ -726,28 +730,40 @@ ionic_dev_rx_queue_start(struct rte_eth_dev *eth_dev, uint16_t rx_queue_id)
* Stop Receive Units for specified queue.
*/
int __rte_cold
-ionic_dev_rx_queue_stop(struct rte_eth_dev *eth_dev, uint16_t rx_queue_id)
+ionic_dev_rx_queue_stop(struct rte_eth_dev *dev, uint16_t rx_queue_id)
{
- uint8_t *rx_queue_state = eth_dev->data->rx_queue_state;
- struct ionic_rx_stats *stats;
- struct ionic_rx_qcq *rxq;
+ ionic_dev_rx_queue_stop_firsthalf(dev, rx_queue_id);
+ ionic_dev_rx_queue_stop_secondhalf(dev, rx_queue_id);
+
+ return 0;
+}
+
+void __rte_cold
+ionic_dev_rx_queue_stop_firsthalf(struct rte_eth_dev *dev,
+ uint16_t rx_queue_id)
+{
+ struct ionic_rx_qcq *rxq = dev->data->rx_queues[rx_queue_id];
IONIC_PRINT(DEBUG, "Stopping RX queue %u", rx_queue_id);
- rxq = eth_dev->data->rx_queues[rx_queue_id];
+ dev->data->rx_queue_state[rx_queue_id] = RTE_ETH_QUEUE_STATE_STOPPED;
+
+ ionic_lif_rxq_deinit_nowait(rxq);
+}
- rx_queue_state[rx_queue_id] = RTE_ETH_QUEUE_STATE_STOPPED;
+void __rte_cold
+ionic_dev_rx_queue_stop_secondhalf(struct rte_eth_dev *dev,
+ uint16_t rx_queue_id)
+{
+ struct ionic_lif *lif = IONIC_ETH_DEV_TO_LIF(dev);
+ struct ionic_rx_qcq *rxq = dev->data->rx_queues[rx_queue_id];
- ionic_lif_rxq_deinit(rxq);
+ ionic_adminq_wait(lif, &rxq->admin_ctx);
/* Free all buffers from descriptor ring */
ionic_rx_empty(rxq);
- stats = &rxq->stats;
- IONIC_PRINT(DEBUG, "RX queue %u pkts %ju mtod %ju",
- rxq->qcq.q.index, stats->packets, stats->mtods);
-
- return 0;
+ ionic_lif_rxq_stats(rxq);
}
int
diff --git a/drivers/net/ionic/ionic_rxtx.h b/drivers/net/ionic/ionic_rxtx.h
index 5348395956..7ca23178cc 100644
--- a/drivers/net/ionic/ionic_rxtx.h
+++ b/drivers/net/ionic/ionic_rxtx.h
@@ -37,14 +37,24 @@ int ionic_dev_rx_queue_setup(struct rte_eth_dev *dev, uint16_t rx_queue_id,
const struct rte_eth_rxconf *rx_conf, struct rte_mempool *mp);
void ionic_dev_rx_queue_release(struct rte_eth_dev *dev, uint16_t qid);
int ionic_dev_rx_queue_start(struct rte_eth_dev *dev, uint16_t rx_queue_id);
-int ionic_dev_rx_queue_stop(struct rte_eth_dev *eth_dev, uint16_t rx_queue_id);
+int ionic_dev_rx_queue_stop(struct rte_eth_dev *dev, uint16_t rx_queue_id);
int ionic_dev_tx_queue_setup(struct rte_eth_dev *dev, uint16_t tx_queue_id,
uint16_t nb_desc, uint32_t socket_id,
const struct rte_eth_txconf *tx_conf);
void ionic_dev_tx_queue_release(struct rte_eth_dev *dev, uint16_t qid);
-int ionic_dev_tx_queue_stop(struct rte_eth_dev *eth_dev, uint16_t tx_queue_id);
int ionic_dev_tx_queue_start(struct rte_eth_dev *dev, uint16_t tx_queue_id);
+int ionic_dev_tx_queue_stop(struct rte_eth_dev *dev, uint16_t tx_queue_id);
+
+/* Helpers for optimized dev_stop() */
+void ionic_dev_rx_queue_stop_firsthalf(struct rte_eth_dev *dev,
+ uint16_t rx_queue_id);
+void ionic_dev_rx_queue_stop_secondhalf(struct rte_eth_dev *dev,
+ uint16_t rx_queue_id);
+void ionic_dev_tx_queue_stop_firsthalf(struct rte_eth_dev *dev,
+ uint16_t tx_queue_id);
+void ionic_dev_tx_queue_stop_secondhalf(struct rte_eth_dev *dev,
+ uint16_t tx_queue_id);
void ionic_rxq_info_get(struct rte_eth_dev *dev, uint16_t queue_id,
struct rte_eth_rxq_info *qinfo);
--
2.17.1
^ permalink raw reply [flat|nested] 52+ messages in thread
* [PATCH v3 13/13] net/ionic: optimize device start operation
2024-02-07 3:13 ` [PATCH v3 00/13] net/ionic: miscellaneous fixes and improvements Andrew Boyer
` (11 preceding siblings ...)
2024-02-07 3:13 ` [PATCH v3 12/13] net/ionic: optimize device stop operation Andrew Boyer
@ 2024-02-07 3:13 ` Andrew Boyer
2024-02-07 15:45 ` [PATCH v3 00/13] net/ionic: miscellaneous fixes and improvements Ferruh Yigit
13 siblings, 0 replies; 52+ messages in thread
From: Andrew Boyer @ 2024-02-07 3:13 UTC (permalink / raw)
To: dev; +Cc: Andrew Boyer
Split the queue_start operation into first-half and second-half helpers.
This allows us to batch up the queue commands during dev_start(), reducing
the outage window when restarting the process by about 1ms per queue.
Signed-off-by: Andrew Boyer <andrew.boyer@amd.com>
---
drivers/net/ionic/ionic_lif.c | 136 +++++++++++++++++++++++----------
drivers/net/ionic/ionic_lif.h | 6 +-
drivers/net/ionic/ionic_rxtx.c | 81 ++++++++++++++++----
drivers/net/ionic/ionic_rxtx.h | 10 +++
4 files changed, 176 insertions(+), 57 deletions(-)
diff --git a/drivers/net/ionic/ionic_lif.c b/drivers/net/ionic/ionic_lif.c
index 45317590fa..93a1011772 100644
--- a/drivers/net/ionic/ionic_lif.c
+++ b/drivers/net/ionic/ionic_lif.c
@@ -1601,13 +1601,16 @@ ionic_lif_set_features(struct ionic_lif *lif)
}
int
-ionic_lif_txq_init(struct ionic_tx_qcq *txq)
+ionic_lif_txq_init_nowait(struct ionic_tx_qcq *txq)
{
struct ionic_qcq *qcq = &txq->qcq;
struct ionic_queue *q = &qcq->q;
struct ionic_lif *lif = qcq->lif;
struct ionic_cq *cq = &qcq->cq;
- struct ionic_admin_ctx ctx = {
+ struct ionic_admin_ctx *ctx = &txq->admin_ctx;
+ int err;
+
+ *ctx = (struct ionic_admin_ctx) {
.pending_work = true,
.cmd.q_init = {
.opcode = IONIC_CMD_Q_INIT,
@@ -1621,32 +1624,41 @@ ionic_lif_txq_init(struct ionic_tx_qcq *txq)
.sg_ring_base = rte_cpu_to_le_64(q->sg_base_pa),
},
};
- int err;
if (txq->flags & IONIC_QCQ_F_SG)
- ctx.cmd.q_init.flags |= rte_cpu_to_le_16(IONIC_QINIT_F_SG);
+ ctx->cmd.q_init.flags |= rte_cpu_to_le_16(IONIC_QINIT_F_SG);
if (txq->flags & IONIC_QCQ_F_CMB) {
- ctx.cmd.q_init.flags |= rte_cpu_to_le_16(IONIC_QINIT_F_CMB);
- ctx.cmd.q_init.ring_base = rte_cpu_to_le_64(q->cmb_base_pa);
+ ctx->cmd.q_init.flags |= rte_cpu_to_le_16(IONIC_QINIT_F_CMB);
+ ctx->cmd.q_init.ring_base = rte_cpu_to_le_64(q->cmb_base_pa);
} else {
- ctx.cmd.q_init.ring_base = rte_cpu_to_le_64(q->base_pa);
+ ctx->cmd.q_init.ring_base = rte_cpu_to_le_64(q->base_pa);
}
IONIC_PRINT(DEBUG, "txq_init.index %d", q->index);
IONIC_PRINT(DEBUG, "txq_init.ring_base 0x%" PRIx64 "", q->base_pa);
IONIC_PRINT(DEBUG, "txq_init.ring_size %d",
- ctx.cmd.q_init.ring_size);
- IONIC_PRINT(DEBUG, "txq_init.ver %u", ctx.cmd.q_init.ver);
+ ctx->cmd.q_init.ring_size);
+ IONIC_PRINT(DEBUG, "txq_init.ver %u", ctx->cmd.q_init.ver);
ionic_q_reset(q);
ionic_cq_reset(cq);
- err = ionic_adminq_post_wait(lif, &ctx);
+ /* Caller responsible for calling ionic_lif_txq_init_done() */
+ err = ionic_adminq_post(lif, ctx);
if (err)
- return err;
+ ctx->pending_work = false;
+ return err;
+}
- q->hw_type = ctx.comp.q_init.hw_type;
- q->hw_index = rte_le_to_cpu_32(ctx.comp.q_init.hw_index);
+void
+ionic_lif_txq_init_done(struct ionic_tx_qcq *txq)
+{
+ struct ionic_lif *lif = txq->qcq.lif;
+ struct ionic_queue *q = &txq->qcq.q;
+ struct ionic_admin_ctx *ctx = &txq->admin_ctx;
+
+ q->hw_type = ctx->comp.q_init.hw_type;
+ q->hw_index = rte_le_to_cpu_32(ctx->comp.q_init.hw_index);
q->db = ionic_db_map(lif, q);
IONIC_PRINT(DEBUG, "txq->hw_type %d", q->hw_type);
@@ -1654,18 +1666,19 @@ ionic_lif_txq_init(struct ionic_tx_qcq *txq)
IONIC_PRINT(DEBUG, "txq->db %p", q->db);
txq->flags |= IONIC_QCQ_F_INITED;
-
- return 0;
}
int
-ionic_lif_rxq_init(struct ionic_rx_qcq *rxq)
+ionic_lif_rxq_init_nowait(struct ionic_rx_qcq *rxq)
{
struct ionic_qcq *qcq = &rxq->qcq;
struct ionic_queue *q = &qcq->q;
struct ionic_lif *lif = qcq->lif;
struct ionic_cq *cq = &qcq->cq;
- struct ionic_admin_ctx ctx = {
+ struct ionic_admin_ctx *ctx = &rxq->admin_ctx;
+ int err;
+
+ *ctx = (struct ionic_admin_ctx) {
.pending_work = true,
.cmd.q_init = {
.opcode = IONIC_CMD_Q_INIT,
@@ -1679,32 +1692,41 @@ ionic_lif_rxq_init(struct ionic_rx_qcq *rxq)
.sg_ring_base = rte_cpu_to_le_64(q->sg_base_pa),
},
};
- int err;
if (rxq->flags & IONIC_QCQ_F_SG)
- ctx.cmd.q_init.flags |= rte_cpu_to_le_16(IONIC_QINIT_F_SG);
+ ctx->cmd.q_init.flags |= rte_cpu_to_le_16(IONIC_QINIT_F_SG);
if (rxq->flags & IONIC_QCQ_F_CMB) {
- ctx.cmd.q_init.flags |= rte_cpu_to_le_16(IONIC_QINIT_F_CMB);
- ctx.cmd.q_init.ring_base = rte_cpu_to_le_64(q->cmb_base_pa);
+ ctx->cmd.q_init.flags |= rte_cpu_to_le_16(IONIC_QINIT_F_CMB);
+ ctx->cmd.q_init.ring_base = rte_cpu_to_le_64(q->cmb_base_pa);
} else {
- ctx.cmd.q_init.ring_base = rte_cpu_to_le_64(q->base_pa);
+ ctx->cmd.q_init.ring_base = rte_cpu_to_le_64(q->base_pa);
}
IONIC_PRINT(DEBUG, "rxq_init.index %d", q->index);
IONIC_PRINT(DEBUG, "rxq_init.ring_base 0x%" PRIx64 "", q->base_pa);
IONIC_PRINT(DEBUG, "rxq_init.ring_size %d",
- ctx.cmd.q_init.ring_size);
- IONIC_PRINT(DEBUG, "rxq_init.ver %u", ctx.cmd.q_init.ver);
+ ctx->cmd.q_init.ring_size);
+ IONIC_PRINT(DEBUG, "rxq_init.ver %u", ctx->cmd.q_init.ver);
ionic_q_reset(q);
ionic_cq_reset(cq);
- err = ionic_adminq_post_wait(lif, &ctx);
+ /* Caller responsible for calling ionic_lif_rxq_init_done() */
+ err = ionic_adminq_post(lif, ctx);
if (err)
- return err;
+ ctx->pending_work = false;
+ return err;
+}
- q->hw_type = ctx.comp.q_init.hw_type;
- q->hw_index = rte_le_to_cpu_32(ctx.comp.q_init.hw_index);
+void
+ionic_lif_rxq_init_done(struct ionic_rx_qcq *rxq)
+{
+ struct ionic_lif *lif = rxq->qcq.lif;
+ struct ionic_queue *q = &rxq->qcq.q;
+ struct ionic_admin_ctx *ctx = &rxq->admin_ctx;
+
+ q->hw_type = ctx->comp.q_init.hw_type;
+ q->hw_index = rte_le_to_cpu_32(ctx->comp.q_init.hw_index);
q->db = ionic_db_map(lif, q);
rxq->flags |= IONIC_QCQ_F_INITED;
@@ -1712,8 +1734,6 @@ ionic_lif_rxq_init(struct ionic_rx_qcq *rxq)
IONIC_PRINT(DEBUG, "rxq->hw_type %d", q->hw_type);
IONIC_PRINT(DEBUG, "rxq->hw_index %d", q->hw_index);
IONIC_PRINT(DEBUG, "rxq->db %p", q->db);
-
- return 0;
}
static int
@@ -1962,9 +1982,11 @@ ionic_lif_configure(struct ionic_lif *lif)
int
ionic_lif_start(struct ionic_lif *lif)
{
+ struct rte_eth_dev *dev = lif->eth_dev;
uint32_t rx_mode;
- uint32_t i;
+ uint32_t i, j, chunk;
int err;
+ bool fatal = false;
err = ionic_lif_rss_setup(lif);
if (err)
@@ -1985,25 +2007,57 @@ ionic_lif_start(struct ionic_lif *lif)
"on port %u",
lif->nrxqcqs, lif->ntxqcqs, lif->port_id);
- for (i = 0; i < lif->nrxqcqs; i++) {
- struct ionic_rx_qcq *rxq = lif->rxqcqs[i];
- if (!(rxq->flags & IONIC_QCQ_F_DEFERRED)) {
- err = ionic_dev_rx_queue_start(lif->eth_dev, i);
+ chunk = ionic_adminq_space_avail(lif);
+
+ for (i = 0; i < lif->nrxqcqs; i += chunk) {
+ if (lif->rxqcqs[0]->flags & IONIC_QCQ_F_DEFERRED) {
+ IONIC_PRINT(DEBUG, "Rx queue start deferred");
+ break;
+ }
+
+ for (j = 0; j < chunk && i + j < lif->nrxqcqs; j++) {
+ err = ionic_dev_rx_queue_start_firsthalf(dev, i + j);
+ if (err) {
+ fatal = true;
+ break;
+ }
+ }
+ for (j = 0; j < chunk && i + j < lif->nrxqcqs; j++) {
+ /* Commands that failed to post return immediately */
+ err = ionic_dev_rx_queue_start_secondhalf(dev, i + j);
if (err)
- return err;
+ /* Don't break */
+ fatal = true;
}
}
+ if (fatal)
+ return -EIO;
- for (i = 0; i < lif->ntxqcqs; i++) {
- struct ionic_tx_qcq *txq = lif->txqcqs[i];
- if (!(txq->flags & IONIC_QCQ_F_DEFERRED)) {
- err = ionic_dev_tx_queue_start(lif->eth_dev, i);
+ for (i = 0; i < lif->ntxqcqs; i += chunk) {
+ if (lif->txqcqs[0]->flags & IONIC_QCQ_F_DEFERRED) {
+ IONIC_PRINT(DEBUG, "Tx queue start deferred");
+ break;
+ }
+
+ for (j = 0; j < chunk && i + j < lif->ntxqcqs; j++) {
+ err = ionic_dev_tx_queue_start_firsthalf(dev, i + j);
+ if (err) {
+ fatal = true;
+ break;
+ }
+ }
+ for (j = 0; j < chunk && i + j < lif->ntxqcqs; j++) {
+ /* Commands that failed to post return immediately */
+ err = ionic_dev_tx_queue_start_secondhalf(dev, i + j);
if (err)
- return err;
+ /* Don't break */
+ fatal = true;
}
}
+ if (fatal)
+ return -EIO;
/* Carrier ON here */
lif->state |= IONIC_LIF_F_UP;
diff --git a/drivers/net/ionic/ionic_lif.h b/drivers/net/ionic/ionic_lif.h
index ee13f5b7c8..591cf1a2ff 100644
--- a/drivers/net/ionic/ionic_lif.h
+++ b/drivers/net/ionic/ionic_lif.h
@@ -228,11 +228,13 @@ int ionic_tx_qcq_alloc(struct ionic_lif *lif, uint32_t socket_id,
struct ionic_tx_qcq **qcq_out);
void ionic_qcq_free(struct ionic_qcq *qcq);
-int ionic_lif_rxq_init(struct ionic_rx_qcq *rxq);
+int ionic_lif_rxq_init_nowait(struct ionic_rx_qcq *rxq);
+void ionic_lif_rxq_init_done(struct ionic_rx_qcq *rxq);
void ionic_lif_rxq_deinit_nowait(struct ionic_rx_qcq *rxq);
void ionic_lif_rxq_stats(struct ionic_rx_qcq *rxq);
-int ionic_lif_txq_init(struct ionic_tx_qcq *txq);
+int ionic_lif_txq_init_nowait(struct ionic_tx_qcq *txq);
+void ionic_lif_txq_init_done(struct ionic_tx_qcq *txq);
void ionic_lif_txq_deinit_nowait(struct ionic_tx_qcq *txq);
void ionic_lif_txq_stats(struct ionic_tx_qcq *txq);
diff --git a/drivers/net/ionic/ionic_rxtx.c b/drivers/net/ionic/ionic_rxtx.c
index 774dc596c0..ad04e987eb 100644
--- a/drivers/net/ionic/ionic_rxtx.c
+++ b/drivers/net/ionic/ionic_rxtx.c
@@ -203,27 +203,54 @@ ionic_dev_tx_queue_setup(struct rte_eth_dev *eth_dev, uint16_t tx_queue_id,
* Start Transmit Units for specified queue.
*/
int __rte_cold
-ionic_dev_tx_queue_start(struct rte_eth_dev *eth_dev, uint16_t tx_queue_id)
+ionic_dev_tx_queue_start(struct rte_eth_dev *dev, uint16_t tx_queue_id)
{
- uint8_t *tx_queue_state = eth_dev->data->tx_queue_state;
- struct ionic_tx_qcq *txq;
int err;
+ err = ionic_dev_tx_queue_start_firsthalf(dev, tx_queue_id);
+ if (err)
+ return err;
+
+ return ionic_dev_tx_queue_start_secondhalf(dev, tx_queue_id);
+}
+
+int __rte_cold
+ionic_dev_tx_queue_start_firsthalf(struct rte_eth_dev *dev,
+ uint16_t tx_queue_id)
+{
+ uint8_t *tx_queue_state = dev->data->tx_queue_state;
+ struct ionic_tx_qcq *txq = dev->data->tx_queues[tx_queue_id];
+
if (tx_queue_state[tx_queue_id] == RTE_ETH_QUEUE_STATE_STARTED) {
IONIC_PRINT(DEBUG, "TX queue %u already started",
tx_queue_id);
return 0;
}
- txq = eth_dev->data->tx_queues[tx_queue_id];
-
IONIC_PRINT(DEBUG, "Starting TX queue %u, %u descs",
tx_queue_id, txq->qcq.q.num_descs);
- err = ionic_lif_txq_init(txq);
+ return ionic_lif_txq_init_nowait(txq);
+}
+
+int __rte_cold
+ionic_dev_tx_queue_start_secondhalf(struct rte_eth_dev *dev,
+ uint16_t tx_queue_id)
+{
+ uint8_t *tx_queue_state = dev->data->tx_queue_state;
+ struct ionic_lif *lif = IONIC_ETH_DEV_TO_LIF(dev);
+ struct ionic_tx_qcq *txq = dev->data->tx_queues[tx_queue_id];
+ int err;
+
+ if (tx_queue_state[tx_queue_id] == RTE_ETH_QUEUE_STATE_STARTED)
+ return 0;
+
+ err = ionic_adminq_wait(lif, &txq->admin_ctx);
if (err)
return err;
+ ionic_lif_txq_init_done(txq);
+
tx_queue_state[tx_queue_id] = RTE_ETH_QUEUE_STATE_STARTED;
return 0;
@@ -680,22 +707,31 @@ ionic_rx_init_descriptors(struct ionic_rx_qcq *rxq)
* Start Receive Units for specified queue.
*/
int __rte_cold
-ionic_dev_rx_queue_start(struct rte_eth_dev *eth_dev, uint16_t rx_queue_id)
+ionic_dev_rx_queue_start(struct rte_eth_dev *dev, uint16_t rx_queue_id)
{
- uint8_t *rx_queue_state = eth_dev->data->rx_queue_state;
- struct ionic_rx_qcq *rxq;
- struct ionic_queue *q;
int err;
+ err = ionic_dev_rx_queue_start_firsthalf(dev, rx_queue_id);
+ if (err)
+ return err;
+
+ return ionic_dev_rx_queue_start_secondhalf(dev, rx_queue_id);
+}
+
+int __rte_cold
+ionic_dev_rx_queue_start_firsthalf(struct rte_eth_dev *dev,
+ uint16_t rx_queue_id)
+{
+ uint8_t *rx_queue_state = dev->data->rx_queue_state;
+ struct ionic_rx_qcq *rxq = dev->data->rx_queues[rx_queue_id];
+ struct ionic_queue *q = &rxq->qcq.q;
+
if (rx_queue_state[rx_queue_id] == RTE_ETH_QUEUE_STATE_STARTED) {
IONIC_PRINT(DEBUG, "RX queue %u already started",
rx_queue_id);
return 0;
}
- rxq = eth_dev->data->rx_queues[rx_queue_id];
- q = &rxq->qcq.q;
-
rxq->frame_size = rxq->qcq.lif->frame_size - RTE_ETHER_CRC_LEN;
/* Recalculate segment count based on MTU */
@@ -707,10 +743,27 @@ ionic_dev_rx_queue_start(struct rte_eth_dev *eth_dev, uint16_t rx_queue_id)
ionic_rx_init_descriptors(rxq);
- err = ionic_lif_rxq_init(rxq);
+ return ionic_lif_rxq_init_nowait(rxq);
+}
+
+int __rte_cold
+ionic_dev_rx_queue_start_secondhalf(struct rte_eth_dev *dev,
+ uint16_t rx_queue_id)
+{
+ uint8_t *rx_queue_state = dev->data->rx_queue_state;
+ struct ionic_lif *lif = IONIC_ETH_DEV_TO_LIF(dev);
+ struct ionic_rx_qcq *rxq = dev->data->rx_queues[rx_queue_id];
+ int err;
+
+ if (rx_queue_state[rx_queue_id] == RTE_ETH_QUEUE_STATE_STARTED)
+ return 0;
+
+ err = ionic_adminq_wait(lif, &rxq->admin_ctx);
if (err)
return err;
+ ionic_lif_rxq_init_done(rxq);
+
/* Allocate buffers for descriptor ring */
if (rxq->flags & IONIC_QCQ_F_SG)
err = ionic_rx_fill_sg(rxq);
diff --git a/drivers/net/ionic/ionic_rxtx.h b/drivers/net/ionic/ionic_rxtx.h
index 7ca23178cc..a342afec54 100644
--- a/drivers/net/ionic/ionic_rxtx.h
+++ b/drivers/net/ionic/ionic_rxtx.h
@@ -46,6 +46,16 @@ void ionic_dev_tx_queue_release(struct rte_eth_dev *dev, uint16_t qid);
int ionic_dev_tx_queue_start(struct rte_eth_dev *dev, uint16_t tx_queue_id);
int ionic_dev_tx_queue_stop(struct rte_eth_dev *dev, uint16_t tx_queue_id);
+/* Helpers for optimized dev_start() */
+int ionic_dev_rx_queue_start_firsthalf(struct rte_eth_dev *dev,
+ uint16_t rx_queue_id);
+int ionic_dev_rx_queue_start_secondhalf(struct rte_eth_dev *dev,
+ uint16_t rx_queue_id);
+int ionic_dev_tx_queue_start_firsthalf(struct rte_eth_dev *dev,
+ uint16_t tx_queue_id);
+int ionic_dev_tx_queue_start_secondhalf(struct rte_eth_dev *dev,
+ uint16_t tx_queue_id);
+
/* Helpers for optimized dev_stop() */
void ionic_dev_rx_queue_stop_firsthalf(struct rte_eth_dev *dev,
uint16_t rx_queue_id);
--
2.17.1
^ permalink raw reply [flat|nested] 52+ messages in thread
* Re: [PATCH v3 00/13] net/ionic: miscellaneous fixes and improvements
2024-02-07 3:13 ` [PATCH v3 00/13] net/ionic: miscellaneous fixes and improvements Andrew Boyer
` (12 preceding siblings ...)
2024-02-07 3:13 ` [PATCH v3 13/13] net/ionic: optimize device start operation Andrew Boyer
@ 2024-02-07 15:45 ` Ferruh Yigit
2024-02-08 15:32 ` Ferruh Yigit
13 siblings, 1 reply; 52+ messages in thread
From: Ferruh Yigit @ 2024-02-07 15:45 UTC (permalink / raw)
To: Andrew Boyer; +Cc: dev
On 2/7/2024 3:13 AM, Andrew Boyer wrote:
> This patchset provides miscellaneous fixes and improvements for
> the net/ionic driver used by AMD Pensando devices.
>
> V3:
> - Resend to fix patchwork threading.
>
> V2:
> - Update device stop and device start patches to use compound literals
> as suggested by review.
>
> Akshay Dorwat (1):
> net/ionic: fix RSS query routine
>
> Andrew Boyer (8):
> net/ionic: add stat for completion queue entries processed
> net/ionic: increase max supported MTU to 9750 bytes
> net/ionic: don't auto-enable Rx scatter-gather a second time
> net/ionic: replace non-standard type in structure definition
> net/ionic: fix device close sequence to avoid crash
> net/ionic: optimize device close operation
> net/ionic: optimize device stop operation
> net/ionic: optimize device start operation
>
> Brad Larson (1):
> net/ionic: add flexible firmware xstat counters
>
> Neel Patel (2):
> net/ionic: fix missing volatile type for cqe pointers
> net/ionic: memcpy descriptors when using Q-in-CMB
>
> Vamsi Krishna Atluri (1):
> net/ionic: report 1G and 200G link speeds when applicable
>
For series,
Acked-by: Ferruh Yigit <ferruh.yigit@amd.com>
^ permalink raw reply [flat|nested] 52+ messages in thread
* Re: [PATCH v3 00/13] net/ionic: miscellaneous fixes and improvements
2024-02-07 15:45 ` [PATCH v3 00/13] net/ionic: miscellaneous fixes and improvements Ferruh Yigit
@ 2024-02-08 15:32 ` Ferruh Yigit
0 siblings, 0 replies; 52+ messages in thread
From: Ferruh Yigit @ 2024-02-08 15:32 UTC (permalink / raw)
To: Andrew Boyer; +Cc: dev
On 2/7/2024 3:45 PM, Ferruh Yigit wrote:
> On 2/7/2024 3:13 AM, Andrew Boyer wrote:
>> This patchset provides miscellaneous fixes and improvements for
>> the net/ionic driver used by AMD Pensando devices.
>>
>> V3:
>> - Resend to fix patchwork threading.
>>
>> V2:
>> - Update device stop and device start patches to use compound literals
>> as suggested by review.
>>
>> Akshay Dorwat (1):
>> net/ionic: fix RSS query routine
>>
>> Andrew Boyer (8):
>> net/ionic: add stat for completion queue entries processed
>> net/ionic: increase max supported MTU to 9750 bytes
>> net/ionic: don't auto-enable Rx scatter-gather a second time
>> net/ionic: replace non-standard type in structure definition
>> net/ionic: fix device close sequence to avoid crash
>> net/ionic: optimize device close operation
>> net/ionic: optimize device stop operation
>> net/ionic: optimize device start operation
>>
>> Brad Larson (1):
>> net/ionic: add flexible firmware xstat counters
>>
>> Neel Patel (2):
>> net/ionic: fix missing volatile type for cqe pointers
>> net/ionic: memcpy descriptors when using Q-in-CMB
>>
>> Vamsi Krishna Atluri (1):
>> net/ionic: report 1G and 200G link speeds when applicable
>>
>
> For series,
> Acked-by: Ferruh Yigit <ferruh.yigit@amd.com>
>
Series applied to dpdk-next-net/main, thanks.
^ permalink raw reply [flat|nested] 52+ messages in thread
end of thread, other threads:[~2024-02-08 15:33 UTC | newest]
Thread overview: 52+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2024-02-02 19:32 [PATCH 00/13] net/ionic: miscellaneous fixes and improvements Andrew Boyer
2024-02-02 19:32 ` [PATCH 01/13] net/ionic: add stat for completion queue entries processed Andrew Boyer
2024-02-03 4:24 ` Stephen Hemminger
2024-02-05 12:49 ` Boyer, Andrew
2024-02-02 19:32 ` [PATCH 02/13] net/ionic: increase max supported MTU to 9750 bytes Andrew Boyer
2024-02-02 19:32 ` [PATCH 03/13] net/ionic: don't auto-enable Rx scatter-gather a second time Andrew Boyer
2024-02-02 19:32 ` [PATCH 04/13] net/ionic: fix missing volatile type for cqe pointers Andrew Boyer
2024-02-03 4:26 ` Stephen Hemminger
2024-02-05 13:33 ` Boyer, Andrew
2024-02-02 19:32 ` [PATCH 05/13] net/ionic: replace non-standard type in structure definition Andrew Boyer
2024-02-02 19:32 ` [PATCH 06/13] net/ionic: memcpy descriptors when using Q-in-CMB Andrew Boyer
2024-02-02 19:32 ` [PATCH 07/13] net/ionic: fix RSS query routine Andrew Boyer
2024-02-02 19:32 ` [PATCH 08/13] net/ionic: report 1G and 200G link speeds when applicable Andrew Boyer
2024-02-02 19:32 ` [PATCH 09/13] net/ionic: add flexible firmware xstat counters Andrew Boyer
2024-02-02 19:32 ` [PATCH 10/13] net/ionic: fix device close sequence to avoid crash Andrew Boyer
2024-02-02 19:32 ` [PATCH 11/13] net/ionic: optimize device close operation Andrew Boyer
2024-02-02 19:32 ` [PATCH 12/13] net/ionic: optimize device stop operation Andrew Boyer
2024-02-02 19:32 ` [PATCH 13/13] net/ionic: optimize device start operation Andrew Boyer
2024-02-03 4:28 ` Stephen Hemminger
2024-02-05 13:21 ` Boyer, Andrew
2024-02-05 18:44 ` [PATCH 00/13] net/ionic: miscellaneous fixes and improvements Patrick Robb
2024-02-05 19:41 ` Boyer, Andrew
2024-02-07 2:18 ` [PATCH v2 " Andrew Boyer
2024-02-07 2:18 ` [PATCH v2 01/13] net/ionic: add stat for completion queue entries processed Andrew Boyer
2024-02-07 2:18 ` [PATCH v2 02/13] net/ionic: increase max supported MTU to 9750 bytes Andrew Boyer
2024-02-07 2:18 ` [PATCH v2 03/13] net/ionic: don't auto-enable Rx scatter-gather a second time Andrew Boyer
2024-02-07 2:18 ` [PATCH v2 04/13] net/ionic: fix missing volatile type for cqe pointers Andrew Boyer
2024-02-07 2:18 ` [PATCH v2 05/13] net/ionic: replace non-standard type in structure definition Andrew Boyer
2024-02-07 2:18 ` [PATCH v2 06/13] net/ionic: memcpy descriptors when using Q-in-CMB Andrew Boyer
2024-02-07 2:18 ` [PATCH v2 07/13] net/ionic: fix RSS query routine Andrew Boyer
2024-02-07 2:18 ` [PATCH v2 08/13] net/ionic: report 1G and 200G link speeds when applicable Andrew Boyer
2024-02-07 2:18 ` [PATCH v2 09/13] net/ionic: add flexible firmware xstat counters Andrew Boyer
2024-02-07 2:18 ` [PATCH v2 10/13] net/ionic: fix device close sequence to avoid crash Andrew Boyer
2024-02-07 2:18 ` [PATCH v2 11/13] net/ionic: optimize device close operation Andrew Boyer
2024-02-07 2:18 ` [PATCH v2 12/13] net/ionic: optimize device stop operation Andrew Boyer
2024-02-07 2:18 ` [PATCH v2 13/13] net/ionic: optimize device start operation Andrew Boyer
2024-02-07 3:13 ` [PATCH v3 00/13] net/ionic: miscellaneous fixes and improvements Andrew Boyer
2024-02-07 3:13 ` [PATCH v3 01/13] net/ionic: add stat for completion queue entries processed Andrew Boyer
2024-02-07 3:13 ` [PATCH v3 02/13] net/ionic: increase max supported MTU to 9750 bytes Andrew Boyer
2024-02-07 3:13 ` [PATCH v3 03/13] net/ionic: don't auto-enable Rx scatter-gather a second time Andrew Boyer
2024-02-07 3:13 ` [PATCH v3 04/13] net/ionic: fix missing volatile type for cqe pointers Andrew Boyer
2024-02-07 3:13 ` [PATCH v3 05/13] net/ionic: replace non-standard type in structure definition Andrew Boyer
2024-02-07 3:13 ` [PATCH v3 06/13] net/ionic: memcpy descriptors when using Q-in-CMB Andrew Boyer
2024-02-07 3:13 ` [PATCH v3 07/13] net/ionic: fix RSS query routine Andrew Boyer
2024-02-07 3:13 ` [PATCH v3 08/13] net/ionic: report 1G and 200G link speeds when applicable Andrew Boyer
2024-02-07 3:13 ` [PATCH v3 09/13] net/ionic: add flexible firmware xstat counters Andrew Boyer
2024-02-07 3:13 ` [PATCH v3 10/13] net/ionic: fix device close sequence to avoid crash Andrew Boyer
2024-02-07 3:13 ` [PATCH v3 11/13] net/ionic: optimize device close operation Andrew Boyer
2024-02-07 3:13 ` [PATCH v3 12/13] net/ionic: optimize device stop operation Andrew Boyer
2024-02-07 3:13 ` [PATCH v3 13/13] net/ionic: optimize device start operation Andrew Boyer
2024-02-07 15:45 ` [PATCH v3 00/13] net/ionic: miscellaneous fixes and improvements Ferruh Yigit
2024-02-08 15:32 ` Ferruh Yigit
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).