[PATCH 0/3] Wait for NPA pools to get filled

[PATCH 0/3] Wait for NPA pools to get filled

[Ashwin Sekhar T K]

NPA could take some time to reflect the pointers which
has been freed into pools. So, after populating a pool
with pointers, wait until the populated pointers are
reflected in the pool.

Ashwin Sekhar T K (3):
  common/cnxk: add support to wait for pool filling
  common/cnxk: wait for sqb pool to fill
  common/cnxk: wait for xaq pool to fill

 drivers/common/cnxk/roc_nix_queue.c | 10 ++++++++++
 drivers/common/cnxk/roc_npa.h       | 26 ++++++++++++++++++++++++++
 drivers/common/cnxk/roc_sso.c       |  9 +++++++++
 3 files changed, 45 insertions(+)

-- 
2.32.0

[PATCH 1/3] common/cnxk: add support to wait for pool filling

[Ashwin Sekhar T K]

Add roc_npa_aura_op_available_wait() API which can be used to wait
until an NPA pool gets filled up to a certain count of pointers.

Signed-off-by: Ashwin Sekhar T K <asekhar@marvell.com>
---
 drivers/common/cnxk/roc_npa.h | 26 ++++++++++++++++++++++++++
 1 file changed, 26 insertions(+)

diff --git a/drivers/common/cnxk/roc_npa.h b/drivers/common/cnxk/roc_npa.h
index 46350fdb48..d05c5c4af4 100644
--- a/drivers/common/cnxk/roc_npa.h
+++ b/drivers/common/cnxk/roc_npa.h
@@ -155,6 +155,32 @@ roc_npa_aura_op_available(uint64_t aura_handle)
 		return reg & 0xFFFFFFFFF;
 }
 
+/* Wait for a given timeout, repeatedly checking whether the available
+ * pointers has reached the given count. Returns the available pointer
+ * count if it has reached the given count or if timeout has expired
+ */
+static inline uint32_t
+roc_npa_aura_op_available_wait(uint64_t aura_handle, uint32_t count,
+			       uint32_t tmo_ms)
+{
+#define OP_AVAIL_WAIT_MS_DEFAULT   (100)
+#define OP_AVAIL_CHECK_INTERVAL_MS (1)
+	uint32_t op_avail;
+	int retry;
+
+	tmo_ms = tmo_ms ? tmo_ms : OP_AVAIL_WAIT_MS_DEFAULT;
+
+	retry = tmo_ms / OP_AVAIL_CHECK_INTERVAL_MS;
+	op_avail = roc_npa_aura_op_available(aura_handle);
+	while (retry && (op_avail < count)) {
+		plt_delay_ms(OP_AVAIL_CHECK_INTERVAL_MS);
+		op_avail = roc_npa_aura_op_available(aura_handle);
+		retry--;
+	}
+
+	return op_avail;
+}
+
 static inline uint64_t
 roc_npa_pool_op_performance_counter(uint64_t aura_handle, const int drop)
 {
-- 
2.32.0

[PATCH 2/3] common/cnxk: wait for sqb pool to fill

[Ashwin Sekhar T K]

Wait for SQB pool to get filled with the freed pointers
before proceeding.

Signed-off-by: Ashwin Sekhar T K <asekhar@marvell.com>
---
 drivers/common/cnxk/roc_nix_queue.c | 10 ++++++++++
 1 file changed, 10 insertions(+)

diff --git a/drivers/common/cnxk/roc_nix_queue.c b/drivers/common/cnxk/roc_nix_queue.c
index c8c8401d81..c638cd43e4 100644
--- a/drivers/common/cnxk/roc_nix_queue.c
+++ b/drivers/common/cnxk/roc_nix_queue.c
@@ -638,11 +638,21 @@ sqb_pool_populate(struct roc_nix *roc_nix, struct roc_nix_sq *sq)
 		roc_npa_aura_op_free(sq->aura_handle, 0, iova);
 		iova += blk_sz;
 	}
+
+	if (roc_npa_aura_op_available_wait(sq->aura_handle, NIX_MAX_SQB, 0) !=
+	    NIX_MAX_SQB) {
+		plt_err("Failed to free all pointers to the pool");
+		rc = NIX_ERR_NO_MEM;
+		goto npa_fail;
+	}
+
 	roc_npa_aura_op_range_set(sq->aura_handle, (uint64_t)sq->sqe_mem, iova);
 	roc_npa_aura_limit_modify(sq->aura_handle, sq->nb_sqb_bufs);
 	sq->aura_sqb_bufs = NIX_MAX_SQB;
 
 	return rc;
+npa_fail:
+	plt_free(sq->sqe_mem);
 nomem:
 	roc_npa_pool_destroy(sq->aura_handle);
 fail:
-- 
2.32.0

[PATCH 3/3] common/cnxk: wait for xaq pool to fill

[Ashwin Sekhar T K]

Wait for XAQ pool to get filled with the freed pointers
before proceeding.

Signed-off-by: Ashwin Sekhar T K <asekhar@marvell.com>
---
 drivers/common/cnxk/roc_sso.c | 9 +++++++++
 1 file changed, 9 insertions(+)

diff --git a/drivers/common/cnxk/roc_sso.c b/drivers/common/cnxk/roc_sso.c
index 45ff16ca0e..c1aa3324be 100644
--- a/drivers/common/cnxk/roc_sso.c
+++ b/drivers/common/cnxk/roc_sso.c
@@ -453,6 +453,13 @@ sso_hwgrp_init_xaq_aura(struct dev *dev, struct roc_sso_xaq_data *xaq,
 	}
 	roc_npa_aura_op_range_set(xaq->aura_handle, (uint64_t)xaq->mem, iova);
 
+	if (roc_npa_aura_op_available_wait(xaq->aura_handle, xaq->nb_xaq, 0) !=
+	    xaq->nb_xaq) {
+		plt_err("Failed to free all pointers to the pool");
+		rc = -ENOMEM;
+		goto npa_fill_fail;
+	}
+
 	/* When SW does addwork (enqueue) check if there is space in XAQ by
 	 * comparing fc_addr above against the xaq_lmt calculated below.
 	 * There should be a minimum headroom of 7 XAQs per HWGRP for SSO
@@ -461,6 +468,8 @@ sso_hwgrp_init_xaq_aura(struct dev *dev, struct roc_sso_xaq_data *xaq,
 	xaq->xaq_lmt = xaq->nb_xaq - (nb_hwgrp * SSO_XAQ_CACHE_CNT);
 
 	return 0;
+npa_fill_fail:
+	roc_npa_pool_destroy(xaq->aura_handle);
 npa_fail:
 	plt_free(xaq->mem);
 free_fc:
-- 
2.32.0

Re: [PATCH 0/3] Wait for NPA pools to get filled

[Jerin Jacob]

On Tue, Nov 30, 2021 at 11:39 AM Ashwin Sekhar T K <asekhar@marvell.com> wrote:
>
> NPA could take some time to reflect the pointers which
> has been freed into pools. So, after populating a pool
> with pointers, wait until the populated pointers are
> reflected in the pool.


Series Acked-by: Jerin Jacob <jerinj@marvell.com>
Series applied to dpdk-next-net-mrvl/for-next-net. Thanks.

>
> Ashwin Sekhar T K (3):
>   common/cnxk: add support to wait for pool filling
>   common/cnxk: wait for sqb pool to fill
>   common/cnxk: wait for xaq pool to fill
>
>  drivers/common/cnxk/roc_nix_queue.c | 10 ++++++++++
>  drivers/common/cnxk/roc_npa.h       | 26 ++++++++++++++++++++++++++
>  drivers/common/cnxk/roc_sso.c       |  9 +++++++++
>  3 files changed, 45 insertions(+)
>
> --
> 2.32.0
>