From mboxrd@z Thu Jan  1 00:00:00 1970
Return-Path: <dev-bounces@dpdk.org>
Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124])
	by inbox.dpdk.org (Postfix) with ESMTP id 24543A0548;
	Sat,  6 Mar 2021 17:31:15 +0100 (CET)
Received: from [217.70.189.124] (localhost [127.0.0.1])
	by mails.dpdk.org (Postfix) with ESMTP id CFEE122A430;
	Sat,  6 Mar 2021 17:30:39 +0100 (CET)
Received: from mx0b-0016f401.pphosted.com (mx0a-0016f401.pphosted.com
 [67.231.148.174])
 by mails.dpdk.org (Postfix) with ESMTP id 164D522A41D
 for <dev@dpdk.org>; Sat,  6 Mar 2021 17:30:37 +0100 (CET)
Received: from pps.filterd (m0045849.ppops.net [127.0.0.1])
 by mx0a-0016f401.pphosted.com (8.16.0.43/8.16.0.43) with SMTP id
 126GPXiB025580 for <dev@dpdk.org>; Sat, 6 Mar 2021 08:30:37 -0800
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=marvell.com;
 h=from : to : cc :
 subject : date : message-id : in-reply-to : references : mime-version :
 content-transfer-encoding : content-type; s=pfpt0220;
 bh=3BuPcYuTR0xyFq/4GbtWETwZ9Adr1E4rEKdWz7oBpcU=;
 b=O6X5yzhMj5e0wJugvs8HSiCxD9qV43S4+/Utd2PZCPBmMiVgkwknrigYB2gtKuCSxfKm
 KDBhZsalC2oGowNoH/2D3yfPI0mh0XH6h7CLEXrOwWY0fkFN67blVI1cmzrm9RG0JOpW
 oxdGnu8I6HXOg0ZNQB9luMep6OQze42BWXbRhN17DlUyVM6iJfS1SQH4BB3jxyIOQWCU
 SvSmBLELceF/G6o/SM/KfGbFTtX8RU4uKf4hDGBCBou4ksgc/hi/aSNUPoA8EBeZ/gHX
 ZvRwALsP5COxzyrR3YCEZGHTJtgSv3HsY/iobJgC1Xz/Ba/ZaSGs/gI4EopM6OYe2Nq7 jg== 
Received: from dc5-exch01.marvell.com ([199.233.59.181])
 by mx0a-0016f401.pphosted.com with ESMTP id 3747yurevs-1
 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-SHA384 bits=256 verify=NOT)
 for <dev@dpdk.org>; Sat, 06 Mar 2021 08:30:37 -0800
Received: from SC-EXCH03.marvell.com (10.93.176.83) by DC5-EXCH01.marvell.com
 (10.69.176.38) with Microsoft SMTP Server (TLS) id 15.0.1497.2;
 Sat, 6 Mar 2021 08:30:35 -0800
Received: from DC5-EXCH01.marvell.com (10.69.176.38) by SC-EXCH03.marvell.com
 (10.93.176.83) with Microsoft SMTP Server (TLS) id 15.0.1497.2;
 Sat, 6 Mar 2021 08:30:35 -0800
Received: from maili.marvell.com (10.69.176.80) by DC5-EXCH01.marvell.com
 (10.69.176.38) with Microsoft SMTP Server id 15.0.1497.2 via Frontend
 Transport; Sat, 6 Mar 2021 08:30:35 -0800
Received: from BG-LT7430.marvell.com (unknown [10.193.68.121])
 by maili.marvell.com (Postfix) with ESMTP id 920AB3F703F;
 Sat,  6 Mar 2021 08:30:33 -0800 (PST)
From: <pbhagavatula@marvell.com>
To: <jerinj@marvell.com>, Pavan Nikhilesh <pbhagavatula@marvell.com>, "Shijith
 Thotton" <sthotton@marvell.com>
CC: <ndabilpuram@marvell.com>, <dev@dpdk.org>
Date: Sat, 6 Mar 2021 21:59:12 +0530
Message-ID: <20210306162942.6845-8-pbhagavatula@marvell.com>
X-Mailer: git-send-email 2.17.1
In-Reply-To: <20210306162942.6845-1-pbhagavatula@marvell.com>
References: <20210306162942.6845-1-pbhagavatula@marvell.com>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit
Content-Type: text/plain
X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10434:6.0.369, 18.0.761
 definitions=2021-03-06_08:2021-03-03,
 2021-03-06 signatures=0
Subject: [dpdk-dev] [PATCH 07/36] event/cnxk: allocate event inflight buffers
X-BeenThere: dev@dpdk.org
X-Mailman-Version: 2.1.29
Precedence: list
List-Id: DPDK patches and discussions <dev.dpdk.org>
List-Unsubscribe: <https://mails.dpdk.org/options/dev>,
 <mailto:dev-request@dpdk.org?subject=unsubscribe>
List-Archive: <http://mails.dpdk.org/archives/dev/>
List-Post: <mailto:dev@dpdk.org>
List-Help: <mailto:dev-request@dpdk.org?subject=help>
List-Subscribe: <https://mails.dpdk.org/listinfo/dev>,
 <mailto:dev-request@dpdk.org?subject=subscribe>
Errors-To: dev-bounces@dpdk.org
Sender: "dev" <dev-bounces@dpdk.org>

From: Pavan Nikhilesh <pbhagavatula@marvell.com>

Allocate buffers in DRAM that hold inflight events.

Signed-off-by: Shijith Thotton <sthotton@marvell.com>
Signed-off-by: Pavan Nikhilesh <pbhagavatula@marvell.com>
---
 drivers/event/cnxk/cn10k_eventdev.c |   7 ++
 drivers/event/cnxk/cn9k_eventdev.c  |   7 ++
 drivers/event/cnxk/cnxk_eventdev.c  | 105 ++++++++++++++++++++++++++++
 drivers/event/cnxk/cnxk_eventdev.h  |  14 +++-
 4 files changed, 132 insertions(+), 1 deletion(-)

diff --git a/drivers/event/cnxk/cn10k_eventdev.c b/drivers/event/cnxk/cn10k_eventdev.c
index 92687c23e..7e3fa20c5 100644
--- a/drivers/event/cnxk/cn10k_eventdev.c
+++ b/drivers/event/cnxk/cn10k_eventdev.c
@@ -55,6 +55,13 @@ cn10k_sso_dev_configure(const struct rte_eventdev *event_dev)
 		return -ENODEV;
 	}
 
+	rc = cnxk_sso_xaq_allocate(dev);
+	if (rc < 0)
+		goto cnxk_rsrc_fini;
+
+	return 0;
+cnxk_rsrc_fini:
+	roc_sso_rsrc_fini(&dev->sso);
 	return rc;
 }
 
diff --git a/drivers/event/cnxk/cn9k_eventdev.c b/drivers/event/cnxk/cn9k_eventdev.c
index 1bd2b3343..71245b660 100644
--- a/drivers/event/cnxk/cn9k_eventdev.c
+++ b/drivers/event/cnxk/cn9k_eventdev.c
@@ -63,6 +63,13 @@ cn9k_sso_dev_configure(const struct rte_eventdev *event_dev)
 		return -ENODEV;
 	}
 
+	rc = cnxk_sso_xaq_allocate(dev);
+	if (rc < 0)
+		goto cnxk_rsrc_fini;
+
+	return 0;
+cnxk_rsrc_fini:
+	roc_sso_rsrc_fini(&dev->sso);
 	return rc;
 }
 
diff --git a/drivers/event/cnxk/cnxk_eventdev.c b/drivers/event/cnxk/cnxk_eventdev.c
index 59cc570fe..927f99117 100644
--- a/drivers/event/cnxk/cnxk_eventdev.c
+++ b/drivers/event/cnxk/cnxk_eventdev.c
@@ -28,12 +28,107 @@ cnxk_sso_info_get(struct cnxk_sso_evdev *dev,
 				  RTE_EVENT_DEV_CAP_CARRY_FLOW_ID;
 }
 
+int
+cnxk_sso_xaq_allocate(struct cnxk_sso_evdev *dev)
+{
+	char pool_name[RTE_MEMZONE_NAMESIZE];
+	uint32_t xaq_cnt, npa_aura_id;
+	const struct rte_memzone *mz;
+	struct npa_aura_s *aura;
+	static int reconfig_cnt;
+	int rc;
+
+	if (dev->xaq_pool) {
+		rc = roc_sso_hwgrp_release_xaq(&dev->sso, dev->nb_event_queues);
+		if (rc < 0) {
+			plt_err("Failed to release XAQ %d", rc);
+			return rc;
+		}
+		rte_mempool_free(dev->xaq_pool);
+		dev->xaq_pool = NULL;
+	}
+
+	/*
+	 * Allocate memory for Add work backpressure.
+	 */
+	mz = rte_memzone_lookup(CNXK_SSO_FC_NAME);
+	if (mz == NULL)
+		mz = rte_memzone_reserve_aligned(CNXK_SSO_FC_NAME,
+						 sizeof(struct npa_aura_s) +
+							 RTE_CACHE_LINE_SIZE,
+						 0, 0, RTE_CACHE_LINE_SIZE);
+	if (mz == NULL) {
+		plt_err("Failed to allocate mem for fcmem");
+		return -ENOMEM;
+	}
+
+	dev->fc_iova = mz->iova;
+	dev->fc_mem = mz->addr;
+
+	aura = (struct npa_aura_s *)((uintptr_t)dev->fc_mem +
+				     RTE_CACHE_LINE_SIZE);
+	memset(aura, 0, sizeof(struct npa_aura_s));
+
+	aura->fc_ena = 1;
+	aura->fc_addr = dev->fc_iova;
+	aura->fc_hyst_bits = 0; /* Store count on all updates */
+
+	/* Taken from HRM 14.3.3(4) */
+	xaq_cnt = dev->nb_event_queues * CNXK_SSO_XAQ_CACHE_CNT;
+	xaq_cnt += (dev->sso.iue / dev->sso.xae_waes) +
+		   (CNXK_SSO_XAQ_SLACK * dev->nb_event_queues);
+
+	plt_sso_dbg("Configuring %d xaq buffers", xaq_cnt);
+	/* Setup XAQ based on number of nb queues. */
+	snprintf(pool_name, 30, "cnxk_xaq_buf_pool_%d", reconfig_cnt);
+	dev->xaq_pool = (void *)rte_mempool_create_empty(
+		pool_name, xaq_cnt, dev->sso.xaq_buf_size, 0, 0,
+		rte_socket_id(), 0);
+
+	if (dev->xaq_pool == NULL) {
+		plt_err("Unable to create empty mempool.");
+		rte_memzone_free(mz);
+		return -ENOMEM;
+	}
+
+	rc = rte_mempool_set_ops_byname(dev->xaq_pool,
+					rte_mbuf_platform_mempool_ops(), aura);
+	if (rc != 0) {
+		plt_err("Unable to set xaqpool ops.");
+		goto alloc_fail;
+	}
+
+	rc = rte_mempool_populate_default(dev->xaq_pool);
+	if (rc < 0) {
+		plt_err("Unable to set populate xaqpool.");
+		goto alloc_fail;
+	}
+	reconfig_cnt++;
+	/* When SW does addwork (enqueue) check if there is space in XAQ by
+	 * comparing fc_addr above against the xaq_lmt calculated below.
+	 * There should be a minimum headroom (CNXK_SSO_XAQ_SLACK / 2) for SSO
+	 * to request XAQ to cache them even before enqueue is called.
+	 */
+	dev->xaq_lmt =
+		xaq_cnt - (CNXK_SSO_XAQ_SLACK / 2 * dev->nb_event_queues);
+	dev->nb_xaq_cfg = xaq_cnt;
+
+	npa_aura_id = roc_npa_aura_handle_to_aura(dev->xaq_pool->pool_id);
+	return roc_sso_hwgrp_alloc_xaq(&dev->sso, npa_aura_id,
+				       dev->nb_event_queues);
+alloc_fail:
+	rte_mempool_free(dev->xaq_pool);
+	rte_memzone_free(mz);
+	return rc;
+}
+
 int
 cnxk_sso_dev_validate(const struct rte_eventdev *event_dev)
 {
 	struct rte_event_dev_config *conf = &event_dev->data->dev_conf;
 	struct cnxk_sso_evdev *dev = cnxk_sso_pmd_priv(event_dev);
 	uint32_t deq_tmo_ns;
+	int rc;
 
 	deq_tmo_ns = conf->dequeue_timeout_ns;
 
@@ -67,6 +162,16 @@ cnxk_sso_dev_validate(const struct rte_eventdev *event_dev)
 		return -EINVAL;
 	}
 
+	if (dev->xaq_pool) {
+		rc = roc_sso_hwgrp_release_xaq(&dev->sso, dev->nb_event_queues);
+		if (rc < 0) {
+			plt_err("Failed to release XAQ %d", rc);
+			return rc;
+		}
+		rte_mempool_free(dev->xaq_pool);
+		dev->xaq_pool = NULL;
+	}
+
 	dev->nb_event_queues = conf->nb_event_queues;
 	dev->nb_event_ports = conf->nb_event_ports;
 
diff --git a/drivers/event/cnxk/cnxk_eventdev.h b/drivers/event/cnxk/cnxk_eventdev.h
index 974c618bc..8478120c0 100644
--- a/drivers/event/cnxk/cnxk_eventdev.h
+++ b/drivers/event/cnxk/cnxk_eventdev.h
@@ -5,6 +5,7 @@
 #ifndef __CNXK_EVENTDEV_H__
 #define __CNXK_EVENTDEV_H__
 
+#include <rte_mbuf_pool_ops.h>
 #include <rte_pci.h>
 
 #include <eventdev_pmd_pci.h>
@@ -13,7 +14,10 @@
 
 #define USEC2NSEC(__us) ((__us)*1E3)
 
-#define CNXK_SSO_MZ_NAME "cnxk_evdev_mz"
+#define CNXK_SSO_FC_NAME       "cnxk_evdev_xaq_fc"
+#define CNXK_SSO_MZ_NAME       "cnxk_evdev_mz"
+#define CNXK_SSO_XAQ_CACHE_CNT (0x7)
+#define CNXK_SSO_XAQ_SLACK     (8)
 
 struct cnxk_sso_evdev {
 	struct roc_sso sso;
@@ -26,6 +30,11 @@ struct cnxk_sso_evdev {
 	uint32_t min_dequeue_timeout_ns;
 	uint32_t max_dequeue_timeout_ns;
 	int32_t max_num_events;
+	uint64_t *fc_mem;
+	uint64_t xaq_lmt;
+	uint64_t nb_xaq_cfg;
+	rte_iova_t fc_iova;
+	struct rte_mempool *xaq_pool;
 	/* CN9K */
 	uint8_t dual_ws;
 } __rte_cache_aligned;
@@ -36,6 +45,9 @@ cnxk_sso_pmd_priv(const struct rte_eventdev *event_dev)
 	return event_dev->data->dev_private;
 }
 
+/* Configuration functions */
+int cnxk_sso_xaq_allocate(struct cnxk_sso_evdev *dev);
+
 /* Common ops API. */
 int cnxk_sso_init(struct rte_eventdev *event_dev);
 int cnxk_sso_fini(struct rte_eventdev *event_dev);
-- 
2.17.1