From mboxrd@z Thu Jan 1 00:00:00 1970
Return-Path: <dev-bounces@dpdk.org>
Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124])
by inbox.dpdk.org (Postfix) with ESMTP id 24543A0548;
Sat, 6 Mar 2021 17:31:15 +0100 (CET)
Received: from [217.70.189.124] (localhost [127.0.0.1])
by mails.dpdk.org (Postfix) with ESMTP id CFEE122A430;
Sat, 6 Mar 2021 17:30:39 +0100 (CET)
Received: from mx0b-0016f401.pphosted.com (mx0a-0016f401.pphosted.com
[67.231.148.174])
by mails.dpdk.org (Postfix) with ESMTP id 164D522A41D
for <dev@dpdk.org>; Sat, 6 Mar 2021 17:30:37 +0100 (CET)
Received: from pps.filterd (m0045849.ppops.net [127.0.0.1])
by mx0a-0016f401.pphosted.com (8.16.0.43/8.16.0.43) with SMTP id
126GPXiB025580 for <dev@dpdk.org>; Sat, 6 Mar 2021 08:30:37 -0800
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=marvell.com;
h=from : to : cc :
subject : date : message-id : in-reply-to : references : mime-version :
content-transfer-encoding : content-type; s=pfpt0220;
bh=3BuPcYuTR0xyFq/4GbtWETwZ9Adr1E4rEKdWz7oBpcU=;
b=O6X5yzhMj5e0wJugvs8HSiCxD9qV43S4+/Utd2PZCPBmMiVgkwknrigYB2gtKuCSxfKm
KDBhZsalC2oGowNoH/2D3yfPI0mh0XH6h7CLEXrOwWY0fkFN67blVI1cmzrm9RG0JOpW
oxdGnu8I6HXOg0ZNQB9luMep6OQze42BWXbRhN17DlUyVM6iJfS1SQH4BB3jxyIOQWCU
SvSmBLELceF/G6o/SM/KfGbFTtX8RU4uKf4hDGBCBou4ksgc/hi/aSNUPoA8EBeZ/gHX
ZvRwALsP5COxzyrR3YCEZGHTJtgSv3HsY/iobJgC1Xz/Ba/ZaSGs/gI4EopM6OYe2Nq7 jg==
Received: from dc5-exch01.marvell.com ([199.233.59.181])
by mx0a-0016f401.pphosted.com with ESMTP id 3747yurevs-1
(version=TLSv1.2 cipher=ECDHE-RSA-AES256-SHA384 bits=256 verify=NOT)
for <dev@dpdk.org>; Sat, 06 Mar 2021 08:30:37 -0800
Received: from SC-EXCH03.marvell.com (10.93.176.83) by DC5-EXCH01.marvell.com
(10.69.176.38) with Microsoft SMTP Server (TLS) id 15.0.1497.2;
Sat, 6 Mar 2021 08:30:35 -0800
Received: from DC5-EXCH01.marvell.com (10.69.176.38) by SC-EXCH03.marvell.com
(10.93.176.83) with Microsoft SMTP Server (TLS) id 15.0.1497.2;
Sat, 6 Mar 2021 08:30:35 -0800
Received: from maili.marvell.com (10.69.176.80) by DC5-EXCH01.marvell.com
(10.69.176.38) with Microsoft SMTP Server id 15.0.1497.2 via Frontend
Transport; Sat, 6 Mar 2021 08:30:35 -0800
Received: from BG-LT7430.marvell.com (unknown [10.193.68.121])
by maili.marvell.com (Postfix) with ESMTP id 920AB3F703F;
Sat, 6 Mar 2021 08:30:33 -0800 (PST)
From: <pbhagavatula@marvell.com>
To: <jerinj@marvell.com>, Pavan Nikhilesh <pbhagavatula@marvell.com>, "Shijith
Thotton" <sthotton@marvell.com>
CC: <ndabilpuram@marvell.com>, <dev@dpdk.org>
Date: Sat, 6 Mar 2021 21:59:12 +0530
Message-ID: <20210306162942.6845-8-pbhagavatula@marvell.com>
X-Mailer: git-send-email 2.17.1
In-Reply-To: <20210306162942.6845-1-pbhagavatula@marvell.com>
References: <20210306162942.6845-1-pbhagavatula@marvell.com>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit
Content-Type: text/plain
X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10434:6.0.369, 18.0.761
definitions=2021-03-06_08:2021-03-03,
2021-03-06 signatures=0
Subject: [dpdk-dev] [PATCH 07/36] event/cnxk: allocate event inflight buffers
X-BeenThere: dev@dpdk.org
X-Mailman-Version: 2.1.29
Precedence: list
List-Id: DPDK patches and discussions <dev.dpdk.org>
List-Unsubscribe: <https://mails.dpdk.org/options/dev>,
<mailto:dev-request@dpdk.org?subject=unsubscribe>
List-Archive: <http://mails.dpdk.org/archives/dev/>
List-Post: <mailto:dev@dpdk.org>
List-Help: <mailto:dev-request@dpdk.org?subject=help>
List-Subscribe: <https://mails.dpdk.org/listinfo/dev>,
<mailto:dev-request@dpdk.org?subject=subscribe>
Errors-To: dev-bounces@dpdk.org
Sender: "dev" <dev-bounces@dpdk.org>
From: Pavan Nikhilesh <pbhagavatula@marvell.com>
Allocate buffers in DRAM that hold inflight events.
Signed-off-by: Shijith Thotton <sthotton@marvell.com>
Signed-off-by: Pavan Nikhilesh <pbhagavatula@marvell.com>
---
drivers/event/cnxk/cn10k_eventdev.c | 7 ++
drivers/event/cnxk/cn9k_eventdev.c | 7 ++
drivers/event/cnxk/cnxk_eventdev.c | 105 ++++++++++++++++++++++++++++
drivers/event/cnxk/cnxk_eventdev.h | 14 +++-
4 files changed, 132 insertions(+), 1 deletion(-)
diff --git a/drivers/event/cnxk/cn10k_eventdev.c b/drivers/event/cnxk/cn10k_eventdev.c
index 92687c23e..7e3fa20c5 100644
--- a/drivers/event/cnxk/cn10k_eventdev.c
+++ b/drivers/event/cnxk/cn10k_eventdev.c
@@ -55,6 +55,13 @@ cn10k_sso_dev_configure(const struct rte_eventdev *event_dev)
return -ENODEV;
}
+ rc = cnxk_sso_xaq_allocate(dev);
+ if (rc < 0)
+ goto cnxk_rsrc_fini;
+
+ return 0;
+cnxk_rsrc_fini:
+ roc_sso_rsrc_fini(&dev->sso);
return rc;
}
diff --git a/drivers/event/cnxk/cn9k_eventdev.c b/drivers/event/cnxk/cn9k_eventdev.c
index 1bd2b3343..71245b660 100644
--- a/drivers/event/cnxk/cn9k_eventdev.c
+++ b/drivers/event/cnxk/cn9k_eventdev.c
@@ -63,6 +63,13 @@ cn9k_sso_dev_configure(const struct rte_eventdev *event_dev)
return -ENODEV;
}
+ rc = cnxk_sso_xaq_allocate(dev);
+ if (rc < 0)
+ goto cnxk_rsrc_fini;
+
+ return 0;
+cnxk_rsrc_fini:
+ roc_sso_rsrc_fini(&dev->sso);
return rc;
}
diff --git a/drivers/event/cnxk/cnxk_eventdev.c b/drivers/event/cnxk/cnxk_eventdev.c
index 59cc570fe..927f99117 100644
--- a/drivers/event/cnxk/cnxk_eventdev.c
+++ b/drivers/event/cnxk/cnxk_eventdev.c
@@ -28,12 +28,107 @@ cnxk_sso_info_get(struct cnxk_sso_evdev *dev,
RTE_EVENT_DEV_CAP_CARRY_FLOW_ID;
}
+int
+cnxk_sso_xaq_allocate(struct cnxk_sso_evdev *dev)
+{
+ char pool_name[RTE_MEMZONE_NAMESIZE];
+ uint32_t xaq_cnt, npa_aura_id;
+ const struct rte_memzone *mz;
+ struct npa_aura_s *aura;
+ static int reconfig_cnt;
+ int rc;
+
+ if (dev->xaq_pool) {
+ rc = roc_sso_hwgrp_release_xaq(&dev->sso, dev->nb_event_queues);
+ if (rc < 0) {
+ plt_err("Failed to release XAQ %d", rc);
+ return rc;
+ }
+ rte_mempool_free(dev->xaq_pool);
+ dev->xaq_pool = NULL;
+ }
+
+ /*
+ * Allocate memory for Add work backpressure.
+ */
+ mz = rte_memzone_lookup(CNXK_SSO_FC_NAME);
+ if (mz == NULL)
+ mz = rte_memzone_reserve_aligned(CNXK_SSO_FC_NAME,
+ sizeof(struct npa_aura_s) +
+ RTE_CACHE_LINE_SIZE,
+ 0, 0, RTE_CACHE_LINE_SIZE);
+ if (mz == NULL) {
+ plt_err("Failed to allocate mem for fcmem");
+ return -ENOMEM;
+ }
+
+ dev->fc_iova = mz->iova;
+ dev->fc_mem = mz->addr;
+
+ aura = (struct npa_aura_s *)((uintptr_t)dev->fc_mem +
+ RTE_CACHE_LINE_SIZE);
+ memset(aura, 0, sizeof(struct npa_aura_s));
+
+ aura->fc_ena = 1;
+ aura->fc_addr = dev->fc_iova;
+ aura->fc_hyst_bits = 0; /* Store count on all updates */
+
+ /* Taken from HRM 14.3.3(4) */
+ xaq_cnt = dev->nb_event_queues * CNXK_SSO_XAQ_CACHE_CNT;
+ xaq_cnt += (dev->sso.iue / dev->sso.xae_waes) +
+ (CNXK_SSO_XAQ_SLACK * dev->nb_event_queues);
+
+ plt_sso_dbg("Configuring %d xaq buffers", xaq_cnt);
+ /* Setup XAQ based on number of nb queues. */
+ snprintf(pool_name, 30, "cnxk_xaq_buf_pool_%d", reconfig_cnt);
+ dev->xaq_pool = (void *)rte_mempool_create_empty(
+ pool_name, xaq_cnt, dev->sso.xaq_buf_size, 0, 0,
+ rte_socket_id(), 0);
+
+ if (dev->xaq_pool == NULL) {
+ plt_err("Unable to create empty mempool.");
+ rte_memzone_free(mz);
+ return -ENOMEM;
+ }
+
+ rc = rte_mempool_set_ops_byname(dev->xaq_pool,
+ rte_mbuf_platform_mempool_ops(), aura);
+ if (rc != 0) {
+ plt_err("Unable to set xaqpool ops.");
+ goto alloc_fail;
+ }
+
+ rc = rte_mempool_populate_default(dev->xaq_pool);
+ if (rc < 0) {
+ plt_err("Unable to set populate xaqpool.");
+ goto alloc_fail;
+ }
+ reconfig_cnt++;
+ /* When SW does addwork (enqueue) check if there is space in XAQ by
+ * comparing fc_addr above against the xaq_lmt calculated below.
+ * There should be a minimum headroom (CNXK_SSO_XAQ_SLACK / 2) for SSO
+ * to request XAQ to cache them even before enqueue is called.
+ */
+ dev->xaq_lmt =
+ xaq_cnt - (CNXK_SSO_XAQ_SLACK / 2 * dev->nb_event_queues);
+ dev->nb_xaq_cfg = xaq_cnt;
+
+ npa_aura_id = roc_npa_aura_handle_to_aura(dev->xaq_pool->pool_id);
+ return roc_sso_hwgrp_alloc_xaq(&dev->sso, npa_aura_id,
+ dev->nb_event_queues);
+alloc_fail:
+ rte_mempool_free(dev->xaq_pool);
+ rte_memzone_free(mz);
+ return rc;
+}
+
int
cnxk_sso_dev_validate(const struct rte_eventdev *event_dev)
{
struct rte_event_dev_config *conf = &event_dev->data->dev_conf;
struct cnxk_sso_evdev *dev = cnxk_sso_pmd_priv(event_dev);
uint32_t deq_tmo_ns;
+ int rc;
deq_tmo_ns = conf->dequeue_timeout_ns;
@@ -67,6 +162,16 @@ cnxk_sso_dev_validate(const struct rte_eventdev *event_dev)
return -EINVAL;
}
+ if (dev->xaq_pool) {
+ rc = roc_sso_hwgrp_release_xaq(&dev->sso, dev->nb_event_queues);
+ if (rc < 0) {
+ plt_err("Failed to release XAQ %d", rc);
+ return rc;
+ }
+ rte_mempool_free(dev->xaq_pool);
+ dev->xaq_pool = NULL;
+ }
+
dev->nb_event_queues = conf->nb_event_queues;
dev->nb_event_ports = conf->nb_event_ports;
diff --git a/drivers/event/cnxk/cnxk_eventdev.h b/drivers/event/cnxk/cnxk_eventdev.h
index 974c618bc..8478120c0 100644
--- a/drivers/event/cnxk/cnxk_eventdev.h
+++ b/drivers/event/cnxk/cnxk_eventdev.h
@@ -5,6 +5,7 @@
#ifndef __CNXK_EVENTDEV_H__
#define __CNXK_EVENTDEV_H__
+#include <rte_mbuf_pool_ops.h>
#include <rte_pci.h>
#include <eventdev_pmd_pci.h>
@@ -13,7 +14,10 @@
#define USEC2NSEC(__us) ((__us)*1E3)
-#define CNXK_SSO_MZ_NAME "cnxk_evdev_mz"
+#define CNXK_SSO_FC_NAME "cnxk_evdev_xaq_fc"
+#define CNXK_SSO_MZ_NAME "cnxk_evdev_mz"
+#define CNXK_SSO_XAQ_CACHE_CNT (0x7)
+#define CNXK_SSO_XAQ_SLACK (8)
struct cnxk_sso_evdev {
struct roc_sso sso;
@@ -26,6 +30,11 @@ struct cnxk_sso_evdev {
uint32_t min_dequeue_timeout_ns;
uint32_t max_dequeue_timeout_ns;
int32_t max_num_events;
+ uint64_t *fc_mem;
+ uint64_t xaq_lmt;
+ uint64_t nb_xaq_cfg;
+ rte_iova_t fc_iova;
+ struct rte_mempool *xaq_pool;
/* CN9K */
uint8_t dual_ws;
} __rte_cache_aligned;
@@ -36,6 +45,9 @@ cnxk_sso_pmd_priv(const struct rte_eventdev *event_dev)
return event_dev->data->dev_private;
}
+/* Configuration functions */
+int cnxk_sso_xaq_allocate(struct cnxk_sso_evdev *dev);
+
/* Common ops API. */
int cnxk_sso_init(struct rte_eventdev *event_dev);
int cnxk_sso_fini(struct rte_eventdev *event_dev);
--
2.17.1