From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 0EA29A0546; Tue, 6 Apr 2021 17:12:26 +0200 (CEST) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id B29691411EA; Tue, 6 Apr 2021 17:11:54 +0200 (CEST) Received: from mx0b-0016f401.pphosted.com (mx0b-0016f401.pphosted.com [67.231.156.173]) by mails.dpdk.org (Postfix) with ESMTP id 4424014120F for ; Tue, 6 Apr 2021 17:11:53 +0200 (CEST) Received: from pps.filterd (m0045851.ppops.net [127.0.0.1]) by mx0b-0016f401.pphosted.com (8.16.0.43/8.16.0.43) with SMTP id 136F0UQp015128 for ; Tue, 6 Apr 2021 08:11:52 -0700 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=marvell.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : mime-version : content-transfer-encoding : content-type; s=pfpt0220; bh=gjRglxpZZtZ3Wnhw0WdVxE7hcJr6gVU2JmmEwXsrAuE=; b=BqkpsgC7XfsKSv+2Vj90coe3cxlMPack1ezI6FTGdqhUYaVpXxQNFiDG9P05sZgARRoB xK/4MLoNgRlZ6TyWxmwouFrkZnhw38ea8vqa7sPhXnYhg4FyafcDJ/7yJqhC4gC5a7xj MigUFiDfxmYAsgYxx+yIzV0eN9sMM/4Juojk1vty0PhOTAsQAtxiRKuaE+dT5Qlwu1Bo cB49FxcgjPFxFr6zhI6U6YLt//7YJjvNoHVdtzhiYNmZu2IH44N5AmtcwlYXvcPs2iHq gxS7CdQFGriMgC9t+rLN8dGC2UGWT8RRq7YruYwKguhJ5ACa/i2MdgCZGp+9Rx8aKguC vQ== Received: from dc5-exch01.marvell.com ([199.233.59.181]) by mx0b-0016f401.pphosted.com with ESMTP id 37redma2qw-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-SHA384 bits=256 verify=NOT) for ; Tue, 06 Apr 2021 08:11:52 -0700 Received: from DC5-EXCH01.marvell.com (10.69.176.38) by DC5-EXCH01.marvell.com (10.69.176.38) with Microsoft SMTP Server (TLS) id 15.0.1497.2; Tue, 6 Apr 2021 08:11:50 -0700 Received: from maili.marvell.com (10.69.176.80) by DC5-EXCH01.marvell.com (10.69.176.38) with Microsoft SMTP Server id 15.0.1497.2 via Frontend Transport; Tue, 6 Apr 2021 08:11:50 -0700 Received: from lab-ci-142.marvell.com (unknown [10.28.36.142]) by maili.marvell.com (Postfix) with ESMTP id 232A23F7040; Tue, 6 Apr 2021 08:11:47 -0700 (PDT) From: Ashwin Sekhar T K To: CC: , , , , , , Date: Tue, 6 Apr 2021 20:41:12 +0530 Message-ID: <20210406151115.1889455-9-asekhar@marvell.com> X-Mailer: git-send-email 2.31.0 In-Reply-To: <20210406151115.1889455-1-asekhar@marvell.com> References: <20210305162149.2196166-1-asekhar@marvell.com> <20210406151115.1889455-1-asekhar@marvell.com> MIME-Version: 1.0 Content-Transfer-Encoding: 8bit Content-Type: text/plain X-Proofpoint-GUID: wTS3vH7Fxh6eCPS04jGwNa05AJ_yeztJ X-Proofpoint-ORIG-GUID: wTS3vH7Fxh6eCPS04jGwNa05AJ_yeztJ X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10434:6.0.369, 18.0.761 definitions=2021-04-06_04:2021-04-01, 2021-04-06 signatures=0 Subject: [dpdk-dev] [PATCH v3 08/11] mempool/cnxk: add batch op init X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" Marvell CN10k mempool supports batch enqueue/dequeue which can dequeue up to 512 pointers and enqueue up to 15 pointers using a single instruction. These batch operations require a DMA memory to enqueue/dequeue pointers. This patch adds the initialization of this DMA memory. Signed-off-by: Ashwin Sekhar T K --- doc/guides/mempool/cnxk.rst | 5 + drivers/mempool/cnxk/cn10k_mempool_ops.c | 147 ++++++++++++++++++++++- drivers/mempool/cnxk/cnxk_mempool.h | 2 + drivers/mempool/cnxk/cnxk_mempool_ops.c | 11 +- 4 files changed, 160 insertions(+), 5 deletions(-) diff --git a/doc/guides/mempool/cnxk.rst b/doc/guides/mempool/cnxk.rst index 783368e690..286ee29003 100644 --- a/doc/guides/mempool/cnxk.rst +++ b/doc/guides/mempool/cnxk.rst @@ -25,6 +25,11 @@ CN9k NPA supports: - Burst alloc of up to 32 pointers. +CN10k NPA supports: + +- Batch dequeue of up to 512 pointers with single instruction. +- Batch enqueue of up to 15 pointers with single instruction. + Prerequisites and Compilation procedure --------------------------------------- diff --git a/drivers/mempool/cnxk/cn10k_mempool_ops.c b/drivers/mempool/cnxk/cn10k_mempool_ops.c index 9b63789006..a3aef0ddb2 100644 --- a/drivers/mempool/cnxk/cn10k_mempool_ops.c +++ b/drivers/mempool/cnxk/cn10k_mempool_ops.c @@ -7,11 +7,136 @@ #include "roc_api.h" #include "cnxk_mempool.h" +#define BATCH_ALLOC_SZ ROC_CN10K_NPA_BATCH_ALLOC_MAX_PTRS +#define BATCH_OP_DATA_TABLE_MZ_NAME "batch_op_data_table_mz" + +enum batch_op_status { + BATCH_ALLOC_OP_NOT_ISSUED = 0, + BATCH_ALLOC_OP_ISSUED = 1, + BATCH_ALLOC_OP_DONE +}; + +struct batch_op_mem { + unsigned int sz; + enum batch_op_status status; + uint64_t objs[BATCH_ALLOC_SZ] __rte_aligned(ROC_ALIGN); +}; + +struct batch_op_data { + uint64_t lmt_addr; + struct batch_op_mem mem[RTE_MAX_LCORE] __rte_aligned(ROC_ALIGN); +}; + +static struct batch_op_data **batch_op_data_tbl; + +static int +batch_op_data_table_create(void) +{ + const struct rte_memzone *mz; + + /* If table is already set, nothing to do */ + if (batch_op_data_tbl) + return 0; + + mz = rte_memzone_lookup(BATCH_OP_DATA_TABLE_MZ_NAME); + if (mz == NULL) { + if (rte_eal_process_type() == RTE_PROC_PRIMARY) { + unsigned int maxpools, sz; + + maxpools = roc_idev_npa_maxpools_get(); + sz = maxpools * sizeof(struct batch_op_data *); + + mz = rte_memzone_reserve_aligned( + BATCH_OP_DATA_TABLE_MZ_NAME, sz, SOCKET_ID_ANY, + 0, ROC_ALIGN); + } + if (mz == NULL) { + plt_err("Failed to reserve batch op data table"); + return -ENOMEM; + } + } + batch_op_data_tbl = mz->addr; + rte_wmb(); + return 0; +} + +static inline struct batch_op_data * +batch_op_data_get(uint64_t pool_id) +{ + uint64_t aura = roc_npa_aura_handle_to_aura(pool_id); + + return batch_op_data_tbl[aura]; +} + +static inline void +batch_op_data_set(uint64_t pool_id, struct batch_op_data *op_data) +{ + uint64_t aura = roc_npa_aura_handle_to_aura(pool_id); + + batch_op_data_tbl[aura] = op_data; +} + +static int +batch_op_init(struct rte_mempool *mp) +{ + struct batch_op_data *op_data; + int i; + + op_data = batch_op_data_get(mp->pool_id); + /* The data should not have been allocated previously */ + RTE_ASSERT(op_data == NULL); + + op_data = rte_zmalloc(NULL, sizeof(struct batch_op_data), ROC_ALIGN); + if (op_data == NULL) + return -ENOMEM; + + for (i = 0; i < RTE_MAX_LCORE; i++) { + op_data->mem[i].sz = 0; + op_data->mem[i].status = BATCH_ALLOC_OP_NOT_ISSUED; + } + + op_data->lmt_addr = roc_idev_lmt_base_addr_get(); + batch_op_data_set(mp->pool_id, op_data); + rte_wmb(); + + return 0; +} + +static void +batch_op_fini(struct rte_mempool *mp) +{ + struct batch_op_data *op_data; + int i; + + op_data = batch_op_data_get(mp->pool_id); + + rte_wmb(); + for (i = 0; i < RTE_MAX_LCORE; i++) { + struct batch_op_mem *mem = &op_data->mem[i]; + + if (mem->status == BATCH_ALLOC_OP_ISSUED) { + mem->sz = roc_npa_aura_batch_alloc_extract( + mem->objs, mem->objs, BATCH_ALLOC_SZ); + mem->status = BATCH_ALLOC_OP_DONE; + } + if (mem->status == BATCH_ALLOC_OP_DONE) { + roc_npa_aura_op_bulk_free(mp->pool_id, mem->objs, + mem->sz, 1); + mem->status = BATCH_ALLOC_OP_NOT_ISSUED; + } + } + + rte_free(op_data); + batch_op_data_set(mp->pool_id, NULL); + rte_wmb(); +} + static int cn10k_mempool_alloc(struct rte_mempool *mp) { uint32_t block_size; size_t padding; + int rc; block_size = mp->elt_size + mp->header_size + mp->trailer_size; /* Align header size to ROC_ALIGN */ @@ -29,15 +154,35 @@ cn10k_mempool_alloc(struct rte_mempool *mp) block_size += padding; } - return cnxk_mempool_alloc(mp); + rc = cnxk_mempool_alloc(mp); + if (rc) + return rc; + + rc = batch_op_init(mp); + if (rc) { + plt_err("Failed to init batch alloc mem rc=%d", rc); + goto error; + } + + return 0; +error: + cnxk_mempool_free(mp); + return rc; } static void cn10k_mempool_free(struct rte_mempool *mp) { + batch_op_fini(mp); cnxk_mempool_free(mp); } +int +cn10k_mempool_plt_init(void) +{ + return batch_op_data_table_create(); +} + static struct rte_mempool_ops cn10k_mempool_ops = { .name = "cn10k_mempool_ops", .alloc = cn10k_mempool_alloc, diff --git a/drivers/mempool/cnxk/cnxk_mempool.h b/drivers/mempool/cnxk/cnxk_mempool.h index 099b7f6998..3405aa7663 100644 --- a/drivers/mempool/cnxk/cnxk_mempool.h +++ b/drivers/mempool/cnxk/cnxk_mempool.h @@ -23,4 +23,6 @@ int __rte_hot cnxk_mempool_enq(struct rte_mempool *mp, void *const *obj_table, int __rte_hot cnxk_mempool_deq(struct rte_mempool *mp, void **obj_table, unsigned int n); +int cn10k_mempool_plt_init(void); + #endif diff --git a/drivers/mempool/cnxk/cnxk_mempool_ops.c b/drivers/mempool/cnxk/cnxk_mempool_ops.c index 42c02bf14e..c7b75f026d 100644 --- a/drivers/mempool/cnxk/cnxk_mempool_ops.c +++ b/drivers/mempool/cnxk/cnxk_mempool_ops.c @@ -174,12 +174,15 @@ cnxk_mempool_populate(struct rte_mempool *mp, unsigned int max_objs, static int cnxk_mempool_plt_init(void) { - if (roc_model_is_cn9k()) + int rc = 0; + + if (roc_model_is_cn9k()) { rte_mbuf_set_platform_mempool_ops("cn9k_mempool_ops"); - else if (roc_model_is_cn10k()) + } else if (roc_model_is_cn10k()) { rte_mbuf_set_platform_mempool_ops("cn10k_mempool_ops"); - - return 0; + rc = cn10k_mempool_plt_init(); + } + return rc; } RTE_INIT(cnxk_mempool_ops_init) -- 2.31.0