From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from EUR01-DB5-obe.outbound.protection.outlook.com (mail-db5eur01on0045.outbound.protection.outlook.com [104.47.2.45]) by dpdk.org (Postfix) with ESMTP id C8B6B37AC for ; Thu, 3 May 2018 18:06:53 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=nxp.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version; bh=cEBW/vtPqloUpDNWPw9uw0aJWzZeAHAGi40qzPOwQzU=; b=nWSWioO/uSrcth0G92OVVTDd9XQ4M9vOfOiffaf8y65Y7jRxfSzSjPWhlL+AA2MLVLN3vXjK34pNzv+1rL1ItZMv28dauVjeDhhKSkhmCNBoyJAWJWFbpZq6M+lt6sKUyWmksby74TFQ1D38OPno/+6t4PT2nMKZidFYjIcoUts= Authentication-Results: spf=none (sender IP is ) smtp.mailfrom=nipun.gupta@nxp.com; Received: from b27504-OptiPlex-790.ap.freescale.net (14.142.187.166) by HE1PR0401MB2427.eurprd04.prod.outlook.com (2603:10a6:3:25::24) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384_P256) id 15.20.715.23; Thu, 3 May 2018 16:06:50 +0000 From: Nipun Gupta To: thomas@monjalon.net, hemant.agrawal@nxp.com, shreyansh.jain@nxp.com Cc: dev@dpdk.org, Nipun Gupta Date: Thu, 3 May 2018 21:36:08 +0530 Message-Id: <1525363570-23542-7-git-send-email-nipun.gupta@nxp.com> X-Mailer: git-send-email 1.9.1 In-Reply-To: <1525363570-23542-1-git-send-email-nipun.gupta@nxp.com> References: <1525280972-27736-1-git-send-email-nipun.gupta@nxp.com> <1525363570-23542-1-git-send-email-nipun.gupta@nxp.com> MIME-Version: 1.0 Content-Type: text/plain X-Originating-IP: [14.142.187.166] X-ClientProxiedBy: MA1PR01CA0105.INDPRD01.PROD.OUTLOOK.COM (2603:1096:a00:1::21) To HE1PR0401MB2427.eurprd04.prod.outlook.com (2603:10a6:3:25::24) X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-HT: Tenant X-Microsoft-Antispam: UriScan:; BCL:0; PCL:0; RULEID:(7020095)(4652020)(48565401081)(4534165)(7168020)(4627221)(201703031133081)(201702281549075)(5600026)(2017052603328)(7153060)(7193020); SRVR:HE1PR0401MB2427; X-Microsoft-Exchange-Diagnostics: 1; HE1PR0401MB2427; 3:Bq/emJ8fLCYramUyyenUCQCOZGMusklJ0shp+CNuAIHtbEloKTScCS7v5q/rY8gZcYVSJ1Im97iuWpejV19vDFVPOfMCXJPbRDHK3iZj9AoGGEGWmD7ne17NY5QtVMHQ80UzMRisvVPdw4cwLHWSfXN66LSS2R94lslE2yFtgENOeYARzML8s1hpElmO+8iYSJQzMGbpKrcBtUmUJV8zMTuFGoEqENhYGao2uz5PMjE+eB0UA6hOoynKRrFzu7Gv; 25:2qWm7s4vErGU31yrcy8FcKJWrJ8/y2Bk13MVsf8u1kwiY59BDaa+rcP10KdHmWN+o9AOj6Ola0b8hZFfu8nL7YZMPY1QNjyNJdM2iQW4XiUffR/666mROoUs5nWGRDd01iCxm2Kk3h6SaG/G4V/J5zxaWzdLBgG36E4zuO7FX8Jd+ar/vWFp1pCBazy5oatw+MJ+yQ5OG0IhS00ZfCgXXpisZ4mgFwmbrJrwksW/pKkxKUVrMiHcM/sfrna7y5cGCQ5I21yuzzrqS78xJzslwxBpD28GoPv3jweyJVAS706IKCv1OlgoQ6clsvmQZLaH1aTghrfceAImiVqf9E+VXA==; 31:GqmgrnQFqQICRxF+rgscnI67gsGR8e7WX1Qm2DVl+R9ceREXIwKbn27Fg3oIL4NJcQ9uMWJGsM4ocyJ7pmgQ3WtZE7ebAQzyZc7c/n+9neAb/yM9WmjvzsCUMxGfKwXOjibADh3Q0Oky2gGeAJCcAlIedqFFwL+ySRkbVqb2xvV9kGZRzu59wfUEt8x01eWaGc9LD44UFzm3zidEgGegQXFofT2KXujgBtNAL+E9ofA= X-MS-TrafficTypeDiagnostic: HE1PR0401MB2427: X-Microsoft-Exchange-Diagnostics: 1; HE1PR0401MB2427; 20:F4N0XyFfDaQbkHo6U0WOrizAuDYF+SEq0LTLOVeDHPccwdl/OJip0vb7gLQHepITQIcKNbrp1R5ZB9kH+zsV2FBizbBlPKo3c4JANuwl//+qH4OAMstI9t9zNfAZm5dLNj/QesyKD8oLttO3ymrNCIg1U0N66SNrcFU1VVYWN1oPJF63FN3lXKqRPqcrZS8l+SMPAeXRa5u9lNovAcAqIAv1kvxvoa68mo3EfIF971pnvTRaLAHz0/CwynjOn/efkdFsGASYBxD4hfgfAfgyW1HabgUuCK2f+Wd75Ek4CH41f9D2BHyXQkOks4MmuWrzzFjNrQ6DuiPDv/eMh0/+mrhcIgiWnBDSzVfzecA9pwHSJ1BfgrJFXqxuwqx/IXkokXDg5MEMOTD34lFpwGaBvz6LOPMhRUriXGSDvrmbTZcaQs4Ofv1icyqqBm9SAHLIRV/fVcfD2jMqpstYSgqIVqxE+0lGnTOoGSdTeVVbS4pULhOmiLPafRR5qXpO+mK+; 4:tBqavK/80HlbAGK3zOfy0orGjvMcBm6fGpWRgjCR4cNj80dEecokkqVh/ItM48WfFNV084K50FOdAMZE8/ZdX2FJC8lYFdm/gnCXZwrSzDYr7JwF9TrTZTlKjTaybzbmxT5DKgM+xQmHqovqq2gzQ0CRRqrzgoeQHCfPIxtpi+jDLEvWpqTg6nQ7G9hIVpVjCMQD+RuMUiy4BfShB6kv+ckKtqRVF0actCM+Vr+hEM0/GgowD3mnk/AHrMAl3/fcRsX3JSldMkw4nDFu9PI8fLH3bW1FCXn44qGGLjeVfiBR75G1GrKRgqZysX8EOH7Y X-Microsoft-Antispam-PRVS: X-Exchange-Antispam-Report-Test: UriScan:(185117386973197); X-Exchange-Antispam-Report-CFA-Test: BCL:0; PCL:0; RULEID:(8211001083)(6040522)(2401047)(5005006)(8121501046)(93006095)(93001095)(10201501046)(3002001)(3231254)(944501410)(52105095)(6055026)(6041310)(201703131423095)(201702281528075)(20161123555045)(201703061421075)(201703061406153)(20161123564045)(20161123562045)(20161123558120)(20161123560045)(6072148)(201708071742011); SRVR:HE1PR0401MB2427; BCL:0; PCL:0; RULEID:; SRVR:HE1PR0401MB2427; X-Forefront-PRVS: 066153096A X-Forefront-Antispam-Report: SFV:NSPM; SFS:(10009020)(366004)(376002)(39380400002)(39860400002)(346002)(396003)(199004)(189003)(5660300001)(26005)(186003)(47776003)(16526019)(6506007)(386003)(51416003)(55236004)(52116002)(2906002)(50226002)(59450400001)(97736004)(486006)(105586002)(25786009)(44832011)(106356001)(76176011)(81156014)(86362001)(68736007)(36756003)(6636002)(6512007)(2616005)(11346002)(53936002)(956004)(50466002)(7736002)(81166006)(8676002)(16586007)(6116002)(478600001)(446003)(5009440100003)(48376002)(6666003)(316002)(476003)(3846002)(305945005)(4326008)(551934003)(66066001)(8936002)(6486002)(110426005)(217873001); DIR:OUT; SFP:1101; SCL:1; SRVR:HE1PR0401MB2427; H:b27504-OptiPlex-790.ap.freescale.net; FPR:; SPF:None; LANG:en; PTR:InfoNoRecords; A:1; MX:1; Received-SPF: None (protection.outlook.com: nxp.com does not designate permitted sender hosts) X-Microsoft-Exchange-Diagnostics: =?us-ascii?Q?1; HE1PR0401MB2427; 23:+VUdLImTEoYoJdB0ag0UOJATxdzSYhD9O8qwf4h?= =?us-ascii?Q?8xQRcq+ikLVCzmMK0wpA/XDI4jC6Uc/CvCTt+VATYVr3M3EJJ8zVhLLBDYG3?= =?us-ascii?Q?FyVy05H26vnU8tPIrV53d6W9YqocftGUy1fcJOARur619aWaLPOHUpkI2J9F?= =?us-ascii?Q?MJBC42NpIEq3vIf5diu+nr478rVZ+mY7kTOL4K+WIvU7W4E0gfOQIfV6tGtb?= =?us-ascii?Q?dBpo9rW4J9sGf95E7GScvXn0OGOxS4NR6nZiP7tXn5EklNDVoLAu4AaHMSCR?= =?us-ascii?Q?QBhUl7ODjRejV3LIIrP1PiMzNmyAXJWwHz/hYwo3mi1xi2H639slaBCcF5rx?= =?us-ascii?Q?Ym0O1IHidY1wsylZpA9VNqQEQLzWrqBIJ43QeyhcSYTuIZZKM7tDOJ6q7Uk4?= =?us-ascii?Q?r6EdqkYoJ3odTpCfXsVuWhzi1Cu16f21albDcIQ+dRNT/ZUicDkupfe1HbQ0?= =?us-ascii?Q?NLvF4J97fM+XYiuBaP/ke0O8z1t3zrX0LWsy+ptZ7e20l8+aPpyqaqBUf/qg?= =?us-ascii?Q?UidsRY/2G9GnvxHSuE0c5xfJubCdD23FrY+/bZaxulazmJNYFx1yAPmUvSHg?= =?us-ascii?Q?9Srfp7BSFc6s7DqMAcV9X7TgWwzq9sdtKz8qgNKrhBgJvOrnjIg8x5oh2ssj?= =?us-ascii?Q?epGtaVi6qXP3kDuVohrXXcw+hpXZwmPu0t0kq6nFAKPvFu1oH919SO6vwTCV?= =?us-ascii?Q?QUdEkXaE7j8nY+EgfSDOPy2kl0TQb8Ytz8v0kUUrtnFmdJl2HvSEG2EY+Glv?= =?us-ascii?Q?fYUoxMldLomlXvr8mtEaF29bCCe1oJf9ACQeRP1q7kN9neaKA69euKnmWNkv?= =?us-ascii?Q?iq/NBozkByuvBJwBypHicJT4lgDL1T9pmGZY0B6XmarecqNtaKf+IhirWjdV?= =?us-ascii?Q?ogzGaizRUdDHXJJnyTu7v/bua3HNl+emcgRiz3X5+3kVeRO6uFVAwPHbNJLo?= =?us-ascii?Q?tO8e74QKOxfPLSjMV7tIc9Zca6iQcMm893MRoaj+CUoT4z4VN/Ft8WOzxFlQ?= =?us-ascii?Q?9dRunn16i4R7fr2xnVC9TLplO2GLfb1jts9YALHEN8BuMhAOmRo9JyS93k/y?= =?us-ascii?Q?5kSl4LHg6LD0X3fdjs7LQe5ilq7w0PIwf1kbjXPch+D88lFFwkgaDIOvjRzj?= =?us-ascii?Q?x5Jwmo+yj9bKixRoO8JZvHdnbOqOmdXw5RWF2v2DumXuEfxl0vVhCqm1k+XG?= =?us-ascii?Q?8mOxUAQzSwipK+wU5LctPqBTCa64ZXw0gHpasSx53jSofaLQEgSlog89nHBv?= =?us-ascii?Q?cUSXEnbQMm7NYtK9PvuxPb2R12nUGkRclI1W9p4umGkfe4/qRJGU+depWxL5?= =?us-ascii?Q?sUiR0A+bdxMdvTsVaf7660+3sxGSD32ldmCE7cMO8EF2ISqSBH6C/Bv8jpv5?= =?us-ascii?Q?1tMaA/g=3D=3D?= X-Microsoft-Antispam-Message-Info: kLvA9NWvikHOVzLNIl9K0ASUFexVqbLQRjd/T7hIA0qCKiN/0mv8K2ZUCnoLb8FUctToHCf5Re4/H8D4DwALlpzaFx0CnaqEd47w3fO0Xs5UIN1QjOWgDvfST5/FkFfdVifwz3wBoDqqLGn71HRQuhuL2NkFnlLBW4vDxOh/Vcr6n0DsY2Hv4RRGb63lQftd X-Microsoft-Exchange-Diagnostics: 1; HE1PR0401MB2427; 6:0oqffO4PrCUblQ5RfGdc4s71/lyF53PMpUUOBF1z1pFmDvjBh18g5Ty+s+iZsI3/g7njT0Zad1VcS+LkA9KX4brB8VSKQtlIxaXtnhclYvuu/rDxyPExx6VB/vstgcbxrMLqTlc4JzZqeNQmlRBY8+A+IQAebn3nOkqEwjF4Tv0h1dQ6n25bPaQJQzl2Yxr/O7On1RxCjFNqkZYYI4shE1U13po5tc0CBWqioDCu7B7e9rI4hY5C9zqS4SA5E/mgFZeAsMcMj7hAQZvgsWpFMbsSQhFQ3Oaflts3z2RljVtV59xAQHruxVtkWFwtBrIgYOuEStbJGtoSLtrpKMAAN9dsvkM3K4L/LvhLF9PkT4nGEzUJ63HvZNAuN5yginU6zT2sb8iccTBC8Vk0a8z3zU9y7uNzEas5i312U+HlWaOJ+7dyi1KSdw9YVyMJE5Rk6fcm9A3kQhtJ1vC9fwA0tw==; 5:DIs49YQn1nXbldhOBDEMPHUbYwqwktqSMSfH8Pnqr9WZgO53T/PMOfXHNK+g2boHCxFDpSS4ppbrisLvyHD98TMdfDEE684GzxzaIZyhA4kfEZqtWmsfB7mENMId/kE6hZNXLMSSSNGLqiik8q5+t6dHdbNwiXVdKUzzxMGGHhg=; 24:CV1Ppiyp4HufEgzkFu5YUP+sAvZz3ufNLRXoZg1ffbI6MFjpMPRXIDiprWlUC2Iag3IuZ+TNv/Mi3I5UAym6Fid/SaSJd8Inb/klzejE+IE= SpamDiagnosticOutput: 1:99 SpamDiagnosticMetadata: NSPM X-Microsoft-Exchange-Diagnostics: 1; HE1PR0401MB2427; 7:6e8AytuZ5kdmS0X+uZMvSNJgZKvYyfHI5XmUMoPfbM5yleSs2sC99vQmFn8YAhXaxXad8sYipaJWTOb9YqGVeX6erYUMggSokZ0UlSGdmaBMl0PIEw+5VQ4UCe3ee/TxAl22Cyk4oK/1Tl2QeAtHJV7MlZ8wUIq/H+0b51Je4lehLTJ6F6RRmGTo0w91M5WTvYe8GQmACBW0M6lai2qG/lU6TNG0IqOt9Hl1gFEvqbmbhvqcx/1kefBIh+eA/QeC X-MS-Office365-Filtering-Correlation-Id: 68ab15d2-c8b0-4e27-a3a7-08d5b10fe275 X-OriginatorOrg: nxp.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 03 May 2018 16:06:50.9622 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: 68ab15d2-c8b0-4e27-a3a7-08d5b10fe275 X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 686ea1d3-bc2b-4c6f-a92c-d99c5c301635 X-MS-Exchange-Transport-CrossTenantHeadersStamped: HE1PR0401MB2427 Subject: [dpdk-dev] [PATCH RESEND v7 6/8] raw/dpaa2_qdma: support configuration APIs X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 03 May 2018 16:06:54 -0000 Signed-off-by: Nipun Gupta Acked-by: Shreyansh Jain --- doc/api/doxy-api-index.md | 1 + doc/api/doxy-api.conf | 1 + drivers/raw/dpaa2_qdma/Makefile | 2 + drivers/raw/dpaa2_qdma/dpaa2_qdma.c | 375 +++++++++++++++++++++ drivers/raw/dpaa2_qdma/dpaa2_qdma.h | 63 ++++ drivers/raw/dpaa2_qdma/meson.build | 2 + drivers/raw/dpaa2_qdma/rte_pmd_dpaa2_qdma.h | 216 ++++++++++++ .../raw/dpaa2_qdma/rte_pmd_dpaa2_qdma_version.map | 12 + 8 files changed, 672 insertions(+) create mode 100644 drivers/raw/dpaa2_qdma/rte_pmd_dpaa2_qdma.h diff --git a/doc/api/doxy-api-index.md b/doc/api/doxy-api-index.md index 26ce7b4..437d903 100644 --- a/doc/api/doxy-api-index.md +++ b/doc/api/doxy-api-index.md @@ -38,6 +38,7 @@ The public API headers are grouped by topics: [i40e] (@ref rte_pmd_i40e.h), [bnxt] (@ref rte_pmd_bnxt.h), [dpaa] (@ref rte_pmd_dpaa.h), + [dpaa2_qdma] (@ref rte_pmd_dpaa2_qdma.h), [crypto_scheduler] (@ref rte_cryptodev_scheduler.h) - **memory**: diff --git a/doc/api/doxy-api.conf b/doc/api/doxy-api.conf index 5686cbb..88bee03 100644 --- a/doc/api/doxy-api.conf +++ b/doc/api/doxy-api.conf @@ -37,6 +37,7 @@ INPUT = doc/api/doxy-api-index.md \ drivers/net/i40e \ drivers/net/ixgbe \ drivers/net/softnic \ + drivers/raw/dpaa2_qdma \ lib/librte_eal/common/include \ lib/librte_eal/common/include/generic \ lib/librte_acl \ diff --git a/drivers/raw/dpaa2_qdma/Makefile b/drivers/raw/dpaa2_qdma/Makefile index 68c785a..d88809e 100644 --- a/drivers/raw/dpaa2_qdma/Makefile +++ b/drivers/raw/dpaa2_qdma/Makefile @@ -32,4 +32,6 @@ LIBABIVER := 1 # SRCS-$(CONFIG_RTE_LIBRTE_PMD_DPAA2_QDMA_RAWDEV) += dpaa2_qdma.c +SYMLINK-$(CONFIG_RTE_LIBRTE_PMD_DPAA2_QDMA_RAWDEV)-include += rte_pmd_dpaa2_qdma.h + include $(RTE_SDK)/mk/rte.lib.mk diff --git a/drivers/raw/dpaa2_qdma/dpaa2_qdma.c b/drivers/raw/dpaa2_qdma/dpaa2_qdma.c index 9288350..b38a282 100644 --- a/drivers/raw/dpaa2_qdma/dpaa2_qdma.c +++ b/drivers/raw/dpaa2_qdma/dpaa2_qdma.c @@ -20,6 +20,7 @@ #include "dpaa2_qdma.h" #include "dpaa2_qdma_logs.h" +#include "rte_pmd_dpaa2_qdma.h" /* Dynamic log type identifier */ int dpaa2_qdma_logtype; @@ -32,6 +33,380 @@ static struct qdma_hw_queue_list qdma_queue_list = TAILQ_HEAD_INITIALIZER(qdma_queue_list); +/* QDMA Virtual Queues */ +struct qdma_virt_queue *qdma_vqs; + +/* QDMA per core data */ +struct qdma_per_core_info qdma_core_info[RTE_MAX_LCORE]; + +static struct qdma_hw_queue * +alloc_hw_queue(uint32_t lcore_id) +{ + struct qdma_hw_queue *queue = NULL; + + DPAA2_QDMA_FUNC_TRACE(); + + /* Get a free queue from the list */ + TAILQ_FOREACH(queue, &qdma_queue_list, next) { + if (queue->num_users == 0) { + queue->lcore_id = lcore_id; + queue->num_users++; + break; + } + } + + return queue; +} + +static void +free_hw_queue(struct qdma_hw_queue *queue) +{ + DPAA2_QDMA_FUNC_TRACE(); + + queue->num_users--; +} + + +static struct qdma_hw_queue * +get_hw_queue(uint32_t lcore_id) +{ + struct qdma_per_core_info *core_info; + struct qdma_hw_queue *queue, *temp; + uint32_t least_num_users; + int num_hw_queues, i; + + DPAA2_QDMA_FUNC_TRACE(); + + core_info = &qdma_core_info[lcore_id]; + num_hw_queues = core_info->num_hw_queues; + + /* + * Allocate a HW queue if there are less queues + * than maximum per core queues configured + */ + if (num_hw_queues < qdma_dev.max_hw_queues_per_core) { + queue = alloc_hw_queue(lcore_id); + if (queue) { + core_info->hw_queues[num_hw_queues] = queue; + core_info->num_hw_queues++; + return queue; + } + } + + queue = core_info->hw_queues[0]; + /* In case there is no queue associated with the core return NULL */ + if (!queue) + return NULL; + + /* Fetch the least loaded H/W queue */ + least_num_users = core_info->hw_queues[0]->num_users; + for (i = 0; i < num_hw_queues; i++) { + temp = core_info->hw_queues[i]; + if (temp->num_users < least_num_users) + queue = temp; + } + + if (queue) + queue->num_users++; + + return queue; +} + +static void +put_hw_queue(struct qdma_hw_queue *queue) +{ + struct qdma_per_core_info *core_info; + int lcore_id, num_hw_queues, i; + + DPAA2_QDMA_FUNC_TRACE(); + + /* + * If this is the last user of the queue free it. + * Also remove it from QDMA core info. + */ + if (queue->num_users == 1) { + free_hw_queue(queue); + + /* Remove the physical queue from core info */ + lcore_id = queue->lcore_id; + core_info = &qdma_core_info[lcore_id]; + num_hw_queues = core_info->num_hw_queues; + for (i = 0; i < num_hw_queues; i++) { + if (queue == core_info->hw_queues[i]) + break; + } + for (; i < num_hw_queues - 1; i++) + core_info->hw_queues[i] = core_info->hw_queues[i + 1]; + core_info->hw_queues[i] = NULL; + } else { + queue->num_users--; + } +} + +int __rte_experimental +rte_qdma_init(void) +{ + DPAA2_QDMA_FUNC_TRACE(); + + rte_spinlock_init(&qdma_dev.lock); + + return 0; +} + +void __rte_experimental +rte_qdma_attr_get(struct rte_qdma_attr *qdma_attr) +{ + DPAA2_QDMA_FUNC_TRACE(); + + qdma_attr->num_hw_queues = qdma_dev.num_hw_queues; +} + +int __rte_experimental +rte_qdma_reset(void) +{ + struct qdma_hw_queue *queue; + int i; + + DPAA2_QDMA_FUNC_TRACE(); + + /* In case QDMA device is not in stopped state, return -EBUSY */ + if (qdma_dev.state == 1) { + DPAA2_QDMA_ERR( + "Device is in running state. Stop before reset."); + return -EBUSY; + } + + /* In case there are pending jobs on any VQ, return -EBUSY */ + for (i = 0; i < qdma_dev.max_vqs; i++) { + if (qdma_vqs[i].in_use && (qdma_vqs[i].num_enqueues != + qdma_vqs[i].num_dequeues)) + DPAA2_QDMA_ERR("Jobs are still pending on VQ: %d", i); + return -EBUSY; + } + + /* Reset HW queues */ + TAILQ_FOREACH(queue, &qdma_queue_list, next) + queue->num_users = 0; + + /* Reset and free virtual queues */ + for (i = 0; i < qdma_dev.max_vqs; i++) { + if (qdma_vqs[i].status_ring) + rte_ring_free(qdma_vqs[i].status_ring); + } + if (qdma_vqs) + rte_free(qdma_vqs); + qdma_vqs = NULL; + + /* Reset per core info */ + memset(&qdma_core_info, 0, + sizeof(struct qdma_per_core_info) * RTE_MAX_LCORE); + + /* Free the FLE pool */ + if (qdma_dev.fle_pool) + rte_mempool_free(qdma_dev.fle_pool); + + /* Reset QDMA device structure */ + qdma_dev.mode = RTE_QDMA_MODE_HW; + qdma_dev.max_hw_queues_per_core = 0; + qdma_dev.fle_pool = NULL; + qdma_dev.fle_pool_count = 0; + qdma_dev.max_vqs = 0; + + return 0; +} + +int __rte_experimental +rte_qdma_configure(struct rte_qdma_config *qdma_config) +{ + int ret; + + DPAA2_QDMA_FUNC_TRACE(); + + /* In case QDMA device is not in stopped state, return -EBUSY */ + if (qdma_dev.state == 1) { + DPAA2_QDMA_ERR( + "Device is in running state. Stop before config."); + return -1; + } + + /* Reset the QDMA device */ + ret = rte_qdma_reset(); + if (ret) { + DPAA2_QDMA_ERR("Resetting QDMA failed"); + return ret; + } + + /* Set mode */ + qdma_dev.mode = qdma_config->mode; + + /* Set max HW queue per core */ + if (qdma_config->max_hw_queues_per_core > MAX_HW_QUEUE_PER_CORE) { + DPAA2_QDMA_ERR("H/W queues per core is more than: %d", + MAX_HW_QUEUE_PER_CORE); + return -EINVAL; + } + qdma_dev.max_hw_queues_per_core = + qdma_config->max_hw_queues_per_core; + + /* Allocate Virtual Queues */ + qdma_vqs = rte_malloc("qdma_virtual_queues", + (sizeof(struct qdma_virt_queue) * qdma_config->max_vqs), + RTE_CACHE_LINE_SIZE); + if (!qdma_vqs) { + DPAA2_QDMA_ERR("qdma_virtual_queues allocation failed"); + return -ENOMEM; + } + qdma_dev.max_vqs = qdma_config->max_vqs; + + /* Allocate FLE pool */ + qdma_dev.fle_pool = rte_mempool_create("qdma_fle_pool", + qdma_config->fle_pool_count, QDMA_FLE_POOL_SIZE, + QDMA_FLE_CACHE_SIZE(qdma_config->fle_pool_count), 0, + NULL, NULL, NULL, NULL, SOCKET_ID_ANY, 0); + if (!qdma_dev.fle_pool) { + DPAA2_QDMA_ERR("qdma_fle_pool create failed"); + rte_free(qdma_vqs); + qdma_vqs = NULL; + return -ENOMEM; + } + qdma_dev.fle_pool_count = qdma_config->fle_pool_count; + + return 0; +} + +int __rte_experimental +rte_qdma_start(void) +{ + DPAA2_QDMA_FUNC_TRACE(); + + qdma_dev.state = 1; + + return 0; +} + +int __rte_experimental +rte_qdma_vq_create(uint32_t lcore_id, uint32_t flags) +{ + char ring_name[32]; + int i; + + DPAA2_QDMA_FUNC_TRACE(); + + rte_spinlock_lock(&qdma_dev.lock); + + /* Get a free Virtual Queue */ + for (i = 0; i < qdma_dev.max_vqs; i++) { + if (qdma_vqs[i].in_use == 0) + break; + } + + /* Return in case no VQ is free */ + if (i == qdma_dev.max_vqs) { + rte_spinlock_unlock(&qdma_dev.lock); + return -ENODEV; + } + + if (qdma_dev.mode == RTE_QDMA_MODE_HW || + (flags & RTE_QDMA_VQ_EXCLUSIVE_PQ)) { + /* Allocate HW queue for a VQ */ + qdma_vqs[i].hw_queue = alloc_hw_queue(lcore_id); + qdma_vqs[i].exclusive_hw_queue = 1; + } else { + /* Allocate a Ring for Virutal Queue in VQ mode */ + sprintf(ring_name, "status ring %d", i); + qdma_vqs[i].status_ring = rte_ring_create(ring_name, + qdma_dev.fle_pool_count, rte_socket_id(), 0); + if (!qdma_vqs[i].status_ring) { + DPAA2_QDMA_ERR("Status ring creation failed for vq"); + rte_spinlock_unlock(&qdma_dev.lock); + return rte_errno; + } + + /* Get a HW queue (shared) for a VQ */ + qdma_vqs[i].hw_queue = get_hw_queue(lcore_id); + qdma_vqs[i].exclusive_hw_queue = 0; + } + + if (qdma_vqs[i].hw_queue == NULL) { + DPAA2_QDMA_ERR("No H/W queue available for VQ"); + if (qdma_vqs[i].status_ring) + rte_ring_free(qdma_vqs[i].status_ring); + qdma_vqs[i].status_ring = NULL; + rte_spinlock_unlock(&qdma_dev.lock); + return -ENODEV; + } + + qdma_vqs[i].in_use = 1; + qdma_vqs[i].lcore_id = lcore_id; + + rte_spinlock_unlock(&qdma_dev.lock); + + return i; +} + +void __rte_experimental +rte_qdma_vq_stats(uint16_t vq_id, + struct rte_qdma_vq_stats *vq_status) +{ + struct qdma_virt_queue *qdma_vq = &qdma_vqs[vq_id]; + + DPAA2_QDMA_FUNC_TRACE(); + + if (qdma_vq->in_use) { + vq_status->exclusive_hw_queue = qdma_vq->exclusive_hw_queue; + vq_status->lcore_id = qdma_vq->lcore_id; + vq_status->num_enqueues = qdma_vq->num_enqueues; + vq_status->num_dequeues = qdma_vq->num_dequeues; + vq_status->num_pending_jobs = vq_status->num_enqueues - + vq_status->num_dequeues; + } +} + +int __rte_experimental +rte_qdma_vq_destroy(uint16_t vq_id) +{ + struct qdma_virt_queue *qdma_vq = &qdma_vqs[vq_id]; + + DPAA2_QDMA_FUNC_TRACE(); + + /* In case there are pending jobs on any VQ, return -EBUSY */ + if (qdma_vq->num_enqueues != qdma_vq->num_dequeues) + return -EBUSY; + + rte_spinlock_lock(&qdma_dev.lock); + + if (qdma_vq->exclusive_hw_queue) + free_hw_queue(qdma_vq->hw_queue); + else { + if (qdma_vqs->status_ring) + rte_ring_free(qdma_vqs->status_ring); + + put_hw_queue(qdma_vq->hw_queue); + } + + memset(qdma_vq, 0, sizeof(struct qdma_virt_queue)); + + rte_spinlock_lock(&qdma_dev.lock); + + return 0; +} + +void __rte_experimental +rte_qdma_stop(void) +{ + DPAA2_QDMA_FUNC_TRACE(); + + qdma_dev.state = 0; +} + +void __rte_experimental +rte_qdma_destroy(void) +{ + DPAA2_QDMA_FUNC_TRACE(); + + rte_qdma_reset(); +} + static const struct rte_rawdev_ops dpaa2_qdma_ops; static int diff --git a/drivers/raw/dpaa2_qdma/dpaa2_qdma.h b/drivers/raw/dpaa2_qdma/dpaa2_qdma.h index 8b3b1b9..fe1da41 100644 --- a/drivers/raw/dpaa2_qdma/dpaa2_qdma.h +++ b/drivers/raw/dpaa2_qdma/dpaa2_qdma.h @@ -5,6 +5,22 @@ #ifndef __DPAA2_QDMA_H__ #define __DPAA2_QDMA_H__ +struct qdma_sdd; +struct qdma_io_meta; + +#define DPAA2_QDMA_MAX_FLE 3 +#define DPAA2_QDMA_MAX_SDD 2 + +/** FLE pool size: 3 Frame list + 2 source/destination descriptor */ +#define QDMA_FLE_POOL_SIZE (sizeof(struct qdma_io_meta) + \ + sizeof(struct qbman_fle) * DPAA2_QDMA_MAX_FLE + \ + sizeof(struct qdma_sdd) * DPAA2_QDMA_MAX_SDD) +/** FLE pool cache size */ +#define QDMA_FLE_CACHE_SIZE(_num) (_num/(RTE_MAX_LCORE * 2)) + +/** Maximum possible H/W Queues on each core */ +#define MAX_HW_QUEUE_PER_CORE 64 + /** * Represents a QDMA device. * A single QDMA device exists which is combination of multiple DPDMAI rawdev's. @@ -45,6 +61,53 @@ struct qdma_hw_queue { uint32_t num_users; }; +/** Represents a QDMA virtual queue */ +struct qdma_virt_queue { + /** Status ring of the virtual queue */ + struct rte_ring *status_ring; + /** Associated hw queue */ + struct qdma_hw_queue *hw_queue; + /** Associated lcore id */ + uint32_t lcore_id; + /** States if this vq is in use or not */ + uint8_t in_use; + /** States if this vq has exclusively associated hw queue */ + uint8_t exclusive_hw_queue; + /* Total number of enqueues on this VQ */ + uint64_t num_enqueues; + /* Total number of dequeues from this VQ */ + uint64_t num_dequeues; +}; + +/** Represents a QDMA per core hw queues allocation in virtual mode */ +struct qdma_per_core_info { + /** list for allocated hw queues */ + struct qdma_hw_queue *hw_queues[MAX_HW_QUEUE_PER_CORE]; + /* Number of hw queues allocated for this core */ + uint16_t num_hw_queues; +}; + +/** Metadata which is stored with each operation */ +struct qdma_io_meta { + /** + * Context which is stored in the FLE pool (just before the FLE). + * QDMA job is stored as a this context as a part of metadata. + */ + uint64_t cnxt; + /** VQ ID is stored as a part of metadata of the enqueue command */ + uint64_t id; +}; + +/** Source/Destination Descriptor */ +struct qdma_sdd { + uint32_t rsv; + /** Stride configuration */ + uint32_t stride; + /** Route-by-port command */ + uint32_t rbpcmd; + uint32_t cmd; +} __attribute__((__packed__)); + /** Represents a DPDMAI raw device */ struct dpaa2_dpdmai_dev { /** Pointer to Next device instance */ diff --git a/drivers/raw/dpaa2_qdma/meson.build b/drivers/raw/dpaa2_qdma/meson.build index b747500..a2eb1d2 100644 --- a/drivers/raw/dpaa2_qdma/meson.build +++ b/drivers/raw/dpaa2_qdma/meson.build @@ -5,3 +5,5 @@ deps += ['rawdev', 'mempool_dpaa2', 'ring'] sources = files('dpaa2_qdma.c') allow_experimental_apis = true + +install_headers('rte_pmd_dpaa2_qdma.h') diff --git a/drivers/raw/dpaa2_qdma/rte_pmd_dpaa2_qdma.h b/drivers/raw/dpaa2_qdma/rte_pmd_dpaa2_qdma.h new file mode 100644 index 0000000..29a1e4b --- /dev/null +++ b/drivers/raw/dpaa2_qdma/rte_pmd_dpaa2_qdma.h @@ -0,0 +1,216 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright 2018 NXP + */ + +#ifndef __RTE_PMD_DPAA2_QDMA_H__ +#define __RTE_PMD_DPAA2_QDMA_H__ + +/** + * @file + * + * NXP dpaa2 QDMA specific structures. + * + */ + +/** Determines the mode of operation */ +enum { + /** + * Allocate a H/W queue per VQ i.e. Exclusive hardware queue for a VQ. + * This mode will have best performance. + */ + RTE_QDMA_MODE_HW, + /** + * A VQ shall not have an exclusive associated H/W queue. + * Rather a H/W Queue will be shared by multiple Virtual Queues. + * This mode will have intermediate data structures to support + * multi VQ to PQ mappings thus having some performance implications. + * Note: Even in this mode there is an option to allocate a H/W + * queue for a VQ. Please see 'RTE_QDMA_VQ_EXCLUSIVE_PQ' flag. + */ + RTE_QDMA_MODE_VIRTUAL +}; + +/** + * If user has configued a Virtual Queue mode, but for some particular VQ + * user needs an exclusive H/W queue associated (for better performance + * on that particular VQ), then user can pass this flag while creating the + * Virtual Queue. A H/W queue will be allocated corresponding to + * VQ which uses this flag. + */ +#define RTE_QDMA_VQ_EXCLUSIVE_PQ (1ULL) + +/** States if the source addresses is physical. */ +#define RTE_QDMA_JOB_SRC_PHY (1ULL) + +/** States if the destination addresses is physical. */ +#define RTE_QDMA_JOB_DEST_PHY (1ULL << 1) + +/** Provides QDMA device attributes */ +struct rte_qdma_attr { + /** total number of hw QDMA queues present */ + uint16_t num_hw_queues; +}; + +/** QDMA device configuration structure */ +struct rte_qdma_config { + /** Number of maximum hw queues to allocate per core. */ + uint16_t max_hw_queues_per_core; + /** Maximum number of VQ's to be used. */ + uint16_t max_vqs; + /** mode of operation - physical(h/w) or virtual */ + uint8_t mode; + /** + * User provides this as input to the driver as a size of the FLE pool. + * FLE's (and corresponding source/destination descriptors) are + * allocated by the driver at enqueue time to store src/dest and + * other data and are freed at the dequeue time. This determines the + * maximum number of inflight jobs on the QDMA device. This should + * be power of 2. + */ + int fle_pool_count; +}; + +/** Provides QDMA device statistics */ +struct rte_qdma_vq_stats { + /** States if this vq has exclusively associated hw queue */ + uint8_t exclusive_hw_queue; + /** Associated lcore id */ + uint32_t lcore_id; + /* Total number of enqueues on this VQ */ + uint64_t num_enqueues; + /* Total number of dequeues from this VQ */ + uint64_t num_dequeues; + /* total number of pending jobs in this VQ */ + uint64_t num_pending_jobs; +}; + +/** Determines a QDMA job */ +struct rte_qdma_job { + /** Source Address from where DMA is (to be) performed */ + uint64_t src; + /** Destination Address where DMA is (to be) done */ + uint64_t dest; + /** Length of the DMA operation in bytes. */ + uint32_t len; + /** See RTE_QDMA_JOB_ flags */ + uint32_t flags; + /** + * User can specify a context which will be maintained + * on the dequeue operation. + */ + uint64_t cnxt; + /** + * Status of the transaction. + * This is filled in the dequeue operation by the driver. + */ + uint8_t status; +}; + +/** + * Initialize the QDMA device. + * + * @returns + * - 0: Success. + * - <0: Error code. + */ +int __rte_experimental +rte_qdma_init(void); + +/** + * Get the QDMA attributes. + * + * @param qdma_attr + * QDMA attributes providing total number of hw queues etc. + */ +void __rte_experimental +rte_qdma_attr_get(struct rte_qdma_attr *qdma_attr); + +/** + * Reset the QDMA device. This API will completely reset the QDMA + * device, bringing it to original state as if only rte_qdma_init() API + * has been called. + * + * @returns + * - 0: Success. + * - <0: Error code. + */ +int __rte_experimental +rte_qdma_reset(void); + +/** + * Configure the QDMA device. + * + * @returns + * - 0: Success. + * - <0: Error code. + */ +int __rte_experimental +rte_qdma_configure(struct rte_qdma_config *qdma_config); + +/** + * Start the QDMA device. + * + * @returns + * - 0: Success. + * - <0: Error code. + */ +int __rte_experimental +rte_qdma_start(void); + +/** + * Create a Virtual Queue on a particular lcore id. + * This API can be called from any thread/core. User can create/destroy + * VQ's at runtime. + * + * @param lcore_id + * LCORE ID on which this particular queue would be associated with. + * @param flags + * RTE_QDMA_VQ_ flags. See macro definitions. + * + * @returns + * - >= 0: Virtual queue ID. + * - <0: Error code. + */ +int __rte_experimental +rte_qdma_vq_create(uint32_t lcore_id, uint32_t flags); + +/** + * Get a Virtual Queue statistics. + * + * @param vq_id + * Virtual Queue ID. + * @param vq_stats + * VQ statistics structure which will be filled in by the driver. + */ +void __rte_experimental +rte_qdma_vq_stats(uint16_t vq_id, + struct rte_qdma_vq_stats *vq_stats); + +/** + * Destroy the Virtual Queue specified by vq_id. + * This API can be called from any thread/core. User can create/destroy + * VQ's at runtime. + * + * @param vq_id + * Virtual Queue ID which needs to be deinialized. + * + * @returns + * - 0: Success. + * - <0: Error code. + */ +int __rte_experimental +rte_qdma_vq_destroy(uint16_t vq_id); + +/** + * Stop QDMA device. + */ +void __rte_experimental +rte_qdma_stop(void); + +/** + * Destroy the QDMA device. + */ +void __rte_experimental +rte_qdma_destroy(void); + +#endif /* __RTE_PMD_DPAA2_QDMA_H__*/ diff --git a/drivers/raw/dpaa2_qdma/rte_pmd_dpaa2_qdma_version.map b/drivers/raw/dpaa2_qdma/rte_pmd_dpaa2_qdma_version.map index 33d2379..7335c35 100644 --- a/drivers/raw/dpaa2_qdma/rte_pmd_dpaa2_qdma_version.map +++ b/drivers/raw/dpaa2_qdma/rte_pmd_dpaa2_qdma_version.map @@ -1,4 +1,16 @@ EXPERIMENTAL { + global: + + rte_qdma_attr_get; + rte_qdma_configure; + rte_qdma_destroy; + rte_qdma_init; + rte_qdma_reset; + rte_qdma_start; + rte_qdma_stop; + rte_qdma_vq_create; + rte_qdma_vq_destroy; + rte_qdma_vq_stats; local: *; }; -- 1.9.1