From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from dpdk.org (dpdk.org [92.243.14.124]) by inbox.dpdk.org (Postfix) with ESMTP id 50D5EA04B6; Sun, 11 Oct 2020 09:36:46 +0200 (CEST) Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id 2C9B81D443; Sun, 11 Oct 2020 09:36:45 +0200 (CEST) Received: from mx0b-0016f401.pphosted.com (mx0b-0016f401.pphosted.com [67.231.156.173]) by dpdk.org (Postfix) with ESMTP id 422011B875 for ; Sun, 11 Oct 2020 09:36:43 +0200 (CEST) Received: from pps.filterd (m0045851.ppops.net [127.0.0.1]) by mx0b-0016f401.pphosted.com (8.16.0.42/8.16.0.42) with SMTP id 09B7LILo006058; Sun, 11 Oct 2020 00:36:39 -0700 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=marvell.com; h=from : to : cc : subject : date : message-id : references : in-reply-to : content-type : content-transfer-encoding : mime-version; s=pfpt0220; bh=aJZfJhgtPAQhcB3NVTaFhRWOdEUEU6M/i6ZUT8Do3sI=; b=PtCHM/X3cPxyVkxLEVVCz9hrcqkp6eTJVRne/iE2LAfsFgjHKYT/lDqUA26hIwTsOSBI 4ZHSyK2O++dU0yeqbt3a3M8FAFoXEiNakDHXHjbRCl7JU3ijoWG8dwO7d13uRQcAv29f GwWr2PwJH/sbRLYBi4VPcBfv7IFaPm55q6czzZtUKXDlx/N2is/kRlr/SrmVwApuHYdQ ZVCUjR1SfdqQwQ9ZR2QHRMie9w8vhmGriAXqesjssupVvR2088nlBULCBuD840OxQeq/ LQ4tSWSo9R2lt37GEfGykEZhIXoZbyMF3uB4rgsCSkfi+MpCTKJWJhyy13tnxdXrpjjK 8A== Received: from sc-exch03.marvell.com ([199.233.58.183]) by mx0b-0016f401.pphosted.com with ESMTP id 343cfj1xep-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-SHA384 bits=256 verify=NOT); Sun, 11 Oct 2020 00:36:38 -0700 Received: from DC5-EXCH01.marvell.com (10.69.176.38) by SC-EXCH03.marvell.com (10.93.176.83) with Microsoft SMTP Server (TLS) id 15.0.1497.2; Sun, 11 Oct 2020 00:36:37 -0700 Received: from SC-EXCH02.marvell.com (10.93.176.82) by DC5-EXCH01.marvell.com (10.69.176.38) with Microsoft SMTP Server (TLS) id 15.0.1497.2; Sun, 11 Oct 2020 00:36:36 -0700 Received: from NAM12-MW2-obe.outbound.protection.outlook.com (104.47.66.40) by SC-EXCH02.marvell.com (10.93.176.82) with Microsoft SMTP Server (TLS) id 15.0.1497.2 via Frontend Transport; Sun, 11 Oct 2020 00:36:35 -0700 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=JpVA8RjkHOfyarnwLzvpeHgyqtwSIplwlZnLfNfGcWrgz06/vsa4XxnDjDiRN4EIBExl4+b5QyCr0r0lx7aXoq88BXFHXK5/fehl4m6csDkyeyhKSwlE1vLspe1LNNIBHp0bm0nf1+fKvtFLmWtV58WgMBkyHsCtQNzFQz/Ed0/cJ7wCuTa0I/AS72H0fVJu7kod3gwtJOWuOcVXR4kHHNpGNQsI4bdJj/26GsxAMZdnzjYVdQs+hp2snGV0H75v1RJM9W+NTEgQnZ58TRyry5T4pqgDHRJ3hVF3j9HSCS2+kdV+CkSx+ibda9odXCevsbZJVh+OUSIIztHk+F5gsw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=aJZfJhgtPAQhcB3NVTaFhRWOdEUEU6M/i6ZUT8Do3sI=; b=dZdHqbpZ04Cwl8XHTFNFFRMUo/8hcF3ClISf/OgBSs6X5t1cMyMknpJ5kGIYDjMz4qbyBumDMTCuIRy+IZgZ/Q+lO2bGhbrtx1H7spOxWCfxd5LMllckRZeFgNEo+zkumI2Wlh+MyD+8z+rWzu01HJGcHcXVYbSxzrK53OR+WjOrDxrp95WCunfLIsOxBwLEi39Fjzq5i8GrWDce15Upa2XkuO+0XwWOLmScMWPXNymNAGDs1XsdTz5Dm0LZuWd1XP5tt2KSMC4+v7IjsFb7Kht/ug5VlqWR04B4/+E/Ydu8W3DoIfZtXEMoBB6DDsprLv+QQz7Y5kho+eF/BBvc2A== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=marvell.com; dmarc=pass action=none header.from=marvell.com; dkim=pass header.d=marvell.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=marvell.onmicrosoft.com; s=selector1-marvell-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=aJZfJhgtPAQhcB3NVTaFhRWOdEUEU6M/i6ZUT8Do3sI=; b=tiK8+zYoRRipAr5bDRr3hUZAnEpv3x+osP8n3lsJ7qTansMmn2TK1gG5BU/yAYmFWNDIt1L8WTzvN2to+uk7m1Gwtodh+YtAOVwWw/xN5vBTyZ5x8xazq9IwrTLoptfHCaXRn/pf/HTPqtCNrF33Sbh3K5iHgmPX0byLAh0ZIdk= Received: from DM5PR18MB2214.namprd18.prod.outlook.com (2603:10b6:4:b9::21) by DM5PR18MB1321.namprd18.prod.outlook.com (2603:10b6:3:14f::12) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.3455.21; Sun, 11 Oct 2020 07:36:33 +0000 Received: from DM5PR18MB2214.namprd18.prod.outlook.com ([fe80::48c2:9ca9:3159:368e]) by DM5PR18MB2214.namprd18.prod.outlook.com ([fe80::48c2:9ca9:3159:368e%5]) with mapi id 15.20.3455.029; Sun, 11 Oct 2020 07:36:33 +0000 From: Liron Himi To: Guy Kaneti , Jerin Jacob Kollanukkaran , Nithin Kumar Dabilpuram , "thomas@monjalon.net" , "mdr@ashroe.eu" , "nhorman@tuxdriver.com" , "bruce.richardson@intel.com" , "anatoly.burakov@intel.com" , "john.mcnamara@intel.com" , "marko.kovacevic@intel.com" CC: "dev@dpdk.org" , Smadar Fuks , "Dovrat Zifroni" , Guy Kaneti , "orika@mellanox.com" , Liron Himi Thread-Topic: [dpdk-dev] [PATCH 2/4] regex/octeontx2: add build infra and device support Thread-Index: AQHWgFssPO4LtbVLuU6OdpFOcAZ8yqmSQaXg Date: Sun, 11 Oct 2020 07:36:33 +0000 Message-ID: References: <20200901122436.17748-1-guyk@marvell.com> <20200901122436.17748-3-guyk@marvell.com> In-Reply-To: <20200901122436.17748-3-guyk@marvell.com> Accept-Language: he-IL, en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-originating-ip: [89.139.26.150] x-ms-publictraffictype: Email x-ms-office365-filtering-correlation-id: 47fc8f0e-ad90-456b-afe2-08d86db86082 x-ms-traffictypediagnostic: DM5PR18MB1321: x-ms-exchange-transport-forked: True x-microsoft-antispam-prvs: x-ms-oob-tlc-oobclassifiers: OLM:121; x-ms-exchange-senderadcheck: 1 x-microsoft-antispam: BCL:0; x-microsoft-antispam-message-info: Vwk/EZxIs1YUW1ZwOaVl3NKkSyF7OCCIIdL7VcQXry6BsYt97QaOXm6IibHF38P5kanJqqAC1iviuM/ucM9KUP78RnxxjMWdEOVyYtkattYA96WeweF3J5F7ijKDdRhTkNVMRaBCXVuz+zOvXJh4ePAtVh+NVMa/0waj2huxsVLUmqORQmBu/RsJqEv1R4s6QpRK+9INDYI9juT5RDRk4n0vtQ2thYq+t3KdO0IQvW4sh/tvsZUqH06T21ifp+eTYyAz4zyo3SXFNHKx+jII+oyHZNJuP2Qs6eTWPSyepa4PHodmwvBDknp11OKuj7x51EwTf23MaJvoVytC9C9OSTB4WvkSLDL8ZPKyO2gfiU+TTQEPp7rhAaiwrWFBBqVF x-forefront-antispam-report: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:DM5PR18MB2214.namprd18.prod.outlook.com; PTR:; CAT:NONE; SFS:(4636009)(39850400004)(136003)(396003)(376002)(366004)(346002)(71200400001)(86362001)(66446008)(30864003)(478600001)(66476007)(66946007)(8936002)(64756008)(26005)(186003)(9686003)(66556008)(4326008)(110136005)(7696005)(54906003)(53546011)(52536014)(2906002)(5660300002)(6506007)(55016002)(76116006)(316002)(83380400001)(107886003)(33656002)(921003)(559001)(579004); DIR:OUT; SFP:1101; x-ms-exchange-antispam-messagedata: MgYzfJAqNhBKNEuguHp5ceDGh6X4FCUiD+++DOB2V0qNJRV5QE546sOY9fH4vjlzxe/dJw8pTW3HM5Lgam1gBbuIJYpiQ14pHJM51hNx9rNGFPc9LdlB13zwWQ26sgMVQ1Or8lBQDz7Y5PJzYacG4mmIRZ2ajpDJ3tAJtRK6uq4MKkjNv8UdFzupUCKWusIPpgi972qCCL6yo9g9tYQ8L1ypqwW7NhtQ9JCHugl97NRhmhOzKw9j8bQKEimUearEkBwl85uIpRM6+f+6PzUV6nAfZvBL+UuZKsx4dFlronLQPwxLw4uBYEyMejKM90ChfvaKyJED7Ijs47yd6PZZuU5WqX07mNpB1EZ/SHbEOWHxPfdNF/D7kMHAUEt3MJawRV5J9V09KsqOgZHO+zCu21l2EyE4JkXyVXfybAo8JjVytKRQeUepfWBcP/l4myrgJS0KzHv+w/XORBY3R+olgCOb4MLtdzQpM4Uclxf9s3oaYjWBTvodS5wnms13WQwYOz43jo9xCnwYgR3BIJRv3yQQx5T07s2A8GqycKQmtqa9FG5QUAyrOrX3Un7EkfTyjtzYuTMqtCsGdR5FsbLMf0h5uS85IpiGy5mbXm0/ptzYw6h3b4RkssOXGQxmjM4CvJEkoq4wgvSr0SQwXK94eA== Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-AuthSource: DM5PR18MB2214.namprd18.prod.outlook.com X-MS-Exchange-CrossTenant-Network-Message-Id: 47fc8f0e-ad90-456b-afe2-08d86db86082 X-MS-Exchange-CrossTenant-originalarrivaltime: 11 Oct 2020 07:36:33.1663 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Hosted X-MS-Exchange-CrossTenant-id: 70e1fb47-1155-421d-87fc-2e58f638b6e0 X-MS-Exchange-CrossTenant-mailboxtype: HOSTED X-MS-Exchange-CrossTenant-userprincipalname: bd8/6xg01en30yOj7Pjs1Jzm8giDf6BVExeng0HDYn3nhExiKIhQghFIPoVsA0YedrOUoUb6Dy37f9voPwxXyw== X-MS-Exchange-Transport-CrossTenantHeadersStamped: DM5PR18MB1321 X-OriginatorOrg: marvell.com X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10434:6.0.235, 18.0.687 definitions=2020-10-10_07:2020-10-09, 2020-10-10 signatures=0 Subject: Re: [dpdk-dev] [PATCH 2/4] regex/octeontx2: add build infra and device support X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" Reviewed-by: Liron Himi -----Original Message----- From: dev On Behalf Of guyk@marvell.com Sent: Tuesday, 1 September 2020 15:25 To: Jerin Jacob Kollanukkaran ; Nithin Kumar Dabilpuram= ; thomas@monjalon.net; mdr@ashroe.eu; nhorman@tux= driver.com; bruce.richardson@intel.com; anatoly.burakov@intel.com; john.mcn= amara@intel.com; marko.kovacevic@intel.com Cc: dev@dpdk.org; Smadar Fuks ; Dovrat Zifroni ; Guy Kaneti ; orika@mellanox.com Subject: [dpdk-dev] [PATCH 2/4] regex/octeontx2: add build infra and device= support From: Guy Kaneti Add meson based build infrastructure along with the OTX2 regexdev (REE) device functions. For regex rule compiler support build: meson configure -Dree_compiler_sdk=3D Signed-off-by: Guy Kaneti --- MAINTAINERS | 3 + config/common_base | 6 + drivers/regex/meson.build | 2 +- drivers/regex/octeontx2/meson.build | 53 + drivers/regex/octeontx2/otx2_regexdev.c | 1001 +++++++++++++++++ drivers/regex/octeontx2/otx2_regexdev.h | 109 ++ .../regex/octeontx2/otx2_regexdev_compiler.c | 229 ++++ .../regex/octeontx2/otx2_regexdev_compiler.h | 11 + .../regex/octeontx2/otx2_regexdev_hw_access.c | 167 +++ .../regex/octeontx2/otx2_regexdev_hw_access.h | 202 ++++ drivers/regex/octeontx2/otx2_regexdev_mbox.c | 401 +++++++ drivers/regex/octeontx2/otx2_regexdev_mbox.h | 38 + .../rte_pmd_octeontx2_regex_version.map | 3 + meson_options.txt | 2 + 14 files changed, 2226 insertions(+), 1 deletion(-) create mode 100644 drivers/regex/octeontx2/meson.build create mode 100644 drivers/regex/octeontx2/otx2_regexdev.c create mode 100644 drivers/regex/octeontx2/otx2_regexdev.h create mode 100644 drivers/regex/octeontx2/otx2_regexdev_compiler.c create mode 100644 drivers/regex/octeontx2/otx2_regexdev_compiler.h create mode 100644 drivers/regex/octeontx2/otx2_regexdev_hw_access.c create mode 100644 drivers/regex/octeontx2/otx2_regexdev_hw_access.h create mode 100644 drivers/regex/octeontx2/otx2_regexdev_mbox.c create mode 100644 drivers/regex/octeontx2/otx2_regexdev_mbox.h create mode 100644 drivers/regex/octeontx2/rte_pmd_octeontx2_regex_version= .map diff --git a/MAINTAINERS b/MAINTAINERS index ed163f5d5..ab8a9313c 100644 --- a/MAINTAINERS +++ b/MAINTAINERS @@ -1137,6 +1137,9 @@ F: drivers/regex/mlx5/ F: doc/guides/regexdevs/mlx5.rst F: doc/guides/regexdevs/features/mlx5.ini =20 +Marvell OCTEON TX2 regex +M: Guy Kaneti +F: drivers/regex/octeontx2/ =20 vDPA Drivers ------------ diff --git a/config/common_base b/config/common_base index fbf0ee70c..2cd687bf6 100644 --- a/config/common_base +++ b/config/common_base @@ -842,6 +842,12 @@ CONFIG_RTE_LIBRTE_PMD_OCTEONTX2_EP_RAWDEV=3Dy # CONFIG_RTE_LIBRTE_PMD_NTB_RAWDEV=3Dy =20 +# +# Compile PMD for Octeontx2 REE regex device +# +CONFIG_RTE_LIBRTE_PMD_OCTEONTX2_REGEX=3Dy +CONFIG_RTE_LIBRTE_PMD_OCTEONTX2_REGEX_COMPILER=3Dn + # # Compile librte_ring # diff --git a/drivers/regex/meson.build b/drivers/regex/meson.build index 8edeba3a0..79bb5d5df 100644 --- a/drivers/regex/meson.build +++ b/drivers/regex/meson.build @@ -1,7 +1,7 @@ # SPDX-License-Identifier: BSD-3-Clause # Copyright 2020 Mellanox Technologies, Ltd =20 -drivers =3D ['mlx5'] +drivers =3D ['mlx5', 'octeontx2'] std_deps =3D ['ethdev', 'kvargs'] # 'ethdev' also pulls in mbuf, net, eal = etc config_flag_fmt =3D 'RTE_LIBRTE_@0@_PMD' driver_name_fmt =3D 'rte_pmd_@0@' diff --git a/drivers/regex/octeontx2/meson.build b/drivers/regex/octeontx2/= meson.build new file mode 100644 index 000000000..c212e3d43 --- /dev/null +++ b/drivers/regex/octeontx2/meson.build @@ -0,0 +1,53 @@ +# SPDX-License-Identifier: BSD-3-Clause +# Copyright(C) 2020 Marvell International Ltd. +# + +if not is_linux + build =3D false + reason =3D 'only supported on Linux' +endif + +path =3D get_option('ree_compiler_sdk') +lib_dir =3D path + '/lib' +inc_dir =3D path + '/include' + +if dpdk_conf.has('RTE_LIBRTE_PMD_OCTEONTX2_REGEX_COMPILER') + lib =3D cc.find_library('librxp_compiler', dirs : [lib_dir], required: fa= lse) + if not lib.found() + build =3D false + reason =3D 'missing dependency, "librxp_compiler"' + else + ext_deps +=3D lib + ext_deps +=3D cc.find_library('libstdc++', required: true) + includes +=3D include_directories(inc_dir) + cflags +=3D ['-DREE_COMPILER_SDK'] + endif +endif + +sources =3D files('otx2_regexdev.c', + 'otx2_regexdev_hw_access.c', + 'otx2_regexdev_mbox.c', + 'otx2_regexdev_compiler.c' + ) + +extra_flags =3D [] +# This integrated controller runs only on a arm64 machine, remove 32bit wa= rnings +if not dpdk_conf.get('RTE_ARCH_64') + extra_flags +=3D ['-Wno-int-to-pointer-cast', '-Wno-pointer-to-int-cast'] +endif + +# for clang 32-bit compiles we need libatomic for 64-bit atomic ops +if cc.get_id() =3D=3D 'clang' and dpdk_conf.get('RTE_ARCH_64') =3D=3D fals= e + ext_deps +=3D cc.find_library('atomic') +endif + +foreach flag: extra_flags + if cc.has_argument(flag) + cflags +=3D flag + endif +endforeach + +name =3D 'octeontx2_regex' +deps +=3D ['bus_pci', 'common_octeontx2', 'regexdev'] + +includes +=3D include_directories('../../common/octeontx2') diff --git a/drivers/regex/octeontx2/otx2_regexdev.c b/drivers/regex/octeon= tx2/otx2_regexdev.c new file mode 100644 index 000000000..2aebd0138 --- /dev/null +++ b/drivers/regex/octeontx2/otx2_regexdev.c @@ -0,0 +1,1001 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright (C) 2020 Marvell International Ltd. + */ + +#include +#include + +#include +#include +#include +#include +#include + +/* REE common headers */ +#include "otx2_common.h" +#include "otx2_dev.h" +#include "otx2_regexdev.h" +#include "otx2_regexdev_compiler.h" +#include "otx2_regexdev_hw_access.h" +#include "otx2_regexdev_mbox.h" + + +/* HW matches are at offset 0x80 from RES_PTR_ADDR + * In op structure matches starts at W5 (0x28) + * There is a need to copy to 0x28 to 0x80 The matches that are at the tai= l + * Which are 88 B. Each match holds 8 B, so up to 11 matches can be copied + */ +#define REE_NUM_MATCHES_ALIGN 11 +/* The REE co-processor will write up to 254 job match structures + * (REE_MATCH_S) starting at address [RES_PTR_ADDR] + 0x80. + */ +#define REE_MATCH_OFFSET 0x80 + +#define REE_MAX_RULES_PER_GROUP 0xFFFF +#define REE_MAX_GROUPS 0xFFFF + +/* This is temporarily here */ +#define REE0_PF 19 +#define REE1_PF 20 + +#define REE_RULE_DB_VERSION 2 +#define REE_RULE_DB_REVISION 0 + +struct ree_rule_db_entry { + uint8_t type; + uint32_t addr; + uint64_t value; +}; + +struct ree_rule_db { + uint32_t version; + uint32_t revision; + uint32_t number_of_entries; + struct ree_rule_db_entry entries[]; +} __rte_packed; + +static void +qp_memzone_name_get(char *name, int size, int dev_id, int qp_id) +{ + snprintf(name, size, "otx2_ree_lf_mem_%u:%u", dev_id, qp_id); +} + +static struct otx2_ree_qp * +ree_qp_create(const struct rte_regexdev *dev, uint16_t qp_id) +{ + struct otx2_ree_data *data =3D dev->data->dev_private; + uint64_t pg_sz =3D sysconf(_SC_PAGESIZE); + struct otx2_ree_vf *vf =3D &data->vf; + const struct rte_memzone *lf_mem; + uint32_t len, iq_len, size_div2; + char name[RTE_MEMZONE_NAMESIZE]; + uint64_t used_len, iova; + struct otx2_ree_qp *qp; + uint8_t *va; + int ret; + + /* Allocate queue pair */ + qp =3D rte_zmalloc("OCTEON TX2 Regex PMD Queue Pair", sizeof(*qp), + OTX2_ALIGN); + if (qp =3D=3D NULL) { + otx2_err("Could not allocate queue pair"); + return NULL; + } + + iq_len =3D OTX2_REE_IQ_LEN; + + /* + * Queue size must be in units of 128B 2 * REE_INST_S (which is 64B), + * and a power of 2. + * effective queue size to software is (size - 1) * 128 + */ + size_div2 =3D iq_len >> 1; + + /* For pending queue */ + len =3D iq_len * RTE_ALIGN(sizeof(struct otx2_ree_rid), 8); + + /* So that instruction queues start as pg size aligned */ + len =3D RTE_ALIGN(len, pg_sz); + + /* For instruction queues */ + len +=3D OTX2_REE_IQ_LEN * sizeof(union otx2_ree_inst); + + /* Waste after instruction queues */ + len =3D RTE_ALIGN(len, pg_sz); + + qp_memzone_name_get(name, RTE_MEMZONE_NAMESIZE, dev->data->dev_id, + qp_id); + + lf_mem =3D rte_memzone_reserve_aligned(name, len, vf->otx2_dev.node, + RTE_MEMZONE_SIZE_HINT_ONLY | RTE_MEMZONE_256MB, + RTE_CACHE_LINE_SIZE); + if (lf_mem =3D=3D NULL) { + otx2_err("Could not allocate reserved memzone"); + goto qp_free; + } + + va =3D lf_mem->addr; + iova =3D lf_mem->iova; + + memset(va, 0, len); + + /* Initialize pending queue */ + qp->pend_q.rid_queue =3D (struct otx2_ree_rid *)va; + qp->pend_q.enq_tail =3D 0; + qp->pend_q.deq_head =3D 0; + qp->pend_q.pending_count =3D 0; + + used_len =3D iq_len * RTE_ALIGN(sizeof(struct otx2_ree_rid), 8); + used_len =3D RTE_ALIGN(used_len, pg_sz); + iova +=3D used_len; + + qp->iq_dma_addr =3D iova; + qp->id =3D qp_id; + qp->base =3D OTX2_REE_LF_BAR2(vf, qp_id); + qp->otx2_regexdev_jobid =3D 0; + qp->write_offset =3D 0; + + ret =3D otx2_ree_iq_enable(dev, qp, OTX2_REE_QUEUE_HI_PRIO, size_div2); + if (ret) { + otx2_err("Could not enable instruction queue"); + goto qp_free; + } + + return qp; + +qp_free: + rte_free(qp); + return NULL; +} + +static int +ree_qp_destroy(const struct rte_regexdev *dev, struct otx2_ree_qp *qp) +{ + const struct rte_memzone *lf_mem; + char name[RTE_MEMZONE_NAMESIZE]; + int ret; + + otx2_ree_iq_disable(qp); + + qp_memzone_name_get(name, RTE_MEMZONE_NAMESIZE, dev->data->dev_id, + qp->id); + + lf_mem =3D rte_memzone_lookup(name); + + ret =3D rte_memzone_free(lf_mem); + if (ret) + return ret; + + rte_free(qp); + + return 0; +} + +static int +ree_queue_pair_release(struct rte_regexdev *dev, uint16_t qp_id) +{ + struct otx2_ree_data *data =3D dev->data->dev_private; + struct otx2_ree_qp *qp =3D data->queue_pairs[qp_id]; + int ret; + + ree_func_trace("Queue=3D%d", qp_id); + + if (qp =3D=3D NULL) + return -EINVAL; + + ret =3D ree_qp_destroy(dev, qp); + if (ret) { + otx2_err("Could not destroy queue pair %d", qp_id); + return ret; + } + + data->queue_pairs[qp_id] =3D NULL; + + return 0; +} + +static struct rte_regexdev * +ree_dev_register(const char *name) +{ + struct rte_regexdev *dev; + + otx2_ree_dbg("Creating regexdev %s\n", name); + + /* allocate device structure */ + dev =3D rte_regexdev_register(name); + if (dev =3D=3D NULL) { + otx2_err("Failed to allocate regex device for %s", name); + return NULL; + } + + /* allocate private device structure */ + if (rte_eal_process_type() =3D=3D RTE_PROC_PRIMARY) { + dev->data->dev_private =3D + rte_zmalloc_socket("regexdev device private", + sizeof(struct otx2_ree_data), + RTE_CACHE_LINE_SIZE, + rte_socket_id()); + + if (dev->data->dev_private =3D=3D NULL) { + otx2_err("Cannot allocate memory for dev %s private data", + name); + + rte_regexdev_unregister(dev); + return NULL; + } + } + + return dev; +} + +static int +ree_dev_unregister(struct rte_regexdev *dev) +{ + otx2_ree_dbg("Closing regex device %s", dev->device->name); + + /* free regex device */ + rte_regexdev_unregister(dev); + + if (rte_eal_process_type() =3D=3D RTE_PROC_PRIMARY) + rte_free(dev->data->dev_private); + + return 0; +} + +static int +ree_dev_fini(struct rte_regexdev *dev) +{ + struct otx2_ree_data *data =3D dev->data->dev_private; + struct rte_pci_device *pci_dev; + int i, ret; + + ree_func_trace(); + + for (i =3D 0; i < data->nb_queue_pairs; i++) { + ret =3D ree_queue_pair_release(dev, i); + if (ret) + return ret; + } + + ret =3D otx2_ree_queues_detach(dev); + if (ret) + otx2_err("Could not detach queues"); + + /* TEMP : should be in lib */ + if (data->queue_pairs) + rte_free(data->queue_pairs); + if (data->rules) + rte_free(data->rules); + + pci_dev =3D container_of(dev->device, struct rte_pci_device, device); + otx2_dev_fini(pci_dev, &(data->vf.otx2_dev)); + + ret =3D ree_dev_unregister(dev); + if (ret) + otx2_err("Could not destroy PMD"); + + return ret; +} + +static inline int +ree_enqueue(struct otx2_ree_qp *qp, struct rte_regex_ops *op, + struct otx2_ree_pending_queue *pend_q) +{ + union otx2_ree_inst inst; + union otx2_ree_res *res; + uint32_t offset; + + if (unlikely(pend_q->pending_count >=3D OTX2_REE_DEFAULT_CMD_QLEN)) { + otx2_err("Pending count %" PRIu64 " is greater than Q size %d", + pend_q->pending_count, OTX2_REE_DEFAULT_CMD_QLEN); + return -EAGAIN; + } + if (unlikely(op->mbuf->data_len > OTX2_REE_MAX_PAYLOAD_SIZE || + op->mbuf->data_len =3D=3D 0)) { + otx2_err("Packet length %d is greater than MAX payload %d", + op->mbuf->data_len, OTX2_REE_MAX_PAYLOAD_SIZE); + return -EAGAIN; + } + + /* W 0 */ + inst.cn98xx.ooj =3D 1; + inst.cn98xx.dg =3D 0; + inst.cn98xx.doneint =3D 0; + /* W 1 */ + inst.cn98xx.inp_ptr_addr =3D rte_pktmbuf_mtod(op->mbuf, uint64_t); + /* W 2 */ + inst.cn98xx.inp_ptr_ctl =3D op->mbuf->data_len & 0x7FFF; + inst.cn98xx.inp_ptr_ctl =3D inst.cn98xx.inp_ptr_ctl << 32; + + /* W 3 */ + inst.cn98xx.res_ptr_addr =3D (uint64_t)op; + /* W 4 */ + inst.cn98xx.wq_ptr =3D 0; + /* W 5 */ + inst.cn98xx.ggrp =3D 0; + inst.cn98xx.tt =3D 0; + inst.cn98xx.tag =3D 0; + /* W 6 */ + inst.cn98xx.ree_job_length =3D op->mbuf->data_len & 0x7FFF; + if (op->req_flags & RTE_REGEX_OPS_REQ_STOP_ON_MATCH_F) + inst.cn98xx.ree_job_ctrl =3D (0x2 << 8); + else if (op->req_flags & RTE_REGEX_OPS_REQ_MATCH_HIGH_PRIORITY_F) + inst.cn98xx.ree_job_ctrl =3D (0x1 << 8); + else + inst.cn98xx.ree_job_ctrl =3D 0; + inst.cn98xx.ree_job_id =3D qp->otx2_regexdev_jobid; + /* W 7 */ + inst.cn98xx.ree_job_subset_id_0 =3D op->group_id0; + if (op->req_flags & RTE_REGEX_OPS_REQ_GROUP_ID1_VALID_F) + inst.cn98xx.ree_job_subset_id_1 =3D op->group_id1; + else + inst.cn98xx.ree_job_subset_id_1 =3D op->group_id0; + if (op->req_flags & RTE_REGEX_OPS_REQ_GROUP_ID2_VALID_F) + inst.cn98xx.ree_job_subset_id_2 =3D op->group_id2; + else + inst.cn98xx.ree_job_subset_id_2 =3D op->group_id0; + if (op->req_flags & RTE_REGEX_OPS_REQ_GROUP_ID3_VALID_F) + inst.cn98xx.ree_job_subset_id_3 =3D op->group_id3; + else + inst.cn98xx.ree_job_subset_id_3 =3D op->group_id0; + + /* Copy REE command to Q */ + offset =3D qp->write_offset * sizeof(inst); + memcpy((void *)(qp->iq_dma_addr + offset), &inst, sizeof(inst)); + + pend_q->rid_queue[pend_q->enq_tail].rid =3D (uintptr_t)op; + pend_q->rid_queue[pend_q->enq_tail].user_id =3D op->user_id; + + /* Mark result as not done */ + res =3D (union otx2_ree_res *)(op); + res->s.done =3D 0; + res->s.ree_err =3D 0; + + /* We will use soft queue length here to limit requests */ + REE_MOD_INC(pend_q->enq_tail, OTX2_REE_DEFAULT_CMD_QLEN); + pend_q->pending_count +=3D 1; + REE_MOD_INC(qp->otx2_regexdev_jobid, 0xFFFFFF); + REE_MOD_INC(qp->write_offset, OTX2_REE_IQ_LEN); + + return 0; +} + +static uint16_t +otx2_ree_enqueue_burst(struct rte_regexdev *dev, uint16_t qp_id, + struct rte_regex_ops **ops, uint16_t nb_ops) +{ + struct otx2_ree_data *data =3D dev->data->dev_private; + struct otx2_ree_qp *qp =3D data->queue_pairs[qp_id]; + struct otx2_ree_pending_queue *pend_q; + uint16_t nb_allowed, count =3D 0; + struct rte_regex_ops *op; + int ret; + + pend_q =3D &qp->pend_q; + + nb_allowed =3D OTX2_REE_DEFAULT_CMD_QLEN - pend_q->pending_count; + if (nb_ops > nb_allowed) + nb_ops =3D nb_allowed; + + for (count =3D 0; count < nb_ops; count++) { + op =3D ops[count]; + ret =3D ree_enqueue(qp, op, pend_q); + + if (unlikely(ret)) + break; + } + + /* + * Make sure all instructions are written before DOORBELL is activated + */ + rte_cio_wmb(); + + /* Update Doorbell */ + otx2_write64(count, qp->base + OTX2_REE_LF_DOORBELL); + + return count; +} + +static inline void +ree_dequeue_post_process(struct rte_regex_ops *ops) +{ + uint8_t ree_res_mcnt, ree_res_dmcnt; + int off =3D REE_MATCH_OFFSET; + struct ree_res_s_98 *res; + uint16_t ree_res_status; + uint64_t match; + + res =3D (struct ree_res_s_98 *)ops; + /* store res values on stack since ops and res + * are using the same memory + */ + ree_res_status =3D res->ree_res_status; + ree_res_mcnt =3D res->ree_res_mcnt; + ree_res_dmcnt =3D res->ree_res_dmcnt; + ops->rsp_flags =3D 0; + ops->nb_actual_matches =3D ree_res_dmcnt; + ops->nb_matches =3D ree_res_mcnt; + if (unlikely(res->ree_err)) { + ops->nb_actual_matches =3D 0; + ops->nb_matches =3D 0; + } + + if (unlikely(ree_res_status !=3D REE_TYPE_RESULT_DESC)) { + if (ree_res_status & OTX2_REE_STATUS_PMI_SOJ_BIT) + ops->rsp_flags |=3D RTE_REGEX_OPS_RSP_PMI_SOJ_F; + if (ree_res_status & OTX2_REE_STATUS_PMI_EOJ_BIT) + ops->rsp_flags |=3D RTE_REGEX_OPS_RSP_PMI_EOJ_F; + if (ree_res_status & OTX2_REE_STATUS_ML_CNT_DET_BIT) + ops->rsp_flags |=3D RTE_REGEX_OPS_RSP_MAX_SCAN_TIMEOUT_F; + if (ree_res_status & OTX2_REE_STATUS_MM_CNT_DET_BIT) + ops->rsp_flags |=3D RTE_REGEX_OPS_RSP_MAX_MATCH_F; + if (ree_res_status & OTX2_REE_STATUS_MP_CNT_DET_BIT) + ops->rsp_flags |=3D RTE_REGEX_OPS_RSP_MAX_PREFIX_F; + } + if (ops->nb_matches > 0) { + /* Move the matches to the correct offset */ + off =3D ((ops->nb_matches < REE_NUM_MATCHES_ALIGN) ? + ops->nb_matches : REE_NUM_MATCHES_ALIGN); + match =3D (uint64_t)ops + REE_MATCH_OFFSET; + match +=3D (ops->nb_matches - off) * + sizeof(union otx2_ree_match); + memcpy((void *)ops->matches, (void *)match, + off * sizeof(union otx2_ree_match)); + } +} + +static uint16_t +otx2_ree_dequeue_burst(struct rte_regexdev *dev, uint16_t qp_id, + struct rte_regex_ops **ops, uint16_t nb_ops) +{ + struct otx2_ree_data *data =3D dev->data->dev_private; + struct otx2_ree_qp *qp =3D data->queue_pairs[qp_id]; + struct otx2_ree_pending_queue *pend_q; + int i, nb_pending, nb_completed =3D 0; + volatile struct ree_res_s_98 *res; + struct otx2_ree_rid *rid; + + pend_q =3D &qp->pend_q; + + nb_pending =3D pend_q->pending_count; + + if (nb_ops > nb_pending) + nb_ops =3D nb_pending; + + for (i =3D 0; i < nb_ops; i++) { + rid =3D &pend_q->rid_queue[pend_q->deq_head]; + res =3D (volatile struct ree_res_s_98 *)(rid->rid); + + /* Check response header done bit if completed */ + if (unlikely(!res->done)) + break; + + ops[i] =3D (struct rte_regex_ops *)(rid->rid); + ops[i]->user_id =3D rid->user_id; + + REE_MOD_INC(pend_q->deq_head, OTX2_REE_DEFAULT_CMD_QLEN); + pend_q->pending_count -=3D 1; + } + + nb_completed =3D i; + + for (i =3D 0; i < nb_completed; i++) + ree_dequeue_post_process(ops[i]); + + return nb_completed; +} + +static int +otx2_ree_dev_info_get(struct rte_regexdev *dev, struct rte_regexdev_info *= info) +{ + struct otx2_ree_data *data =3D dev->data->dev_private; + struct otx2_ree_vf *vf =3D &data->vf; + + ree_func_trace(); + + if (info =3D=3D NULL) + return -EINVAL; + + info->driver_name =3D dev->device->driver->name; + info->dev =3D dev->device; + + info->max_queue_pairs =3D vf->max_queues; + info->max_matches =3D vf->max_matches; + info->max_payload_size =3D OTX2_REE_MAX_PAYLOAD_SIZE; + info->max_rules_per_group =3D data->max_rules_per_group; + info->max_groups =3D data->max_groups; + info->regexdev_capa =3D data->regexdev_capa; + info->rule_flags =3D data->rule_flags; + + return 0; +} + +static int +otx2_ree_dev_config(struct rte_regexdev *dev, + const struct rte_regexdev_config *cfg) +{ + struct otx2_ree_data *data =3D dev->data->dev_private; + struct otx2_ree_vf *vf =3D &data->vf; + const struct ree_rule_db *rule_db; + uint32_t rule_db_len; + int ret; + + ree_func_trace(); + + if (cfg->nb_queue_pairs > vf->max_queues) { + otx2_err("Invalid number of queue pairs requested"); + return -EINVAL; + } + + if (cfg->nb_max_matches !=3D vf->max_matches) { + otx2_err("Invalid number of max matches requested"); + return -EINVAL; + } + + if (cfg->dev_cfg_flags !=3D 0) { + otx2_err("Invalid device configuration flags requested"); + return -EINVAL; + } + + /* Unregister error interrupts */ + if (vf->err_intr_registered) + otx2_ree_err_intr_unregister(dev); + + /* Detach queues */ + if (vf->nb_queues) { + ret =3D otx2_ree_queues_detach(dev); + if (ret) { + otx2_err("Could not detach REE queues"); + return ret; + } + } + + /* TEMP : should be in lib */ + if (data->queue_pairs =3D=3D NULL) { /* first time configuration */ + data->queue_pairs =3D rte_zmalloc("regexdev->queue_pairs", + sizeof(data->queue_pairs[0]) * + cfg->nb_queue_pairs, RTE_CACHE_LINE_SIZE); + + if (data->queue_pairs =3D=3D NULL) { + data->nb_queue_pairs =3D 0; + otx2_err("Failed to get memory for qp meta data, nb_queues %u", + cfg->nb_queue_pairs); + return -ENOMEM; + } + } else { /* re-configure */ + uint16_t old_nb_queues =3D data->nb_queue_pairs; + void **qp; + unsigned int i; + + qp =3D data->queue_pairs; + + for (i =3D cfg->nb_queue_pairs; i < old_nb_queues; i++) { + ret =3D ree_queue_pair_release(dev, i); + if (ret < 0) + return ret; + } + + qp =3D rte_realloc(qp, sizeof(qp[0]) * cfg->nb_queue_pairs, + RTE_CACHE_LINE_SIZE); + if (qp =3D=3D NULL) { + otx2_err("Failed to realloc qp meta data, nb_queues %u", + cfg->nb_queue_pairs); + return -ENOMEM; + } + + if (cfg->nb_queue_pairs > old_nb_queues) { + uint16_t new_qs =3D cfg->nb_queue_pairs - old_nb_queues; + memset(qp + old_nb_queues, 0, sizeof(qp[0]) * new_qs); + } + + data->queue_pairs =3D qp; + } + data->nb_queue_pairs =3D cfg->nb_queue_pairs; + + /* Attach queues */ + otx2_ree_dbg("Attach %d queues", cfg->nb_queue_pairs); + ret =3D otx2_ree_queues_attach(dev, cfg->nb_queue_pairs); + if (ret) { + otx2_err("Could not attach queues"); + return -ENODEV; + } + + ret =3D otx2_ree_msix_offsets_get(dev); + if (ret) { + otx2_err("Could not get MSI-X offsets"); + goto queues_detach; + } + + if (cfg->rule_db && cfg->rule_db_len) { + otx2_ree_dbg("rule_db length %d", cfg->rule_db_len); + rule_db =3D (const struct ree_rule_db *)cfg->rule_db; + rule_db_len =3D rule_db->number_of_entries * + sizeof(struct ree_rule_db_entry); + otx2_ree_dbg("rule_db number of entries %d", + rule_db->number_of_entries); + if (rule_db_len > cfg->rule_db_len) { + otx2_err("Could not program rule db"); + ret =3D -EINVAL; + goto queues_detach; + } + ret =3D otx2_ree_rule_db_prog(dev, (const char *)rule_db->entries, + rule_db_len, NULL, OTX2_REE_NON_INC_PROG); + if (ret) { + otx2_err("Could not program rule db"); + goto queues_detach; + } + } + + dev->enqueue =3D otx2_ree_enqueue_burst; + dev->dequeue =3D otx2_ree_dequeue_burst; + + rte_mb(); + return 0; + +queues_detach: + otx2_ree_queues_detach(dev); + return ret; +} + +static int +otx2_ree_stop(struct rte_regexdev *dev) +{ + RTE_SET_USED(dev); + + ree_func_trace(); + return 0; +} + +static int +otx2_ree_start(struct rte_regexdev *dev) +{ + uint32_t rule_db_len =3D 0; + int ret; + + ree_func_trace(); + + ret =3D otx2_ree_rule_db_len_get(dev, &rule_db_len, NULL); + if (ret) + return ret; + if (rule_db_len =3D=3D 0) { + otx2_err("Rule db not programmed"); + return -EFAULT; + } + + return 0; +} + +static int +otx2_ree_close(struct rte_regexdev *dev) +{ + return ree_dev_fini(dev); +} + +static int +otx2_ree_queue_pair_setup(struct rte_regexdev *dev, uint16_t qp_id, + const struct rte_regexdev_qp_conf *qp_conf) +{ + struct otx2_ree_data *data =3D dev->data->dev_private; + struct otx2_ree_qp *qp; + + ree_func_trace("Queue=3D%d", qp_id); + + if (data->queue_pairs[qp_id] !=3D NULL) + ree_queue_pair_release(dev, qp_id); + + if (qp_conf->nb_desc > OTX2_REE_DEFAULT_CMD_QLEN) { + otx2_err("Could not setup queue pair for %u descriptors", + qp_conf->nb_desc); + return -EINVAL; + } + if (qp_conf->qp_conf_flags !=3D 0) { + otx2_err("Could not setup queue pair with configuration flags 0x%x", + qp_conf->qp_conf_flags); + return -EINVAL; + } + + qp =3D ree_qp_create(dev, qp_id); + if (qp =3D=3D NULL) { + otx2_err("Could not create queue pair %d", qp_id); + return -ENOMEM; + } + qp->cb =3D qp_conf->cb; + data->queue_pairs[qp_id] =3D qp; + + return 0; +} + +static int +otx2_ree_rule_db_compile_activate(struct rte_regexdev *dev) +{ + return otx2_ree_rule_db_compile_prog(dev); +} + +static int +otx2_ree_rule_db_update(struct rte_regexdev *dev, + const struct rte_regexdev_rule *rules, uint16_t nb_rules) +{ + struct otx2_ree_data *data =3D dev->data->dev_private; + struct rte_regexdev_rule *old_ptr; + uint32_t i, sum_nb_rules; + + ree_func_trace("nb_rules=3D%d", nb_rules); + + for (i =3D 0; i < nb_rules; i++) { + if (rules[i].op =3D=3D RTE_REGEX_RULE_OP_REMOVE) + break; + if (rules[i].group_id >=3D data->max_groups) + break; + if (rules[i].rule_id >=3D data->max_rules_per_group) + break; + /* logical implication + * p q p -> q + * 0 0 1 + * 0 1 1 + * 1 0 0 + * 1 1 1 + */ + if ((~(rules[i].rule_flags) | data->rule_flags) =3D=3D 0) + break; + } + nb_rules =3D i; + + if (data->nb_rules =3D=3D 0) { + + data->rules =3D rte_malloc("rte_regexdev_rules", + nb_rules*sizeof(struct rte_regexdev_rule), 0); + if (data->rules =3D=3D NULL) + return -ENOMEM; + + memcpy(data->rules, rules, + nb_rules*sizeof(struct rte_regexdev_rule)); + data->nb_rules =3D nb_rules; + } else { + + old_ptr =3D data->rules; + sum_nb_rules =3D data->nb_rules + nb_rules; + data->rules =3D rte_realloc(data->rules, + sum_nb_rules * sizeof(struct rte_regexdev_rule), + 0); + if (data->rules =3D=3D NULL) { + data->rules =3D old_ptr; + return -ENOMEM; + } + memcpy(&data->rules[data->nb_rules], rules, + nb_rules*sizeof(struct rte_regexdev_rule)); + data->nb_rules =3D sum_nb_rules; + } + return nb_rules; +} + +static int +otx2_ree_rule_db_import(struct rte_regexdev *dev, const char *rule_db, + uint32_t rule_db_len) +{ + + const struct ree_rule_db *ree_rule_db; + uint32_t ree_rule_db_len; + int ret; + + ree_func_trace("rule_db_len=3D%d", rule_db_len); + + ree_rule_db =3D (const struct ree_rule_db *)rule_db; + ree_rule_db_len =3D ree_rule_db->number_of_entries * + sizeof(struct ree_rule_db_entry); + if (ree_rule_db_len > rule_db_len) { + otx2_err("Could not program rule db"); + return -EINVAL; + } + ret =3D otx2_ree_rule_db_prog(dev, (const char *)ree_rule_db->entries, + ree_rule_db_len, NULL, OTX2_REE_NON_INC_PROG); + if (ret) { + otx2_err("Could not program rule db"); + return -ENOSPC; + } + return 0; +} + +static int +otx2_ree_rule_db_export(struct rte_regexdev *dev, char *rule_db) +{ + struct ree_rule_db *ree_rule_db; + uint32_t rule_dbi_len; + uint32_t rule_db_len; + int ret; + + ree_func_trace(); + + ret =3D otx2_ree_rule_db_len_get(dev, &rule_db_len, &rule_dbi_len); + if (ret) + return ret; + + if (rule_db =3D=3D NULL) { + rule_db_len +=3D sizeof(struct ree_rule_db); + return rule_db_len; + } + + ree_rule_db =3D (struct ree_rule_db *)rule_db; + ret =3D otx2_ree_rule_db_get(dev, (char *)ree_rule_db->entries, + rule_db_len, NULL, 0); + if (ret) { + otx2_err("Could not export rule db"); + return -EFAULT; + } + ree_rule_db->number_of_entries =3D + rule_db_len/sizeof(struct ree_rule_db_entry); + ree_rule_db->revision =3D REE_RULE_DB_REVISION; + ree_rule_db->version =3D REE_RULE_DB_VERSION; + + return 0; +} + +static int +ree_get_blkaddr(struct otx2_dev *dev) +{ + int pf; + + pf =3D otx2_get_pf(dev->pf_func); + if (pf =3D=3D REE0_PF) + return RVU_BLOCK_ADDR_REE0; + else if (pf =3D=3D REE1_PF) + return RVU_BLOCK_ADDR_REE1; + else + return 0; +} + +static struct rte_regexdev_ops otx2_ree_ops =3D { + .dev_info_get =3D otx2_ree_dev_info_get, + .dev_configure =3D otx2_ree_dev_config, + .dev_qp_setup =3D otx2_ree_queue_pair_setup, + .dev_start =3D otx2_ree_start, + .dev_stop =3D otx2_ree_stop, + .dev_close =3D otx2_ree_close, + .dev_attr_get =3D NULL, + .dev_attr_set =3D NULL, + .dev_rule_db_update =3D otx2_ree_rule_db_update, + .dev_rule_db_compile_activate =3D + otx2_ree_rule_db_compile_activate, + .dev_db_import =3D otx2_ree_rule_db_import, + .dev_db_export =3D otx2_ree_rule_db_export, + .dev_xstats_names_get =3D NULL, + .dev_xstats_get =3D NULL, + .dev_xstats_by_name_get =3D NULL, + .dev_xstats_reset =3D NULL, + .dev_selftest =3D NULL, + .dev_dump =3D NULL, +}; + +static int +otx2_ree_pci_probe(struct rte_pci_driver *pci_drv __rte_unused, + struct rte_pci_device *pci_dev) +{ + char name[RTE_REGEXDEV_NAME_MAX_LEN]; + struct otx2_ree_data *data; + struct otx2_dev *otx2_dev; + struct rte_regexdev *dev; + uint8_t max_matches =3D 0; + struct otx2_ree_vf *vf; + uint16_t nb_queues =3D 0; + int ret; + + rte_pci_device_name(&pci_dev->addr, name, sizeof(name)); + + dev =3D ree_dev_register(name); + if (dev =3D=3D NULL) { + ret =3D -ENODEV; + goto exit; + } + + dev->dev_ops =3D &otx2_ree_ops; + dev->device =3D &pci_dev->device; + + /* Get private data space allocated */ + data =3D dev->data->dev_private; + vf =3D &data->vf; + + otx2_dev =3D &vf->otx2_dev; + + /* Initialize the base otx2_dev object */ + ret =3D otx2_dev_init(pci_dev, otx2_dev); + if (ret) { + otx2_err("Could not initialize otx2_dev"); + goto dev_unregister; + } + /* Get REE block address */ + vf->block_address =3D ree_get_blkaddr(otx2_dev); + if (!vf->block_address) { + otx2_err("Could not determine block PF number"); + goto otx2_dev_fini; + } + + /* Get number of queues available on the device */ + ret =3D otx2_ree_available_queues_get(dev, &nb_queues); + if (ret) { + otx2_err("Could not determine the number of queues available"); + goto otx2_dev_fini; + } + + /* Don't exceed the limits set per VF */ + nb_queues =3D RTE_MIN(nb_queues, OTX2_REE_MAX_QUEUES_PER_VF); + + if (nb_queues =3D=3D 0) { + otx2_err("No free queues available on the device"); + goto otx2_dev_fini; + } + + vf->max_queues =3D nb_queues; + + otx2_ree_dbg("Max queues supported by device: %d", vf->max_queues); + + /* Get number of maximum matches supported on the device */ + ret =3D otx2_ree_max_matches_get(dev, &max_matches); + if (ret) { + otx2_err("Could not determine the maximum matches supported"); + goto otx2_dev_fini; + } + /* Don't exceed the limits set per VF */ + max_matches =3D RTE_MIN(max_matches, OTX2_REE_MAX_MATCHES_PER_VF); + if (max_matches =3D=3D 0) { + otx2_err("Could not determine the maximum matches supported"); + goto otx2_dev_fini; + } + + vf->max_matches =3D max_matches; + + otx2_ree_dbg("Max matches supported by device: %d", vf->max_matches); + data->rule_flags =3D RTE_REGEX_PCRE_RULE_ALLOW_EMPTY_F | + RTE_REGEX_PCRE_RULE_ANCHORED_F; + data->regexdev_capa =3D 0; + data->max_groups =3D REE_MAX_GROUPS; + data->max_rules_per_group =3D REE_MAX_RULES_PER_GROUP; + data->nb_rules =3D 0; + + dev->state =3D RTE_REGEXDEV_READY; + return 0; + +otx2_dev_fini: + otx2_dev_fini(pci_dev, otx2_dev); +dev_unregister: + ree_dev_unregister(dev); +exit: + otx2_err("Could not create device (vendor_id: 0x%x device_id: 0x%x)", + pci_dev->id.vendor_id, pci_dev->id.device_id); + return ret; +} + +static int +otx2_ree_pci_remove(struct rte_pci_device *pci_dev) +{ + char name[RTE_REGEXDEV_NAME_MAX_LEN]; + struct rte_regexdev *dev =3D NULL; + + if (pci_dev =3D=3D NULL) + return -EINVAL; + + rte_pci_device_name(&pci_dev->addr, name, sizeof(name)); + + dev =3D rte_regexdev_get_device_by_name(name); + + if (dev =3D=3D NULL) + return -ENODEV; + + return ree_dev_fini(dev); +} + +static struct rte_pci_id pci_id_ree_table[] =3D { + { + RTE_PCI_DEVICE(PCI_VENDOR_ID_CAVIUM, + PCI_DEVID_OCTEONTX2_RVU_REE_PF) + }, +}; + +static struct rte_pci_driver otx2_regexdev_pmd =3D { + .id_table =3D pci_id_ree_table, + .drv_flags =3D RTE_PCI_DRV_NEED_MAPPING, + .probe =3D otx2_ree_pci_probe, + .remove =3D otx2_ree_pci_remove, +}; + + +RTE_PMD_REGISTER_PCI(REGEXDEV_NAME_OCTEONTX2_PMD, otx2_regexdev_pmd); +RTE_PMD_REGISTER_PCI_TABLE(REGEXDEV_NAME_OCTEONTX2_PMD, pci_id_ree_table); diff --git a/drivers/regex/octeontx2/otx2_regexdev.h b/drivers/regex/octeon= tx2/otx2_regexdev.h new file mode 100644 index 000000000..d710535f5 --- /dev/null +++ b/drivers/regex/octeontx2/otx2_regexdev.h @@ -0,0 +1,109 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright (C) 2020 Marvell International Ltd. + */ + +#ifndef _OTX2_REGEXDEV_H_ +#define _OTX2_REGEXDEV_H_ + +#include +#include + +#include "otx2_dev.h" + +#define ree_func_trace otx2_ree_dbg + +/* Marvell OCTEON TX2 Regex PMD device name */ +#define REGEXDEV_NAME_OCTEONTX2_PMD regex_octeontx2 + +#define OTX2_REE_MAX_LFS 36 +#define OTX2_REE_MAX_QUEUES_PER_VF 36 +#define OTX2_REE_MAX_MATCHES_PER_VF 254 + +#define OTX2_REE_MAX_PAYLOAD_SIZE (1 << 14) + +#define OTX2_REE_NON_INC_PROG 0 +#define OTX2_REE_INC_PROG 1 + +#define REE_MOD_INC(i, l) ((i) =3D=3D (l - 1) ? (i) =3D 0 : (i)++) + + +/** + * Device vf data + */ +struct otx2_ree_vf { + struct otx2_dev otx2_dev; + /**< Base class */ + uint16_t max_queues; + /**< Max queues supported */ + uint8_t nb_queues; + /**< Number of regex queues attached */ + uint16_t max_matches; + /**< Max matches supported*/ + uint16_t lf_msixoff[OTX2_REE_MAX_LFS]; + /**< MSI-X offsets */ + uint8_t block_address; + /**< REE Block Address */ + uint8_t err_intr_registered:1; + /**< Are error interrupts registered? */ +}; + +/** + * Device private data + */ +struct otx2_ree_data { + uint32_t regexdev_capa; + uint64_t rule_flags; + /**< Feature flags exposes HW/SW features for the given device */ + uint16_t max_rules_per_group; + /**< Maximum rules supported per subset by this device */ + uint16_t max_groups; + /**< Maximum subset supported by this device */ + void **queue_pairs; + /**< Array of pointers to queue pairs. */ + uint16_t nb_queue_pairs; + /**< Number of device queue pairs. */ + struct otx2_ree_vf vf; + /**< vf data */ + struct rte_regexdev_rule *rules; + /**< rules to be compiled */ + uint16_t nb_rules; + /**< number of rules */ +} __rte_cache_aligned; + +struct otx2_ree_rid { + uintptr_t rid; + /** Request id of a ree operation */ + uint64_t user_id; + /* Client data */ + /**< IOVA address of the pattern to be matched. */ +}; + +struct otx2_ree_pending_queue { + uint64_t pending_count; + /** Pending requests count */ + struct otx2_ree_rid *rid_queue; + /** Array of pending requests */ + uint16_t enq_tail; + /** Tail of queue to be used for enqueue */ + uint16_t deq_head; + /** Head of queue to be used for dequeue */ +}; + +struct otx2_ree_qp { + uint32_t id; + /**< Queue pair id */ + uintptr_t base; + /**< Base address where BAR is mapped */ + struct otx2_ree_pending_queue pend_q; + /**< Pending queue */ + rte_iova_t iq_dma_addr; + /**< Instruction queue address */ + uint32_t otx2_regexdev_jobid; + /**< Job ID */ + uint32_t write_offset; + /**< write offset */ + regexdev_stop_flush_t cb; + /**< Callback function called during rte_regex_dev_stop()*/ +}; + +#endif /* _OTX2_REGEXDEV_H_ */ diff --git a/drivers/regex/octeontx2/otx2_regexdev_compiler.c b/drivers/reg= ex/octeontx2/otx2_regexdev_compiler.c new file mode 100644 index 000000000..785459f74 --- /dev/null +++ b/drivers/regex/octeontx2/otx2_regexdev_compiler.c @@ -0,0 +1,229 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright (C) 2020 Marvell International Ltd. + */ + +#include +#include + +#include "otx2_regexdev.h" +#include "otx2_regexdev_compiler.h" +#include "otx2_regexdev_mbox.h" + +#ifdef REE_COMPILER_SDK +#include + +static int +ree_rule_db_compile(const struct rte_regexdev_rule *rules, + uint16_t nb_rules, struct rxp_rof **rof, struct rxp_rof **rofi, + struct rxp_rof *rof_for_incremental_compile, + struct rxp_rof *rofi_for_incremental_compile) +{ + /*INPUT*/ + struct rxp_prefix_selection_control_list *prefix_selection_control_list + =3D NULL; + struct rxp_blacklist_data_sample *blacklist_sample_data =3D NULL; + struct rxp_rule_ids_to_remove *rule_ids_to_remove =3D NULL; + struct rxp_roff *roff_for_incremental_compile =3D NULL; + + /*OPTIONS - setting default values*/ + enum rxp_virtual_prefix_mode virtual_prefix_mode =3D + RXP_VIRTUAL_PREFIX_MODE_0; + enum rxp_prefix_capacity prefix_capacity =3D RXP_PREFIX_CAPACITY_32K; + /**< rxp_global_regex_options_flags*/ + enum rxp_compiler_objective objective =3D RXP_COMPILER_OBJECTIVE_5; + enum rxp_tpe_data_width tpe_data_width =3D RXP_TPE_DATA_WIDTH_4; + uint32_t compiler_options =3D RXP_COMPILER_OPTIONS_FORCE; + /**< rxp_compiler_options_flags*/ + enum rxp_verbose_level verbose =3D RXP_VERBOSE_LEVEL_3; + enum rxp_version set_rxp_version =3D RXP_VERSION_V5_8; + uint32_t compiler_output_flags =3D 0; + /**< rxp_compiler_output_flags*/ + uint32_t global_regex_options =3D 0; + /**< rxp_global_regex_options_flags*/ + float set_auto_blacklist =3D 0; + uint32_t max_rep_max =3D 65535; + uint32_t divide_ruleset =3D 1; + struct rxp_ruleset ruleset; + float ptpb_threshold =3D 0; + uint32_t set_max =3D 0; + uint32_t threads =3D 1; + + /*OUTPUT*/ + struct rxp_rule_direction_analysis *rule_direction_analysis =3D NULL; + struct rxp_compilation_statistics *compilation_statistics =3D NULL; + struct rxp_prefix_selection_control_list *generated_pscl =3D NULL; + struct rxp_uncompiled_rules_log *uncompiled_rules_log =3D NULL; + struct rxp_critical_rules_rank *critical_rules_rank =3D NULL; + struct rxp_compiled_rules_log *compiled_rules_log =3D NULL; + struct rxp_roff *roff =3D NULL; + + uint16_t i; + int ret; + + ruleset.number_of_entries =3D nb_rules; + ruleset.rules =3D rte_malloc("rxp_rule_entry", + nb_rules*sizeof(struct rxp_rule_entry), 0); + + if (ruleset.rules =3D=3D NULL) { + otx2_err("Could not allocate memory for rule compilation\n"); + return -EFAULT; + } + if (rof_for_incremental_compile) + compiler_options |=3D RXP_COMPILER_OPTIONS_INCREMENTAL; + if (rofi_for_incremental_compile) + compiler_options |=3D RXP_COMPILER_OPTIONS_CHECKSUM; + + for (i =3D 0; i < nb_rules; i++) { + ruleset.rules[i].number_of_prefix_entries =3D 0; + ruleset.rules[i].prefix =3D NULL; + ruleset.rules[i].rule =3D rules[i].pcre_rule; + ruleset.rules[i].rule_id =3D rules[i].rule_id; + ruleset.rules[i].subset_id =3D rules[i].group_id; + ruleset.rules[i].rule_direction_type =3D + RXP_RULE_DIRECTION_TYPE_NONE; + } + + ret =3D rxp_compile_advanced( + /*INPUT*/ + &ruleset, + prefix_selection_control_list, + rof_for_incremental_compile, + roff_for_incremental_compile, + rofi_for_incremental_compile, + rule_ids_to_remove, + blacklist_sample_data, + + /*OPTIONS*/ + compiler_options, + prefix_capacity, + global_regex_options, + set_auto_blacklist, + set_max, + objective, + ptpb_threshold, + max_rep_max, + threads, + set_rxp_version, + verbose, + tpe_data_width, + virtual_prefix_mode, + compiler_output_flags, + divide_ruleset, + + /*OUTPUT*/ + &compilation_statistics, + &compiled_rules_log, + &critical_rules_rank, + &rule_direction_analysis, + &uncompiled_rules_log, + rof, + &roff, + rofi, + &generated_pscl); + rte_free(ruleset.rules); + + return ret; +} + +int +otx2_ree_rule_db_compile_prog(struct rte_regexdev *dev) +{ + struct otx2_ree_data *data =3D dev->data->dev_private; + char compiler_version[] =3D "20.5.2.eda0fa2"; + char timestamp[] =3D "19700101_000001"; + uint32_t rule_db_len, rule_dbi_len; + struct rxp_rof *rofi_inc_p =3D NULL; + struct rxp_rof_entry rule_dbi[6]; + char *rofi_rof_entries =3D NULL; + struct rxp_rof *rofi =3D NULL; + struct rxp_rof *rof =3D NULL; + struct rxp_rof rofi_inc; + struct rxp_rof rof_inc; + char *rule_db =3D NULL; + int ret; + + ree_func_trace(); + + ret =3D otx2_ree_rule_db_len_get(dev, &rule_db_len, &rule_dbi_len); + if (ret !=3D 0) { + otx2_err("Could not get rule db length"); + return ret; + } + + if (rule_db_len > 0) { + otx2_ree_dbg("Incremental compile, rule db len %d rule dbi len %d", + rule_db_len, rule_dbi_len); + rule_db =3D rte_malloc("ree_rule_db", rule_db_len, 0); + if (!rule_db) { + otx2_err("Could not allocate memory for rule db"); + return -EFAULT; + } + + ret =3D otx2_ree_rule_db_get(dev, rule_db, rule_db_len, + (char *)rule_dbi, rule_dbi_len); + if (ret) { + otx2_err("Could not read rule db"); + rte_free(rule_db); + return -EFAULT; + } + rof_inc.rof_revision =3D 0; + rof_inc.rof_version =3D 2; + rof_inc.rof_entries =3D (struct rxp_rof_entry *)rule_db; + rof_inc.rxp_compiler_version =3D compiler_version; + rof_inc.timestamp =3D timestamp; + rof_inc.number_of_entries =3D + (rule_db_len/sizeof(struct rxp_rof_entry)); + + if (rule_dbi_len > 0) { + /* incremental compilation not the first time */ + rofi_inc.rof_revision =3D 0; + rofi_inc.rof_version =3D 2; + rofi_inc.rof_entries =3D rule_dbi; + rofi_inc.rxp_compiler_version =3D compiler_version; + rofi_inc.timestamp =3D timestamp; + rofi_inc.number_of_entries =3D + (rule_dbi_len/sizeof(struct rxp_rof_entry)); + rofi_inc_p =3D &rofi_inc; + } + ret =3D ree_rule_db_compile(data->rules, data->nb_rules, &rof, + &rofi, &rof_inc, rofi_inc_p); + if (rofi->number_of_entries =3D=3D 0) { + otx2_ree_dbg("No change to rule db"); + ret =3D 0; + goto free_structs; + } + rule_dbi_len =3D rofi->number_of_entries * + sizeof(struct rxp_rof_entry); + rofi_rof_entries =3D (char *)rofi->rof_entries; + } else { + /* full compilation */ + ret =3D ree_rule_db_compile(data->rules, data->nb_rules, &rof, + &rofi, NULL, NULL); + } + if (ret !=3D 0) { + otx2_err("Could not compile rule db"); + goto free_structs; + } + rule_db_len =3D rof->number_of_entries * sizeof(struct rxp_rof_entry); + ret =3D otx2_ree_rule_db_prog(dev, (char *)rof->rof_entries, rule_db_len, + rofi_rof_entries, rule_dbi_len); + if (ret) + otx2_err("Could not program rule db"); + +free_structs: + rxp_free_structs(NULL, NULL, NULL, NULL, NULL, &rof, NULL, &rofi, NULL, + 1); + + if (rule_db) + rte_free(rule_db); + + return ret; +} +#else +int +otx2_ree_rule_db_compile_prog(struct rte_regexdev *dev) +{ + RTE_SET_USED(dev); + return -ENOTSUP; +} +#endif diff --git a/drivers/regex/octeontx2/otx2_regexdev_compiler.h b/drivers/reg= ex/octeontx2/otx2_regexdev_compiler.h new file mode 100644 index 000000000..8d2625bf7 --- /dev/null +++ b/drivers/regex/octeontx2/otx2_regexdev_compiler.h @@ -0,0 +1,11 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright (C) 2020 Marvell International Ltd. + */ + +#ifndef _OTX2_REGEXDEV_COMPILER_H_ +#define _OTX2_REGEXDEV_COMPILER_H_ + +int +otx2_ree_rule_db_compile_prog(struct rte_regexdev *dev); + +#endif /* _OTX2_REGEXDEV_COMPILER_H_ */ diff --git a/drivers/regex/octeontx2/otx2_regexdev_hw_access.c b/drivers/re= gex/octeontx2/otx2_regexdev_hw_access.c new file mode 100644 index 000000000..620d5c912 --- /dev/null +++ b/drivers/regex/octeontx2/otx2_regexdev_hw_access.c @@ -0,0 +1,167 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright (C) 2020 Marvell International Ltd. + */ + +#include "otx2_common.h" +#include "otx2_dev.h" +#include "otx2_regexdev_hw_access.h" +#include "otx2_regexdev_mbox.h" + +static void +ree_lf_err_intr_handler(void *param) +{ + uintptr_t base =3D (uintptr_t)param; + uint8_t lf_id; + uint64_t intr; + + lf_id =3D (base >> 12) & 0xFF; + + intr =3D otx2_read64(base + OTX2_REE_LF_MISC_INT); + if (intr =3D=3D 0) + return; + + otx2_ree_dbg("LF %d MISC_INT: 0x%" PRIx64 "", lf_id, intr); + + /* Clear interrupt */ + otx2_write64(intr, base + OTX2_REE_LF_MISC_INT); +} + +static void +ree_lf_err_intr_unregister(const struct rte_regexdev *dev, uint16_t msix_o= ff, + uintptr_t base) +{ + struct rte_pci_device *pci_dev =3D RTE_DEV_TO_PCI(dev->device); + struct rte_intr_handle *handle =3D &pci_dev->intr_handle; + + /* Disable error interrupts */ + otx2_write64(~0ull, base + OTX2_REE_LF_MISC_INT_ENA_W1C); + + otx2_unregister_irq(handle, ree_lf_err_intr_handler, (void *)base, + msix_off); +} + +void +otx2_ree_err_intr_unregister(const struct rte_regexdev *dev) +{ + struct otx2_ree_data *data =3D dev->data->dev_private; + struct otx2_ree_vf *vf =3D &data->vf; + uintptr_t base; + uint32_t i; + + for (i =3D 0; i < vf->nb_queues; i++) { + base =3D OTX2_REE_LF_BAR2(vf, i); + ree_lf_err_intr_unregister(dev, vf->lf_msixoff[i], base); + } + + vf->err_intr_registered =3D 0; +} + +static int +ree_lf_err_intr_register(const struct rte_regexdev *dev, uint16_t msix_off= , + uintptr_t base) +{ + struct rte_pci_device *pci_dev =3D RTE_DEV_TO_PCI(dev->device); + struct rte_intr_handle *handle =3D &pci_dev->intr_handle; + int ret; + + /* Disable error interrupts */ + otx2_write64(~0ull, base + OTX2_REE_LF_MISC_INT_ENA_W1C); + + /* Register error interrupt handler */ + ret =3D otx2_register_irq(handle, ree_lf_err_intr_handler, (void *)base, + msix_off); + if (ret) + return ret; + + /* Enable error interrupts */ + otx2_write64(~0ull, base + OTX2_REE_LF_MISC_INT_ENA_W1S); + + return 0; +} + +int +otx2_ree_err_intr_register(const struct rte_regexdev *dev) +{ + struct otx2_ree_data *data =3D dev->data->dev_private; + struct otx2_ree_vf *vf =3D &data->vf; + uint32_t i, j, ret; + uintptr_t base; + + for (i =3D 0; i < vf->nb_queues; i++) { + if (vf->lf_msixoff[i] =3D=3D MSIX_VECTOR_INVALID) { + otx2_err("Invalid REE LF MSI-X offset: 0x%x", + vf->lf_msixoff[i]); + return -EINVAL; + } + } + + for (i =3D 0; i < vf->nb_queues; i++) { + base =3D OTX2_REE_LF_BAR2(vf, i); + ret =3D ree_lf_err_intr_register(dev, vf->lf_msixoff[i], base); + if (ret) + goto intr_unregister; + } + + vf->err_intr_registered =3D 1; + return 0; + +intr_unregister: + /* Unregister the ones already registered */ + for (j =3D 0; j < i; j++) { + base =3D OTX2_REE_LF_BAR2(vf, j); + ree_lf_err_intr_unregister(dev, vf->lf_msixoff[j], base); + } + return ret; +} + +int +otx2_ree_iq_enable(const struct rte_regexdev *dev, const struct otx2_ree_q= p *qp, + uint8_t pri, uint32_t size_div2) +{ + union otx2_ree_lf_sbuf_addr base; + union otx2_ree_lf_ena lf_ena; + + /* Set instruction queue size and priority */ + otx2_ree_config_lf(dev, qp->id, pri, size_div2); + + /* Set instruction queue base address */ + /* Should be written after SBUF_CTL and before LF_ENA */ + + base.u =3D otx2_read64(qp->base + OTX2_REE_LF_SBUF_ADDR); + base.s.ptr =3D qp->iq_dma_addr >> 7; + otx2_write64(base.u, qp->base + OTX2_REE_LF_SBUF_ADDR); + + /* Enable instruction queue */ + + lf_ena.u =3D otx2_read64(qp->base + OTX2_REE_LF_ENA); + lf_ena.s.ena =3D 1; + otx2_write64(lf_ena.u, qp->base + OTX2_REE_LF_ENA); + + return 0; +} + +void +otx2_ree_iq_disable(struct otx2_ree_qp *qp) +{ + union otx2_ree_lf_ena lf_ena; + + /* Stop instruction execution */ + lf_ena.u =3D otx2_read64(qp->base + OTX2_REE_LF_ENA); + lf_ena.s.ena =3D 0x0; + otx2_write64(lf_ena.u, qp->base + OTX2_REE_LF_ENA); +} + +int +otx2_ree_max_matches_get(const struct rte_regexdev *dev, uint8_t *max_matc= hes) +{ + union otx2_ree_af_reexm_max_match reexm_max_match; + int ret; + + ret =3D otx2_ree_af_reg_read(dev, REE_AF_REEXM_MAX_MATCH, + &reexm_max_match.u); + if (ret) + return ret; + + *max_matches =3D reexm_max_match.s.max; + return 0; +} diff --git a/drivers/regex/octeontx2/otx2_regexdev_hw_access.h b/drivers/re= gex/octeontx2/otx2_regexdev_hw_access.h new file mode 100644 index 000000000..dedf5f328 --- /dev/null +++ b/drivers/regex/octeontx2/otx2_regexdev_hw_access.h @@ -0,0 +1,202 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright (C) 2020 Marvell International Ltd. + */ + +#ifndef _OTX2_REGEXDEV_HW_ACCESS_H_ +#define _OTX2_REGEXDEV_HW_ACCESS_H_ + +#include + +#include "otx2_regexdev.h" + +/* REE instruction queue length */ +#define OTX2_REE_IQ_LEN (1 << 13) + +#define OTX2_REE_DEFAULT_CMD_QLEN OTX2_REE_IQ_LEN + +/* Status register bits */ +#define OTX2_REE_STATUS_PMI_EOJ_BIT (1 << 14) +#define OTX2_REE_STATUS_PMI_SOJ_BIT (1 << 13) +#define OTX2_REE_STATUS_MP_CNT_DET_BIT (1 << 7) +#define OTX2_REE_STATUS_MM_CNT_DET_BIT (1 << 6) +#define OTX2_REE_STATUS_ML_CNT_DET_BIT (1 << 5) +#define OTX2_REE_STATUS_MST_CNT_DET_BIT (1 << 4) +#define OTX2_REE_STATUS_MPT_CNT_DET_BIT (1 << 3) + +/* Register offsets */ +/* REE LF registers */ +#define OTX2_REE_LF_DONE_INT 0x120ull +#define OTX2_REE_LF_DONE_INT_W1S 0x130ull +#define OTX2_REE_LF_DONE_INT_ENA_W1S 0x138ull +#define OTX2_REE_LF_DONE_INT_ENA_W1C 0x140ull +#define OTX2_REE_LF_MISC_INT 0x300ull +#define OTX2_REE_LF_MISC_INT_W1S 0x310ull +#define OTX2_REE_LF_MISC_INT_ENA_W1S 0x320ull +#define OTX2_REE_LF_MISC_INT_ENA_W1C 0x330ull +#define OTX2_REE_LF_ENA 0x10ull +#define OTX2_REE_LF_SBUF_ADDR 0x20ull +#define OTX2_REE_LF_DONE 0x100ull +#define OTX2_REE_LF_DONE_ACK 0x110ull +#define OTX2_REE_LF_DONE_WAIT 0x148ull +#define OTX2_REE_LF_DOORBELL 0x400ull +#define OTX2_REE_LF_OUTSTAND_JOB 0x410ull + +/* BAR 0 */ +#define OTX2_REE_AF_QUE_SBUF_CTL(a) (0x1200ull | (uint64_t)(a) << 3) +#define OTX2_REE_PRIV_LF_CFG(a) (0x41000ull | (uint64_t)(a) << 3) + +#define OTX2_REE_LF_BAR2(vf, q_id) \ + ((vf)->otx2_dev.bar2 + \ + (((vf)->block_address << 20) | ((q_id) << 12))) + + +#define OTX2_REE_QUEUE_HI_PRIO 0x1 + +enum ree_desc_type_e { + REE_TYPE_JOB_DESC =3D 0x0, + REE_TYPE_RESULT_DESC =3D 0x1, + REE_TYPE_ENUM_LAST =3D 0x2 +}; + +union otx2_ree_priv_lf_cfg { + uint64_t u; + struct { + uint64_t slot : 8; + uint64_t pf_func : 16; + uint64_t reserved_24_62 : 39; + uint64_t ena : 1; + } s; +}; + + +union otx2_ree_lf_sbuf_addr { + uint64_t u; + struct { + uint64_t off : 7; + uint64_t ptr : 46; + uint64_t reserved_53_63 : 11; + } s; +}; + +union otx2_ree_lf_ena { + uint64_t u; + struct { + uint64_t ena : 1; + uint64_t reserved_1_63 : 63; + } s; +}; + +union otx2_ree_af_reexm_max_match { + uint64_t u; + struct { + uint64_t max : 8; + uint64_t reserved_8_63 : 56; + } s; +}; + +union otx2_ree_lf_done { + uint64_t u; + struct { + uint64_t done : 20; + uint64_t reserved_20_63 : 44; + } s; +}; + +union otx2_ree_inst { + uint64_t u[8]; + struct { + uint64_t doneint : 1; + uint64_t reserved_1_3 : 3; + uint64_t dg : 1; + uint64_t reserved_5_7 : 3; + uint64_t ooj : 1; + uint64_t reserved_9_15 : 7; + uint64_t reserved_16_63 : 48; + uint64_t inp_ptr_addr : 64; + uint64_t inp_ptr_ctl : 64; + uint64_t res_ptr_addr : 64; + uint64_t wq_ptr : 64; + uint64_t tag : 32; + uint64_t tt : 2; + uint64_t ggrp : 10; + uint64_t reserved_364_383 : 20; + uint64_t reserved_384_391 : 8; + uint64_t ree_job_id : 24; + uint64_t ree_job_ctrl : 16; + uint64_t ree_job_length : 15; + uint64_t reserved_447_447 : 1; + uint64_t ree_job_subset_id_0 : 16; + uint64_t ree_job_subset_id_1 : 16; + uint64_t ree_job_subset_id_2 : 16; + uint64_t ree_job_subset_id_3 : 16; + } cn98xx; +}; + +union otx2_ree_res_status { + uint64_t u; + struct { + uint64_t job_type : 3; + uint64_t mpt_cnt_det : 1; + uint64_t mst_cnt_det : 1; + uint64_t ml_cnt_det : 1; + uint64_t mm_cnt_det : 1; + uint64_t mp_cnt_det : 1; + uint64_t mode : 2; + uint64_t reserved_10_11 : 2; + uint64_t reserved_12_12 : 1; + uint64_t pmi_soj : 1; + uint64_t pmi_eoj : 1; + uint64_t reserved_15_15 : 1; + uint64_t reserved_16_63 : 48; + } s; +}; + +union otx2_ree_res { + uint64_t u[8]; + struct ree_res_s_98 { + uint64_t done : 1; + uint64_t hwjid : 7; + uint64_t ree_res_job_id : 24; + uint64_t ree_res_status : 16; + uint64_t ree_res_dmcnt : 8; + uint64_t ree_res_mcnt : 8; + uint64_t ree_meta_ptcnt : 16; + uint64_t ree_meta_icnt : 16; + uint64_t ree_meta_lcnt : 16; + uint64_t ree_pmi_min_byte_ptr : 16; + uint64_t ree_err : 1; + uint64_t reserved_129_190 : 62; + uint64_t doneint : 1; + uint64_t reserved_192_255 : 64; + uint64_t reserved_256_319 : 64; + uint64_t reserved_320_383 : 64; + uint64_t reserved_384_447 : 64; + uint64_t reserved_448_511 : 64; + } s; +}; + +union otx2_ree_match { + uint64_t u; + struct { + uint64_t ree_rule_id : 32; + uint64_t start_ptr : 14; + uint64_t reserved_46_47 : 2; + uint64_t match_length : 15; + uint64_t reserved_63_63 : 1; + } s; +}; + +void otx2_ree_err_intr_unregister(const struct rte_regexdev *dev); + +int otx2_ree_err_intr_register(const struct rte_regexdev *dev); + +int otx2_ree_iq_enable(const struct rte_regexdev *dev, + const struct otx2_ree_qp *qp, + uint8_t pri, uint32_t size_div128); + +void otx2_ree_iq_disable(struct otx2_ree_qp *qp); + +int otx2_ree_max_matches_get(const struct rte_regexdev *dev, + uint8_t *max_matches); + +#endif /* _OTX2_REGEXDEV_HW_ACCESS_H_ */ diff --git a/drivers/regex/octeontx2/otx2_regexdev_mbox.c b/drivers/regex/o= cteontx2/otx2_regexdev_mbox.c new file mode 100644 index 000000000..6d58d367d --- /dev/null +++ b/drivers/regex/octeontx2/otx2_regexdev_mbox.c @@ -0,0 +1,401 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright (C) 2020 Marvell International Ltd. + */ + +#include "otx2_common.h" +#include "otx2_dev.h" +#include "otx2_regexdev_mbox.h" +#include "otx2_regexdev.h" + +int +otx2_ree_available_queues_get(const struct rte_regexdev *dev, + uint16_t *nb_queues) +{ + struct otx2_ree_data *data =3D dev->data->dev_private; + struct otx2_ree_vf *vf =3D &data->vf; + struct free_rsrcs_rsp *rsp; + struct otx2_dev *otx2_dev; + int ret; + + otx2_dev =3D &vf->otx2_dev; + otx2_mbox_alloc_msg_free_rsrc_cnt(otx2_dev->mbox); + + ret =3D otx2_mbox_process_msg(otx2_dev->mbox, (void *)&rsp); + if (ret) + return -EIO; + + if (vf->block_address =3D=3D RVU_BLOCK_ADDR_REE0) + *nb_queues =3D rsp->ree0; + else + *nb_queues =3D rsp->ree1; + return 0; +} + +int +otx2_ree_queues_attach(const struct rte_regexdev *dev, uint8_t nb_queues) +{ + struct otx2_ree_data *data =3D dev->data->dev_private; + struct otx2_ree_vf *vf =3D &data->vf; + struct rsrc_attach_req *req; + struct otx2_mbox *mbox; + + /* Ask AF to attach required LFs */ + mbox =3D vf->otx2_dev.mbox; + req =3D otx2_mbox_alloc_msg_attach_resources(mbox); + + /* 1 LF =3D 1 queue */ + req->reelfs =3D nb_queues; + req->ree_blkaddr =3D vf->block_address; + + if (otx2_mbox_process(mbox) < 0) + return -EIO; + + /* Update number of attached queues */ + vf->nb_queues =3D nb_queues; + + return 0; +} + +int +otx2_ree_queues_detach(const struct rte_regexdev *dev) +{ + struct otx2_ree_data *data =3D dev->data->dev_private; + struct otx2_ree_vf *vf =3D &data->vf; + struct rsrc_detach_req *req; + struct otx2_mbox *mbox; + + mbox =3D vf->otx2_dev.mbox; + req =3D otx2_mbox_alloc_msg_detach_resources(mbox); + req->reelfs =3D true; + req->partial =3D true; + if (otx2_mbox_process(mbox) < 0) + return -EIO; + + /* Queues have been detached */ + vf->nb_queues =3D 0; + + return 0; +} + +int +otx2_ree_msix_offsets_get(const struct rte_regexdev *dev) +{ + struct otx2_ree_data *data =3D dev->data->dev_private; + struct otx2_ree_vf *vf =3D &data->vf; + struct msix_offset_rsp *rsp; + struct otx2_mbox *mbox; + uint32_t i, ret; + + /* Get REE MSI-X vector offsets */ + mbox =3D vf->otx2_dev.mbox; + otx2_mbox_alloc_msg_msix_offset(mbox); + + ret =3D otx2_mbox_process_msg(mbox, (void *)&rsp); + if (ret) + return ret; + + for (i =3D 0; i < vf->nb_queues; i++) { + if (vf->block_address =3D=3D RVU_BLOCK_ADDR_REE0) + vf->lf_msixoff[i] =3D rsp->ree0_lf_msixoff[i]; + else + vf->lf_msixoff[i] =3D rsp->ree1_lf_msixoff[i]; + otx2_ree_dbg("lf_msixoff[%d] 0x%x", i, vf->lf_msixoff[i]); + } + + return 0; +} + +static int +ree_send_mbox_msg(struct otx2_ree_vf *vf) +{ + struct otx2_mbox *mbox =3D vf->otx2_dev.mbox; + int ret; + + otx2_mbox_msg_send(mbox, 0); + + ret =3D otx2_mbox_wait_for_rsp(mbox, 0); + if (ret < 0) { + otx2_err("Could not get mailbox response"); + return ret; + } + + return 0; +} + +int +otx2_ree_config_lf(const struct rte_regexdev *dev, uint8_t lf, uint8_t pri= , + uint32_t size) +{ + struct otx2_ree_data *data =3D dev->data->dev_private; + struct otx2_ree_vf *vf =3D &data->vf; + struct ree_lf_req_msg *req; + struct otx2_mbox *mbox; + int ret; + + mbox =3D vf->otx2_dev.mbox; + req =3D otx2_mbox_alloc_msg_ree_config_lf(mbox); + + req->lf =3D lf; + req->pri =3D pri ? 1 : 0; + req->size =3D size; + req->blkaddr =3D vf->block_address; + + ret =3D otx2_mbox_process(mbox); + if (ret < 0) { + otx2_err("Could not get mailbox response"); + return ret; + } + return 0; +} + +int +otx2_ree_af_reg_read(const struct rte_regexdev *dev, uint64_t reg, + uint64_t *val) +{ + struct otx2_ree_data *data =3D dev->data->dev_private; + struct otx2_ree_vf *vf =3D &data->vf; + struct ree_rd_wr_reg_msg *msg; + struct otx2_mbox_dev *mdev; + struct otx2_mbox *mbox; + int ret, off; + + mbox =3D vf->otx2_dev.mbox; + mdev =3D &mbox->dev[0]; + msg =3D (struct ree_rd_wr_reg_msg *)otx2_mbox_alloc_msg_rsp(mbox, 0, + sizeof(*msg), sizeof(*msg)); + if (msg =3D=3D NULL) { + otx2_err("Could not allocate mailbox message"); + return -EFAULT; + } + + msg->hdr.id =3D MBOX_MSG_REE_RD_WR_REGISTER; + msg->hdr.sig =3D OTX2_MBOX_REQ_SIG; + msg->hdr.pcifunc =3D vf->otx2_dev.pf_func; + msg->is_write =3D 0; + msg->reg_offset =3D reg; + msg->ret_val =3D val; + msg->blkaddr =3D vf->block_address; + + ret =3D ree_send_mbox_msg(vf); + if (ret < 0) + return ret; + + off =3D mbox->rx_start + + RTE_ALIGN(sizeof(struct mbox_hdr), MBOX_MSG_ALIGN); + msg =3D (struct ree_rd_wr_reg_msg *) ((uintptr_t)mdev->mbase + off); + + *val =3D msg->val; + + return 0; +} + +int +otx2_ree_af_reg_write(const struct rte_regexdev *dev, uint64_t reg, + uint64_t val) +{ + struct otx2_ree_data *data =3D dev->data->dev_private; + struct otx2_ree_vf *vf =3D &data->vf; + struct ree_rd_wr_reg_msg *msg; + struct otx2_mbox *mbox; + + mbox =3D vf->otx2_dev.mbox; + msg =3D (struct ree_rd_wr_reg_msg *)otx2_mbox_alloc_msg_rsp(mbox, 0, + sizeof(*msg), sizeof(*msg)); + if (msg =3D=3D NULL) { + otx2_err("Could not allocate mailbox message"); + return -EFAULT; + } + + msg->hdr.id =3D MBOX_MSG_REE_RD_WR_REGISTER; + msg->hdr.sig =3D OTX2_MBOX_REQ_SIG; + msg->hdr.pcifunc =3D vf->otx2_dev.pf_func; + msg->is_write =3D 1; + msg->reg_offset =3D reg; + msg->val =3D val; + msg->blkaddr =3D vf->block_address; + + return ree_send_mbox_msg(vf); +} + +int +otx2_ree_rule_db_get(const struct rte_regexdev *dev, char *rule_db, + uint32_t rule_db_len, char *rule_dbi, uint32_t rule_dbi_len) +{ + struct otx2_ree_data *data =3D dev->data->dev_private; + struct ree_rule_db_get_req_msg *req; + struct ree_rule_db_get_rsp_msg *rsp; + char *rule_db_ptr =3D (char *)rule_db; + struct otx2_ree_vf *vf =3D &data->vf; + struct otx2_mbox *mbox; + int ret, last =3D 0; + uint32_t len =3D 0; + + mbox =3D vf->otx2_dev.mbox; + if (!rule_db) { + otx2_err("Couldn't return rule db due to NULL pointer"); + return -EFAULT; + } + + while (!last) { + req =3D (struct ree_rule_db_get_req_msg *) + otx2_mbox_alloc_msg_rsp(mbox, 0, sizeof(*req), + sizeof(*rsp)); + if (!req) { + otx2_err("Could not allocate mailbox message"); + return -EFAULT; + } + + req->hdr.id =3D MBOX_MSG_REE_RULE_DB_GET; + req->hdr.sig =3D OTX2_MBOX_REQ_SIG; + req->hdr.pcifunc =3D vf->otx2_dev.pf_func; + req->blkaddr =3D vf->block_address; + req->is_dbi =3D 0; + req->offset =3D len; + ret =3D otx2_mbox_process_msg(mbox, (void *)&rsp); + if (ret) + return ret; + if (rule_db_len < len + rsp->len) { + otx2_err("Rule db size is too small"); + return -EFAULT; + } + otx2_mbox_memcpy(rule_db_ptr, rsp->rule_db, rsp->len); + len +=3D rsp->len; + rule_db_ptr =3D rule_db_ptr + rsp->len; + last =3D rsp->is_last; + } + + if (rule_dbi) { + req =3D (struct ree_rule_db_get_req_msg *) + otx2_mbox_alloc_msg_rsp(mbox, 0, sizeof(*req), + sizeof(*rsp)); + if (!req) { + otx2_err("Could not allocate mailbox message"); + return -EFAULT; + } + + req->hdr.id =3D MBOX_MSG_REE_RULE_DB_GET; + req->hdr.sig =3D OTX2_MBOX_REQ_SIG; + req->hdr.pcifunc =3D vf->otx2_dev.pf_func; + req->blkaddr =3D vf->block_address; + req->is_dbi =3D 1; + req->offset =3D 0; + + ret =3D otx2_mbox_process_msg(mbox, (void *)&rsp); + if (ret) + return ret; + if (rule_dbi_len < rsp->len) { + otx2_err("Rule dbi size is too small"); + return -EFAULT; + } + otx2_mbox_memcpy(rule_dbi, rsp->rule_db, rsp->len); + } + return 0; +} + +int +otx2_ree_rule_db_len_get(const struct rte_regexdev *dev, + uint32_t *rule_db_len, + uint32_t *rule_dbi_len) +{ + struct otx2_ree_data *data =3D dev->data->dev_private; + struct ree_rule_db_len_rsp_msg *rsp; + struct otx2_ree_vf *vf =3D &data->vf; + struct ree_req_msg *req; + struct otx2_mbox *mbox; + int ret; + + mbox =3D vf->otx2_dev.mbox; + req =3D (struct ree_req_msg *) + otx2_mbox_alloc_msg_rsp(mbox, 0, sizeof(*req), sizeof(*rsp)); + if (!req) { + otx2_err("Could not allocate mailbox message"); + return -EFAULT; + } + + req->hdr.id =3D MBOX_MSG_REE_RULE_DB_LEN_GET; + req->hdr.sig =3D OTX2_MBOX_REQ_SIG; + req->hdr.pcifunc =3D vf->otx2_dev.pf_func; + req->blkaddr =3D vf->block_address; + ret =3D otx2_mbox_process_msg(mbox, (void *)&rsp); + if (ret) + return ret; + if (rule_db_len !=3D NULL) + *rule_db_len =3D rsp->len; + if (rule_dbi_len !=3D NULL) + *rule_dbi_len =3D rsp->inc_len; + + return 0; +} + +static int +ree_db_msg(const struct rte_regexdev *dev, const char *db, uint32_t db_len= , + int inc, int dbi) +{ + struct otx2_ree_data *data =3D dev->data->dev_private; + uint32_t len_left =3D db_len, offset =3D 0; + struct ree_rule_db_prog_req_msg *req; + struct otx2_ree_vf *vf =3D &data->vf; + const char *rule_db_ptr =3D db; + struct otx2_mbox *mbox; + struct msg_rsp *rsp; + int ret; + + mbox =3D vf->otx2_dev.mbox; + while (len_left) { + req =3D (struct ree_rule_db_prog_req_msg *) + otx2_mbox_alloc_msg_rsp(mbox, 0, sizeof(*req), + sizeof(*rsp)); + if (!req) { + otx2_err("Could not allocate mailbox message"); + return -EFAULT; + } + req->hdr.id =3D MBOX_MSG_REE_RULE_DB_PROG; + req->hdr.sig =3D OTX2_MBOX_REQ_SIG; + req->hdr.pcifunc =3D vf->otx2_dev.pf_func; + req->offset =3D offset; + req->total_len =3D db_len; + req->len =3D REE_RULE_DB_REQ_BLOCK_SIZE; + req->is_incremental =3D inc; + req->is_dbi =3D dbi; + req->blkaddr =3D vf->block_address; + + if (len_left < REE_RULE_DB_REQ_BLOCK_SIZE) { + req->is_last =3D true; + req->len =3D len_left; + } + otx2_mbox_memcpy(req->rule_db, rule_db_ptr, req->len); + ret =3D otx2_mbox_process_msg(mbox, (void *)&rsp); + if (ret) { + otx2_err("Programming mailbox processing failed"); + return ret; + } + len_left -=3D req->len; + offset +=3D req->len; + rule_db_ptr =3D rule_db_ptr + req->len; + } + return 0; +} + +int +otx2_ree_rule_db_prog(const struct rte_regexdev *dev, const char *rule_db, + uint32_t rule_db_len, const char *rule_dbi, + uint32_t rule_dbi_len) +{ + int inc, ret; + + if (rule_db_len =3D=3D 0) { + otx2_err("Couldn't program empty rule db"); + return -EFAULT; + } + inc =3D (rule_dbi_len !=3D 0); + if ((rule_db =3D=3D NULL) || (inc && (rule_dbi =3D=3D NULL))) { + otx2_err("Couldn't program NULL rule db"); + return -EFAULT; + } + if (inc) { + ret =3D ree_db_msg(dev, rule_dbi, rule_dbi_len, inc, 1); + if (ret) + return ret; + } + return ree_db_msg(dev, rule_db, rule_db_len, inc, 0); +} diff --git a/drivers/regex/octeontx2/otx2_regexdev_mbox.h b/drivers/regex/o= cteontx2/otx2_regexdev_mbox.h new file mode 100644 index 000000000..953efa672 --- /dev/null +++ b/drivers/regex/octeontx2/otx2_regexdev_mbox.h @@ -0,0 +1,38 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright (C) 2020 Marvell International Ltd. + */ + +#ifndef _OTX2_REGEXDEV_MBOX_H_ +#define _OTX2_REGEXDEV_MBOX_H_ + +#include + +int otx2_ree_available_queues_get(const struct rte_regexdev *dev, + uint16_t *nb_queues); + +int otx2_ree_queues_attach(const struct rte_regexdev *dev, uint8_t nb_queu= es); + +int otx2_ree_queues_detach(const struct rte_regexdev *dev); + +int otx2_ree_msix_offsets_get(const struct rte_regexdev *dev); + +int otx2_ree_config_lf(const struct rte_regexdev *dev, uint8_t lf, uint8_t= pri, + uint32_t size); + +int otx2_ree_af_reg_read(const struct rte_regexdev *dev, uint64_t reg, + uint64_t *val); + +int otx2_ree_af_reg_write(const struct rte_regexdev *dev, uint64_t reg, + uint64_t val); + +int otx2_ree_rule_db_get(const struct rte_regexdev *dev, char *rule_db, + uint32_t rule_db_len, char *rule_dbi, uint32_t rule_dbi_len); + +int otx2_ree_rule_db_len_get(const struct rte_regexdev *dev, + uint32_t *rule_db_len, uint32_t *rule_dbi_len); + +int otx2_ree_rule_db_prog(const struct rte_regexdev *dev, const char *rule= _db, + uint32_t rule_db_len, const char *rule_dbi, + uint32_t rule_dbi_len); + +#endif /* _OTX2_REGEXDEV_MBOX_H_ */ diff --git a/drivers/regex/octeontx2/rte_pmd_octeontx2_regex_version.map b/= drivers/regex/octeontx2/rte_pmd_octeontx2_regex_version.map new file mode 100644 index 000000000..34eb991c7 --- /dev/null +++ b/drivers/regex/octeontx2/rte_pmd_octeontx2_regex_version.map @@ -0,0 +1,3 @@ +DPDK_20.08 { + local: *; +}; diff --git a/meson_options.txt b/meson_options.txt index 9bf18ab6b..214b5f7f5 100644 --- a/meson_options.txt +++ b/meson_options.txt @@ -36,3 +36,5 @@ option('tests', type: 'boolean', value: true, description: 'build unit tests') option('use_hpet', type: 'boolean', value: false, description: 'use HPET timer in EAL') +option('ree_compiler_sdk', type: 'string', value: '', + description: 'path to REE compiler SDK optional library for regex device'= ) \ No newline at end of file --=20 2.28.0