From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from NAM05-CO1-obe.outbound.protection.outlook.com (mail-eopbgr720049.outbound.protection.outlook.com [40.107.72.49]) by dpdk.org (Postfix) with ESMTP id 845471B124 for ; Tue, 9 Oct 2018 11:08:51 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=CAVIUMNETWORKS.onmicrosoft.com; s=selector1-cavium-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=85z9XqSF8Is+Xeppdjmtxa2ojZpWKzZ6g/d8VQgJ/OM=; b=efli5hVjxG8J9N0FiHEGBcinUDBapGheC1+38LhipJfgO7EMDy9T+f+140MtDE/S1tI3LpwS90OgjjemKWtitma7a/WUlRtfBwdbrzLspFkrfFa9Bh2ycpxN5qEZrvsGIYVn90G2lf6SlF7+j9wMzwGZrLqjVz9Ns4V3WkuO3p8= Authentication-Results: spf=none (sender IP is ) smtp.mailfrom=Anoob.Joseph@cavium.com; Received: from ajoseph83.caveonetworks.com.com (115.113.156.2) by BN7PR07MB4897.namprd07.prod.outlook.com (2603:10b6:406:ef::26) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.1207.28; Tue, 9 Oct 2018 09:08:45 +0000 From: Anoob Joseph To: Akhil Goyal , Pablo de Lara , Thomas Monjalon Cc: Ankur Dwivedi , Jerin Jacob , Narayana Prasad , Anoob Joseph , Murthy NSSR , Nithin Dabilpuram , Ragothaman Jayaraman , Srisivasubramanian S , Tejasree Kondoj , dev@dpdk.org Date: Tue, 9 Oct 2018 14:37:36 +0530 Message-Id: <1539076076-19786-4-git-send-email-anoob.joseph@caviumnetworks.com> X-Mailer: git-send-email 2.7.4 In-Reply-To: <1539076076-19786-1-git-send-email-anoob.joseph@caviumnetworks.com> References: <1538744363-30340-1-git-send-email-anoob.joseph@caviumnetworks.com> <1539076076-19786-1-git-send-email-anoob.joseph@caviumnetworks.com> MIME-Version: 1.0 Content-Type: text/plain X-Originating-IP: [115.113.156.2] X-ClientProxiedBy: PN1PR0101CA0004.INDPRD01.PROD.OUTLOOK.COM (2603:1096:c00:e::14) To BN7PR07MB4897.namprd07.prod.outlook.com (2603:10b6:406:ef::26) X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: bbbaa216-a713-4fd6-d093-08d62dc6d37d X-Microsoft-Antispam: BCL:0; PCL:0; RULEID:(7020095)(4652040)(8989299)(4534185)(7168020)(4627221)(201703031133081)(201702281549075)(8990200)(5600074)(711020)(2017052603328)(7153060)(7193020); SRVR:BN7PR07MB4897; X-Microsoft-Exchange-Diagnostics: 1; BN7PR07MB4897; 3:S+ShpIsr5IYGYtpL8WsXoL4S/sEYUfIRl0T/QtGLy+NHXDIgwnrdSPvNoib7Xo7G/HRUuDkU7ejKTBDVVUsONvW3pLV8HTp3DStsgyTgad9cfAUekXEX/KZ1WIkzjjxm3MK//FQpcUa69pIqvEi+bEag4JGdBlZfEZYEO0Qvf414d+3Q7kUia54c9z8Z23wYXXqhFMq9/KLVhY8C5HnAr9vndv/gvurV+tMeQIWytqWNiI1Cpwh5Fm42HzIvE4to; 25:0QESoJh/oH7aqq7WG98QKCd/B7R9hX66/bLoV78+5eHOi3e5yUqBjNUdFXaUwhvEFU5mJXU12ShlMIneOKHxUfnS1mXaYSUICYbj2Gp1digT5xCIchIXFLtiNhZytYUDlKYbH9vD/B7/PNuRbB6E3YP3YQwNK73QTxSxnkSdk8A6Vo72pEVCSNYnj1g3tZweJzotCPnVvEkoq9SRDJOo+qlPy13wjUHFUDipq7aG2kyGcwTABbur4VmNud4Nowqscy/4R9o7PFxYqmdx+m7DtAkCV4na0qmimQzN2x1BjmvM5JIA0MZmplM/trahM9goS/iG7sVmGEAY9UAwrZVfeQ==; 31:K0NKXS984Wp9/UysOY1yCL+PibpdrZThQZ4kJSREbn2GaQrnvR318s4jWjG1fsqhXrsfXkvzve6EWt8MfbYULvS5Q9M+4MK1HyMdKr52ZJg3/ye7c072E8CGpJ40Zq7PxVTPKAfD2iw6o4oexlS1Lu5ZTI+0aGwLs0CRwAwxkgFHZCpXjuObaYdRdLka4vUP2WYy2JlACw5kLKnBLNbbe0mzGtYiay4xEqRb3P9EYgE= X-MS-TrafficTypeDiagnostic: BN7PR07MB4897: X-Microsoft-Exchange-Diagnostics: 1; BN7PR07MB4897; 20:z1FaqB9MbDxUPp2PorWorsRTX23YbNigcX/FmYDVPMYBehtgVtQJNgUVjjXqweU82zkX7lJKhyg8gMIGlb5rn5zQT8u8hBEMkWB6Tcc04+pqDqLV+NzZyqpzprKXlBljjRSaoliuGckiGEQ+v1lhO3R4z5BDgkjf0Pff5YgWW+QnDa6HT5OGtk9fIEc+oNx/7hZmNWFF2EF+kR7qkVbsuqKTfq1w1PkhXKJYo6jeDJfMP3Gmfgzbca8uPdQDLI+l4Cq486rAvFsoYtxZ8JA5NOs/q5AOxdWb2YkmU6o8nIFMxa5Wp5ytWzgvsb89eOdUNqnq931NPgfV+JwfvDB8LAbXh8/V4c8OrfY1+NV4F3Bvl32j9dF5JMRpTIYEj9rSGEiPEfoRgdGqV9cwCGGi7n90HkJ/rL4OBup+40KrmNoMkED63kYHz9iY0eOMAd2ATAUCEAlF/YoRCJrlFUHubVLY89iKGi2yhkNpaOWPPHhIq8ggQkoz9caUUuECsx5jzriK6qxieIetPuMxE2Zmn4Oe4GTUkIN5/kqN5mrI/M3o7g8phG7yPsqoNU0vq/e8iMzFbBJx49PUtD05cDCDxXXoKidUS2h8ERmivhoMllw=; 4:rbYhIoIdjP5I/fee79hJxvfkldbL60/46UrGlCUeR8oYnEsT28xFVYyayB1T/DxlwQhJQkcT9sHOiKJIaetH5zp3tE7Zzm89fqe1UM31lqFjKwWHJ8DGbCHGOOqzfajdoKVcd2MlDQxusl28QBcKUbliOf4+WnXpwrlgACg9jpuiTtUQvO4XvI2o8oIySXPocCCAfWluFgmrY9+dwNhwPbmdUTzfbDxW/zw/gUhmFavpe9iD6c6X6fEm9qAll8Rgs/smOCjMu61kw7aQQeMLLA== X-Microsoft-Antispam-PRVS: X-Exchange-Antispam-Report-Test: UriScan:; X-MS-Exchange-SenderADCheck: 1 X-Exchange-Antispam-Report-CFA-Test: BCL:0; PCL:0; RULEID:(8211001083)(6040522)(2401047)(5005006)(8121501046)(3002001)(93006095)(10201501046)(3231355)(944501410)(52105095)(149066)(150057)(6041310)(20161123560045)(201703131423095)(201702281528075)(20161123555045)(201703061421075)(201703061406153)(20161123562045)(20161123564045)(20161123558120)(201708071742011)(7699051); SRVR:BN7PR07MB4897; BCL:0; PCL:0; RULEID:; SRVR:BN7PR07MB4897; X-Forefront-PRVS: 08200063E9 X-Forefront-Antispam-Report: SFV:NSPM; SFS:(10009020)(396003)(366004)(376002)(136003)(346002)(39860400002)(199004)(189003)(42882007)(47776003)(106356001)(6666003)(186003)(5660300001)(26005)(68736007)(16526019)(8936002)(25786009)(6116002)(2906002)(81156014)(50226002)(305945005)(81166006)(7736002)(3846002)(105586002)(478600001)(8676002)(54906003)(110136005)(48376002)(16586007)(14444005)(6486002)(6506007)(50466002)(316002)(53936002)(386003)(51416003)(52116002)(76176011)(11346002)(6512007)(44832011)(446003)(36756003)(55236004)(956004)(2616005)(4744004)(486006)(4326008)(66066001)(97736004)(72206003)(476003); DIR:OUT; SFP:1101; SCL:1; SRVR:BN7PR07MB4897; H:ajoseph83.caveonetworks.com.com; FPR:; SPF:None; LANG:en; PTR:InfoNoRecords; A:1; MX:1; Received-SPF: None (protection.outlook.com: cavium.com does not designate permitted sender hosts) X-Microsoft-Exchange-Diagnostics: =?us-ascii?Q?1; BN7PR07MB4897; 23:ajpQ3cPh3JoP1PUlBvkAnBV++ftSHLrPaW8SFngk2?= =?us-ascii?Q?uIRQfCeOPCFFEzbdwrc1+GKXr5AGGl2MzMO1XljzueF1IafLqMARSjDXiEwL?= =?us-ascii?Q?PT57GFSWeLvddb6ihDRH1jWUsy9Cum8OFZ8GRnLWib6mXTtD7eYqF0RT0coc?= =?us-ascii?Q?MdnAnnz4qP4q8EUrtFfWr6XYO2dnNBsSjIJTRA2ro9XqARYZe7yOMGI0FqZr?= =?us-ascii?Q?oB5K6i36F/I6zYIy2i1hv2aUuc1xeooZuH4oIAS5zfzEtWtn0t4fJf8/p6gi?= =?us-ascii?Q?uDwN+z80ucUVvR4WxiLo1cI3Q9mZiA8kHXLWNg6rnzsw3NZOQwNsTYRrUwQx?= =?us-ascii?Q?WHz0x3FOiw5RbqJ3UKR6l73+1FD3B0xkpDtTfRCwJDgWPerJBN7Iy82552mh?= =?us-ascii?Q?yCkgIP5q6pUoU5c3105G/SKhPHMjQI0Kd5vhkqQJs+AYFs9fUlJK7aUMnq/Q?= =?us-ascii?Q?623ChtxJc1BU4gsiCccHmqxlvRfgpVMqfyZ0iJS+atSaNET4GY+vMaRkOYwn?= =?us-ascii?Q?KHisPdyShPpYRbxSUfrhm3G4NFY8KON0tU90JOfe3eCvLAF7aA40GN9tPMhQ?= =?us-ascii?Q?EDyB2igYeRjkdwUHBqvQKM8LYXJpxYyoJdPILiWeN86kE2dGNL7jtjS7+8pi?= =?us-ascii?Q?pnZBqwqlETA85at7sX+MAYuRgGjRKsPlKZQZo76qaRJ77HjcLX8GFjwkU8fd?= =?us-ascii?Q?cUk5uIYyFo0TnFsGXX2WqKhIbGrlYPaKDLOVYOu+JoMlZM+C5B6R+MSB4EY1?= =?us-ascii?Q?JYZVhhYt3wsGdeCd8x9F/Yhw5sYrGhF7HhMVUUxJWJzCfAEcFSwILCZvd1ld?= =?us-ascii?Q?m5edSyiM0dZwrVi/grfFZYTkcHayP/WHPyIBjBa+4CulrSR/8F5qC/CT9j0B?= =?us-ascii?Q?CXKGegyW6c0PyW/dzyEHk5nKx+AKEl+6BzfFvcjxN68fI6oUgpQNf5Gmy34y?= =?us-ascii?Q?VOSk+V4b3J9WPJXBYNT8VJ8Wy6lOepElfe0uiJkkfGWCcttAnpV9bMIzsEWZ?= =?us-ascii?Q?NweVYqgz36inPGchxzb6RscKw7OMTejZuHjIRWFai/byG/Mg9WJUSv6f0hbx?= =?us-ascii?Q?atXzsjTps/7NB/oJ0bOcyc1nK9SEuTvMojLZhX7tKbKRWfxy9+Zd9jC87mGW?= =?us-ascii?Q?aCNSen/7oZbZO/LmupZvWOucY0SnBbO+8OhAuTCOgnRs3CtsseUOknA6iJvn?= =?us-ascii?Q?6zHWljVpTOIENE3apIM7CgUC9SzC9/GaPDmYnJuQ54sOKtHAxAV1r2fFC+rr?= =?us-ascii?Q?Af/8vS8oIEgeeaW+smayR/Wstful9mHvpRHIYt+dskSvM2U28F/E2JMN9qIJ?= =?us-ascii?B?QT09?= X-Microsoft-Antispam-Message-Info: 74sS0rs99Pyax/yKTP54xUQMKc3dD0vZ/WupJSUm56qHBL1Ynm4kzADNZsMyVgCpMW0vK8AHvCtbFpNFtYvpsKjX5d1q/M9lrZqnsspNQWiWXIMspNVlCexy+66FSDqyVUrUMmkQiplCqECpbif1raZxo8Wpj2Q1dYz5CWpOsP2lCD85VAocKykXojIAfryXSNZcfXVb90TDLBi/EY3Y5M8S+1Ua9wi9mybrAvBkPk9ZlDW7zuimLqq4hxCz2kA/VJC406IsNPMEfnBgc7yf0SZpKTZyurqYEph0trKbcisG6Ymq1iHpApQespi8SHxxCde9PFojc7LkOJBfrrQFSLHctClG+PwxcxX8UwZPpO0= X-Microsoft-Exchange-Diagnostics: 1; BN7PR07MB4897; 6:7zAx2UMgECU1prWoLBe1Fb6jqRUSNOqH++KB2bG+DkIO12ANFtYenUUbKABBSC67vvCFqsEX/e6mA/mvLvKejQKtI5ngjBwuzP6PLmO7TppydoMQ/2lKo0f33bvj6k2szRnhfnUPMq6HWGBkb3Yw+1F4QQfDMJypxd5zH7Mk7hwRLjnavezkCbTzIK6o0HfAQjdvrE4jVVWRmozunrDLwSXe6DW6QCMsEPVMJZobkGSrLKG5rmhqTIAH8TdgfOAkTzKL/s2ZhEccnmPuaREgjRUxaShain0ou0eircDKhb3lJxu9fBFDLGHDeULVApZMzYLFw0xDRJws7/eO39cbfw22crhOxArix5kROhb3Y7ZVXD9w8FVJOe0BQIfEWX5py+WSKk9TMOpVUXA9zk+luuiTG04f345naVxpWfSDO8D72eJSjKUIvYwwmxNwsR/z+wcjDsLCBR/p0GIFTyu/yw==; 5:lf5+p+hrUWON224Df0+r+3a7l9EDBQfXusztVSg51L+2QyevJFgSrCan3Ek8a9kiXmxh8yIq2Sk3c2BBFbCqMZCehbQtzDutDxl4eYl8lRyqZq0I1VRkSoieYgO5BdRt0NK2PoVv0N+d9fe0Ff4K3KVOZHTu4qPEotHsCCAJpyA=; 7:p9x7mDJLdANUm/FoVNysLYSsrijrB5deDMlIAP0sz3utXahPPNWJS66e+DrsPe6HHty+IRIWHKEM4/oEftRbA9NR8peeQM3P1AGBWVtj30c7GuDpUPhDigph6Vm/k2xgwBeh9xi4jc+GKBFjImDFSaiUy6ybgdns86NorF2aV6LCei2SWX6EYBm+QYJvPaqGd34eCKg+m7Xcbtn0Y7IkIOhiqFffhAcfeuw2PaBfTH4Dk1vpSo3LqSQG4G9liC4a SpamDiagnosticOutput: 1:99 SpamDiagnosticMetadata: NSPM X-OriginatorOrg: caviumnetworks.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 09 Oct 2018 09:08:45.8651 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: bbbaa216-a713-4fd6-d093-08d62dc6d37d X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 711e4ccf-2e9b-4bcf-a551-4094005b6194 X-MS-Exchange-Transport-CrossTenantHeadersStamped: BN7PR07MB4897 Subject: [dpdk-dev] [PATCH v4 03/23] crypto/octeontx: add hardware register access for misc poll X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 09 Oct 2018 09:08:52 -0000 From: Ankur Dwivedi Adding hardware register accesses required for misc poll Signed-off-by: Ankur Dwivedi Signed-off-by: Anoob Joseph Signed-off-by: Murthy NSSR Signed-off-by: Nithin Dabilpuram Signed-off-by: Ragothaman Jayaraman Signed-off-by: Srisivasubramanian S Signed-off-by: Tejasree Kondoj --- drivers/common/cpt/cpt_hw_types.h | 519 ++++++++++++++++++++++ drivers/crypto/octeontx/otx_cryptodev_hw_access.c | 192 +++++++- drivers/crypto/octeontx/otx_cryptodev_hw_access.h | 11 + 3 files changed, 721 insertions(+), 1 deletion(-) create mode 100644 drivers/common/cpt/cpt_hw_types.h diff --git a/drivers/common/cpt/cpt_hw_types.h b/drivers/common/cpt/cpt_hw_types.h new file mode 100644 index 0000000..0a98621 --- /dev/null +++ b/drivers/common/cpt/cpt_hw_types.h @@ -0,0 +1,519 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright(c) 2018 Cavium, Inc + */ + +#ifndef _CPT_HW_TYPES_H_ +#define _CPT_HW_TYPES_H_ + +#include + +/* + * This file defines HRM specific structs. + * + */ + +#define CPT_VF_INTR_MBOX_MASK (1<<0) +#define CPT_VF_INTR_DOVF_MASK (1<<1) +#define CPT_VF_INTR_IRDE_MASK (1<<2) +#define CPT_VF_INTR_NWRP_MASK (1<<3) +#define CPT_VF_INTR_SWERR_MASK (1<<4) +#define CPT_VF_INTR_HWERR_MASK (1<<5) +#define CPT_VF_INTR_FAULT_MASK (1<<6) + +/* + * CPT_INST_S software command definitions + * Words EI (0-3) + */ +typedef union { + uint64_t u64; + struct { + uint16_t opcode; + uint16_t param1; + uint16_t param2; + uint16_t dlen; + } s; +} vq_cmd_word0_t; + +typedef union { + uint64_t u64; + struct { +#if RTE_BYTE_ORDER == RTE_BIG_ENDIAN + uint64_t grp : 3; + uint64_t cptr : 61; +#else + uint64_t cptr : 61; + uint64_t grp : 3; +#endif + } s; +} vq_cmd_word3_t; + +typedef struct cpt_vq_command { + vq_cmd_word0_t cmd; + uint64_t dptr; + uint64_t rptr; + vq_cmd_word3_t cptr; +} cpt_vq_cmd_t; + +/** + * Structure cpt_inst_s + * + * CPT Instruction Structure + * This structure specifies the instruction layout. + * Instructions are stored in memory as little-endian unless + * CPT()_PF_Q()_CTL[INST_BE] is set. + */ +typedef union cpt_inst_s { + uint64_t u[8]; + struct cpt_inst_s_8s { +#if (RTE_BYTE_ORDER == RTE_BIG_ENDIAN) /* Word 0 - Big Endian */ + uint64_t reserved_17_63 : 47; + /* [ 16: 16] Done interrupt. + * 0 = No interrupts related to this instruction. + * 1 = When the instruction completes,CPT()_VQ()_DONE[DONE] + * will be incremented, and based on the rules described + * there an interrupt may occur. + */ + uint64_t doneint : 1; + uint64_t reserved_0_15 : 16; +#else /* Word 0 - Little Endian */ + uint64_t reserved_0_15 : 16; + uint64_t doneint : 1; + uint64_t reserved_17_63 : 47; +#endif /* Word 0 - End */ +#if (RTE_BYTE_ORDER == RTE_BIG_ENDIAN) /* Word 1 - Big Endian */ + /* [127: 64] Result IOVA. + * If nonzero, specifies where to write CPT_RES_S. + * If zero, no result structure will be written. + * Address must be 16-byte aligned. + * + * Bits <63:49> are ignored by hardware; software should + * use a sign-extended bit <48> for forward compatibility. + */ + uint64_t res_addr : 64; +#else /* Word 1 - Little Endian */ + uint64_t res_addr : 64; +#endif /* Word 1 - End */ +#if (RTE_BYTE_ORDER == RTE_BIG_ENDIAN) /* Word 2 - Big Endian */ + uint64_t reserved_172_191 : 20; + /* [171:162] If [WQ_PTR] is nonzero, the SSO guest-group to + * use when CPT submits work to SSO. + * For the SSO to not discard the add-work request, FPA_PF_MAP() + * must map [GRP] and CPT()_PF_Q()_GMCTL[GMID] as valid. + */ + uint64_t grp : 10; + /* [161:160] If [WQ_PTR] is nonzero, the SSO tag type to use + * when CPT submits work to SSO. + */ + uint64_t tt : 2; + /* [159:128] If [WQ_PTR] is nonzero, the SSO tag to use when + * CPT submits work to SSO. + */ + uint64_t tag : 32; +#else /* Word 2 - Little Endian */ + uint64_t tag : 32; + uint64_t tt : 2; + uint64_t grp : 10; + uint64_t reserved_172_191 : 20; +#endif /* Word 2 - End */ +#if (RTE_BYTE_ORDER == RTE_BIG_ENDIAN) /* Word 3 - Big Endian */ + /** [255:192] If [WQ_PTR] is nonzero, it is a pointer to a + * work-queue entry that CPT submits work to SSO after all + * context, output data, and result write operations are + * visible to other CNXXXX units and the cores. + * Bits <2:0> must be zero. + * Bits <63:49> are ignored by hardware; software should use a + * sign-extended bit <48> for forward compatibility. + * Internal:Bits <63:49>, <2:0> are ignored by hardware, + * treated as always 0x0. + **/ + uint64_t wq_ptr : 64; +#else /* Word 3 - Little Endian */ + uint64_t wq_ptr : 64; +#endif /* Word 3 - End */ +#if (RTE_BYTE_ORDER == RTE_BIG_ENDIAN) /* Word 4 - Big Endian */ + union { + /** [319:256] Engine instruction word 0. Passed to the + * AE/SE. + **/ + uint64_t ei0 : 64; + vq_cmd_word0_t vq_cmd_w0; + }; +#else /* Word 4 - Little Endian */ + union { + uint64_t ei0 : 64; + vq_cmd_word0_t vq_cmd_w0; + }; +#endif /* Word 4 - End */ +#if (RTE_BYTE_ORDER == RTE_BIG_ENDIAN) /* Word 5 - Big Endian */ + union { + /** [383:320] Engine instruction word 1. Passed to the + * AE/SE. + **/ + uint64_t ei1 : 64; + uint64_t dptr; + }; +#else /* Word 5 - Little Endian */ + union { + uint64_t ei1 : 64; + uint64_t dptr; + }; +#endif /* Word 5 - End */ +#if (RTE_BYTE_ORDER == RTE_BIG_ENDIAN) /* Word 6 - Big Endian */ + union { + /** [447:384] Engine instruction word 2. Passed to the + * AE/SE. + **/ + uint64_t ei2 : 64; + uint64_t rptr; + }; +#else /* Word 6 - Little Endian */ + union { + uint64_t ei2 : 64; + uint64_t rptr; + }; +#endif /* Word 6 - End */ +#if (RTE_BYTE_ORDER == RTE_BIG_ENDIAN) /* Word 7 - Big Endian */ + union { + /** [511:448] Engine instruction word 3. Passed to the + * AE/SE. + **/ + uint64_t ei3 : 64; + vq_cmd_word3_t vq_cmd_w3; + }; +#else /* Word 7 - Little Endian */ + union { + uint64_t ei3 : 64; + vq_cmd_word3_t vq_cmd_w3; + }; +#endif /* Word 7 - End */ + } s8x; +} cpt_inst_s_t; + +/** + * Structure cpt_res_s + * + * CPT Result Structure + * The CPT coprocessor writes the result structure after it completes a + * CPT_INST_S instruction. The result structure is exactly 16 bytes, and each + * instruction completion produces exactly one result structure. + * + * This structure is stored in memory as little-endian unless + * CPT()_PF_Q()_CTL[INST_BE] is set. + */ +typedef union cpt_res_s { + uint64_t u[2]; + struct cpt_res_s_8s { +#if (RTE_BYTE_ORDER == RTE_BIG_ENDIAN) /* Word 0 - Big Endian */ + uint64_t reserved_17_63 : 47; + /** [ 16: 16] Done interrupt. This bit is copied from the + * corresponding instruction's CPT_INST_S[DONEINT]. + **/ + uint64_t doneint : 1; + uint64_t reserved_8_15 : 8; + /** [ 7: 0] Indicates completion/error status of the CPT + * coprocessor for the associated instruction, as enumerated by + * CPT_COMP_E. Core software may write the memory location + * containing [COMPCODE] to 0x0 before ringing the doorbell, and + * then poll for completion by checking for a nonzero value. + * + * Once the core observes a nonzero [COMPCODE] value in this + * case, the CPT coprocessor will have also completed L2/DRAM + * write operations. + **/ + uint64_t compcode : 8; +#else /* Word 0 - Little Endian */ + uint64_t compcode : 8; + uint64_t reserved_8_15 : 8; + uint64_t doneint : 1; + uint64_t reserved_17_63 : 47; +#endif /* Word 0 - End */ +#if (RTE_BYTE_ORDER == RTE_BIG_ENDIAN) /* Word 1 - Big Endian */ + uint64_t reserved_64_127 : 64; +#else /* Word 1 - Little Endian */ + uint64_t reserved_64_127 : 64; +#endif /* Word 1 - End */ + } s8x; +} cpt_res_s_t; + +/** + * Register (NCB) cpt#_vq#_ctl + * + * CPT VF Queue Control Registers + * This register configures queues. This register should be changed (other than + * clearing [ENA]) only when quiescent (see CPT()_VQ()_INPROG[INFLIGHT]). + */ +typedef union { + uint64_t u; + struct cptx_vqx_ctl_s { +#if (RTE_BYTE_ORDER == RTE_BIG_ENDIAN) /* Word 0 - Big Endian */ + uint64_t reserved_1_63 : 63; + /** [ 0: 0](R/W/H) Enables the logical instruction queue. + * See also CPT()_PF_Q()_CTL[CONT_ERR] and + * CPT()_VQ()_INPROG[INFLIGHT]. + * 1 = Queue is enabled. + * 0 = Queue is disabled. + **/ + uint64_t ena : 1; +#else /* Word 0 - Little Endian */ + uint64_t ena : 1; + uint64_t reserved_1_63 : 63; +#endif /* Word 0 - End */ + } s; +} cptx_vqx_ctl_t; + +/** + * Register (NCB) cpt#_vq#_done + * + * CPT Queue Done Count Registers + * These registers contain the per-queue instruction done count. + */ +typedef union { + uint64_t u; + struct cptx_vqx_done_s { +#if (RTE_BYTE_ORDER == RTE_BIG_ENDIAN) /* Word 0 - Big Endian */ + uint64_t reserved_20_63 : 44; + /** [ 19: 0](R/W/H) Done count. When CPT_INST_S[DONEINT] set + * and that instruction completes,CPT()_VQ()_DONE[DONE] is + * incremented when the instruction finishes. Write to this + * field are for diagnostic use only; instead software writes + * CPT()_VQ()_DONE_ACK with the number of decrements for this + * field. + * + * Interrupts are sent as follows: + * + * When CPT()_VQ()_DONE[DONE] = 0, then no results are pending, + * the interrupt coalescing timer is held to zero, and an + * interrupt is not sent. + * + * When CPT()_VQ()_DONE[DONE] != 0, then the interrupt + * coalescing timer counts. If the counter is >= CPT()_VQ()_DONE + * _WAIT[TIME_WAIT]*1024, or CPT()_VQ()_DONE[DONE] >= CPT()_VQ() + * _DONE_WAIT[NUM_WAIT], i.e. enough time has passed or enough + * results have arrived, then the interrupt is sent. Otherwise, + * it is not sent due to coalescing. + * + * When CPT()_VQ()_DONE_ACK is written (or CPT()_VQ()_DONE is + * written but this is not typical), the interrupt coalescing + * timer restarts. Note after decrementing this interrupt + * equation is recomputed, for example if CPT()_VQ()_DONE[DONE] + * >= CPT()_VQ()_DONE_WAIT[NUM_WAIT] and because the timer is + * zero, the interrupt will be resent immediately. (This covers + * the race case between software acknowledging an interrupt and + * a result returning.) + * + * When CPT()_VQ()_DONE_ENA_W1S[DONE] = 0, interrupts are not + * sent, but the counting described above still occurs. + * + * Since CPT instructions complete out-of-order, if software is + * using completion interrupts the suggested scheme is to + * request a DONEINT on each request, and when an interrupt + * arrives perform a "greedy" scan for completions; even if a + * later command is acknowledged first this will not result in + * missing a completion. + * + * Software is responsible for making sure [DONE] does not + * overflow; for example by insuring there are not more than + * 2^20-1 instructions in flight that may request interrupts. + **/ + uint64_t done : 20; +#else /* Word 0 - Little Endian */ + uint64_t done : 20; + uint64_t reserved_20_63 : 44; +#endif /* Word 0 - End */ + } s; +} cptx_vqx_done_t; + +/** + * Register (NCB) cpt#_vq#_done_ack + * + * CPT Queue Done Count Ack Registers + * This register is written by software to acknowledge interrupts. + */ +typedef union { + uint64_t u; + struct cptx_vqx_done_ack_s { +#if (RTE_BYTE_ORDER == RTE_BIG_ENDIAN) /* Word 0 - Big Endian */ + uint64_t reserved_20_63 : 44; + /** [ 19: 0](R/W/H) Number of decrements to CPT()_VQ()_DONE + * [DONE]. Reads CPT()_VQ()_DONE[DONE]. + * + * Written by software to acknowledge interrupts. If CPT()_VQ()_ + * DONE[DONE] is still nonzero the interrupt will be re-sent if + * the conditions described in CPT()_VQ()_DONE[DONE] are + * satisfied. + **/ + uint64_t done_ack : 20; +#else /* Word 0 - Little Endian */ + uint64_t done_ack : 20; + uint64_t reserved_20_63 : 44; +#endif /* Word 0 - End */ + } s; +} cptx_vqx_done_ack_t; + +/** + * Register (NCB) cpt#_vq#_done_wait + * + * CPT Queue Done Interrupt Coalescing Wait Registers + * Specifies the per queue interrupt coalescing settings. + */ +typedef union { + uint64_t u; + struct cptx_vqx_done_wait_s { +#if (RTE_BYTE_ORDER == RTE_BIG_ENDIAN) /* Word 0 - Big Endian */ + uint64_t reserved_48_63 : 16; + /** [ 47: 32](R/W) Time hold-off. When CPT()_VQ()_DONE[DONE] = + * 0, or CPT()_VQ()_DONE_ACK is written a timer is cleared. When + * the timer reaches [TIME_WAIT]*1024 then interrupt coalescing + * ends; see CPT()_VQ()_DONE[DONE]. If 0x0, time coalescing is + * disabled. + **/ + uint64_t time_wait : 16; + uint64_t reserved_20_31 : 12; + /** [ 19: 0](R/W) Number of messages hold-off. When + * CPT()_VQ()_DONE[DONE] >= [NUM_WAIT] then interrupt coalescing + * ends; see CPT()_VQ()_DONE[DONE]. If 0x0, same behavior as + * 0x1. + **/ + uint64_t num_wait : 20; +#else /* Word 0 - Little Endian */ + uint64_t num_wait : 20; + uint64_t reserved_20_31 : 12; + uint64_t time_wait : 16; + uint64_t reserved_48_63 : 16; +#endif /* Word 0 - End */ + } s; +} cptx_vqx_done_wait_t; + +/** + * Register (NCB) cpt#_vq#_doorbell + * + * CPT Queue Doorbell Registers + * Doorbells for the CPT instruction queues. + */ +typedef union { + uint64_t u; + struct cptx_vqx_doorbell_s { +#if (RTE_BYTE_ORDER == RTE_BIG_ENDIAN) /* Word 0 - Big Endian */ + uint64_t reserved_20_63 : 44; + uint64_t dbell_cnt : 20; + /** [ 19: 0](R/W/H) Number of instruction queue 64-bit words + * to add to the CPT instruction doorbell count. Readback value + * is the the current number of pending doorbell requests. + * + * If counter overflows CPT()_VQ()_MISC_INT[DBELL_DOVF] is set. + * + * To reset the count back to zero, write one to clear + * CPT()_VQ()_MISC_INT_ENA_W1C[DBELL_DOVF], then write a value + * of 2^20 minus the read [DBELL_CNT], then write one to + * CPT()_VQ()_MISC_INT_W1C[DBELL_DOVF] and + * CPT()_VQ()_MISC_INT_ENA_W1S[DBELL_DOVF]. + * + * Must be a multiple of 8. All CPT instructions are 8 words + * and require a doorbell count of multiple of 8. + **/ +#else /* Word 0 - Little Endian */ + uint64_t dbell_cnt : 20; + uint64_t reserved_20_63 : 44; +#endif /* Word 0 - End */ + } s; +} cptx_vqx_doorbell_t; + +/** + * Register (NCB) cpt#_vq#_inprog + * + * CPT Queue In Progress Count Registers + * These registers contain the per-queue instruction in flight registers. + */ +typedef union { + uint64_t u; + struct cptx_vqx_inprog_s { +#if (RTE_BYTE_ORDER == RTE_BIG_ENDIAN) /* Word 0 - Big Endian */ + uint64_t reserved_8_63 : 56; + /** [ 7: 0](RO/H) Inflight count. Counts the number of + * instructions for the VF for which CPT is fetching, executing + * or responding to instructions. However this does not include + * any interrupts that are awaiting software handling + * (CPT()_VQ()_DONE[DONE] != 0x0). + * + * A queue may not be reconfigured until: + * 1. CPT()_VQ()_CTL[ENA] is cleared by software. + * 2. [INFLIGHT] is polled until equals to zero. + **/ + uint64_t inflight : 8; +#else /* Word 0 - Little Endian */ + uint64_t inflight : 8; + uint64_t reserved_8_63 : 56; +#endif /* Word 0 - End */ + } s; +} cptx_vqx_inprog_t; + +/** + * Register (NCB) cpt#_vq#_misc_int + * + * CPT Queue Misc Interrupt Register + * These registers contain the per-queue miscellaneous interrupts. + */ +typedef union { + uint64_t u; + struct cptx_vqx_misc_int_s { +#if (RTE_BYTE_ORDER == RTE_BIG_ENDIAN) /* Word 0 - Big Endian */ + uint64_t reserved_7_63 : 57; + /** [ 6: 6](R/W1C/H) Translation fault detected. */ + uint64_t fault : 1; + /** [ 5: 5](R/W1C/H) Hardware error from engines. */ + uint64_t hwerr : 1; + /** [ 4: 4](R/W1C/H) Software error from engines. */ + uint64_t swerr : 1; + /** [ 3: 3](R/W1C/H) NCB result write response error. */ + uint64_t nwrp : 1; + /** [ 2: 2](R/W1C/H) Instruction NCB read response error. */ + uint64_t irde : 1; + /** [ 1: 1](R/W1C/H) Doorbell overflow. */ + uint64_t dovf : 1; + /** [ 0: 0](R/W1C/H) PF to VF mailbox interrupt. Set when + * CPT()_VF()_PF_MBOX(0) is written. + **/ + uint64_t mbox : 1; +#else /* Word 0 - Little Endian */ + uint64_t mbox : 1; + uint64_t dovf : 1; + uint64_t irde : 1; + uint64_t nwrp : 1; + uint64_t swerr : 1; + uint64_t hwerr : 1; + uint64_t fault : 1; + uint64_t reserved_5_63 : 59; +#endif /* Word 0 - End */ + } s; +} cptx_vqx_misc_int_t; + +/** + * Register (NCB) cpt#_vq#_saddr + * + * CPT Queue Starting Buffer Address Registers + * These registers set the instruction buffer starting address. + */ +typedef union { + uint64_t u; + struct cptx_vqx_saddr_s { +#if (RTE_BYTE_ORDER == RTE_BIG_ENDIAN) /* Word 0 - Big Endian */ + uint64_t reserved_49_63 : 15; + /** [ 48: 6](R/W/H) Instruction buffer IOVA <48:6> + * (64-byte aligned). When written, it is the initial buffer + * starting address; when read, it is the next read pointer to + * be requested from L2C. The PTR field is overwritten with the + * next pointer each time that the command buffer segment is + * exhausted. New commands will then be read from the newly + * specified command buffer pointer. + **/ + uint64_t ptr : 43; + uint64_t reserved_0_5 : 6; +#else /* Word 0 - Little Endian */ + uint64_t reserved_0_5 : 6; + uint64_t ptr : 43; + uint64_t reserved_49_63 : 15; +#endif /* Word 0 - End */ + } s; +} cptx_vqx_saddr_t; + +#endif /*_CPT_HW_TYPES_H_ */ diff --git a/drivers/crypto/octeontx/otx_cryptodev_hw_access.c b/drivers/crypto/octeontx/otx_cryptodev_hw_access.c index 99fe3cf..369d62b 100644 --- a/drivers/crypto/octeontx/otx_cryptodev_hw_access.c +++ b/drivers/crypto/octeontx/otx_cryptodev_hw_access.c @@ -3,11 +3,19 @@ */ #include +#include #include #include "otx_cryptodev_hw_access.h" #include "cpt_pmd_logs.h" +#include "cpt_hw_types.h" + +/* + * VF HAL functions + * Access its own BAR0/4 registers by passing VF number as 0. + * OS/PCI maps them accordingly. + */ static int otx_cpt_vf_init(struct cpt_vf *cptvf) @@ -19,10 +27,192 @@ otx_cpt_vf_init(struct cpt_vf *cptvf) return ret; } +/* + * Read Interrupt status of the VF + * + * @param cptvf cptvf structure + */ +static uint64_t +otx_cpt_read_vf_misc_intr_status(struct cpt_vf *cptvf) +{ + return CPT_READ_CSR(CPT_CSR_REG_BASE(cptvf), CPTX_VQX_MISC_INT(0, 0)); +} + +/* + * Clear mailbox interrupt of the VF + * + * @param cptvf cptvf structure + */ +static void +otx_cpt_clear_mbox_intr(struct cpt_vf *cptvf) +{ + cptx_vqx_misc_int_t vqx_misc_int; + + vqx_misc_int.u = CPT_READ_CSR(CPT_CSR_REG_BASE(cptvf), + CPTX_VQX_MISC_INT(0, 0)); + /* W1C for the VF */ + vqx_misc_int.s.mbox = 1; + CPT_WRITE_CSR(CPT_CSR_REG_BASE(cptvf), + CPTX_VQX_MISC_INT(0, 0), vqx_misc_int.u); +} + +/* + * Clear instruction NCB read error interrupt of the VF + * + * @param cptvf cptvf structure + */ +static void +otx_cpt_clear_irde_intr(struct cpt_vf *cptvf) +{ + cptx_vqx_misc_int_t vqx_misc_int; + + vqx_misc_int.u = CPT_READ_CSR(CPT_CSR_REG_BASE(cptvf), + CPTX_VQX_MISC_INT(0, 0)); + /* W1C for the VF */ + vqx_misc_int.s.irde = 1; + CPT_WRITE_CSR(CPT_CSR_REG_BASE(cptvf), + CPTX_VQX_MISC_INT(0, 0), vqx_misc_int.u); +} + +/* + * Clear NCB result write response error interrupt of the VF + * + * @param cptvf cptvf structure + */ +static void +otx_cpt_clear_nwrp_intr(struct cpt_vf *cptvf) +{ + cptx_vqx_misc_int_t vqx_misc_int; + + vqx_misc_int.u = CPT_READ_CSR(CPT_CSR_REG_BASE(cptvf), + CPTX_VQX_MISC_INT(0, 0)); + /* W1C for the VF */ + vqx_misc_int.s.nwrp = 1; + CPT_WRITE_CSR(CPT_CSR_REG_BASE(cptvf), + CPTX_VQX_MISC_INT(0, 0), vqx_misc_int.u); +} + +/* + * Clear swerr interrupt of the VF + * + * @param cptvf cptvf structure + */ +static void +otx_cpt_clear_swerr_intr(struct cpt_vf *cptvf) +{ + cptx_vqx_misc_int_t vqx_misc_int; + + vqx_misc_int.u = CPT_READ_CSR(CPT_CSR_REG_BASE(cptvf), + CPTX_VQX_MISC_INT(0, 0)); + /* W1C for the VF */ + vqx_misc_int.s.swerr = 1; + CPT_WRITE_CSR(CPT_CSR_REG_BASE(cptvf), + CPTX_VQX_MISC_INT(0, 0), vqx_misc_int.u); +} + +/* + * Clear hwerr interrupt of the VF + * + * @param cptvf cptvf structure + */ +static void +otx_cpt_clear_hwerr_intr(struct cpt_vf *cptvf) +{ + cptx_vqx_misc_int_t vqx_misc_int; + + vqx_misc_int.u = CPT_READ_CSR(CPT_CSR_REG_BASE(cptvf), + CPTX_VQX_MISC_INT(0, 0)); + /* W1C for the VF */ + vqx_misc_int.s.hwerr = 1; + CPT_WRITE_CSR(CPT_CSR_REG_BASE(cptvf), + CPTX_VQX_MISC_INT(0, 0), vqx_misc_int.u); +} + +/* + * Clear translation fault interrupt of the VF + * + * @param cptvf cptvf structure + */ +static void +otx_cpt_clear_fault_intr(struct cpt_vf *cptvf) +{ + cptx_vqx_misc_int_t vqx_misc_int; + + vqx_misc_int.u = CPT_READ_CSR(CPT_CSR_REG_BASE(cptvf), + CPTX_VQX_MISC_INT(0, 0)); + /* W1C for the VF */ + vqx_misc_int.s.fault = 1; + CPT_WRITE_CSR(CPT_CSR_REG_BASE(cptvf), + CPTX_VQX_MISC_INT(0, 0), vqx_misc_int.u); +} + +/* + * Clear doorbell overflow interrupt of the VF + * + * @param cptvf cptvf structure + */ +static void +otx_cpt_clear_dovf_intr(struct cpt_vf *cptvf) +{ + cptx_vqx_misc_int_t vqx_misc_int; + + vqx_misc_int.u = CPT_READ_CSR(CPT_CSR_REG_BASE(cptvf), + CPTX_VQX_MISC_INT(0, 0)); + /* W1C for the VF */ + vqx_misc_int.s.dovf = 1; + CPT_WRITE_CSR(CPT_CSR_REG_BASE(cptvf), + CPTX_VQX_MISC_INT(0, 0), vqx_misc_int.u); +} + void otx_cpt_poll_misc(struct cpt_vf *cptvf) { - RTE_SET_USED(cptvf); + uint64_t intr; + + intr = otx_cpt_read_vf_misc_intr_status(cptvf); + + if (!intr) + return; + + /* Check for MISC interrupt types */ + if (likely(intr & CPT_VF_INTR_MBOX_MASK)) { + CPT_LOG_DP_DEBUG("%s: Mailbox interrupt 0x%lx on CPT VF %d", + cptvf->dev_name, (unsigned int long)intr, cptvf->vfid); + otx_cpt_clear_mbox_intr(cptvf); + } else if (unlikely(intr & CPT_VF_INTR_IRDE_MASK)) { + otx_cpt_clear_irde_intr(cptvf); + CPT_LOG_DP_DEBUG("%s: Instruction NCB read error interrupt " + "0x%lx on CPT VF %d", cptvf->dev_name, + (unsigned int long)intr, cptvf->vfid); + } else if (unlikely(intr & CPT_VF_INTR_NWRP_MASK)) { + otx_cpt_clear_nwrp_intr(cptvf); + CPT_LOG_DP_DEBUG("%s: NCB response write error interrupt 0x%lx" + " on CPT VF %d", cptvf->dev_name, + (unsigned int long)intr, cptvf->vfid); + } else if (unlikely(intr & CPT_VF_INTR_SWERR_MASK)) { + otx_cpt_clear_swerr_intr(cptvf); + CPT_LOG_DP_DEBUG("%s: Software error interrupt 0x%lx on CPT VF " + "%d", cptvf->dev_name, (unsigned int long)intr, + cptvf->vfid); + } else if (unlikely(intr & CPT_VF_INTR_HWERR_MASK)) { + otx_cpt_clear_hwerr_intr(cptvf); + CPT_LOG_DP_DEBUG("%s: Hardware error interrupt 0x%lx on CPT VF " + "%d", cptvf->dev_name, (unsigned int long)intr, + cptvf->vfid); + } else if (unlikely(intr & CPT_VF_INTR_FAULT_MASK)) { + otx_cpt_clear_fault_intr(cptvf); + CPT_LOG_DP_DEBUG("%s: Translation fault interrupt 0x%lx on CPT VF " + "%d", cptvf->dev_name, (unsigned int long)intr, + cptvf->vfid); + } else if (unlikely(intr & CPT_VF_INTR_DOVF_MASK)) { + otx_cpt_clear_dovf_intr(cptvf); + CPT_LOG_DP_DEBUG("%s: Doorbell overflow interrupt 0x%lx on CPT VF " + "%d", cptvf->dev_name, (unsigned int long)intr, + cptvf->vfid); + } else + CPT_LOG_DP_ERR("%s: Unhandled interrupt 0x%lx in CPT VF %d", + cptvf->dev_name, (unsigned int long)intr, + cptvf->vfid); } int diff --git a/drivers/crypto/octeontx/otx_cryptodev_hw_access.h b/drivers/crypto/octeontx/otx_cryptodev_hw_access.h index 1e1877c..73473ed 100644 --- a/drivers/crypto/octeontx/otx_cryptodev_hw_access.h +++ b/drivers/crypto/octeontx/otx_cryptodev_hw_access.h @@ -7,6 +7,7 @@ #include +#include #include #include "cpt_common.h" @@ -16,6 +17,16 @@ /* Default command queue length */ #define DEFAULT_CMD_QCHUNKS 2 +#define CPT_CSR_REG_BASE(cpt) ((cpt)->reg_base) + +/* Read hw register */ +#define CPT_READ_CSR(__hw_addr, __offset) \ + rte_read64_relaxed((uint8_t *)__hw_addr + __offset) + +/* Write hw register */ +#define CPT_WRITE_CSR(__hw_addr, __offset, __val) \ + rte_write64_relaxed((__val), ((uint8_t *)__hw_addr + __offset)) + /* cpt instance */ struct cpt_instance { uint32_t queue_id; -- 2.7.4