From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from NAM02-SN1-obe.outbound.protection.outlook.com (mail-sn1nam02on0050.outbound.protection.outlook.com [104.47.36.50]) by dpdk.org (Postfix) with ESMTP id BFD6F5F17 for ; Fri, 8 Jun 2018 18:48:54 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=CAVIUMNETWORKS.onmicrosoft.com; s=selector1-cavium-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=5lCTEv37vxRNquAWLPZve66DT8tfxb7QA10irfWlaDo=; b=DTcahU0TnEbypu/JT4HAbdac/uHQ+rcwPrqRCpw3f4psbxqfKXZgKQ8B2NuGuYgg8SHL6/C6d5AoggfW1S8vUZsYXCQWAf/NV7pG7Xm8wCpLVSHHzaVlabHonxZpMbajz81BJFcIthD7Vq0p+OHw/JXpgrM+TyqTExkFI6ghKAk= Authentication-Results: spf=none (sender IP is ) smtp.mailfrom=Anoob.Joseph@cavium.com; Received: from ajoseph83.caveonetworks.com.caveonetworks.com (115.113.156.2) by SN6PR07MB4911.namprd07.prod.outlook.com (2603:10b6:805:3c::29) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.820.15; Fri, 8 Jun 2018 16:48:48 +0000 From: Anoob Joseph To: Akhil Goyal , Pablo de Lara , Thomas Monjalon Cc: Nithin Dabilpuram , Ankur Dwivedi , Jerin Jacob , Murthy NSSR , Narayana Prasad , Ragothaman Jayaraman , Srisivasubramanian Srinivasan , dev@dpdk.org Date: Fri, 8 Jun 2018 22:15:12 +0530 Message-Id: <1528476325-15585-4-git-send-email-anoob.joseph@caviumnetworks.com> X-Mailer: git-send-email 2.7.4 In-Reply-To: <1528476325-15585-1-git-send-email-anoob.joseph@caviumnetworks.com> References: <1528476325-15585-1-git-send-email-anoob.joseph@caviumnetworks.com> MIME-Version: 1.0 Content-Type: text/plain X-Originating-IP: [115.113.156.2] X-ClientProxiedBy: BM1PR01CA0071.INDPRD01.PROD.OUTLOOK.COM (2603:1096:b00:1::11) To SN6PR07MB4911.namprd07.prod.outlook.com (2603:10b6:805:3c::29) X-MS-PublicTrafficType: Email X-Microsoft-Antispam: UriScan:; BCL:0; PCL:0; RULEID:(7020095)(4652020)(5600026)(4534165)(7168020)(4627221)(201703031133081)(201702281549075)(2017052603328)(7153060)(7193020); SRVR:SN6PR07MB4911; X-Microsoft-Exchange-Diagnostics: 1; SN6PR07MB4911; 3:brY/XMZg0P2bUdUeD3e7Lh1KMpxBqlCA4lotgTmrhTTQjPchKtVrL/LrQri1Vd1W2X/X5FVSrVBIySjy+RrDIxDNly8thb16p/M1qDSJg1YWagVbjlN8cgy4IxYSIznZKTUPweMMmr59T9YQRpEJzDQhJkvUqUIhGZY1/jVRq/waPG/AYoHmmFBkb7FwjPsEj30q4flKATIod4E1bOZ0Dh9RsT+z5RIKpK9ZcDn+BXV6H+dXPM42DT9vudJ/Ie1B; 25:eofc0vYiF2SiPf6rI+Gw07XCN/RDi8HhKSzxF27n22MFLPirEZIL8oATjhEzPH0Fs5uzan40y51cqYyVGOfhiSx8ANzV5XKV5RFMtu2dyGdMrxuxyLA0FN1t/KMyFnXGo5cz4fMIW9LP06yqeHW8M6hmkYs0cRBX+R0mZeGFYJtu1SR0aw2pfy6HJFT5y6c/xACNO+HIZw1+JmuX9rSsBpz1v1V7ao8mx766r3CMQ57y45CUx0q00bACAf6qNcWVwdREIsWa2Mw2PblAwAxgfBr3QcWSz7sPM0cBkUlt+k+2qD5fopkxGbZKVAm3eJgYIHFXJfpVF+t4aTCAiNiLXA==; 31:4MDvkHC34X3B3j12sVFhdrWUAq+C3rK2C9NF67y7fKda0ehA0snRtfzBpnHPc+vJS2ejhkysap3Ug7XKf5PE7z0yWJt0iXaB/gxQ00EA9q+B5PJOVofHF9LHl4hiR1goSgc6JVGnd0wVSxsjEP7bdE6j29wS3QvWw+jEtmj4MMxdJ5+hFx2GrmjtFBaSL4YNdK8Wus3k8lEP56lXxBQOoLvcUD//R0Gy4pggxl3r7KU= X-MS-TrafficTypeDiagnostic: SN6PR07MB4911: X-Microsoft-Exchange-Diagnostics: 1; SN6PR07MB4911; 20:SGgiLFZuhVI4xJ8gtPoty1k/Is7kDDXglq4dyEmUtWwt+Z83KwdV/SZkCcUJZ7CJut1yvtnqCsDU0A5CePzhBmOgfBuzJMQ4idXbqz2mAObchff7tyK2ZeM5aRdZ5/pTo6gVEmkbCFVA3C41vj4bV8bo+uAibquyb7Ul8nMPhJMVdAMXXuNXR38fHhrcMFf5wC587wNqtqUaKn2xbUbfe5HtItqAvMADamTAfTqny+ae1L8lbaTnp9ECONPgCSan4Wd7XZleF3e7G/Gv36MFxzVwaOMRzOYb24dSDzIC/EzoT5gF+tlG3szOMqRe+7ViwuzmdgJbVR3unnrHv9+fjU8zYWOhJQ08nRYE2vl83pBhz26quOyKuATNxr5Y1+9TdE7dfIn3DGXLyjPwFE9WtU/jVMAAMvhJuwUQKSTf437VTecF+hpnCszWe6Mnx0n4x5Pea6mIQ/HpnzTn2OFLqdIMvBUcx3iB+hxtL3WK30v9k1NzJ/z+oIRtrY3YkeM3sgQZB+NyKvmn7Amuxou2zF4qM8TsSXZegNcN6+TYNzBxQnNg36WPzig6eLTs3yns8KrpKMXkoe0aAdzBERrvSWRSzKyBtO7VfAm4+WQlBKU=; 4:neLkQEnvHbuukE90aOaufyEuSiUQj5iUZeS7uhtkZ2Wb4F+KQ1H++hZlAVYuN3mO8MHnBhP/AdWnBIRk73kvYRtVer4qZ2Ih24njJ3v1kiJwUEYxFpbP9S67+DTmpZsg4V+j1/X1/QTQ5XbBeo6LWOQYMvl1eK+CCGY1bcSCAgnnVJcaWY6knN214knENhwPbHnuwCacbiApAcJ2xVn8LpqaxKRQoAycjCzXrPc2UcnaQSCWH8fLMyOat+/Y21BxaNfb9IotcORwOw2yt97XVg== X-Microsoft-Antispam-PRVS: X-Exchange-Antispam-Report-Test: UriScan:; X-MS-Exchange-SenderADCheck: 1 X-Exchange-Antispam-Report-CFA-Test: BCL:0; PCL:0; RULEID:(8211001083)(6040522)(2401047)(8121501046)(5005006)(10201501046)(93006095)(3002001)(3231254)(944501410)(52105095)(149027)(150027)(6041310)(201703131423095)(201702281528075)(20161123555045)(201703061421075)(201703061406153)(20161123560045)(20161123562045)(20161123558120)(20161123564045)(6072148)(201708071742011)(7699016); SRVR:SN6PR07MB4911; BCL:0; PCL:0; RULEID:; SRVR:SN6PR07MB4911; X-Forefront-PRVS: 06973FFAD3 X-Forefront-Antispam-Report: SFV:NSPM; SFS:(10009020)(376002)(396003)(346002)(39380400002)(366004)(39860400002)(189003)(199004)(26005)(52116002)(956004)(305945005)(8676002)(81156014)(50226002)(8936002)(81166006)(7736002)(68736007)(105586002)(106356001)(3846002)(6116002)(72206003)(478600001)(6486002)(97736004)(6506007)(16586007)(6512007)(53946003)(53936002)(47776003)(486006)(25786009)(5660300001)(50466002)(48376002)(6666003)(4326008)(54906003)(76176011)(36756003)(110136005)(66066001)(8656006)(2906002)(316002)(55236004)(11346002)(16526019)(386003)(186003)(59450400001)(446003)(476003)(42882007)(44832011)(2616005)(51416003); DIR:OUT; SFP:1101; SCL:1; SRVR:SN6PR07MB4911; H:ajoseph83.caveonetworks.com.caveonetworks.com; FPR:; SPF:None; LANG:en; PTR:InfoNoRecords; MX:1; A:1; Received-SPF: None (protection.outlook.com: cavium.com does not designate permitted sender hosts) X-Microsoft-Exchange-Diagnostics: =?us-ascii?Q?1; SN6PR07MB4911; 23:UA1OPyd53Z34ZJcCqcVBxdWwh/R0jBKBeAB99uU3s?= =?us-ascii?Q?ClVcOR47clFbTgbQi0k/ppSpg8Jcz6JAACXV0Jp3DcRk0d4CE4T5U/tqP1Af?= =?us-ascii?Q?h/VUcnsmPFjkK8ywtGuadU074p5gipCjWYPmdhwA7vyM2ieftwDX4eimt3h9?= =?us-ascii?Q?kV3RwOLC61FtVq80bZD4wUbjQeU/Z5MFwxsTA+23RrBs7vOkuRj0nDL+pj6g?= =?us-ascii?Q?vQY3u3o3mETiHLEqwU6Y8dTS17r0NHXw8Sv2SSxcsycxC72vLYWs5Ft2R3pJ?= =?us-ascii?Q?it3En/JFI7J3m5AgIHrjt896txaYsT6d5TDI4o49RBAyrZHthsCA9fZYeSsC?= =?us-ascii?Q?TsN0/xSJ9HumakkYzDYIOwyMWPgRKdoz+W8oc5JCSprlmZcRT38DPR7k+aZk?= =?us-ascii?Q?6N7DinwArxDeN2pGybesEAYiODZrAVaR2qM4Qts1dO47+YP/GhifIJwdXvT2?= =?us-ascii?Q?wSdWK2v40bgXSfd0CEBnequ2FgsLbuXwa3X2CJ7oUMBfxmRIVblq/cin1ixK?= =?us-ascii?Q?q3lZTFyLSTi8xsrjsgL+PD3ZFzfJmvU9Uw4pnuItenBqJhcqTpWKob47F++S?= =?us-ascii?Q?vnGFhC3vVjdqRmyoujKPPj3VvCWNDBuXm7HpIz0G1lj/3YNaDeK7sig2ilC4?= =?us-ascii?Q?EO/sUFdlwE7oFiUQtFLJn7zLbo3XMHmP5idJbaiAWWddez5abzBZKXIMFhxU?= =?us-ascii?Q?MQB9GezoO8tbBrrIPEOFMzXvQJ5jWtfqQzZb7dNoQNUqq++5J7qkZRqQi2wZ?= =?us-ascii?Q?HFRaGHHsl6MpVDyfwahyvWf36Nic0lTYgOQXkw8fu4BDil/M3PIMu8rCV1HV?= =?us-ascii?Q?z702snJw8LYiH+WsaZ9Lc5jDRf9TqRUUtTvFKargHI/eo4A5l2mCwMyKMM+P?= =?us-ascii?Q?8ZwBuFoB41jPS6SBFtWS6d9/gyirLDiOUr8f+za67F8YLTVbYILLfzjiZepS?= =?us-ascii?Q?FOwwySU9LFmqz9ygtQxQwl+PB5CEW3UzIJF5kEIXKZQzn0032X50FqiPsomf?= =?us-ascii?Q?LfRHq7MFEjx28CW5hou6qf/VdQt5mFjXtaWGOf/8aAL3FeuLgFPl+lKxinQt?= =?us-ascii?Q?bahdXL8Hk3kFzN1xD4WUINseBtTTHaRREuuMHgmeFLeyxtERyjndiRi7fLkL?= =?us-ascii?Q?fB+px76YjlasLrXAY8S+iyALDGVMCwkdaQOW53GWbm3bJNSKbWNj8clfTea/?= =?us-ascii?Q?v+66W8RgBhbvZ0HXJuO1JwCWSh2oI65GQupEScjM0epJlyltPJaSS7X5oNnI?= =?us-ascii?Q?X3FIFNcN/6cG3nU5Bfi4wjm1zMxZCEZEFC0CY90bKDcpgYQhHg5f8pBHjDZa?= =?us-ascii?Q?6t8UdPloerw0OkrfKgUAFT+OH7Aqfiwkva2uuUVPtKr?= X-Microsoft-Antispam-Message-Info: yuwl3DZXExsw2tTbL7LONeN5lPV9XAXjf2h+9J/VjzbL7tbPLBcp9mlFo4eoIx4ZVRsGxwzSoANuQmHT+4TMHK3izy7SOFr7OThV5f8qDHYKbuo4BZNwvN3IXhwywB349vPOQQyL/aQICkQckPjTetAd0Q4bKmBtdrrn7enscaNs5y3v6kpyy5ESayC81pno X-Microsoft-Exchange-Diagnostics: 1; SN6PR07MB4911; 6:GJrZJDyZNfzGBWZ7P9v/4J7yjS47qahsed57hT2r4ObMF1oIQVbxLgK5ImYTleCBvwQycLmiP9UJbqH8+ez+shMqr/i7veP+i9WZwDMuCZVEWVMUrKMgyYt2ejOyvYIPJoJOH++mnc5t/POZPbABrcmWxhs0+lXiGunYVPqlCRL6zs8UJpz1OYTtb+GqXMIO+vmqG3zZF7IGcpAqbeWpMKYo22q42fYNieUVWFrICqhw8qgaCM/r2G19HhsfbdHZTKhrkh1ayZrDHA5MCGp88hmsx9a0kGXAMqD5iu1+wM2y4QFc206tDepgPGswW1lnw0TmycsPtIRQ1nM00XYU7wiZplWVG5DZJdUQPi8YM6QWPy7DjYdOfbvmOO1p7TyGLUQUt2c+yGAPmzKufydLLfChkxBTVvL2siO3AodJC2HSW+ILvP7jAqnFrkkquwiopJE95XNeiPNM7k0xS/ihpw==; 5:b3Zu8jSSymmm4dlv7vk+PikJtFYEIw5YVauXPh9cEDalSpHGLdZ7oQyu6pY29VfrAWdwgI9QforKaMW7XJo5iUS+wLeDQtdcAW/Zg7h21Gox+B39Z2DkAjXUZdHQP4cde2OW6ivrsidt0ZR6OJ7kQKyElR6wtzYNY+dUppCt1wA=; 24:cF1Fwn9FocytQDMJpkeBktT23uLJuShrHzOjGoVBZ8An3gD1odtHTffWRPMjajhfbiiUFRiK5PvLYNDv/+OVWSFTuJCKWAAQWgmGdl5Iq0E= SpamDiagnosticOutput: 1:99 SpamDiagnosticMetadata: NSPM X-Microsoft-Exchange-Diagnostics: 1; SN6PR07MB4911; 7:n7tN3pXrQElkHiRYTJkstxNn+PJ/bhjYXdnx8WpHQNLzFkOZJtFe/2LTaeCib2s987Fzn8ZFTu0OmZvyRmFoABdmGWCQeYtqRI7J+l/KCBDD78FWekHT9XuFe7fHENlFSS/FRyWz87WgYZ1v/Is1MU0PKbZdL2IdTZ7MOF479vGIoFCbAz27NCXXf4QeR8+IxZknIHLCKoI04S8WOZWNIgeOsmI15cQmiY7/sW3g1XoID40xbzVzYFui3qayrTWt X-MS-Office365-Filtering-Correlation-Id: de22e199-9281-45c6-a731-08d5cd5fb753 X-OriginatorOrg: caviumnetworks.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 08 Jun 2018 16:48:48.8768 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: de22e199-9281-45c6-a731-08d5cd5fb753 X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 711e4ccf-2e9b-4bcf-a551-4094005b6194 X-MS-Exchange-Transport-CrossTenantHeadersStamped: SN6PR07MB4911 Subject: [dpdk-dev] [PATCH 03/16] crypto/cpt/base: add hardware initialization API for CPT X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 08 Jun 2018 16:48:55 -0000 From: Nithin Dabilpuram Adds hardware device initialization specific api for Cavium CPT device. Signed-off-by: Ankur Dwivedi Signed-off-by: Murthy NSSR Signed-off-by: Nithin Dabilpuram Signed-off-by: Ragothaman Jayaraman Signed-off-by: Srisivasubramanian Srinivasan --- drivers/crypto/cpt/base/cpt8xxx_device.c | 200 ++++++++++++++++ drivers/crypto/cpt/base/cpt8xxx_device.h | 85 +++++++ drivers/crypto/cpt/base/cpt_debug.h | 231 +++++++++++++++++++ drivers/crypto/cpt/base/cpt_device.c | 383 +++++++++++++++++++++++++++++++ drivers/crypto/cpt/base/cpt_device.h | 162 +++++++++++++ drivers/crypto/cpt/base/cpt_vf_mbox.c | 176 ++++++++++++++ drivers/crypto/cpt/base/cpt_vf_mbox.h | 60 +++++ 7 files changed, 1297 insertions(+) create mode 100644 drivers/crypto/cpt/base/cpt8xxx_device.c create mode 100644 drivers/crypto/cpt/base/cpt8xxx_device.h create mode 100644 drivers/crypto/cpt/base/cpt_debug.h create mode 100644 drivers/crypto/cpt/base/cpt_device.c create mode 100644 drivers/crypto/cpt/base/cpt_device.h create mode 100644 drivers/crypto/cpt/base/cpt_vf_mbox.c create mode 100644 drivers/crypto/cpt/base/cpt_vf_mbox.h diff --git a/drivers/crypto/cpt/base/cpt8xxx_device.c b/drivers/crypto/cpt/base/cpt8xxx_device.c new file mode 100644 index 0000000..cdce96f --- /dev/null +++ b/drivers/crypto/cpt/base/cpt8xxx_device.c @@ -0,0 +1,200 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright(c) 2017 Cavium, Inc + */ + +#include "cpt8xxx_device.h" + +/* + * VF HAL functions + * Access its own BAR0/4 registers by passing VF number as 0. + * OS/PCI maps them accordingly. + */ + +/* Send a mailbox message to PF + * @vf: vf from which this message to be sent + * @mbx: Message to be sent + */ +void cptvf_send_msg_to_pf(struct cpt_vf *cptvf, cpt_mbox_t *mbx) +{ + /* Writing mbox(1) causes interrupt */ + CPT_WRITE_CSR(CPT_CSR_REG_BASE(cptvf), + CPTX_VFX_PF_MBOXX(0, 0, 0), mbx->msg); + CPT_WRITE_CSR(CPT_CSR_REG_BASE(cptvf), + CPTX_VFX_PF_MBOXX(0, 0, 1), mbx->data); +} + +/* Read Interrupt status of the VF + * @vf: vf number + */ +uint64_t cptvf_read_vf_misc_intr_status(struct cpt_vf *cptvf) +{ + return CPT_READ_CSR(CPT_CSR_REG_BASE(cptvf), CPTX_VQX_MISC_INT(0, 0)); +} + +/* Clear mailbox interrupt of the VF + * @vf: vf number + */ +void cptvf_clear_mbox_intr(struct cpt_vf *cptvf) +{ + cptx_vqx_misc_int_t vqx_misc_int; + + vqx_misc_int.u = CPT_READ_CSR(CPT_CSR_REG_BASE(cptvf), + CPTX_VQX_MISC_INT(0, 0)); + /* W1C for the VF */ + vqx_misc_int.s.mbox = 1; + CPT_WRITE_CSR(CPT_CSR_REG_BASE(cptvf), + CPTX_VQX_MISC_INT(0, 0), vqx_misc_int.u); +} + +/* Clear swerr interrupt of the VF + * @vf: vf number + */ +void cptvf_clear_swerr_intr(struct cpt_vf *cptvf) +{ + cptx_vqx_misc_int_t vqx_misc_int; + + vqx_misc_int.u = CPT_READ_CSR(CPT_CSR_REG_BASE(cptvf), + CPTX_VQX_MISC_INT(0, 0)); + /* W1C for the VF */ + vqx_misc_int.s.swerr = 1; + CPT_WRITE_CSR(CPT_CSR_REG_BASE(cptvf), + CPTX_VQX_MISC_INT(0, 0), vqx_misc_int.u); +} + +/* Clear doorbell overflow interrupt of the VF + * @vf: vf number + */ +void cptvf_clear_dovf_intr(struct cpt_vf *cptvf) +{ + cptx_vqx_misc_int_t vqx_misc_int; + + vqx_misc_int.u = CPT_READ_CSR(CPT_CSR_REG_BASE(cptvf), + CPTX_VQX_MISC_INT(0, 0)); + /* W1C for the VF */ + vqx_misc_int.s.dovf = 1; + CPT_WRITE_CSR(CPT_CSR_REG_BASE(cptvf), + CPTX_VQX_MISC_INT(0, 0), vqx_misc_int.u); +} + +/* Clear instruction NCB read error interrupt of the VF + * @vf: vf number + */ +void cptvf_clear_irde_intr(struct cpt_vf *cptvf) +{ + cptx_vqx_misc_int_t vqx_misc_int; + + vqx_misc_int.u = CPT_READ_CSR(CPT_CSR_REG_BASE(cptvf), + CPTX_VQX_MISC_INT(0, 0)); + /* W1C for the VF */ + vqx_misc_int.s.irde = 1; + CPT_WRITE_CSR(CPT_CSR_REG_BASE(cptvf), + CPTX_VQX_MISC_INT(0, 0), vqx_misc_int.u); +} + +/* Clear NCB result write response error interrupt of the VF + * @vf: vf number + */ +void cptvf_clear_nwrp_intr(struct cpt_vf *cptvf) +{ + cptx_vqx_misc_int_t vqx_misc_int; + + vqx_misc_int.u = CPT_READ_CSR(CPT_CSR_REG_BASE(cptvf), + CPTX_VQX_MISC_INT(0, 0)); + /* W1C for the VF */ + vqx_misc_int.s.nwrp = 1; + CPT_WRITE_CSR(CPT_CSR_REG_BASE(cptvf), + CPTX_VQX_MISC_INT(0, 0), vqx_misc_int.u); +} + +/* Clear hwerr interrupt of the VF + * @vf: vf number + */ +void cptvf_clear_hwerr_intr(struct cpt_vf *cptvf) +{ + cptx_vqx_misc_int_t vqx_misc_int; + + vqx_misc_int.u = CPT_READ_CSR(CPT_CSR_REG_BASE(cptvf), + CPTX_VQX_MISC_INT(0, 0)); + /* W1C for the VF */ + vqx_misc_int.s.hwerr = 1; + CPT_WRITE_CSR(CPT_CSR_REG_BASE(cptvf), + CPTX_VQX_MISC_INT(0, 0), vqx_misc_int.u); +} + +/* Clear translation fault interrupt of the VF + * @vf: vf number + */ +void cptvf_clear_fault_intr(struct cpt_vf *cptvf) +{ + cptx_vqx_misc_int_t vqx_misc_int; + + vqx_misc_int.u = CPT_READ_CSR(CPT_CSR_REG_BASE(cptvf), + CPTX_VQX_MISC_INT(0, 0)); + /* W1C for the VF */ + vqx_misc_int.s.fault = 1; + CPT_WRITE_CSR(CPT_CSR_REG_BASE(cptvf), + CPTX_VQX_MISC_INT(0, 0), vqx_misc_int.u); +} + +/* Write to VQX_CTL register + */ +void cptvf_write_vq_ctl(struct cpt_vf *cptvf, bool val) +{ + cptx_vqx_ctl_t vqx_ctl; + + vqx_ctl.u = CPT_READ_CSR(CPT_CSR_REG_BASE(cptvf), + CPTX_VQX_CTL(0, 0)); + vqx_ctl.s.ena = val; + CPT_WRITE_CSR(CPT_CSR_REG_BASE(cptvf), + CPTX_VQX_CTL(0, 0), vqx_ctl.u); +} + +/* Write to VQX_INPROG register + */ +void cptvf_write_vq_inprog(struct cpt_vf *cptvf, uint8_t val) +{ + cptx_vqx_inprog_t vqx_inprg; + + vqx_inprg.u = CPT_READ_CSR(CPT_CSR_REG_BASE(cptvf), + CPTX_VQX_INPROG(0, 0)); + vqx_inprg.s.inflight = val; + CPT_WRITE_CSR(CPT_CSR_REG_BASE(cptvf), + CPTX_VQX_INPROG(0, 0), vqx_inprg.u); +} + +/* Write to VQX_DONE_WAIT NUMWAIT register + */ +void cptvf_write_vq_done_numwait(struct cpt_vf *cptvf, uint32_t val) +{ + cptx_vqx_done_wait_t vqx_dwait; + + vqx_dwait.u = CPT_READ_CSR(CPT_CSR_REG_BASE(cptvf), + CPTX_VQX_DONE_WAIT(0, 0)); + vqx_dwait.s.num_wait = val; + CPT_WRITE_CSR(CPT_CSR_REG_BASE(cptvf), + CPTX_VQX_DONE_WAIT(0, 0), vqx_dwait.u); +} + +/* Write to VQX_DONE_WAIT NUM_WAIT register + */ +void cptvf_write_vq_done_timewait(struct cpt_vf *cptvf, uint16_t val) +{ + cptx_vqx_done_wait_t vqx_dwait; + + vqx_dwait.u = CPT_READ_CSR(CPT_CSR_REG_BASE(cptvf), + CPTX_VQX_DONE_WAIT(0, 0)); + vqx_dwait.s.time_wait = val; + CPT_WRITE_CSR(CPT_CSR_REG_BASE(cptvf), + CPTX_VQX_DONE_WAIT(0, 0), vqx_dwait.u); +} + +/* Write to VQX_SADDR register + */ +void cptvf_write_vq_saddr(struct cpt_vf *cptvf, uint64_t val) +{ + cptx_vqx_saddr_t vqx_saddr; + + vqx_saddr.u = val; + CPT_WRITE_CSR(CPT_CSR_REG_BASE(cptvf), + CPTX_VQX_SADDR(0, 0), vqx_saddr.u); +} diff --git a/drivers/crypto/cpt/base/cpt8xxx_device.h b/drivers/crypto/cpt/base/cpt8xxx_device.h new file mode 100644 index 0000000..b7d7dcd --- /dev/null +++ b/drivers/crypto/cpt/base/cpt8xxx_device.h @@ -0,0 +1,85 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright(c) 2017 Cavium, Inc + */ + +#ifndef __CPT81XX_DEVICE_H +#define __CPT81XX_DEVICE_H + +#include "cpt_device.h" +#include "cpt_vf_mbox.h" +/* + * CPT Registers map for 81xx + */ + +/* VF registers */ +#define CPTX_VQX_CTL(a, b) (0x0000100ll + 0x1000000000ll * \ + ((a) & 0x0) + 0x100000ll * (b)) +#define CPTX_VQX_SADDR(a, b) (0x0000200ll + 0x1000000000ll * \ + ((a) & 0x0) + 0x100000ll * (b)) +#define CPTX_VQX_DONE_WAIT(a, b) (0x0000400ll + 0x1000000000ll * \ + ((a) & 0x0) + 0x100000ll * (b)) +#define CPTX_VQX_INPROG(a, b) (0x0000410ll + 0x1000000000ll * \ + ((a) & 0x0) + 0x100000ll * (b)) +#define CPTX_VQX_DONE(a, b) (0x0000420ll + 0x1000000000ll * \ + ((a) & 0x1) + 0x100000ll * (b)) +#define CPTX_VQX_DONE_ACK(a, b) (0x0000440ll + 0x1000000000ll * \ + ((a) & 0x1) + 0x100000ll * (b)) +#define CPTX_VQX_DONE_INT_W1S(a, b) (0x0000460ll + 0x1000000000ll * \ + ((a) & 0x1) + 0x100000ll * (b)) +#define CPTX_VQX_DONE_INT_W1C(a, b) (0x0000468ll + 0x1000000000ll * \ + ((a) & 0x1) + 0x100000ll * (b)) +#define CPTX_VQX_DONE_ENA_W1S(a, b) (0x0000470ll + 0x1000000000ll * \ + ((a) & 0x1) + 0x100000ll * (b)) +#define CPTX_VQX_DONE_ENA_W1C(a, b) (0x0000478ll + 0x1000000000ll * \ + ((a) & 0x1) + 0x100000ll * (b)) +#define CPTX_VQX_MISC_INT(a, b) (0x0000500ll + 0x1000000000ll * \ + ((a) & 0x1) + 0x100000ll * (b)) +#define CPTX_VQX_MISC_INT_W1S(a, b) (0x0000508ll + 0x1000000000ll * \ + ((a) & 0x1) + 0x100000ll * (b)) +#define CPTX_VQX_MISC_ENA_W1S(a, b) (0x0000510ll + 0x1000000000ll * \ + ((a) & 0x1) + 0x100000ll * (b)) +#define CPTX_VQX_MISC_ENA_W1C(a, b) (0x0000518ll + 0x1000000000ll * \ + ((a) & 0x1) + 0x100000ll * (b)) +#define CPTX_VQX_DOORBELL(a, b) (0x0000600ll + 0x1000000000ll * \ + ((a) & 0x1) + 0x100000ll * (b)) +#define CPTX_VFX_PF_MBOXX(a, b, c) (0x0001000ll + 0x1000000000ll * \ + ((a) & 0x1) + 0x100000ll * (b) + \ + 8ll * ((c) & 0x1)) +/* VF HAL functions */ +void cptvf_send_msg_to_pf(struct cpt_vf *cptvf, cpt_mbox_t *mbx); +void cptvf_clear_mbox_intr(struct cpt_vf *cptvf); +void cptvf_clear_swerr_intr(struct cpt_vf *cptvf); +void cptvf_clear_dovf_intr(struct cpt_vf *cptvf); +void cptvf_clear_irde_intr(struct cpt_vf *cptvf); +void cptvf_clear_nwrp_intr(struct cpt_vf *cptvf); +void cptvf_clear_fault_intr(struct cpt_vf *cptvf); +void cptvf_clear_hwerr_intr(struct cpt_vf *cptvf); +void cptvf_write_vq_ctl(struct cpt_vf *cptvf, bool val); +void cptvf_write_vq_saddr(struct cpt_vf *cptvf, uint64_t val); +void cptvf_write_vq_done_timewait(struct cpt_vf *cptvf, uint16_t val); +void cptvf_write_vq_done_numwait(struct cpt_vf *cptvf, uint32_t val); +void cptvf_write_vq_inprog(struct cpt_vf *cptvf, uint8_t val); +uint64_t cptvf_read_vf_misc_intr_status(struct cpt_vf *cptvf); + +/* Write to VQX_DOORBELL register + */ +static inline void cptvf_write_vq_doorbell(struct cpt_vf *cptvf, uint32_t val) +{ + cptx_vqx_doorbell_t vqx_dbell; + + vqx_dbell.u = 0; + vqx_dbell.s.dbell_cnt = val * 8; /* Num of Instructions * 8 words */ + CPT_WRITE_CSR(CPT_CSR_REG_BASE(cptvf), + CPTX_VQX_DOORBELL(0, 0), vqx_dbell.u); +} + +static inline uint32_t cptvf_read_vq_doorbell(struct cpt_vf *cptvf) +{ + cptx_vqx_doorbell_t vqx_dbell; + + vqx_dbell.u = CPT_READ_CSR(CPT_CSR_REG_BASE(cptvf), + CPTX_VQX_DOORBELL(0, 0)); + return vqx_dbell.s.dbell_cnt; +} + +#endif /* __CPT81XX_DEVICE_H */ diff --git a/drivers/crypto/cpt/base/cpt_debug.h b/drivers/crypto/cpt/base/cpt_debug.h new file mode 100644 index 0000000..afa05df --- /dev/null +++ b/drivers/crypto/cpt/base/cpt_debug.h @@ -0,0 +1,231 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright(c) 2017 Cavium, Inc + */ + +#ifndef __CPT_DEBUG_H +#define __CPT_DEBUG_H +#include +#include +#include "cpt_request_mgr.h" +#include + +#ifdef CPT_DEBUG +static inline void * +os_iova2va(phys_addr_t physaddr) +{ + return rte_mem_iova2virt(physaddr); +} + +static inline void __cpt_dump_buffer(const char *prefix_str, + void *buf, size_t len, int rowsize) +{ + size_t i = 0; + unsigned char *ptr = (unsigned char *)buf; + + PRINT("\n%s[%p]", prefix_str, buf); + PRINT("\n%.8lx: ", i); + + if (buf == NULL) { + PRINT("\n!!!NULL ptr\n"); + abort(); + } + + for (i = 0; i < len; i++) { + if (i && !(i % rowsize)) + PRINT("\n%.8lx: ", i); + PRINT("%02x ", ptr[i]); + } + PRINT("\n\n"); +} + +static inline void cpt_dump_buffer(const char *prefix_str, + void *buf, size_t len) +{ + __cpt_dump_buffer(prefix_str, buf, len, 8); +} + +#define cpt_fn_trace(fmt, ...) \ + do { \ + if (msg_req_trace(debug)) \ + cpt_info(fmt, ##__VA_ARGS__); \ + } while (0) + +static inline void dump_cpt_request_info(struct cpt_request_info *req, + cpt_inst_s_t *inst) +{ + vq_cmd_word0_t vq_cmd_w0; + vq_cmd_word3_t vq_cmd_w3; + uint16_t opcode, param1, param2, dlen; + + vq_cmd_w0.u64 = be64toh(inst->s.ei0); + opcode = be16toh(vq_cmd_w0.s.opcode); + param1 = be16toh(vq_cmd_w0.s.param1); + param2 = be16toh(vq_cmd_w0.s.param2); + dlen = be16toh(vq_cmd_w0.s.dlen); + vq_cmd_w3.u64 = inst->s.ei3; + + PRINT("\ncpt Request Info...\n"); + PRINT("\tdma_mode: %u\n", req->dma_mode); + PRINT("\tis_se : %u\n", req->se_req); + PRINT("\tgrp : 0\n"); + + PRINT("\nRequest Info...\n"); + PRINT("\topcode: 0x%0x\n", opcode); + PRINT("\tparam1: 0x%0x\n", param1); + PRINT("\tparam2: 0x%0x\n", param2); + PRINT("\tdlen: %u\n", dlen); + PRINT("\tctx_handle vaddr %p, dma 0x%lx\n", + os_iova2va((uint64_t)vq_cmd_w3.s.cptr), + (uint64_t)vq_cmd_w3.s.cptr); +} + +static inline void +dump_cpt_request_sglist(cpt_inst_s_t *inst, + const char *header, bool data, + bool glist) +{ + int i; + char suffix[64]; + vq_cmd_word0_t vq_cmd_w0; + uint16_t opcode, dlen; + const char *list = glist ? "glist" : "slist"; + + vq_cmd_w0.u64 = be64toh(inst->s.ei0); + opcode = be16toh(vq_cmd_w0.s.opcode); + dlen = be16toh(vq_cmd_w0.s.dlen); + + if (opcode & DMA_MODE) { + uint8_t *in_buffer = os_iova2va(inst->s.ei1); + uint16_t list_cnt, components; + struct sglist_comp *sg_ptr = NULL; + struct { + void *vaddr; + phys_addr_t dma_addr; + uint32_t size; + } list_ptr[MAX_SG_CNT]; + + PRINT("%s: DMA Mode\n", header); + snprintf(suffix, sizeof(suffix), + "DPTR: vaddr %p, dma 0x%lx len %u: ", + in_buffer, inst->s.ei1, dlen); + + cpt_dump_buffer(suffix, + in_buffer, + dlen); + + sg_ptr = (void *)(in_buffer + 8); + list_cnt = be16toh((((uint16_t *)in_buffer)[2])); + if (!glist) { + components = list_cnt / 4; + if (list_cnt % 4) + components++; + sg_ptr += components; + list_cnt = be16toh((((uint16_t *)in_buffer)[3])); + } + PRINT("current %s: %u\n", list, list_cnt); + if (!(list_cnt <= MAX_SG_CNT)) + abort(); + + components = list_cnt / 4; + + for (i = 0; i < components; i++) { + list_ptr[i*4+0].size = be16toh(sg_ptr->u.s.len[0]); + list_ptr[i*4+1].size = be16toh(sg_ptr->u.s.len[1]); + list_ptr[i*4+2].size = be16toh(sg_ptr->u.s.len[2]); + list_ptr[i*4+3].size = be16toh(sg_ptr->u.s.len[3]); + list_ptr[i*4+0].dma_addr = be64toh(sg_ptr->ptr[0]); + list_ptr[i*4+1].dma_addr = be64toh(sg_ptr->ptr[1]); + list_ptr[i*4+2].dma_addr = be64toh(sg_ptr->ptr[2]); + list_ptr[i*4+3].dma_addr = be64toh(sg_ptr->ptr[3]); + + list_ptr[i*4+0].vaddr = + os_iova2va(list_ptr[i*4+0].dma_addr); + list_ptr[i*4+1].vaddr = + os_iova2va(list_ptr[i*4+1].dma_addr); + list_ptr[i*4+2].vaddr = + os_iova2va(list_ptr[i*4+2].dma_addr); + list_ptr[i*4+3].vaddr = + os_iova2va(list_ptr[i*4+3].dma_addr); + sg_ptr++; + } + components = list_cnt % 4; + + switch (components) { + case 3: + list_ptr[i*4+2].size = be16toh(sg_ptr->u.s.len[2]); + list_ptr[i*4+2].dma_addr = be64toh(sg_ptr->ptr[2]); + list_ptr[i*4+2].vaddr = + os_iova2va(list_ptr[i*4+2].dma_addr); + /* fall through */ + case 2: + list_ptr[i*4+1].size = be16toh(sg_ptr->u.s.len[1]); + list_ptr[i*4+1].dma_addr = be64toh(sg_ptr->ptr[1]); + list_ptr[i*4+1].vaddr = + os_iova2va(list_ptr[i*4+1].dma_addr); + /* fall through */ + case 1: + list_ptr[i*4+0].size = be16toh(sg_ptr->u.s.len[0]); + list_ptr[i*4+0].dma_addr = be64toh(sg_ptr->ptr[0]); + list_ptr[i*4+0].vaddr = + os_iova2va(list_ptr[i*4+0].dma_addr); + break; + default: + break; + } + + for (i = 0; i < list_cnt; i++) { + snprintf(suffix, sizeof(suffix), + "%s[%d]: vaddr %p, dma 0x%lx len %u: ", + list, i, list_ptr[i].vaddr, + list_ptr[i].dma_addr, + list_ptr[i].size); + if (data) + cpt_dump_buffer(suffix, + list_ptr[i].vaddr, + list_ptr[i].size); + else + PRINT("%s\n", suffix); + } + } else { + PRINT("%s: Direct Mode\n", header); + + if (glist) { + snprintf(suffix, sizeof(suffix), + "DPTR: vaddr %p, dma 0x%lx len %u: ", + os_iova2va(inst->s.ei1), + inst->s.ei1, dlen); + if (data) + cpt_dump_buffer(suffix, + os_iova2va(inst->s.ei1), + dlen); + else + PRINT("%s\n", suffix); + } else { + snprintf(suffix, sizeof(suffix), + "RPTR: vaddr %p, dma 0x%lx len %u+..: ", + os_iova2va(inst->s.ei2), + inst->s.ei2, dlen); + /* + * In direct mode, we don't have rlen + * to dump exactly, so dump dlen + 32 + */ + if (data) + cpt_dump_buffer(suffix, + os_iova2va(inst->s.ei2), + dlen + 32); + else + PRINT("%s\n", suffix); + } + } +} + + +#else + +#define cpt_dump_buffer(__str, __buf, __len) +#define cpt_fn_trace(fmt, ...) +#define dump_cpt_request_info(req, ist) +#define dump_cpt_request_sglist(ist, header, data, flag) +#endif /* CPT_DEBUG */ + +#endif /* __CPT_DEBUG_H */ diff --git a/drivers/crypto/cpt/base/cpt_device.c b/drivers/crypto/cpt/base/cpt_device.c new file mode 100644 index 0000000..b7cd5b5 --- /dev/null +++ b/drivers/crypto/cpt/base/cpt_device.c @@ -0,0 +1,383 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright(c) 2017 Cavium, Inc + */ + +#include "cpt_device.h" +#include "cpt_debug.h" +#include "cpt8xxx_device.h" +#include "cpt_vf_mbox.h" +#include "cpt_request_mgr.h" + +#include + +static void cptvf_vfvq_init(struct cpt_vf *cptvf) +{ + uint64_t base_addr = 0; + + /* Disable the VQ */ + cptvf_write_vq_ctl(cptvf, 0); + + /* Reset the doorbell */ + cptvf_write_vq_doorbell(cptvf, 0); + /* Clear inflight */ + cptvf_write_vq_inprog(cptvf, 0); + + /* Write VQ SADDR */ + base_addr = (uint64_t)(cptvf->cqueue.chead[0].dma_addr); + cptvf_write_vq_saddr(cptvf, base_addr); + + /* Configure timerhold / coalescence */ + cptvf_write_vq_done_timewait(cptvf, CPT_TIMER_THOLD); + cptvf_write_vq_done_numwait(cptvf, CPT_COUNT_THOLD); + + /* Enable the VQ */ + cptvf_write_vq_ctl(cptvf, 1); + + /* Flag the VF ready */ + cptvf->flags |= CPT_FLAG_DEVICE_READY; +} + +static int cpt_vf_init(struct cpt_vf *cptvf) +{ + int err = -1; + + /* Mark as VF driver */ + cptvf->flags |= CPT_FLAG_VF_DRIVER; + + /* Check ready with PF */ + /* Gets chip ID / device Id from PF if ready */ + err = cptvf_check_pf_ready(cptvf); + if (err) { + PMD_DRV_LOG(ERR, "%s: PF not responding to READY msg\n", + cptvf->dev_name); + err = -EBUSY; + goto cptvf_err; + } + + PMD_DRV_LOG(DEBUG, "%s: cpt_vf_init() done\n", cptvf->dev_name); + return 0; + +cptvf_err: + return err; +} + +static int cpt_vq_init(struct cpt_vf *cptvf, uint8_t group) +{ + int err; + + /* Convey VQ LEN to PF */ + err = cptvf_send_vq_size_msg(cptvf); + if (err) { + PMD_DRV_LOG(ERR, "%s: PF not responding to QLEN msg\n", + cptvf->dev_name); + err = -EBUSY; + goto cleanup; + } + + /* CPT VF device initialization */ + cptvf_vfvq_init(cptvf); + + /* Send msg to PF to assign currnet Q to required group */ + cptvf->vfgrp = group; + err = cptvf_send_vf_to_grp_msg(cptvf, group); + if (err) { + PMD_DRV_LOG(ERR, "%s: PF not responding to VF_GRP msg\n", + cptvf->dev_name); + err = -EBUSY; + goto cleanup; + } + + PMD_DRV_LOG(DEBUG, "%s: cpt_vq_init() done\n", cptvf->dev_name); + return 0; + +cleanup: + return err; +} + +void cptvf_poll_misc(void *dev) +{ + uint64_t intr; + struct cpt_vf *cptvf = dev; + + intr = cptvf_read_vf_misc_intr_status(cptvf); + + if (!intr) + return; + + /*Check for MISC interrupt types*/ + if (likely(intr & CPT_VF_INTR_MBOX_MASK)) { + PMD_DRV_LOG(DEBUG, "%s: Mailbox interrupt 0x%lx on CPT VF %d\n", + cptvf->dev_name, intr, cptvf->vfid); + cptvf_handle_mbox_intr(cptvf); + cptvf_clear_mbox_intr(cptvf); + } else if (unlikely(intr & CPT_VF_INTR_IRDE_MASK)) { + cptvf_clear_irde_intr(cptvf); + PMD_DRV_LOG(DEBUG, "%s: Instruction NCB read error interrupt" + " 0x%lx on CPT VF %d\n", cptvf->dev_name, intr, + cptvf->vfid); + } else if (unlikely(intr & CPT_VF_INTR_NWRP_MASK)) { + cptvf_clear_nwrp_intr(cptvf); + PMD_DRV_LOG(DEBUG, "%s: NCB response write error interrupt" + " 0x%lx on CPT VF %d\n", cptvf->dev_name, intr, cptvf->vfid); + } else if (unlikely(intr & CPT_VF_INTR_SWERR_MASK)) { + cptvf_clear_swerr_intr(cptvf); + PMD_DRV_LOG(DEBUG, "%s: Software error interrupt 0x%lx on" + " CPT VF %d\n", cptvf->dev_name, intr, cptvf->vfid); + } else if (unlikely(intr & CPT_VF_INTR_HWERR_MASK)) { + cptvf_clear_hwerr_intr(cptvf); + PMD_DRV_LOG(DEBUG, "%s: Hardware error interrupt 0x%lx on" + " CPT VF %d\n", cptvf->dev_name, intr, cptvf->vfid); + } else if (unlikely(intr & CPT_VF_INTR_FAULT_MASK)) { + cptvf_clear_fault_intr(cptvf); + PMD_DRV_LOG(DEBUG, "%s: Translation fault interrupt 0x%lx on" + " CPT VF %d\n", cptvf->dev_name, intr, cptvf->vfid); + } else + PMD_DRV_LOG(ERR, "%s: Unhandled interrupt 0x%lx in CPT VF %d\n", + cptvf->dev_name, intr, cptvf->vfid); +} + +int cptvf_deinit_device(struct cpt_vf *dev) +{ + struct cpt_vf *cptvf = (struct cpt_vf *)dev; + + /* Do misc work one last time */ + cptvf_poll_misc(cptvf); + + /* TODO anything else ?? */ + + return 0; +} + +int cptvf_init_device(struct cpt_vf *cptvf, + void *pdev, + void *reg_base, + char *name, + uint32_t flags) +{ + (void) flags; + + memset(cptvf, 0, sizeof(struct cpt_vf)); + + /* Bar0 base address */ + cptvf->reg_base = reg_base; + strncpy(cptvf->dev_name, name, 32); + + cptvf->nr_queues = 1; + cptvf->max_queues = 1; + cptvf->pdev = pdev; + + /* To clear if there are any pending mbox msgs */ + cptvf_poll_misc(cptvf); + + if (cpt_vf_init(cptvf)) { + PMD_DRV_LOG(ERR, "Failed to initialize CPT VF device\n"); + return -1; + } + + return 0; +} + + +int cptvf_get_resource(struct cpt_vf *dev, + uint8_t group, cpt_instance_t **instance) +{ + int ret = -ENOENT, len, qlen, i; + int chunk_len, chunks, chunk_size; + struct cpt_vf *cptvf = dev; + cpt_instance_t *cpt_instance; + struct command_chunk *chunk_head = NULL, *chunk_prev = NULL; + struct command_chunk *chunk = NULL; + uint8_t *mem; + const struct rte_memzone *rz; + uint64_t dma_addr = 0, alloc_len, used_len; + uint64_t *next_ptr; + uint64_t pg_sz = sysconf(_SC_PAGESIZE); + + PMD_DRV_LOG(DEBUG, "Initializing csp resource %s\n", cptvf->dev_name); + + cpt_instance = &cptvf->instance; + + memset(&cptvf->cqueue, 0, sizeof(cptvf->cqueue)); + memset(&cptvf->pqueue, 0, sizeof(cptvf->pqueue)); + + /* Chunks are of fixed size buffers */ + chunks = DEFAULT_CMD_QCHUNKS; + chunk_len = DEFAULT_CMD_QCHUNK_SIZE; + + qlen = chunks * chunk_len; + /* Chunk size includes 8 bytes of next chunk ptr */ + chunk_size = chunk_len * CPT_INST_SIZE + CPT_NEXT_CHUNK_PTR_SIZE; + + /* For command chunk structures */ + len = chunks * RTE_ALIGN(sizeof(struct command_chunk), 8); + + /* For pending queue */ + len += qlen * RTE_ALIGN(sizeof(rid_t), 8); + + /* So that instruction queues start as pg size aligned */ + len = RTE_ALIGN(len, pg_sz); + + /* For Instruction queues */ + len += chunks * RTE_ALIGN(chunk_size, 128); + + /* Wastage after instruction queues */ + len = RTE_ALIGN(len, pg_sz); + + rz = rte_memzone_reserve_aligned(cptvf->dev_name, len, cptvf->node, + RTE_MEMZONE_SIZE_HINT_ONLY | + RTE_MEMZONE_256MB, + RTE_CACHE_LINE_SIZE); + if (!rz) { + ret = rte_errno; + goto cleanup; + } + + mem = rz->addr; + dma_addr = rz->phys_addr; + alloc_len = len; + + memset(mem, 0, len); + + cpt_instance->rsvd = (uint64_t)rz; + + /* Pending queue setup */ + cptvf->pqueue.rid_queue = (rid_t *)mem; + cptvf->pqueue.soft_qlen = qlen; + cptvf->pqueue.enq_tail = 0; + cptvf->pqueue.deq_head = 0; + cptvf->pqueue.pending_count = 0; + + mem += qlen * RTE_ALIGN(sizeof(rid_t), 8); + len -= qlen * RTE_ALIGN(sizeof(rid_t), 8); + dma_addr += qlen * RTE_ALIGN(sizeof(rid_t), 8); + + /* Alignement wastage */ + used_len = alloc_len - len; + mem += RTE_ALIGN(used_len, pg_sz) - used_len; + len -= RTE_ALIGN(used_len, pg_sz) - used_len; + dma_addr += RTE_ALIGN(used_len, pg_sz) - used_len; + + /* Init instruction queues */ + chunk_head = &cptvf->cqueue.chead[0]; + i = qlen; + + chunk_prev = NULL; + for (i = 0; i < DEFAULT_CMD_QCHUNKS; i++) { + int csize; + + chunk = &cptvf->cqueue.chead[i]; + chunk->head = mem; + chunk->dma_addr = dma_addr; + + csize = RTE_ALIGN(chunk_size, 128); + mem += csize; + dma_addr += csize; + len -= csize; + + if (chunk_prev) { + next_ptr = (uint64_t *)(chunk_prev->head + + chunk_size - 8); + *next_ptr = (uint64_t)chunk->dma_addr; + } + chunk_prev = chunk; + } + /* Circular loop */ + next_ptr = (uint64_t *)(chunk_prev->head + chunk_size - 8); + *next_ptr = (uint64_t)chunk_head->dma_addr; + + assert(!len); + + cptvf->qlen = qlen; + /* This is used for CPT(0)_PF_Q(0..15)_CTL.size config */ + cptvf->qsize = chunk_size / 8; + cptvf->cqueue.qhead = chunk_head->head; + cptvf->cqueue.idx = 0; + cptvf->cqueue.cchunk = 0; + + if (cpt_vq_init(cptvf, group)) { + PMD_DRV_LOG(ERR, "Failed to initialize CPT VQ of device %s\n", + cptvf->dev_name); + ret = -EBUSY; + goto cleanup; + } + + *instance = cpt_instance; + + PMD_DRV_LOG(DEBUG, "Crypto device (%s) initialized\n", + cptvf->dev_name); + + return 0; +cleanup: + rte_memzone_free(rz); + *instance = NULL; + return ret; +} + +int cptvf_put_resource(cpt_instance_t *instance) +{ + struct cpt_vf *cptvf = (struct cpt_vf *)instance; + struct rte_memzone *rz; + + if (!cptvf) { + PMD_DRV_LOG(ERR, "Invalid CPTVF handle\n"); + return -EINVAL; + } + + PMD_DRV_LOG(DEBUG, "Releasing csp device %s\n", cptvf->dev_name); + + rz = (struct rte_memzone *)instance->rsvd; + rte_memzone_free(rz); + return 0; +} + +int cptvf_start_device(struct cpt_vf *cptvf) +{ + int rc; + + rc = cptvf_send_vf_up(cptvf); + if (rc) { + PMD_DRV_LOG(ERR, "Failed to mark CPT VF device %s UP, rc = %d\n" + , cptvf->dev_name, rc); + return -EFAULT; + } + + if ((cptvf->vftype != SE_TYPE) && + (cptvf->vftype != AE_TYPE)) { + PMD_DRV_LOG(ERR, "Fatal error, unexpected vf type %u, for CPT" + " VF device %s\n", cptvf->vftype, cptvf->dev_name); + return -ENOENT; + } + + return 0; +} + +void cptvf_stop_device(struct cpt_vf *cptvf) +{ + int rc; + uint32_t pending, retries = 5; + + /* Wait for pending entries to complete */ + pending = cptvf_read_vq_doorbell(cptvf); + while (pending) { + PRINT("%s: Waiting for pending %u cmds to complete\n", + cptvf->dev_name, pending); + sleep(1); + pending = cptvf_read_vq_doorbell(cptvf); + retries--; + if (!retries) + break; + } + + if (!retries && pending) { + PMD_DRV_LOG(ERR, "%s: Timeout waiting for commands(%u)\n", + cptvf->dev_name, pending); + return; + } + + rc = cptvf_send_vf_down(cptvf); + if (rc) { + PMD_DRV_LOG(ERR, "Failed to bring down vf %s, rc %d\n", + cptvf->dev_name, rc); + return; + } +} diff --git a/drivers/crypto/cpt/base/cpt_device.h b/drivers/crypto/cpt/base/cpt_device.h new file mode 100644 index 0000000..951c7ae --- /dev/null +++ b/drivers/crypto/cpt/base/cpt_device.h @@ -0,0 +1,162 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright(c) 2017 Cavium, Inc + */ + +#ifndef __CPT_DEVICE_H +#define __CPT_DEVICE_H + +#include "cpt.h" +#include "cpt_hw_types.h" +#include +#include +#include +#include + +/* Device ID */ +#define PCI_VENDOR_ID_CAVIUM 0x177d +#define CPT_81XX_PCI_VF_DEVICE_ID 0xa041 + +#define CPT_NUM_QS_PER_VF (1) + +#define CPT_MBOX_MSG_TIMEOUT 2000 /* In Milli Seconds */ + +/**< flags to indicate the features supported */ +#define CPT_FLAG_VF_DRIVER (uint16_t)(1 << 3) +#define CPT_FLAG_DEVICE_READY (uint16_t)(1 << 4) + +#ifndef ROUNDUP4 +#define ROUNDUP4(val) (((val) + 3) & 0xfffffffc) +#endif + +#ifndef ROUNDUP8 +#define ROUNDUP8(val) (((val) + 7) & 0xfffffff8) +#endif + +#ifndef ROUNDUP16 +#define ROUNDUP16(val) (((val) + 15) & 0xfffffff0) +#endif + +/* Default command queue length */ +#define DEFAULT_CMD_QCHUNKS 2 +#define DEFAULT_CMD_QCHUNK_SIZE 1023 +#define DEFAULT_CMD_QLEN (DEFAULT_CMD_QCHUNK_SIZE * DEFAULT_CMD_QCHUNKS) + +/* Default command timeout in seconds */ +#define DEFAULT_COMMAND_TIMEOUT 4 + +#define CPT_COUNT_THOLD 32 +#define CPT_TIMER_THOLD 0x3F + + +#define AE_TYPE 1 +#define SE_TYPE 2 + +typedef enum { + CPT_81XX = 1, + CPT_AE_83XX, + CPT_SE_83XX, + INVALID_CPT +} cpt_pf_type_t; + +typedef struct rid { + uint64_t rid; /* Request id of a crypto operation */ +/* void *op; */ /* Opaque operation handle returned */ +} rid_t; /* Array of pending request's */ + +typedef struct pending_queue { + uint16_t enq_tail; + uint16_t deq_head; + uint16_t soft_qlen; /* Software expected queue length */ + uint16_t p_doorbell; + rid_t *rid_queue; /* Array of pending request's */ + uint64_t pending_count; /* Pending requests count */ +} pending_queue_t; + +struct command_chunk { + uint8_t *head; /**< 128-byte aligned real_vaddr */ + phys_addr_t dma_addr; /**< 128-byte aligned real_dma_addr */ +}; + +/** + * comamnd queue structure + */ +struct command_queue { + uint32_t idx; + /**< Command queue host write idx */ + uint32_t cchunk; + uint8_t *qhead; + /**< Command queue head, instructions are inserted here */ + struct command_chunk chead[DEFAULT_CMD_QCHUNKS]; + /**< Command chunk list head */ +}; + +/** + * CPT VF device structure + */ +struct cpt_vf { + cpt_instance_t instance; + + /* base address where BAR is mapped */ + uint8_t *reg_base; /**< Register start address */ + + /* Command and Pending queues */ + struct command_queue cqueue;/**< Command queue information */ + struct pending_queue pqueue;/**< Pending queue information */ + + /* Below fields are accessed only in control path */ + + /* + * This points to environment specific pdev that + * represents the pci dev + */ + void *pdev; + uint32_t qlen; + /* + * Qsize * CPT_INST_SIZE + + * alignment size(CPT_INST_SIZE + + * next chunk pointer size (8) + */ + uint32_t qsize; + /**< Calculated queue size */ + uint32_t nr_queues; + uint32_t max_queues; + + uint32_t chip_id; + /**< CPT Device ID */ + uint16_t flags; + /**< Flags to hold device status bits */ + uint8_t vfid; + /**< Device Index (0...CPT_MAX_VQ_NUM */ + uint8_t vftype; + /**< VF type of cpt_vf_type_t (SE_TYPE(2) or AE_TYPE(1) */ + uint8_t vfgrp; + /**< VF group (0 - 8) */ + uint8_t node; + /**< Operating node: Bits (46:44) in BAR0 address */ + + /* VF-PF mailbox communication */ + bool pf_acked; + bool pf_nacked; + char dev_name[32]; +} ____cacheline_aligned_in_smp; + +#define CPT_CSR_REG_BASE(cpt) ((cpt)->reg_base) + +#define CPT_READ_CSR(__hw_addr, __offset) \ + rte_read64_relaxed((uint8_t *)__hw_addr + __offset) +#define CPT_WRITE_CSR(__hw_addr, __offset, __val) \ + rte_write64_relaxed((__val), ((uint8_t *)__hw_addr + __offset)) + +void cptvf_poll_misc(void *dev); +int cptvf_deinit_device(struct cpt_vf *dev); +int cptvf_init_device(struct cpt_vf *cptvf, + void *pdev, + void *reg_base, + char *name, + uint32_t flags); +int cptvf_get_resource(struct cpt_vf *dev, + uint8_t group, cpt_instance_t **instance); +int cptvf_put_resource(cpt_instance_t *instance); +int cptvf_start_device(struct cpt_vf *cptvf); +void cptvf_stop_device(struct cpt_vf *cptvf); +#endif /* __CPT_DEVICE_H */ diff --git a/drivers/crypto/cpt/base/cpt_vf_mbox.c b/drivers/crypto/cpt/base/cpt_vf_mbox.c new file mode 100644 index 0000000..00d98bb --- /dev/null +++ b/drivers/crypto/cpt/base/cpt_vf_mbox.c @@ -0,0 +1,176 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright(c) 2017 Cavium, Inc + */ + +#include "cpt8xxx_device.h" +#include "cpt_vf_mbox.h" +#include + +/* Poll handler to handle mailbox messages from VFs */ +void cptvf_handle_mbox_intr(struct cpt_vf *cptvf) +{ + cpt_mbox_t mbx = {0, 0}; + + /* + * MBOX[0] contains msg + * MBOX[1] contains data + */ + mbx.msg = CPT_READ_CSR(CPT_CSR_REG_BASE(cptvf), + CPTX_VFX_PF_MBOXX(0, 0, 0)); + mbx.data = CPT_READ_CSR(CPT_CSR_REG_BASE(cptvf), + CPTX_VFX_PF_MBOXX(0, 0, 1)); + + PMD_DRV_LOG(DEBUG, "%s: Mailbox msg 0x%lx from PF\n", + cptvf->dev_name, mbx.msg); + switch (mbx.msg) { + case CPT_MSG_READY: + { + cpt_chipid_vfid_t cid; + + cid.u64 = mbx.data; + cptvf->pf_acked = true; + cptvf->vfid = cid.s.vfid; + cptvf->chip_id = cid.s.chip_id; + PMD_DRV_LOG(DEBUG, "%s: Received VFID %d chip_id %d\n", + cptvf->dev_name, + cptvf->vfid, cid.s.chip_id); + } + break; + case CPT_MSG_QBIND_GRP: + cptvf->pf_acked = true; + cptvf->vftype = mbx.data; + PMD_DRV_LOG(DEBUG, "%s: VF %d type %s group %d\n", + cptvf->dev_name, cptvf->vfid, + ((mbx.data == SE_TYPE) ? "SE" : "AE"), + cptvf->vfgrp); + break; + case CPT_MBOX_MSG_TYPE_ACK: + cptvf->pf_acked = true; + break; + case CPT_MBOX_MSG_TYPE_NACK: + cptvf->pf_nacked = true; + break; + default: + PMD_DRV_LOG(DEBUG, "%s: Invalid msg from PF, msg 0x%lx\n", + cptvf->dev_name, mbx.msg); + break; + } +} + +static int32_t +cptvf_send_msg_to_pf_timeout(struct cpt_vf *cptvf, cpt_mbox_t *mbx) +{ + int timeout = CPT_MBOX_MSG_TIMEOUT; + int sleep_ms = 10; + + cptvf->pf_acked = false; + cptvf->pf_nacked = false; + + cptvf_send_msg_to_pf(cptvf, mbx); + + /* Wait for previous message to be acked, timeout 2sec */ + while (!cptvf->pf_acked) { + if (cptvf->pf_nacked) + return -EINVAL; + usleep(sleep_ms * 1000); + cptvf_poll_misc(cptvf); + if (cptvf->pf_acked) + break; + timeout -= sleep_ms; + if (!timeout) { + PMD_DRV_LOG(ERR, "%s: PF didn't ack mbox msg %lx(vfid " + "%u)\n", + cptvf->dev_name, + (mbx->msg & 0xFF), cptvf->vfid); + return -EBUSY; + } + } + return 0; +} + +/* + * Checks if VF is able to comminicate with PF + * and also gets the CPT number this VF is associated to. + */ +int cptvf_check_pf_ready(struct cpt_vf *cptvf) +{ + cpt_mbox_t mbx = {0, 0}; + + mbx.msg = CPT_MSG_READY; + if (cptvf_send_msg_to_pf_timeout(cptvf, &mbx)) { + PMD_DRV_LOG(ERR, "%s: PF didn't respond to READY msg\n", + cptvf->dev_name); + return 1; + } + return 0; +} + +/* + * Communicate VQs size to PF to program CPT(0)_PF_Q(0-15)_CTL of the VF. + * Must be ACKed. + */ +int cptvf_send_vq_size_msg(struct cpt_vf *cptvf) +{ + cpt_mbox_t mbx = {0, 0}; + + mbx.msg = CPT_MSG_QLEN; + + mbx.data = cptvf->qsize; + if (cptvf_send_msg_to_pf_timeout(cptvf, &mbx)) { + PMD_DRV_LOG(ERR, "%s: PF didn't respond to vq_size msg\n", + cptvf->dev_name); + return 1; + } + return 0; +} + +/* + * Communicate VF group required to PF and get the VQ binded to that group + */ +int cptvf_send_vf_to_grp_msg(struct cpt_vf *cptvf, uint32_t group) +{ + cpt_mbox_t mbx = {0, 0}; + + mbx.msg = CPT_MSG_QBIND_GRP; + + /* Convey group of the VF */ + mbx.data = group; + if (cptvf_send_msg_to_pf_timeout(cptvf, &mbx)) { + PMD_DRV_LOG(ERR, "%s: PF didn't respond to vf_type msg\n", + cptvf->dev_name); + return 1; + } + return 0; +} + +/* + * Communicate to PF that VF is UP and running + */ +int cptvf_send_vf_up(struct cpt_vf *cptvf) +{ + cpt_mbox_t mbx = {0, 0}; + + mbx.msg = CPT_MSG_VF_UP; + if (cptvf_send_msg_to_pf_timeout(cptvf, &mbx)) { + PMD_DRV_LOG(ERR, "%s: PF didn't respond to UP msg\n", + cptvf->dev_name); + return 1; + } + return 0; +} + +/* + * Communicate to PF that VF is DOWN and running + */ +int cptvf_send_vf_down(struct cpt_vf *cptvf) +{ + cpt_mbox_t mbx = {0, 0}; + + mbx.msg = CPT_MSG_VF_DOWN; + if (cptvf_send_msg_to_pf_timeout(cptvf, &mbx)) { + PMD_DRV_LOG(ERR, "%s: PF didn't respond to DOWN msg\n", + cptvf->dev_name); + return 1; + } + return 0; +} diff --git a/drivers/crypto/cpt/base/cpt_vf_mbox.h b/drivers/crypto/cpt/base/cpt_vf_mbox.h new file mode 100644 index 0000000..8e7a05f --- /dev/null +++ b/drivers/crypto/cpt/base/cpt_vf_mbox.h @@ -0,0 +1,60 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright(c) 2017 Cavium, Inc + */ + +#ifndef __CPTVF_MBOX_H +#define __CPTVF_MBOX_H + +#include "cpt.h" +#include "cpt_device.h" + +#define CPT_MBOX_MSG_TYPE_REQ 0 +#define CPT_MBOX_MSG_TYPE_ACK 1 +#define CPT_MBOX_MSG_TYPE_NACK 2 +#define CPT_MBOX_MSG_TYPE_NOP 3 + +typedef enum { + CPT_MSG_VF_UP = 1, + CPT_MSG_VF_DOWN, + CPT_MSG_READY, + CPT_MSG_QLEN, + CPT_MSG_QBIND_GRP, + CPT_MSG_VQ_PRIORITY, + CPT_MSG_PF_TYPE, +} cpt_mbox_opcode_t; + +/* CPT mailbox structure */ +typedef struct { + uint64_t msg; /* Message type MBOX[0] */ + uint64_t data;/* Data MBOX[1] */ +} cpt_mbox_t; + +typedef union { + uint64_t u64; + struct { +#if __BYTE_ORDER__ == __ORDER_BIG_ENDIAN__ + uint32_t chip_id; + uint8_t vfid; + uint8_t reserved[3]; +#else + uint8_t reserved[3]; + uint8_t vfid; + uint32_t chip_id; +#endif + } s; +} cpt_chipid_vfid_t; + + +void cptvf_mbox_send_ack(struct cpt_vf *cptvf, cpt_mbox_t *mbx); +void cptvf_mbox_send_nack(struct cpt_vf *cptvf, cpt_mbox_t *mbx); +int cptvf_check_pf_ready(struct cpt_vf *cptvf); +int cptvf_send_vq_size_msg(struct cpt_vf *cptvf); +int cptvf_send_vf_to_grp_msg(struct cpt_vf *cptvf, uint32_t group); +int cptvf_send_vf_priority_msg(struct cpt_vf *cptvf, uint32_t priority); +int cptvf_send_vf_down(struct cpt_vf *cptvf); +int cptvf_send_vf_up(struct cpt_vf *cptvf); +void cptvf_handle_mbox_intr(struct cpt_vf *cptvf); +/* Synchronous raw operation to get vf cfg */ +int cptvf_get_pf_type_raw(char *dev_name, void *reg_base, + cpt_pf_type_t *pf_type); +#endif /* __CPTVF_MBOX_H */ -- 1.9.3