From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from dpdk.org (dpdk.org [92.243.14.124]) by inbox.dpdk.org (Postfix) with ESMTP id 8B37FA04F3; Fri, 3 Jan 2020 11:59:12 +0100 (CET) Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id BABEF1BFD8; Fri, 3 Jan 2020 11:59:11 +0100 (CET) Received: from mx0b-0016f401.pphosted.com (mx0a-0016f401.pphosted.com [67.231.148.174]) by dpdk.org (Postfix) with ESMTP id 381554C8B for ; Fri, 3 Jan 2020 11:59:10 +0100 (CET) Received: from pps.filterd (m0045849.ppops.net [127.0.0.1]) by mx0a-0016f401.pphosted.com (8.16.0.42/8.16.0.42) with SMTP id 003AuLhH002947; Fri, 3 Jan 2020 02:59:05 -0800 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=marvell.com; h=from : to : cc : subject : date : message-id : references : in-reply-to : content-type : content-transfer-encoding : mime-version; s=pfpt0818; bh=Q0BexfOUuXCLFlIGnzPbxB3K7QHxpAJrR2Caxxm6IA4=; b=bHNRwbKsshlN/dyi1SMDmYjP7Hn3Q9uhNzsX36UJE0Pmef7FWbRt6pTiOsehFBfLcKBL 8ElZdIg4vxEhNwujIzDR7tlTEk+VHJayN70iv/9kSyu/qZDi5MRF7SEpqo2VfAfpNWxN A806RbU8odoLAXYGGQEeJOEDRnuU1WkyJwohcffHXTatT8kQl+p438ztnna6dHgdNX2D rugMd135wnmYp+E9+IzGo8/PVm+c/OVq1mUSQsPPmGbOtaSWhWZ5ZOhRokdcBbflDMiQ 4LhP79auLHQ8FEX6Aqd+LN+s3rCYmsqSKSPzb92+HegU9FW0XmIFYJ7DrC/db+5Uitib pg== Received: from sc-exch04.marvell.com ([199.233.58.184]) by mx0a-0016f401.pphosted.com with ESMTP id 2xa2g909ne-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-SHA384 bits=256 verify=NOT); Fri, 03 Jan 2020 02:59:05 -0800 Received: from SC-EXCH04.marvell.com (10.93.176.84) by SC-EXCH04.marvell.com (10.93.176.84) with Microsoft SMTP Server (TLS) id 15.0.1497.2; Fri, 3 Jan 2020 02:59:03 -0800 Received: from NAM12-BN8-obe.outbound.protection.outlook.com (104.47.55.175) by SC-EXCH04.marvell.com (10.93.176.84) with Microsoft SMTP Server (TLS) id 15.0.1497.2 via Frontend Transport; Fri, 3 Jan 2020 02:59:03 -0800 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=K29gA8zPjXXlPhnwDvL5hh1+wJtVeSw4zDmpaQ6zT5tIATC21N5juKmeK6rVS0G4DZHrV5pC4Jz6Q4nIgWHqOxvlbdIKYrFNwOlhgN81Pg8In8+WtvZ7CqR3Pse0BqIJcmrxLKiNi/K3tThBZlK9ex7uNx1sidscLEJx96WoJny+FOWWNIOGaelES/XWUiGaIoDZx6AFvEIegYiub2ODZSryS1dXm5If09bjPZhr632xl62ly8HBHwi2CUWblH/YChwMxPZAQuAvJTigDSt3Zz1s6vTlDT94SN9HleqM33kQRem5gdKX8/DJeabx7IwQCwpg1v1ZYuqF/yCw8bsj0A== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=Q0BexfOUuXCLFlIGnzPbxB3K7QHxpAJrR2Caxxm6IA4=; b=cI400ayAsPrNJm4wr0ZY3dBCc0bPMsgx1sltuEJPBxxRk1JnQ8E2I9GGtOuMMu3zSi71yqJcpdkNFer5cf3Nb1ZnmpTAgxBBB1PjTxYZxQ+zrJubbgKf7lsxtgPaMEi9IOFB7J3bvViUGE3S2dkV/hxqdxpN4RS20fQbvoG3zaCPp+MUE81GIrl9xZrBoRS5KvBDMS3J5Vx/2beNgJPoHiFhCc7M8VLo1h5JGKSeGzavlGe0oF9nfr45m2CdiX5ADiB4pZHVB86SMZvVPJ4Trl4GL1XQzEvxVigzkkIRFz6kbKxPowOAYJaq17anfYgv+IjKv3aD1YHPqORphdAaHw== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=marvell.com; dmarc=pass action=none header.from=marvell.com; dkim=pass header.d=marvell.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=marvell.onmicrosoft.com; s=selector1-marvell-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=Q0BexfOUuXCLFlIGnzPbxB3K7QHxpAJrR2Caxxm6IA4=; b=CCDDhMf68PkYvtX1Wssksa0+vEqCEPMgbAxIAfzlyWQ5I9gf9blCR9BKMVuzYFKWL7OXY7wQidAarFd70u0jbsfIVGouKA636Wbe/SUiSpSOeRtsI1ljJVf49BMcZ+x/7COIsQxMGDWGyEi/WnwjmXJHSEVZ827bqllUeA1zIws= Received: from MWHPR18MB1376.namprd18.prod.outlook.com (10.173.243.18) by MWHPR18MB1389.namprd18.prod.outlook.com (10.175.5.143) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.2602.13; Fri, 3 Jan 2020 10:59:01 +0000 Received: from MWHPR18MB1376.namprd18.prod.outlook.com ([fe80::3d27:53fc:7a3a:f73a]) by MWHPR18MB1376.namprd18.prod.outlook.com ([fe80::3d27:53fc:7a3a:f73a%5]) with mapi id 15.20.2602.012; Fri, 3 Jan 2020 10:59:00 +0000 From: Mahipal Challa To: Gavin Hu , "dev@dpdk.org" CC: Jerin Jacob Kollanukkaran , "Narayana Prasad Raju Athreya" , Subrahmanyam Nilla , Venkateshwarlu Nalla , nd Thread-Topic: [dpdk-dev] [dpdk-dev PATCH v2 2/6] raw/octeontx2_ep: add device configuration Thread-Index: AQHVvLLSIt4FDDeGOUyt0vN7xV755afXMlEAgAGTHbA= Date: Fri, 3 Jan 2020 10:59:00 +0000 Message-ID: References: <1577450386-16966-1-git-send-email-mchalla@marvell.com> <1577450386-16966-3-git-send-email-mchalla@marvell.com> In-Reply-To: Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-originating-ip: [115.113.156.2] x-ms-publictraffictype: Email x-ms-office365-filtering-correlation-id: 6358e87b-f9d2-428e-abe8-08d7903bf08a x-ms-traffictypediagnostic: MWHPR18MB1389: x-ms-exchange-transport-forked: True x-microsoft-antispam-prvs: x-ms-oob-tlc-oobclassifiers: OLM:8273; x-forefront-prvs: 0271483E06 x-forefront-antispam-report: SFV:NSPM; SFS:(10009020)(4636009)(136003)(39850400004)(396003)(366004)(346002)(376002)(13464003)(199004)(189003)(19627235002)(478600001)(55016002)(5660300002)(110136005)(8936002)(81156014)(54906003)(33656002)(71200400001)(9686003)(316002)(81166006)(86362001)(52536014)(966005)(66946007)(7696005)(26005)(66476007)(76116006)(2906002)(186003)(30864003)(4326008)(55236004)(53546011)(66446008)(66556008)(64756008)(6506007)(569006); DIR:OUT; SFP:1101; SCL:1; SRVR:MWHPR18MB1389; H:MWHPR18MB1376.namprd18.prod.outlook.com; FPR:; SPF:None; LANG:en; PTR:InfoNoRecords; A:1; MX:1; received-spf: None (protection.outlook.com: marvell.com does not designate permitted sender hosts) x-ms-exchange-senderadcheck: 1 x-microsoft-antispam: BCL:0; x-microsoft-antispam-message-info: 3LVmBxqJkyJRjNt8d5hfpoExfd5sLQaLm9Oz84h4B/FngXopGcNyudymT66MLn0r4NbEQKu17lUghVLnNl146f1/pTbA1VKWvWAf6yz8E6U9h4WBXU1XeDcpi+EeKCCcQRIjFhjTcTxx6adwHRUTdLDsuMYM9u6/bxACSptTYnymOq2P9tjvJ7726C5sBUCzfBVkvzB0rHsqIr/zKoytXLmoTHoKvNiLL7bBdGAANSI+c8IJ+ju1wUF3RuGenRty7hEaB6JoYBbZA8ypUFI2FauFbTOwIDVpCFCdy7U+1/fBjScbXk2lWnD4lJu48/FC4gCzIllDr6R8UBxr6FedImbniaE80Oig4JD+NNpnOAv0qiXYxAMuKlrNvEq9Aph4yaut/w2aNrlU2qn8SSKUYNBLcucMTH/MTRkDJKXnytnOJFWwcD1QVEvZeYiz1vM/xhsY67ERAHQbXTKo9Ayzljm0VquuXdxlgP3Gj6ytRggxfPRalrZxAQVv8Vf8CdBkYvBlSWO67TVe/oxZZqyZeV7jdbfelPPB7FGuLvGrj7U= Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 X-MS-Exchange-CrossTenant-Network-Message-Id: 6358e87b-f9d2-428e-abe8-08d7903bf08a X-MS-Exchange-CrossTenant-originalarrivaltime: 03 Jan 2020 10:59:00.7095 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Hosted X-MS-Exchange-CrossTenant-id: 70e1fb47-1155-421d-87fc-2e58f638b6e0 X-MS-Exchange-CrossTenant-mailboxtype: HOSTED X-MS-Exchange-CrossTenant-userprincipalname: v7S2t/gDXvmA+qu8fovuKGYTDGsv2sFzA9EgcRNBKh+QnfBrtzjWHnsMCFzB5KBZlS9N8OexTYxWbhEQixrK5w== X-MS-Exchange-Transport-CrossTenantHeadersStamped: MWHPR18MB1389 X-OriginatorOrg: marvell.com X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10434:6.0.95,18.0.572 definitions=2020-01-03_02:2020-01-02,2020-01-03 signatures=0 Subject: Re: [dpdk-dev] [dpdk-dev PATCH v2 2/6] raw/octeontx2_ep: add device configuration X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" Hi Gavin, Please see response inline. > -----Original Message----- > From: Gavin Hu > Sent: Thursday, January 2, 2020 3:48 PM > To: Mahipal Challa ; dev@dpdk.org > Cc: Jerin Jacob Kollanukkaran ; Narayana Prasad Raju > Athreya ; Subrahmanyam Nilla > ; Venkateshwarlu Nalla ; nd > > Subject: [EXT] RE: [dpdk-dev] [dpdk-dev PATCH v2 2/6] raw/octeontx2_ep: > add device configuration >=20 > External Email >=20 > ---------------------------------------------------------------------- > Hi Mahipal, >=20 > 3 comments inline. >=20 > > -----Original Message----- > > From: dev On Behalf Of Mahipal Challa > > Sent: Friday, December 27, 2019 8:40 PM > > To: dev@dpdk.org > > Cc: jerinj@marvell.com; pathreya@marvell.com; snilla@marvell.com; > > venkatn@marvell.com > > Subject: [dpdk-dev] [dpdk-dev PATCH v2 2/6] raw/octeontx2_ep: add > device > > configuration > > > > Register "dev_configure" API to configure/initialize the SDP > > VF PCIe devices. > > > > Signed-off-by: Mahipal Challa > > --- > > doc/guides/rawdevs/octeontx2_ep.rst | 29 ++ > > drivers/common/octeontx2/hw/otx2_sdp.h | 184 +++++++++ > > drivers/common/octeontx2/otx2_common.c | 9 + > > drivers/common/octeontx2/otx2_common.h | 4 + > > .../octeontx2/rte_common_octeontx2_version.map | 6 + > > drivers/raw/octeontx2_ep/Makefile | 3 + > > drivers/raw/octeontx2_ep/meson.build | 4 +- > > drivers/raw/octeontx2_ep/otx2_ep_enqdeq.c | 294 +++++++++++++= + > > drivers/raw/octeontx2_ep/otx2_ep_enqdeq.h | 11 + > > drivers/raw/octeontx2_ep/otx2_ep_rawdev.c | 148 +++++++ > > drivers/raw/octeontx2_ep/otx2_ep_rawdev.h | 434 > > ++++++++++++++++++++- > > drivers/raw/octeontx2_ep/otx2_ep_vf.c | 408 > +++++++++++++++++++ > > drivers/raw/octeontx2_ep/otx2_ep_vf.h | 10 + > > 13 files changed, 1542 insertions(+), 2 deletions(-) > > > > diff --git a/doc/guides/rawdevs/octeontx2_ep.rst > > b/doc/guides/rawdevs/octeontx2_ep.rst > > index 5f5ed01..2507fcf 100644 > > --- a/doc/guides/rawdevs/octeontx2_ep.rst > > +++ b/doc/guides/rawdevs/octeontx2_ep.rst > > @@ -39,3 +39,32 @@ entry `sriov_numvfs` for the corresponding PF > driver. > > > > Once the required VFs are enabled, to be accessible from DPDK, VFs nee= d > to > > be > > bound to vfio-pci driver. > > + > > +Device Setup > > +------------ > > + > > +The OCTEON TX2 SDP End Point VF devices will need to be bound to a > > +user-space IO driver for use. The script ``dpdk-devbind.py`` script > > +included with DPDK can be used to view the state of the devices and to > bind > > +them to a suitable DPDK-supported kernel driver. When querying the > status > > +of the devices, they will appear under the category of "Misc (rawdev) > > +devices", i.e. the command ``dpdk-devbind.py --status-dev misc`` can b= e > > +used to see the state of those devices alone. > > + > > +Device Configuration > > +-------------------- > > + > > +Configuring SDP EP rawdev device is done using the > > ``rte_rawdev_configure()`` > > +API, which takes the mempool as parameter. PMD uses this pool to > > send/receive > > +packets to/from the HW. > > + > > +The following code shows how the device is configured > > + > > +.. code-block:: c > > + > > + struct sdp_rawdev_info config =3D {0}; > > + struct rte_rawdev_info rdev_info =3D {.dev_private =3D &config}; > > + config.enqdeq_mpool =3D (void *)rte_mempool_create(...); > > + > > + rte_rawdev_configure(dev_id, (rte_rawdev_obj_t)&rdev_info); > > + > > diff --git a/drivers/common/octeontx2/hw/otx2_sdp.h > > b/drivers/common/octeontx2/hw/otx2_sdp.h > > new file mode 100644 > > index 0000000..7e03317 > > --- /dev/null > > +++ b/drivers/common/octeontx2/hw/otx2_sdp.h > > @@ -0,0 +1,184 @@ > > +/* SPDX-License-Identifier: BSD-3-Clause > > + * Copyright(C) 2019 Marvell International Ltd. > > + */ > > + > > +#ifndef __OTX2_SDP_HW_H_ > > +#define __OTX2_SDP_HW_H_ > > + > > +/* SDP VF IOQs */ > > +#define SDP_MIN_RINGS_PER_VF (1) > > +#define SDP_MAX_RINGS_PER_VF (8) > > + > > +/* SDP VF IQ configuration */ > > +#define SDP_VF_MAX_IQ_DESCRIPTORS (512) > > +#define SDP_VF_MIN_IQ_DESCRIPTORS (128) > > + > > +#define SDP_VF_DB_MIN (1) > > +#define SDP_VF_DB_TIMEOUT (1) > > +#define SDP_VF_INTR_THRESHOLD (0xFFFFFFFF) > > + > > +#define SDP_VF_64BYTE_INSTR (64) > > +#define SDP_VF_32BYTE_INSTR (32) > > + > > +/* SDP VF OQ configuration */ > > +#define SDP_VF_MAX_OQ_DESCRIPTORS (512) > > +#define SDP_VF_MIN_OQ_DESCRIPTORS (128) > > +#define SDP_VF_OQ_BUF_SIZE (2048) > > +#define SDP_VF_OQ_REFIL_THRESHOLD (16) > > + > > +#define SDP_VF_OQ_INFOPTR_MODE (1) > > +#define SDP_VF_OQ_BUFPTR_MODE (0) > > + > > +#define SDP_VF_OQ_INTR_PKT (1) > > +#define SDP_VF_OQ_INTR_TIME (10) > > +#define SDP_VF_CFG_IO_QUEUES SDP_MAX_RINGS_PER_VF > > + > > +/* Wait time in milliseconds for FLR */ > > +#define SDP_VF_PCI_FLR_WAIT (100) > > +#define SDP_VF_BUSY_LOOP_COUNT (10000) > > + > > +#define SDP_VF_MAX_IO_QUEUES SDP_MAX_RINGS_PER_VF > > +#define SDP_VF_MIN_IO_QUEUES SDP_MIN_RINGS_PER_VF > > + > > +/* SDP VF IOQs per rawdev */ > > +#define SDP_VF_MAX_IOQS_PER_RAWDEV SDP_VF_MAX_IO_QUEUES > > +#define SDP_VF_DEFAULT_IOQS_PER_RAWDEV > SDP_VF_MIN_IO_QUEUES > > + > > +/* SDP VF Register definitions */ > > +#define SDP_VF_RING_OFFSET (0x1ull << 17) > > + > > +/* SDP VF IQ Registers */ > > +#define SDP_VF_R_IN_CONTROL_START (0x10000) > > +#define SDP_VF_R_IN_ENABLE_START (0x10010) > > +#define SDP_VF_R_IN_INSTR_BADDR_START (0x10020) > > +#define SDP_VF_R_IN_INSTR_RSIZE_START (0x10030) > > +#define SDP_VF_R_IN_INSTR_DBELL_START (0x10040) > > +#define SDP_VF_R_IN_CNTS_START (0x10050) > > +#define SDP_VF_R_IN_INT_LEVELS_START (0x10060) > > +#define SDP_VF_R_IN_PKT_CNT_START (0x10080) > > +#define SDP_VF_R_IN_BYTE_CNT_START (0x10090) > > + > > +#define SDP_VF_R_IN_CONTROL(ring) \ > > + (SDP_VF_R_IN_CONTROL_START + ((ring) * SDP_VF_RING_OFFSET)) > > + > > +#define SDP_VF_R_IN_ENABLE(ring) \ > > + (SDP_VF_R_IN_ENABLE_START + ((ring) * SDP_VF_RING_OFFSET)) > > + > > +#define SDP_VF_R_IN_INSTR_BADDR(ring) \ > > + (SDP_VF_R_IN_INSTR_BADDR_START + ((ring) * > > SDP_VF_RING_OFFSET)) > > + > > +#define SDP_VF_R_IN_INSTR_RSIZE(ring) \ > > + (SDP_VF_R_IN_INSTR_RSIZE_START + ((ring) * > SDP_VF_RING_OFFSET)) > > + > > +#define SDP_VF_R_IN_INSTR_DBELL(ring) \ > > + (SDP_VF_R_IN_INSTR_DBELL_START + ((ring) * > > SDP_VF_RING_OFFSET)) > > + > > +#define SDP_VF_R_IN_CNTS(ring) \ > > + (SDP_VF_R_IN_CNTS_START + ((ring) * SDP_VF_RING_OFFSET)) > > + > > +#define SDP_VF_R_IN_INT_LEVELS(ring) \ > > + (SDP_VF_R_IN_INT_LEVELS_START + ((ring) * SDP_VF_RING_OFFSET)) > > + > > +#define SDP_VF_R_IN_PKT_CNT(ring) \ > > + (SDP_VF_R_IN_PKT_CNT_START + ((ring) * SDP_VF_RING_OFFSET)) > > + > > +#define SDP_VF_R_IN_BYTE_CNT(ring) \ > > + (SDP_VF_R_IN_BYTE_CNT_START + ((ring) * SDP_VF_RING_OFFSET)) > > + > > +/* SDP VF IQ Masks */ > > +#define SDP_VF_R_IN_CTL_RPVF_MASK (0xF) > > +#define SDP_VF_R_IN_CTL_RPVF_POS (48) > > + > > +#define SDP_VF_R_IN_CTL_IDLE (0x1ull << 28) > > +#define SDP_VF_R_IN_CTL_RDSIZE (0x3ull << 25) /* Setting to m= ax(4) > */ > > +#define SDP_VF_R_IN_CTL_IS_64B (0x1ull << 24) > > +#define SDP_VF_R_IN_CTL_D_NSR (0x1ull << 8) > > +#define SDP_VF_R_IN_CTL_D_ESR (0x1ull << 6) > > +#define SDP_VF_R_IN_CTL_D_ROR (0x1ull << 5) > > +#define SDP_VF_R_IN_CTL_NSR (0x1ull << 3) > > +#define SDP_VF_R_IN_CTL_ESR (0x1ull << 1) > > +#define SDP_VF_R_IN_CTL_ROR (0x1ull << 0) > > + > > +#define SDP_VF_R_IN_CTL_MASK \ > > + (SDP_VF_R_IN_CTL_RDSIZE | SDP_VF_R_IN_CTL_IS_64B) > > + > > +/* SDP VF OQ Registers */ > > +#define SDP_VF_R_OUT_CNTS_START (0x10100) > > +#define SDP_VF_R_OUT_INT_LEVELS_START (0x10110) > > +#define SDP_VF_R_OUT_SLIST_BADDR_START (0x10120) > > +#define SDP_VF_R_OUT_SLIST_RSIZE_START (0x10130) > > +#define SDP_VF_R_OUT_SLIST_DBELL_START (0x10140) > > +#define SDP_VF_R_OUT_CONTROL_START (0x10150) > > +#define SDP_VF_R_OUT_ENABLE_START (0x10160) > > +#define SDP_VF_R_OUT_PKT_CNT_START (0x10180) > > +#define SDP_VF_R_OUT_BYTE_CNT_START (0x10190) > > + > > +#define SDP_VF_R_OUT_CONTROL(ring) \ > > + (SDP_VF_R_OUT_CONTROL_START + ((ring) * SDP_VF_RING_OFFSET)) > > + > > +#define SDP_VF_R_OUT_ENABLE(ring) \ > > + (SDP_VF_R_OUT_ENABLE_START + ((ring) * SDP_VF_RING_OFFSET)) > > + > > +#define SDP_VF_R_OUT_SLIST_BADDR(ring) \ > > + (SDP_VF_R_OUT_SLIST_BADDR_START + ((ring) * > > SDP_VF_RING_OFFSET)) > > + > > +#define SDP_VF_R_OUT_SLIST_RSIZE(ring) \ > > + (SDP_VF_R_OUT_SLIST_RSIZE_START + ((ring) * > > SDP_VF_RING_OFFSET)) > > + > > +#define SDP_VF_R_OUT_SLIST_DBELL(ring) \ > > + (SDP_VF_R_OUT_SLIST_DBELL_START + ((ring) * > > SDP_VF_RING_OFFSET)) > > + > > +#define SDP_VF_R_OUT_CNTS(ring) \ > > + (SDP_VF_R_OUT_CNTS_START + ((ring) * SDP_VF_RING_OFFSET)) > > + > > +#define SDP_VF_R_OUT_INT_LEVELS(ring) \ > > + (SDP_VF_R_OUT_INT_LEVELS_START + ((ring) * > > SDP_VF_RING_OFFSET)) > > + > > +#define SDP_VF_R_OUT_PKT_CNT(ring) \ > > + (SDP_VF_R_OUT_PKT_CNT_START + ((ring) * SDP_VF_RING_OFFSET)) > > + > > +#define SDP_VF_R_OUT_BYTE_CNT(ring) \ > > + (SDP_VF_R_OUT_BYTE_CNT_START + ((ring) * > SDP_VF_RING_OFFSET)) > > + > > +/* SDP VF OQ Masks */ > > +#define SDP_VF_R_OUT_CTL_IDLE (1ull << 40) > > +#define SDP_VF_R_OUT_CTL_ES_I (1ull << 34) > > +#define SDP_VF_R_OUT_CTL_NSR_I (1ull << 33) > > +#define SDP_VF_R_OUT_CTL_ROR_I (1ull << 32) > > +#define SDP_VF_R_OUT_CTL_ES_D (1ull << 30) > > +#define SDP_VF_R_OUT_CTL_NSR_D (1ull << 29) > > +#define SDP_VF_R_OUT_CTL_ROR_D (1ull << 28) > > +#define SDP_VF_R_OUT_CTL_ES_P (1ull << 26) > > +#define SDP_VF_R_OUT_CTL_NSR_P (1ull << 25) > > +#define SDP_VF_R_OUT_CTL_ROR_P (1ull << 24) > > +#define SDP_VF_R_OUT_CTL_IMODE (1ull << 23) > > + > > +#define SDP_VF_R_OUT_INT_LEVELS_BMODE (1ull << 63) > > +#define SDP_VF_R_OUT_INT_LEVELS_TIMET (32) > > + > > +/* SDP Instruction Header */ > > +struct sdp_instr_ih { > > + /* Data Len */ > > + uint64_t tlen:16; > > + > > + /* Reserved1 */ > > + uint64_t rsvd1:20; > > + > > + /* PKIND for SDP */ > > + uint64_t pkind:6; > > + > > + /* Front Data size */ > > + uint64_t fsz:6; > > + > > + /* No. of entries in gather list */ > > + uint64_t gsz:14; > > + > > + /* Gather indicator */ > > + uint64_t gather:1; > > + > > + /* Reserved2 */ > > + uint64_t rsvd2:1; > > +}; > Should __attributes((packed)) be added? > I am not sure about this, but I see here it is added. > https://urldefense.proofpoint.com/v2/url?u=3Dhttps- > 3A__code.dpdk.org_dpdk_latest_source_drivers_crypto_caam-5Fjr_caam- > 5Fjr-5Fdesc.h-23L165&d=3DDwIFAg&c=3DnKjWec2b6R0mOyPaz7xtfQ&r=3D- > s3WtB2eBJ7B88AJuqlVnFqXScHMHYESWbIBMvf7lwI&m=3DvLoVSSkkB3cLz- > ZlbcX68kWjRPvjSykTAzLXKH2sMKw&s=3DEZ- > RqeuNNUdrnzynfqMreySb4GyFFKaOSkj0XHsLnmk&e=3D [Mahipal]: It seems there is no harm in packing, will add this in v3. > > + > > +#endif /* __OTX2_SDP_HW_H_ */ > > + > > diff --git a/drivers/common/octeontx2/otx2_common.c > > b/drivers/common/octeontx2/otx2_common.c > > index 7e45366..1a257cf 100644 > > --- a/drivers/common/octeontx2/otx2_common.c > > +++ b/drivers/common/octeontx2/otx2_common.c > > @@ -205,6 +205,10 @@ int otx2_npa_lf_obj_ref(void) > > * @internal > > */ > > int otx2_logtype_dpi; > > +/** > > + * @internal > > + */ > > +int otx2_logtype_ep; > > > > RTE_INIT(otx2_log_init); > > static void > > @@ -245,4 +249,9 @@ int otx2_npa_lf_obj_ref(void) > > otx2_logtype_dpi =3D rte_log_register("pmd.raw.octeontx2.dpi"); > > if (otx2_logtype_dpi >=3D 0) > > rte_log_set_level(otx2_logtype_dpi, RTE_LOG_NOTICE); > > + > > + otx2_logtype_ep =3D rte_log_register("pmd.raw.octeontx2.ep"); > > + if (otx2_logtype_ep >=3D 0) > > + rte_log_set_level(otx2_logtype_ep, RTE_LOG_NOTICE); > > + > > } > > diff --git a/drivers/common/octeontx2/otx2_common.h > > b/drivers/common/octeontx2/otx2_common.h > > index f62c45d..6bc55ca 100644 > > --- a/drivers/common/octeontx2/otx2_common.h > > +++ b/drivers/common/octeontx2/otx2_common.h > > @@ -16,6 +16,7 @@ > > #include "hw/otx2_nix.h" > > #include "hw/otx2_npc.h" > > #include "hw/otx2_npa.h" > > +#include "hw/otx2_sdp.h" > > #include "hw/otx2_sso.h" > > #include "hw/otx2_ssow.h" > > #include "hw/otx2_tim.h" > > @@ -85,6 +86,7 @@ struct otx2_idev_cfg { > > extern int otx2_logtype_tm; > > extern int otx2_logtype_tim; > > extern int otx2_logtype_dpi; > > +extern int otx2_logtype_ep; > > > > #define otx2_err(fmt, args...) \ > > RTE_LOG(ERR, PMD, "%s():%u " fmt "\n", \ > > @@ -107,6 +109,7 @@ struct otx2_idev_cfg { > > #define otx2_tm_dbg(fmt, ...) otx2_dbg(tm, fmt, ##__VA_ARGS__) > > #define otx2_tim_dbg(fmt, ...) otx2_dbg(tim, fmt, ##__VA_ARGS__) > > #define otx2_dpi_dbg(fmt, ...) otx2_dbg(dpi, fmt, ##__VA_ARGS__) > > +#define otx2_sdp_dbg(fmt, ...) otx2_dbg(ep, fmt, ##__VA_ARGS__) > > > > /* PCI IDs */ > > #define PCI_VENDOR_ID_CAVIUM 0x177D > > @@ -121,6 +124,7 @@ struct otx2_idev_cfg { > > #define PCI_DEVID_OCTEONTX2_RVU_CPT_VF 0xA0FE > > #define PCI_DEVID_OCTEONTX2_RVU_AF_VF 0xA0f8 > > #define PCI_DEVID_OCTEONTX2_DPI_VF 0xA081 > > +#define PCI_DEVID_OCTEONTX2_EP_VF 0xB203 /* OCTEON > > TX2 EP mode */ > > #define PCI_DEVID_OCTEONTX2_RVU_SDP_PF 0xA0f6 > > #define PCI_DEVID_OCTEONTX2_RVU_SDP_VF 0xA0f7 > > > > diff --git > a/drivers/common/octeontx2/rte_common_octeontx2_version.map > > b/drivers/common/octeontx2/rte_common_octeontx2_version.map > > index adad21a..a51d719 100644 > > --- a/drivers/common/octeontx2/rte_common_octeontx2_version.map > > +++ b/drivers/common/octeontx2/rte_common_octeontx2_version.map > > @@ -33,3 +33,9 @@ DPDK_20.0 { > > > > local: *; > > }; > > + > > +EXPERIMENTAL { > > + global: > > + > > + otx2_logtype_ep; > > +}; > > diff --git a/drivers/raw/octeontx2_ep/Makefile > > b/drivers/raw/octeontx2_ep/Makefile > > index 8cec6bd..02853fb 100644 > > --- a/drivers/raw/octeontx2_ep/Makefile > > +++ b/drivers/raw/octeontx2_ep/Makefile > > @@ -36,5 +36,8 @@ LIBABIVER :=3D 1 > > # All source are stored in SRCS-y > > # > > SRCS-$(CONFIG_RTE_LIBRTE_PMD_OCTEONTX2_EP_RAWDEV) +=3D > > otx2_ep_rawdev.c > > +SRCS-$(CONFIG_RTE_LIBRTE_PMD_OCTEONTX2_EP_RAWDEV) +=3D > > otx2_ep_enqdeq.c > > +SRCS-$(CONFIG_RTE_LIBRTE_PMD_OCTEONTX2_EP_RAWDEV) +=3D > > otx2_ep_vf.c > > + > > > > include $(RTE_SDK)/mk/rte.lib.mk > > diff --git a/drivers/raw/octeontx2_ep/meson.build > > b/drivers/raw/octeontx2_ep/meson.build > > index e513131..99e6c6d 100644 > > --- a/drivers/raw/octeontx2_ep/meson.build > > +++ b/drivers/raw/octeontx2_ep/meson.build > > @@ -3,4 +3,6 @@ > > # > > > > deps +=3D ['bus_pci', 'common_octeontx2', 'rawdev'] > > -sources =3D files('otx2_ep_rawdev.c') > > +sources =3D files('otx2_ep_rawdev.c', > > + 'otx2_ep_enqdeq.c', > > + 'otx2_ep_vf.c') > > diff --git a/drivers/raw/octeontx2_ep/otx2_ep_enqdeq.c > > b/drivers/raw/octeontx2_ep/otx2_ep_enqdeq.c > > new file mode 100644 > > index 0000000..8857004 > > --- /dev/null > > +++ b/drivers/raw/octeontx2_ep/otx2_ep_enqdeq.c > > @@ -0,0 +1,294 @@ > > +/* SPDX-License-Identifier: BSD-3-Clause > > + * Copyright(C) 2019 Marvell International Ltd. > > + */ > > + > > +#include > > +#include > > +#include > > +#include > > + > > +#include > > +#include > > +#include > > +#include > > +#include > > +#include > > + > > +#include > > +#include > > +#include > > + > > +#include "otx2_common.h" > > +#include "otx2_ep_enqdeq.h" > > + > > +/* IQ initialization */ > > +static int > > +sdp_init_instr_queue(struct sdp_device *sdpvf, int iq_no) > > +{ > > + const struct sdp_config *conf; > > + struct sdp_instr_queue *iq; > > + uint32_t q_size; > > + > > + conf =3D sdpvf->conf; > > + iq =3D sdpvf->instr_queue[iq_no]; > > + q_size =3D conf->iq.instr_type * conf->num_iqdef_descs; > > + > > + /* IQ memory creation for Instruction submission to OCTEON TX2 */ > > + iq->iq_mz =3D rte_memzone_reserve_aligned("iqmz", > > + q_size, > > + rte_socket_id(), > > + RTE_MEMZONE_IOVA_CONTIG, > > + RTE_CACHE_LINE_SIZE); > > + if (iq->iq_mz =3D=3D NULL) { > > + otx2_err("IQ[%d] memzone alloc failed", iq_no); > > + goto iq_init_fail; > > + } > > + > > + iq->base_addr_dma =3D iq->iq_mz->iova; > > + iq->base_addr =3D (uint8_t *)iq->iq_mz->addr; > > + > > + if (conf->num_iqdef_descs & (conf->num_iqdef_descs - 1)) { > > + otx2_err("IQ[%d] descs not in power of 2", iq_no); >=20 > Should the reserved memzone be freed by calling rte_memzone_free here? [Mahipal]: All the error handling is part of the "[v2,3/6] raw/octeontx2_ep= : add device uninitialization" patch. >=20 > > + goto iq_init_fail; > > + } > > + > > + iq->nb_desc =3D conf->num_iqdef_descs; > > + > > + /* Create a IQ request list to hold requests that have been > > + * posted to OCTEON TX2. This list will be used for freeing the IQ > > + * data buffer(s) later once the OCTEON TX2 fetched the requests. > > + */ > > + iq->req_list =3D rte_zmalloc_socket("request_list", > > + (iq->nb_desc * SDP_IQREQ_LIST_SIZE), > > + RTE_CACHE_LINE_SIZE, > > + rte_socket_id()); > > + if (iq->req_list =3D=3D NULL) { > > + otx2_err("IQ[%d] req_list alloc failed", iq_no); > > + goto iq_init_fail; > > + } > > + > > + otx2_info("IQ[%d]: base: %p basedma: %lx count: %d", > > + iq_no, iq->base_addr, (unsigned long)iq- > >base_addr_dma, > > + iq->nb_desc); > > + > > + iq->sdp_dev =3D sdpvf; > > + iq->q_no =3D iq_no; > > + iq->fill_cnt =3D 0; > > + iq->host_write_index =3D 0; > > + iq->otx_read_index =3D 0; > > + iq->flush_index =3D 0; > > + > > + /* Initialize the spinlock for this instruction queue */ > > + rte_spinlock_init(&iq->lock); > > + rte_spinlock_init(&iq->post_lock); > > + > > + rte_atomic64_clear(&iq->iq_flush_running); > > + > > + sdpvf->io_qmask.iq |=3D (1ull << iq_no); > > + > > + /* Set 32B/64B mode for each input queue */ > > + if (conf->iq.instr_type =3D=3D 64) > > + sdpvf->io_qmask.iq64B |=3D (1ull << iq_no); > > + > > + iq->iqcmd_64B =3D (conf->iq.instr_type =3D=3D 64); > > + > > + /* Set up IQ registers */ > > + sdpvf->fn_list.setup_iq_regs(sdpvf, iq_no); > > + > > + return 0; > > + > > +iq_init_fail: > > + return -ENOMEM; > > + > > +} > > + > > +int > > +sdp_setup_iqs(struct sdp_device *sdpvf, uint32_t iq_no) > > +{ > > + struct sdp_instr_queue *iq; > > + > > + iq =3D (struct sdp_instr_queue *)rte_zmalloc("sdp_IQ", sizeof(*iq), > > + RTE_CACHE_LINE_SIZE); > > + if (iq =3D=3D NULL) > > + return -ENOMEM; > > + > > + sdpvf->instr_queue[iq_no] =3D iq; > > + > > + if (sdp_init_instr_queue(sdpvf, iq_no)) { > > + otx2_err("IQ init is failed"); > > + goto delete_IQ; > > + } > > + otx2_info("IQ[%d] is created.", sdpvf->num_iqs); > > + > > + sdpvf->num_iqs++; > > + > > + > > + return 0; > > + > > +delete_IQ: > > + return -ENOMEM; > > +} > > + > > +static void > > +sdp_droq_reset_indices(struct sdp_droq *droq) > > +{ > > + droq->read_idx =3D 0; > > + droq->write_idx =3D 0; > > + droq->refill_idx =3D 0; > > + droq->refill_count =3D 0; > > + rte_atomic64_set(&droq->pkts_pending, 0); > C11 atomics are recommended to replace the rte_atomic APIs. > More technical reasons are here: > https://urldefense.proofpoint.com/v2/url?u=3Dhttps-3A__www.dpdk.org_wp- > 2Dcontent_uploads_sites_35_2019_10_StateofC11Code.pdf&d=3DDwIFAg&c=3Dn > KjWec2b6R0mOyPaz7xtfQ&r=3D- > s3WtB2eBJ7B88AJuqlVnFqXScHMHYESWbIBMvf7lwI&m=3DvLoVSSkkB3cLz- > ZlbcX68kWjRPvjSykTAzLXKH2sMKw&s=3DEtNWtg_uAWv9X0z0WMPWWwpUmR > noDXbVLUPy0O2bwYA&e=3D=09 [Mahipal]: Thanks for this suggestion, but we would like to go with this fo= r now.=20 > > +} > > + > > +static int > > +sdp_droq_setup_ring_buffers(struct sdp_device *sdpvf, > > + struct sdp_droq *droq) > > +{ > > + struct sdp_droq_desc *desc_ring =3D droq->desc_ring; > > + uint32_t idx; > > + void *buf; > > + > > + for (idx =3D 0; idx < droq->nb_desc; idx++) { > > + rte_mempool_get(sdpvf->enqdeq_mpool, &buf); > > + if (buf =3D=3D NULL) { > > + otx2_err("OQ buffer alloc failed"); > > + /* sdp_droq_destroy_ring_buffers(droq);*/ > > + return -ENOMEM; > > + } > > + > > + droq->recv_buf_list[idx].buffer =3D buf; > > + droq->info_list[idx].length =3D 0; > > + > > + /* Map ring buffers into memory */ > > + desc_ring[idx].info_ptr =3D (uint64_t)(droq->info_list_dma + > > + (idx * SDP_DROQ_INFO_SIZE)); > > + > > + desc_ring[idx].buffer_ptr =3D rte_mem_virt2iova(buf); > > + } > > + > > + sdp_droq_reset_indices(droq); > > + > > + return 0; > > +} > > + > > +static void * > > +sdp_alloc_info_buffer(struct sdp_device *sdpvf __rte_unused, > > + struct sdp_droq *droq) > > +{ > > + droq->info_mz =3D rte_memzone_reserve_aligned("OQ_info_list", > > + (droq->nb_desc * SDP_DROQ_INFO_SIZE), > > + rte_socket_id(), > > + RTE_MEMZONE_IOVA_CONTIG, > > + RTE_CACHE_LINE_SIZE); > > + > > + if (droq->info_mz =3D=3D NULL) > > + return NULL; > > + > > + droq->info_list_dma =3D droq->info_mz->iova; > > + droq->info_alloc_size =3D droq->info_mz->len; > > + droq->info_base_addr =3D (size_t)droq->info_mz->addr; > > + > > + return droq->info_mz->addr; > > +} > > + > > +/* OQ initialization */ > > +static int > > +sdp_init_droq(struct sdp_device *sdpvf, uint32_t q_no) > > +{ > > + const struct sdp_config *conf =3D sdpvf->conf; > > + uint32_t c_refill_threshold; > > + uint32_t desc_ring_size; > > + struct sdp_droq *droq; > > + > > + otx2_info("OQ[%d] Init start", q_no); > > + > > + droq =3D sdpvf->droq[q_no]; > > + droq->sdp_dev =3D sdpvf; > > + droq->q_no =3D q_no; > > + > > + c_refill_threshold =3D conf->oq.refill_threshold; > > + droq->nb_desc =3D conf->num_oqdef_descs; > > + droq->buffer_size =3D conf->oqdef_buf_size; > > + > > + /* OQ desc_ring set up */ > > + desc_ring_size =3D droq->nb_desc * SDP_DROQ_DESC_SIZE; > > + droq->desc_ring_mz =3D rte_memzone_reserve_aligned("sdp_oqmz", > > + desc_ring_size, > > + rte_socket_id(), > > + > > RTE_MEMZONE_IOVA_CONTIG, > > + RTE_CACHE_LINE_SIZE); > > + > > + if (droq->desc_ring_mz =3D=3D NULL) { > > + otx2_err("OQ:%d desc_ring allocation failed", q_no); > > + goto init_droq_fail; > > + } > > + > > + droq->desc_ring_dma =3D droq->desc_ring_mz->iova; > > + droq->desc_ring =3D (struct sdp_droq_desc *)droq->desc_ring_mz- > > >addr; > > + > > + otx2_sdp_dbg("OQ[%d]: desc_ring: virt: 0x%p, dma: %lx", > > + q_no, droq->desc_ring, (unsigned long)droq- > > >desc_ring_dma); > > + otx2_sdp_dbg("OQ[%d]: num_desc: %d", q_no, droq->nb_desc); > > + > > + > > + /* OQ info_list set up */ > > + droq->info_list =3D sdp_alloc_info_buffer(sdpvf, droq); > > + if (droq->info_list =3D=3D NULL) { > > + otx2_err("memory allocation failed for OQ[%d] info_list", > > q_no); > > + goto init_droq_fail; > > + } > > + > > + /* OQ buf_list set up */ > > + droq->recv_buf_list =3D rte_zmalloc_socket("recv_buf_list", > > + (droq->nb_desc * > > SDP_DROQ_RECVBUF_SIZE), > > + RTE_CACHE_LINE_SIZE, rte_socket_id()); > > + if (droq->recv_buf_list =3D=3D NULL) { > > + otx2_err("OQ recv_buf_list alloc failed"); > > + goto init_droq_fail; > > + } > > + > > + if (sdp_droq_setup_ring_buffers(sdpvf, droq)) > > + goto init_droq_fail; > > + > > + droq->refill_threshold =3D c_refill_threshold; > > + rte_spinlock_init(&droq->lock); > > + > > + > > + /* Set up OQ registers */ > > + sdpvf->fn_list.setup_oq_regs(sdpvf, q_no); > > + > > + sdpvf->io_qmask.oq |=3D (1ull << q_no); > > + > > + return 0; > > + > > +init_droq_fail: > > + return -ENOMEM; > > +} > > + > > +/* OQ configuration and setup */ > > +int > > +sdp_setup_oqs(struct sdp_device *sdpvf, uint32_t oq_no) > > +{ > > + struct sdp_droq *droq; > > + > > + /* Allocate new droq. */ > > + droq =3D (struct sdp_droq *)rte_zmalloc("sdp_OQ", > > + sizeof(*droq), RTE_CACHE_LINE_SIZE); > > + if (droq =3D=3D NULL) { > > + otx2_err("Droq[%d] Creation Failed", oq_no); > > + return -ENOMEM; > > + } > > + sdpvf->droq[oq_no] =3D droq; > > + > > + if (sdp_init_droq(sdpvf, oq_no)) { > > + otx2_err("Droq[%d] Initialization failed", oq_no); > > + goto delete_OQ; > > + } > > + otx2_info("OQ[%d] is created.", oq_no); > > + > > + sdpvf->num_oqs++; > > + > > + return 0; > > + > > +delete_OQ: > > + return -ENOMEM; > > +} > > diff --git a/drivers/raw/octeontx2_ep/otx2_ep_enqdeq.h > > b/drivers/raw/octeontx2_ep/otx2_ep_enqdeq.h > > new file mode 100644 > > index 0000000..4c28283 > > --- /dev/null > > +++ b/drivers/raw/octeontx2_ep/otx2_ep_enqdeq.h > > @@ -0,0 +1,11 @@ > > +/* SPDX-License-Identifier: BSD-3-Clause > > + * Copyright(C) 2019 Marvell International Ltd. > > + */ > > + > > +#ifndef _OTX2_EP_ENQDEQ_H_ > > +#define _OTX2_EP_ENQDEQ_H_ > > + > > +#include > > +#include "otx2_ep_rawdev.h" > > + > > +#endif /* _OTX2_EP_ENQDEQ_H_ */ > > diff --git a/drivers/raw/octeontx2_ep/otx2_ep_rawdev.c > > b/drivers/raw/octeontx2_ep/otx2_ep_rawdev.c > > index 04b4fac..0c56609 100644 > > --- a/drivers/raw/octeontx2_ep/otx2_ep_rawdev.c > > +++ b/drivers/raw/octeontx2_ep/otx2_ep_rawdev.c > > @@ -17,6 +17,7 @@ > > > > #include "otx2_common.h" > > #include "otx2_ep_rawdev.h" > > +#include "otx2_ep_vf.h" > > > > static const struct rte_pci_id pci_sdp_vf_map[] =3D { > > { > > @@ -28,6 +29,152 @@ > > }, > > }; > > > > +/* SDP_VF default configuration */ > > +const struct sdp_config default_sdp_conf =3D { > > + /* IQ attributes */ > > + .iq =3D { > > + .max_iqs =3D SDP_VF_CFG_IO_QUEUES, > > + .instr_type =3D SDP_VF_64BYTE_INSTR, > > + .pending_list_size =3D (SDP_VF_MAX_IQ_DESCRIPTORS * > > + SDP_VF_CFG_IO_QUEUES), > > + }, > > + > > + /* OQ attributes */ > > + .oq =3D { > > + .max_oqs =3D SDP_VF_CFG_IO_QUEUES, > > + .info_ptr =3D SDP_VF_OQ_INFOPTR_MODE, > > + .refill_threshold =3D SDP_VF_OQ_REFIL_THRESHOLD, > > + }, > > + > > + .num_iqdef_descs =3D SDP_VF_MAX_IQ_DESCRIPTORS, > > + .num_oqdef_descs =3D SDP_VF_MAX_OQ_DESCRIPTORS, > > + .oqdef_buf_size =3D SDP_VF_OQ_BUF_SIZE, > > + > > +}; > > + > > +const struct sdp_config* > > +sdp_get_defconf(struct sdp_device *sdp_dev __rte_unused) > > +{ > > + const struct sdp_config *default_conf =3D NULL; > > + > > + default_conf =3D &default_sdp_conf; > > + > > + return default_conf; > > +} > > + > > +static int > > +sdp_chip_specific_setup(struct sdp_device *sdpvf) > > +{ > > + struct rte_pci_device *pdev =3D sdpvf->pci_dev; > > + uint32_t dev_id =3D pdev->id.device_id; > > + int ret; > > + > > + switch (dev_id) { > > + case PCI_DEVID_OCTEONTX2_EP_VF: > > + sdpvf->chip_id =3D PCI_DEVID_OCTEONTX2_EP_VF; > > + ret =3D sdp_vf_setup_device(sdpvf); > > + > > + break; > > + default: > > + otx2_err("Unsupported device"); > > + ret =3D -EINVAL; > > + } > > + > > + if (!ret) > > + otx2_info("SDP dev_id[%d]", dev_id); > > + > > + return ret; > > +} > > + > > +/* SDP VF device initialization */ > > +static int > > +sdp_vfdev_init(struct sdp_device *sdpvf) > > +{ > > + uint32_t rawdev_queues, q; > > + > > + if (sdp_chip_specific_setup(sdpvf)) { > > + otx2_err("Chip specific setup failed"); > > + goto setup_fail; > > + } > > + > > + if (sdpvf->fn_list.setup_device_regs(sdpvf)) { > > + otx2_err("Failed to configure device registers"); > > + goto setup_fail; > > + } > > + > > + rawdev_queues =3D (uint32_t)(sdpvf->sriov_info.rings_per_vf); > > + > > + /* Rawdev queues setup for enqueue/dequeue */ > > + for (q =3D 0; q < rawdev_queues; q++) { > > + if (sdp_setup_iqs(sdpvf, q)) { > > + otx2_err("Failed to setup IQs"); > > + goto iq_fail; > > + } > > + } > > + otx2_info("Total[%d] IQs setup", sdpvf->num_iqs); > > + > > + for (q =3D 0; q < rawdev_queues; q++) { > > + if (sdp_setup_oqs(sdpvf, q)) { > > + otx2_err("Failed to setup OQs"); > > + goto oq_fail; > > + } > > + } > > + otx2_info("Total [%d] OQs setup", sdpvf->num_oqs); > > + > > + /* Enable IQ/OQ for this device */ > > + sdpvf->fn_list.enable_io_queues(sdpvf); > > + > > + /* Send OQ desc credits for OQs, credits are always > > + * sent after the OQs are enabled. > > + */ > > + for (q =3D 0; q < rawdev_queues; q++) { > > + rte_write32(sdpvf->droq[q]->nb_desc, > > + sdpvf->droq[q]->pkts_credit_reg); > > + > > + rte_io_mb(); > > + otx2_info("OQ[%d] dbells [%d]", q, > > + rte_read32(sdpvf->droq[q]->pkts_credit_reg)); > > + } > > + > > + rte_wmb(); > > + > > + otx2_info("SDP Device is Ready"); > > + > > + return 0; > > + > > +oq_fail: > > +iq_fail: > > +setup_fail: > > + return -ENOMEM; > > +} > > + > > +static int > > +sdp_rawdev_configure(const struct rte_rawdev *dev, rte_rawdev_obj_t > > config) > > +{ > > + struct sdp_rawdev_info *app_info =3D (struct sdp_rawdev_info > *)config; > > + struct sdp_device *sdpvf; > > + > > + if (app_info =3D=3D NULL) { > > + otx2_err("Application config info [NULL]"); > > + return -EINVAL; > > + } > > + > > + sdpvf =3D (struct sdp_device *)dev->dev_private; > > + > > + sdpvf->conf =3D app_info->app_conf; > > + sdpvf->enqdeq_mpool =3D app_info->enqdeq_mpool; > > + > > + sdp_vfdev_init(sdpvf); > > + > > + return 0; > > + > > +} > > + > > +/* SDP VF endpoint rawdev ops */ > > +static const struct rte_rawdev_ops sdp_rawdev_ops =3D { > > + .dev_configure =3D sdp_rawdev_configure, > > +}; > > + > > static int > > otx2_sdp_rawdev_probe(struct rte_pci_driver *pci_drv __rte_unused, > > struct rte_pci_device *pci_dev) > > @@ -66,6 +213,7 @@ > > return -ENOMEM; > > } > > > > + sdp_rawdev->dev_ops =3D &sdp_rawdev_ops; > > sdp_rawdev->device =3D &pci_dev->device; > > sdp_rawdev->driver_name =3D pci_dev->driver->driver.name; > > > > diff --git a/drivers/raw/octeontx2_ep/otx2_ep_rawdev.h > > b/drivers/raw/octeontx2_ep/otx2_ep_rawdev.h > > index 7ae7a08..bb36b6a 100644 > > --- a/drivers/raw/octeontx2_ep/otx2_ep_rawdev.h > > +++ b/drivers/raw/octeontx2_ep/otx2_ep_rawdev.h > > @@ -5,17 +5,449 @@ > > #ifndef _OTX2_EP_RAWDEV_H_ > > #define _OTX2_EP_RAWDEV_H_ > > > > -#define PCI_DEVID_OCTEONTX2_EP_VF 0xB203 /* OCTEON TX2 EP > mode > > */ > > +#include > > +#include > > + > > + > > +/* Input Request Header format */ > > +struct sdp_instr_irh { > > + /* Request ID */ > > + uint64_t rid:16; > > + > > + /* PCIe port to use for response */ > > + uint64_t pcie_port:3; > > + > > + /* Scatter indicator 1=3Dscatter */ > > + uint64_t scatter:1; > > + > > + /* Size of Expected result OR no. of entries in scatter list */ > > + uint64_t rlenssz:14; > > + > > + /* Desired destination port for result */ > > + uint64_t dport:6; > > + > > + /* Opcode Specific parameters */ > > + uint64_t param:8; > > + > > + /* Opcode for the return packet */ > > + uint64_t opcode:16; > > +}; > > + > > +/* SDP 32B instruction format */ > > +struct sdp_instr_32B { > > + /* Pointer where the input data is available. */ > > + uint64_t dptr; > > + > > + /* SDP Instruction Header. */ > > + uint64_t ih; > > + > > + /** Pointer where the response for a RAW mode packet > > + * will be written by OCTEON TX2. > > + */ > > + uint64_t rptr; > > + > > + /* Input Request Header. Additional info about the input. */ > > + uint64_t irh; > > +}; > > +#define SDP_32B_INSTR_SIZE (sizeof(sdp_instr_32B)) > > + > > +/* SDP 64B instruction format */ > > +struct sdp_instr_64B { > > + /* Pointer where the input data is available. */ > > + uint64_t dptr; > > + > > + /* SDP Instruction Header. */ > > + uint64_t ih; > > + > > + /** Pointer where the response for a RAW mode packet > > + * will be written by OCTEON TX2. > > + */ > > + uint64_t rptr; > > + > > + /* Input Request Header. */ > > + uint64_t irh; > > + > > + /* Additional headers available in a 64-byte instruction. */ > > + uint64_t exhdr[4]; > > +}; > > +#define SDP_64B_INSTR_SIZE (sizeof(sdp_instr_64B)) > > + > > +struct sdp_soft_instr { > > + /** Input data pointer. It is either pointing directly to input data > > + * or to a gather list. > > + */ > > + void *dptr; > > + > > + /** Response from OCTEON TX2 comes at this address. It is either > > + * directlty pointing to output data buffer or to a scatter list. > > + */ > > + void *rptr; > > + > > + /* The instruction header. All input commands have this field. */ > > + struct sdp_instr_ih ih; > > + > > + /* Input request header. */ > > + struct sdp_instr_irh irh; > > + > > + /** The PCI instruction to be sent to OCTEON TX2. This is stored in > the > > + * instr to retrieve the physical address of buffers when instr is > > + * freed. > > + */ > > + struct sdp_instr_64B command; > > + > > + /** If a gather list was allocated, this ptr points to the buffer use= d > > + * for the gather list. The gather list has to be 8B aligned, so thi= s > > + * value may be different from dptr. > > + */ > > + void *gather_ptr; > > + > > + /* Total data bytes transferred in the gather mode request. */ > > + uint64_t gather_bytes; > > + > > + /** If a scatter list was allocated, this ptr points to the buffer us= ed > > + * for the scatter list. The scatter list has to be 8B aligned, so > > + * this value may be different from rptr. > > + */ > > + void *scatter_ptr; > > + > > + /* Total data bytes to be received in the scatter mode request. */ > > + uint64_t scatter_bytes; > > + > > + /* IQ number to which this instruction has to be submitted. */ > > + uint32_t q_no; > > + > > + /* IQ instruction request type. */ > > + uint32_t reqtype; > > +}; > > +#define SDP_SOFT_INSTR_SIZE (sizeof(sdp_soft_instr)) > > + > > +/* SDP IQ request list */ > > +struct sdp_instr_list { > > + void *buf; > > + uint32_t reqtype; > > +}; > > +#define SDP_IQREQ_LIST_SIZE (sizeof(struct sdp_instr_list)) > > + > > +/* Structure to define the configuration attributes for each Input que= ue. > */ > > +struct sdp_iq_config { > > + /* Max number of IQs available */ > > + uint16_t max_iqs; > > + > > + /* Command size - 32 or 64 bytes */ > > + uint16_t instr_type; > > + > > + /* Pending list size, usually set to the sum of the size of all IQs *= / > > + uint32_t pending_list_size; > > +}; > > + > > +/** The instruction (input) queue. > > + * The input queue is used to post raw (instruction) mode data or pac= ket > > data > > + * to OCTEON TX2 device from the host. Each IQ of a SDP EP VF device = has > > one > > + * such structure to represent it. > > + */ > > +struct sdp_instr_queue { > > + /* A spinlock to protect access to the input ring. */ > > + rte_spinlock_t lock; > > + rte_spinlock_t post_lock; > > + > > + struct sdp_device *sdp_dev; > > + rte_atomic64_t iq_flush_running; > > + > > + uint32_t q_no; > > + uint32_t pkt_in_done; > > + > > + /* Flag for 64 byte commands. */ > > + uint32_t iqcmd_64B:1; > > + uint32_t rsvd:17; > > + uint32_t status:8; > > + > > + /* Number of descriptors in this ring. */ > > + uint32_t nb_desc; > > + > > + /* Input ring index, where the driver should write the next packet */ > > + uint32_t host_write_index; > > + > > + /* Input ring index, where the OCTEON TX2 should read the next > > packet */ > > + uint32_t otx_read_index; > > + > > + /** This index aids in finding the window in the queue where > > OCTEON TX2 > > + * has read the commands. > > + */ > > + uint32_t flush_index; > > + > > + /* This keeps track of the instructions pending in this queue. */ > > + rte_atomic64_t instr_pending; > > + > > + uint32_t reset_instr_cnt; > > + > > + /* Pointer to the Virtual Base addr of the input ring. */ > > + uint8_t *base_addr; > > + > > + /* This IQ request list */ > > + struct sdp_instr_list *req_list; > > + > > + /* SDP doorbell register for the ring. */ > > + void *doorbell_reg; > > + > > + /* SDP instruction count register for this ring. */ > > + void *inst_cnt_reg; > > + > > + /* Number of instructions pending to be posted to OCTEON TX2. */ > > + uint32_t fill_cnt; > > + > > + /* DMA mapped base address of the input descriptor ring. */ > > + uint64_t base_addr_dma; > > + > > + /* Memory zone */ > > + const struct rte_memzone *iq_mz; > > +}; > > + > > +/* DROQ packet format for application i/f. */ > > +struct sdp_droq_pkt { > > + /* DROQ packet data buffer pointer. */ > > + uint8_t *data; > > + > > + /* DROQ packet data length */ > > + uint32_t len; > > + > > + uint32_t misc; > > +}; > > + > > +/** Descriptor format. > > + * The descriptor ring is made of descriptors which have 2 64-bit val= ues: > > + * -# Physical (bus) address of the data buffer. > > + * -# Physical (bus) address of a sdp_droq_info structure. > > + * The device DMA's incoming packets and its information at the addre= ss > > + * given by these descriptor fields. > > + */ > > +struct sdp_droq_desc { > > + /* The buffer pointer */ > > + uint64_t buffer_ptr; > > + > > + /* The Info pointer */ > > + uint64_t info_ptr; > > +}; > > +#define SDP_DROQ_DESC_SIZE (sizeof(struct sdp_droq_desc)) > > + > > +/* Receive Header */ > > +union sdp_rh { > > + uint64_t rh64; > > +}; > > +#define SDP_RH_SIZE (sizeof(union sdp_rh)) > > + > > +/** Information about packet DMA'ed by OCTEON TX2. > > + * The format of the information available at Info Pointer after OCTE= ON > TX2 > > + * has posted a packet. Not all descriptors have valid information. O= nly > > + * the Info field of the first descriptor for a packet has informatio= n > > + * about the packet. > > + */ > > +struct sdp_droq_info { > > + /* The Output Receive Header. */ > > + union sdp_rh rh; > > + > > + /* The Length of the packet. */ > > + uint64_t length; > > +}; > > +#define SDP_DROQ_INFO_SIZE (sizeof(struct sdp_droq_info)) > > + > > +/** Pointer to data buffer. > > + * Driver keeps a pointer to the data buffer that it made available t= o > > + * the OCTEON TX2 device. Since the descriptor ring keeps physical (b= us) > > + * addresses, this field is required for the driver to keep track of > > + * the virtual address pointers. > > + */ > > +struct sdp_recv_buffer { > > + /* Packet buffer, including meta data. */ > > + void *buffer; > > + > > + /* Data in the packet buffer. */ > > + /* uint8_t *data; */ > > +}; > > +#define SDP_DROQ_RECVBUF_SIZE (sizeof(struct sdp_recv_buffer)) > > + > > +/* Structure to define the configuration attributes for each Output qu= eue. > */ > > +struct sdp_oq_config { > > + /* Max number of OQs available */ > > + uint16_t max_oqs; > > + > > + /* If set, the Output queue uses info-pointer mode. (Default: 1 ) */ > > + uint16_t info_ptr; > > + > > + /** The number of buffers that were consumed during packet > > processing by > > + * the driver on this Output queue before the driver attempts to > > + * replenish the descriptor ring with new buffers. > > + */ > > + uint32_t refill_threshold; > > +}; > > + > > +/* The Descriptor Ring Output Queue(DROQ) structure. */ > > +struct sdp_droq { > > + /* A spinlock to protect access to this ring. */ > > + rte_spinlock_t lock; > > + > > + struct sdp_device *sdp_dev; > > + /* The 8B aligned descriptor ring starts at this address. */ > > + struct sdp_droq_desc *desc_ring; > > + > > + uint32_t q_no; > > + uint32_t last_pkt_count; > > + > > + /* Driver should read the next packet at this index */ > > + uint32_t read_idx; > > + > > + /* OCTEON TX2 will write the next packet at this index */ > > + uint32_t write_idx; > > + > > + /* At this index, the driver will refill the descriptor's buffer */ > > + uint32_t refill_idx; > > + > > + /* Packets pending to be processed */ > > + rte_atomic64_t pkts_pending; > > + > > + /* Number of descriptors in this ring. */ > > + uint32_t nb_desc; > > + > > + /* The number of descriptors pending to refill. */ > > + uint32_t refill_count; > > + > > + uint32_t refill_threshold; > > + > > + /* The 8B aligned info ptrs begin from this address. */ > > + struct sdp_droq_info *info_list; > > + > > + /* receive buffer list contains virtual addresses of the buffers. */ > > + struct sdp_recv_buffer *recv_buf_list; > > + > > + /* The size of each buffer pointed by the buffer pointer. */ > > + uint32_t buffer_size; > > + > > + /** Pointer to the mapped packet credit register. > > + * Host writes number of info/buffer ptrs available to this register > > + */ > > + void *pkts_credit_reg; > > + > > + /** Pointer to the mapped packet sent register. OCTEON TX2 writes > > the > > + * number of packets DMA'ed to host memory in this register. > > + */ > > + void *pkts_sent_reg; > > + > > + /* DMA mapped address of the DROQ descriptor ring. */ > > + size_t desc_ring_dma; > > + > > + /* Info_ptr list is allocated at this virtual address. */ > > + size_t info_base_addr; > > + > > + /* DMA mapped address of the info list */ > > + size_t info_list_dma; > > + > > + /* Allocated size of info list. */ > > + uint32_t info_alloc_size; > > + > > + /* Memory zone **/ > > + const struct rte_memzone *desc_ring_mz; > > + const struct rte_memzone *info_mz; > > +}; > > +#define SDP_DROQ_SIZE (sizeof(struct sdp_droq)) > > + > > +/* IQ/OQ mask */ > > +struct sdp_io_enable { > > + uint64_t iq; > > + uint64_t oq; > > + uint64_t iq64B; > > +}; > > + > > +/* Structure to define the configuration. */ > > +struct sdp_config { > > + /* Input Queue attributes. */ > > + struct sdp_iq_config iq; > > + > > + /* Output Queue attributes. */ > > + struct sdp_oq_config oq; > > + > > + /* Num of desc for IQ rings */ > > + uint32_t num_iqdef_descs; > > + > > + /* Num of desc for OQ rings */ > > + uint32_t num_oqdef_descs; > > + > > + /* OQ buffer size */ > > + uint32_t oqdef_buf_size; > > +}; > > + > > +/* Required functions for each VF device */ > > +struct sdp_fn_list { > > + void (*setup_iq_regs)(struct sdp_device *sdpvf, uint32_t q_no); > > + void (*setup_oq_regs)(struct sdp_device *sdpvf, uint32_t q_no); > > + int (*setup_device_regs)(struct sdp_device *sdpvf); > > + void (*enable_io_queues)(struct sdp_device *sdpvf); > > + void (*enable_iq)(struct sdp_device *sdpvf, uint32_t q_no); > > + void (*enable_oq)(struct sdp_device *sdpvf, uint32_t q_no); > > +}; > > + > > +/* SRIOV information */ > > +struct sdp_sriov_info { > > + /* Number of rings assigned to VF */ > > + uint32_t rings_per_vf; > > + > > + /* Number of VF devices enabled */ > > + uint32_t num_vfs; > > +}; > > + > > + > > +/* Information to be passed from application */ > > +struct sdp_rawdev_info { > > + struct rte_mempool *enqdeq_mpool; > > + const struct sdp_config *app_conf; > > +}; > > > > /* SDP EP VF device */ > > struct sdp_device { > > /* PCI device pointer */ > > struct rte_pci_device *pci_dev; > > + uint16_t chip_id; > > + uint16_t pf_num; > > uint16_t vf_num; > > > > + /* This device's PCIe port used for traffic. */ > > + uint16_t pcie_port; > > + uint32_t pkind; > > + > > + /* The state of this device */ > > + rte_atomic64_t status; > > + > > /* Memory mapped h/w address */ > > uint8_t *hw_addr; > > > > + struct sdp_fn_list fn_list; > > + > > + /* Num IQs */ > > + uint32_t num_iqs; > > + > > + /* The input instruction queues */ > > + struct sdp_instr_queue > > *instr_queue[SDP_VF_MAX_IOQS_PER_RAWDEV]; > > + > > + /* Num OQs */ > > + uint32_t num_oqs; > > + > > + /* The DROQ output queues */ > > + struct sdp_droq *droq[SDP_VF_MAX_IOQS_PER_RAWDEV]; > > + > > + /* IOQ data buffer pool */ > > + struct rte_mempool *enqdeq_mpool; > > + > > + /* IOQ mask */ > > + struct sdp_io_enable io_qmask; > > + > > + /* SR-IOV info */ > > + struct sdp_sriov_info sriov_info; > > + > > + /* Device configuration */ > > + const struct sdp_config *conf; > > }; > > > > +const struct sdp_config *sdp_get_defconf(struct sdp_device *sdp_dev); > > +int sdp_setup_iqs(struct sdp_device *sdpvf, uint32_t iq_no); > > + > > +int sdp_setup_oqs(struct sdp_device *sdpvf, uint32_t oq_no); > > + > > #endif /* _OTX2_EP_RAWDEV_H_ */ > > diff --git a/drivers/raw/octeontx2_ep/otx2_ep_vf.c > > b/drivers/raw/octeontx2_ep/otx2_ep_vf.c > > new file mode 100644 > > index 0000000..b6120eb > > --- /dev/null > > +++ b/drivers/raw/octeontx2_ep/otx2_ep_vf.c > > @@ -0,0 +1,408 @@ > > +/* SPDX-License-Identifier: BSD-3-Clause > > + * Copyright(C) 2019 Marvell International Ltd. > > + */ > > + > > +#include > > +#include > > +#include > > + > > +#include "otx2_common.h" > > +#include "otx2_ep_rawdev.h" > > +#include "otx2_ep_vf.h" > > + > > +static int > > +sdp_vf_reset_iq(struct sdp_device *sdpvf, int q_no) > > +{ > > + uint64_t loop =3D SDP_VF_BUSY_LOOP_COUNT; > > + volatile uint64_t d64 =3D 0ull; > > + > > + /* There is no RST for a ring. > > + * Clear all registers one by one after disabling the ring > > + */ > > + > > + otx2_write64(d64, sdpvf->hw_addr + SDP_VF_R_IN_ENABLE(q_no)); > > + otx2_write64(d64, sdpvf->hw_addr + > > SDP_VF_R_IN_INSTR_BADDR(q_no)); > > + otx2_write64(d64, sdpvf->hw_addr + > > SDP_VF_R_IN_INSTR_RSIZE(q_no)); > > + > > + d64 =3D 0xFFFFFFFF; /* ~0ull */ > > + otx2_write64(d64, sdpvf->hw_addr + > > SDP_VF_R_IN_INSTR_DBELL(q_no)); > > + d64 =3D otx2_read64(sdpvf->hw_addr + > > SDP_VF_R_IN_INSTR_DBELL(q_no)); > > + > > + while ((d64 !=3D 0) && loop--) { > > + otx2_write64(d64, sdpvf->hw_addr + > > + SDP_VF_R_IN_INSTR_DBELL(q_no)); > > + > > + rte_delay_ms(1); > > + > > + d64 =3D otx2_read64(sdpvf->hw_addr + > > + SDP_VF_R_IN_INSTR_DBELL(q_no)); > > + } > > + > > + loop =3D SDP_VF_BUSY_LOOP_COUNT; > > + d64 =3D otx2_read64(sdpvf->hw_addr + SDP_VF_R_IN_CNTS(q_no)); > > + while ((d64 !=3D 0) && loop--) { > > + otx2_write64(d64, sdpvf->hw_addr + > > SDP_VF_R_IN_CNTS(q_no)); > > + > > + rte_delay_ms(1); > > + > > + d64 =3D otx2_read64(sdpvf->hw_addr + > > SDP_VF_R_IN_CNTS(q_no)); > > + } > > + > > + d64 =3D 0ull; > > + otx2_write64(d64, sdpvf->hw_addr + > > SDP_VF_R_IN_INT_LEVELS(q_no)); > > + otx2_write64(d64, sdpvf->hw_addr + SDP_VF_R_IN_PKT_CNT(q_no)); > > + otx2_write64(d64, sdpvf->hw_addr + > > SDP_VF_R_IN_BYTE_CNT(q_no)); > > + > > + return 0; > > +} > > + > > +static int > > +sdp_vf_reset_oq(struct sdp_device *sdpvf, int q_no) > > +{ > > + uint64_t loop =3D SDP_VF_BUSY_LOOP_COUNT; > > + volatile uint64_t d64 =3D 0ull; > > + > > + otx2_write64(d64, sdpvf->hw_addr + > SDP_VF_R_OUT_ENABLE(q_no)); > > + > > + otx2_write64(d64, sdpvf->hw_addr + > > SDP_VF_R_OUT_SLIST_BADDR(q_no)); > > + > > + otx2_write64(d64, sdpvf->hw_addr + > > SDP_VF_R_OUT_SLIST_RSIZE(q_no)); > > + > > + d64 =3D 0xFFFFFFFF; > > + otx2_write64(d64, sdpvf->hw_addr + > > SDP_VF_R_OUT_SLIST_DBELL(q_no)); > > + d64 =3D otx2_read64(sdpvf->hw_addr + > > SDP_VF_R_OUT_SLIST_DBELL(q_no)); > > + > > + while ((d64 !=3D 0) && loop--) { > > + otx2_write64(d64, sdpvf->hw_addr + > > + SDP_VF_R_OUT_SLIST_DBELL(q_no)); > > + > > + rte_delay_ms(1); > > + > > + d64 =3D otx2_read64(sdpvf->hw_addr + > > + SDP_VF_R_OUT_SLIST_DBELL(q_no)); > > + } > > + > > + loop =3D SDP_VF_BUSY_LOOP_COUNT; > > + d64 =3D otx2_read64(sdpvf->hw_addr + SDP_VF_R_OUT_CNTS(q_no)); > > + while ((d64 !=3D 0) && (loop--)) { > > + otx2_write64(d64, sdpvf->hw_addr + > > SDP_VF_R_OUT_CNTS(q_no)); > > + > > + rte_delay_ms(1); > > + > > + d64 =3D otx2_read64(sdpvf->hw_addr + > > SDP_VF_R_OUT_CNTS(q_no)); > > + } > > + > > + d64 =3D 0ull; > > + otx2_write64(d64, sdpvf->hw_addr + > > SDP_VF_R_OUT_INT_LEVELS(q_no)); > > + otx2_write64(d64, sdpvf->hw_addr + > > SDP_VF_R_OUT_PKT_CNT(q_no)); > > + otx2_write64(d64, sdpvf->hw_addr + > > SDP_VF_R_OUT_BYTE_CNT(q_no)); > > + > > + return 0; > > +} > > + > > +static void > > +sdp_vf_setup_global_iq_reg(struct sdp_device *sdpvf, int q_no) > > +{ > > + volatile uint64_t reg_val =3D 0ull; > > + > > + /* Select ES, RO, NS, RDSIZE,DPTR Fomat#0 for IQs > > + * IS_64B is by default enabled. > > + */ > > + reg_val =3D otx2_read64(sdpvf->hw_addr + > > SDP_VF_R_IN_CONTROL(q_no)); > > + > > + reg_val |=3D SDP_VF_R_IN_CTL_RDSIZE; > > + reg_val |=3D SDP_VF_R_IN_CTL_IS_64B; > > + reg_val |=3D SDP_VF_R_IN_CTL_ESR; > > + > > + otx2_write64(reg_val, sdpvf->hw_addr + > > SDP_VF_R_IN_CONTROL(q_no)); > > + > > +} > > + > > +static void > > +sdp_vf_setup_global_oq_reg(struct sdp_device *sdpvf, int q_no) > > +{ > > + volatile uint64_t reg_val =3D 0ull; > > + > > + reg_val =3D otx2_read64(sdpvf->hw_addr + > > SDP_VF_R_OUT_CONTROL(q_no)); > > + > > + reg_val |=3D (SDP_VF_R_OUT_CTL_IMODE); > > + > > + reg_val &=3D ~(SDP_VF_R_OUT_CTL_ROR_P); > > + reg_val &=3D ~(SDP_VF_R_OUT_CTL_NSR_P); > > + reg_val &=3D ~(SDP_VF_R_OUT_CTL_ROR_I); > > + reg_val &=3D ~(SDP_VF_R_OUT_CTL_NSR_I); > > + reg_val &=3D ~(SDP_VF_R_OUT_CTL_ES_I); > > + reg_val &=3D ~(SDP_VF_R_OUT_CTL_ROR_D); > > + reg_val &=3D ~(SDP_VF_R_OUT_CTL_NSR_D); > > + reg_val &=3D ~(SDP_VF_R_OUT_CTL_ES_D); > > + > > + /* INFO/DATA ptr swap is required */ > > + reg_val |=3D (SDP_VF_R_OUT_CTL_ES_P); > > + > > + otx2_write64(reg_val, sdpvf->hw_addr + > > SDP_VF_R_OUT_CONTROL(q_no)); > > + > > +} > > + > > +static int > > +sdp_vf_reset_input_queues(struct sdp_device *sdpvf) > > +{ > > + uint32_t q_no =3D 0; > > + > > + otx2_sdp_dbg("%s :", __func__); > > + > > + for (q_no =3D 0; q_no < sdpvf->sriov_info.rings_per_vf; q_no++) > > + sdp_vf_reset_iq(sdpvf, q_no); > > + > > + return 0; > > +} > > + > > +static int > > +sdp_vf_reset_output_queues(struct sdp_device *sdpvf) > > +{ > > + uint64_t q_no =3D 0ull; > > + > > + otx2_sdp_dbg(" %s :", __func__); > > + > > + for (q_no =3D 0; q_no < sdpvf->sriov_info.rings_per_vf; q_no++) > > + sdp_vf_reset_oq(sdpvf, q_no); > > + > > + return 0; > > +} > > + > > +static void > > +sdp_vf_setup_global_input_regs(struct sdp_device *sdpvf) > > +{ > > + uint64_t q_no =3D 0ull; > > + > > + sdp_vf_reset_input_queues(sdpvf); > > + > > + for (q_no =3D 0; q_no < (sdpvf->sriov_info.rings_per_vf); q_no++) > > + sdp_vf_setup_global_iq_reg(sdpvf, q_no); > > +} > > + > > +static void > > +sdp_vf_setup_global_output_regs(struct sdp_device *sdpvf) > > +{ > > + uint32_t q_no; > > + > > + sdp_vf_reset_output_queues(sdpvf); > > + > > + for (q_no =3D 0; q_no < (sdpvf->sriov_info.rings_per_vf); q_no++) > > + sdp_vf_setup_global_oq_reg(sdpvf, q_no); > > + > > +} > > + > > +static int > > +sdp_vf_setup_device_regs(struct sdp_device *sdpvf) > > +{ > > + sdp_vf_setup_global_input_regs(sdpvf); > > + sdp_vf_setup_global_output_regs(sdpvf); > > + > > + return 0; > > +} > > + > > +static void > > +sdp_vf_setup_iq_regs(struct sdp_device *sdpvf, uint32_t iq_no) > > +{ > > + struct sdp_instr_queue *iq =3D sdpvf->instr_queue[iq_no]; > > + volatile uint64_t reg_val =3D 0ull; > > + > > + reg_val =3D otx2_read64(sdpvf->hw_addr + > > SDP_VF_R_IN_CONTROL(iq_no)); > > + > > + /* Wait till IDLE to set to 1, not supposed to configure BADDR > > + * as long as IDLE is 0 > > + */ > > + if (!(reg_val & SDP_VF_R_IN_CTL_IDLE)) { > > + do { > > + reg_val =3D otx2_read64(sdpvf->hw_addr + > > + SDP_VF_R_IN_CONTROL(iq_no)); > > + } while (!(reg_val & SDP_VF_R_IN_CTL_IDLE)); > > + } > > + > > + /* Write the start of the input queue's ring and its size */ > > + otx2_write64(iq->base_addr_dma, sdpvf->hw_addr + > > + SDP_VF_R_IN_INSTR_BADDR(iq_no)); > > + otx2_write64(iq->nb_desc, sdpvf->hw_addr + > > + SDP_VF_R_IN_INSTR_RSIZE(iq_no)); > > + > > + /* Remember the doorbell & instruction count register addr > > + * for this queue > > + */ > > + iq->doorbell_reg =3D (uint8_t *) sdpvf->hw_addr + > > + SDP_VF_R_IN_INSTR_DBELL(iq_no); > > + iq->inst_cnt_reg =3D (uint8_t *) sdpvf->hw_addr + > > + SDP_VF_R_IN_CNTS(iq_no); > > + > > + otx2_sdp_dbg("InstQ[%d]:dbell reg @ 0x%p instcnt_reg @ 0x%p", > > + iq_no, iq->doorbell_reg, iq->inst_cnt_reg); > > + > > + /* Store the current instrn counter(used in flush_iq calculation) */ > > + iq->reset_instr_cnt =3D rte_read32(iq->inst_cnt_reg); > > + > > + /* IN INTR_THRESHOLD is set to max(FFFFFFFF) which disable the IN > > INTR > > + * to raise > > + */ > > + reg_val =3D otx2_read64(sdpvf->hw_addr + > > SDP_VF_R_IN_INT_LEVELS(iq_no)); > > + reg_val =3D 0xffffffff; > > + > > + otx2_write64(reg_val, sdpvf->hw_addr + > > SDP_VF_R_IN_INT_LEVELS(iq_no)); > > + > > +} > > + > > +static void > > +sdp_vf_setup_oq_regs(struct sdp_device *sdpvf, uint32_t oq_no) > > +{ > > + volatile uint64_t reg_val =3D 0ull; > > + uint64_t oq_ctl =3D 0ull; > > + > > + struct sdp_droq *droq =3D sdpvf->droq[oq_no]; > > + > > + /* Wait on IDLE to set to 1, supposed to configure BADDR > > + * as log as IDLE is 0 > > + */ > > + reg_val =3D otx2_read64(sdpvf->hw_addr + > > SDP_VF_R_OUT_CONTROL(oq_no)); > > + > > + while (!(reg_val & SDP_VF_R_OUT_CTL_IDLE)) { > > + reg_val =3D otx2_read64(sdpvf->hw_addr + > > + SDP_VF_R_OUT_CONTROL(oq_no)); > > + } > > + > > + otx2_write64(droq->desc_ring_dma, sdpvf->hw_addr + > > + SDP_VF_R_OUT_SLIST_BADDR(oq_no)); > > + otx2_write64(droq->nb_desc, sdpvf->hw_addr + > > + SDP_VF_R_OUT_SLIST_RSIZE(oq_no)); > > + > > + oq_ctl =3D otx2_read64(sdpvf->hw_addr + > > SDP_VF_R_OUT_CONTROL(oq_no)); > > + > > + /* Clear the ISIZE and BSIZE (22-0) */ > > + oq_ctl &=3D ~(0x7fffffull); > > + > > + /* Populate the BSIZE (15-0) */ > > + oq_ctl |=3D (droq->buffer_size & 0xffff); > > + > > + /* Populate ISIZE(22-16) */ > > + oq_ctl |=3D ((SDP_RH_SIZE << 16) & 0x7fffff); > > + otx2_write64(oq_ctl, sdpvf->hw_addr + > > SDP_VF_R_OUT_CONTROL(oq_no)); > > + > > + /* Mapped address of the pkt_sent and pkts_credit regs */ > > + droq->pkts_sent_reg =3D (uint8_t *) sdpvf->hw_addr + > > + SDP_VF_R_OUT_CNTS(oq_no); > > + droq->pkts_credit_reg =3D (uint8_t *) sdpvf->hw_addr + > > + SDP_VF_R_OUT_SLIST_DBELL(oq_no); > > + > > + reg_val =3D otx2_read64(sdpvf->hw_addr + > > SDP_VF_R_OUT_INT_LEVELS(oq_no)); > > + > > + /* Clear PKT_CNT register */ > > + rte_write64(0xFFFFFFFFF, (uint8_t *)sdpvf->hw_addr + > > + SDP_VF_R_OUT_PKT_CNT(oq_no)); > > + > > + /* Clear the OQ doorbell */ > > + rte_write32(0xFFFFFFFF, droq->pkts_credit_reg); > > + while ((rte_read32(droq->pkts_credit_reg) !=3D 0ull)) { > > + rte_write32(0xFFFFFFFF, droq->pkts_credit_reg); > > + rte_delay_ms(1); > > + } > > + otx2_sdp_dbg("SDP_R[%d]_credit:%x", oq_no, > > + rte_read32(droq->pkts_credit_reg)); > > + > > + /* Clear the OQ_OUT_CNTS doorbell */ > > + reg_val =3D rte_read32(droq->pkts_sent_reg); > > + rte_write32((uint32_t)reg_val, droq->pkts_sent_reg); > > + > > + otx2_sdp_dbg("SDP_R[%d]_sent: %x", oq_no, > > + rte_read32(droq->pkts_sent_reg)); > > + > > + while (((rte_read32(droq->pkts_sent_reg)) !=3D 0ull)) { > > + reg_val =3D rte_read32(droq->pkts_sent_reg); > > + rte_write32((uint32_t)reg_val, droq->pkts_sent_reg); > > + rte_delay_ms(1); > > + } > > + > > +} > > + > > +static void > > +sdp_vf_enable_iq(struct sdp_device *sdpvf, uint32_t q_no) > > +{ > > + volatile uint64_t reg_val =3D 0ull; > > + uint64_t loop =3D SDP_VF_BUSY_LOOP_COUNT; > > + > > + /* Resetting doorbells during IQ enabling also to handle abrupt > > + * guest reboot. IQ reset does not clear the doorbells. > > + */ > > + otx2_write64(0xFFFFFFFF, sdpvf->hw_addr + > > + SDP_VF_R_IN_INSTR_DBELL(q_no)); > > + > > + while (((otx2_read64(sdpvf->hw_addr + > > + SDP_VF_R_IN_INSTR_DBELL(q_no))) !=3D 0ull) && loop--) { > > + > > + rte_delay_ms(1); > > + } > > + > > + reg_val =3D otx2_read64(sdpvf->hw_addr + > > SDP_VF_R_IN_ENABLE(q_no)); > > + reg_val |=3D 0x1ull; > > + > > + otx2_write64(reg_val, sdpvf->hw_addr + > > SDP_VF_R_IN_ENABLE(q_no)); > > + > > + otx2_info("IQ[%d] enable done", q_no); > > + > > +} > > + > > +static void > > +sdp_vf_enable_oq(struct sdp_device *sdpvf, uint32_t q_no) > > +{ > > + volatile uint64_t reg_val =3D 0ull; > > + > > + reg_val =3D otx2_read64(sdpvf->hw_addr + > > SDP_VF_R_OUT_ENABLE(q_no)); > > + reg_val |=3D 0x1ull; > > + otx2_write64(reg_val, sdpvf->hw_addr + > > SDP_VF_R_OUT_ENABLE(q_no)); > > + > > + otx2_info("OQ[%d] enable done", q_no); > > +} > > + > > +static void > > +sdp_vf_enable_io_queues(struct sdp_device *sdpvf) > > +{ > > + uint32_t q_no =3D 0; > > + > > + for (q_no =3D 0; q_no < sdpvf->num_iqs; q_no++) > > + sdp_vf_enable_iq(sdpvf, q_no); > > + > > + for (q_no =3D 0; q_no < sdpvf->num_oqs; q_no++) > > + sdp_vf_enable_oq(sdpvf, q_no); > > +} > > + > > +int > > +sdp_vf_setup_device(struct sdp_device *sdpvf) > > +{ > > + uint64_t reg_val =3D 0ull; > > + > > + /* If application doesn't provide its conf, use driver default conf *= / > > + if (sdpvf->conf =3D=3D NULL) { > > + sdpvf->conf =3D sdp_get_defconf(sdpvf); > > + if (sdpvf->conf =3D=3D NULL) { > > + otx2_err("SDP VF default config not found"); > > + return -ENOMEM; > > + } > > + otx2_info("Default config is used"); > > + } > > + > > + /* Get IOQs (RPVF] count */ > > + reg_val =3D otx2_read64(sdpvf->hw_addr + > SDP_VF_R_IN_CONTROL(0)); > > + > > + sdpvf->sriov_info.rings_per_vf =3D ((reg_val >> > > SDP_VF_R_IN_CTL_RPVF_POS) > > + & SDP_VF_R_IN_CTL_RPVF_MASK); > > + > > + otx2_info("SDP RPVF: %d", sdpvf->sriov_info.rings_per_vf); > > + > > + sdpvf->fn_list.setup_iq_regs =3D sdp_vf_setup_iq_regs; > > + sdpvf->fn_list.setup_oq_regs =3D sdp_vf_setup_oq_regs; > > + sdpvf->fn_list.setup_device_regs =3D sdp_vf_setup_device_regs; > > + sdpvf->fn_list.enable_io_queues =3D sdp_vf_enable_io_queues; > > + sdpvf->fn_list.enable_iq =3D sdp_vf_enable_iq; > > + sdpvf->fn_list.enable_oq =3D sdp_vf_enable_oq; > > + > > + > > + return 0; > > + > > +} > > + > > diff --git a/drivers/raw/octeontx2_ep/otx2_ep_vf.h > > b/drivers/raw/octeontx2_ep/otx2_ep_vf.h > > new file mode 100644 > > index 0000000..974f856 > > --- /dev/null > > +++ b/drivers/raw/octeontx2_ep/otx2_ep_vf.h > > @@ -0,0 +1,10 @@ > > +/* SPDX-License-Identifier: BSD-3-Clause > > + * Copyright(C) 2019 Marvell International Ltd. > > + */ > > +#ifndef _OTX2_EP_VF_H_ > > +#define _OTX2_EP_VF_H_ > > +int > > +sdp_vf_setup_device(struct sdp_device *sdpvf); > > + > > +#endif /*_OTX2_EP_VF_H_ */ > > + > > -- > > 1.8.3.1 Thanks, Mahipal