From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 0DF7043204; Thu, 26 Oct 2023 08:47:30 +0200 (CEST) Received: from mails.dpdk.org (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 9B3A742E4B; Thu, 26 Oct 2023 08:44:33 +0200 (CEST) Received: from NAM10-DM6-obe.outbound.protection.outlook.com (mail-dm6nam10on2112.outbound.protection.outlook.com [40.107.93.112]) by mails.dpdk.org (Postfix) with ESMTP id 62DDD40A8A for ; Thu, 26 Oct 2023 08:44:26 +0200 (CEST) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=hYSmRMv0eIDBtpS5+KuV925kZDx61pxLdWStPsPA8jcjW+nZaNDSARtG5qcxtFrMddke72fci706KV9oOIQJe9W6KqaGB8/PiF5p6ScKtBupfRxztkMnD2ksUhhBMODTESyBaRtdU/AmzB5RV9HCYhy9295nqY090HRJfRPThnIwGvYUia6LrRsA2VaOaFBjOwOG90AkhT6ApibUaD5rPx5WSCp0MEwb4gx/lLJ+Tt+CdykZqt3mrifHmqBb3Tk2P//iUTTR1Ev6l2RsQTBe2GmwilV+uKyBMOmHqre6aJOH0s7UgvDVvYwXOZ/1MIEu0xh4orWTnWUYlyg7uifbqg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=//vcj9C4dCYeyhEFcez77fR7dZTUeDRee4UCLmkkU5I=; b=TxTf9GyE39vKrOgUEB8mUHrXzWDX2cJAyfLEJ9Br6EabWCDf6aIE4rM5ti1QJmsL8EPUsbenjMcV/BlpuinhilQF/68kqpSutxNfbNbwTT00qwkjq59xpRZJTCb88/blONnqKhub5Ex50yIMOISHyWbYxCcChesrAtxAXfV45kdzuUPgcl0WMMUQYOlmQvumQsIuFu17HMGHoyhaDgKkMXX0jMBZfEjazrgJjaUq9zyueyN5g6hFqkuAomJ3Nc8mndnBB8/oRbWxwWNFchUvB2auJLaw/MQLBZxdJZDpXHxXUuypbjXG32mUy2YWfQLbR3LMkazFLnzUB1EN+z62Sw== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=corigine.com; dmarc=pass action=none header.from=corigine.com; dkim=pass header.d=corigine.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=corigine.onmicrosoft.com; s=selector2-corigine-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=//vcj9C4dCYeyhEFcez77fR7dZTUeDRee4UCLmkkU5I=; b=A1VdaS65JmIf+wNhg63t+SLtLYscsB0Dhn6ldw2GNMw5xAbShbScFT7sy133B+gFOZfe8bHr5tIeA8TcmY8sBXFvRUBWHAsqZw7Q99BkuvFSAX5novKMZ0hWVrMq9KIWKWAWvOQcvV7zRsb4cYuM1bv3rUb2bFcX+gEamTO01rU= Authentication-Results: dkim=none (message not signed) header.d=none;dmarc=none action=none header.from=corigine.com; Received: from SJ0PR13MB5545.namprd13.prod.outlook.com (2603:10b6:a03:424::5) by DM4PR13MB5908.namprd13.prod.outlook.com (2603:10b6:8:50::21) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.6933.19; Thu, 26 Oct 2023 06:44:25 +0000 Received: from SJ0PR13MB5545.namprd13.prod.outlook.com ([fe80::45b3:d83c:cff2:4a1b]) by SJ0PR13MB5545.namprd13.prod.outlook.com ([fe80::45b3:d83c:cff2:4a1b%6]) with mapi id 15.20.6933.022; Thu, 26 Oct 2023 06:44:24 +0000 From: Chaoyong He To: dev@dpdk.org Cc: oss-drivers@corigine.com, Chaoyong He , Shujing Dong , Long Wu , Peng Zhang Subject: [PATCH v3 21/25] vdpa/nfp: add the hardware init logic Date: Thu, 26 Oct 2023 14:43:20 +0800 Message-Id: <20231026064324.177531-22-chaoyong.he@corigine.com> X-Mailer: git-send-email 2.39.1 In-Reply-To: <20231026064324.177531-1-chaoyong.he@corigine.com> References: <20231024022849.3092506-1-chaoyong.he@corigine.com> <20231026064324.177531-1-chaoyong.he@corigine.com> Content-Transfer-Encoding: 8bit Content-Type: text/plain X-ClientProxiedBy: BYAPR07CA0034.namprd07.prod.outlook.com (2603:10b6:a02:bc::47) To SJ0PR13MB5545.namprd13.prod.outlook.com (2603:10b6:a03:424::5) MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: SJ0PR13MB5545:EE_|DM4PR13MB5908:EE_ X-MS-Office365-Filtering-Correlation-Id: 880c915f-718e-4c80-c1fc-08dbd5eefe31 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: WJ2iPezUbZwXvRY5m5w5phh56usuOOHGEYMloTxNSpFMYTaQfHCj71YIByLE8kdH/ByRWWPtAyHunZA1/R8jc7HuiKMxOvXxEGOYpvw9ZNGsrjFiCAepH41zXEGf6UIH5RXAA5Y3dNvBJctihMTU6R2xBscwDatwRav8ypXS0DzcNNwnE0NqXK/HvbVKqyzfytSA9Ckzt8lZAmrHTCCw1xhEj/p2iF/jjLpYFrda5H5JBelMSCJlXvvv3nH993lzPaKZLz9Chvubw16qWeGmNtagRhSEQf4bHKEtfQpSbQuQBiPVR/RQcA9+erT8ejMqxIbpY71fkTUZN2PWqnEAfq4Ch7/dZPfVcTVcuayGkJCBJhXUqd2J96RxnfT6kbj9/P9jSF57Ih6y/J/psTzkNk3ZLieEA29z7vwSuU7DXGMIY8fQCNeFkWAsXmeC+ird4PEdgMsyIOp1TQno5p+Yp1G6+QzhrxmVlctoXuIN+MLzjJtDShcz9efC/6iyiNqvco5SM7dAX04DuJs2bTmVbUO+zG3SDdOm1dfk+7iatAzrchn8vJszHkDQLUfyvU+KgCz6dx5+4ErrYP7m4Tz4b8CFYkFZMqW29YARJ4tRnB0NzCoW6UqRoWQHvqQw7SkQD9s+nH1uH4epR/jxicqZ2ryKr0eTMBV3/2SgyGxRuhM= X-Forefront-Antispam-Report: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:SJ0PR13MB5545.namprd13.prod.outlook.com; PTR:; CAT:NONE; SFS:(13230031)(136003)(376002)(366004)(396003)(346002)(39840400004)(230922051799003)(186009)(1800799009)(451199024)(64100799003)(26005)(478600001)(1076003)(316002)(6666004)(6512007)(86362001)(6506007)(6916009)(66946007)(66556008)(38100700002)(66476007)(54906003)(36756003)(38350700005)(2616005)(6486002)(107886003)(44832011)(2906002)(83380400001)(4326008)(8676002)(8936002)(5660300002)(41300700001)(52116002); DIR:OUT; SFP:1102; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: =?us-ascii?Q?uvrU6+2tNZnBtKZfipQCOFAdczRQk0JLAqmcHGjhJIO5V7LWguo4a96hof20?= =?us-ascii?Q?se1fXDvZcek5LfqYKPeThD05u2/+aTLRAy6X6Kt0RKsgMtekIylQoJjfdBrv?= =?us-ascii?Q?atjl8KmPpgC9L+nHVFKnJIX67OJ0/SqT6BIcnyvDSYa48p0Kf1o4H45aLp+L?= =?us-ascii?Q?L+S0ePWo3zMWYCWE/cPQDoJQo9qqUiL9hEZoqethtQmZlrffU52aV343CjXb?= =?us-ascii?Q?PAO4pyciBUjVMsLOOkiaAgqipe2z867NieQY2y0gFw9WQUDO/oBxhd0HtAN1?= =?us-ascii?Q?yNC9S18CgZxScSyCSbVmGTwWVw9rMeSr9ajxfsin97VexMqvdA4SHqY0hccG?= =?us-ascii?Q?H7ulmJ55M09vr+0Fi8Oha70iabLJZr5jAbzz4TzE+Pp9EsfEfS3mmMv9uNVK?= =?us-ascii?Q?rqJrDj8zgh8pgYiih1+rXk4L0ZgZUq1lcy7TORBxDnCJDcmto6YJMRm3+2vd?= =?us-ascii?Q?o/KJ4n7o2xIWWkiWCvoNC4oCeRqDSlX28NRSmix9lff0I5qw//iINsQlJF8H?= =?us-ascii?Q?AvtCe+/3KhOiLwlVpBvMp1E5jYfQReCd+aX998wa0ylIw8nuKZh9npvzQa7R?= =?us-ascii?Q?4TOL01Jrh8r+wuA09xkWCapJhxMcsdjJHjlDJp99KXiuA9e+QzTHtLMXNKgl?= =?us-ascii?Q?QNj3HIJ6w77RwdCc9CZRxqPA5Nxw6qDEqGfJL6IZvvxHHDLwH4z/MlVsKhY1?= =?us-ascii?Q?bseSzfqd5W/X+CTwrmNNAKEPLCgk8QwU6a/0pHo6vVPISdOcTeGsvNnfo6Gq?= =?us-ascii?Q?RbvYP0TEKKK2XCXsdzEs/oQf2wtfKwP/E3CKQob7z3/xAaPqxRNhS9s0OjQi?= =?us-ascii?Q?/dfH95ZbDfujJWOekmUgahmNPiPeU2sxi4avNxOak35Pd0FmHNZGGWVq5DCL?= =?us-ascii?Q?JNwGNhEmr+oPuEKsom1N0S6/SHciIQMzVRnshp5kUwqfZQXaCiYO1a+Yr4v8?= =?us-ascii?Q?hX735ZLMpcLTeroTtUCoB7k+u3QNfMnBueTzh4wAXhXmyha1BmjLPw5m61Cb?= =?us-ascii?Q?ymkcallvAvGLxRwOVDtao/FDWI4vrTn4LDPR3bNQ7fLEpxes4pp8mjRSlGPX?= =?us-ascii?Q?/x59aYPaEdbZNyNVVNLcoFuKr9Ag9ahoImYrlJvDsLpfXCraXLz5+8UAyPho?= =?us-ascii?Q?Y6vZK+6T9y6ufjr0rzojCkFQOOSFTGDeBwLJLolY2DKOdZZs3Q2J3HPMxQrG?= =?us-ascii?Q?WHocZIN5T3DJjXjoBlZr1y8D0VC5xqlCDFqCzf0+IFyDc0OdBdy6RC8uRoIJ?= =?us-ascii?Q?bCnjDr0SYg/0aB0GVweLlm0bpwi04+zDMdq8vRr5ovJhGztjf98LQrSAzsMY?= =?us-ascii?Q?10kz0nAKYEqavIGMJhCz7dnNnB4ShE4c3XbaZzHOJPK4YVPBeILZ9mL9ZAEn?= =?us-ascii?Q?NyrPJ6qFXGYhefNhpNABi4nmyCFarj/qQ6bITcUnUMcZBFV1l9eKxSAPKwHs?= =?us-ascii?Q?suLhi3gUNbRnkkdl0leHD7hoJYJHknc5nqckh1d3X5aNLFPvx77o40dHKfbB?= =?us-ascii?Q?sxYem4i7tt+J7nWNfmSIwhhrYjMeRFnB35eipmprIOF5RKh36mv5KSAOzRyf?= =?us-ascii?Q?LJotZr/xiu/DnMrNVkszTu9DyBS6R8Huoh28U4BxoBQjvutNXbSO2Djy9L62?= =?us-ascii?Q?eg=3D=3D?= X-OriginatorOrg: corigine.com X-MS-Exchange-CrossTenant-Network-Message-Id: 880c915f-718e-4c80-c1fc-08dbd5eefe31 X-MS-Exchange-CrossTenant-AuthSource: SJ0PR13MB5545.namprd13.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 26 Oct 2023 06:44:24.9739 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: fe128f2c-073b-4c20-818e-7246a585940c X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: ibK15eEInCGXGdq4OcZHTC0UDrwA+GlGcla4Hy5AyeK91j7Z9gKHUVhfh0BqSPtpUn3A+ZtCyXywfu7M3QamUjHm3n7PRsGEo4uS1lFt8lY= X-MS-Exchange-Transport-CrossTenantHeadersStamped: DM4PR13MB5908 X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Add the hardware initialize logic. Signed-off-by: Chaoyong He Signed-off-by: Shujing Dong Reviewed-by: Long Wu Reviewed-by: Peng Zhang --- drivers/vdpa/nfp/meson.build | 1 + drivers/vdpa/nfp/nfp_vdpa.c | 10 ++++++ drivers/vdpa/nfp/nfp_vdpa_core.c | 54 ++++++++++++++++++++++++++++++++ drivers/vdpa/nfp/nfp_vdpa_core.h | 32 +++++++++++++++++++ 4 files changed, 97 insertions(+) create mode 100644 drivers/vdpa/nfp/nfp_vdpa_core.c create mode 100644 drivers/vdpa/nfp/nfp_vdpa_core.h diff --git a/drivers/vdpa/nfp/meson.build b/drivers/vdpa/nfp/meson.build index 79a0364917..258f3eaaa0 100644 --- a/drivers/vdpa/nfp/meson.build +++ b/drivers/vdpa/nfp/meson.build @@ -8,6 +8,7 @@ endif sources = files( 'nfp_vdpa.c', + 'nfp_vdpa_core.c', 'nfp_vdpa_log.c', ) diff --git a/drivers/vdpa/nfp/nfp_vdpa.c b/drivers/vdpa/nfp/nfp_vdpa.c index 824b0d681a..00d8f7e007 100644 --- a/drivers/vdpa/nfp/nfp_vdpa.c +++ b/drivers/vdpa/nfp/nfp_vdpa.c @@ -10,6 +10,7 @@ #include #include +#include "nfp_vdpa_core.h" #include "nfp_vdpa_log.h" #define NFP_VDPA_DRIVER_NAME nfp_vdpa @@ -17,11 +18,14 @@ struct nfp_vdpa_dev { struct rte_pci_device *pci_dev; struct rte_vdpa_device *vdev; + struct nfp_vdpa_hw hw; int vfio_container_fd; int vfio_group_fd; int vfio_dev_fd; int iommu_group; + + uint16_t max_queues; }; struct nfp_vdpa_dev_node { @@ -135,6 +139,12 @@ nfp_vdpa_pci_probe(struct rte_pci_device *pci_dev) if (ret != 0) goto free_device; + ret = nfp_vdpa_hw_init(&device->hw, pci_dev); + if (ret != 0) + goto vfio_teardown; + + device->max_queues = NFP_VDPA_MAX_QUEUES; + device->vdev = rte_vdpa_register_device(&pci_dev->device, &nfp_vdpa_ops); if (device->vdev == NULL) { DRV_VDPA_LOG(ERR, "Failed to register device %s", pci_dev->name); diff --git a/drivers/vdpa/nfp/nfp_vdpa_core.c b/drivers/vdpa/nfp/nfp_vdpa_core.c new file mode 100644 index 0000000000..a7e15fa88a --- /dev/null +++ b/drivers/vdpa/nfp/nfp_vdpa_core.c @@ -0,0 +1,54 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright (c) 2023 Corigine, Inc. + * All rights reserved. + */ + +#include "nfp_vdpa_core.h" + +#include + +#include "nfp_vdpa_log.h" + +/* Available and used descs are in same order */ +#ifndef VIRTIO_F_IN_ORDER +#define VIRTIO_F_IN_ORDER 35 +#endif + +int +nfp_vdpa_hw_init(struct nfp_vdpa_hw *vdpa_hw, + struct rte_pci_device *pci_dev) +{ + uint32_t queue; + struct nfp_hw *hw; + uint8_t *notify_base; + + hw = &vdpa_hw->super; + hw->ctrl_bar = pci_dev->mem_resource[0].addr; + if (hw->ctrl_bar == NULL) { + DRV_CORE_LOG(ERR, "hw->ctrl_bar is NULL. BAR0 not configured."); + return -ENODEV; + } + + notify_base = hw->ctrl_bar + NFP_VDPA_NOTIFY_ADDR_BASE; + for (queue = 0; queue < NFP_VDPA_MAX_QUEUES; queue++) { + uint32_t idx = queue * 2; + + /* RX */ + vdpa_hw->notify_addr[idx] = notify_base; + notify_base += NFP_VDPA_NOTIFY_ADDR_INTERVAL; + /* TX */ + vdpa_hw->notify_addr[idx + 1] = notify_base; + notify_base += NFP_VDPA_NOTIFY_ADDR_INTERVAL; + + vdpa_hw->notify_region = queue; + DRV_CORE_LOG(DEBUG, "notify_addr[%d] at %p, notify_addr[%d] at %p", + idx, vdpa_hw->notify_addr[idx], + idx + 1, vdpa_hw->notify_addr[idx + 1]); + } + + vdpa_hw->features = (1ULL << VIRTIO_F_VERSION_1) | + (1ULL << VIRTIO_F_IN_ORDER) | + (1ULL << VHOST_USER_F_PROTOCOL_FEATURES); + + return 0; +} diff --git a/drivers/vdpa/nfp/nfp_vdpa_core.h b/drivers/vdpa/nfp/nfp_vdpa_core.h new file mode 100644 index 0000000000..c9403e0ea4 --- /dev/null +++ b/drivers/vdpa/nfp/nfp_vdpa_core.h @@ -0,0 +1,32 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright (c) 2023 Corigine, Inc. + * All rights reserved. + */ + +#ifndef __NFP_VDPA_CORE_H__ +#define __NFP_VDPA_CORE_H__ + +#include +#include +#include + +#define NFP_VDPA_MAX_QUEUES 1 + +#define NFP_VDPA_NOTIFY_ADDR_BASE 0x4000 +#define NFP_VDPA_NOTIFY_ADDR_INTERVAL 0x1000 + +struct nfp_vdpa_hw { + struct nfp_hw super; + + uint64_t features; + uint64_t req_features; + + uint8_t *notify_addr[NFP_VDPA_MAX_QUEUES * 2]; + + uint8_t mac_addr[RTE_ETHER_ADDR_LEN]; + uint8_t notify_region; +}; + +int nfp_vdpa_hw_init(struct nfp_vdpa_hw *vdpa_hw, struct rte_pci_device *dev); + +#endif /* __NFP_VDPA_CORE_H__ */ -- 2.39.1