From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 25576431EB; Tue, 24 Oct 2023 04:32:43 +0200 (CEST) Received: from mails.dpdk.org (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id BB89742DD7; Tue, 24 Oct 2023 04:30:24 +0200 (CEST) Received: from NAM11-DM6-obe.outbound.protection.outlook.com (mail-dm6nam11on2102.outbound.protection.outlook.com [40.107.223.102]) by mails.dpdk.org (Postfix) with ESMTP id B9B7442D96 for ; Tue, 24 Oct 2023 04:30:18 +0200 (CEST) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=kCNz/woAamIc3qzby2aI/nRDvIJg3f1O8raB5P8NnOXvV+tr7wB5kEVtLWCM3BdUE/59EJjZzOLdRKWhvqFwI1G9ulXGlLIsJndv4rK7ZL0Zn2pc7MbC0NJ4tvWHxRHkWt/jqpVCV5L0ULj2s512quDjB+AEtw0o6d9DVxqjWn7OH/Qu45k+JmK+4zUyjJUKyTUzYc/yoHmqbqGqa/47FajowH7Gv1yNt/umT0ffOlXd0jDqzsoRe0ZpXkDhh32H7ECdGZl+IOLKfhBkCMdvSxnxmTg67PoC/CEvU+8tWAHbmxVnX8K7cZumgBTwlod8FNWy3hlZF5CX7IA3jtw2NQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=ueD306333BC6aL/GrzldxBPxOnMsxU/MlgBa+o8r1+8=; b=U1MK/2djTn9uGtZxmJAkkHjpD6AZqA36YPcjq2wzPtIOGH/DlsSFpyYXsRRT/5Z7n1JcZwXJfi9DcAYv8EJqEQ90PuV32f2RI3tYPv6zGLQVm53NNv4h8HDbdh64Cq4p9+sQrmX0IQmnnYv2/zoo6yxF2Z2+pd9eZqxu5QEg0PAJwr9pxzQ62QOjG7PTJYU8OhuWWqXwUxaXAxFk7SkeCPP+7tznxx2CMVF9LCd9TWVUyzZPF5swgIwn+FrFgm72B39nxDdBkok87XYj6M3cR0wnuOgMM6CL501svN4DWgiZc9DhI573rgvJe2WIycCASTawwXLkmqAAfnwSmOPkFg== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=corigine.com; dmarc=pass action=none header.from=corigine.com; dkim=pass header.d=corigine.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=corigine.onmicrosoft.com; s=selector2-corigine-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=ueD306333BC6aL/GrzldxBPxOnMsxU/MlgBa+o8r1+8=; b=BVpRKKPNEdRIzV2hUTXmcFffG5mAWrgNc1vnKLEJAQyO1De0zSyUiBhzXuCHK0mYpyt/qnr8NK07c4ZvBsQUfE29uzWB/tmfQhUTQmP9MqH14/ke4cMx+pfchYgtRik5DoEqGVKcVTcfMswibMO5H8fBBlr31JL46mFjkVlFCx4= Authentication-Results: dkim=none (message not signed) header.d=none;dmarc=none action=none header.from=corigine.com; Received: from SJ0PR13MB5545.namprd13.prod.outlook.com (2603:10b6:a03:424::5) by PH8PR13MB6182.namprd13.prod.outlook.com (2603:10b6:510:259::13) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.6907.21; Tue, 24 Oct 2023 02:30:17 +0000 Received: from SJ0PR13MB5545.namprd13.prod.outlook.com ([fe80::142e:d47c:7320:8a9d]) by SJ0PR13MB5545.namprd13.prod.outlook.com ([fe80::142e:d47c:7320:8a9d%4]) with mapi id 15.20.6907.021; Tue, 24 Oct 2023 02:30:17 +0000 From: Chaoyong He To: dev@dpdk.org Cc: oss-drivers@corigine.com, Chaoyong He , Shujing Dong , Long Wu , Peng Zhang Subject: [PATCH v2 21/25] vdpa/nfp: add the hardware init logic Date: Tue, 24 Oct 2023 10:28:45 +0800 Message-Id: <20231024022849.3092506-22-chaoyong.he@corigine.com> X-Mailer: git-send-email 2.39.1 In-Reply-To: <20231024022849.3092506-1-chaoyong.he@corigine.com> References: <20231017054545.1692509-1-chaoyong.he@corigine.com> <20231024022849.3092506-1-chaoyong.he@corigine.com> Content-Transfer-Encoding: 8bit Content-Type: text/plain X-ClientProxiedBy: BYAPR04CA0015.namprd04.prod.outlook.com (2603:10b6:a03:40::28) To SJ0PR13MB5545.namprd13.prod.outlook.com (2603:10b6:a03:424::5) MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: SJ0PR13MB5545:EE_|PH8PR13MB6182:EE_ X-MS-Office365-Filtering-Correlation-Id: c3ac0de1-7dd7-4c43-2bf2-08dbd43928eb X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: YDlcE0Ac0s/8t/TxNeiq//WiyFO/ThhfRo2Lfx9W6cmaJNzLs5eSuU/Do9+rcX87EZMGgH66iCVxImq4Juqha34lwkQJF6vk2eoVLYZwWIWuzdEv5RicxTbnDaBZSqeHc5W/n4RREBjWJSCD+6lHDSepm91VsMV2ylyx3ZmNrnSWhT4qvymkXB9E19Ffj6TOUD8dOkiyVam2ouB+Dj0g0ROzxBaaSGJDi5v58mAV3rE5KsnXfmbLYso/gANYvDQxgP+YzghDVhq774XEva9EFD53CTeCv4LGgiYCFQYYd1AACBeb/91R+i4Wm63YukLF5Z2fnQFJZKvbgHcTHDb859dqzODIHV3c6hKQqMup7jXdT2ZTEbBz6+MdyURPowoI1FWKp1afciq2jDSaXZIAp0BCI6z/T5Gtg415sB6/Q4ijVKnkshAixi4vwCCZQx0qEeLKVrxlf27VBvX/bmi6RCJF6VWc6Y7nA0z+XXCQMqHHItf2L/QrZwIF1HcphOLJ7xrIryqIaapMV0fnVUaaV85QZLkZO4w3HxYcpdgLUjFsVFHQdGHwWbFD+JYIfyJyWcGJvy53u0OJkIFXF9rmLUI+KBV67Ta7rpEn2Wqye4MRccnPqOMRDPY2qirkycQy/E6EaJGQiJCPN4gCnJVVdb4G8+nc+KgrxZ4RfgweWMg= X-Forefront-Antispam-Report: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:SJ0PR13MB5545.namprd13.prod.outlook.com; PTR:; CAT:NONE; SFS:(13230031)(366004)(396003)(376002)(39830400003)(346002)(136003)(230922051799003)(1800799009)(64100799003)(186009)(451199024)(86362001)(66946007)(316002)(2906002)(54906003)(66476007)(6916009)(66556008)(5660300002)(8936002)(44832011)(8676002)(4326008)(36756003)(41300700001)(2616005)(38100700002)(478600001)(1076003)(52116002)(6506007)(107886003)(6666004)(38350700005)(6512007)(26005)(83380400001)(6486002); DIR:OUT; SFP:1102; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: =?us-ascii?Q?UShskndJgAsWe4Xw5vWndfA1f0DFdj6g8aBM+4AZO3lOho+j8YbJDN9ngIjx?= =?us-ascii?Q?vxHFOJsNefqBQsnhwx/elsAFYxsBl+ZxNaNrU3P6Gg9kg99uRfuQgLQWQ4Ge?= =?us-ascii?Q?lcyeIxkbmKy74DU+GM9bVZE0jbg/uPBCCu3AhIvuPix9jL5baXKSuaBYsQ2r?= =?us-ascii?Q?vCfRum5tmt7O0ZC/Ngysu4N+sJswS7WYdy/XLylPxPeohqpYbjexX3wnHs8e?= =?us-ascii?Q?L0mWTogE0dGaQoBKMOt6I5DxAlpLw59WoM9xWW6+V42ydTbidSOwSD3WhiU+?= =?us-ascii?Q?cGabanvTo4+kGD6bf6AkNmIxvg5QKOfawjbfNk6+4C4IQNhj6PZvME1Wj8bN?= =?us-ascii?Q?4mGhaZ8SJnGQGXR9XWeCnWOY8mor+lu3DuHi5kHbnU3E/+mfNIAH4+tyoZps?= =?us-ascii?Q?m/FtT/ya1F/+0zmBgNcSsm4LURzlPOk7Ie/GHL/XXIfMiISVAz0IB7+kbKjo?= =?us-ascii?Q?ui69ZA1buY+RKHIVaryxDXpmQCNhA78KTMTO2BS5SGBz88mBGSZlqvsUrfsX?= =?us-ascii?Q?3jeq0UPFlK9xk5QO8ZmGg0sFH+wuE2HO3eeinPjYM+WssILG2dN96ttVAPQN?= =?us-ascii?Q?VXs8MFjJbTT4W/0oCbEzSuA0sCOvJWpOcYecIbE7jA+mo9aOguS2npY6K58h?= =?us-ascii?Q?mrQM3eoLNjHwdCSvxFgx/FjwXCJg/ox56+5/j8P7DFKvX1oN0jx3YnFmncPU?= =?us-ascii?Q?RTGPm+ci33mN56t44RZahdi2I7E0qIgKUMDwSd804B2KNQkTbH7uj6N2Y1Qd?= =?us-ascii?Q?vWRQTlEw8Cs5llsBHrnzGkGwphHee0FD0nmjzPphKj7dLxdEtn+hhyZtKsQw?= =?us-ascii?Q?AhUbAgL5PIz3OnL66q0N6xBBfGpKUmCN9jQIpYe9uYSFA+BDJXrVXSU2ikvx?= =?us-ascii?Q?c7fi0RNzeQ2+JBnOsN7gtR5myLAKbWVjNQ2aTLSXc10bHBsbU8meLcNCIbVF?= =?us-ascii?Q?FKLb+sG4oz7oED6Ia4Y1MTb6ysVI6052SgLGQ1VHMMxbaA6Ojgq0AbEAmhqs?= =?us-ascii?Q?2ajkAUqCLsAWRcWTOb8iffZa9jgWEBzzsj8JaCNozeWTF5RJavtz2eRgrPpI?= =?us-ascii?Q?HVqK0kBKSw0PJbQbDupZ2fDUT4OgaEY7F5RovldLZwZT9joVp26s7jC2D0g/?= =?us-ascii?Q?2JgOscCStOUka2XvTmjsQCcizyWjCzfLIsg1xsixgtgiFXr+s+R8pdM2fzr4?= =?us-ascii?Q?iADEDhFehc0iyxjI/4nmaIDH8ZYdL+jUtQfI48NKWuO80czeHFATQdxrKvt3?= =?us-ascii?Q?E5oV0xcB3o0cwcx/Vw7kslXRjtyPJnA7B5SToVzcTGwj5M8jG2ylwv/5o1XS?= =?us-ascii?Q?tt6OvGKBNtTT5B6jkw9YVZCXkicpqaxXrgYI8JKGbmpQVMW41PPvH9r82zSb?= =?us-ascii?Q?PUBqmHxqI67adOF24sbXj10Y9IhO1ya2y/NAV72UiQYIz+XbVgYTx/qMwDu2?= =?us-ascii?Q?AKX4I8EbbgVOPKyXVidZS4vtGncOn06weSnPsjG7r7SJt76KSpNWIa/bAYLs?= =?us-ascii?Q?cSzd8QDA6tP0uh6nGCzkxrajSS+IxvdMOzj4wdk7kvL1Woq6I9408+B6TAlu?= =?us-ascii?Q?AjoUTIob361MWpI+obSFBoyJCoc2+7ZCES0Wpaq4Rl/GE8+dMmbnM0yXWJ5H?= =?us-ascii?Q?eQ=3D=3D?= X-OriginatorOrg: corigine.com X-MS-Exchange-CrossTenant-Network-Message-Id: c3ac0de1-7dd7-4c43-2bf2-08dbd43928eb X-MS-Exchange-CrossTenant-AuthSource: SJ0PR13MB5545.namprd13.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 24 Oct 2023 02:30:17.0832 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: fe128f2c-073b-4c20-818e-7246a585940c X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: l5bfy0aJayBWc8YML4pGfvckq4JlcKbQaVCgoO5R1k/+KcnMXSPxQ9oXq8lcYJgbnda2cx7J14B9dYiAD6vf36rGfQZLQ0oZf2FgCv9JmKo= X-MS-Exchange-Transport-CrossTenantHeadersStamped: PH8PR13MB6182 X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Add the hardware initialize logic. Signed-off-by: Chaoyong He Signed-off-by: Shujing Dong Reviewed-by: Long Wu Reviewed-by: Peng Zhang --- drivers/vdpa/nfp/meson.build | 1 + drivers/vdpa/nfp/nfp_vdpa.c | 10 ++++++ drivers/vdpa/nfp/nfp_vdpa_core.c | 54 ++++++++++++++++++++++++++++++++ drivers/vdpa/nfp/nfp_vdpa_core.h | 32 +++++++++++++++++++ 4 files changed, 97 insertions(+) create mode 100644 drivers/vdpa/nfp/nfp_vdpa_core.c create mode 100644 drivers/vdpa/nfp/nfp_vdpa_core.h diff --git a/drivers/vdpa/nfp/meson.build b/drivers/vdpa/nfp/meson.build index e1cc244578..27eb1a6276 100644 --- a/drivers/vdpa/nfp/meson.build +++ b/drivers/vdpa/nfp/meson.build @@ -13,6 +13,7 @@ endif sources = files( 'nfp_vdpa.c', + 'nfp_vdpa_core.c', 'nfp_vdpa_log.c', ) diff --git a/drivers/vdpa/nfp/nfp_vdpa.c b/drivers/vdpa/nfp/nfp_vdpa.c index 824b0d681a..00d8f7e007 100644 --- a/drivers/vdpa/nfp/nfp_vdpa.c +++ b/drivers/vdpa/nfp/nfp_vdpa.c @@ -10,6 +10,7 @@ #include #include +#include "nfp_vdpa_core.h" #include "nfp_vdpa_log.h" #define NFP_VDPA_DRIVER_NAME nfp_vdpa @@ -17,11 +18,14 @@ struct nfp_vdpa_dev { struct rte_pci_device *pci_dev; struct rte_vdpa_device *vdev; + struct nfp_vdpa_hw hw; int vfio_container_fd; int vfio_group_fd; int vfio_dev_fd; int iommu_group; + + uint16_t max_queues; }; struct nfp_vdpa_dev_node { @@ -135,6 +139,12 @@ nfp_vdpa_pci_probe(struct rte_pci_device *pci_dev) if (ret != 0) goto free_device; + ret = nfp_vdpa_hw_init(&device->hw, pci_dev); + if (ret != 0) + goto vfio_teardown; + + device->max_queues = NFP_VDPA_MAX_QUEUES; + device->vdev = rte_vdpa_register_device(&pci_dev->device, &nfp_vdpa_ops); if (device->vdev == NULL) { DRV_VDPA_LOG(ERR, "Failed to register device %s", pci_dev->name); diff --git a/drivers/vdpa/nfp/nfp_vdpa_core.c b/drivers/vdpa/nfp/nfp_vdpa_core.c new file mode 100644 index 0000000000..a7e15fa88a --- /dev/null +++ b/drivers/vdpa/nfp/nfp_vdpa_core.c @@ -0,0 +1,54 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright (c) 2023 Corigine, Inc. + * All rights reserved. + */ + +#include "nfp_vdpa_core.h" + +#include + +#include "nfp_vdpa_log.h" + +/* Available and used descs are in same order */ +#ifndef VIRTIO_F_IN_ORDER +#define VIRTIO_F_IN_ORDER 35 +#endif + +int +nfp_vdpa_hw_init(struct nfp_vdpa_hw *vdpa_hw, + struct rte_pci_device *pci_dev) +{ + uint32_t queue; + struct nfp_hw *hw; + uint8_t *notify_base; + + hw = &vdpa_hw->super; + hw->ctrl_bar = pci_dev->mem_resource[0].addr; + if (hw->ctrl_bar == NULL) { + DRV_CORE_LOG(ERR, "hw->ctrl_bar is NULL. BAR0 not configured."); + return -ENODEV; + } + + notify_base = hw->ctrl_bar + NFP_VDPA_NOTIFY_ADDR_BASE; + for (queue = 0; queue < NFP_VDPA_MAX_QUEUES; queue++) { + uint32_t idx = queue * 2; + + /* RX */ + vdpa_hw->notify_addr[idx] = notify_base; + notify_base += NFP_VDPA_NOTIFY_ADDR_INTERVAL; + /* TX */ + vdpa_hw->notify_addr[idx + 1] = notify_base; + notify_base += NFP_VDPA_NOTIFY_ADDR_INTERVAL; + + vdpa_hw->notify_region = queue; + DRV_CORE_LOG(DEBUG, "notify_addr[%d] at %p, notify_addr[%d] at %p", + idx, vdpa_hw->notify_addr[idx], + idx + 1, vdpa_hw->notify_addr[idx + 1]); + } + + vdpa_hw->features = (1ULL << VIRTIO_F_VERSION_1) | + (1ULL << VIRTIO_F_IN_ORDER) | + (1ULL << VHOST_USER_F_PROTOCOL_FEATURES); + + return 0; +} diff --git a/drivers/vdpa/nfp/nfp_vdpa_core.h b/drivers/vdpa/nfp/nfp_vdpa_core.h new file mode 100644 index 0000000000..c9403e0ea4 --- /dev/null +++ b/drivers/vdpa/nfp/nfp_vdpa_core.h @@ -0,0 +1,32 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright (c) 2023 Corigine, Inc. + * All rights reserved. + */ + +#ifndef __NFP_VDPA_CORE_H__ +#define __NFP_VDPA_CORE_H__ + +#include +#include +#include + +#define NFP_VDPA_MAX_QUEUES 1 + +#define NFP_VDPA_NOTIFY_ADDR_BASE 0x4000 +#define NFP_VDPA_NOTIFY_ADDR_INTERVAL 0x1000 + +struct nfp_vdpa_hw { + struct nfp_hw super; + + uint64_t features; + uint64_t req_features; + + uint8_t *notify_addr[NFP_VDPA_MAX_QUEUES * 2]; + + uint8_t mac_addr[RTE_ETHER_ADDR_LEN]; + uint8_t notify_region; +}; + +int nfp_vdpa_hw_init(struct nfp_vdpa_hw *vdpa_hw, struct rte_pci_device *dev); + +#endif /* __NFP_VDPA_CORE_H__ */ -- 2.39.1