From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id ABE1C429AE; Fri, 21 Apr 2023 16:54:27 +0200 (CEST) Received: from mails.dpdk.org (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 4C4AC4113C; Fri, 21 Apr 2023 16:54:27 +0200 (CEST) Received: from NAM10-DM6-obe.outbound.protection.outlook.com (mail-dm6nam10on2051.outbound.protection.outlook.com [40.107.93.51]) by mails.dpdk.org (Postfix) with ESMTP id 9570F410FB for ; Fri, 21 Apr 2023 16:54:25 +0200 (CEST) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=NQnSGjA3wP61ZiTSG/fqrtTo0ZKO5snSGHqix3jxVSTPwWl+7pHJA92AHl4CgWueoJY4fatbn+i7SLocbLMZdQcXt3jf/xdBlvvh1Xm0A+mZp8Y8wjc6ja9IECH0bWGpl/ujANGacRWYpxN5T0cq3xUOR8fXTVf+RmFXezTBBWGpQhHGqHkB4hI2PJYHHTKPEtySBeQ+ybGqKDHKQgUcy9VQ/+6MI5pJLq8CChCwixVfegkWdBpsIiEfjFsLV0b8ggGnzK9vjOtKAwcMcnu2j2eWeAmBxNohoyIBHqs6qav5iLU7Y2tGw21kg64cNRO8W16FCO/KAQAiB0vbtcwtOQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=pZO/pZAjoVUwgR/bSXZArls3XMHTWQkyefLQJt3M9fE=; b=HSSEEQ69Wwsce7vKPQVwcUVePwAlZP6+tFK/QAByVKoz4iSoGXDx374ea+A7N/vH257AdcIg7nuIssl8jYyw4o2o1pZNrhOluZcCfSHDSuA+0X6GrwbYP7/s2/t7tqARheBLhIDCW2VNktYFEGNPWNWYaCADZTrJRmAXMRwVn6+ua45YQDFdg1EexDtU1gtJNomE/0lNw8oP9hoMvB1A5goBZtaCWa56DuQv9wse2Y4x/s0hTAeECE+E17zYDmH+BQkoajT54hCykxLJe5L8N+Puqwy0p6q9RKDUP9MUUOMdUlKq/BCwju4AM2T3lrGYgKHw7gPZ3jFpD7w3SbCtqQ== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 165.204.84.17) smtp.rcpttodomain=dpdk.org smtp.mailfrom=amd.com; dmarc=pass (p=quarantine sp=quarantine pct=100) action=none header.from=amd.com; dkim=none (message not signed); arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amd.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=pZO/pZAjoVUwgR/bSXZArls3XMHTWQkyefLQJt3M9fE=; b=BEiYdMXGP9i3nFgbF4u5MMdTJqcCCx9CjICq93Sod3BA6hbWdQPO8RXo9MLn7XhRa3tof094CMIqKFFF0qs8ZhpZvKgo22k1+E1qaootf8rSszat7jaoWSBjtxUrjXLk211DOf/RRDYDNKA7Il6GuV6p2fQOp4vYEgSM1LWKVL0= Received: from MW4PR04CA0385.namprd04.prod.outlook.com (2603:10b6:303:81::30) by CH0PR12MB5186.namprd12.prod.outlook.com (2603:10b6:610:b9::19) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.6319.22; Fri, 21 Apr 2023 14:54:23 +0000 Received: from CO1NAM11FT040.eop-nam11.prod.protection.outlook.com (2603:10b6:303:81:cafe::ed) by MW4PR04CA0385.outlook.office365.com (2603:10b6:303:81::30) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.6319.25 via Frontend Transport; Fri, 21 Apr 2023 14:54:22 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 165.204.84.17) smtp.mailfrom=amd.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=amd.com; Received-SPF: Pass (protection.outlook.com: domain of amd.com designates 165.204.84.17 as permitted sender) receiver=protection.outlook.com; client-ip=165.204.84.17; helo=SATLEXMB03.amd.com; pr=C Received: from SATLEXMB03.amd.com (165.204.84.17) by CO1NAM11FT040.mail.protection.outlook.com (10.13.174.140) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.20.6319.27 via Frontend Transport; Fri, 21 Apr 2023 14:54:22 +0000 Received: from SATLEXMB08.amd.com (10.181.40.132) by SATLEXMB03.amd.com (10.181.40.144) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2375.34; Fri, 21 Apr 2023 09:54:21 -0500 Received: from SATLEXMB04.amd.com (10.181.40.145) by SATLEXMB08.amd.com (10.181.40.132) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2375.34; Fri, 21 Apr 2023 07:54:21 -0700 Received: from xhdipdslab41.xilinx.com (10.180.168.240) by SATLEXMB04.amd.com (10.181.40.145) with Microsoft SMTP Server id 15.1.2375.34 via Frontend Transport; Fri, 21 Apr 2023 09:54:19 -0500 From: Nipun Gupta To: , , , CC: , , Nipun Gupta Subject: [PATCH v3 0/5] Support AMD CDX bus, for FPGA based CDX devices. The CDX Date: Fri, 21 Apr 2023 20:24:01 +0530 Message-ID: <20230421145406.12831-1-nipun.gupta@amd.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20230124140746.594066-1-nipun.gupta@amd.com> References: <20230124140746.594066-1-nipun.gupta@amd.com> MIME-Version: 1.0 Content-Type: text/plain X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: CO1NAM11FT040:EE_|CH0PR12MB5186:EE_ X-MS-Office365-Filtering-Correlation-Id: 474a6286-f717-45ee-7713-08db42784b31 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: 2KpJU1HIEuiY6Cg2AXhLsEPXEVcMU7EzqiT1DKUhoqxB9PAkCOpbeSGJSQKTkHrVyqvCtyGF5YomRTel8HZY7sIBUdemN1XKvuD+1YorRLAwkScKvy/HBJqrTZsVgLc4T9m09UR2FbdXOsC/ac00GPBBFVeKwuIDemKi3zCb8fSSCznkzRwUp0ANx7/4KI+q9PSa648zWEDyMZTqXDjfAEMlnC4zasYBygvo0C7WGXssRqgVEQVuglrmNpXGmghOUyHLG6MlpNFz0AsbLv9dx0udz00Mgjdj+EXRrxD60GsAAgfOKHWU9N1s8+cPkVbhEEDrcgOH0jHZvM9Jf9q+dVMMl1kAr7M0qhgKVi6I5M3GxeMnDuh6WUQRyRhQh5ePXB8APu44CGrlkD412cSNLARkKmrRe1njMCnPVZZTrfEc7elto1BU19q58WiiBkHfRfIXGWTgpWwWE/qJ4/AnmIN4Fku4fshV044h4wgN1ijExazB8Bel21Rv4QaheUGLhLJMsWKFcQXi2I3P3mrAlISbMh44vbAIwUeDB7EZJSd87cEfAVSzKtLDLS11kF6cHM0OL78ykV0tlPeHhuJetDtb3j1xk5jMyizqQ1lmaJjijLrxaoBPhJjf2tAoN7OkgmNOpRWUnuggWLZkAPbM5gm/SavHcBSm/U9xlDGGKTxufcG6nevDtkMhuUZfx/gedTfuuZEb2Vnlngt7nNP/28svJ09a5mou2RDiJ5KBET3vazmvka1YXmRdRLHt7BRV0ZOnTRvZyN2b7QViygjH9A== X-Forefront-Antispam-Report: CIP:165.204.84.17; CTRY:US; LANG:en; SCL:1; SRV:; IPV:CAL; SFV:NSPM; H:SATLEXMB03.amd.com; PTR:InfoDomainNonexistent; CAT:NONE; SFS:(13230028)(4636009)(376002)(39860400002)(136003)(346002)(396003)(451199021)(40470700004)(36840700001)(46966006)(110136005)(54906003)(6636002)(83380400001)(478600001)(47076005)(36860700001)(2616005)(40480700001)(966005)(26005)(1076003)(6666004)(70206006)(70586007)(81166007)(41300700001)(356005)(4326008)(336012)(426003)(316002)(82740400003)(4743002)(186003)(5660300002)(44832011)(8676002)(2906002)(40460700003)(8936002)(86362001)(36756003)(82310400005)(36900700001); DIR:OUT; SFP:1101; X-OriginatorOrg: amd.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 21 Apr 2023 14:54:22.6536 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: 474a6286-f717-45ee-7713-08db42784b31 X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=3dd8961f-e488-4e60-8e11-a82d994e183d; Ip=[165.204.84.17]; Helo=[SATLEXMB03.amd.com] X-MS-Exchange-CrossTenant-AuthSource: CO1NAM11FT040.eop-nam11.prod.protection.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: CH0PR12MB5186 X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org It uses sysfs interface and the vfio-cdx driver to discover and initialize the CDX devices. The CDX bus and VFIO support is available at Xilinx open source tree: https://github.com/Xilinx/linux-xlnx (drivers/cdx/ and drivers/vfio/cdx) Linux CDX bus patches has been added into linux next: https://git.kernel.org/pub/scm/linux/kernel/git/next/linux-next.git/tree/drivers/cdx VFIO patches are also submitted in upstream: https://www.spinics.net/lists/kvm/msg310623.html CDX is a Hardware Architecture designed for AMD FPGA devices. It consists of mechanism for interaction between FPGA, Firmware and the APUs (Application CPUs). Firmware resides on RPU (Realtime CPUs) which interacts with the FPGA program manager and the APUs. The RPU provides memory-mapped interface (RPU if) which is used to communicate with APUs. VFIO CDX driver provides the CDX device resources like MMIO and interrupts to map to user-space. DPDK CDX bus uses sysfs interface and the vfio-cdx driver to discover and initialize the CDX devices for user-space applications. Changes v2->v3: - merged cdx bus compilation enablement in the first patch - fixed issue reported by check-git-log.sh - updated release notes - updated offset to uint64_t instead of off_t in cdx_map_resource Changes v1->v2: - Moved file rte_cdx_bus.h to internal bus_cdx_driver.h and added this file to deivce_cdx_headers - Moved cdx.h to private.h - Removed rte_ prefix from the static symbols in .c files. Changes RFC->v1: - Marked few API's as internal which were not required to be provided to user. Nipun Gupta (5): bus/cdx: introduce cdx bus bus/cdx: add DMA map and unmap support bus/cdx: add support for MSI bus/cdx: support plug unplug and dev iterator config/arm: add AMD CDX MAINTAINERS | 5 + config/arm/arm64_cdx_linux_gcc | 17 + config/arm/meson.build | 14 + doc/guides/rel_notes/release_23_07.rst | 6 + drivers/bus/cdx/bus_cdx_driver.h | 227 ++++++++ drivers/bus/cdx/cdx.c | 693 +++++++++++++++++++++++++ drivers/bus/cdx/cdx_logs.h | 37 ++ drivers/bus/cdx/cdx_vfio.c | 615 ++++++++++++++++++++++ drivers/bus/cdx/meson.build | 13 + drivers/bus/cdx/private.h | 49 ++ drivers/bus/cdx/version.map | 13 + drivers/bus/meson.build | 1 + lib/eal/common/eal_common_interrupts.c | 21 + lib/eal/common/eal_interrupts.h | 1 + lib/eal/include/rte_interrupts.h | 32 ++ lib/eal/version.map | 2 + 16 files changed, 1746 insertions(+) create mode 100644 config/arm/arm64_cdx_linux_gcc create mode 100644 drivers/bus/cdx/bus_cdx_driver.h create mode 100644 drivers/bus/cdx/cdx.c create mode 100644 drivers/bus/cdx/cdx_logs.h create mode 100644 drivers/bus/cdx/cdx_vfio.c create mode 100644 drivers/bus/cdx/meson.build create mode 100644 drivers/bus/cdx/private.h create mode 100644 drivers/bus/cdx/version.map -- 2.17.1