From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id BF6B642943; Fri, 14 Apr 2023 18:45:56 +0200 (CEST) Received: from mails.dpdk.org (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 9A0E841153; Fri, 14 Apr 2023 18:45:56 +0200 (CEST) Received: from NAM12-BN8-obe.outbound.protection.outlook.com (mail-bn8nam12on2059.outbound.protection.outlook.com [40.107.237.59]) by mails.dpdk.org (Postfix) with ESMTP id 6A19340144 for ; Fri, 14 Apr 2023 18:45:55 +0200 (CEST) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=BAs/5Y3YR3+7m8AEKCpG6ALcXvyI3FzMCHfHZsQp/O685OWn7ecg/EnITMAaFGaPvyTBsWvSoJ8iywJN7MOpCDP2isOsUAaUjfTshmCPZYsx259cfkV7G75g/XiCqg0d0BGZ8VhFAYKD1Zg44z++3qAqrJAnUBkWGQeAomonob4vJ3v7OHJIizHjQiBP5r915oxALWfP+HKjvqkeiJex4NKZOC3TJ3HZ4tvY2vCIFxAJ4q9v+xDtSp80wl7QAJtfO3cmWY5Ze745y4qtCArjiIH4VayJiEmfvJ/pDU8jmZIW/nuuVddfsPCjcgiQ92kLaaHT+WJP85IJP8jn6aBRSQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=FZcyKq6gaFRuaWB4HuYP9WqR6P8rSLJOF+eyImwRRZs=; b=fa3gu8sw2LmWaI6MU4UScQzATKRWHpACNGZIV7UFoaj14ygnIcn/0f7a7KYBsvnzTuAOxIyZ1pvk1FIAJ61qt1f9It1/Ed47onoA8HB50JOZsQ7L9DfGazfH963kq4yf5M0xYyL5OSjTaF3cApv4NlUmTkwh2vxJ6SW3qYVGUIX08OUqRkO3QqDSRX1VsT28XrqYckS3o0i2M/gUxOMtjfOSbwG6dzn4zbS28s7nRbzu+6aMTQs97u0ciLv6m0lEmfV21bYHBhCGIi/63wLoCpSHZ3kIcSqqj97CVzSNhCnfwhgF7CL3YJZl4SxdQdUDDjdo7Gyk2fQXNdGMUq0f9A== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=amd.com; dmarc=pass action=none header.from=amd.com; dkim=pass header.d=amd.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amd.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=FZcyKq6gaFRuaWB4HuYP9WqR6P8rSLJOF+eyImwRRZs=; b=q9t4QXsG/KEKaZP2sFv5IKeyq/jka0+NbA8vMhWP0QjOVn0sEaMpWlruCIcTXhxOcXVp3uV9UsfNVrn/7OER3QefXwXthD3EvorEsybCtIkLesFEXC2bE1Q1TQ9QMhvwYUGqjdWWrV2Gt/ebROKIvW4IB4bLElUPfbFeGsYaT18= Authentication-Results: dkim=none (message not signed) header.d=none;dmarc=none action=none header.from=amd.com; Received: from CH2PR12MB4294.namprd12.prod.outlook.com (2603:10b6:610:a9::11) by IA0PR12MB8228.namprd12.prod.outlook.com (2603:10b6:208:402::10) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.6277.43; Fri, 14 Apr 2023 16:45:53 +0000 Received: from CH2PR12MB4294.namprd12.prod.outlook.com ([fe80::5e2c:c0ed:88a6:a4c7]) by CH2PR12MB4294.namprd12.prod.outlook.com ([fe80::5e2c:c0ed:88a6:a4c7%7]) with mapi id 15.20.6298.030; Fri, 14 Apr 2023 16:45:53 +0000 Message-ID: <39f6afd7-69be-867f-eaa5-2ff35a795fb3@amd.com> Date: Fri, 14 Apr 2023 17:45:45 +0100 User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:102.0) Gecko/20100101 Thunderbird/102.10.0 Content-Language: en-US To: Nipun Gupta , dev@dpdk.org, thomas@monjalon.net, david.marchand@redhat.com Cc: harpreet.anand@amd.com, nikhil.agarwal@amd.com References: <20230124140746.594066-1-nipun.gupta@amd.com> <20230413132704.17586-1-nipun.gupta@amd.com> From: Ferruh Yigit Subject: Re: [PATCH v2 0/6] add support for CDX bus In-Reply-To: <20230413132704.17586-1-nipun.gupta@amd.com> Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 7bit X-ClientProxiedBy: LO2P265CA0355.GBRP265.PROD.OUTLOOK.COM (2603:10a6:600:d::31) To CH2PR12MB4294.namprd12.prod.outlook.com (2603:10b6:610:a9::11) MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: CH2PR12MB4294:EE_|IA0PR12MB8228:EE_ X-MS-Office365-Filtering-Correlation-Id: 2f20bada-4b48-403b-a58e-08db3d07b542 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: CId+w/rLqX3tV8yt/NtG99VgqacPJ2/w/NRBfoGL7sFbBxhzyoH8/ixqCbSnKKCAfFIRIr7UdOvSHEChONw6UBeW9lQ9iaKP6I9ZDq3bbL+asfNgaH4Q9mA5hg58FjHCEd0+sTRepIZAa+h7UWNhBleOzTksuldJ4vbSl+B6Pfa78HPGbWh7DDsze2txMcpie1WnWh0737tNxa1ynNqLFy32q7gCdH0ICdiLlGD2ueCIacSj6xvZ/f9dRQ8FE216oKmP99KAWMCT9cdMmgd3+5Ixa2KFngFawFS6jNaSFjpkADsHExeo6By6HlHBG4GDU5lqQroijeI8SRFsbBukpkl9h1dhestWMHICqt9omqkuygXgptbU+UiJ6o7WRCJGwL1tqyAVRr019632OhhwOrXv0FxDhH468fvV+6mvHOhIsl2AyaIgPWtw87l32OR2y3bAtfEVTAmfCG9SqcvoLWpsQ75TV7eUpDPadoVt4v0cWWJKXzrh7wkD1dyAnuOweoaRCaS1SxSEy+juMHEwkSHypI6j9dBP8wdd3AQei1c4SXMr5LqMnaD6E6lUbJCssKuBw/tGpWZ+qHFONtwOOGhlHFXiT7UVtYIp6leNzJh4l1jDIn0NYru4vewKS0xSJg6Cj25BX4AtdzGfeHisRaYBYwdnL33PuhrHvpEhtIDAOLjE7Va+xkesyaW+EIi/ X-Forefront-Antispam-Report: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:CH2PR12MB4294.namprd12.prod.outlook.com; PTR:; CAT:NONE; SFS:(13230028)(4636009)(346002)(136003)(396003)(366004)(39860400002)(376002)(451199021)(478600001)(8936002)(5660300002)(38100700002)(66476007)(66556008)(8676002)(66946007)(41300700001)(4326008)(316002)(31686004)(36756003)(6512007)(53546011)(6506007)(26005)(86362001)(2616005)(186003)(31696002)(83380400001)(44832011)(6666004)(2906002)(966005)(6486002)(45980500001)(43740500002); DIR:OUT; SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: =?utf-8?B?dTFES1dwUVRPcEpvYVZpeGc2RzVGY09HMTNyQTM0WmI0dU1QRFdrWE5WMU5z?= =?utf-8?B?Z3VUWXE5ejJQK3JmcmZvdFZ0ZnBCeG50WUJFdzRySVBqYUNaY3F5NkxDRjBK?= =?utf-8?B?NmFOQ1ZSd0RWQU42WW5IdTBwVGE2OGRVYWwwTytoT3dzWE9wVUxLbjlmemVu?= =?utf-8?B?Sjk1OVhqWHpiNEUzbkVFWnBpdkw2V1VwWEMxTGRZTmVUb1J5RVBhK0FENUNa?= =?utf-8?B?d2lKNVcwZkFaTTlIdEcrazlsNWZyTndNM1lJMXN4UVdMZ3p1Y2ZLa2JqT01m?= =?utf-8?B?d0pCMEdaN1NTNWVyRWZwSDhlK0Y1NDF4QVJlUERtaEROM3lKRjVoYUZzU2lt?= =?utf-8?B?VGExbTlTeTV4c1JqM1ZrSHFTYlcxa2NFOG91UmJZV29mb1hwMDBGdmlqZXYw?= =?utf-8?B?ZzVXZUZxNGh3RjRFRjVFSTlBTzJBeSsxZnZIdXhYc3pFZTlQTXdmWTREMUZC?= =?utf-8?B?Yjg5VU5YeGh1eFFUdnNTZWNPYkp3VlBLVzhwNTdEeVp4UXlpT2VGS0FUc25I?= =?utf-8?B?eE9jQnZxQUdzSG9QUUxlVGV3RDk1eGw1bStXU0VSQlBzWXo2RDVBdjlyOXRG?= =?utf-8?B?SWxJbWloczIrV3ZWS0gyMkx4U1RWTWlBclZxeUp3UHNVZFQyZmtkZzBJdVl2?= =?utf-8?B?ZUNQbFc1dTFuN2lOMTlDaTFSdDJXRXp4TGlxN0F2MXpHcmZFV1JkZk5rT1Z3?= =?utf-8?B?WjJCaWlkbVBQTzhCdExvV1BkZDJjZzZsSW9wZ0Q1bjZBK3g0QzJPbnJJa2gz?= =?utf-8?B?bjhFbDFhME8rNGMxc3p2ci9LVkY0dTQ1d2FERDJJT2U0M0dhaGgrSTZPSHhw?= =?utf-8?B?bzIwNzMyYzZqNUIyM3FOMmFrRFQveUJVTjB1aWlLUGE1d2l6VUYzU3ljZ2du?= =?utf-8?B?R21idnFLN1ZwbGRkMDlBNWZydnE2b0VCZzhnNUFxQ1QzcDYxWUg0dlJxeXNE?= =?utf-8?B?VHJXbTB2VlBENkVYOWxWdDc0L3hSRWtoOE1rSTduRzRjT2FablJOYkFzeTI2?= =?utf-8?B?bndPTEhVZ2Y2YW50R2hpRnd1WWdLNVpGeEhMTEdtcDR0TzZrTjgwSnpJRDk3?= =?utf-8?B?SnFaUUJsVHZTcFR2djZmV2k4ampjdSsxYW45ek9JRng0UVJLYVpmdmZTcFky?= =?utf-8?B?SFZPNGIwZUxrdTg3T20yTzRFSzRibmY1VEZiY0JmWmE5UDhEendtakhEdlBN?= =?utf-8?B?NEVRQ002Z0VsT2VoZEYzUitnMHlvM0k2bk53eUhyMjFDWitHaWZHK0JLL2cw?= =?utf-8?B?RExJNG1pcnNYeldValQwZHVqQlg5NnROaWlZaUs1MXZidWlCTDZNR1AyMndr?= =?utf-8?B?N2ZXTlh3U0U2RW5NZ3NJTTVZVnYvaVpMMWY4NzJTOGQ4SjE0akJNZTY4U3B3?= =?utf-8?B?WmlGQWl2a2NGSUxoQi9JN09nVEdxU3ptRDEyNTNEUXl1Q3p2cUhvUTQvZjBt?= =?utf-8?B?TDRlQUJlazFENmhxaURHZDVLRlJ3WkpxdmtzbWlSRnlicHM2ZUpsSnQzMVlx?= =?utf-8?B?RGdNTlppam5wT25zcTl6WWdxZHFhT1BpbThBaVV0VE9lY2pDMUJQZEJIUW4x?= =?utf-8?B?SzFrN1E4N1QrejdQL1dJYlorcW9UcEo3Q1dET0JPdHZEOTdIMWwzU1hwdDVk?= =?utf-8?B?MHMrMFhGY1NhUENteXl1Tkppc3RvM1Zyd2FoUTlsUElLQUZxUjdZQmlGelFv?= =?utf-8?B?cnFOM09kaUp1bHl4d0RPZXhxUU51UEJEdmdqT01Ja2NqbTA5anZvczZNME5p?= =?utf-8?B?M3RUdlpudmNGV0pKaC9NVmpzWFFsWVp3RkJKZm1uU0lodkc2bllvemlWYjNQ?= =?utf-8?B?QTVMbHdHZWJqV095UG9UVEduU2dYc0FFMDMwSXhtazFma2dBdUQ2RFM5aCtD?= =?utf-8?B?SnRDTWk0K0M4bXRNc1VGcnNidzNxQkhnWXFUL2xoSHhwWTNTSEZhWW9FR3pQ?= =?utf-8?B?TDVITjF6dDNUZHF0Q2htSHg1U0VLOFY4Ly82NldleTA3aFdwNVYvMWZtSHZv?= =?utf-8?B?SkxKZmZJbXhWVjJZTGZ5Skw1NStlOUsvUnIrZFRHbUtHcStFMXVDNUU5ZFF4?= =?utf-8?B?RVl1WldhcXJnV2hpZXFVZ2wvektmcTVFN25kVUR2SU13dzBzUkJoZmxLM2RS?= =?utf-8?Q?XoNqPQru7TI0foCLsze6EsNDj?= X-OriginatorOrg: amd.com X-MS-Exchange-CrossTenant-Network-Message-Id: 2f20bada-4b48-403b-a58e-08db3d07b542 X-MS-Exchange-CrossTenant-AuthSource: CH2PR12MB4294.namprd12.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 14 Apr 2023 16:45:53.4909 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: rilWypEdajlZIkIRCV6U9hORiz5aCmeGxy1sIGxX84/eXKv6OqOY1b72imaZOixx X-MS-Exchange-Transport-CrossTenantHeadersStamped: IA0PR12MB8228 X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org On 4/13/2023 2:26 PM, Nipun Gupta wrote: > Support AMD CDX bus, for FPGA based CDX devices. The CDX > devices are memory mapped on system bus for embedded CPUs. > > It uses sysfs interface and the vfio-cdx driver to discover > and initialize the CDX devices. > > The patches are intended for DPDK 23.07 release, and have been sent > as an RFC as patches are yet to be merged in Linux. > not RFC anymore > The CDX bus and VFIO support is available at Xilinx open source tree: > https://github.com/Xilinx/linux-xlnx (drivers/cdx/ and drivers/vfio/cdx) > > Linux CDX bus patches has been added into linux next: > https://git.kernel.org/pub/scm/linux/kernel/git/next/linux-next.git/tree/drivers/cdx > > VFIO patches are also submitted in upstream: > https://www.spinics.net/lists/kvm/msg310623.html > > CDX is a Hardware Architecture designed for AMD FPGA devices. It > consists of mechanism for interaction between FPGA, Firmware and > the APUs (Application CPUs). > Firmware resides on RPU (Realtime CPUs) which interacts with > the FPGA program manager and the APUs. The RPU provides memory-mapped > interface (RPU if) which is used to communicate with APUs. > > VFIO CDX driver provides the CDX device resources like MMIO and interrupts > to map to user-space. DPDK CDX bus uses sysfs interface and the vfio-cdx > driver to discover and initialize the CDX devices for user-space > applications. > Overall looks good to me, there are a few warnings by `check-git-log.sh`, can you please check? Also what do you think to add a release notes update for new bus? > Changes v1->v2: > - Moved file rte_cdx_bus.h to internal bus_cdx_driver.h > and added this file to deivce_cdx_headers > - Moved cdx.h to private.h > - Removed rte_ prefix from the static symbols in .c files. > > Changes RFC->v1: > - Marked few API's as internal which were not required > to be provided to user. > > Nipun Gupta (6): > bus/cdx: introduce cdx bus > bus/cdx: add dma map and unmap support > bus/cdx: add support for MSI > bus/cdx: support plug unplug and dev iterator > bus: enable cdx bus > config/arm: add AMD CDX > > MAINTAINERS | 5 + > config/arm/arm64_cdx_linux_gcc | 17 + > config/arm/meson.build | 14 + > drivers/bus/cdx/bus_cdx_driver.h | 227 ++++++++ > drivers/bus/cdx/cdx.c | 694 +++++++++++++++++++++++++ > drivers/bus/cdx/cdx_logs.h | 37 ++ > drivers/bus/cdx/cdx_vfio.c | 619 ++++++++++++++++++++++ > drivers/bus/cdx/meson.build | 12 + > drivers/bus/cdx/private.h | 49 ++ > drivers/bus/cdx/version.map | 13 + > drivers/bus/meson.build | 1 + > lib/eal/common/eal_common_interrupts.c | 21 + > lib/eal/common/eal_interrupts.h | 1 + > lib/eal/include/rte_interrupts.h | 32 ++ > lib/eal/version.map | 2 + > 15 files changed, 1744 insertions(+) > create mode 100644 config/arm/arm64_cdx_linux_gcc > create mode 100644 drivers/bus/cdx/bus_cdx_driver.h > create mode 100644 drivers/bus/cdx/cdx.c > create mode 100644 drivers/bus/cdx/cdx_logs.h > create mode 100644 drivers/bus/cdx/cdx_vfio.c > create mode 100644 drivers/bus/cdx/meson.build > create mode 100644 drivers/bus/cdx/private.h > create mode 100644 drivers/bus/cdx/version.map >