From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id AE9F5A0542; Mon, 29 Aug 2022 10:52:02 +0200 (CEST) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 89D8D4069D; Mon, 29 Aug 2022 10:52:02 +0200 (CEST) Received: from NAM02-DM3-obe.outbound.protection.outlook.com (mail-dm3nam02on2069.outbound.protection.outlook.com [40.107.95.69]) by mails.dpdk.org (Postfix) with ESMTP id 2D1A540694 for ; Mon, 29 Aug 2022 10:52:01 +0200 (CEST) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=nF6cpxy2JfbD0KG59rzs5bIFKqVWN/dJXhRIa0gKOQLouUQyuRoK5Ffqak1zVGQ27pteNfysM+yCIz0i7ymGRxBI0Ly/oJy68BFKaJTmK5kyE+7yC4xZhH3NGt0o2BWSv8nwycwhttxhh8p6OguZwwJum8anDNgzj6kNwJGQmFCtY/gfrNlGOlnLWs0OUHCBhQg/PmLlggy/oA1yfiS2LaWOq8gZErVxyMCp0vQl35Xil29B4fFg2D6PCGCo/BLiKWH3XX2pVvTx5ORN8uAcZ4QmOHfZVc7V6VBp+xdoeP5hz7nwI9lm+UrN+SERhkhgL+2Ly/8iRRU/1sqOohij0A== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=QJ4XSj47DBo4lNG3VHIC7AsQh0mXPWbnbn6/BD/pXjs=; b=lRp3yLNfPfFDmro0N+P6jZd4ha/rlXVQpuPAXCidw+cFHQVhpnG9mtxm2Qnl4xiJyFBfURINoBMtnNmT2IvcvEH6NUjMg4Qu2mdxomJbOZK7uvJYvQJjMZHDfGdHAgTK5E+ig1jSVLEKjIaLTV49kU4m26Cc12zSR6oJ+N92EXpvs2IHMZXzlec0WlKLsyK/ZduMK9P26/W5WCIYZFYV7sWgd3FxndExFYOiv5dRtFEffR/mUBbsRDLRWO9Te/jMq6Ga3CFgEJ7ai4LvnlEvwhwthRrqkeB9QP06KnQhl1p+YV1bYVOSjmOHHJRJxdN7//lcinqoLGfMB8sxeI4SfA== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 149.199.80.198) smtp.rcpttodomain=monjalon.net smtp.mailfrom=xilinx.com; dmarc=pass (p=none sp=none pct=100) action=none header.from=xilinx.com; dkim=none (message not signed); arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=xilinx.onmicrosoft.com; s=selector2-xilinx-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=QJ4XSj47DBo4lNG3VHIC7AsQh0mXPWbnbn6/BD/pXjs=; b=DX+YOkbLP3S0jQFawEXltZ9EwLjaLt10FyUxLOwMcoGaCb/nbP5j9PrjFLh31qacnKBgtTqNFxrhan91rd2+WsvlQCm17AxRGSOf/1Wa00H3GRMEDQkbXWYVfR8mApUmFh+xka5qWFgJxmG821XJmgc1j8njDMB46vNNYVmBQeg= Received: from DM6PR02CA0118.namprd02.prod.outlook.com (2603:10b6:5:1b4::20) by SJ0PR02MB8546.namprd02.prod.outlook.com (2603:10b6:a03:3fc::8) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5566.15; Mon, 29 Aug 2022 08:51:58 +0000 Received: from DM3NAM02FT010.eop-nam02.prod.protection.outlook.com (2603:10b6:5:1b4:cafe::ba) by DM6PR02CA0118.outlook.office365.com (2603:10b6:5:1b4::20) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5566.15 via Frontend Transport; Mon, 29 Aug 2022 08:51:58 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 149.199.80.198) smtp.mailfrom=xilinx.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=xilinx.com; Received-SPF: Pass (protection.outlook.com: domain of xilinx.com designates 149.199.80.198 as permitted sender) receiver=protection.outlook.com; client-ip=149.199.80.198; helo=xir-pvapexch02.xlnx.xilinx.com; pr=C Received: from xir-pvapexch02.xlnx.xilinx.com (149.199.80.198) by DM3NAM02FT010.mail.protection.outlook.com (10.13.5.124) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.20.5566.15 via Frontend Transport; Mon, 29 Aug 2022 08:51:58 +0000 Received: from xir-pvapexch02.xlnx.xilinx.com (172.21.17.17) by xir-pvapexch02.xlnx.xilinx.com (172.21.17.17) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2375.24; Mon, 29 Aug 2022 09:51:44 +0100 Received: from smtp.xilinx.com (172.21.105.198) by xir-pvapexch02.xlnx.xilinx.com (172.21.17.17) with Microsoft SMTP Server id 15.1.2375.24 via Frontend Transport; Mon, 29 Aug 2022 09:51:44 +0100 Envelope-to: thomas@monjalon.net, longli@microsoft.com, david.marchand@redhat.com, dev@dpdk.org, sharmaajay@microsoft.com, sthemmin@microsoft.com, viacheslavo@nvidia.com, matan@nvidia.com Received: from [10.71.194.74] (port=51992) by smtp.xilinx.com with esmtp (Exim 4.90) (envelope-from ) id 1oSaUi-0003z0-GF; Mon, 29 Aug 2022 09:51:44 +0100 Message-ID: <5e010c8f-2bb9-51a3-7e99-5eee27236228@xilinx.com> Date: Mon, 29 Aug 2022 09:51:44 +0100 MIME-Version: 1.0 User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:102.0) Gecko/20100101 Thunderbird/102.2.0 Subject: Re: [Patch v4 01/17] net/mana: add basic driver, build environment and doc Content-Language: en-US To: Thomas Monjalon , Long Li CC: David Marchand , "dev@dpdk.org" , Ajay Sharma , Stephen Hemminger , Viacheslav Ovsiienko , Matan Azrad References: <1657324171-31369-1-git-send-email-longli@linuxonhyperv.com> <0443d4db-0fe3-60d6-b62f-14976965f1a7@xilinx.com> <1857175.eGJsNajkDb@thomas> From: Ferruh Yigit In-Reply-To: <1857175.eGJsNajkDb@thomas> Content-Type: text/plain; charset="UTF-8"; format=flowed Content-Transfer-Encoding: 7bit X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: 12f1f964-bee9-4357-61c6-08da899bbb76 X-MS-TrafficTypeDiagnostic: SJ0PR02MB8546:EE_ X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: 3SiPV9QLgsz3jNPd3SrZgf6wxnwe5NUvUYndTkJOi54rOgiDsHR0v2xM/CcTQJUWwsGXCh9aNTKcIgrM4Gz3OMLJ87s9IzT/hODIcPqMm/fwXXR84uSelW0UcjvvL7f2FZbnhLBI+UTJ9VLbGnY0OQNJ2WXj0avgAts2BW2L9x6+W+uqFXBB9dnQmB/jlWIAF46j1Fjl3xwVJ/PZvrpOGVLGm9wtO7BMhDgpCYmxj7vTiWaETwoEf5NwOvAAbN1wsQ4cms45vc+JM+qIiKbAWMrTGVaXtHFllo4kCrq7WVBVF29qQAO2b0a2lK051EhmsJ5uvf/xfz0UXARM0SreLTgBnvEuy8DYX+FLuAy66MncTIlyYYsi+B4Mg0KeP31p/8Mq4WVhnKhEcCG/TM/+ORJ2lVmEPdtoVjsJ/6WeURVMHCsiLaRyMnicjmzzhA1lj92STDhVizQPLDmTgg3fSCaKgUQ0qfkOQTz2loqhH7P4q715T7IoTi0PargxULHJMDTiRR4QN7jWD9UFOkG3QUsOVoEW+9Rriik/osnIfE0UnMrj3yNTMjwwyYMN2SY1a1uTGan8T7Q8gYwaeSs4AAT1HKMR5NlJ+Y2dtw6GoPT7cQ3+FHNvEpK1iRmK9vk4uJ6jR5a5GGjb+ytGLzcnOLDY3BKnJBzPnunvUWAACVLDspmFecGBpmDoPrQR9eTG5MWk4RTrzMffaH4staaz1Wl2PTWvh9CLF2Z/5kImZv5GnsV0TCH7uKjKxHlJ63CKS+nxnVwscxRVlizfJCNInhEyLBCJiG7WUS5oZvoBf7023qHbHMmJtvQAyKpcWQAPQWq4ayWWw7eA0fo3jJddaA== X-Forefront-Antispam-Report: CIP:149.199.80.198; CTRY:IE; LANG:en; SCL:1; SRV:; IPV:CAL; SFV:NSPM; H:xir-pvapexch02.xlnx.xilinx.com; PTR:unknown-80-198.xilinx.com; CAT:NONE; SFS:(13230016)(4636009)(376002)(39860400002)(136003)(396003)(346002)(40470700004)(46966006)(36840700001)(47076005)(186003)(426003)(2616005)(316002)(44832011)(36756003)(31686004)(2906002)(53546011)(26005)(336012)(31696002)(40460700003)(356005)(82740400003)(82310400005)(36860700001)(40480700001)(5660300002)(8936002)(478600001)(41300700001)(54906003)(9786002)(7636003)(70206006)(110136005)(4326008)(70586007)(8676002)(50156003)(43740500002); DIR:OUT; SFP:1101; X-OriginatorOrg: xilinx.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 29 Aug 2022 08:51:58.2036 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: 12f1f964-bee9-4357-61c6-08da899bbb76 X-MS-Exchange-CrossTenant-Id: 657af505-d5df-48d0-8300-c31994686c5c X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=657af505-d5df-48d0-8300-c31994686c5c; Ip=[149.199.80.198]; Helo=[xir-pvapexch02.xlnx.xilinx.com] X-MS-Exchange-CrossTenant-AuthSource: DM3NAM02FT010.eop-nam02.prod.protection.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: SJ0PR02MB8546 X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org On 8/29/2022 8:58 AM, Thomas Monjalon wrote: > 22/08/2022 20:27, Long Li: >>> Subject: Re: [Patch v4 01/17] net/mana: add basic driver, build environment >>> and doc >>> >>> On 8/22/2022 4:03 PM, Ferruh Yigit wrote: >>>>> + struct rte_pci_device *pci_dev, >>>>> + struct rte_ether_addr *mac_addr) { >>>>> + struct ibv_device **ibv_list; >>>>> + int ibv_idx; >>>>> + struct ibv_context *ctx; >>>>> + struct ibv_device_attr_ex dev_attr; >>>>> + int num_devices; >>>>> + int ret = 0; >>>>> + uint8_t port; >>>>> + struct mana_priv *priv = NULL; >>>>> + struct rte_eth_dev *eth_dev = NULL; >>>>> + bool found_port; >>>>> + >>>>> + ibv_list = ibv_get_device_list(&num_devices); >>>>> + for (ibv_idx = 0; ibv_idx < num_devices; ibv_idx++) { >>>>> + struct ibv_device *ibdev = ibv_list[ibv_idx]; >>>>> + struct rte_pci_addr pci_addr; >>>>> + >>>>> + DRV_LOG(INFO, "Probe device name %s dev_name %s >>>>> ibdev_path %s", >>>>> + ibdev->name, ibdev->dev_name, >>>>> +ibdev->ibdev_path); >>>>> + >>>>> + if (mana_ibv_device_to_pci_addr(ibdev, &pci_addr)) >>>>> + continue; >>>>> + >>>>> + /* Ignore if this IB device is not this PCI device */ >>>>> + if (pci_dev->addr.domain != pci_addr.domain || >>>>> + pci_dev->addr.bus != pci_addr.bus || >>>>> + pci_dev->addr.devid != pci_addr.devid || >>>>> + pci_dev->addr.function != pci_addr.function) >>>>> + continue; >>>>> + >>>> >>>> As far as I understand, intention of this loop is to find 'ibdev' >>>> matching this device, code gooes through all "ibv device list" for >>>> this, I wonder if there is a easy way for doing this, like a sysfs >>>> entry to help getting this information? >>>> And how mlx4/5 does this? >>> >>> Since there are multiple RDMA devices now, does it make sense to have >>> RDMA bus driver, which can hide some PCIe details under bus, and driver can >>> get PCI and ibdev information during probe? >> >> Mellanox drivers use a similar way to go through the list of IB devices. >> >> Matan, Viacheslav, what are your thoughts on implementing a bus for RDMA devices? > > These are PCI devices first. > Is it a good idea to have a bus driver on top of another one? > What would be the responsibility of such a bus driver? > As far as I can see a 'ibdev' device struct is required to be able to configure the device, so rdma bus can be on top of PCI and find matching 'ibdev' device for it, and driver probe can get pci and ibdev information directly, instead of each rdma device driver walking through all ibdev devices (ibv_get_device_list()).