From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 65CDFA054F; Wed, 7 Sep 2022 15:24:31 +0200 (CEST) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id F188E40143; Wed, 7 Sep 2022 15:24:30 +0200 (CEST) Received: from NAM12-BN8-obe.outbound.protection.outlook.com (mail-bn8nam12on2055.outbound.protection.outlook.com [40.107.237.55]) by mails.dpdk.org (Postfix) with ESMTP id 26BE5400D6 for ; Wed, 7 Sep 2022 15:24:29 +0200 (CEST) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=cKZGtfH04MuXx3mfqX9OD85WMiVNUVLs2IVjB3TWe+PT+aMhGnrWpVL5lGFyBstvVYWb8fes2qYognYilBkOlioIp86AArfX6h41qnJmxCOA0BHIrYdLuv07Hk+eCHuIW/HMjFG2Z93gQkztfstj6FAsM+IkH+A9C8Ow4+KiVWlNWelPK+CC8VApieBzmq94gVYg0WFtIb7Aqs+MGpP4RBcROzbRPOFUFboF5Fd8qVi0dQ2AojIwk37OUbx1Doi1tbzkw/z4WbdoGrnuP2woUC8bd75FgElaCAhv4vDL22tYwyjurU7ypCL2568FNtZpqjg+szeSj39HpMw3nIlTMA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=Y2MavzW5/FHp9N7pUB1f76xsL7EOhbwuQVz8AQ6ezJ8=; b=PYXoLHwR1xeyWzB8LAUw9ym0VZy9K4RfDr6ycpKg1TuQrrd5efiwaELM78udcMo5gNzl4Bf9SHSqVVOmXx78jtYD2Rh3spYJHOWQ61gGQpPPbGOX+OX2hD5LTd+MFSXOCgVOwskbqP7qh8jAwtPp/EYeauniPnKltHT8wNSczhjtMoUnapjJHTMIVw8ZRD2mYxQPXgvIP1T4pyrM/hHwv8XtR3E2GgBGCt6ktUB+/zBO1AqAVc4VGYRNZARtRqK/Bj9rXemPE8MZ10HT0b+uS0yUJAysdC2fzGyNXOgpt4MxGkiqaAVZcZclQ4GBtgYWEvOEbI0kmACuDLs8g0hevQ== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 149.199.80.198) smtp.rcpttodomain=corigine.com smtp.mailfrom=xilinx.com; dmarc=pass (p=none sp=none pct=100) action=none header.from=xilinx.com; dkim=none (message not signed); arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=xilinx.onmicrosoft.com; s=selector2-xilinx-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=Y2MavzW5/FHp9N7pUB1f76xsL7EOhbwuQVz8AQ6ezJ8=; b=bmHPr5etLr3zL6bLRd4wj0W2W3cR5LKaWue015k3KLrqwyjnyUjX9IkFeP1cwcwf9m6aSJRuyCU8uGzINyfl+zO6apifRguBm5iIrVCZK3Qq8tMfjpjeNPth+1ml+DFFDqWlPmQpAbDp9H/UAJIw1St6Jgbney2EiIQyclWtMXo= Received: from BN0PR10CA0002.namprd10.prod.outlook.com (2603:10b6:408:143::16) by CY4PR02MB2213.namprd02.prod.outlook.com (2603:10b6:903:e::19) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5588.11; Wed, 7 Sep 2022 13:24:19 +0000 Received: from BN1NAM02FT032.eop-nam02.prod.protection.outlook.com (2603:10b6:408:143:cafe::27) by BN0PR10CA0002.outlook.office365.com (2603:10b6:408:143::16) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5612.14 via Frontend Transport; Wed, 7 Sep 2022 13:24:19 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 149.199.80.198) smtp.mailfrom=xilinx.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=xilinx.com; Received-SPF: Pass (protection.outlook.com: domain of xilinx.com designates 149.199.80.198 as permitted sender) receiver=protection.outlook.com; client-ip=149.199.80.198; helo=xir-pvapexch02.xlnx.xilinx.com; pr=C Received: from xir-pvapexch02.xlnx.xilinx.com (149.199.80.198) by BN1NAM02FT032.mail.protection.outlook.com (10.13.3.192) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.20.5612.13 via Frontend Transport; Wed, 7 Sep 2022 13:24:19 +0000 Received: from xir-pvapexch02.xlnx.xilinx.com (172.21.17.17) by xir-pvapexch02.xlnx.xilinx.com (172.21.17.17) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2375.24; Wed, 7 Sep 2022 14:24:06 +0100 Received: from smtp.xilinx.com (172.21.105.197) by xir-pvapexch02.xlnx.xilinx.com (172.21.17.17) with Microsoft SMTP Server id 15.1.2375.24 via Frontend Transport; Wed, 7 Sep 2022 14:24:06 +0100 Envelope-to: chaoyong.he@corigine.com, dev@dpdk.org, oss-drivers@corigine.com, niklas.soderlund@corigine.com, andrew.rybchenko@oktetlabs.ru Received: from [10.71.194.74] (port=15458) by smtp.xilinx.com with esmtp (Exim 4.90) (envelope-from ) id 1oVv2D-0002kM-Rg; Wed, 07 Sep 2022 14:24:06 +0100 Message-ID: Date: Wed, 7 Sep 2022 14:24:05 +0100 MIME-Version: 1.0 User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:102.0) Gecko/20100101 Thunderbird/102.2.1 Subject: Re: [PATCH v7 04/12] net/nfp: add initial flower firmware support Content-Language: en-US To: Chaoyong He , "dev@dpdk.org" CC: oss-drivers , Niklas Soderlund , Andrew Rybchenko References: <1660299750-10668-1-git-send-email-chaoyong.he@corigine.com> <1660299750-10668-5-git-send-email-chaoyong.he@corigine.com> From: Ferruh Yigit In-Reply-To: Content-Type: text/plain; charset="UTF-8"; format=flowed Content-Transfer-Encoding: 7bit X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: BN1NAM02FT032:EE_|CY4PR02MB2213:EE_ X-MS-Office365-Filtering-Correlation-Id: 97a1950f-d17e-48b0-6fd2-08da90d44526 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: dxYGxb8KIsxXsYN2AIbeg4BgKut/4lVZLpwYbW3i48c99r1ctcs0/v0p+PHPEXpAv8nlM8PdHyQk9p08Pff2Gv2egV81LlVQzvFDz40FG/LUHLwAVvbzL5VpWUY9bdcJX+Ry4OfJPnFzt1m0QIQowCp2fJVaw2eMhPRxyOiIukK3xXLklMCZCOCJnmwejRhg5rzeCl4BYmHnARfxuH/9LpI1D8NRxZ427ANoQHUNPksq3H7+oLwCqGkH7z+lTeUHG6u/Dkyf7Nkxd3+UefmzdYstDTrsFkVrGATEJYgLfPqvCiQpRRmNliFEu8Rt2vzdCSITTPbkwFDhx2sMOs/RTlVBHr0AU2s8doQrIBvYgIfRwcyH3q76L0Lk7YwgXQx2wIYwjMr5epbtl0T2qifuGBym957vUbox2XLHz4jHH/rT5LFLRscUt1ZE9YfCWyKibfhkZ6XPa4VP+gF8AZCjn6O/LZpKTlFqJulHt40bvW9ZRJv8Y8AK2eR0AjP54M0Iv9W/VMH8lr0+tJmAKi8OenyYNie2mFsaz2pJKFFT4miIEtTxfpigBeX4d5Q5z1P5X5NHZ6c6ASoqRYv3r8A1vqY7QPsiTcPnwOE/kSNb6eXEhhQKObIe97Xkl8IdHdQutQlk/aWEf2q5GI5QIytqLw27smRKlONCTiIN7UrSleHwLHGhWzQLl75ini2egDEO+3LKDigva3BV5cb47E7hyifEmCfnso3rJJ2ZvpEIZsGmwWiQRZDwIP1cw1J1jjifrmOe4t1KC8loRlDRW5LlGyRO2GHaQASZk33Hmp7IelXS9c6iOAKSG3uv0K54tN7WO2Jf0OJ8mbBGGrQC0VWFBg== X-Forefront-Antispam-Report: CIP:149.199.80.198; CTRY:IE; LANG:en; SCL:1; SRV:; IPV:CAL; SFV:NSPM; H:xir-pvapexch02.xlnx.xilinx.com; PTR:unknown-80-198.xilinx.com; CAT:NONE; SFS:(13230016)(4636009)(376002)(39860400002)(396003)(346002)(136003)(40470700004)(36840700001)(46966006)(7636003)(316002)(36756003)(40480700001)(2906002)(82740400003)(356005)(40460700003)(31696002)(83380400001)(53546011)(82310400005)(41300700001)(26005)(478600001)(36860700001)(70206006)(8676002)(186003)(4326008)(426003)(47076005)(8936002)(2616005)(336012)(9786002)(5660300002)(44832011)(70586007)(54906003)(110136005)(31686004)(50156003)(43740500002); DIR:OUT; SFP:1101; X-OriginatorOrg: xilinx.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 07 Sep 2022 13:24:19.1772 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: 97a1950f-d17e-48b0-6fd2-08da90d44526 X-MS-Exchange-CrossTenant-Id: 657af505-d5df-48d0-8300-c31994686c5c X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=657af505-d5df-48d0-8300-c31994686c5c; Ip=[149.199.80.198]; Helo=[xir-pvapexch02.xlnx.xilinx.com] X-MS-Exchange-CrossTenant-AuthSource: BN1NAM02FT032.eop-nam02.prod.protection.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: CY4PR02MB2213 X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org On 9/7/2022 9:22 AM, Chaoyong He wrote: > CAUTION: This message has originated from an External Source. Please use proper judgment and caution when opening attachments, clicking links, or responding to this email. > > >> -----Original Message----- >> From: Ferruh Yigit >> Sent: Monday, September 5, 2022 11:40 PM >> To: Chaoyong He ; dev@dpdk.org >> Cc: oss-drivers ; Niklas Soderlund >> ; Heinrich Kuhn >> >> Subject: Re: [PATCH v7 04/12] net/nfp: add initial flower firmware support >> >> On 8/12/2022 11:22 AM, Chaoyong He wrote: >>> Adds the basic probing infrastructure to support the flower firmware. >>> This firmware is geared towards offloading OVS and can generally be >>> found in /lib/firmware/netronome/flower. It is also used by the NFP >>> kernel driver when OVS offload with TC is desired. >> >> '/lib/firmware/netronome/flower' FW is loaded (automatically ?) by kernel >> driver, right? >> Is there anything related to the DPDK with the name/path of the FW? >> >> >> And I wonder if these kind of information should be part of driver >> documentation? >> >> <...> >> >>> @@ -965,6 +968,16 @@ >>> goto hwqueues_cleanup; >>> } >>> break; >>> + case NFP_APP_FLOWER_NIC: >>> + PMD_INIT_LOG(INFO, "Initializing Flower"); >>> + pci_dev->device.driver = &pci_drv->driver; >> >> Why this assignment is required? Driver shouldn't need to update this >> bus related data struct. > > Following the framework's logic: > function 'rte_pci_probe_one_driver()' in file drivers/bus/pci/pci_common.c > > ``` > ... > ret = dr->probe(dr, dev); // here call our nfp_pf_pci_probe() > if (ret) { > ... > } else { > dev->device.driver = &dr->driver; > } > > ``` > > Here the framework will do the assignment. > > But in our logic, if we won't add this assignment, we will run into a coredump: > ``` > Program terminated with signal SIGSEGV, Segmentation fault. > #0 rte_eth_dev_info_get (port_id=0, dev_info=0x7ffe9c3d3f70) at ../lib/ethdev/rte_ethdev.c:3138 > 3138 dev_info->driver_name = dev->device->driver->name; > (gdb) bt > #0 rte_eth_dev_info_get (port_id=0, dev_info=0x7ffe9c3d3f70) at ../lib/ethdev/rte_ethdev.c:3138 > #1 0x00007fa95128565f in rte_eth_dev_configure (port_id=0, nb_rx_q=1, nb_tx_q=1, > dev_conf=0x7fa94ce338e0 ) at ../lib/ethdev/rte_ethdev.c:1110 > #2 0x00007fa94cdf74b7 in nfp_flower_init_ctrl_vnic (hw=0x2001e8530) > at ../drivers/net/nfp/flower/nfp_flower.c:1035 > #3 0x00007fa94cdf7b32 in nfp_init_app_fw_flower (pf_dev=0x2001e8c80) > at ../drivers/net/nfp/flower/nfp_flower.c:1219 > #4 0x00007fa94ce30e33 in nfp_pf_init (pci_dev=0xcd22c0, pci_drv=0x7fa94ce44460 ) > at ../drivers/net/nfp/nfp_ethdev.c:976 > #5 0x00007fa94ce31429 in nfp_pf_pci_probe (pci_drv=0x7fa94ce44460 , dev=0xcd22c0) > at ../drivers/net/nfp/nfp_ethdev.c:1152 > #6 0x00007fa94e8f8f34 in rte_pci_probe_one_driver (dr=0x7fa94ce44460 , dev=0xcd22c0) > at ../drivers/bus/pci/pci_common.c:269 > #7 0x00007fa94e8f91c0 in pci_probe_all_drivers (dev=0xcd22c0) at ../drivers/bus/pci/pci_common.c:353 > #8 0x00007fa94e8f9244 in pci_probe () at ../drivers/bus/pci/pci_common.c:380 > #9 0x00007fa951163c0e in rte_bus_probe () at ../lib/eal/common/eal_common_bus.c:72 > #10 0x00007fa951193903 in rte_eal_init (argc=10, argv=0xcc5150) at ../lib/eal/linux/eal.c:1279 > #11 0x0000000000601531 in dpdk_init__ (ovs_other_config=0xccefd0) at lib/dpdk.c:466 > #12 0x0000000000601913 in dpdk_init (ovs_other_config=0xccefd0) at lib/dpdk.c:545 > #13 0x00000000004129ed in bridge_run () at vswitchd/bridge.c:3252 > #14 0x0000000000418364 in main (argc=11, argv=0x7ffe9c3d49c8) at vswitchd/ovs-vswitchd.c:129 > > ``` > > Our nfp card use `control message` to exchange message between PMD and firmware when we use flower firmware. > The control message is in the form of pkt and we use a `ctrl vNIC` ehtdev as the agent to send and receive these pkts. > e.g., if we want to create representor port, the PMD must send the corresponding control message to firmware. > To be able to send and receive pkt, we must do some configure steps to this ethdev firstly. > But the framework has not do the assignment step at this moment. > And this is where the problem comes from. > I am not sure about PMD calling 'rte_eth_dev_configure()', this API is intended for application. And even worse, not sure if it is allowed to call 'rte_eth_dev_configure()' before driver probe completed successfully. It looks to me assigning 'pci_dev->device.driver' within the PMD is hack to allow it run 'rte_eth_dev_configure()' before probe() completed. Do you really need to call ''rte_eth_dev_configure()' to prepare ethdev for the 'control message' exchange? What exactly needs to be configured for this? > Logically, if our probe process is successful, assignment again won't import any problem. > If our probe process failed, our logic will undo the assignment and the framework will go as its original logic. > So maybe we can keep the logic now? > Or there exist another way more standard? >