From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id BA20DA00C3; Tue, 13 Sep 2022 11:08:44 +0200 (CEST) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 99EF340156; Tue, 13 Sep 2022 11:08:44 +0200 (CEST) Received: from NAM10-DM6-obe.outbound.protection.outlook.com (mail-dm6nam10on2070.outbound.protection.outlook.com [40.107.93.70]) by mails.dpdk.org (Postfix) with ESMTP id A7C1940151 for ; Tue, 13 Sep 2022 11:08:43 +0200 (CEST) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=h+tZNybZ5zJlqHyKG4+ZGPpxK7pSNdCSdIG45Jjnw2QHRhqJ6FJ6paUoEfYDo2UD6TwZJHYmSElUNmCEvqXyrrYNi2Xjf1PYvvKWHZXgMsLtwKx5nEHfeZjnWRzhjsVz03tdybMJF3DMTnW9LQKZx1bFCW6CVDKJgRVTA2p/NA7RKjnRLnZZ3aiThINahK3De3w2t+sYeRLbGHRFo66aNDlsOlF6ndWifqX6+/MeBQ1mpn4XHP3EoXtQYLrXs9I3wIbYRo/r9cbQqToeZYz8nYWFJYJY0ea8twd7ZibC37jzwoMlPy5pNmb3Mww8ai0X64hZiO0c1henoEMnEBaeEA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=VytJ4b7yL+nbHWPRuumdndIOurwD23kj53HfzsFd7is=; b=S7uXproJtLpSbtVAr866P9BNxZuiHvNFBxKW7O3G+s6QErx2C2uJK0l9Wv3zApvtp536hgSZVF0KrDZP7I/ucdnR0CodLTuHs5NftY7zFtY9PHgHMttfUtdFpykf8P8Gi/C+Y5b/wj5db0iKbXlbhrvutI979NHRgovUtnIbCzyMHTpNpwdm6q3JEFzEXOVXgwmILxaPVhIB+9wI9sVmw1myJlhJd8tHk7WpLxwSdgLyTGmhoSFc7gJgrJvun++dW4JCRwTK3osJYljMvhSjDw0IHxFZE/v+zfg9grKaORqkPgPYjHuO4k/k7xvAvTiAqqegahApFByPtaVdSpmaIA== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 149.199.80.198) smtp.rcpttodomain=corigine.com smtp.mailfrom=xilinx.com; dmarc=pass (p=none sp=none pct=100) action=none header.from=xilinx.com; dkim=none (message not signed); arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=xilinx.onmicrosoft.com; s=selector2-xilinx-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=VytJ4b7yL+nbHWPRuumdndIOurwD23kj53HfzsFd7is=; b=qDt0FUgYNV186hrrVnr88e9vfnK9o9hwqBqFBc+KmYJqlkorJ2sbdcT/b/ZZm3/cffFJwZvSHHTkNNuFLvEG1Yx7roH+nF9uXfiOqK5/EIs/COP21KN3cdnCX1zDv7ssFg1tgMpdL8NhiUuSt/y5kpCdiY6IrZDdVSFHaVA5YPI= Received: from DM6PR02CA0134.namprd02.prod.outlook.com (2603:10b6:5:1b4::36) by CO6PR02MB8834.namprd02.prod.outlook.com (2603:10b6:303:145::13) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5612.22; Tue, 13 Sep 2022 09:08:40 +0000 Received: from DM3NAM02FT042.eop-nam02.prod.protection.outlook.com (2603:10b6:5:1b4:cafe::1b) by DM6PR02CA0134.outlook.office365.com (2603:10b6:5:1b4::36) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5612.12 via Frontend Transport; Tue, 13 Sep 2022 09:08:39 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 149.199.80.198) smtp.mailfrom=xilinx.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=xilinx.com; Received-SPF: Pass (protection.outlook.com: domain of xilinx.com designates 149.199.80.198 as permitted sender) receiver=protection.outlook.com; client-ip=149.199.80.198; helo=xir-pvapexch02.xlnx.xilinx.com; pr=C Received: from xir-pvapexch02.xlnx.xilinx.com (149.199.80.198) by DM3NAM02FT042.mail.protection.outlook.com (10.13.4.213) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.20.5612.13 via Frontend Transport; Tue, 13 Sep 2022 09:08:39 +0000 Received: from xir-pvapexch02.xlnx.xilinx.com (172.21.17.17) by xir-pvapexch02.xlnx.xilinx.com (172.21.17.17) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2375.24; Tue, 13 Sep 2022 10:08:38 +0100 Received: from smtp.xilinx.com (172.21.105.197) by xir-pvapexch02.xlnx.xilinx.com (172.21.17.17) with Microsoft SMTP Server id 15.1.2375.24 via Frontend Transport; Tue, 13 Sep 2022 10:08:38 +0100 Envelope-to: chaoyong.he@corigine.com, jerinj@marvell.com, thomas@monjalon.net, andrew.rybchenko@oktetlabs.ru, oss-drivers@corigine.com, niklas.soderlund@corigine.com, dev@dpdk.org Received: from [10.71.194.74] (port=39545) by smtp.xilinx.com with esmtp (Exim 4.90) (envelope-from ) id 1oY1uI-0003XD-DZ; Tue, 13 Sep 2022 10:08:38 +0100 Message-ID: <9aed41cf-c1f0-fa2e-cc2c-f4dc1e17f390@xilinx.com> Date: Tue, 13 Sep 2022 10:08:38 +0100 MIME-Version: 1.0 User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:102.0) Gecko/20100101 Thunderbird/102.2.1 Subject: Re: [PATCH v8 05/12] net/nfp: add flower PF setup logic Content-Language: en-US To: Chaoyong He , Jerin Jacob Kollanukkaran , Thomas Monjalon , Andrew Rybchenko CC: oss-drivers , Niklas Soderlund , "dev@dpdk.org" References: <1662626702-17254-1-git-send-email-chaoyong.he@corigine.com> <1662626702-17254-6-git-send-email-chaoyong.he@corigine.com> <49888fb1-e16a-9d55-9855-7e7807922dea@xilinx.com> <82a90702-3873-02bd-e804-ba66aad2d0ef@xilinx.com> From: Ferruh Yigit In-Reply-To: Content-Type: text/plain; charset="UTF-8"; format=flowed Content-Transfer-Encoding: 8bit X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: DM3NAM02FT042:EE_|CO6PR02MB8834:EE_ X-MS-Office365-Filtering-Correlation-Id: 93173cd1-0d97-40e2-3d5a-08da95678c90 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: /GxMCRhXhSRK2xymyXrX1ZqX0R/xyTA4umqNL6RcUN/b3Y+TF9Z8LYRPbcDIisSouNy0GTHbr1n9mhgk2pwv/MgR4QTNXpzPHn4xuvBuN16AeqT4h4jpifKq7nm+Y9Fs9NMY4wpP/vAzRwt/3mjYFR2kW6nPli4QAbgZpyakuHUGHCE6l66Ns3Vn7Z1ZyAgNFCGfz2JIK8MfaYmo3X42i2l8WTwjPUGKOc1OyhoUxqKHTa/di0HMoJCYZnz0LuCeZ8IcrQi8m9Ohe+19mrgSznklDrB8RECdrA2TB0bpOfPEmvL1hVE1S+8YSx4ULr1ji89Rs6MiunqiwqNeAC29q2J7rFZp3SsRykHa3bpsKJRzDparbAEDjirAGLR86uAAJqOLp7pp5mBuCTrSfNYs+DMUIhmzFAgyNs1SWasumQ4cvJonRDtJOn37ZSyiDrHIuAXyHMx5SXNEeQFsGotdtGYU9A8e0mTT2MyYtL7pSIkVJ+Eavyp3/szuHoDsjy2AQi6Bc6F92+tlbw1GWRydGEtPgw+wGleUt/IoAkYj2Og2iyADOVDrh/h08Isw+C960vZcVk36an6Qc0nWegGHFk/H7om9onN0XMCZ5qj/eI3WZqgWOu0QurczxFN3ZDNIghtcA1NdUo4mYMVQvYSruZ8eOxe0zgJMX2xRg70n9dE1flHaskCl8Y0Ks2PnvRZGNFJAJeSYu4NlTATGjb/ISW6npNAohgLEpBORz+bPtfcpELDvpftfaYR9+1BpVjiR7KafXZRv2/vea4adC5CpM9AkzfDhQA9+IrvFIQdi939afAs9z4gX4zzl6NvkToQaAywEtTRnnUxh0JdnJJTeNg== X-Forefront-Antispam-Report: CIP:149.199.80.198; CTRY:IE; LANG:en; SCL:1; SRV:; IPV:CAL; SFV:NSPM; H:xir-pvapexch02.xlnx.xilinx.com; PTR:unknown-80-198.xilinx.com; CAT:NONE; SFS:(13230022)(4636009)(396003)(346002)(39860400002)(376002)(136003)(451199015)(36840700001)(46966006)(40470700004)(9786002)(31696002)(2906002)(31686004)(426003)(5660300002)(8676002)(478600001)(316002)(4326008)(40460700003)(356005)(110136005)(54906003)(40480700001)(186003)(70206006)(44832011)(82740400003)(41300700001)(8936002)(70586007)(83380400001)(47076005)(7636003)(53546011)(36756003)(26005)(82310400005)(336012)(2616005)(36860700001)(50156003)(43740500002); DIR:OUT; SFP:1101; X-OriginatorOrg: xilinx.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 13 Sep 2022 09:08:39.6279 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: 93173cd1-0d97-40e2-3d5a-08da95678c90 X-MS-Exchange-CrossTenant-Id: 657af505-d5df-48d0-8300-c31994686c5c X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=657af505-d5df-48d0-8300-c31994686c5c; Ip=[149.199.80.198]; Helo=[xir-pvapexch02.xlnx.xilinx.com] X-MS-Exchange-CrossTenant-AuthSource: DM3NAM02FT042.eop-nam02.prod.protection.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: CO6PR02MB8834 X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org On 9/13/2022 7:51 AM, Chaoyong He wrote: > >> On 9/9/2022 3:36 AM, Chaoyong He wrote: >>>> On 9/8/2022 9:44 AM, Chaoyong He wrote: >>>>> Adds the vNIC initialization logic for the flower PF vNIC. The >>>>> flower firmware exposes this vNIC for the purposes of fallback >>>>> traffic in the switchdev use-case. >>>>> >>>>> Adds minimal dev_ops for this PF device. Because the device is being >>>>> exposed externally to DPDK it should also be configured using DPDK >>>>> helpers like rte_eth_configure(). For these helpers to work the >>>>> flower logic needs to implements a minimal set of dev_ops. >>>>> >>>>> Signed-off-by: Chaoyong He >>>>> Reviewed-by: Niklas Söderlund >> >> <...> >> >>>>> +static int >>>>> +nfp_flower_init_pf_vnic(struct nfp_net_hw *hw) { >>>>> + int ret; >>>>> + uint16_t i; >>>>> + uint16_t n_txq; >>>>> + uint16_t n_rxq; >>>>> + uint16_t port_id; >>>>> + unsigned int numa_node; >>>>> + struct rte_mempool *mp; >>>>> + struct nfp_pf_dev *pf_dev; >>>>> + struct rte_eth_dev *eth_dev; >>>>> + struct nfp_app_fw_flower *app_fw_flower; >>>>> + >>>>> + static const struct rte_eth_conf port_conf = { >>>>> + .rxmode = { >>>>> + .mq_mode = RTE_ETH_MQ_RX_RSS, >>>>> + .offloads = RTE_ETH_RX_OFFLOAD_CHECKSUM, >>>>> + }, >>>>> + .txmode = { >>>>> + .mq_mode = RTE_ETH_MQ_TX_NONE, >>>>> + }, >>>>> + }; >>>>> + >>>>> + /* Set up some pointers here for ease of use */ >>>>> + pf_dev = hw->pf_dev; >>>>> + app_fw_flower = NFP_PRIV_TO_APP_FW_FLOWER(pf_dev- >>>>> app_fw_priv); >>>>> + >>>>> + /* >>>>> + * Perform the "common" part of setting up a flower vNIC. >>>>> + * Mostly reading configuration from hardware. >>>>> + */ >>>>> + ret = nfp_flower_init_vnic_common(hw, "pf_vnic"); >>>>> + if (ret != 0) >>>>> + goto done; >>>>> + >>>>> + hw->eth_dev = rte_eth_dev_allocate("nfp_pf_vnic"); >>>>> + if (hw->eth_dev == NULL) { >>>>> + ret = -ENOMEM; >>>>> + goto done; >>>>> + } >>>>> + >>>>> + /* Grab the pointer to the newly created rte_eth_dev here */ >>>>> + eth_dev = hw->eth_dev; >>>>> + >>>>> + numa_node = rte_socket_id(); >>>>> + >>>>> + /* Fill in some of the eth_dev fields */ >>>>> + eth_dev->device = &pf_dev->pci_dev->device; >>>>> + eth_dev->data->dev_private = hw; >>>>> + >>>>> + /* Create a mbuf pool for the PF */ >>>>> + app_fw_flower->pf_pktmbuf_pool = nfp_flower_pf_mp_create(); >>>>> + if (app_fw_flower->pf_pktmbuf_pool == NULL) { >>>>> + ret = -ENOMEM; >>>>> + goto port_release; >>>>> + } >>>>> + >>>>> + mp = app_fw_flower->pf_pktmbuf_pool; >>>>> + >>>>> + /* Add Rx/Tx functions */ >>>>> + eth_dev->dev_ops = &nfp_flower_pf_vnic_ops; >>>>> + >>>>> + /* PF vNIC gets a random MAC */ >>>>> + eth_dev->data->mac_addrs = rte_zmalloc("mac_addr", >>>> RTE_ETHER_ADDR_LEN, 0); >>>>> + if (eth_dev->data->mac_addrs == NULL) { >>>>> + ret = -ENOMEM; >>>>> + goto mempool_cleanup; >>>>> + } >>>>> + >>>>> + rte_eth_random_addr(eth_dev->data->mac_addrs->addr_bytes); >>>>> + rte_eth_dev_probing_finish(eth_dev); >>>>> + >>>>> + /* Configure the PF device now */ >>>>> + n_rxq = hw->max_rx_queues; >>>>> + n_txq = hw->max_tx_queues; >>>>> + port_id = hw->eth_dev->data->port_id; >>>>> + >>>>> + ret = rte_eth_dev_configure(port_id, n_rxq, n_txq, &port_conf); >>>> >>>> Still not sure about PMD calling 'rte_eth_dev_configure()', can you >>>> please give more details on what specific configuration is expected with >> that call? >>> >>> The main configuration we need is the number of rx/tx queue. >>> So we should use the internal api `eth_dev_rx/tx_queue_config` to instead? >>> >> >> nb_rx_q/nb_tx_q are parameters provided by user (via >> rte_eth_dev_configure()), won't is wrong for PMD to set a value on its own? >> >> Why nb_rx_q/nb_tx_q are required in the probe() stage? Probe stage is not >> to configure the device. > > Our nfp card use `control message` to exchange message between PMD and firmware when we use flower firmware. > The control message is in the form of pkt and we use a `ctrl vNIC` ehtdev as the agent to send and receive these pkts. > e.g., if we want to create representor port, the PMD must send the corresponding control message to firmware. > > This `ctrl vNIC` is totally user app Invisible, to make it able to send and receive pkt, we must do some configure steps to this ethdev ourselves firstly. > We can don't use 'rte_eth_dev_configure()', but we still should do the needed configure steps to make sure the device can work. > How ethdev instance becomes app invisible, unless owner set properly I think apps can able to see it. And if needs to be internal, do you really need to create an ethdev? Why not communicate with HW via internal functions? Cc'ed Jerin, their driver also communicate with FW before probing. @Jerin, can you please review this patch and help on the design? @Thomas, @Andrew, can you please check the design too?