From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id E14C5A0548; Thu, 8 Sep 2022 10:46:05 +0200 (CEST) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 71B3242B6C; Thu, 8 Sep 2022 10:45:39 +0200 (CEST) Received: from NAM11-DM6-obe.outbound.protection.outlook.com (mail-dm6nam11on2101.outbound.protection.outlook.com [40.107.223.101]) by mails.dpdk.org (Postfix) with ESMTP id E5C5042B6C for ; Thu, 8 Sep 2022 10:45:37 +0200 (CEST) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=SXzzurxWn5BrPKxVaAXS8fHX3zphIKK8sd+cX9+5Opqe97s+23VlZnoNUx/JluOrg/0NxVZ2sTBXl3y7SntM9fVV0YRUNdzvfwzJJOpecbkOb49el9IuK7FIzOq91J1BFg9XZhJuEDsng5os0w1Cn0THaziTlfjlo4eKQjZbdXZELC+esMHzT5PcfxmqTPBXkYIq0KSmtgxv42kBdNFlQDRAIxeQg0UzDnFav0ZNnlzUSb1mz/5duSlwhL22mT4LBAaQUMmcXa3/rHCsIDHYdIHCPrhEiH0LJ2iXHvJf3t/KJ3QNkL5SGRTgR1+UM0z4or8tVxGOJVovEVAaKevHqw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=uTDLHxqnGiuqN8uRfdkwcC44LdKTOeRc466kpXJ0wG0=; b=keKFVkc8GtoCEYw2W8EA2C1A4sp307S3Kkr9vpRIH2CjInWeV4fMvFWT4vlX0vs5XdNaV7j//6TBZo7FidQD0gM9zM0YIoDqEZmcm8ybafsg0BtuCTBJTG+JN34QKSe88Cy7yV/809ZEfwT1qTuy85mUopDSWSODXq1JN8eUa/S7QSQLXOAsVsFC38J1f+prc4268hUu4z+5K2HnD5JXAyPc/+peVf1mB812lFyzqV2M2teU7rRV4/cDiemjOKuxIa5bvlX6f8R0OMMTPe7ZBhzOwAw/7/Aem+G8VgJsVDXAjfYHrQZfOcRPbWE+2ELVzpCdZ3WWlXOA7om/fBXS2A== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=corigine.com; dmarc=pass action=none header.from=corigine.com; dkim=pass header.d=corigine.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=corigine.onmicrosoft.com; s=selector2-corigine-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=uTDLHxqnGiuqN8uRfdkwcC44LdKTOeRc466kpXJ0wG0=; b=pTxe1qgjGugv6KjNma+L9UWyN7uecMF/S3dHPczfbEbsdC4JAdrYvYfapp05Qsc3FDRs5C4hTs/hjCWU+a7uZSz6MzZSWyJZjV6uZVmk3HIdiFN2Op4nK3NPM3WjQ/L5tyRtdkzNIlIJFUnsZ3g6k8uXcHuLtd4k6595D/DLiBE= Authentication-Results: dkim=none (message not signed) header.d=none;dmarc=none action=none header.from=corigine.com; Received: from SJ0PR13MB5545.namprd13.prod.outlook.com (2603:10b6:a03:424::5) by MW4PR13MB5940.namprd13.prod.outlook.com (2603:10b6:303:1b8::10) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5588.3; Thu, 8 Sep 2022 08:45:36 +0000 Received: from SJ0PR13MB5545.namprd13.prod.outlook.com ([fe80::402d:6abc:83a8:2431]) by SJ0PR13MB5545.namprd13.prod.outlook.com ([fe80::402d:6abc:83a8:2431%6]) with mapi id 15.20.5612.009; Thu, 8 Sep 2022 08:45:36 +0000 From: Chaoyong He To: dev@dpdk.org Cc: oss-drivers@corigine.com, niklas.soderlund@corigine.com, Chaoyong He Subject: [PATCH v8 05/12] net/nfp: add flower PF setup logic Date: Thu, 8 Sep 2022 16:44:55 +0800 Message-Id: <1662626702-17254-6-git-send-email-chaoyong.he@corigine.com> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1662626702-17254-1-git-send-email-chaoyong.he@corigine.com> References: <1662626702-17254-1-git-send-email-chaoyong.he@corigine.com> Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-ClientProxiedBy: SI2PR01CA0023.apcprd01.prod.exchangelabs.com (2603:1096:4:192::17) To SJ0PR13MB5545.namprd13.prod.outlook.com (2603:10b6:a03:424::5) MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: SJ0PR13MB5545:EE_|MW4PR13MB5940:EE_ X-MS-Office365-Filtering-Correlation-Id: 656b0701-5021-4b49-486e-08da91767f96 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: D3pqufzQkS1VGfrayHIxwsSR+69EMW8BzGQnFGegWaZhif9oz+UYpwyGfBdokYse74MCmN+bbr2s2v4WEoNFySnNAgd6pdZUGVA8YhJlAHD2dSLZxpLD42HTTqVdz0cY+tfGR8jtScUzoj7dWVdgmC1V6q6oasS0H+1pAw+724ceLinn0JGjFd6aJsZ30sTaLyDl1Li+xrmFjXgloJksqKgDcK+xqa8fY6IXczkqGT8lMbwOd8OhkhWXUdx6QwWnJrxycX8LmPXEuyrpnMDBSgbWFIjCk150fIf1tcF1iRbbtSuFlRrhulQqQ3nvglioCykn1jQ7GcMs2AeiMDEyhXy9oU0uzQhVklY6jgrWdzUnkleYZ24NE5TbHlImIZmMWlA6teYFvuc6XOcoP7JqCNiW1/STinPajkJIYaSH6DRWh5BJ/+dFzMi237gedsYMiBXexDV727XCGxSSGK91lxfYnX6WQZ06rnWnQGd5UY+EYDmn1qPJ0Ox9oD7a3ldr0c5mb851Mz3jZu4QcL+UY5HiHZdixmKH+bkGsxXwPo/KyAXpvtvgRJthUBZjGBqfQuvRT5cFINzLN5Hh9MdlzELwAMvJXXtWij1k+ZUpU5FQXOIn9izfquWmRAL/VoblpXCzpgeMhey4lMqvixsEn7G4k15eQS6XI0DwI5g0CAQiIXD9n6l/LRUj5ioxOVhBS/0ibUNjMWhzOIOjsFeW/TkWGVUjKgR258BpaDPPZv7A85o2X/Zj6wcoMlDydA7u3JvVGj0fOCHA4lfWLywKwA== X-Forefront-Antispam-Report: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:SJ0PR13MB5545.namprd13.prod.outlook.com; PTR:; CAT:NONE; SFS:(13230016)(4636009)(396003)(366004)(39840400004)(346002)(376002)(136003)(107886003)(44832011)(2616005)(30864003)(41300700001)(6916009)(38350700002)(83380400001)(66574015)(26005)(6512007)(186003)(36756003)(38100700002)(4326008)(2906002)(6506007)(316002)(66946007)(478600001)(86362001)(5660300002)(66476007)(52116002)(6486002)(6666004)(8936002)(8676002)(66556008); DIR:OUT; SFP:1102; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: =?utf-8?B?OGNkZ29pRElFN21MMTBXZ2hLQzFMaGRJNlltRXViNnFmb2R0dzY4RVNmMm5E?= =?utf-8?B?Qzc2bVVSUll1T0hudzlFWEhLWjhvWi84a250RGpCT3dFc3dXaFlVb3prZVJJ?= =?utf-8?B?NU5xcTJwVkxLRS83UHNSbDhEc2Z2RXpIdGVKM3I5OURlSytEaTdubUFQRnBq?= =?utf-8?B?cFZKK1dOS09SK05WUE5Gam5uQ1lXSUtqTmlNUms2aTdXeWlMZlllb2RYVjJp?= =?utf-8?B?Z3ZER21pZTYrbWNDWEg4clo1WmVtZ1VLK1ZlYWgxRUdobzVrUzU0UU1YdUhk?= =?utf-8?B?YkVZZ2htTjMwWHhKb21HY2Vid3A4cXFNT0R2ajdHeEZUWjlTYk5jZ0hGU2lw?= =?utf-8?B?UXRkZDl2M2hsaTdVNlNwRy8yTVZYcXVURXdJMXREVzVMZDdCK2VwL3N1VGI4?= =?utf-8?B?a0d4d3dGamlsbVNKSVBrUEZRSm1uVVExTGpZRlpEaFJ1Um5JYXk4Y0c4UUwx?= =?utf-8?B?UGY4Nm5VZUg5bXU1RkVOdk9KWExWQ1NNTlVpRms4YXdrTGYyRkoyTHBFZ2Jw?= =?utf-8?B?WVVIbm14NmlUZHNtMW4xUTRzMllvdXRNZWtDYVNvL0ZFZmlPYjNGaHVGK0Jr?= =?utf-8?B?SGVOMnRJZnMvMzJpZ0UxZEVNTTV6MHBEWkltVC8wVDZESWJGVHJLYmlnU09B?= =?utf-8?B?ZTVQYmJFRHdCZTAxRGJSaTVucG9DYmVkQVRXLzAyQWpSZ2Yyb3U5dU54MEUv?= =?utf-8?B?QVpyVTgxZFIwZFk2L2FldUh3U29lQjNXaUkzQVRibXZjb2dLKzdqMGxVUjI1?= =?utf-8?B?Y0tQVzF4dXNORWxsOXNUL0tuNnFrOWRYSmNzTStzQ2YrWjVrQnhxNlVaczd3?= =?utf-8?B?d1ZxRnJib3pNdjh4bW84QWUvZmZUV2JWY0U5YVZ4eDdFQTBaMDVVU2hkOUpZ?= =?utf-8?B?bERzcnJLR1BCam14ZE9VWVNSbUUrdTJwQUZGTUdjaWJhYVdYT3M4R0hrUXpa?= =?utf-8?B?T29VNXBnR3ZzZWkrdnl1M0tTTWNiZGNQSVRTcC9nOW80U1d4RjFCLzRxcUZp?= =?utf-8?B?OXgyYkVNR05RbjZxZTZxR24wYkk5dVArdGQ1L09keStYMS9iRlM0N1VMdHN5?= =?utf-8?B?VzZ5WWtsSkg4M1duUVlOb3drRWQ1U0Z6L0I5T1FnSVpzbzhrdjN1TFlianFU?= =?utf-8?B?UFNheTQvb1d4aUM1dVJ2UTJMZ0JkWEt0RUhvd0RIZTdnNDA3QXBPM2R2Tk9E?= =?utf-8?B?SlZWUkwxQXg2RlpVTmNQUnM1YzN2QVJrN2Q3TS9ia1RNSnpHN2N4QWhaZDlm?= =?utf-8?B?Zm1WWWg1UHJXaWorNmYrcWtqblpZSGxMRUVKUWxSQVZoaUxMQkw4YnJzUFpB?= =?utf-8?B?NXZmUmsxanZ2Wk9oWThYdUJIZVpzNUR1T2hyWFJ5Vm41KzZsZ1NvNnQrTUZC?= =?utf-8?B?TGcxTkphd0EvaHZwRVQvdmlhWjdqT1hOM3B2OFE1Mk83U29vT2JnRzBxSnpo?= =?utf-8?B?RS9EdHRSRjNRY3hhYlN3TkllK0lkVSsyZGxwSFRVMVFEWVJMM0VpRFFZOW14?= =?utf-8?B?STVyaEpNMHJ3MzBpVUFqNXNsblplRjlSRkc4clNNTlJTUndSNnlvUjJnSFBm?= =?utf-8?B?YUVmQnkycmhiTGJPcXo1L3FobWtnNlJ1cjA0Z1dXeTdLYVdEakpTYmJtTDBG?= =?utf-8?B?akZ0MSthSHorSS8vbjRpZkJscUY5K2JmRTM4YjdqbmU5Z0VkcUJOMWNkM1Q4?= =?utf-8?B?Tmp3ZXhDNnVCeVBiNHI0MFBjOVo3aGgrT1Y5MzdVZlRVUldvRFhhSnhUOGZC?= =?utf-8?B?eEZjWDFXK1I1RnRaL3NVWEgrMW5xNTE0dnZoNVc5bUdOSnZPNVVMRHF6c2pJ?= =?utf-8?B?Q21PZXVBNlpFSiszTVl1THpYRmoyQlZrdXJOVlIrM2hza0ZRRXljTWZXZDBP?= =?utf-8?B?TEJnUFY2amRJczE1UmpHbDJSbnZMTGhpTHI1aXUvOWtyUVJsYW5BcFZZWGwr?= =?utf-8?B?aVV0elMwbmZrZ0FZU1VKc1l5a0FFNExsUXVpaGdFczZhTHdVOFkrNUlrVm1p?= =?utf-8?B?ZVgySXpaQjZQVjk0OWo0cnpKb1F4SElnMTcwbXVLRVVLOTUxUFZUWGhIWFJ6?= =?utf-8?B?OVpZUnY2bnMyQzdjblVIRjlEckM2dkVqcnVrS1hkbVM4SmxTVno3aUhRbFl0?= =?utf-8?B?QVdWeUdiVW44M0xvTmFDcjFBN1pCYy9vM1dYNURvblhLbGFnZ0JTUzZnMExr?= =?utf-8?B?RXc9PQ==?= X-OriginatorOrg: corigine.com X-MS-Exchange-Transport-CrossTenantHeadersStamped: MW4PR13MB5940 X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Adds the vNIC initialization logic for the flower PF vNIC. The flower firmware exposes this vNIC for the purposes of fallback traffic in the switchdev use-case. Adds minimal dev_ops for this PF device. Because the device is being exposed externally to DPDK it should also be configured using DPDK helpers like rte_eth_configure(). For these helpers to work the flower logic needs to implements a minimal set of dev_ops. Signed-off-by: Chaoyong He Reviewed-by: Niklas Söderlund --- drivers/net/nfp/flower/nfp_flower.c | 398 ++++++++++++++++++++++++- drivers/net/nfp/flower/nfp_flower.h | 6 + drivers/net/nfp/flower/nfp_flower_ovs_compat.h | 37 +++ drivers/net/nfp/nfp_common.h | 3 + 4 files changed, 441 insertions(+), 3 deletions(-) create mode 100644 drivers/net/nfp/flower/nfp_flower_ovs_compat.h diff --git a/drivers/net/nfp/flower/nfp_flower.c b/drivers/net/nfp/flower/nfp_flower.c index e0506bd..ad0dff1 100644 --- a/drivers/net/nfp/flower/nfp_flower.c +++ b/drivers/net/nfp/flower/nfp_flower.c @@ -13,7 +13,353 @@ #include "../nfp_logs.h" #include "../nfp_ctrl.h" #include "../nfp_cpp_bridge.h" +#include "../nfp_rxtx.h" +#include "../nfpcore/nfp_mip.h" +#include "../nfpcore/nfp_rtsym.h" +#include "../nfpcore/nfp_nsp.h" #include "nfp_flower.h" +#include "nfp_flower_ovs_compat.h" + +#define MAX_PKT_BURST 32 +#define MEMPOOL_CACHE_SIZE 512 +#define DEFAULT_FLBUF_SIZE 9216 + +#define PF_VNIC_NB_DESC 1024 + +static const struct rte_eth_rxconf rx_conf = { + .rx_free_thresh = DEFAULT_RX_FREE_THRESH, + .rx_drop_en = 1, +}; + +static const struct rte_eth_txconf tx_conf = { + .tx_thresh = { + .pthresh = DEFAULT_TX_PTHRESH, + .hthresh = DEFAULT_TX_HTHRESH, + .wthresh = DEFAULT_TX_WTHRESH, + }, + .tx_free_thresh = DEFAULT_TX_FREE_THRESH, +}; + +static const struct eth_dev_ops nfp_flower_pf_vnic_ops = { + .dev_infos_get = nfp_net_infos_get, +}; + +static void +nfp_flower_pf_mp_init(__rte_unused struct rte_mempool *mp, + __rte_unused void *opaque_arg, + void *packet, + __rte_unused unsigned int i) +{ + struct dp_packet *pkt = packet; + pkt->source = DPBUF_DPDK; + pkt->l2_pad_size = 0; + pkt->l2_5_ofs = UINT16_MAX; + pkt->l3_ofs = UINT16_MAX; + pkt->l4_ofs = UINT16_MAX; + pkt->packet_type = 0; /* PT_ETH */ +} + +static struct rte_mempool * +nfp_flower_pf_mp_create(void) +{ + uint32_t nb_mbufs; + uint32_t pkt_size; + unsigned int numa_node; + uint32_t aligned_mbuf_size; + uint32_t mbuf_priv_data_len; + struct rte_mempool *pktmbuf_pool; + uint32_t n_rxd = PF_VNIC_NB_DESC; + uint32_t n_txd = PF_VNIC_NB_DESC; + + nb_mbufs = RTE_MAX(n_rxd + n_txd + MAX_PKT_BURST + MEMPOOL_CACHE_SIZE, 81920U); + + /* + * The size of the mbuf's private area (i.e. area that holds OvS' + * dp_packet data) + */ + mbuf_priv_data_len = sizeof(struct dp_packet) - sizeof(struct rte_mbuf); + /* The size of the entire dp_packet. */ + pkt_size = sizeof(struct dp_packet) + RTE_MBUF_DEFAULT_BUF_SIZE; + /* mbuf size, rounded up to cacheline size. */ + aligned_mbuf_size = RTE_CACHE_LINE_ROUNDUP(pkt_size); + mbuf_priv_data_len += (aligned_mbuf_size - pkt_size); + + numa_node = rte_socket_id(); + pktmbuf_pool = rte_pktmbuf_pool_create("flower_pf_mbuf_pool", nb_mbufs, + MEMPOOL_CACHE_SIZE, mbuf_priv_data_len, + RTE_MBUF_DEFAULT_BUF_SIZE, numa_node); + if (pktmbuf_pool == NULL) { + PMD_INIT_LOG(ERR, "Cannot init pf vnic mbuf pool"); + return NULL; + } + + rte_mempool_obj_iter(pktmbuf_pool, nfp_flower_pf_mp_init, NULL); + + return pktmbuf_pool; +} + +static int +nfp_flower_init_vnic_common(struct nfp_net_hw *hw, const char *vnic_type) +{ + uint32_t start_q; + uint64_t rx_bar_off; + uint64_t tx_bar_off; + const int stride = 4; + struct nfp_pf_dev *pf_dev; + struct rte_pci_device *pci_dev; + + pf_dev = hw->pf_dev; + pci_dev = hw->pf_dev->pci_dev; + + /* NFP can not handle DMA addresses requiring more than 40 bits */ + if (rte_mem_check_dma_mask(40)) { + PMD_INIT_LOG(ERR, "Device %s can not be used: restricted dma mask to 40 bits!\n", + pci_dev->device.name); + return -ENODEV; + }; + + hw->device_id = pci_dev->id.device_id; + hw->vendor_id = pci_dev->id.vendor_id; + hw->subsystem_device_id = pci_dev->id.subsystem_device_id; + hw->subsystem_vendor_id = pci_dev->id.subsystem_vendor_id; + + PMD_INIT_LOG(DEBUG, "%s vNIC ctrl bar: %p", vnic_type, hw->ctrl_bar); + + /* Read the number of available rx/tx queues from hardware */ + hw->max_rx_queues = nn_cfg_readl(hw, NFP_NET_CFG_MAX_RXRINGS); + hw->max_tx_queues = nn_cfg_readl(hw, NFP_NET_CFG_MAX_TXRINGS); + + /* Work out where in the BAR the queues start */ + start_q = nn_cfg_readl(hw, NFP_NET_CFG_START_TXQ); + tx_bar_off = (uint64_t)start_q * NFP_QCP_QUEUE_ADDR_SZ; + start_q = nn_cfg_readl(hw, NFP_NET_CFG_START_RXQ); + rx_bar_off = (uint64_t)start_q * NFP_QCP_QUEUE_ADDR_SZ; + + hw->tx_bar = pf_dev->hw_queues + tx_bar_off; + hw->rx_bar = pf_dev->hw_queues + rx_bar_off; + + /* Get some of the read-only fields from the config BAR */ + hw->ver = nn_cfg_readl(hw, NFP_NET_CFG_VERSION); + hw->cap = nn_cfg_readl(hw, NFP_NET_CFG_CAP); + hw->max_mtu = nn_cfg_readl(hw, NFP_NET_CFG_MAX_MTU); + /* Set the current MTU to the maximum supported */ + hw->mtu = hw->max_mtu; + hw->flbufsz = DEFAULT_FLBUF_SIZE; + + /* read the Rx offset configured from firmware */ + if (NFD_CFG_MAJOR_VERSION_of(hw->ver) < 2) + hw->rx_offset = NFP_NET_RX_OFFSET; + else + hw->rx_offset = nn_cfg_readl(hw, NFP_NET_CFG_RX_OFFSET_ADDR); + + hw->ctrl = 0; + hw->stride_rx = stride; + hw->stride_tx = stride; + + /* Reuse cfg queue setup function */ + nfp_net_cfg_queue_setup(hw); + + PMD_INIT_LOG(INFO, "%s vNIC max_rx_queues: %u, max_tx_queues: %u", + vnic_type, hw->max_rx_queues, hw->max_tx_queues); + + /* Initializing spinlock for reconfigs */ + rte_spinlock_init(&hw->reconfig_lock); + + return 0; +} + +static int +nfp_flower_init_pf_vnic(struct nfp_net_hw *hw) +{ + int ret; + uint16_t i; + uint16_t n_txq; + uint16_t n_rxq; + uint16_t port_id; + unsigned int numa_node; + struct rte_mempool *mp; + struct nfp_pf_dev *pf_dev; + struct rte_eth_dev *eth_dev; + struct nfp_app_fw_flower *app_fw_flower; + + static const struct rte_eth_conf port_conf = { + .rxmode = { + .mq_mode = RTE_ETH_MQ_RX_RSS, + .offloads = RTE_ETH_RX_OFFLOAD_CHECKSUM, + }, + .txmode = { + .mq_mode = RTE_ETH_MQ_TX_NONE, + }, + }; + + /* Set up some pointers here for ease of use */ + pf_dev = hw->pf_dev; + app_fw_flower = NFP_PRIV_TO_APP_FW_FLOWER(pf_dev->app_fw_priv); + + /* + * Perform the "common" part of setting up a flower vNIC. + * Mostly reading configuration from hardware. + */ + ret = nfp_flower_init_vnic_common(hw, "pf_vnic"); + if (ret != 0) + goto done; + + hw->eth_dev = rte_eth_dev_allocate("nfp_pf_vnic"); + if (hw->eth_dev == NULL) { + ret = -ENOMEM; + goto done; + } + + /* Grab the pointer to the newly created rte_eth_dev here */ + eth_dev = hw->eth_dev; + + numa_node = rte_socket_id(); + + /* Fill in some of the eth_dev fields */ + eth_dev->device = &pf_dev->pci_dev->device; + eth_dev->data->dev_private = hw; + + /* Create a mbuf pool for the PF */ + app_fw_flower->pf_pktmbuf_pool = nfp_flower_pf_mp_create(); + if (app_fw_flower->pf_pktmbuf_pool == NULL) { + ret = -ENOMEM; + goto port_release; + } + + mp = app_fw_flower->pf_pktmbuf_pool; + + /* Add Rx/Tx functions */ + eth_dev->dev_ops = &nfp_flower_pf_vnic_ops; + + /* PF vNIC gets a random MAC */ + eth_dev->data->mac_addrs = rte_zmalloc("mac_addr", RTE_ETHER_ADDR_LEN, 0); + if (eth_dev->data->mac_addrs == NULL) { + ret = -ENOMEM; + goto mempool_cleanup; + } + + rte_eth_random_addr(eth_dev->data->mac_addrs->addr_bytes); + rte_eth_dev_probing_finish(eth_dev); + + /* Configure the PF device now */ + n_rxq = hw->max_rx_queues; + n_txq = hw->max_tx_queues; + port_id = hw->eth_dev->data->port_id; + + ret = rte_eth_dev_configure(port_id, n_rxq, n_txq, &port_conf); + if (ret != 0) { + PMD_INIT_LOG(ERR, "Could not configure PF device %d", ret); + goto mac_cleanup; + } + + /* Set up the Rx queues */ + for (i = 0; i < n_rxq; i++) { + ret = nfp_net_rx_queue_setup(eth_dev, i, PF_VNIC_NB_DESC, numa_node, + &rx_conf, mp); + if (ret) { + PMD_INIT_LOG(ERR, "Configure flower PF vNIC Rx queue %d failed", i); + goto rx_queue_cleanup; + } + } + + /* Set up the Tx queues */ + for (i = 0; i < n_txq; i++) { + ret = nfp_net_nfd3_tx_queue_setup(eth_dev, i, PF_VNIC_NB_DESC, numa_node, + &tx_conf); + if (ret) { + PMD_INIT_LOG(ERR, "Configure flower PF vNIC Tx queue %d failed", i); + goto tx_queue_cleanup; + } + } + + return 0; + +tx_queue_cleanup: + for (i = 0; i < n_txq; i++) + nfp_net_tx_queue_release(eth_dev, i); +rx_queue_cleanup: + for (i = 0; i < n_rxq; i++) + nfp_net_rx_queue_release(eth_dev, i); +mac_cleanup: + rte_free(eth_dev->data->mac_addrs); +mempool_cleanup: + rte_mempool_free(mp); +port_release: + rte_eth_dev_release_port(hw->eth_dev); +done: + return ret; +} + +static void +nfp_flower_cleanup_pf_vnic(struct nfp_net_hw *hw) +{ + uint16_t i; + struct nfp_app_fw_flower *app_fw_flower; + + app_fw_flower = NFP_PRIV_TO_APP_FW_FLOWER(hw->pf_dev->app_fw_priv); + + for (i = 0; i < hw->max_tx_queues; i++) + nfp_net_tx_queue_release(hw->eth_dev, i); + + for (i = 0; i < hw->max_tx_queues; i++) + nfp_net_rx_queue_release(hw->eth_dev, i); + + rte_free(hw->eth_dev->data->mac_addrs); + rte_mempool_free(app_fw_flower->pf_pktmbuf_pool); + rte_eth_dev_release_port(hw->eth_dev); +} + +static int +nfp_flower_pf_vnic_service(void *arg) +{ + int ret; + struct nfp_app_fw_flower *app_fw_flower = arg; + + ret = nfp_flower_init_pf_vnic(app_fw_flower->pf_hw); + if (ret != 0) { + PMD_DRV_LOG(ERR, "Could not initialize flower PF vNIC"); + goto pf_vnic_cleanup; + } + + return 0; + +pf_vnic_cleanup: + nfp_flower_cleanup_pf_vnic(app_fw_flower->pf_hw); + + return ret; +} + +static int +nfp_flower_enable_pf_vnic_service(struct nfp_app_fw_flower *app_fw_flower) +{ + int ret; + uint32_t service_id; + struct rte_service_spec pf_vnic_service = { + .name = "flower_pf_vnic_service", + .callback = nfp_flower_pf_vnic_service, + }; + + /* Pass a pointer to the flower app to the service */ + pf_vnic_service.callback_userdata = (void *)app_fw_flower; + + /* Register the flower services */ + ret = rte_service_component_register(&pf_vnic_service, &service_id); + if (ret != 0) { + PMD_INIT_LOG(ERR, "Could not register flower pf vnic service"); + return -EINVAL; + } + + PMD_INIT_LOG(INFO, "Flower pf vnic service registered"); + + /* Map them to available service cores*/ + ret = nfp_map_service(service_id); + if (ret != 0) { + PMD_INIT_LOG(ERR, "Could not map flower pf vnic service"); + return -EINVAL; + } + + return 0; +} int nfp_init_app_fw_flower(struct nfp_pf_dev *pf_dev) @@ -43,8 +389,40 @@ goto app_cleanup; } + /* Grab the number of physical ports present on hardware */ + app_fw_flower->nfp_eth_table = nfp_eth_read_ports(pf_dev->cpp); + if (app_fw_flower->nfp_eth_table == NULL) { + PMD_INIT_LOG(ERR, "error reading nfp ethernet table"); + ret = -EIO; + goto vnic_cleanup; + } + + /* Map the PF ctrl bar */ + pf_dev->ctrl_bar = nfp_rtsym_map(pf_dev->sym_tbl, "_pf0_net_bar0", + 32768, &pf_dev->ctrl_area); + if (pf_dev->ctrl_bar == NULL) { + PMD_INIT_LOG(ERR, "Cloud not map the PF vNIC ctrl bar"); + ret = -ENODEV; + goto eth_tbl_cleanup; + } + + /* Fill in the PF vNIC and populate app struct */ + app_fw_flower->pf_hw = pf_hw; + pf_hw->ctrl_bar = pf_dev->ctrl_bar; + pf_hw->pf_dev = pf_dev; + pf_hw->cpp = pf_dev->cpp; + + /* Start up flower pf vnic service */ + ret = nfp_flower_enable_pf_vnic_service(app_fw_flower); + if (ret != 0) + PMD_INIT_LOG(WARNING, "Enable nfp pf vnic service failed."); + return 0; +eth_tbl_cleanup: + free(app_fw_flower->nfp_eth_table); +vnic_cleanup: + rte_free(pf_hw); app_cleanup: rte_free(app_fw_flower); done: @@ -52,8 +430,22 @@ } int -nfp_secondary_init_app_fw_flower(__rte_unused struct nfp_cpp *cpp) +nfp_secondary_init_app_fw_flower(struct nfp_cpp *cpp) { - PMD_INIT_LOG(ERR, "Flower firmware not supported"); - return -ENOTSUP; + struct rte_eth_dev *eth_dev; + const char *port_name = "pf_vnic_eth_dev"; + + PMD_INIT_LOG(DEBUG, "Secondary attaching to port %s", port_name); + + eth_dev = rte_eth_dev_attach_secondary(port_name); + if (eth_dev == NULL) { + PMD_INIT_LOG(ERR, "Secondary process attach to port %s failed", port_name); + return -ENODEV; + } + + eth_dev->process_private = cpp; + eth_dev->dev_ops = &nfp_flower_pf_vnic_ops; + rte_eth_dev_probing_finish(eth_dev); + + return 0; } diff --git a/drivers/net/nfp/flower/nfp_flower.h b/drivers/net/nfp/flower/nfp_flower.h index 51e05e8..981d88d 100644 --- a/drivers/net/nfp/flower/nfp_flower.h +++ b/drivers/net/nfp/flower/nfp_flower.h @@ -8,8 +8,14 @@ /* The flower application's private structure */ struct nfp_app_fw_flower { + /* Pointer to a mempool for the PF vNIC */ + struct rte_mempool *pf_pktmbuf_pool; + /* Pointer to the PF vNIC */ struct nfp_net_hw *pf_hw; + + /* the eth table as reported by firmware */ + struct nfp_eth_table *nfp_eth_table; }; int nfp_init_app_fw_flower(struct nfp_pf_dev *pf_dev); diff --git a/drivers/net/nfp/flower/nfp_flower_ovs_compat.h b/drivers/net/nfp/flower/nfp_flower_ovs_compat.h new file mode 100644 index 0000000..085de8a --- /dev/null +++ b/drivers/net/nfp/flower/nfp_flower_ovs_compat.h @@ -0,0 +1,37 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright (c) 2022 Corigine, Inc. + * All rights reserved. + */ + +#ifndef _NFP_FLOWER_OVS_COMPAT_H_ +#define _NFP_FLOWER_OVS_COMPAT_H_ + +enum dp_packet_source { + DPBUF_MALLOC, /* Obtained via malloc(). */ + DPBUF_STACK, /* Un-movable stack space or static buffer. */ + DPBUF_STUB, /* Starts on stack, may expand into heap. */ + DPBUF_DPDK, /* buffer data is from DPDK allocated memory. */ + DPBUF_AFXDP, /* Buffer data from XDP frame. */ +}; + +#define DP_PACKET_CONTEXT_SIZE 64 + +/* + * Buffer for holding packet data. A dp_packet is automatically reallocated + * as necessary if it grows too large for the available memory. + * By default the packet type is set to Ethernet (0). + */ +struct dp_packet { + struct rte_mbuf mbuf; /* DPDK mbuf */ + enum dp_packet_source source; /* Source of memory allocated as 'base'. */ + + uint16_t l2_pad_size; /* Detected l2 padding size. Padding is non-pullable. */ + uint16_t l2_5_ofs; /* MPLS label stack offset, or UINT16_MAX */ + uint16_t l3_ofs; /* Network-level header offset, or UINT16_MAX. */ + uint16_t l4_ofs; /* Transport-level header offset, or UINT16_MAX. */ + uint32_t cutlen; /* length in bytes to cut from the end. */ + uint32_t packet_type; /* Packet type as defined in OpenFlow */ + uint64_t data[DP_PACKET_CONTEXT_SIZE / 8]; +}; + +#endif /* _NFP_FLOWER_OVS_COMPAT_ */ diff --git a/drivers/net/nfp/nfp_common.h b/drivers/net/nfp/nfp_common.h index cefe717..aa6fdd4 100644 --- a/drivers/net/nfp/nfp_common.h +++ b/drivers/net/nfp/nfp_common.h @@ -446,6 +446,9 @@ int nfp_net_rss_hash_conf_get(struct rte_eth_dev *dev, #define NFP_PRIV_TO_APP_FW_NIC(app_fw_priv)\ ((struct nfp_app_fw_nic *)app_fw_priv) +#define NFP_PRIV_TO_APP_FW_FLOWER(app_fw_priv)\ + ((struct nfp_app_fw_flower *)app_fw_priv) + #endif /* _NFP_COMMON_H_ */ /* * Local variables: -- 1.8.3.1