From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from dpdk.org (dpdk.org [92.243.14.124]) by inbox.dpdk.org (Postfix) with ESMTP id 1C068A04F1; Wed, 11 Dec 2019 10:53:00 +0100 (CET) Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id 600981BF7F; Wed, 11 Dec 2019 10:52:15 +0100 (CET) Received: from EUR01-VE1-obe.outbound.protection.outlook.com (mail-eopbgr140118.outbound.protection.outlook.com [40.107.14.118]) by dpdk.org (Postfix) with ESMTP id C4DE71BF7B for ; Wed, 11 Dec 2019 10:52:13 +0100 (CET) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=ETUG6+4rxU6zI4tqkdTnXfhehLALO1n60PsFNFbD8ONWKaHODEYDvwYL4Pj7RqZhGQZ++9CMoyQjUcidy03hfB5Rh78QFZj+bif9+e+2H6N8/acHCInffgDC4gsZSH9n/jOcUOpw1oi65Lx9PP5DQd2JOIvHmLYVUmgXdCy/SQSJQ6Ua84N8Dms7lo3PtTXA9Qhfgh8R0avrgubYph6sW6bY2yrw+Kg3vJz0a+xWEVN5fshYoTpKryn+3I1iOIvW/WidNif8uAWN2w3xFJp7Mg7AIik+GYQua8YeZQpN637HFxA5h4l+DxD7hAL6vYhg0dD+Ry5wZpqWhqZ3YT5ySw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=ab8RVS9NWNovwuZAivcbTYcYGkM4OWmonAfZZogaKtA=; b=S9RgHzDpXq/vwDAi1FaBzI54DlQpndQFR/oQlWSVBRTNTi1n2r+G6F8WH+kWOuvXBqeYnytRYlL63hpUgC9ke9WpBng9wddf8EaYIeJvkrBeOc5geSyK/U7K9exYrWUTCTF++39PPxxtHoMQB6ujtcM9l/iI5ifHfV6DCF387gLwuapF6hBgWuUDcZgVv6f4QGluUKIyLyDE/2+S5V+x4Auwt9eG5reaEFSjBVAM/muYUGhuxHWSSR+pAgHEf1381fn8WZSoFWzi6JGjUaEJLiqglQ5a7lUoH4cAUBrtY9ctSGd9SyRST33RHiYay0laCL/UeOE2skgK6dpmdEN19g== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=silicom.co.il; dmarc=pass action=none header.from=silicom.co.il; dkim=pass header.d=silicom.co.il; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=SILICOMLTD.onmicrosoft.com; s=selector2-SILICOMLTD-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=ab8RVS9NWNovwuZAivcbTYcYGkM4OWmonAfZZogaKtA=; b=l4l1PVryz+PNOo2+QYVOWwl4earJLi4DbmDYHl7Zjx5j4rlKeaNlO2SRIyKVrPEPES/9JXA20xXBWEytCGN6K5hrXOHTEKN89sLervwwtO9YFNmAbZkNMMBgdw4+qGfsnnF4hSQPtBv5cWdmiZp40LvcmJQ0am4Ra9Qbpe+Ub5c= Received: from DB7PR04MB5196.eurprd04.prod.outlook.com (20.176.234.140) by DB7PR04MB4203.eurprd04.prod.outlook.com (52.135.131.12) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.2538.14; Wed, 11 Dec 2019 09:52:12 +0000 Received: from DB7PR04MB5196.eurprd04.prod.outlook.com ([fe80::cdaa:fcae:322b:59ed]) by DB7PR04MB5196.eurprd04.prod.outlook.com ([fe80::cdaa:fcae:322b:59ed%7]) with mapi id 15.20.2516.018; Wed, 11 Dec 2019 09:52:12 +0000 From: Xiaojun Liu To: "xiao.w.wang@intel.com" , "qi.z.zhang@intel.com" , "ngai-mint.kwan@intel.com" , "jakub.fornal@intel.co" , "jacob.e.keller@intel.com" CC: "dev@dpdk.org" , Xiaojun Liu Thread-Topic: [PATCH v2 5/7] net/fm10k: add switch initialization Thread-Index: AQHVsAipNkEFpzp5C0C2tvX5WIs5vQ== Date: Wed, 11 Dec 2019 09:52:12 +0000 Message-ID: <1576057875-7677-6-git-send-email-xiaojun.liu@silicom.co.il> References: <1576057875-7677-1-git-send-email-xiaojun.liu@silicom.co.il> In-Reply-To: <1576057875-7677-1-git-send-email-xiaojun.liu@silicom.co.il> Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-clientproxiedby: HK2PR02CA0180.apcprd02.prod.outlook.com (2603:1096:201:21::16) To DB7PR04MB5196.eurprd04.prod.outlook.com (2603:10a6:10:1a::12) authentication-results: spf=none (sender IP is ) smtp.mailfrom=xiaojun.liu@silicom.co.il; x-ms-exchange-messagesentrepresentingtype: 1 x-mailer: git-send-email 1.8.3.1 x-originating-ip: [113.110.226.253] x-ms-publictraffictype: Email x-ms-office365-filtering-correlation-id: b1e83beb-513d-422d-8f7c-08d77e1fcbdf x-ms-traffictypediagnostic: DB7PR04MB4203: x-ms-exchange-transport-forked: True x-microsoft-antispam-prvs: x-ms-oob-tlc-oobclassifiers: OLM:9508; x-forefront-prvs: 024847EE92 x-forefront-antispam-report: SFV:NSPM; SFS:(10019020)(346002)(366004)(136003)(39850400004)(376002)(396003)(199004)(189003)(2906002)(71200400001)(81156014)(64756008)(316002)(66556008)(6506007)(44832011)(8676002)(52116002)(81166006)(26005)(66476007)(54906003)(110136005)(66946007)(186003)(36756003)(6512007)(4326008)(86362001)(2616005)(107886003)(6486002)(66446008)(8936002)(5660300002)(478600001); DIR:OUT; SFP:1102; SCL:1; SRVR:DB7PR04MB4203; H:DB7PR04MB5196.eurprd04.prod.outlook.com; FPR:; SPF:None; LANG:en; PTR:InfoNoRecords; A:1; MX:1; received-spf: None (protection.outlook.com: silicom.co.il does not designate permitted sender hosts) x-ms-exchange-senderadcheck: 1 x-microsoft-antispam: BCL:0; x-microsoft-antispam-message-info: Nh1uptADYTAcqf2IVHyZ9SVuUsoABr/dspX6aqfoMpH8KbVmefAHs3Bmu22KC83/Qtcr1K1xPvNbixhhlDfRkLdrU3g9CdBpWvZ040zUpXwL1SeAJiLADITVtp70TNyC5P/mqhq5K1euk7IQS3G/dQOAzh/qiSj+ZnWt83O4UEiZpRir+dNdNtBEBhX6C7xSLOdRIE0oUxLrt211aV8Fm+ndGMWAFE38JeHBIQ7HGkvHx5jkXIhhsh3HtpW0H5X7acNp4qM/BTNA5IdS8XQq+ah2cnDjZ3N1KaJnL0TIyaNEkueXU2Wb4ePf6fIRqDl/8FVNRe+MCjkFcyJnuh8Hk9vuFRvuTmN6WTCX0mEVrvcm8gw62pcrDGSgKchFia7qiL5ohuJKzEIhjMz0yLsvanZLRlMLSM2L9tjgDW+NyN+Be/5LbnBbsGKue3RViFhS Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 X-OriginatorOrg: silicom.co.il X-MS-Exchange-CrossTenant-Network-Message-Id: b1e83beb-513d-422d-8f7c-08d77e1fcbdf X-MS-Exchange-CrossTenant-originalarrivaltime: 11 Dec 2019 09:52:12.7145 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Hosted X-MS-Exchange-CrossTenant-id: c9e326d8-ce47-4930-8612-cc99d3c87ad1 X-MS-Exchange-CrossTenant-mailboxtype: HOSTED X-MS-Exchange-CrossTenant-userprincipalname: klthDy7XfHu4Qxun0fH3vwcq7/5TgjyUzJRJqvpkhLMhjugnBwz4cyxDJ5e2rjvJ5Z9b4DNui3UulwkPI6qV8KhqkGLscEEJdxew0zRf3pk= X-MS-Exchange-Transport-CrossTenantHeadersStamped: DB7PR04MB4203 Subject: [dpdk-dev] [PATCH v2 5/7] net/fm10k: add switch initialization X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" Modify fm10k/fm10k_ethdev.c. Split dev init to 2 parts. First only register the port in switch management; second init hook will be called after all the pf are registered and switch initialization. It will finish dev init. Also add switch interrupt support. To avoid configuration for both kernel driver and userspace SDK outside DPDK, we add switch management in FM10K DPDK PMD driver. To enable switch management, you need add CONFIG_RTE_FM10K_MANAGEMENT=3Dy in config/common_linux when building. Signed-off-by: Xiaojun Liu --- drivers/net/fm10k/fm10k_ethdev.c | 182 +++++++++++++++++++++++++++++++++++= ++-- 1 file changed, 176 insertions(+), 6 deletions(-) diff --git a/drivers/net/fm10k/fm10k_ethdev.c b/drivers/net/fm10k/fm10k_eth= dev.c index 99c4366..4c81952 100644 --- a/drivers/net/fm10k/fm10k_ethdev.c +++ b/drivers/net/fm10k/fm10k_ethdev.c @@ -13,6 +13,10 @@ =20 #include "fm10k.h" #include "base/fm10k_api.h" +#ifdef ENABLE_FM10K_MANAGEMENT +#include "switch/fm10k_regs.h" +#include "switch/fm10k_switch.h" +#endif =20 /* Default delay to acquire mailbox lock */ #define FM10K_MBXLOCK_DELAY_US 20 @@ -39,6 +43,10 @@ #define GLORT_PF_MASK 0xFFC0 #define GLORT_FD_MASK GLORT_PF_MASK #define GLORT_FD_INDEX GLORT_FD_Q_BASE +#ifdef ENABLE_FM10K_MANAGEMENT +/* When the switch is ready, the status will be changed */ +static int fm10k_switch_ready; +#endif =20 int fm10k_logtype_init; int fm10k_logtype_driver; @@ -2588,6 +2596,9 @@ static uint64_t fm10k_get_tx_port_offloads_capa(struc= t rte_eth_dev *dev) FM10K_DEV_PRIVATE_TO_INFO(dev->data->dev_private); int status_mbx; s32 err; +#ifdef ENABLE_FM10K_MANAGEMENT + uint32_t writeback =3D 0; +#endif =20 if (hw->mac.type !=3D fm10k_mac_pf) return; @@ -2601,11 +2612,20 @@ static uint64_t fm10k_get_tx_port_offloads_capa(str= uct rte_eth_dev *dev) } =20 /* Handle switch up/down */ - if (cause & FM10K_EICR_SWITCHNOTREADY) - PMD_INIT_LOG(ERR, "INT: Switch is not ready"); + if (cause & FM10K_EICR_SWITCHNOTREADY) { + PMD_INIT_LOG(INFO, "INT: Switch is not ready"); +#ifdef ENABLE_FM10K_MANAGEMENT + fm10k_switch_ready =3D 0; + writeback |=3D FM10K_EICR_SWITCHNOTREADY; +#endif + } =20 if (cause & FM10K_EICR_SWITCHREADY) { PMD_INIT_LOG(INFO, "INT: Switch is ready"); +#ifdef ENABLE_FM10K_MANAGEMENT + fm10k_switch_ready =3D 1; + writeback |=3D FM10K_EICR_SWITCHREADY; +#endif if (dev_info->sm_down =3D=3D 1) { fm10k_mbx_lock(hw); =20 @@ -2656,6 +2676,7 @@ static uint64_t fm10k_get_tx_port_offloads_capa(struc= t rte_eth_dev *dev) } =20 /* Handle mailbox message */ +#ifndef ENABLE_FM10K_MANAGEMENT fm10k_mbx_lock(hw); err =3D hw->mbx.ops.process(hw, &hw->mbx); fm10k_mbx_unlock(hw); @@ -2663,10 +2684,33 @@ static uint64_t fm10k_get_tx_port_offloads_capa(str= uct rte_eth_dev *dev) if (err =3D=3D FM10K_ERR_RESET_REQUESTED) { PMD_INIT_LOG(INFO, "INT: Switch is down"); dev_info->sm_down =3D 1; - _rte_eth_dev_callback_process(dev, RTE_ETH_EVENT_INTR_LSC, + _rte_eth_dev_callback_process + (dev, + RTE_ETH_EVENT_INTR_LSC, NULL); } =20 +#else + if (cause & FM10K_EICR_MAILBOX) { + fm10k_mbx_lock(hw); + err =3D hw->mbx.ops.process(hw, &hw->mbx); + fm10k_mbx_unlock(hw); + writeback |=3D FM10K_EICR_MAILBOX; + if (err =3D=3D FM10K_ERR_RESET_REQUESTED) { + PMD_INIT_LOG(INFO, "INT: Switch is down"); + dev_info->sm_down =3D 1; + _rte_eth_dev_callback_process + (dev, + RTE_ETH_EVENT_INTR_LSC, + NULL); + } + } + + /* Handle switch interrupt */ + if (cause & FM10K_SW_EICR_SWITCH_INT) + fm10k_switch_intr(hw); +#endif + /* Handle SRAM error */ if (cause & FM10K_EICR_SRAMERROR) { PMD_INIT_LOG(ERR, "INT: SRAM error on PEP"); @@ -2678,15 +2722,27 @@ static uint64_t fm10k_get_tx_port_offloads_capa(str= uct rte_eth_dev *dev) /* Todo: print out error message after shared code updates */ } =20 +#ifndef ENABLE_FM10K_MANAGEMENT /* Clear these 3 events if having any */ cause &=3D FM10K_EICR_SWITCHNOTREADY | FM10K_EICR_MAILBOX | FM10K_EICR_SWITCHREADY; if (cause) FM10K_WRITE_REG(hw, FM10K_EICR, cause); +#else + if (writeback) + FM10K_WRITE_REG(hw, FM10K_EICR, writeback); +#endif =20 /* Re-enable interrupt from device side */ - FM10K_WRITE_REG(hw, FM10K_ITR(0), FM10K_ITR_AUTOMASK | +#ifndef ENABLE_FM10K_MANAGEMENT + FM10K_WRITE_REG(hw, FM10K_ITR(0), + FM10K_ITR_AUTOMASK | FM10K_ITR_MASK_CLEAR); +#else + FM10K_WRITE_REG(hw, FM10K_ITR(0), + FM10K_SW_ITR_AUTO_MASK | + FM10K_SW_MAKE_REG_FIELD(ITR_MASK, FM10K_SW_ITR_MASK_W_ENABLE)); +#endif /* Re-enable interrupt from host side */ rte_intr_ack(dev->intr_handle); } @@ -3071,13 +3127,87 @@ static void __attribute__((cold)) info->sm_down =3D false; } =20 +#ifdef ENABLE_FM10K_MANAGEMENT +static int eth_fm10k_dev_init_hook(struct fm10k_hw *hw) +{ + int i, switch_ready; + struct rte_eth_dev *dev =3D + (struct rte_eth_dev *)fm10k_switch_dpdk_port_rte_dev_get(hw); + + /* Make sure Switch Manager is ready before going forward. */ + if (hw->mac.type =3D=3D fm10k_mac_pf) { + switch_ready =3D 0; + + for (i =3D 0; i < MAX_QUERY_SWITCH_STATE_TIMES; i++) { + fm10k_mbx_lock(hw); + switch_ready =3D fm10k_switch_ready; + fm10k_mbx_unlock(hw); + if (switch_ready) + break; + /* Delay some time to acquire async LPORT_MAP info. */ + rte_delay_us(WAIT_SWITCH_MSG_US); + } + + if (switch_ready =3D=3D 0) { + PMD_INIT_LOG(ERR, "switch is not ready"); + return -1; + } + } + + /* + * Below function will trigger operations on mailbox, acquire lock to + * avoid race condition from interrupt handler. Operations on mailbox + * FIFO will trigger interrupt to PF/SM, in which interrupt handler + * will handle and generate an interrupt to our side. Then, FIFO in + * mailbox will be touched. + */ + if (hw->mac.dglort_map =3D=3D FM10K_DGLORTMAP_NONE) { + PMD_INIT_LOG(ERR, "dglort_map is not ready"); + return -1; + } + + fm10k_mbx_lock(hw); + /* Enable port first */ + hw->mac.ops.update_lport_state(hw, hw->mac.dglort_map, + MAX_LPORT_NUM, 1); + /* Set unicast mode by default. App can change to other mode in other + * API func. + */ + hw->mac.ops.update_xcast_mode(hw, hw->mac.dglort_map, + FM10K_XCAST_MODE_NONE); + fm10k_mbx_unlock(hw); + + /* Make sure default VID is ready before going forward. */ + if (hw->mac.type =3D=3D fm10k_mac_pf) { + for (i =3D 0; i < MAX_QUERY_SWITCH_STATE_TIMES; i++) { + if (hw->mac.default_vid) + break; + /* Delay some time to acquire async port VLAN info. */ + rte_delay_us(WAIT_SWITCH_MSG_US); + } + + if (hw->mac.default_vid =3D=3D 0) + hw->mac.default_vid =3D 1; + } + + /* Add default mac address */ + fm10k_MAC_filter_set(dev, hw->mac.addr, true, + MAIN_VSI_POOL_NUMBER); + + return 0; +} +#endif + static int eth_fm10k_dev_init(struct rte_eth_dev *dev) { struct fm10k_hw *hw =3D FM10K_DEV_PRIVATE_TO_HW(dev->data->dev_private); struct rte_pci_device *pdev =3D RTE_ETH_DEV_TO_PCI(dev); struct rte_intr_handle *intr_handle =3D &pdev->intr_handle; - int diag, i; + int diag; +#ifndef ENABLE_FM10K_MANAGEMENT + int i; +#endif struct fm10k_macvlan_filter_info *macvlan; =20 PMD_INIT_FUNC_TRACE(); @@ -3114,7 +3244,9 @@ static void __attribute__((cold)) " Try to blacklist unused devices."); return -EIO; } - +#ifdef ENABLE_FM10K_MANAGEMENT + hw->sw_addr =3D (void *)pdev->mem_resource[4].addr; +#endif /* Store fm10k_adapter pointer */ hw->back =3D dev->data->dev_private; =20 @@ -3125,6 +3257,25 @@ static void __attribute__((cold)) return -EIO; } =20 +#ifdef ENABLE_FM10K_MANAGEMENT + if (hw->mac.type =3D=3D fm10k_mac_pf) { + if (hw->hw_addr =3D=3D NULL || hw->sw_addr =3D=3D NULL) { + PMD_INIT_LOG(ERR, "Bad mem resource." + " Try to blacklist unused devices."); + return -EIO; + } + } else { + if (hw->hw_addr =3D=3D NULL) { + PMD_INIT_LOG(ERR, "Bad mem resource." + " Try to blacklist unused devices."); + return -EIO; + } + } + + /* Store fm10k_adapter pointer */ + hw->back =3D dev->data->dev_private; +#endif + /* Initialize parameters */ fm10k_params_init(dev); =20 @@ -3205,6 +3356,7 @@ static void __attribute__((cold)) hw->mac.ops.update_int_moderator(hw); =20 /* Make sure Switch Manager is ready before going forward. */ +#ifndef ENABLE_FM10K_MANAGEMENT if (hw->mac.type =3D=3D fm10k_mac_pf) { int switch_ready =3D 0; =20 @@ -3264,11 +3416,25 @@ static void __attribute__((cold)) MAIN_VSI_POOL_NUMBER); =20 return 0; +#else + if (hw->mac.type =3D=3D fm10k_mac_pf) { + bool master =3D FM10K_READ_REG(hw, + FM10K_CTRL) & FM10K_CTRL_BAR4_ALLOWED; + return fm10k_switch_dpdk_port_start(hw, + dev, 1, master, eth_fm10k_dev_init_hook); + } else { /* It may not work for VF */ + return fm10k_switch_dpdk_port_start(hw, + dev, 0, false, eth_fm10k_dev_init_hook); + } +#endif } =20 static int eth_fm10k_dev_uninit(struct rte_eth_dev *dev) { +#ifdef ENABLE_FM10K_MANAGEMENT + struct fm10k_hw *hw =3D FM10K_DEV_PRIVATE_TO_HW(dev->data->dev_private); +#endif PMD_INIT_FUNC_TRACE(); =20 /* only uninitialize in the primary process */ @@ -3278,6 +3444,10 @@ static void __attribute__((cold)) /* safe to close dev here */ fm10k_dev_close(dev); =20 +#ifdef ENABLE_FM10K_MANAGEMENT + fm10k_switch_dpdk_port_stop(hw); +#endif + return 0; } =20 --=20 1.8.3.1