From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 43738A0C48; Thu, 8 Jul 2021 16:11:24 +0200 (CEST) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id BCBCA40696; Thu, 8 Jul 2021 16:11:23 +0200 (CEST) Received: from NAM02-DM3-obe.outbound.protection.outlook.com (mail-dm3nam07on2089.outbound.protection.outlook.com [40.107.95.89]) by mails.dpdk.org (Postfix) with ESMTP id 43F214014F for ; Thu, 8 Jul 2021 16:11:22 +0200 (CEST) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=lJUlGkQIId8A35lNcmsGsEYlZI+dsVDDhgP9LznXvfrQ3Fu8RBwGhwHrm24zX+ifOL3EWsHpQcfPkTSezaEPrzYpkIuNuuLka6TtltA6skp5tPaiQ6NXB7SJ2a+wHsiwo4iVJwBdmfYSs4vEpP3pAjK67RTd04DG9I2dRUbeKG/CraLTsQerFUD1YZrZA56zyc2RUMJbiSSdrz2XPax0N+tSdau/g+3TtS2LpHHSFX9rkwvUQFukjNit+I9AHLW5jQ7Hi9cGQB6OrvmKvkRl81FVyiV/fuP7LsOO0agOz9+bvk9GlO/qhjWG7V0ClruF7/T8/oe/2VYqIqRT/G/Bgg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=cRpoBqI/QJyknV14o5lBq6yFgvY01KInOa8v4fTs2W0=; b=Ee2sMDhgW+TJjkklCZG0UQqJ0AYSaJRmYlkyjnCNeR6u85z8qbPPSu9QNYzW4dJDtdkcXqGR8BQf8OQKrHqNpjOHlRXXj3mCOiEVMiUCtzSfSCAKXnb+Smw20LmS4vItR/8jqmvmuG59bMRsIWPkxrsHHzLZX6AQmm6i182ngB19/thx1Fbe7A1cGuPmi6Kvb0AqJ7W+5oYx5xvU0OxLpck9SrnUgLzKPeKurY8+ciye/3OhkMbpuuEClcBMxKoWvhbEYB6uTudprkfSlqjRvhOiytPPpc7liaNS26u/hnoZRnzGf0d0/MK1ZILRwCWiaYhXseuEHpLcKInicNVLlg== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=vmware.com; dmarc=pass action=none header.from=vmware.com; dkim=pass header.d=vmware.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=vmware.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=cRpoBqI/QJyknV14o5lBq6yFgvY01KInOa8v4fTs2W0=; b=WCd716tCByDo96aZ0yDtFGU7UKKoWU8j+c3aBtfXxjT9TuPad7uhe3S2pkSxj/o56OLagktcpR044D1ryfLkDlKC5lTxepWoSUPKsPMU6GYH0DrO230dNm9PIi6jZ6dpldBFynWrCP127hyxTBhaCU8LJgDouM76aaxuKFZac8k= Authentication-Results: vmware.com; dkim=none (message not signed) header.d=none;vmware.com; dmarc=none action=none header.from=vmware.com; Received: from BYAPR05MB5911.namprd05.prod.outlook.com (2603:10b6:a03:cc::28) by BYAPR05MB4245.namprd05.prod.outlook.com (2603:10b6:a03::26) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4331.11; Thu, 8 Jul 2021 14:11:19 +0000 Received: from BYAPR05MB5911.namprd05.prod.outlook.com ([fe80::4c4:e666:46fb:31d]) by BYAPR05MB5911.namprd05.prod.outlook.com ([fe80::4c4:e666:46fb:31d%7]) with mapi id 15.20.4331.010; Thu, 8 Jul 2021 14:11:19 +0000 From: Jochen Behrens To: yongwang@vmware.com Cc: dev@dpdk.org, jbehrens@vmware.com Date: Thu, 8 Jul 2021 07:02:25 -0700 Message-Id: <20210708140225.583-1-jbehrens@vmware.com> X-Mailer: git-send-email 2.17.1 Content-Type: text/plain X-ClientProxiedBy: SJ0PR05CA0185.namprd05.prod.outlook.com (2603:10b6:a03:330::10) To BYAPR05MB5911.namprd05.prod.outlook.com (2603:10b6:a03:cc::28) MIME-Version: 1.0 X-MS-Exchange-MessageSentRepresentingType: 1 Received: from jb-bio-46.eng.vmware.com (66.170.99.2) by SJ0PR05CA0185.namprd05.prod.outlook.com (2603:10b6:a03:330::10) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4331.12 via Frontend Transport; Thu, 8 Jul 2021 14:11:19 +0000 X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: 8b8ccfac-bc80-4415-ca5a-08d9421a4214 X-MS-TrafficTypeDiagnostic: BYAPR05MB4245: X-MS-Exchange-Transport-Forked: True X-Microsoft-Antispam-PRVS: X-MS-Oob-TLC-OOBClassifiers: OLM:10000; X-MS-Exchange-SenderADCheck: 1 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: TbJp3JXzFUE9wA+lcc+4glPhj0cIM8y/eDSgT9QJcjPScXtJgdDS5+k/txffWMxViztSWmqQKuZlAqeD8qA1c0lQIwtUrxHH3036enx/jzWt483DjoYujbaMTWgIatjkZlSYgFKibF8UF4f6INoWzl2fyvsCTzEk0POMAdTnfQWU3nRHPNCN+Cbppxal3Wq+23WhUDzgtH3BZQN1PrTMt9leuejIQTlJZNg0+4Jnl7LHk7qUuOeq3G4U41weO/zWorOHrtBDmiEe18ib7YsiVzKOZYJYkhJGGdjkWB61YNdB95HQWXX1SwAlkW/iAfmSbk/zR9tWM9ybbZULTgAx+P9SYpDhgqY3vJBBFmKim1u2OLV+UTQyXJRO2u8Gsr6j33RusnBOuFSO/okSlIGiJPABAA5oW7f9t906lUOemENaSp9hBtiOYExzVrhpqYakCp3jZhSdY48ipKS+jGIBAelK+DvUfSSpiqgfbmZK8G0lTRPgiWKZj85gH3dejE8QT7j3ff3k1HRcG71eT7O/EzcthQyEpqpLn308LWh2fHNusi6aU9XGTF3GIWjjui/8iOVDbLgAkH5jsR72wWx2MGLc8WMJAkw/GWIoPPWbf5vFV/JvpVdUu5aQRBDzWJ4/6xgmhdHpzN4eNmdAavk7IiUtjwZxmJA42RToB8Ck9sZiL5vDbzkQ4m+pNF4REqQyVpcDyTq8O7UUzfWdCmXTz4c9lPMrMqz2j5F3EdD1cdM= X-Forefront-Antispam-Report: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:BYAPR05MB5911.namprd05.prod.outlook.com; PTR:; CAT:NONE; SFS:(4636009)(39860400002)(396003)(376002)(136003)(366004)(346002)(8936002)(8676002)(6486002)(86362001)(83380400001)(2616005)(6666004)(2906002)(316002)(6636002)(38100700002)(66556008)(30864003)(38350700002)(34206002)(7696005)(19627235002)(4326008)(107886003)(52116002)(478600001)(36756003)(37006003)(956004)(1076003)(186003)(66476007)(5660300002)(66946007)(26005)(21314003); DIR:OUT; SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: =?us-ascii?Q?ThL802bZXzTOl09REoK1vDMNkTiNUCqRdH+biw4zOBPc6nf50zUpMZ2RqHYE?= =?us-ascii?Q?1hBqlF6dL+7WHj6/NLe/EUjhfMKLwSq7seLR1HF8pZkG+v099hXNUP5sIxoF?= =?us-ascii?Q?1qA14rKWpOqeyXjlDVDJVr52FuQACQ5IREH6WxM17P+N39xJmuqNmvojD7aQ?= =?us-ascii?Q?Z3M4ffNkOowmMjiTb3QRlsPa1WkjBjrwxXt0OBinvHlyaf38m2JT2cjk8Ylk?= =?us-ascii?Q?1dm5XC3NKIFkzjaqznYXlEymCUnsshWIWnxIgdpuF2FeZqj+q+fDLpJHYelP?= =?us-ascii?Q?yUWeBpd/CcuNsN+aQMjnyCi5hS/shWlVO3hYt7Zmw+PhIW/la5ATmaHfV+8y?= =?us-ascii?Q?PG+Tx6gp+TbV+kgr16Euw1lYenF2ErnZp2Y5Hz3dMg/mq9OB8Src0RBi/A1y?= =?us-ascii?Q?EXNyQ45bWzAwXonGxHUY5FDDMJa9dm3ghfIJuDbNE3W4YN5yKZx3bMgPDXkB?= =?us-ascii?Q?HqwdFQt8BzwMGeIG4I5yGI1pDdtgB1kUp93KZQEH3/inzCFrTTiO/4b0Qn6B?= =?us-ascii?Q?Zd6YbWEXizNgZdWOE4N5uhib2Tnx+JxT1CewTQFtmVSFiNDGu9mqPMR+Iuqz?= =?us-ascii?Q?uXz9kLtvBmNY27wlMB/EV0D6m/y9rEp7TYoYWyT7E70Fl1Tdo7dh1f5/nlQ2?= =?us-ascii?Q?0BVEg4kSYhoY5lpWHsFLioY+v5zIDy9iwyj2v1Oxz7+VuvvxLKYNVWE7ALQS?= =?us-ascii?Q?Nx291MPsbfGSWrM9HjO+hFS2rfS8DYUNSpNOn79BiBWSac4W3tmc0WgWChro?= =?us-ascii?Q?Aw9hSzvscInstBBYM6EwuOU+/Fu17i9wMbFHpZM0jAiMSEswF1tmQekzvMuP?= =?us-ascii?Q?G4/4iHkwPwKIX0UtyxtxXsKDte9BNRrRq81eX0NtN2jdIVXjsNpiuU/NycIM?= =?us-ascii?Q?39laBuxwHCUGM+9yrc6cj31ZlCPt3XtRGUv3M1SvLsJIEUexkE5iZWvJUn2F?= =?us-ascii?Q?NRmh4XilBZGKcCeo7Jvw4UHgkq68rA492qNTla0zp0/d03kJoUEj65DkzOW8?= =?us-ascii?Q?dPQmIGFmS89pgrtBkI6rykuqQQuEWqTbQwQ/SBKDAeN6fpEQdeFTao4RGElc?= =?us-ascii?Q?Rearo9DzVHhgG+y0TvFtm1c18ggfyXVlOeRxNPZl7zkJcs+t+rvgHg2S/UOw?= =?us-ascii?Q?vPsuB27m06T8FWk3hLaEYL4+jd9L0n7ZXOVMlVi6sTlJwgKkvNL0gxGpzR+k?= =?us-ascii?Q?3yAYTPOUFFxVJs7a7pGo4FiIT0CKm1pugU6+c+blNxyCGxm5JU0i47nzxAW+?= =?us-ascii?Q?O3mvzDWXbjIVxt6s1XeMzungc+WzXbCavSKW3Asazj2GMMRtR1l0o5ON6sul?= =?us-ascii?Q?jlFcjeKCcAg+6lPva2U0mEjE?= X-OriginatorOrg: vmware.com X-MS-Exchange-CrossTenant-Network-Message-Id: 8b8ccfac-bc80-4415-ca5a-08d9421a4214 X-MS-Exchange-CrossTenant-AuthSource: BYAPR05MB5911.namprd05.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 08 Jul 2021 14:11:19.6296 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: b39138ca-3cee-4b4a-a4d6-cd83d9dd62f0 X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: sW22vVb49+7HQKi/VmhRq6oVUKmZps/bdYAlwfY617gufW9+WIrKnzCtWKZLjmPX/1m3KOOKeLKD9woVXWCgcA== X-MS-Exchange-Transport-CrossTenantHeadersStamped: BYAPR05MB4245 Subject: [dpdk-dev] [PATCH v2] net/vmxnet3: add interrupt support in MSI-X X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" Add support for MSI-X interrupt vectors to the vmxnet3 driver. This will allow more efficient deployments in cloud environments. By default it will try to allocate 1 vector (0) for link event and one MSI-X vector for each Rx queue. To simplify things, it will only be enabled if the number of Tx and Rx queues are equal (so that Tx/Rx share the same vector). If for any reason vmxnet3 cannot enable intr mode, it will fall back to the LSC only mode. v2: addressed formatting nits in comments Signed-off-by: Yong Wang Signed-off-by: Jochen Behrens --- drivers/net/vmxnet3/vmxnet3_ethdev.c | 340 +++++++++++++++++++++------ drivers/net/vmxnet3/vmxnet3_ethdev.h | 12 + 2 files changed, 284 insertions(+), 68 deletions(-) diff --git a/drivers/net/vmxnet3/vmxnet3_ethdev.c b/drivers/net/vmxnet3/vmxnet3_ethdev.c index 5bffbb8a0e..1a3291273a 100644 --- a/drivers/net/vmxnet3/vmxnet3_ethdev.c +++ b/drivers/net/vmxnet3/vmxnet3_ethdev.c @@ -38,8 +38,6 @@ #include "vmxnet3_logs.h" #include "vmxnet3_ethdev.h" -#define PROCESS_SYS_EVENTS 0 - #define VMXNET3_TX_MAX_SEG UINT8_MAX #define VMXNET3_TX_OFFLOAD_CAP \ @@ -96,7 +94,12 @@ static int vmxnet3_dev_vlan_filter_set(struct rte_eth_dev *dev, static int vmxnet3_dev_vlan_offload_set(struct rte_eth_dev *dev, int mask); static int vmxnet3_mac_addr_set(struct rte_eth_dev *dev, struct rte_ether_addr *mac_addr); +static void vmxnet3_process_events(struct rte_eth_dev *dev); static void vmxnet3_interrupt_handler(void *param); +static int vmxnet3_dev_rx_queue_intr_enable(struct rte_eth_dev *dev, + uint16_t queue_id); +static int vmxnet3_dev_rx_queue_intr_disable(struct rte_eth_dev *dev, + uint16_t queue_id); /* * The set of PCI devices this driver supports @@ -133,6 +136,8 @@ static const struct eth_dev_ops vmxnet3_eth_dev_ops = { .rx_queue_release = vmxnet3_dev_rx_queue_release, .tx_queue_setup = vmxnet3_dev_tx_queue_setup, .tx_queue_release = vmxnet3_dev_tx_queue_release, + .rx_queue_intr_enable = vmxnet3_dev_rx_queue_intr_enable, + .rx_queue_intr_disable = vmxnet3_dev_rx_queue_intr_disable, }; struct vmxnet3_xstats_name_off { @@ -183,30 +188,61 @@ gpa_zone_reserve(struct rte_eth_dev *dev, uint32_t size, } /* - * This function is based on vmxnet3_disable_intr() + * Enable the given interrupt */ static void -vmxnet3_disable_intr(struct vmxnet3_hw *hw) +vmxnet3_enable_intr(struct vmxnet3_hw *hw, unsigned int intr_idx) { - int i; + PMD_INIT_FUNC_TRACE(); + VMXNET3_WRITE_BAR0_REG(hw, VMXNET3_REG_IMR + intr_idx * 8, 0); +} +/* + * Disable the given interrupt + */ +static void +vmxnet3_disable_intr(struct vmxnet3_hw *hw, unsigned int intr_idx) +{ PMD_INIT_FUNC_TRACE(); + VMXNET3_WRITE_BAR0_REG(hw, VMXNET3_REG_IMR + intr_idx * 8, 1); +} - hw->shared->devRead.intrConf.intrCtrl |= VMXNET3_IC_DISABLE_ALL; - for (i = 0; i < hw->num_intrs; i++) - VMXNET3_WRITE_BAR0_REG(hw, VMXNET3_REG_IMR + i * 8, 1); +/* + * Enable all intrs used by the device + */ +static void +vmxnet3_enable_all_intrs(struct vmxnet3_hw *hw) +{ + Vmxnet3_DSDevRead *devRead = &hw->shared->devRead; + + PMD_INIT_FUNC_TRACE(); + + devRead->intrConf.intrCtrl &= rte_cpu_to_le_32(~VMXNET3_IC_DISABLE_ALL); + + if (hw->intr.lsc_only) { + vmxnet3_enable_intr(hw, devRead->intrConf.eventIntrIdx); + } else { + int i; + + for (i = 0; i < hw->intr.num_intrs; i++) + vmxnet3_enable_intr(hw, i); + } } +/* + * Disable all intrs used by the device + */ static void -vmxnet3_enable_intr(struct vmxnet3_hw *hw) +vmxnet3_disable_all_intrs(struct vmxnet3_hw *hw) { int i; PMD_INIT_FUNC_TRACE(); - hw->shared->devRead.intrConf.intrCtrl &= ~VMXNET3_IC_DISABLE_ALL; + hw->shared->devRead.intrConf.intrCtrl |= + rte_cpu_to_le_32(VMXNET3_IC_DISABLE_ALL); for (i = 0; i < hw->num_intrs; i++) - VMXNET3_WRITE_BAR0_REG(hw, VMXNET3_REG_IMR + i * 8, 0); + vmxnet3_disable_intr(hw, i); } /* @@ -304,7 +340,7 @@ eth_vmxnet3_dev_init(struct rte_eth_dev *eth_dev) return -EIO; } - PMD_INIT_LOG(DEBUG, "Using device version %d\n", hw->version); + PMD_INIT_LOG(INFO, "Using device v%d", hw->version); /* Check UPT version compatibility with driver. */ ver = VMXNET3_READ_BAR1_REG(hw, VMXNET3_REG_UVRS); @@ -408,6 +444,40 @@ static struct rte_pci_driver rte_vmxnet3_pmd = { .remove = eth_vmxnet3_pci_remove, }; +static void +vmxnet3_alloc_intr_resources(struct rte_eth_dev *dev) +{ + struct vmxnet3_hw *hw = dev->data->dev_private; + uint32_t cfg; + int nvec = 1; /* for link event */ + + /* intr settings */ + VMXNET3_WRITE_BAR1_REG(hw, VMXNET3_REG_CMD, + VMXNET3_CMD_GET_CONF_INTR); + cfg = VMXNET3_READ_BAR1_REG(hw, VMXNET3_REG_CMD); + hw->intr.type = cfg & 0x3; + hw->intr.mask_mode = (cfg >> 2) & 0x3; + + if (hw->intr.type == VMXNET3_IT_AUTO) + hw->intr.type = VMXNET3_IT_MSIX; + + if (hw->intr.type == VMXNET3_IT_MSIX) { + /* only support shared tx/rx intr */ + if (hw->num_tx_queues != hw->num_rx_queues) + goto msix_err; + + nvec += hw->num_rx_queues; + hw->intr.num_intrs = nvec; + return; + } + +msix_err: + /* the tx/rx queue interrupt will be disabled */ + hw->intr.num_intrs = 2; + hw->intr.lsc_only = TRUE; + PMD_INIT_LOG(INFO, "Enabled MSI-X with %d vectors", hw->intr.num_intrs); +} + static int vmxnet3_dev_configure(struct rte_eth_dev *dev) { @@ -494,6 +564,8 @@ vmxnet3_dev_configure(struct rte_eth_dev *dev) hw->rss_confPA = mz->iova; } + vmxnet3_alloc_intr_resources(dev); + return 0; } @@ -514,6 +586,85 @@ vmxnet3_write_mac(struct vmxnet3_hw *hw, const uint8_t *addr) VMXNET3_WRITE_BAR1_REG(hw, VMXNET3_REG_MACH, val); } +/* + * Configure the hardware to generate MSI-X interrupts. + * If setting up MSIx fails, try setting up MSI (only 1 interrupt vector + * which will be disabled to allow lsc to work). + * + * Returns 0 on success and -1 otherwise. + */ +static int +vmxnet3_configure_msix(struct rte_eth_dev *dev) +{ + struct vmxnet3_hw *hw = dev->data->dev_private; + struct rte_intr_handle *intr_handle = dev->intr_handle; + uint16_t intr_vector; + int i; + + hw->intr.event_intr_idx = 0; + + /* only vfio-pci driver can support interrupt mode. */ + if (!rte_intr_cap_multiple(intr_handle) || + dev->data->dev_conf.intr_conf.rxq == 0) + return -1; + + intr_vector = dev->data->nb_rx_queues; + if (intr_vector > VMXNET3_MAX_RX_QUEUES) { + PMD_INIT_LOG(ERR, "At most %d intr queues supported", + VMXNET3_MAX_RX_QUEUES); + return -ENOTSUP; + } + + if (rte_intr_efd_enable(intr_handle, intr_vector)) { + PMD_INIT_LOG(ERR, "Failed to enable fastpath event fd"); + return -1; + } + + if (rte_intr_dp_is_en(intr_handle) && !intr_handle->intr_vec) { + intr_handle->intr_vec = + rte_zmalloc("intr_vec", + dev->data->nb_rx_queues * sizeof(int), 0); + if (intr_handle->intr_vec == NULL) { + PMD_INIT_LOG(ERR, "Failed to allocate %d Rx queues intr_vec", + dev->data->nb_rx_queues); + rte_intr_efd_disable(intr_handle); + return -ENOMEM; + } + } + + if (!rte_intr_allow_others(intr_handle) && + dev->data->dev_conf.intr_conf.lsc != 0) { + PMD_INIT_LOG(ERR, "not enough intr vector to support both Rx interrupt and LSC"); + rte_free(intr_handle->intr_vec); + intr_handle->intr_vec = NULL; + rte_intr_efd_disable(intr_handle); + return -1; + } + + /* if we cannot allocate one MSI-X vector per queue, don't enable + * interrupt mode. + */ + if (hw->intr.num_intrs != (intr_handle->nb_efd + 1)) { + PMD_INIT_LOG(ERR, "Device configured with %d Rx intr vectors, expecting %d", + hw->intr.num_intrs, intr_handle->nb_efd + 1); + rte_free(intr_handle->intr_vec); + intr_handle->intr_vec = NULL; + rte_intr_efd_disable(intr_handle); + return -1; + } + + for (i = 0; i < dev->data->nb_rx_queues; i++) + intr_handle->intr_vec[i] = i + 1; + + for (i = 0; i < hw->intr.num_intrs; i++) + hw->intr.mod_levels[i] = UPT1_IML_ADAPTIVE; + + PMD_INIT_LOG(INFO, "intr type %u, mode %u, %u vectors allocated", + hw->intr.type, hw->intr.mask_mode, hw->intr.num_intrs); + + return 0; +} + static int vmxnet3_dev_setup_memreg(struct rte_eth_dev *dev) { @@ -605,6 +756,7 @@ vmxnet3_setup_driver_shared(struct rte_eth_dev *dev) { struct rte_eth_conf port_conf = dev->data->dev_conf; struct vmxnet3_hw *hw = dev->data->dev_private; + struct rte_intr_handle *intr_handle = dev->intr_handle; uint32_t mtu = dev->data->mtu; Vmxnet3_DriverShared *shared = hw->shared; Vmxnet3_DSDevRead *devRead = &shared->devRead; @@ -630,15 +782,6 @@ vmxnet3_setup_driver_shared(struct rte_eth_dev *dev) devRead->misc.numTxQueues = hw->num_tx_queues; devRead->misc.numRxQueues = hw->num_rx_queues; - /* - * Set number of interrupts to 1 - * PMD by default disables all the interrupts but this is MUST - * to activate device. It needs at least one interrupt for - * link events to handle - */ - hw->num_intrs = devRead->intrConf.numIntrs = 1; - devRead->intrConf.intrCtrl |= VMXNET3_IC_DISABLE_ALL; - for (i = 0; i < hw->num_tx_queues; i++) { Vmxnet3_TxQueueDesc *tqd = &hw->tqd_start[i]; vmxnet3_tx_queue_t *txq = dev->data->tx_queues[i]; @@ -655,9 +798,13 @@ vmxnet3_setup_driver_shared(struct rte_eth_dev *dev) tqd->conf.compRingSize = txq->comp_ring.size; tqd->conf.dataRingSize = txq->data_ring.size; tqd->conf.txDataRingDescSize = txq->txdata_desc_size; - tqd->conf.intrIdx = txq->comp_ring.intr_idx; - tqd->status.stopped = TRUE; - tqd->status.error = 0; + + if (hw->intr.lsc_only) + tqd->conf.intrIdx = 1; + else + tqd->conf.intrIdx = intr_handle->intr_vec[i]; + tqd->status.stopped = TRUE; + tqd->status.error = 0; memset(&tqd->stats, 0, sizeof(tqd->stats)); } @@ -674,16 +821,25 @@ vmxnet3_setup_driver_shared(struct rte_eth_dev *dev) rqd->conf.rxRingSize[0] = rxq->cmd_ring[0].size; rqd->conf.rxRingSize[1] = rxq->cmd_ring[1].size; rqd->conf.compRingSize = rxq->comp_ring.size; - rqd->conf.intrIdx = rxq->comp_ring.intr_idx; - if (VMXNET3_VERSION_GE_3(hw)) { - rqd->conf.rxDataRingBasePA = rxq->data_ring.basePA; - rqd->conf.rxDataRingDescSize = rxq->data_desc_size; - } - rqd->status.stopped = TRUE; - rqd->status.error = 0; + + if (hw->intr.lsc_only) + rqd->conf.intrIdx = 1; + else + rqd->conf.intrIdx = intr_handle->intr_vec[i]; + rqd->status.stopped = TRUE; + rqd->status.error = 0; memset(&rqd->stats, 0, sizeof(rqd->stats)); } + /* intr settings */ + devRead->intrConf.autoMask = hw->intr.mask_mode == VMXNET3_IMM_AUTO; + devRead->intrConf.numIntrs = hw->intr.num_intrs; + for (i = 0; i < hw->intr.num_intrs; i++) + devRead->intrConf.modLevels[i] = hw->intr.mod_levels[i]; + + devRead->intrConf.eventIntrIdx = hw->intr.event_intr_idx; + devRead->intrConf.intrCtrl |= rte_cpu_to_le_32(VMXNET3_IC_DISABLE_ALL); + /* RxMode set to 0 of VMXNET3_RXM_xxx */ devRead->rxFilterConf.rxMode = 0; @@ -733,24 +889,18 @@ vmxnet3_dev_start(struct rte_eth_dev *dev) /* Save stats before it is reset by CMD_ACTIVATE */ vmxnet3_hw_stats_save(hw); + /* configure MSI-X */ + ret = vmxnet3_configure_msix(dev); + if (ret < 0) { + /* revert to lsc only */ + hw->intr.num_intrs = 2; + hw->intr.lsc_only = TRUE; + } + ret = vmxnet3_setup_driver_shared(dev); if (ret != VMXNET3_SUCCESS) return ret; - /* check if lsc interrupt feature is enabled */ - if (dev->data->dev_conf.intr_conf.lsc) { - struct rte_pci_device *pci_dev = RTE_DEV_TO_PCI(dev->device); - - /* Setup interrupt callback */ - rte_intr_callback_register(&pci_dev->intr_handle, - vmxnet3_interrupt_handler, dev); - - if (rte_intr_enable(&pci_dev->intr_handle) < 0) { - PMD_INIT_LOG(ERR, "interrupt enable failed"); - return -EIO; - } - } - /* Exchange shared data with device */ VMXNET3_WRITE_BAR1_REG(hw, VMXNET3_REG_DSAL, VMXNET3_GET_ADDR_LO(hw->sharedPA)); @@ -790,9 +940,6 @@ vmxnet3_dev_start(struct rte_eth_dev *dev) } } - /* Disable interrupts */ - vmxnet3_disable_intr(hw); - /* * Load RX queues with blank mbufs and update next2fill index for device * Update RxMode of the device @@ -808,19 +955,29 @@ vmxnet3_dev_start(struct rte_eth_dev *dev) /* Setting proper Rx Mode and issue Rx Mode Update command */ vmxnet3_dev_set_rxmode(hw, VMXNET3_RXM_UCAST | VMXNET3_RXM_BCAST, 1); - if (dev->data->dev_conf.intr_conf.lsc) { - vmxnet3_enable_intr(hw); + /* Setup interrupt callback */ + rte_intr_callback_register(dev->intr_handle, + vmxnet3_interrupt_handler, dev); - /* - * Update link state from device since this won't be - * done upon starting with lsc in use. This is done - * only after enabling interrupts to avoid any race - * where the link state could change without an - * interrupt being fired. - */ - __vmxnet3_dev_link_update(dev, 0); + if (rte_intr_enable(dev->intr_handle) < 0) { + PMD_INIT_LOG(ERR, "interrupt enable failed"); + return -EIO; } + /* enable all intrs */ + vmxnet3_enable_all_intrs(hw); + + vmxnet3_process_events(dev); + + /* + * Update link state from device since this won't be + * done upon starting with lsc in use. This is done + * only after enabling interrupts to avoid any race + * where the link state could change without an + * interrupt being fired. + */ + __vmxnet3_dev_link_update(dev, 0); + return VMXNET3_SUCCESS; } @@ -832,6 +989,8 @@ vmxnet3_dev_stop(struct rte_eth_dev *dev) { struct rte_eth_link link; struct vmxnet3_hw *hw = dev->data->dev_private; + struct rte_intr_handle *intr_handle = dev->intr_handle; + int ret; PMD_INIT_FUNC_TRACE(); @@ -840,16 +999,32 @@ vmxnet3_dev_stop(struct rte_eth_dev *dev) return 0; } - /* disable interrupts */ - vmxnet3_disable_intr(hw); + do { + /* Unregister has lock to make sure there is no running cb. + * This has to happen first since vmxnet3_interrupt_handler + * reenables interrupts by calling vmxnet3_enable_intr + */ + ret = rte_intr_callback_unregister(intr_handle, + vmxnet3_interrupt_handler, + (void *)-1); + } while (ret == -EAGAIN); - if (dev->data->dev_conf.intr_conf.lsc) { - struct rte_pci_device *pci_dev = RTE_DEV_TO_PCI(dev->device); + if (ret < 0) + PMD_DRV_LOG(ERR, "Error attempting to unregister intr cb: %d", + ret); - rte_intr_disable(&pci_dev->intr_handle); + PMD_INIT_LOG(DEBUG, "Disabled %d intr callbacks", ret); - rte_intr_callback_unregister(&pci_dev->intr_handle, - vmxnet3_interrupt_handler, dev); + /* disable interrupts */ + vmxnet3_disable_all_intrs(hw); + + rte_intr_disable(intr_handle); + + /* Clean datapath event and queue/vector mapping */ + rte_intr_efd_disable(intr_handle); + if (intr_handle->intr_vec != NULL) { + rte_free(intr_handle->intr_vec); + intr_handle->intr_vec = NULL; } /* quiesce the device first */ @@ -1480,12 +1655,41 @@ static void vmxnet3_interrupt_handler(void *param) { struct rte_eth_dev *dev = param; - struct rte_pci_device *pci_dev = RTE_DEV_TO_PCI(dev->device); + struct vmxnet3_hw *hw = dev->data->dev_private; + Vmxnet3_DSDevRead *devRead = &hw->shared->devRead; + uint32_t events; + + PMD_INIT_FUNC_TRACE(); + vmxnet3_disable_intr(hw, devRead->intrConf.eventIntrIdx); + + events = VMXNET3_READ_BAR1_REG(hw, VMXNET3_REG_ECR); + if (events == 0) + goto done; + RTE_LOG(DEBUG, PMD, "Reading events: 0x%X", events); vmxnet3_process_events(dev); +done: + vmxnet3_enable_intr(hw, devRead->intrConf.eventIntrIdx); +} + +static int +vmxnet3_dev_rx_queue_intr_enable(struct rte_eth_dev *dev, uint16_t queue_id) +{ + struct vmxnet3_hw *hw = dev->data->dev_private; + + vmxnet3_enable_intr(hw, dev->intr_handle->intr_vec[queue_id]); - if (rte_intr_ack(&pci_dev->intr_handle) < 0) - PMD_DRV_LOG(ERR, "interrupt enable failed"); + return 0; +} + +static int +vmxnet3_dev_rx_queue_intr_disable(struct rte_eth_dev *dev, uint16_t queue_id) +{ + struct vmxnet3_hw *hw = dev->data->dev_private; + + vmxnet3_disable_intr(hw, dev->intr_handle->intr_vec[queue_id]); + + return 0; } RTE_PMD_REGISTER_PCI(net_vmxnet3, rte_vmxnet3_pmd); diff --git a/drivers/net/vmxnet3/vmxnet3_ethdev.h b/drivers/net/vmxnet3/vmxnet3_ethdev.h index f93bb474b2..59bee9723c 100644 --- a/drivers/net/vmxnet3/vmxnet3_ethdev.h +++ b/drivers/net/vmxnet3/vmxnet3_ethdev.h @@ -28,6 +28,8 @@ #define VMXNET3_RSS_MAX_KEY_SIZE 40 #define VMXNET3_RSS_MAX_IND_TABLE_SIZE 128 +#define VMXNET3_MAX_MSIX_VECT (VMXNET3_MAX_TX_QUEUES + \ + VMXNET3_MAX_RX_QUEUES + 1) #define VMXNET3_RSS_OFFLOAD_ALL ( \ ETH_RSS_IPV4 | \ @@ -63,6 +65,15 @@ typedef struct vmxnet3_mf_table { uint16_t num_addrs; /* number of multicast addrs */ } vmxnet3_mf_table_t; +struct vmxnet3_intr { + enum vmxnet3_intr_mask_mode mask_mode; + enum vmxnet3_intr_type type; /* MSI-X, MSI, or INTx? */ + uint8_t num_intrs; /* # of intr vectors */ + uint8_t event_intr_idx; /* idx of the intr vector for event */ + uint8_t mod_levels[VMXNET3_MAX_MSIX_VECT]; /* moderation level */ + bool lsc_only; /* no Rx queue interrupt */ +}; + struct vmxnet3_hw { uint8_t *hw_addr0; /* BAR0: PT-Passthrough Regs */ uint8_t *hw_addr1; /* BAR1: VD-Virtual Device Regs */ @@ -102,6 +113,7 @@ struct vmxnet3_hw { uint64_t rss_confPA; vmxnet3_mf_table_t *mf_table; uint32_t shadow_vfta[VMXNET3_VFT_SIZE]; + struct vmxnet3_intr intr; Vmxnet3_MemRegs *memRegs; uint64_t memRegsPA; #define VMXNET3_VFT_TABLE_SIZE (VMXNET3_VFT_SIZE * sizeof(uint32_t)) -- 2.17.1