From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 33D36A034C; Mon, 2 May 2022 21:48:06 +0200 (CEST) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 0159E4283C; Mon, 2 May 2022 21:47:41 +0200 (CEST) Received: from NAM10-DM6-obe.outbound.protection.outlook.com (mail-dm6nam10on2043.outbound.protection.outlook.com [40.107.93.43]) by mails.dpdk.org (Postfix) with ESMTP id 867954282F for ; Mon, 2 May 2022 21:47:38 +0200 (CEST) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=ifVJHHS0xeBj0CMviUCyLcWfIXXYYmKgc7fdIpB7bXn4E02Twe8C1mV1pSfUi5ayg7IQcas3JVT9TKl4FUK2YyAZiSDrih5hhQkgve71y0EU9vI+nyXMbtb6Lv+SPFN+JjcjQ16D3SuS4K7ON9cuz8+MMxVofMnfeuXZwk/NOdyN/QB5swJ1EtOAgaa5AyuBROUrYv0nQ2xRA0XS7HHCH7mayIu2MXi6YvRUOn2n0Nc7XmbmXxoPRcwvfr9ZQf30i/CJFTDCUQa1H7yC2XP2UnqbLTGMXZQKfhP9a/Iv9IY447aDRayggS5HWza2S6z57Ci1c3SYpOCXScgy1mhI2Q== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=XUBgJ6O11uohU3/Yda56oF9mCcFa5s8SQdchWM48cgE=; b=ZSgmAPklPkLl90QlLtnRStfv2DktSjpWddFElMk0RTEscDF0GOQ4P/2jRrwWRuZKmd8mqRmbgf9hfTjDHgGWULRZMgMDMgRffDmZfJQtRvrYAexGxHR4+FQ6jPPm3KhyN2IglpQM/yjCjPuM1OI39QZmqefcrG5BGznwUY4ldJ9bktD1NWqt9wnpA1JQ3+QqsjgGXNdFwaNtpa4RCKQrp1M9c0PsNsW2RwYj/MMYiAHFTN5FM9GgP0NVirL2KmIZB+QcrrHOFiwmD29U9wZEufEhc7MzVSMZmKBBqvMjXQDsfpjSAbdxv7qTAn5uahzcFMZYz6qk1ADBfP6QvN/q0Q== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=vmware.com; dmarc=pass action=none header.from=vmware.com; dkim=pass header.d=vmware.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=vmware.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=XUBgJ6O11uohU3/Yda56oF9mCcFa5s8SQdchWM48cgE=; b=QvZUG378q/gRX84uyzVWIwkLVviAsSSHLXdJFQGS+W73ObzKDVQPhXpupiERzDHvEoZhhn/Oo4rdUn0PhritTJk6/E/AfT8SMUWmMJj1cX76trrOsZ0bFyoWhgek+Jkjnps6px3JRxvhWbsOI4nIuXQbxr8AZt4d2tghcrrj2qo= Authentication-Results: dkim=none (message not signed) header.d=none;dmarc=none action=none header.from=vmware.com; Received: from BYAPR05MB5624.namprd05.prod.outlook.com (2603:10b6:a03:1d::13) by MWHPR05MB3294.namprd05.prod.outlook.com (2603:10b6:301:41::24) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5206.12; Mon, 2 May 2022 19:47:35 +0000 Received: from BYAPR05MB5624.namprd05.prod.outlook.com ([fe80::5161:a7c5:8c7d:f3b2]) by BYAPR05MB5624.namprd05.prod.outlook.com ([fe80::5161:a7c5:8c7d:f3b2%7]) with mapi id 15.20.5206.024; Mon, 2 May 2022 19:47:34 +0000 From: Pankaj Gupta To: jbehrens@vmware.com, yongwang@vmware.com Cc: dev@dpdk.org, pagupta@vmware.com Subject: [PATCH 5/8] vmxnet3, version 6 Date: Mon, 2 May 2022 15:45:54 -0400 Message-Id: <20220502194557.16016-6-pagupta@vmware.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20220502194557.16016-1-pagupta@vmware.com> References: <20220502194557.16016-1-pagupta@vmware.com> Content-Type: text/plain X-ClientProxiedBy: BYAPR01CA0049.prod.exchangelabs.com (2603:10b6:a03:94::26) To BYAPR05MB5624.namprd05.prod.outlook.com (2603:10b6:a03:1d::13) MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: 77ca3734-c76d-47b8-6bf7-08da2c74999c X-MS-TrafficTypeDiagnostic: MWHPR05MB3294:EE_ X-Microsoft-Antispam-PRVS: X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: Rp6E4uTbgEm5krke0rKw/7wX9vuVseVeLfJyUhYWr3pXBAPohhjqJem8xjoGM8Hla4FNIHMhQPw6Yz4di0pxQ4VgO34PClva4mGMj5aMGBE8/CAJxJ1s+pkAnAtYGRfxE4SW7cXr7KRUZZ2AIK19w1RC/k5+OGvz04M7G42yciKuVNzByelcPpWKmWOgjkanFyIRZ8fRWu4tfsAC2JNzzgfgaaDIMNymzFnvnrg2rx7usOkfBSD+g8uj61hmkM4xpZxBk6ZutAN7w17QAyJWmN6oPpRrw5cHmFQIYPgfEIr/uO948m3xrai6a4FW1Tm26Ji1td90KQ0N1kupi/S6BGBQc58DrRgSu4w0K3KqkZ9KLGdo7xe6Kh7pjDbDm5+jT7uV/ZuQBioWyOr//Rca4MwT+MG7eyhOgylLSiCzkHpF07O4uQZ5mb9pjiuBdTSsaF6XgJWhp2T50D+cEFRWAw8hdsnFoKsSZ8f7DTcmqubJJLq9rc7hD02HDMuocrnrDc+XGuyLPWqwJLlPGJUML7X7hkDEtoRBUXoF0SE2eUASZ5V0+SINGd4DnqPa/OZewwKLY4KVDCiRlQq/ksJWZe/IGBdmgSoa7zJzki7494DwmyxV4yLc0JR9Ql7/1asIMbtOxiocM1qYhxvE2TxNvJY40hdZxGBZbcxQbyTz58putmq3AC8Yf5VGF/AUSu3XDXfIyg9IXz7s6fNk6QyYY6+btHpD/M6nKtbzXhIgo9A= X-Forefront-Antispam-Report: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:BYAPR05MB5624.namprd05.prod.outlook.com; PTR:; CAT:NONE; SFS:(13230001)(4636009)(366004)(6666004)(1076003)(4326008)(6486002)(508600001)(86362001)(186003)(83380400001)(38350700002)(8676002)(52116002)(6636002)(2616005)(316002)(19627235002)(36756003)(107886003)(66476007)(38100700002)(66556008)(66946007)(8936002)(6506007)(5660300002)(2906002)(26005)(6512007)(30864003)(21314003); DIR:OUT; SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: =?us-ascii?Q?Cozro1RGru/IyOvjaDRHXSGAk1Sme1ZsjiT9usEfF/R44ikVmITq9q07bd8g?= =?us-ascii?Q?MTav5K+dMzil4YHOm+DqyxsO3rhbiNrdizNnWKn6bcm6fQ6bxxtT1B+cUcLT?= =?us-ascii?Q?YxCbw0w6/EnKBLWc5oR608RGFNgurAmeIzEO+xJU8kME0collP8vXHgCxKO9?= =?us-ascii?Q?vGyonC0JL51BI+OFRzkjMo+XpMMJFUYPytFOWMRT1s22/D16ZCeR6IzVxyYZ?= =?us-ascii?Q?PN4Zpzs0S7wrjetNng2D9dvpppFXeqOIlwnttcpSsaqpDUt6uqTIRoeD22t7?= =?us-ascii?Q?RsWj0FQb6OGmDszmAQPW3BweMaOE5XKitLIrP+u6iccFRjP0hPuG3JAK6W1z?= =?us-ascii?Q?weEIVsdV+h8Vi1P1A6b8xkTmy4IQdBRTa1dpLMkGYnSuGoxFWD0AK+riws3m?= =?us-ascii?Q?M2dNf1f0nNDh7HOAgLynE+FRdXAUqLz90CSduGSc3DxmM9qvD1TrNLiQa3X0?= =?us-ascii?Q?vEv1ijwrC3D1g1jdIr0sU0APAlmN2nD4H8+/8kSIwzO9LrM1Q5UPXiZHAYkx?= =?us-ascii?Q?0SZDpthU8GF1sLvZ2gz6eahoKy2ldWVVbpUjjI4DIRU/kLTzQCDvitLsitNB?= =?us-ascii?Q?iWuRnEPwFMxnvHikKwSrLWcNwZKXwuSgJNCA0AuVNZxHgWrT/aayRunLjAFR?= =?us-ascii?Q?tBshTo9buGaIk5Hi9DMdLmQCwieJpQzI68TYYkTVpLYhljcqtlmx8IuUkSmE?= =?us-ascii?Q?OMKA7pN/VMeBSP9FS5N1ffYmausd4vd0irZltE03OFCXf/nOtf7waCBy/ofM?= =?us-ascii?Q?l3dsYr7GzWdagLLndLJ+qupWHKNhJg35xN0SMEYAZFh2wO/bC1jwq/9YufD2?= =?us-ascii?Q?lymS2XIgbymCNTYNK2X6fFwRmbDcGGrw9SPtzqSLcyGWNykyMjWnzLwFdOgA?= =?us-ascii?Q?LsrA7CrpbWstTNbckzS5J46ilkwO7JN751Zs6Nu9YGtLKbudrNOQRd+FTgL2?= =?us-ascii?Q?YlmNC+YHET7PGeDT6FY7jDDP2zFoUGpdFEFeE/rXWm5sDk6CH5vnwNd9bMz+?= =?us-ascii?Q?Va24RFhAa+w9a/vDqyPb5AYTkiZmYfvzqFMb6j/j6/kTwHTa4XglQcI0ZViw?= =?us-ascii?Q?/sDC9MM/csBUV+Pp+FqdlaCtNL9K/vyVketlzhhmeys5QD0X7xG3SsgrYDC9?= =?us-ascii?Q?Q/TtDNh1T5WK3LXIYfgtxcKr1Swsykyo6bWdpVgBzhwwkhy1Ke4ACa5yICIN?= =?us-ascii?Q?A03CvyWlSO6SY4lG9dfyuH3nkpg+kIHkHjPJfxKLenLfnOW7uTutbyUIS/dt?= =?us-ascii?Q?Ir0OAPN34mr2tPl5fIB+xBc2h3JiM3yUyYF4bUc63+3FSXoMZrOaaEzD87Db?= =?us-ascii?Q?aB9iCh5Hxts1iH0i36hEYne4KIbt9lfJOS3vjEEkp+vBZ0wKKlBuEfRwOWZe?= =?us-ascii?Q?aLwEY40tf5IUWNEdDb/0/If5Ep+dqBzxtLwp/9I0JZ5JyZK4JPomCBFh8eF0?= =?us-ascii?Q?9oZOVV2AkNK4d+VVZ4FtLl3qrCFXZVQq3XVzw3axrliCw6+5h/Qwk5sZcW9h?= =?us-ascii?Q?fiQgJE61OFSbdf1MnDWCGfjRtTLejsIjWbEQsjfbV8ngIzrt3cmIGFS04ihh?= =?us-ascii?Q?dgFpq2CdLj/S1VcDOaB64DEYYAFLgloJmP5ZnaXn2BQkQD6PmNQ6qE7+xuIi?= =?us-ascii?Q?VxYR4RNuCI3DX6pWTpSRs/TDYKJMKbIqxr2INYRgzZOorWH0B3nQRNX27ieT?= =?us-ascii?Q?mQPgmtzhrtGBRbft6Eej5avrGb+1YO24Jw2Z5QnxRRIFfj1YXM9pFaur+dKl?= =?us-ascii?Q?mekv8xkpaQ=3D=3D?= X-OriginatorOrg: vmware.com X-MS-Exchange-CrossTenant-Network-Message-Id: 77ca3734-c76d-47b8-6bf7-08da2c74999c X-MS-Exchange-CrossTenant-AuthSource: BYAPR05MB5624.namprd05.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 02 May 2022 19:47:33.3371 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: b39138ca-3cee-4b4a-a4d6-cd83d9dd62f0 X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: 2zGOaZsgSnzBDJK29bpr3h1VyDb6anNF18Vg3PJrnQgMkdQrZIvDldP3n2bwpJZT9n6OOfAMIuZodpfN9+Z57A== X-MS-Exchange-Transport-CrossTenantHeadersStamped: MWHPR05MB3294 X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org vmxnet3 version 6 supports some new features, including but not limited to: - Increased max MTU up to 9190 - Increased max number of queues, both for RX and TX - Removes power-of-two limitations - Extended interrupt structures, required implementation for additional number of queues Tested, using testpmd, for different hardware version on ESXi 7.0 Update 2. Signed-off-by: Pankaj Gupta --- drivers/net/vmxnet3/base/vmxnet3_defs.h | 135 +++++++++----- drivers/net/vmxnet3/vmxnet3_ethdev.c | 222 +++++++++++++++++------- drivers/net/vmxnet3/vmxnet3_ethdev.h | 10 +- drivers/net/vmxnet3/vmxnet3_rxtx.c | 2 +- 4 files changed, 259 insertions(+), 110 deletions(-) diff --git a/drivers/net/vmxnet3/base/vmxnet3_defs.h b/drivers/net/vmxnet3/base/vmxnet3_defs.h index 8d62b3e116..ceac5d64db 100644 --- a/drivers/net/vmxnet3/base/vmxnet3_defs.h +++ b/drivers/net/vmxnet3/base/vmxnet3_defs.h @@ -72,38 +72,42 @@ #endif typedef enum { - VMXNET3_CMD_FIRST_SET = 0xCAFE0000, - VMXNET3_CMD_ACTIVATE_DEV = VMXNET3_CMD_FIRST_SET, - VMXNET3_CMD_QUIESCE_DEV, - VMXNET3_CMD_RESET_DEV, - VMXNET3_CMD_UPDATE_RX_MODE, - VMXNET3_CMD_UPDATE_MAC_FILTERS, - VMXNET3_CMD_UPDATE_VLAN_FILTERS, - VMXNET3_CMD_UPDATE_RSSIDT, - VMXNET3_CMD_UPDATE_IML, - VMXNET3_CMD_UPDATE_PMCFG, - VMXNET3_CMD_UPDATE_FEATURE, - VMXNET3_CMD_STOP_EMULATION, - VMXNET3_CMD_LOAD_PLUGIN, - VMXNET3_CMD_ACTIVATE_VF, - VMXNET3_CMD_RESERVED3, - VMXNET3_CMD_RESERVED4, - VMXNET3_CMD_REGISTER_MEMREGS, - VMXNET3_CMD_SET_RSS_FIELDS, - - VMXNET3_CMD_FIRST_GET = 0xF00D0000, - VMXNET3_CMD_GET_QUEUE_STATUS = VMXNET3_CMD_FIRST_GET, - VMXNET3_CMD_GET_STATS, - VMXNET3_CMD_GET_LINK, - VMXNET3_CMD_GET_PERM_MAC_LO, - VMXNET3_CMD_GET_PERM_MAC_HI, - VMXNET3_CMD_GET_DID_LO, - VMXNET3_CMD_GET_DID_HI, - VMXNET3_CMD_GET_DEV_EXTRA_INFO, - VMXNET3_CMD_GET_CONF_INTR, - VMXNET3_CMD_GET_ADAPTIVE_RING_INFO, - VMXNET3_CMD_GET_TXDATA_DESC_SIZE, - VMXNET3_CMD_RESERVED5, + VMXNET3_CMD_FIRST_SET = 0xCAFE0000, + VMXNET3_CMD_ACTIVATE_DEV = VMXNET3_CMD_FIRST_SET, + VMXNET3_CMD_QUIESCE_DEV, + VMXNET3_CMD_RESET_DEV, + VMXNET3_CMD_UPDATE_RX_MODE, + VMXNET3_CMD_UPDATE_MAC_FILTERS, + VMXNET3_CMD_UPDATE_VLAN_FILTERS, + VMXNET3_CMD_UPDATE_RSSIDT, + VMXNET3_CMD_UPDATE_IML, + VMXNET3_CMD_UPDATE_PMCFG, + VMXNET3_CMD_UPDATE_FEATURE, + VMXNET3_CMD_STOP_EMULATION, + VMXNET3_CMD_LOAD_PLUGIN, + VMXNET3_CMD_ACTIVATE_VF, + VMXNET3_CMD_RESERVED3, + VMXNET3_CMD_RESERVED4, + VMXNET3_CMD_REGISTER_MEMREGS, + VMXNET3_CMD_SET_RSS_FIELDS, + + VMXNET3_CMD_FIRST_GET = 0xF00D0000, + VMXNET3_CMD_GET_QUEUE_STATUS = VMXNET3_CMD_FIRST_GET, + VMXNET3_CMD_GET_STATS, + VMXNET3_CMD_GET_LINK, + VMXNET3_CMD_GET_PERM_MAC_LO, + VMXNET3_CMD_GET_PERM_MAC_HI, + VMXNET3_CMD_GET_DID_LO, + VMXNET3_CMD_GET_DID_HI, + VMXNET3_CMD_GET_DEV_EXTRA_INFO, + VMXNET3_CMD_GET_CONF_INTR, + VMXNET3_CMD_GET_ADAPTIVE_RING_INFO, + VMXNET3_CMD_GET_TXDATA_DESC_SIZE, + VMXNET3_CMD_RESERVED5, + VMXNET3_CMD_RESERVED6, + VMXNET3_CMD_RESERVED7, + VMXNET3_CMD_RESERVED8, + VMXNET3_CMD_GET_MAX_QUEUES_CONF, } Vmxnet3_Cmd; /* Adaptive Ring Info Flags */ @@ -571,6 +575,24 @@ enum vmxnet3_intr_type { /* addition 1 for events */ #define VMXNET3_MAX_INTRS 25 +/* Version 6 and later will use below macros */ +#define VMXNET3_EXT_MAX_TX_QUEUES 32 +#define VMXNET3_EXT_MAX_RX_QUEUES 32 + +/* Version-dependent MAX RX/TX queues macro */ +#define MAX_RX_QUEUES \ + (VMXNET3_VERSION_GE_6(hw) ? \ + VMXNET3_EXT_MAX_RX_QUEUES : \ + VMXNET3_MAX_RX_QUEUES) +#define MAX_TX_QUEUES \ + (VMXNET3_VERSION_GE_6(hw) ? \ + VMXNET3_EXT_MAX_TX_QUEUES : \ + VMXNET3_MAX_TX_QUEUES) + +/* addition 1 for events */ +#define VMXNET3_EXT_MAX_INTRS 65 +#define VMXNET3_FIRST_SET_INTRS 64 + /* value of intrCtrl */ #define VMXNET3_IC_DISABLE_ALL 0x1 /* bit 0 */ @@ -587,6 +609,21 @@ struct Vmxnet3_IntrConf { #include "vmware_pack_end.h" Vmxnet3_IntrConf; +typedef +#include "vmware_pack_begin.h" +struct Vmxnet3_IntrConfExt { + uint8 autoMask; + uint8 numIntrs; /* # of interrupts */ + uint8 eventIntrIdx; + uint8 reserved; + __le32 intrCtrl; + __le32 reserved1; + uint8 modLevels[VMXNET3_EXT_MAX_INTRS]; /* moderation level for each intr */ + uint8 reserved2[3]; +} +#include "vmware_pack_end.h" +Vmxnet3_IntrConfExt; + /* one bit per VLAN ID, the size is in the units of uint32 */ #define VMXNET3_VFT_SIZE (4096 / (sizeof(uint32) * 8)) @@ -692,6 +729,15 @@ struct Vmxnet3_DSDevRead { #include "vmware_pack_end.h" Vmxnet3_DSDevRead; +typedef +#include "vmware_pack_begin.h" +struct Vmxnet3_DSDevReadExt { + /* read-only region for device, read by dev in response to a SET cmd */ + struct Vmxnet3_IntrConfExt intrConfExt; +} +#include "vmware_pack_end.h" +Vmxnet3_DSDevReadExt; + typedef #include "vmware_pack_begin.h" struct Vmxnet3_TxQueueDesc { @@ -778,18 +824,18 @@ Vmxnet3_CmdInfo; typedef #include "vmware_pack_begin.h" struct Vmxnet3_DriverShared { - __le32 magic; - __le32 pad; /* make devRead start at 64-bit boundaries */ - Vmxnet3_DSDevRead devRead; - __le32 ecr; - __le32 reserved; - - union { - __le32 reserved1[4]; - Vmxnet3_CmdInfo cmdInfo; /* only valid in the context of executing the - * relevant command - */ - } cu; + __le32 magic; + __le32 size; /* size of DriverShared */ + Vmxnet3_DSDevRead devRead; + __le32 ecr; + __le32 reserved; + + union { + __le32 reserved1[4]; + /* only valid in the context of executing the relevant command */ + Vmxnet3_CmdInfo cmdInfo; + } cu; + struct Vmxnet3_DSDevReadExt devReadExt; } #include "vmware_pack_end.h" Vmxnet3_DriverShared; @@ -821,6 +867,7 @@ do {\ ((vfTable[vid >> 5] & (1 << (vid & 31))) != 0) #define VMXNET3_MAX_MTU 9000 +#define VMXNET3_V6_MAX_MTU 9190 #define VMXNET3_MIN_MTU 60 #define VMXNET3_LINK_UP (10000 << 16 | 1) // 10 Gbps, up diff --git a/drivers/net/vmxnet3/vmxnet3_ethdev.c b/drivers/net/vmxnet3/vmxnet3_ethdev.c index f77399f145..e213b6d864 100644 --- a/drivers/net/vmxnet3/vmxnet3_ethdev.c +++ b/drivers/net/vmxnet3/vmxnet3_ethdev.c @@ -222,24 +222,20 @@ vmxnet3_disable_intr(struct vmxnet3_hw *hw, unsigned int intr_idx) } /* - * Enable all intrs used by the device + * Simple helper to get intrCtrl and eventIntrIdx based on config and hw version */ static void -vmxnet3_enable_all_intrs(struct vmxnet3_hw *hw) +vmxnet3_get_intr_ctrl_ev(struct vmxnet3_hw *hw, + uint8 **out_eventIntrIdx, + uint32 **out_intrCtrl) { - Vmxnet3_DSDevRead *devRead = &hw->shared->devRead; - - PMD_INIT_FUNC_TRACE(); - devRead->intrConf.intrCtrl &= rte_cpu_to_le_32(~VMXNET3_IC_DISABLE_ALL); - - if (hw->intr.lsc_only) { - vmxnet3_enable_intr(hw, devRead->intrConf.eventIntrIdx); + if (VMXNET3_VERSION_GE_6(hw) && hw->queuesExtEnabled) { + *out_eventIntrIdx = &hw->shared->devReadExt.intrConfExt.eventIntrIdx; + *out_intrCtrl = &hw->shared->devReadExt.intrConfExt.intrCtrl; } else { - int i; - - for (i = 0; i < hw->intr.num_intrs; i++) - vmxnet3_enable_intr(hw, i); + *out_eventIntrIdx = &hw->shared->devRead.intrConf.eventIntrIdx; + *out_intrCtrl = &hw->shared->devRead.intrConf.intrCtrl; } } @@ -250,15 +246,42 @@ static void vmxnet3_disable_all_intrs(struct vmxnet3_hw *hw) { int i; + uint8 *eventIntrIdx; + uint32 *intrCtrl; PMD_INIT_FUNC_TRACE(); + vmxnet3_get_intr_ctrl_ev(hw, &eventIntrIdx, &intrCtrl); + + *intrCtrl |= rte_cpu_to_le_32(VMXNET3_IC_DISABLE_ALL); - hw->shared->devRead.intrConf.intrCtrl |= - rte_cpu_to_le_32(VMXNET3_IC_DISABLE_ALL); - for (i = 0; i < hw->num_intrs; i++) + for (i = 0; i < hw->intr.num_intrs; i++) vmxnet3_disable_intr(hw, i); } +/* + * Enable all intrs used by the device + */ +static void +vmxnet3_enable_all_intrs(struct vmxnet3_hw *hw) +{ + uint8 *eventIntrIdx; + uint32 *intrCtrl; + + PMD_INIT_FUNC_TRACE(); + vmxnet3_get_intr_ctrl_ev(hw, &eventIntrIdx, &intrCtrl); + + *intrCtrl &= rte_cpu_to_le_32(~VMXNET3_IC_DISABLE_ALL); + + if (hw->intr.lsc_only) { + vmxnet3_enable_intr(hw, *eventIntrIdx); + } else { + int i; + + for (i = 0; i < hw->intr.num_intrs; i++) + vmxnet3_enable_intr(hw, i); + } +} + /* * Gets tx data ring descriptor size. */ @@ -333,7 +356,11 @@ eth_vmxnet3_dev_init(struct rte_eth_dev *eth_dev) /* Check h/w version compatibility with driver. */ ver = VMXNET3_READ_BAR1_REG(hw, VMXNET3_REG_VRRS); - if (ver & (1 << VMXNET3_REV_5)) { + if (ver & (1 << VMXNET3_REV_6)) { + VMXNET3_WRITE_BAR1_REG(hw, VMXNET3_REG_VRRS, + 1 << VMXNET3_REV_6); + hw->version = VMXNET3_REV_6 + 1; + } else if (ver & (1 << VMXNET3_REV_5)) { VMXNET3_WRITE_BAR1_REG(hw, VMXNET3_REG_VRRS, 1 << VMXNET3_REV_5); hw->version = VMXNET3_REV_5 + 1; @@ -508,15 +535,22 @@ vmxnet3_dev_configure(struct rte_eth_dev *dev) if (dev->data->dev_conf.rxmode.mq_mode & RTE_ETH_MQ_RX_RSS_FLAG) dev->data->dev_conf.rxmode.offloads |= RTE_ETH_RX_OFFLOAD_RSS_HASH; - if (dev->data->nb_tx_queues > VMXNET3_MAX_TX_QUEUES || - dev->data->nb_rx_queues > VMXNET3_MAX_RX_QUEUES) { - PMD_INIT_LOG(ERR, "ERROR: Number of queues not supported"); - return -EINVAL; + if (!VMXNET3_VERSION_GE_6(hw)) { + if (!rte_is_power_of_2(dev->data->nb_rx_queues)) { + PMD_INIT_LOG(ERR, + "ERROR: Number of rx queues not power of 2"); + return -EINVAL; + } } - if (!rte_is_power_of_2(dev->data->nb_rx_queues)) { - PMD_INIT_LOG(ERR, "ERROR: Number of rx queues not power of 2"); - return -EINVAL; + /* At this point, the number of queues requested has already + * been validated against dev_infos max queues by EAL + */ + if (dev->data->nb_rx_queues > VMXNET3_MAX_RX_QUEUES || + dev->data->nb_tx_queues > VMXNET3_MAX_TX_QUEUES) { + hw->queuesExtEnabled = 1; + } else { + hw->queuesExtEnabled = 0; } size = dev->data->nb_rx_queues * sizeof(struct Vmxnet3_TxQueueDesc) + @@ -627,9 +661,9 @@ vmxnet3_configure_msix(struct rte_eth_dev *dev) return -1; intr_vector = dev->data->nb_rx_queues; - if (intr_vector > VMXNET3_MAX_RX_QUEUES) { + if (intr_vector > MAX_RX_QUEUES) { PMD_INIT_LOG(ERR, "At most %d intr queues supported", - VMXNET3_MAX_RX_QUEUES); + MAX_RX_QUEUES); return -ENOTSUP; } @@ -777,6 +811,7 @@ vmxnet3_setup_driver_shared(struct rte_eth_dev *dev) uint32_t mtu = dev->data->mtu; Vmxnet3_DriverShared *shared = hw->shared; Vmxnet3_DSDevRead *devRead = &shared->devRead; + struct Vmxnet3_DSDevReadExt *devReadExt = &shared->devReadExt; uint64_t rx_offloads = dev->data->dev_conf.rxmode.offloads; uint32_t i; int ret; @@ -853,13 +888,27 @@ vmxnet3_setup_driver_shared(struct rte_eth_dev *dev) } /* intr settings */ - devRead->intrConf.autoMask = hw->intr.mask_mode == VMXNET3_IMM_AUTO; - devRead->intrConf.numIntrs = hw->intr.num_intrs; - for (i = 0; i < hw->intr.num_intrs; i++) - devRead->intrConf.modLevels[i] = hw->intr.mod_levels[i]; + if (VMXNET3_VERSION_GE_6(hw) && hw->queuesExtEnabled) { + devReadExt->intrConfExt.autoMask = hw->intr.mask_mode == + VMXNET3_IMM_AUTO; + devReadExt->intrConfExt.numIntrs = hw->intr.num_intrs; + for (i = 0; i < hw->intr.num_intrs; i++) + devReadExt->intrConfExt.modLevels[i] = + hw->intr.mod_levels[i]; + + devReadExt->intrConfExt.eventIntrIdx = hw->intr.event_intr_idx; + devReadExt->intrConfExt.intrCtrl |= + rte_cpu_to_le_32(VMXNET3_IC_DISABLE_ALL); + } else { + devRead->intrConf.autoMask = hw->intr.mask_mode == + VMXNET3_IMM_AUTO; + devRead->intrConf.numIntrs = hw->intr.num_intrs; + for (i = 0; i < hw->intr.num_intrs; i++) + devRead->intrConf.modLevels[i] = hw->intr.mod_levels[i]; - devRead->intrConf.eventIntrIdx = hw->intr.event_intr_idx; - devRead->intrConf.intrCtrl |= rte_cpu_to_le_32(VMXNET3_IC_DISABLE_ALL); + devRead->intrConf.eventIntrIdx = hw->intr.event_intr_idx; + devRead->intrConf.intrCtrl |= rte_cpu_to_le_32(VMXNET3_IC_DISABLE_ALL); + } /* RxMode set to 0 of VMXNET3_RXM_xxx */ devRead->rxFilterConf.rxMode = 0; @@ -937,18 +986,24 @@ vmxnet3_dev_start(struct rte_eth_dev *dev) return -EINVAL; } - /* Setup memory region for rx buffers */ - ret = vmxnet3_dev_setup_memreg(dev); - if (ret == 0) { - VMXNET3_WRITE_BAR1_REG(hw, VMXNET3_REG_CMD, - VMXNET3_CMD_REGISTER_MEMREGS); - ret = VMXNET3_READ_BAR1_REG(hw, VMXNET3_REG_CMD); - if (ret != 0) - PMD_INIT_LOG(DEBUG, - "Failed in setup memory region cmd\n"); - ret = 0; + /* Check memregs restrictions first */ + if (dev->data->nb_rx_queues <= VMXNET3_MAX_RX_QUEUES && + dev->data->nb_tx_queues <= VMXNET3_MAX_TX_QUEUES) { + ret = vmxnet3_dev_setup_memreg(dev); + if (ret == 0) { + VMXNET3_WRITE_BAR1_REG(hw, VMXNET3_REG_CMD, + VMXNET3_CMD_REGISTER_MEMREGS); + ret = VMXNET3_READ_BAR1_REG(hw, VMXNET3_REG_CMD); + if (ret != 0) + PMD_INIT_LOG(DEBUG, + "Failed in setup memory region cmd\n"); + ret = 0; + } else { + PMD_INIT_LOG(DEBUG, "Failed to setup memory region\n"); + } } else { - PMD_INIT_LOG(DEBUG, "Failed to setup memory region\n"); + PMD_INIT_LOG(WARNING, "Memregs can't init (rx: %d, tx: %d)", + dev->data->nb_rx_queues, dev->data->nb_tx_queues); } if (VMXNET3_VERSION_GE_4(hw) && @@ -1203,8 +1258,6 @@ vmxnet3_hw_stats_save(struct vmxnet3_hw *hw) VMXNET3_WRITE_BAR1_REG(hw, VMXNET3_REG_CMD, VMXNET3_CMD_GET_STATS); - RTE_BUILD_BUG_ON(RTE_ETHDEV_QUEUE_STAT_CNTRS < VMXNET3_MAX_TX_QUEUES); - for (i = 0; i < hw->num_tx_queues; i++) vmxnet3_hw_tx_stats_get(hw, i, &hw->saved_tx_stats[i]); for (i = 0; i < hw->num_rx_queues; i++) @@ -1306,7 +1359,6 @@ vmxnet3_dev_stats_get(struct rte_eth_dev *dev, struct rte_eth_stats *stats) VMXNET3_WRITE_BAR1_REG(hw, VMXNET3_REG_CMD, VMXNET3_CMD_GET_STATS); - RTE_BUILD_BUG_ON(RTE_ETHDEV_QUEUE_STAT_CNTRS < VMXNET3_MAX_TX_QUEUES); for (i = 0; i < hw->num_tx_queues; i++) { vmxnet3_tx_stats_get(hw, i, &txStats); @@ -1323,7 +1375,6 @@ vmxnet3_dev_stats_get(struct rte_eth_dev *dev, struct rte_eth_stats *stats) stats->oerrors += txStats.pktsTxError + txStats.pktsTxDiscard; } - RTE_BUILD_BUG_ON(RTE_ETHDEV_QUEUE_STAT_CNTRS < VMXNET3_MAX_RX_QUEUES); for (i = 0; i < hw->num_rx_queues; i++) { vmxnet3_rx_stats_get(hw, i, &rxStats); @@ -1377,9 +1428,29 @@ vmxnet3_dev_info_get(struct rte_eth_dev *dev, struct rte_eth_dev_info *dev_info) { struct vmxnet3_hw *hw = dev->data->dev_private; + int queues = 0; + + if (VMXNET3_VERSION_GE_6(hw)) { + VMXNET3_WRITE_BAR1_REG(hw, VMXNET3_REG_CMD, + VMXNET3_CMD_GET_MAX_QUEUES_CONF); + queues = VMXNET3_READ_BAR1_REG(hw, VMXNET3_REG_CMD); + + if (queues > 0) { +#define MIN(x, y) (((x) < (y)) ? (x) : (y)) + dev_info->max_rx_queues = + MIN(VMXNET3_EXT_MAX_RX_QUEUES, ((queues >> 8) & 0xff)); + dev_info->max_tx_queues = + MIN(VMXNET3_EXT_MAX_TX_QUEUES, (queues & 0xff)); +#undef MIN + } else { + dev_info->max_rx_queues = VMXNET3_MAX_RX_QUEUES; + dev_info->max_tx_queues = VMXNET3_MAX_TX_QUEUES; + } + } else { + dev_info->max_rx_queues = VMXNET3_MAX_RX_QUEUES; + dev_info->max_tx_queues = VMXNET3_MAX_TX_QUEUES; + } - dev_info->max_rx_queues = VMXNET3_MAX_RX_QUEUES; - dev_info->max_tx_queues = VMXNET3_MAX_TX_QUEUES; dev_info->min_rx_bufsize = 1518 + RTE_PKTMBUF_HEADROOM; dev_info->max_rx_pktlen = 16384; /* includes CRC, cf MAXFRS register */ dev_info->min_mtu = VMXNET3_MIN_MTU; @@ -1430,24 +1501,50 @@ vmxnet3_dev_supported_ptypes_get(struct rte_eth_dev *dev) } static int -vmxnet3_dev_mtu_set(struct rte_eth_dev *dev, __rte_unused uint16_t mtu) +vmxnet3_mac_addr_set(struct rte_eth_dev *dev, struct rte_ether_addr *mac_addr) { - if (dev->data->dev_started) { - PMD_DRV_LOG(ERR, "Port %d must be stopped to configure MTU", - dev->data->port_id); - return -EBUSY; - } + struct vmxnet3_hw *hw = dev->data->dev_private; + rte_ether_addr_copy(mac_addr, (struct rte_ether_addr *)(hw->perm_addr)); + vmxnet3_write_mac(hw, mac_addr->addr_bytes); return 0; } static int -vmxnet3_mac_addr_set(struct rte_eth_dev *dev, struct rte_ether_addr *mac_addr) +vmxnet3_dev_mtu_set(struct rte_eth_dev *dev, uint16_t mtu) { struct vmxnet3_hw *hw = dev->data->dev_private; + uint32_t frame_size = mtu + RTE_ETHER_HDR_LEN + RTE_ETHER_CRC_LEN + 4; + + if (mtu < VMXNET3_MIN_MTU) + return -EINVAL; + + if (VMXNET3_VERSION_GE_6(hw)) { + if (mtu > VMXNET3_V6_MAX_MTU) + return -EINVAL; + } else { + if (mtu > VMXNET3_MAX_MTU) { + PMD_DRV_LOG(ERR, "MTU %d too large in device version v%d", + mtu, hw->version); + return -EINVAL; + } + } + + dev->data->mtu = mtu; + /* update max frame size */ + dev->data->dev_conf.rxmode.mtu = frame_size; + + if (dev->data->dev_started == 0) + return 0; + + /* changing mtu for vmxnet3 pmd does not require a restart + * as it does not need to repopulate the rx rings to support + * different mtu size. We stop and restart the device here + * just to pass the mtu info to the backend. + */ + vmxnet3_dev_stop(dev); + vmxnet3_dev_start(dev); - rte_ether_addr_copy(mac_addr, (struct rte_ether_addr *)(hw->perm_addr)); - vmxnet3_write_mac(hw, mac_addr->addr_bytes); return 0; } @@ -1668,11 +1765,14 @@ vmxnet3_interrupt_handler(void *param) { struct rte_eth_dev *dev = param; struct vmxnet3_hw *hw = dev->data->dev_private; - Vmxnet3_DSDevRead *devRead = &hw->shared->devRead; uint32_t events; + uint8 *eventIntrIdx; + uint32 *intrCtrl; PMD_INIT_FUNC_TRACE(); - vmxnet3_disable_intr(hw, devRead->intrConf.eventIntrIdx); + + vmxnet3_get_intr_ctrl_ev(hw, &eventIntrIdx, &intrCtrl); + vmxnet3_disable_intr(hw, *eventIntrIdx); events = VMXNET3_READ_BAR1_REG(hw, VMXNET3_REG_ECR); if (events == 0) @@ -1681,7 +1781,7 @@ vmxnet3_interrupt_handler(void *param) RTE_LOG(DEBUG, PMD, "Reading events: 0x%X", events); vmxnet3_process_events(dev); done: - vmxnet3_enable_intr(hw, devRead->intrConf.eventIntrIdx); + vmxnet3_enable_intr(hw, *eventIntrIdx); } static int diff --git a/drivers/net/vmxnet3/vmxnet3_ethdev.h b/drivers/net/vmxnet3/vmxnet3_ethdev.h index ceaeb66392..5a303717b1 100644 --- a/drivers/net/vmxnet3/vmxnet3_ethdev.h +++ b/drivers/net/vmxnet3/vmxnet3_ethdev.h @@ -70,7 +70,7 @@ struct vmxnet3_intr { enum vmxnet3_intr_type type; /* MSI-X, MSI, or INTx? */ uint8_t num_intrs; /* # of intr vectors */ uint8_t event_intr_idx; /* idx of the intr vector for event */ - uint8_t mod_levels[VMXNET3_MAX_MSIX_VECT]; /* moderation level */ + uint8_t mod_levels[VMXNET3_EXT_MAX_INTRS]; /* moderation level */ bool lsc_only; /* no Rx queue interrupt */ }; @@ -108,6 +108,7 @@ struct vmxnet3_hw { uint64_t queueDescPA; uint16_t queue_desc_len; uint16_t mtu; + bool queuesExtEnabled; VMXNET3_RSSConf *rss_conf; uint64_t rss_confPA; @@ -117,19 +118,20 @@ struct vmxnet3_hw { Vmxnet3_MemRegs *memRegs; uint64_t memRegsPA; #define VMXNET3_VFT_TABLE_SIZE (VMXNET3_VFT_SIZE * sizeof(uint32_t)) - UPT1_TxStats saved_tx_stats[VMXNET3_MAX_TX_QUEUES]; - UPT1_RxStats saved_rx_stats[VMXNET3_MAX_RX_QUEUES]; - + UPT1_TxStats saved_tx_stats[VMXNET3_EXT_MAX_TX_QUEUES]; + UPT1_RxStats saved_rx_stats[VMXNET3_EXT_MAX_RX_QUEUES]; UPT1_TxStats snapshot_tx_stats[VMXNET3_MAX_TX_QUEUES]; UPT1_RxStats snapshot_rx_stats[VMXNET3_MAX_RX_QUEUES]; }; +#define VMXNET3_REV_6 5 /* Vmxnet3 Rev. 6 */ #define VMXNET3_REV_5 4 /* Vmxnet3 Rev. 5 */ #define VMXNET3_REV_4 3 /* Vmxnet3 Rev. 4 */ #define VMXNET3_REV_3 2 /* Vmxnet3 Rev. 3 */ #define VMXNET3_REV_2 1 /* Vmxnet3 Rev. 2 */ #define VMXNET3_REV_1 0 /* Vmxnet3 Rev. 1 */ +#define VMXNET3_VERSION_GE_6(hw) ((hw)->version >= VMXNET3_REV_6 + 1) #define VMXNET3_VERSION_GE_5(hw) ((hw)->version >= VMXNET3_REV_5 + 1) #define VMXNET3_VERSION_GE_4(hw) ((hw)->version >= VMXNET3_REV_4 + 1) #define VMXNET3_VERSION_GE_3(hw) ((hw)->version >= VMXNET3_REV_3 + 1) diff --git a/drivers/net/vmxnet3/vmxnet3_rxtx.c b/drivers/net/vmxnet3/vmxnet3_rxtx.c index e15b377d8c..a6665fbf70 100644 --- a/drivers/net/vmxnet3/vmxnet3_rxtx.c +++ b/drivers/net/vmxnet3/vmxnet3_rxtx.c @@ -1400,7 +1400,7 @@ vmxnet3_rss_configure(struct rte_eth_dev *dev) /* loading hashKeySize */ dev_rss_conf->hashKeySize = VMXNET3_RSS_MAX_KEY_SIZE; /* loading indTableSize: Must not exceed VMXNET3_RSS_MAX_IND_TABLE_SIZE (128)*/ - dev_rss_conf->indTableSize = (uint16_t)(hw->num_rx_queues * 4); + dev_rss_conf->indTableSize = (uint16_t)(MAX_RX_QUEUES * 4); if (port_rss_conf->rss_key == NULL) { /* Default hash key */ -- 2.17.1