From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 97279A0547; Thu, 21 Oct 2021 10:57:13 +0200 (CEST) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 4711A411DA; Thu, 21 Oct 2021 10:57:04 +0200 (CEST) Received: from NAM12-MW2-obe.outbound.protection.outlook.com (mail-mw2nam12on2044.outbound.protection.outlook.com [40.107.244.44]) by mails.dpdk.org (Postfix) with ESMTP id 28834411CB for ; Thu, 21 Oct 2021 10:57:01 +0200 (CEST) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=SXH1wygVwWxuaIykF5+X2SK3YESXAMasgD2O/elnXmoOffiff6XHQIU627dJ1M91g5RTemvNOzp8uaJ4XOFj2z2x95pthlD/KD8KUpsMYHx7i7UHG8gJU2kk1rQmDQPuCd9dCi5VKqzFadYhmT54N4HQS4Wh1rtnQ0KFrIDtu4ZWd5x9k/721jCXFPXXEWYhWDdIMVhRDxdYLNkGHX5gFGRr7iQ45506egkA4MuFaAxgnhHQsvW0BwV/HsYRH1i8WjJyFiCGJXMdgS1sS03L4YvWOlQ6okeCxRZcBVDYdCnYM6RzFxeOKkkjR6qcHISQa/s1XsQRNjhRRX2wYBO7KA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=oEbssCLKTFfJbHRWPeG+Dnxdef5ujNCdAeMkHCzW+Nc=; b=ZeQYBGBtcIpRHwLMePvtb0dP1yxvWQ5uVCM58AuTa4UDzehZDt3OFGefaI39yfBBz06/oZfZSaEDET0F45KBGUqzfV1nQfWvwgzrj9PZfk1KDQluPdc7yj9vzG3QX80ZLCigaPcAyNDeY61LEf66BEOdzNlUVLAn6dBExdrmEnaywlSi1d3qUwIc1If4KUibkWJDxZbQKTEw7RvBq5BhwMrk6+crH6WJDtfJNb2NTfdKXs4RbaROr2/6oaPYk3GKgqvITUAked2ZgqNCXeF3AIvYw4/sL/pwLVk3Pk2PtRL7ILuURmdM/u3dbGpjIuP/IjTxZvU2isRPLDTCwyoEmQ== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 216.228.112.34) smtp.rcpttodomain=dpdk.org smtp.mailfrom=nvidia.com; dmarc=pass (p=quarantine sp=quarantine pct=100) action=none header.from=nvidia.com; dkim=none (message not signed); arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=oEbssCLKTFfJbHRWPeG+Dnxdef5ujNCdAeMkHCzW+Nc=; b=i80Tf6hZOUmve8f+2ObsbAr5EP8HJus4iz3ORESPpKgdFVhpwjWP1FaP5VRPtOc5KNt/9wbpupPIZZZgTATVw+7txIrdtLgxGdwFUUAHuBBxc0huQ0bZPCLxNXRLBgsMsoQ3KSUBhmhE2mQaFqlYZnKmh9CduBSHzG2Ztk13DuCG/4S/reZyXlWs7IGoy2FEIyV20d/TRKhIX1a+yt0aqLG3kxsCIidEI8hfpnD0S05piXOBvdQ+Th8O1+CeJULBVJuxxBTbPwMvStItiFRTfGvXetwP07LvsNFw5WvcETpuInD6s3qdrdM3edn02uACSo158uY7SutaL4RK78l/vQ== Received: from DM6PR13CA0016.namprd13.prod.outlook.com (2603:10b6:5:bc::29) by CY4PR12MB1624.namprd12.prod.outlook.com (2603:10b6:910:a::15) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4628.16; Thu, 21 Oct 2021 08:56:55 +0000 Received: from DM6NAM11FT056.eop-nam11.prod.protection.outlook.com (2603:10b6:5:bc:cafe::8d) by DM6PR13CA0016.outlook.office365.com (2603:10b6:5:bc::29) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4649.8 via Frontend Transport; Thu, 21 Oct 2021 08:56:55 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 216.228.112.34) smtp.mailfrom=nvidia.com; dpdk.org; dkim=none (message not signed) header.d=none;dpdk.org; dmarc=pass action=none header.from=nvidia.com; Received-SPF: Pass (protection.outlook.com: domain of nvidia.com designates 216.228.112.34 as permitted sender) receiver=protection.outlook.com; client-ip=216.228.112.34; helo=mail.nvidia.com; Received: from mail.nvidia.com (216.228.112.34) by DM6NAM11FT056.mail.protection.outlook.com (10.13.173.99) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384) id 15.20.4628.16 via Frontend Transport; Thu, 21 Oct 2021 08:56:55 +0000 Received: from nvidia.com (172.20.187.6) by HQMAIL107.nvidia.com (172.20.187.13) with Microsoft SMTP Server (TLS) id 15.0.1497.18; Thu, 21 Oct 2021 08:56:51 +0000 From: Rongwei Liu To: , , , , Ray Kinsella CC: , , Jiawei Wang Date: Thu, 21 Oct 2021 11:56:35 +0300 Message-ID: <20211021085637.3627922-2-rongweil@nvidia.com> X-Mailer: git-send-email 2.27.0 In-Reply-To: <20211021085637.3627922-1-rongweil@nvidia.com> References: <20211021085637.3627922-1-rongweil@nvidia.com> MIME-Version: 1.0 Content-Transfer-Encoding: 8bit Content-Type: text/plain X-Originating-IP: [172.20.187.6] X-ClientProxiedBy: HQMAIL105.nvidia.com (172.20.187.12) To HQMAIL107.nvidia.com (172.20.187.13) X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: 2eef103c-2a05-42a5-ec6f-08d99470bbc3 X-MS-TrafficTypeDiagnostic: CY4PR12MB1624: X-LD-Processed: 43083d15-7273-40c1-b7db-39efd9ccc17a,ExtAddr X-Microsoft-Antispam-PRVS: X-MS-Oob-TLC-OOBClassifiers: OLM:7219; X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: FLkQzVHnH5C9I8UP88Aajc5Up0RIN3Egc42qaOb8KbN2WSl9MFk/b4hPIzB3AwDKhNCtnPEucSBBZ/pFDMBX8itchH5oONUQEb2S9tBbL8ccHwh6K862UxIi44n40g52QmifxJbE1rdXDKleM+lyj0DdS9XccJ8gMjIfgfVckZqEQ9BalwWrFMzqluk2WdKiFvA+FTSZMbwe0VP/gy8jiYe7uCuTtredsMP0hqCooEJCyQrsiLaWgvmNmiOMWJzMXlkw+4LJ4QIFnwhtelMJB76FGk3iyxoXTTnmjxqMjHdMYK4UCvvupxCVstzWtAV54Mz0ziwuogi3F+zSByXSbhxEF4MscsjDWbRZim73zpHD/8Pb2cbdbNca6/8kpujsCPDC6eXD6oWzp8B0U2l13vnAQcMQWp4fV0LxJikhAzcUUuqHk5qP2fs3AvQNcHnRh8uryyTozSSy93zleWkHLHJDi4GvN/nrzKPMwnH3rKL9GB2E+OwwFmtIo9ebBJLkZBN3rgLdlD0c3dhZgxHztKP8rH7EnLpklgxMqD6dY7zX0i4evBNpAlyLZg0aDpugdezeKgACkMjN0Pzl/D9NP69/jBlmYd7ZUsG61JqNzXDTBPhHgqsAcmberD0OsoFw9HxtnMBJxTWU7QYpBLXvEMWMXVgy49biIuCeIX7h4V+7vUb07jTgnm0jTC+dJNpPJbt3ilC1fdsc/SB4RU/vmw== X-Forefront-Antispam-Report: CIP:216.228.112.34; CTRY:US; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:mail.nvidia.com; PTR:schybrid03.nvidia.com; CAT:NONE; SFS:(4636009)(46966006)(36840700001)(70586007)(70206006)(426003)(83380400001)(26005)(186003)(16526019)(508600001)(4326008)(110136005)(82310400003)(6286002)(7696005)(5660300002)(36906005)(7636003)(316002)(6666004)(54906003)(107886003)(47076005)(8936002)(356005)(36860700001)(336012)(55016002)(86362001)(1076003)(2616005)(8676002)(2906002)(36756003); DIR:OUT; SFP:1101; X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 21 Oct 2021 08:56:55.1816 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: 2eef103c-2a05-42a5-ec6f-08d99470bbc3 X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=43083d15-7273-40c1-b7db-39efd9ccc17a; Ip=[216.228.112.34]; Helo=[mail.nvidia.com] X-MS-Exchange-CrossTenant-AuthSource: DM6NAM11FT056.eop-nam11.prod.protection.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: CY4PR12MB1624 Subject: [dpdk-dev] [PATCH v1 1/2] common/mlx5: support lag context query X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" Added a new api mlx5_devx_cmd_query_lag() to query lag property from firmware including state/affinity/mode etc. Signed-off-by: Jiawei Wang Signed-off-by: Rongwei Liu Acked-by: Matan Azrad --- drivers/common/mlx5/mlx5_devx_cmds.c | 40 +++++++++++++++++++++++++ drivers/common/mlx5/mlx5_devx_cmds.h | 13 ++++++++ drivers/common/mlx5/mlx5_prm.h | 45 +++++++++++++++++++++++++++- drivers/common/mlx5/version.map | 1 + 4 files changed, 98 insertions(+), 1 deletion(-) diff --git a/drivers/common/mlx5/mlx5_devx_cmds.c b/drivers/common/mlx5/mlx5_devx_cmds.c index 6538bce57b..fb7c8e986f 100644 --- a/drivers/common/mlx5/mlx5_devx_cmds.c +++ b/drivers/common/mlx5/mlx5_devx_cmds.c @@ -2800,3 +2800,43 @@ mlx5_devx_cmd_create_crypto_login_obj(void *ctx, crypto_login_obj->id = MLX5_GET(general_obj_out_cmd_hdr, out, obj_id); return crypto_login_obj; } + +/** + * Query LAG context. + * + * @param[in] ctx + * Pointer to ibv_context, returned from mlx5dv_open_device. + * @param[out] lag_ctx + * Pointer to struct mlx5_devx_lag_context, to be set by the routine. + * + * @return + * 0 on success, a negative value otherwise. + */ +int +mlx5_devx_cmd_query_lag(void *ctx, + struct mlx5_devx_lag_context *lag_ctx) +{ + uint32_t in[MLX5_ST_SZ_DW(query_lag_in)] = {0}; + uint32_t out[MLX5_ST_SZ_DW(query_lag_out)] = {0}; + void *lctx; + int rc; + + MLX5_SET(query_lag_in, in, opcode, MLX5_CMD_OP_QUERY_LAG); + rc = mlx5_glue->devx_general_cmd(ctx, in, sizeof(in), out, sizeof(out)); + if (rc) + goto error; + lctx = MLX5_ADDR_OF(query_lag_out, out, context); + lag_ctx->fdb_selection_mode = MLX5_GET(lag_context, lctx, + fdb_selection_mode); + lag_ctx->port_select_mode = MLX5_GET(lag_context, lctx, + port_select_mode); + lag_ctx->lag_state = MLX5_GET(lag_context, lctx, lag_state); + lag_ctx->tx_remap_affinity_2 = MLX5_GET(lag_context, lctx, + tx_remap_affinity_2); + lag_ctx->tx_remap_affinity_1 = MLX5_GET(lag_context, lctx, + tx_remap_affinity_1); + return 0; +error: + rc = (rc > 0) ? -rc : rc; + return rc; +} diff --git a/drivers/common/mlx5/mlx5_devx_cmds.h b/drivers/common/mlx5/mlx5_devx_cmds.h index 6948cadd37..5e4f3b749e 100644 --- a/drivers/common/mlx5/mlx5_devx_cmds.h +++ b/drivers/common/mlx5/mlx5_devx_cmds.h @@ -197,6 +197,15 @@ struct mlx5_hca_attr { uint32_t umr_indirect_mkey_disabled:1; }; +/* LAG Context. */ +struct mlx5_devx_lag_context { + uint32_t fdb_selection_mode:1; + uint32_t port_select_mode:3; + uint32_t lag_state:3; + uint32_t tx_remap_affinity_1:4; + uint32_t tx_remap_affinity_2:4; +}; + struct mlx5_devx_wq_attr { uint32_t wq_type:4; uint32_t wq_signature:1; @@ -681,4 +690,8 @@ struct mlx5_devx_obj * mlx5_devx_cmd_create_crypto_login_obj(void *ctx, struct mlx5_devx_crypto_login_attr *attr); +__rte_internal +int +mlx5_devx_cmd_query_lag(void *ctx, + struct mlx5_devx_lag_context *lag_ctx); #endif /* RTE_PMD_MLX5_DEVX_CMDS_H_ */ diff --git a/drivers/common/mlx5/mlx5_prm.h b/drivers/common/mlx5/mlx5_prm.h index 54e62aa153..eab80eaead 100644 --- a/drivers/common/mlx5/mlx5_prm.h +++ b/drivers/common/mlx5/mlx5_prm.h @@ -1048,6 +1048,7 @@ enum { MLX5_CMD_OP_DEALLOC_PD = 0x801, MLX5_CMD_OP_ACCESS_REGISTER = 0x805, MLX5_CMD_OP_ALLOC_TRANSPORT_DOMAIN = 0x816, + MLX5_CMD_OP_QUERY_LAG = 0x842, MLX5_CMD_OP_CREATE_TIR = 0x900, MLX5_CMD_OP_MODIFY_TIR = 0x901, MLX5_CMD_OP_CREATE_SQ = 0X904, @@ -1507,7 +1508,8 @@ struct mlx5_ifc_cmd_hca_cap_bits { u8 uar_4k[0x1]; u8 reserved_at_241[0x9]; u8 uar_sz[0x6]; - u8 reserved_at_250[0x8]; + u8 port_selection_cap[0x1]; + u8 reserved_at_251[0x7]; u8 log_pg_sz[0x8]; u8 bf[0x1]; u8 driver_version[0x1]; @@ -1974,6 +1976,14 @@ struct mlx5_ifc_query_nic_vport_context_in_bits { u8 reserved_at_68[0x18]; }; +/* + * lag_tx_port_affinity: 0 auto-selection, 1 PF1, 2 PF2 vice versa. + * Each TIS binds to one PF by setting lag_tx_port_affinity (>0). + * Once LAG enabled, we create multiple TISs and bind each one to + * different PFs, then TIS[i] gets affinity i+1 and goes to PF i+1. + */ +#define MLX5_IFC_LAG_MAP_TIS_AFFINITY(index, num) ((num) ? \ + (index) % (num) + 1 : 0) struct mlx5_ifc_tisc_bits { u8 strict_lag_tx_port_affinity[0x1]; u8 reserved_at_1[0x3]; @@ -2007,6 +2017,39 @@ struct mlx5_ifc_query_tis_in_bits { u8 reserved_at_60[0x20]; }; +/* port_select_mode definition. */ +enum mlx5_lag_mode_type { + MLX5_LAG_MODE_TIS = 0, + MLX5_LAG_MODE_HASH = 1, +}; + +struct mlx5_ifc_lag_context_bits { + u8 fdb_selection_mode[0x1]; + u8 reserved_at_1[0x14]; + u8 port_select_mode[0x3]; + u8 reserved_at_18[0x5]; + u8 lag_state[0x3]; + u8 reserved_at_20[0x14]; + u8 tx_remap_affinity_2[0x4]; + u8 reserved_at_38[0x4]; + u8 tx_remap_affinity_1[0x4]; +}; + +struct mlx5_ifc_query_lag_in_bits { + u8 opcode[0x10]; + u8 uid[0x10]; + u8 reserved_at_20[0x10]; + u8 op_mod[0x10]; + u8 reserved_at_40[0x40]; +}; + +struct mlx5_ifc_query_lag_out_bits { + u8 status[0x8]; + u8 reserved_at_8[0x18]; + u8 syndrome[0x20]; + struct mlx5_ifc_lag_context_bits context; +}; + struct mlx5_ifc_alloc_transport_domain_out_bits { u8 status[0x8]; u8 reserved_at_8[0x18]; diff --git a/drivers/common/mlx5/version.map b/drivers/common/mlx5/version.map index 5be01505b5..0fda60bd11 100644 --- a/drivers/common/mlx5/version.map +++ b/drivers/common/mlx5/version.map @@ -53,6 +53,7 @@ INTERNAL { mlx5_devx_cmd_modify_virtq; mlx5_devx_cmd_qp_query_tis_td; mlx5_devx_cmd_query_hca_attr; + mlx5_devx_cmd_query_lag; mlx5_devx_cmd_query_parse_samples; mlx5_devx_cmd_query_virtio_q_counters; # WINDOWS_NO_EXPORT mlx5_devx_cmd_query_virtq; -- 2.27.0