From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 2F7EDA0C45; Wed, 20 Oct 2021 05:20:11 +0200 (CEST) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 56D9241164; Wed, 20 Oct 2021 05:20:05 +0200 (CEST) Received: from NAM10-MW2-obe.outbound.protection.outlook.com (mail-mw2nam10on2070.outbound.protection.outlook.com [40.107.94.70]) by mails.dpdk.org (Postfix) with ESMTP id C6D1440142 for ; Wed, 20 Oct 2021 05:20:03 +0200 (CEST) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=GpR7m6SfFMEc+KNTdNI173Aobmfi44gOvRvBpw30WMFFvLgtzRkNQjbUgdgdpnDzOlMwjX206w0GMHUXinRpTDVgsKdAWb8jSRgmvPSonYTExB0iyoNb5vjmsnvhZfAgm9vYgFnxPVsmxN+VIfAGKbeDnRaemaYRSXKFPb0jy6vsG844u8EcT5H1B42oNDWKyCu9v6aS3Y6OHAmHm47s/FxF9vdo4Aw4wp6he7YbTfecMWEsjz/z2yKsbzJjY2NxeOBeadh6aoZ/Joiuw2ln8dQqz/EDX+zUdNU4lqS4vD9vqWIRF98ZsFD0Am/yBd8dK4DaB5JhHokyTVllvC8MsA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=zPy5c36WJKX5dtU7rtIFTiDmuvq5qExk0W+A8FGJuLI=; b=XyPYrIXdm+XHkgcCvSn0m8U6tYt1wKFjC2MWohwH1YIG4tZ7t/oHUt9CPSLfgpsl+YZJm3KkqT1IUf84dSP6GfF9RahdzipIzPZ2sv0pHfXybi7kLuj3AP6tvOMbH76VVyJoqriZfu02KHB0/QlcaJHLJmKqap1EVWe045tZTCixjDL+yuyuJsOZMbT/ifGNcRYa4ZxB06XwUUHennV3OEwDXOkHQ/VtGPyNYVRxKC6M58KpOd+SWRnEBHDfKqjsiaVEQweS/aRMHSaWbM57jirNqcjxPwnlGeckqTUAXXBzt3r/DINARug2NNHNTOVl9OvmMUF9XB+GYuGkNUHeEg== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 216.228.112.34) smtp.rcpttodomain=dpdk.org smtp.mailfrom=nvidia.com; dmarc=pass (p=quarantine sp=quarantine pct=100) action=none header.from=nvidia.com; dkim=none (message not signed); arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=zPy5c36WJKX5dtU7rtIFTiDmuvq5qExk0W+A8FGJuLI=; b=KlUFwq2OQSJLF8C9rDPFkK4ax2+RrkRk7u1ibbJHZqLCUi8YHiKMCp3fjp7Ak5zd34qppnSpll8gNyu8onVtOQow8uLaYRfBenQnOc36nLvbfb2amfdRJtPPTYkQrmuLqI2C0murVMLNEEyM4wdobgEk/tKhngt6jOs3T4T1LD0BV4mpW90nh1PFTXMFS7vDzbsQgU9HNTVK6/9s7eO+tKMxbn8eP1ji4+ms1Ka7RQdfFLjyfYfUYqtEFjdjXk3Lz03nMQX+ZjLGRbzs/QYYI1deTsCxZ7wnzwHWC7uczRXT/rLp8tp89di67zCl6OgztynC2KLTXjFDiXydIOGDrg== Received: from BN9P221CA0010.NAMP221.PROD.OUTLOOK.COM (2603:10b6:408:10a::13) by MN2PR12MB3437.namprd12.prod.outlook.com (2603:10b6:208:c3::20) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4608.16; Wed, 20 Oct 2021 03:20:02 +0000 Received: from BN8NAM11FT063.eop-nam11.prod.protection.outlook.com (2603:10b6:408:10a:cafe::6f) by BN9P221CA0010.outlook.office365.com (2603:10b6:408:10a::13) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4608.18 via Frontend Transport; Wed, 20 Oct 2021 03:20:02 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 216.228.112.34) smtp.mailfrom=nvidia.com; dpdk.org; dkim=none (message not signed) header.d=none;dpdk.org; dmarc=pass action=none header.from=nvidia.com; Received-SPF: Pass (protection.outlook.com: domain of nvidia.com designates 216.228.112.34 as permitted sender) receiver=protection.outlook.com; client-ip=216.228.112.34; helo=mail.nvidia.com; Received: from mail.nvidia.com (216.228.112.34) by BN8NAM11FT063.mail.protection.outlook.com (10.13.177.110) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384) id 15.20.4608.15 via Frontend Transport; Wed, 20 Oct 2021 03:20:01 +0000 Received: from nvidia.com (172.20.187.6) by HQMAIL107.nvidia.com (172.20.187.13) with Microsoft SMTP Server (TLS) id 15.0.1497.18; Wed, 20 Oct 2021 03:19:55 +0000 From: Rongwei Liu To: , , , , Ray Kinsella CC: , , Jiawei Wang Date: Wed, 20 Oct 2021 06:19:37 +0300 Message-ID: <20211020031938.3190843-2-rongweil@nvidia.com> X-Mailer: git-send-email 2.27.0 In-Reply-To: <20211020031938.3190843-1-rongweil@nvidia.com> References: <20211020031938.3190843-1-rongweil@nvidia.com> MIME-Version: 1.0 Content-Transfer-Encoding: 8bit Content-Type: text/plain X-Originating-IP: [172.20.187.6] X-ClientProxiedBy: HQMAIL111.nvidia.com (172.20.187.18) To HQMAIL107.nvidia.com (172.20.187.13) X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: f284f11e-bb67-455d-d7c0-08d99378810d X-MS-TrafficTypeDiagnostic: MN2PR12MB3437: X-LD-Processed: 43083d15-7273-40c1-b7db-39efd9ccc17a,ExtAddr X-Microsoft-Antispam-PRVS: X-MS-Oob-TLC-OOBClassifiers: OLM:7219; X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: Kn2UDS/y4oV8jco5dLJ2Wnttrt+tcR+wPQNP8rIF2F60NUvi3/b9vEMXHmDT1PL8+mz0JFH9zTn3njefrFE8DlM4vqNZ+JHNmJFfau/BF0bAS3bFYc6g3JgJbQQG0KgbsPfWzujYIqciH+/15bQSYoxAlAiSFeLJ9SxRCbvgU/hgtA/FG+c+YLZeRKO0X5jN66oMdxyNO9CtCF+sZI6TgvAzY50uvVuq4wTzMQtC+JCacD9pExlU3rChLQwxwmMGA1tir8ChhsnpWYw4vx5PVVrQqiL4RSMRZA8tJ9MMHjy4Dt4x0x9WrK1IPExrKg7FGV0ZzuPCX4rGCZItYwo9ZtrVaVsyI6CAMMdBOCAJOGC/+/gf4fQ/7Uo8SIzu2yRAWkVKpDQjTWk6/zWp1/BlXbcr/hZy9V+jT4PfA98v4UX++7UDHumHGedrVMkmc5zLnZCxE1ASe5X+DLV9d5VLr3331WC6jIiXbXG/spSoSMjFtn0lFNQIBueV1DvmoT45oldaeq7n0GFU4fT9KjinuHKkAhuA9VHbp0jHTfESAQurgjkI7giQqvwdXUyr4kCdVv4zqhySF6FQ7XykjIN+VoA3FSOzL+qhFNvu8paGGL+RlYH+Gx6d6XwSx5rlzapU8nI55AgUjoxT/LYMsFJ3V+FlV8taYKwsNDQ3J643ppXf43Xi6BWlM1xWRG3GYm5GdINYTeYoQ8MjtKjejrN31w== X-Forefront-Antispam-Report: CIP:216.228.112.34; CTRY:US; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:mail.nvidia.com; PTR:schybrid03.nvidia.com; CAT:NONE; SFS:(4636009)(36840700001)(46966006)(110136005)(54906003)(336012)(36906005)(316002)(6286002)(6666004)(1076003)(26005)(2616005)(70206006)(70586007)(186003)(8676002)(8936002)(426003)(82310400003)(4326008)(16526019)(7636003)(2906002)(86362001)(7696005)(36756003)(5660300002)(356005)(55016002)(508600001)(47076005)(36860700001)(83380400001)(107886003); DIR:OUT; SFP:1101; X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 20 Oct 2021 03:20:01.5835 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: f284f11e-bb67-455d-d7c0-08d99378810d X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=43083d15-7273-40c1-b7db-39efd9ccc17a; Ip=[216.228.112.34]; Helo=[mail.nvidia.com] X-MS-Exchange-CrossTenant-AuthSource: BN8NAM11FT063.eop-nam11.prod.protection.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: MN2PR12MB3437 Subject: [dpdk-dev] [PATCH 1/2] common/mlx5: support lag context query X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" Added a new api mlx5_devx_cmd_query_lag() to query lag property from firmware including state/affinity/mode etc. Signed-off-by: Jiawei Wang Signed-off-by: Rongwei Liu Acked-by: Matan Azrad --- drivers/common/mlx5/mlx5_devx_cmds.c | 40 +++++++++++++++++++++++++ drivers/common/mlx5/mlx5_devx_cmds.h | 13 ++++++++ drivers/common/mlx5/mlx5_prm.h | 45 +++++++++++++++++++++++++++- drivers/common/mlx5/version.map | 1 + 4 files changed, 98 insertions(+), 1 deletion(-) diff --git a/drivers/common/mlx5/mlx5_devx_cmds.c b/drivers/common/mlx5/mlx5_devx_cmds.c index 6538bce57b..fb7c8e986f 100644 --- a/drivers/common/mlx5/mlx5_devx_cmds.c +++ b/drivers/common/mlx5/mlx5_devx_cmds.c @@ -2800,3 +2800,43 @@ mlx5_devx_cmd_create_crypto_login_obj(void *ctx, crypto_login_obj->id = MLX5_GET(general_obj_out_cmd_hdr, out, obj_id); return crypto_login_obj; } + +/** + * Query LAG context. + * + * @param[in] ctx + * Pointer to ibv_context, returned from mlx5dv_open_device. + * @param[out] lag_ctx + * Pointer to struct mlx5_devx_lag_context, to be set by the routine. + * + * @return + * 0 on success, a negative value otherwise. + */ +int +mlx5_devx_cmd_query_lag(void *ctx, + struct mlx5_devx_lag_context *lag_ctx) +{ + uint32_t in[MLX5_ST_SZ_DW(query_lag_in)] = {0}; + uint32_t out[MLX5_ST_SZ_DW(query_lag_out)] = {0}; + void *lctx; + int rc; + + MLX5_SET(query_lag_in, in, opcode, MLX5_CMD_OP_QUERY_LAG); + rc = mlx5_glue->devx_general_cmd(ctx, in, sizeof(in), out, sizeof(out)); + if (rc) + goto error; + lctx = MLX5_ADDR_OF(query_lag_out, out, context); + lag_ctx->fdb_selection_mode = MLX5_GET(lag_context, lctx, + fdb_selection_mode); + lag_ctx->port_select_mode = MLX5_GET(lag_context, lctx, + port_select_mode); + lag_ctx->lag_state = MLX5_GET(lag_context, lctx, lag_state); + lag_ctx->tx_remap_affinity_2 = MLX5_GET(lag_context, lctx, + tx_remap_affinity_2); + lag_ctx->tx_remap_affinity_1 = MLX5_GET(lag_context, lctx, + tx_remap_affinity_1); + return 0; +error: + rc = (rc > 0) ? -rc : rc; + return rc; +} diff --git a/drivers/common/mlx5/mlx5_devx_cmds.h b/drivers/common/mlx5/mlx5_devx_cmds.h index 6948cadd37..5e4f3b749e 100644 --- a/drivers/common/mlx5/mlx5_devx_cmds.h +++ b/drivers/common/mlx5/mlx5_devx_cmds.h @@ -197,6 +197,15 @@ struct mlx5_hca_attr { uint32_t umr_indirect_mkey_disabled:1; }; +/* LAG Context. */ +struct mlx5_devx_lag_context { + uint32_t fdb_selection_mode:1; + uint32_t port_select_mode:3; + uint32_t lag_state:3; + uint32_t tx_remap_affinity_1:4; + uint32_t tx_remap_affinity_2:4; +}; + struct mlx5_devx_wq_attr { uint32_t wq_type:4; uint32_t wq_signature:1; @@ -681,4 +690,8 @@ struct mlx5_devx_obj * mlx5_devx_cmd_create_crypto_login_obj(void *ctx, struct mlx5_devx_crypto_login_attr *attr); +__rte_internal +int +mlx5_devx_cmd_query_lag(void *ctx, + struct mlx5_devx_lag_context *lag_ctx); #endif /* RTE_PMD_MLX5_DEVX_CMDS_H_ */ diff --git a/drivers/common/mlx5/mlx5_prm.h b/drivers/common/mlx5/mlx5_prm.h index 54e62aa153..eab80eaead 100644 --- a/drivers/common/mlx5/mlx5_prm.h +++ b/drivers/common/mlx5/mlx5_prm.h @@ -1048,6 +1048,7 @@ enum { MLX5_CMD_OP_DEALLOC_PD = 0x801, MLX5_CMD_OP_ACCESS_REGISTER = 0x805, MLX5_CMD_OP_ALLOC_TRANSPORT_DOMAIN = 0x816, + MLX5_CMD_OP_QUERY_LAG = 0x842, MLX5_CMD_OP_CREATE_TIR = 0x900, MLX5_CMD_OP_MODIFY_TIR = 0x901, MLX5_CMD_OP_CREATE_SQ = 0X904, @@ -1507,7 +1508,8 @@ struct mlx5_ifc_cmd_hca_cap_bits { u8 uar_4k[0x1]; u8 reserved_at_241[0x9]; u8 uar_sz[0x6]; - u8 reserved_at_250[0x8]; + u8 port_selection_cap[0x1]; + u8 reserved_at_251[0x7]; u8 log_pg_sz[0x8]; u8 bf[0x1]; u8 driver_version[0x1]; @@ -1974,6 +1976,14 @@ struct mlx5_ifc_query_nic_vport_context_in_bits { u8 reserved_at_68[0x18]; }; +/* + * lag_tx_port_affinity: 0 auto-selection, 1 PF1, 2 PF2 vice versa. + * Each TIS binds to one PF by setting lag_tx_port_affinity (>0). + * Once LAG enabled, we create multiple TISs and bind each one to + * different PFs, then TIS[i] gets affinity i+1 and goes to PF i+1. + */ +#define MLX5_IFC_LAG_MAP_TIS_AFFINITY(index, num) ((num) ? \ + (index) % (num) + 1 : 0) struct mlx5_ifc_tisc_bits { u8 strict_lag_tx_port_affinity[0x1]; u8 reserved_at_1[0x3]; @@ -2007,6 +2017,39 @@ struct mlx5_ifc_query_tis_in_bits { u8 reserved_at_60[0x20]; }; +/* port_select_mode definition. */ +enum mlx5_lag_mode_type { + MLX5_LAG_MODE_TIS = 0, + MLX5_LAG_MODE_HASH = 1, +}; + +struct mlx5_ifc_lag_context_bits { + u8 fdb_selection_mode[0x1]; + u8 reserved_at_1[0x14]; + u8 port_select_mode[0x3]; + u8 reserved_at_18[0x5]; + u8 lag_state[0x3]; + u8 reserved_at_20[0x14]; + u8 tx_remap_affinity_2[0x4]; + u8 reserved_at_38[0x4]; + u8 tx_remap_affinity_1[0x4]; +}; + +struct mlx5_ifc_query_lag_in_bits { + u8 opcode[0x10]; + u8 uid[0x10]; + u8 reserved_at_20[0x10]; + u8 op_mod[0x10]; + u8 reserved_at_40[0x40]; +}; + +struct mlx5_ifc_query_lag_out_bits { + u8 status[0x8]; + u8 reserved_at_8[0x18]; + u8 syndrome[0x20]; + struct mlx5_ifc_lag_context_bits context; +}; + struct mlx5_ifc_alloc_transport_domain_out_bits { u8 status[0x8]; u8 reserved_at_8[0x18]; diff --git a/drivers/common/mlx5/version.map b/drivers/common/mlx5/version.map index d3c5040aac..95f8bddb94 100644 --- a/drivers/common/mlx5/version.map +++ b/drivers/common/mlx5/version.map @@ -53,6 +53,7 @@ INTERNAL { mlx5_devx_cmd_modify_virtq; mlx5_devx_cmd_qp_query_tis_td; mlx5_devx_cmd_query_hca_attr; + mlx5_devx_cmd_query_lag; mlx5_devx_cmd_query_parse_samples; mlx5_devx_cmd_query_virtio_q_counters; # WINDOWS_NO_EXPORT mlx5_devx_cmd_query_virtq; -- 2.27.0