From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 07CBBA0C48 for ; Thu, 8 Jul 2021 16:15:39 +0200 (CEST) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 01A424014F; Thu, 8 Jul 2021 16:15:39 +0200 (CEST) Received: from NAM12-BN8-obe.outbound.protection.outlook.com (mail-bn8nam12on2040.outbound.protection.outlook.com [40.107.237.40]) by mails.dpdk.org (Postfix) with ESMTP id 2A1544014F for ; Thu, 8 Jul 2021 16:15:38 +0200 (CEST) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=k74jbqJii5XzIgg4zdcZ34TMfPiT8et9LwfIB2apquBEDRHh4HM0A+G27WQ+fr2woFnZKUjKSQwz8Fui3sZY2fzHVcZVzc6Sm9GeY1nQ4h9v4387HgV+OMLTHENSZxDvhoTIg8+cwgrA5E2scRMWDDFV9Z3Qr+RTmotRMzyaPmk8TlkL6sX1g5xZ0RuAqCu+FTrcO1Onz7ngf1SlsHfyphMEC3RLFtopzKxEKQCtD7PsumXVWaMpYlrjt8DI0hPkD30DO/FGkUIeSzLvcPJ0Pi3R1rE/UhSTrnqW2+sFanlRWGDeHlHHpOyVVNTsANNX1u4koHhvSPYaTD8bd4t5Rg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=2oW0pP8YLVm6yeW4ccP3oLgVMagkcCG3zQSz1+3TscM=; b=iF7SNHjirg4t/fv2oaZJwZ8ikjw7LCxHErtJN5Xhj1V/of0r67eyUYNmYTQb8kybM7/BghCYGRjiO7MaOiz9FJ9hC6dd6GoeDmQZRvCvecum8GURFbaaKs89S26cDgqJegwlqwzxvK4hyeIRkkSekuvV7UlOEBylbCmfdg8YrJaFUZ0643oAJ1HVnzc7+hc6BIXIjp9u7Wk8FAo39PlG7i2Foi/PgQbUTxRbky88Hj0HALOekb71MgbbPASBGqwBN8W/Jf4ebCGWWklnS0khh3KB9t3WuWqxalCWMHjIhkmWxV/T7HA3qVdc+sSInStUPTTy75i2RCe+RYLFSyGmcQ== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 216.228.112.34) smtp.rcpttodomain=dpdk.org smtp.mailfrom=nvidia.com; dmarc=pass (p=none sp=none pct=100) action=none header.from=nvidia.com; dkim=none (message not signed); arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=2oW0pP8YLVm6yeW4ccP3oLgVMagkcCG3zQSz1+3TscM=; b=mTLRv4koWD4BENLgnxCZNpekLT0kuiT0X/VzFmVKvMRfX5OAYCFp9tx1DBrtp0kot5hraxZrN8Fkccejeb3dmTTy2xwm5QkEijKY7gneI4dvic/WRTKvlKNRb/2zDpOHOvWiLnFo0uNCahp0En8sartWFGjTKHtqd3MeCvirxZhM1nWv6aD3YsS4Oz+uj3YS64zn9T6EwPo8CNEeM/n4cLeAP413oHdxDRayenWkIuhk426o7N/TJivNsfBZKWV/omJJIsTgaQMDbMvzSumKVlH847ZVoUNaXEtbJ/PtpaKbiXQWT3p2k7rONFvWmYYXb7s/8tupcoZsZ3LqWlcdGA== Received: from DM6PR02CA0056.namprd02.prod.outlook.com (2603:10b6:5:177::33) by BN6PR12MB1522.namprd12.prod.outlook.com (2603:10b6:405:11::14) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4308.20; Thu, 8 Jul 2021 14:15:35 +0000 Received: from DM6NAM11FT036.eop-nam11.prod.protection.outlook.com (2603:10b6:5:177:cafe::7a) by DM6PR02CA0056.outlook.office365.com (2603:10b6:5:177::33) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4308.19 via Frontend Transport; Thu, 8 Jul 2021 14:15:35 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 216.228.112.34) smtp.mailfrom=nvidia.com; dpdk.org; dkim=none (message not signed) header.d=none;dpdk.org; dmarc=pass action=none header.from=nvidia.com; Received-SPF: Pass (protection.outlook.com: domain of nvidia.com designates 216.228.112.34 as permitted sender) receiver=protection.outlook.com; client-ip=216.228.112.34; helo=mail.nvidia.com; Received: from mail.nvidia.com (216.228.112.34) by DM6NAM11FT036.mail.protection.outlook.com (10.13.172.64) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384) id 15.20.4308.20 via Frontend Transport; Thu, 8 Jul 2021 14:15:34 +0000 Received: from nvidia.com (172.20.187.5) by HQMAIL107.nvidia.com (172.20.187.13) with Microsoft SMTP Server (TLS) id 15.0.1497.2; Thu, 8 Jul 2021 14:15:32 +0000 From: Matan Azrad To: , Xueming Li Date: Thu, 8 Jul 2021 17:15:12 +0300 Message-ID: <20210708141512.2255977-1-matan@nvidia.com> X-Mailer: git-send-email 2.25.1 MIME-Version: 1.0 Content-Transfer-Encoding: 8bit Content-Type: text/plain X-Originating-IP: [172.20.187.5] X-ClientProxiedBy: HQMAIL101.nvidia.com (172.20.187.10) To HQMAIL107.nvidia.com (172.20.187.13) X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: c292a5a7-3b07-439e-e3b3-08d9421ada73 X-MS-TrafficTypeDiagnostic: BN6PR12MB1522: X-Microsoft-Antispam-PRVS: X-MS-Oob-TLC-OOBClassifiers: OLM:255; X-MS-Exchange-SenderADCheck: 1 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: nGsguuy6l6NVYTDefcEUbllWua0SnM3I9ntLOqOC7baM5hu1j0pkVl23iaZa4j6dS7zoauaZCS42ExfCCoFnzior1L0ySKPhYYOPLTjt5krBETcL7AO0FAzx9ti7n4GPno/gNQFVaIjd83DN6KHEaWbt7mdURV4Wkjx/rGucoTEECRfZfqA8Ft+mOSQmWs1D8RmOBmxPluGXPi46ZI3JGxVWWUuZoV8KNTwmw7gYZ6HI3pBf9MBS3j95YTUXKw/INDSnOSoXD1ip58TZo4WeqjTgqGzbCo/k/RUcu81kVJgZPeg1vI+BIWs7MU8tHvZvsvoItTtC8qQbNIlGhgFBfgKmfc3rzyBn+0QTpTzNJhkqJVe6RxEM8U69cs/vwBNSou8CqKBe9zYZGQ+ImqL1TVyvUJdM9WKBeypP23M6hNiDAj4jZIyHY4HBRCsH1jdVKKqiB53QTx/u7BxE9nM9oMnzkjs9KY8AcxrpSQ3IjkuWoHH+c76iM5t+qCPMOm1P22gjHZIE1FisPtIcLsMAPazc0srOBFx8s5tWE7iAk2RwBy+OOgLY68KOLObTi5rJ/G1mx9A8KSWh7xFxYv3L9JQzelgyYjqjYaGDf6yXZA+xk8MzhVxz17G2amMnWAXx/8RD1veZjd3SbilCH0gBttl0d8b4MXE2JBHvAuVapSRdpO6cLEewR3B+iazhMYSTlJ1PIllkT6GNwc/1raGm9g== X-Forefront-Antispam-Report: CIP:216.228.112.34; CTRY:US; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:mail.nvidia.com; PTR:schybrid03.nvidia.com; CAT:NONE; SFS:(4636009)(39860400002)(346002)(136003)(396003)(376002)(36840700001)(46966006)(2616005)(36906005)(70206006)(5660300002)(478600001)(26005)(16526019)(70586007)(316002)(36860700001)(8936002)(82310400003)(426003)(1076003)(186003)(8676002)(86362001)(47076005)(83380400001)(82740400003)(36756003)(55016002)(7696005)(2906002)(356005)(6666004)(110136005)(6286002)(7636003)(6636002)(336012); DIR:OUT; SFP:1101; X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 08 Jul 2021 14:15:34.9967 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: c292a5a7-3b07-439e-e3b3-08d9421ada73 X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=43083d15-7273-40c1-b7db-39efd9ccc17a; Ip=[216.228.112.34]; Helo=[mail.nvidia.com] X-MS-Exchange-CrossTenant-AuthSource: DM6NAM11FT036.eop-nam11.prod.protection.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: BN6PR12MB1522 Subject: [dpdk-stable] [PATCH] net/mlx5: fix imissed statistics X-BeenThere: stable@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: patches for DPDK stable branches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: stable-bounces@dpdk.org Sender: "stable" upstream commit e6988afdc75a628bba20bbc291831e63902b20a6 ] The imissed port statistic counts packets that were dropped by the device Rx queues. In mlx5, the imissed counter summarizes 2 counters: - packets dropped by the SW queue handling counted by SW. - packets dropped by the HW queues due to "out of buffer" events detected when no SW buffer is available for the incoming packets. There is HW counter object that should be created per device, and all the Rx queues should be assigned to this counter in configuration time. This part was missed when the Rx queues were created by DevX what remained the "out of buffer" counter clean forever in this case. Add 2 options to assign the DevX Rx queues to queue counter: - Create queue counter per device by DevX and assign all the queues to it. - Query the kernel counter and assign all the queues to it. Use the first option by default and if it is failed, fallback to the second option. Fixes: e79c9be91515 ("net/mlx5: support Rx hairpin queues") Fixes: dc9ceff73c99 ("net/mlx5: create advanced RxQ via DevX") Cc: stable@dpdk.org Signed-off-by: Matan Azrad --- drivers/net/mlx5/linux/mlx5_os.c | 52 ++++++++++++++++++++++++++++++++ drivers/net/mlx5/mlx5.c | 4 +++ drivers/net/mlx5/mlx5.h | 2 ++ drivers/net/mlx5/mlx5_devx.c | 2 ++ 4 files changed, 60 insertions(+) diff --git a/drivers/net/mlx5/linux/mlx5_os.c b/drivers/net/mlx5/linux/mlx5_os.c index 800df92997..78633ab71b 100644 --- a/drivers/net/mlx5/linux/mlx5_os.c +++ b/drivers/net/mlx5/linux/mlx5_os.c @@ -651,6 +651,53 @@ mlx5_flow_counter_mode_config(struct rte_eth_dev *dev __rte_unused) #endif } +static void +mlx5_queue_counter_id_prepare(struct rte_eth_dev *dev) +{ + struct mlx5_priv *priv = dev->data->dev_private; + void *ctx = priv->sh->ctx; + + priv->q_counters = mlx5_devx_cmd_queue_counter_alloc(ctx); + if (!priv->q_counters) { + struct ibv_cq *cq = mlx5_glue->create_cq(ctx, 1, NULL, NULL, 0); + struct ibv_wq *wq; + + DRV_LOG(DEBUG, "Port %d queue counter object cannot be created " + "by DevX - fall-back to use the kernel driver global " + "queue counter.", dev->data->port_id); + /* Create WQ by kernel and query its queue counter ID. */ + if (cq) { + wq = mlx5_glue->create_wq(ctx, + &(struct ibv_wq_init_attr){ + .wq_type = IBV_WQT_RQ, + .max_wr = 1, + .max_sge = 1, + .pd = priv->sh->pd, + .cq = cq, + }); + if (wq) { + /* Counter is assigned only on RDY state. */ + int ret = mlx5_glue->modify_wq(wq, + &(struct ibv_wq_attr){ + .attr_mask = IBV_WQ_ATTR_STATE, + .wq_state = IBV_WQS_RDY, + }); + + if (ret == 0) + mlx5_devx_cmd_wq_query(wq, + &priv->counter_set_id); + claim_zero(mlx5_glue->destroy_wq(wq)); + } + claim_zero(mlx5_glue->destroy_cq(cq)); + } + } else { + priv->counter_set_id = priv->q_counters->id; + } + if (priv->counter_set_id == 0) + DRV_LOG(INFO, "Part of the port %d statistics will not be " + "available.", dev->data->port_id); +} + /** * Spawn an Ethernet device from Verbs information. * @@ -1512,6 +1559,7 @@ mlx5_dev_spawn(struct rte_device *dpdk_dev, mlx5_rxq_ibv_obj_dummy_lb_create; priv->obj_ops.lb_dummy_queue_release = mlx5_rxq_ibv_obj_dummy_lb_release; + mlx5_queue_counter_id_prepare(eth_dev); } else { priv->obj_ops = ibv_obj_ops; } @@ -2446,6 +2494,10 @@ mlx5_os_read_dev_stat(struct mlx5_priv *priv, const char *ctr_name, int fd; if (priv->sh) { + if (priv->q_counters != NULL && + strcmp(ctr_name, "out_of_buffer") == 0) + return mlx5_devx_cmd_queue_counter_query(priv->sh->ctx, + 0, (uint32_t *)stat); MKSTR(path, "%s/ports/%d/hw_counters/%s", priv->sh->ibdev_path, priv->dev_port, diff --git a/drivers/net/mlx5/mlx5.c b/drivers/net/mlx5/mlx5.c index 7953a27834..1a874befc3 100644 --- a/drivers/net/mlx5/mlx5.c +++ b/drivers/net/mlx5/mlx5.c @@ -1344,6 +1344,10 @@ mlx5_dev_close(struct rte_eth_dev *dev) priv->txqs = NULL; } mlx5_proc_priv_uninit(dev); + if (priv->q_counters) { + mlx5_devx_cmd_destroy(priv->q_counters); + priv->q_counters = NULL; + } if (priv->drop_queue.hrxq) mlx5_drop_action_destroy(dev); if (priv->mreg_cp_tbl) diff --git a/drivers/net/mlx5/mlx5.h b/drivers/net/mlx5/mlx5.h index 831838768a..2923d36b34 100644 --- a/drivers/net/mlx5/mlx5.h +++ b/drivers/net/mlx5/mlx5.h @@ -1006,6 +1006,8 @@ struct mlx5_priv { LIST_HEAD(fdir, mlx5_fdir_flow) fdir_flows; /* fdir flows. */ rte_spinlock_t shared_act_sl; /* Shared actions spinlock. */ uint32_t rss_shared_actions; /* RSS shared actions. */ + struct mlx5_devx_obj *q_counters; /* DevX queue counter object. */ + uint32_t counter_set_id; /* Queue counter ID to set in DevX objects. */ }; #define PORT_ID(priv) ((priv)->dev_data->port_id) diff --git a/drivers/net/mlx5/mlx5_devx.c b/drivers/net/mlx5/mlx5_devx.c index 74421a6f85..8add3b2cac 100644 --- a/drivers/net/mlx5/mlx5_devx.c +++ b/drivers/net/mlx5/mlx5_devx.c @@ -392,6 +392,7 @@ mlx5_rxq_create_devx_rq_resources(struct rte_eth_dev *dev, uint16_t idx) (uintptr_t)rxq_ctrl->rq_dbr_offset); /* Create RQ using DevX API. */ mlx5_devx_wq_attr_fill(priv, rxq_ctrl, &rq_attr.wq_attr); + rq_attr.counter_set_id = priv->counter_set_id; rq = mlx5_devx_cmd_create_rq(priv->sh->ctx, &rq_attr, rxq_ctrl->socket); if (!rq) goto error; @@ -600,6 +601,7 @@ mlx5_rxq_obj_hairpin_new(struct rte_eth_dev *dev, uint16_t idx) attr.wq_attr.log_hairpin_num_packets = attr.wq_attr.log_hairpin_data_sz - MLX5_HAIRPIN_QUEUE_STRIDE; + attr.counter_set_id = priv->counter_set_id; tmpl->rq = mlx5_devx_cmd_create_rq(priv->sh->ctx, &attr, rxq_ctrl->socket); if (!tmpl->rq) { -- 2.25.1