From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 9EDD6A04FF; Tue, 24 May 2022 17:21:15 +0200 (CEST) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id C157942BA5; Tue, 24 May 2022 17:21:09 +0200 (CEST) Received: from NAM10-DM6-obe.outbound.protection.outlook.com (mail-dm6nam10on2041.outbound.protection.outlook.com [40.107.93.41]) by mails.dpdk.org (Postfix) with ESMTP id 9985A42BA0 for ; Tue, 24 May 2022 17:21:08 +0200 (CEST) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=m7zaaP4f6DQ4JKrpYA+xJk5dMUItfZXHWX2nLjRt/XZ3+TETNZByh895Monyf4KcsQ8PdUWPkf/gYr08cacic/8EU7wyh5XMux4irBZjkS9lh/FB1p9PbXfS8JQ53Ed15r2G7jcJt+HuKvZCx3vJBbVShzMrYA+5iVvsMcHvp/FyqppKxCQTTvj7pGSQ+JusACWFd9kr2vFJzIZMS5C0wkYzxZhz4edpJ2ZfV9zHJ/b16oP5rP5yLGfs/PZlvp4YmOpKvhfIT2FiL8QR9h4HwkJXz0I1C2j8/Cowlrb1vrkAXQQlGrciPel8dYZkaOS4mn2HOg2TdmvkzKNaNanMlQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=smchEUvmAnQkWebPNartqR9FdG6PK0stx+jv7Nz15/s=; b=SV6iQtwoESNElNOw2nSoiN+IFk1xgJWQLui0oNdY//vvGaLW2EcF/VrK5/TM/8dAIdY6fOeueoO6pD6D2wGLTPuUEJM7AsDBR5h4fO4ViJu+AI2Za9Ws5FAgaKQ08XbA5+kxf4SJbO8nfF7GOFV3iL6VLBOLw1M3QBvS20QlVJpuEXKJwIMZOS+M3l3nG6Rr+paVhzZfUwplkbCsr7zbB921NGyXTmTIttncOuOO2kxJg8vBeUTU8EB13EqqASBRPwPH4rQSD/Lr5Aofah1tf39khu9Jdzk7ZyzA75ji8wBfIDB6OFAYfA9/P6jRcaIAdYaPL5mNiUjETR1xPBgakA== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 12.22.5.235) smtp.rcpttodomain=dpdk.org smtp.mailfrom=nvidia.com; dmarc=pass (p=reject sp=reject pct=100) action=none header.from=nvidia.com; dkim=none (message not signed); arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=smchEUvmAnQkWebPNartqR9FdG6PK0stx+jv7Nz15/s=; b=ubk3cGMfL76/aWWKzWhnPzVptTrTrOBmfYaUVAhnuykaQ4YLgope1kpGzuwdWcTFdnoRUJreEIJqh3nK1mZnAYpxka07EweR4bMMBxrBPpsSfgaxXgDhJyj67z8BcsEhiY4Q66CWLnenpHJoVpX4DH+L5HJ/h+t3yIWXxXHgykRgh01ceR8ExGk1ywK734E4DQUskUxdwCedQ9mV8xq/ScDYUXlONQyi27Xijr/u05KihfXQojUT7isa/lGgtIkZKmKHocM+KnBYkkKWentMij0rgpTXNO5aKD1kwW2ONbzxWn9Hv/mpt3sFDs0NivV03qPNv4OfkZ/+B/6GlgbdtA== Received: from MW4PR03CA0319.namprd03.prod.outlook.com (2603:10b6:303:dd::24) by SA0PR12MB4477.namprd12.prod.outlook.com (2603:10b6:806:92::12) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5273.13; Tue, 24 May 2022 15:21:06 +0000 Received: from CO1NAM11FT011.eop-nam11.prod.protection.outlook.com (2603:10b6:303:dd:cafe::cb) by MW4PR03CA0319.outlook.office365.com (2603:10b6:303:dd::24) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5293.13 via Frontend Transport; Tue, 24 May 2022 15:21:06 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 12.22.5.235) smtp.mailfrom=nvidia.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=nvidia.com; Received-SPF: Pass (protection.outlook.com: domain of nvidia.com designates 12.22.5.235 as permitted sender) receiver=protection.outlook.com; client-ip=12.22.5.235; helo=mail.nvidia.com; pr=C Received: from mail.nvidia.com (12.22.5.235) by CO1NAM11FT011.mail.protection.outlook.com (10.13.175.186) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384) id 15.20.5273.14 via Frontend Transport; Tue, 24 May 2022 15:21:05 +0000 Received: from rnnvmail201.nvidia.com (10.129.68.8) by DRHQMAIL107.nvidia.com (10.27.9.16) with Microsoft SMTP Server (TLS) id 15.0.1497.32; Tue, 24 May 2022 15:21:05 +0000 Received: from nvidia.com (10.126.231.35) by rnnvmail201.nvidia.com (10.129.68.8) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.986.22; Tue, 24 May 2022 08:21:03 -0700 From: Spike Du To: , , , CC: , Subject: [PATCH v3 3/7] ethdev: introduce Rx queue based limit watermark Date: Tue, 24 May 2022 18:20:37 +0300 Message-ID: <20220524152041.737154-4-spiked@nvidia.com> X-Mailer: git-send-email 2.27.0 In-Reply-To: <20220524152041.737154-1-spiked@nvidia.com> References: <20220522055900.417282-1-spiked@nvidia.com> <20220524152041.737154-1-spiked@nvidia.com> MIME-Version: 1.0 Content-Transfer-Encoding: 8bit Content-Type: text/plain X-Originating-IP: [10.126.231.35] X-ClientProxiedBy: rnnvmail202.nvidia.com (10.129.68.7) To rnnvmail201.nvidia.com (10.129.68.8) X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: 1663f3c8-e4af-466e-0381-08da3d9905a3 X-MS-TrafficTypeDiagnostic: SA0PR12MB4477:EE_ X-LD-Processed: 43083d15-7273-40c1-b7db-39efd9ccc17a,ExtAddr X-Microsoft-Antispam-PRVS: X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: rJ2G3XOqvm03vTcb6bq2QoaJrFZEbZQ9lWFxdF2yCPpeccTkfBRyhysMhn/lL3qyqy/iGz7XHW195E4B+pG3YShCshQL4ptnkPVD12oE44bj91It+aQ/lTJFNvh1X5APh5Iw/0mgDUi5wtYcNR81sBF9y9C7u+hcoMYZ3FXQgrVFSya7ymQuQbUsftNYvIg3g0E3zJvIuye9AWepJdOmXnsq0TLTV5brtBgZmVLVLjw+BtpjTTWWuRRSd3dm55TTwUzqAKXe9xOw6Y9dv7CVNvj6puZLOjjH8HHLE+Lmqc0UyqVONheI6dmBJfuFqofZYGn4zDgkxKUeQ5mYWO4LCiBSsrgKIpNgxsiHZ/VEMqzLlxZ3MX0L6ASO6RHC9oCbsM1JBEP0wUZQx5GE/0fujTcBy0Uc4PeNdpKtXrelvqkerv+j8ilOFlF/QkY6+Wizwz0YDHvz4AmrAm/2G80wv1JXotPJu2eFfnrZwxLzKT3kmwxL6O8Y4zq3/+Q2dspRql/ueFQYnfG3vHX1Te0yR9byrFWlFIH1Vjp/F05xRk5bJ078/8qYHIZpnqbsyhsP1VErUC9GosOay+iOHzY0hGsLV5XTk/xSF3h/rXlVirA6agIQmK3W2yIUzqd7O95rwBBw7UmCVDIHfgy7wJ7GC5zS5DosrkobQ6Rf3QM2CQ+naIqRz9nFnGWlC23i0UAk3dfgEizbDcMSiwZSGMAvxw== X-Forefront-Antispam-Report: CIP:12.22.5.235; CTRY:US; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:mail.nvidia.com; PTR:InfoNoRecords; CAT:NONE; SFS:(13230001)(4636009)(36840700001)(46966006)(40470700004)(2906002)(70206006)(6666004)(36860700001)(508600001)(36756003)(4326008)(40460700003)(55016003)(83380400001)(107886003)(8676002)(2616005)(16526019)(5660300002)(86362001)(186003)(70586007)(82310400005)(1076003)(26005)(81166007)(7696005)(356005)(316002)(336012)(6286002)(426003)(47076005)(8936002)(54906003)(110136005)(36900700001); DIR:OUT; SFP:1101; X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 24 May 2022 15:21:05.9269 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: 1663f3c8-e4af-466e-0381-08da3d9905a3 X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=43083d15-7273-40c1-b7db-39efd9ccc17a; Ip=[12.22.5.235]; Helo=[mail.nvidia.com] X-MS-Exchange-CrossTenant-AuthSource: CO1NAM11FT011.eop-nam11.prod.protection.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: SA0PR12MB4477 X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org LWM (limit watermark) describes the fullness of a Rx queue. If the Rx queue fullness is above LWM, the device will trigger the event RTE_ETH_EVENT_RX_LWM. LWM is defined as a percentage of Rx queue size with valid value of [0,99]. Setting LWM to 0 means disable it, which is the default. Add LWM's configuration and query driver callbacks in eth_dev_ops. Signed-off-by: Spike Du --- lib/ethdev/ethdev_driver.h | 22 ++++++++++++ lib/ethdev/rte_ethdev.c | 52 ++++++++++++++++++++++++++++ lib/ethdev/rte_ethdev.h | 71 ++++++++++++++++++++++++++++++++++++++ lib/ethdev/version.map | 2 ++ 4 files changed, 147 insertions(+) diff --git a/lib/ethdev/ethdev_driver.h b/lib/ethdev/ethdev_driver.h index 69d9dc21d8..49e4ef0fbb 100644 --- a/lib/ethdev/ethdev_driver.h +++ b/lib/ethdev/ethdev_driver.h @@ -470,6 +470,23 @@ typedef int (*eth_rx_queue_setup_t)(struct rte_eth_dev *dev, const struct rte_eth_rxconf *rx_conf, struct rte_mempool *mb_pool); +/** + * @internal Set Rx queue limit watermark. + * @see rte_eth_rx_lwm_set() + */ +typedef int (*eth_rx_queue_lwm_set_t)(struct rte_eth_dev *dev, + uint16_t rx_queue_id, + uint8_t lwm); + +/** + * @internal Query queue limit watermark event. + * @see rte_eth_rx_lwm_query() + */ + +typedef int (*eth_rx_queue_lwm_query_t)(struct rte_eth_dev *dev, + uint16_t *rx_queue_id, + uint8_t *lwm); + /** @internal Setup a transmit queue of an Ethernet device. */ typedef int (*eth_tx_queue_setup_t)(struct rte_eth_dev *dev, uint16_t tx_queue_id, @@ -1168,6 +1185,11 @@ struct eth_dev_ops { /** Priority flow control queue configure */ priority_flow_ctrl_queue_config_t priority_flow_ctrl_queue_config; + /** Set Rx queue limit watermark. */ + eth_rx_queue_lwm_set_t rx_queue_lwm_set; + /** Query Rx queue limit watermark event. */ + eth_rx_queue_lwm_query_t rx_queue_lwm_query; + /** Set Unicast Table Array */ eth_uc_hash_table_set_t uc_hash_table_set; /** Set Unicast hash bitmap */ diff --git a/lib/ethdev/rte_ethdev.c b/lib/ethdev/rte_ethdev.c index a175867651..e10e874aae 100644 --- a/lib/ethdev/rte_ethdev.c +++ b/lib/ethdev/rte_ethdev.c @@ -4424,6 +4424,58 @@ int rte_eth_set_queue_rate_limit(uint16_t port_id, uint16_t queue_idx, queue_idx, tx_rate)); } +int rte_eth_rx_lwm_set(uint16_t port_id, uint16_t queue_id, + uint8_t lwm) +{ + struct rte_eth_dev *dev; + struct rte_eth_dev_info dev_info; + int ret; + + RTE_ETH_VALID_PORTID_OR_ERR_RET(port_id, -ENODEV); + dev = &rte_eth_devices[port_id]; + + ret = rte_eth_dev_info_get(port_id, &dev_info); + if (ret != 0) + return ret; + + if (queue_id > dev_info.max_rx_queues) { + RTE_ETHDEV_LOG(ERR, + "Set queue LWM: port %u: invalid queue ID=%u.\n", + port_id, queue_id); + return -EINVAL; + } + + if (lwm > 99) + return -EINVAL; + RTE_FUNC_PTR_OR_ERR_RET(*dev->dev_ops->rx_queue_lwm_set, -ENOTSUP); + return eth_err(port_id, (*dev->dev_ops->rx_queue_lwm_set)(dev, + queue_id, lwm)); +} + +int rte_eth_rx_lwm_query(uint16_t port_id, uint16_t *queue_id, + uint8_t *lwm) +{ + struct rte_eth_dev_info dev_info; + struct rte_eth_dev *dev; + int ret; + + RTE_ETH_VALID_PORTID_OR_ERR_RET(port_id, -ENODEV); + dev = &rte_eth_devices[port_id]; + + ret = rte_eth_dev_info_get(port_id, &dev_info); + if (ret != 0) + return ret; + + if (queue_id == NULL) + return -EINVAL; + if (*queue_id >= dev_info.max_rx_queues) + *queue_id = 0; + + RTE_FUNC_PTR_OR_ERR_RET(*dev->dev_ops->rx_queue_lwm_query, -ENOTSUP); + return eth_err(port_id, (*dev->dev_ops->rx_queue_lwm_query)(dev, + queue_id, lwm)); +} + RTE_INIT(eth_dev_init_fp_ops) { uint32_t i; diff --git a/lib/ethdev/rte_ethdev.h b/lib/ethdev/rte_ethdev.h index 04225bba4d..541178fa76 100644 --- a/lib/ethdev/rte_ethdev.h +++ b/lib/ethdev/rte_ethdev.h @@ -1931,6 +1931,14 @@ struct rte_eth_rxq_info { uint8_t queue_state; /**< one of RTE_ETH_QUEUE_STATE_*. */ uint16_t nb_desc; /**< configured number of RXDs. */ uint16_t rx_buf_size; /**< hardware receive buffer size. */ + /** + * Per-queue Rx limit watermark defined as percentage of Rx queue + * size. If Rx queue receives traffic higher than this percentage, + * the event RTE_ETH_EVENT_RX_LWM is triggered. + * Value 0 means watermark monitoring is disabled, no event is + * triggered. + */ + uint8_t lwm; } __rte_cache_min_aligned; /** @@ -3672,6 +3680,64 @@ int rte_eth_dev_get_vlan_offload(uint16_t port_id); */ int rte_eth_dev_set_vlan_pvid(uint16_t port_id, uint16_t pvid, int on); +/** + * @warning + * @b EXPERIMENTAL: this API may change without prior notice. + * + * Set Rx queue based limit watermark. + * + * @param port_id + * The port identifier of the Ethernet device. + * @param queue_id + * The index of the receive queue. + * @param lwm + * The limit watermark percentage of Rx queue size which describes + * the fullness of Rx queue. If the Rx queue fullness is above LWM, + * the device will trigger the event RTE_ETH_EVENT_RX_LWM. + * [1-99] to set a new LWM. + * 0 to disable watermark monitoring. + * + * @return + * - 0 if successful. + * - negative if failed. + */ +__rte_experimental +int rte_eth_rx_lwm_set(uint16_t port_id, uint16_t queue_id, uint8_t lwm); + +/** + * @warning + * @b EXPERIMENTAL: this API may change without prior notice. + * + * Query Rx queue based limit watermark event. + * The function queries all queues in the port circularly until one + * pending LWM event is found or no pending LWM event is found. + * + * @param port_id + * The port identifier of the Ethernet device. + * @param queue_id + * The API caller sets the starting Rx queue id in the pointer. + * If the queue_id is bigger than maximum queue id of the port, + * it's rewinded to 0 so that application can keep calling + * this function to handle all pending LWM events in the queues + * with a simple increment between calls. + * If a Rx queue has pending LWM event, the pointer is updated + * with this Rx queue id; otherwise this pointer's content is + * unchanged. + * @param lwm + * The pointer to the limit watermark percentage of Rx queue. + * If Rx queue with pending LWM event is found, the queue's LWM + * percentage is stored in this pointer, otherwise the pointer's + * content is unchanged. + * + * @return + * - 1 if a Rx queue with pending LWM event is found. + * - 0 if no Rx queue with pending LWM event is found. + * - -EINVAL if queue_id is NULL. + */ +__rte_experimental +int rte_eth_rx_lwm_query(uint16_t port_id, uint16_t *queue_id, + uint8_t *lwm); + typedef void (*buffer_tx_error_fn)(struct rte_mbuf **unsent, uint16_t count, void *userdata); @@ -3877,6 +3943,11 @@ enum rte_eth_event_type { RTE_ETH_EVENT_DESTROY, /**< port is released */ RTE_ETH_EVENT_IPSEC, /**< IPsec offload related event */ RTE_ETH_EVENT_FLOW_AGED,/**< New aged-out flows is detected */ + /** + * Watermark value is exceeded in a queue. + * @see rte_eth_rx_lwm_set() + */ + RTE_ETH_EVENT_RX_LWM, RTE_ETH_EVENT_MAX /**< max value of this enum */ }; diff --git a/lib/ethdev/version.map b/lib/ethdev/version.map index daca7851f2..2e60765bbd 100644 --- a/lib/ethdev/version.map +++ b/lib/ethdev/version.map @@ -285,6 +285,8 @@ EXPERIMENTAL { rte_mtr_color_in_protocol_priority_get; rte_mtr_color_in_protocol_set; rte_mtr_meter_vlan_table_update; + rte_eth_rx_lwm_set; + rte_eth_rx_lwm_query; }; INTERNAL { -- 2.27.0