From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 52A50A034C; Sun, 22 May 2022 07:59:42 +0200 (CEST) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 7B0AE427F8; Sun, 22 May 2022 07:59:35 +0200 (CEST) Received: from NAM11-DM6-obe.outbound.protection.outlook.com (mail-dm6nam11on2075.outbound.protection.outlook.com [40.107.223.75]) by mails.dpdk.org (Postfix) with ESMTP id A4B034281B for ; Sun, 22 May 2022 07:59:33 +0200 (CEST) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=E1tEOCar99nW4jheJ2uaPOCLoLHMxQObzpxAPYKGzXyjehwsah3OINKiELb1LURVEBfmvNTui/CMG9qVdcYWxkIRUthN1dn5zTwPjzOGSWRo5cLXSICcQ+7UvPhXeDqD4YrXw6eohNhxsUYscjyQgZIuva/P8Nq+Fl390RWrHZF/QKoBGSxqT+XpX7WFn1uHUX8agNo7H+SLjAE8C6vxuwPWUP0a3O2c+4rMmcZIvU1TTmi7Qh51wmP+CGmL84H3HGOmyAXqBnqfz22hq55N2qTYtB2f4v2CsKwuD0czJ6INDCNvxqob0vQgUQk/mTAOoleIszzSxXMsJcBkZa1wfw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=WazGxw9jpK+RcpsXCbYEkGq7MBuLA92Gg5unwtckczw=; b=dQ1LYZdlp84O0whFbQiLXh7ifecHNS+SLy/RvNlF7vNpOkHkd9JZnrdUHOmPENtsc5Vl6ZvcJj8n1qHYSPxjpUkNfBDpwNkzpu/d9vl4F1XRyZW5+xZ7sN7ymWTQ5fXP8J+FzcqX0AydeLiHa3JOHRlJhqOEwncITVrUTHVnkkWdS0lYJwui8R7fA1/7RHPmlG1BPWOe+gh6bQ44zOJM2SZu1v3rkXjtrd+j1IvGASO2cRRRGEn0n2ko8SdHWK/3tNK/akvFpCcV4ftfHit/u8hfz6Xs1/uowWFC+AWFrHQqmAV/T37TS2g9Ys4ElUqXhOVS2PAEzzg0SWG+QDUeOA== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 12.22.5.235) smtp.rcpttodomain=dpdk.org smtp.mailfrom=nvidia.com; dmarc=pass (p=reject sp=reject pct=100) action=none header.from=nvidia.com; dkim=none (message not signed); arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=WazGxw9jpK+RcpsXCbYEkGq7MBuLA92Gg5unwtckczw=; b=j+Fc4DdgdqYJCca2LohL/kHuVzRIslCRqY79f0d/NslQ4Aol+v0yrrPLyu0EvN6wjqp1/kglIUEW1+/HzRK+cDwm4xi5A+qTHtn8w/0be0iwmo5iAmAN5Z0PkiNa69QEf4m/dBp6e2x466/CvRaRXrS3Ioj5I8Yn2xBeyWgbORCQ/tC2/7llywIHV9VtflB7FtBwkooGEiJZBhaMUBaJKWO2dN2iKHX3TY1jfitWzSgQCj9K3q3Wj89+uH0zytUiEY+3nmTHuL8V+As1C5XnRyUpV+G1yHDO8MW4lbMaCGmqKzfx8ZQpy2L45edzCl4txbIpA2AhU4YD9fHvpzGChA== Received: from MWHPR04CA0042.namprd04.prod.outlook.com (2603:10b6:300:ee::28) by CO6PR12MB5460.namprd12.prod.outlook.com (2603:10b6:5:357::11) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5273.14; Sun, 22 May 2022 05:59:32 +0000 Received: from CO1NAM11FT050.eop-nam11.prod.protection.outlook.com (2603:10b6:300:ee:cafe::a3) by MWHPR04CA0042.outlook.office365.com (2603:10b6:300:ee::28) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5273.17 via Frontend Transport; Sun, 22 May 2022 05:59:32 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 12.22.5.235) smtp.mailfrom=nvidia.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=nvidia.com; Received-SPF: Pass (protection.outlook.com: domain of nvidia.com designates 12.22.5.235 as permitted sender) receiver=protection.outlook.com; client-ip=12.22.5.235; helo=mail.nvidia.com; pr=C Received: from mail.nvidia.com (12.22.5.235) by CO1NAM11FT050.mail.protection.outlook.com (10.13.174.79) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384) id 15.20.5273.14 via Frontend Transport; Sun, 22 May 2022 05:59:31 +0000 Received: from rnnvmail201.nvidia.com (10.129.68.8) by DRHQMAIL107.nvidia.com (10.27.9.16) with Microsoft SMTP Server (TLS) id 15.0.1497.32; Sun, 22 May 2022 05:59:31 +0000 Received: from nvidia.com (10.126.231.35) by rnnvmail201.nvidia.com (10.129.68.8) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.986.22; Sat, 21 May 2022 22:59:28 -0700 From: Spike Du To: , , , CC: , Subject: [RFC v2 3/7] ethdev: introduce Rx queue based limit watermark Date: Sun, 22 May 2022 08:58:56 +0300 Message-ID: <20220522055900.417282-4-spiked@nvidia.com> X-Mailer: git-send-email 2.27.0 In-Reply-To: <20220522055900.417282-1-spiked@nvidia.com> References: <20220506035645.4101714-1-spiked@nvidia.com> <20220522055900.417282-1-spiked@nvidia.com> MIME-Version: 1.0 Content-Transfer-Encoding: 8bit Content-Type: text/plain X-Originating-IP: [10.126.231.35] X-ClientProxiedBy: rnnvmail203.nvidia.com (10.129.68.9) To rnnvmail201.nvidia.com (10.129.68.8) X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: be422f1f-e01e-4921-3928-08da3bb83d94 X-MS-TrafficTypeDiagnostic: CO6PR12MB5460:EE_ X-LD-Processed: 43083d15-7273-40c1-b7db-39efd9ccc17a,ExtAddr X-Microsoft-Antispam-PRVS: X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: 2aZq9cO2XquICxlWVAyIULnJH4pUu+XUtNGpQ45aQXSXzfFpUycH1JTsOG1c33jW1cP2zgIAjwBrXxI6iMZdSsr8xQdW7F0EaK6df+qIALitsljOSd7tKmcWgCHCZwhvCIjI5pEasvwHSVdiSvmGS12lQ1v6gj5+gW92HrWxAglh040wyRT3zSkKNXagm/td/Sr1njZ1KMoQ35O8qKun+d+Q1wP7D5Atr4JXPYOKj0ak5MbDieZsVcw+vIeQafoYjn453DC81xFFI8yCUuUiZ55HCn0MVVRAevAeBlOc8a10VaYDa+gaYxVBlGqzrGVUmlTV6ubZTse0Cel2kpLJ22LN6q/MuAe/lI3BEDaUzFlVXegtFUGs7E74BIxfdr4DCE9tPtgXdYHubSsYoKPeRo/uQL0NuuKPD8KWfl0TsN4PtA3+HJgTNBSuB9E+03+s/jN+yA7JBCg7T35zu4Vk19Mzr3mYXIuUsX3peMqO2qHN4jURJQbTky26ih+avX1laOeRaHNgoqnPAckxJcyBS5bveeXeGrx5CBtHYOsx/vAvg/pawnniJnYZWT9iFBCgeQS0K2v90mlQRghxzn8i89xModSSg2tqusfyGGjwc3+fkf8L5ek2/Re0RuWEV90956fd3s0ABr7pRmGB4+WOnFkqTa42HtJat9XqcJN7AWOwmvelxIFjwYEAm/qZffxuAVSpYYYWo1TxWjN/+bsCSg== X-Forefront-Antispam-Report: CIP:12.22.5.235; CTRY:US; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:mail.nvidia.com; PTR:InfoNoRecords; CAT:NONE; SFS:(13230001)(4636009)(40470700004)(46966006)(36840700001)(5660300002)(336012)(47076005)(426003)(81166007)(86362001)(26005)(2616005)(107886003)(6286002)(2906002)(7696005)(6666004)(1076003)(186003)(16526019)(356005)(70206006)(40460700003)(8676002)(4326008)(110136005)(70586007)(36860700001)(55016003)(316002)(54906003)(8936002)(508600001)(36756003)(83380400001)(82310400005)(36900700001); DIR:OUT; SFP:1101; X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 22 May 2022 05:59:31.8398 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: be422f1f-e01e-4921-3928-08da3bb83d94 X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=43083d15-7273-40c1-b7db-39efd9ccc17a; Ip=[12.22.5.235]; Helo=[mail.nvidia.com] X-MS-Exchange-CrossTenant-AuthSource: CO1NAM11FT050.eop-nam11.prod.protection.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: CO6PR12MB5460 X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org LWM(limit watermark) describes the fullness of a Rx queue. If the Rx queue fullness is above LWM, the device will trigger the event RTE_ETH_EVENT_RX_LWM. LWM is defined as a percentage of Rx queue size with valid value of [0,99]. Setting LWM to 0 means disable it, which is the default. When translate the percentage to queue descriptor number, the numbe should be bigger than 0 and less than queue size. Add LWM's configuration and query driver callbacks in eth_dev_ops. Signed-off-by: Spike Du --- lib/ethdev/ethdev_driver.h | 22 ++++++++++++ lib/ethdev/rte_ethdev.c | 52 +++++++++++++++++++++++++++ lib/ethdev/rte_ethdev.h | 74 +++++++++++++++++++++++++++++++++++++- lib/ethdev/version.map | 4 +++ 4 files changed, 151 insertions(+), 1 deletion(-) diff --git a/lib/ethdev/ethdev_driver.h b/lib/ethdev/ethdev_driver.h index 69d9dc21d8..12ec5e7e19 100644 --- a/lib/ethdev/ethdev_driver.h +++ b/lib/ethdev/ethdev_driver.h @@ -470,6 +470,23 @@ typedef int (*eth_rx_queue_setup_t)(struct rte_eth_dev *dev, const struct rte_eth_rxconf *rx_conf, struct rte_mempool *mb_pool); +/** + * @internal Set Rx queue limit watermark. + * see @rte_eth_rx_lwm_set() + */ +typedef int (*eth_rx_queue_lwm_set_t)(struct rte_eth_dev *dev, + uint16_t rx_queue_id, + uint8_t lwm); + +/** + * @internal Query queue limit watermark. + * see @rte_eth_rx_lwm_query() + */ + +typedef int (*eth_rx_queue_lwm_query_t)(struct rte_eth_dev *dev, + uint16_t *rx_queue_id, + uint8_t *lwm); + /** @internal Setup a transmit queue of an Ethernet device. */ typedef int (*eth_tx_queue_setup_t)(struct rte_eth_dev *dev, uint16_t tx_queue_id, @@ -1168,6 +1185,11 @@ struct eth_dev_ops { /** Priority flow control queue configure */ priority_flow_ctrl_queue_config_t priority_flow_ctrl_queue_config; + /** Set Rx queue limit watermark */ + eth_rx_queue_lwm_set_t rx_queue_lwm_set; + /** Query Rx queue limit watermark */ + eth_rx_queue_lwm_query_t rx_queue_lwm_query; + /** Set Unicast Table Array */ eth_uc_hash_table_set_t uc_hash_table_set; /** Set Unicast hash bitmap */ diff --git a/lib/ethdev/rte_ethdev.c b/lib/ethdev/rte_ethdev.c index 8520aec561..0a46c71288 100644 --- a/lib/ethdev/rte_ethdev.c +++ b/lib/ethdev/rte_ethdev.c @@ -4429,6 +4429,58 @@ int rte_eth_set_queue_rate_limit(uint16_t port_id, uint16_t queue_idx, queue_idx, tx_rate)); } +int rte_eth_rx_lwm_set(uint16_t port_id, uint16_t queue_id, + uint8_t lwm) +{ + struct rte_eth_dev *dev; + struct rte_eth_dev_info dev_info; + int ret; + + RTE_ETH_VALID_PORTID_OR_ERR_RET(port_id, -ENODEV); + dev = &rte_eth_devices[port_id]; + + ret = rte_eth_dev_info_get(port_id, &dev_info); + if (ret != 0) + return ret; + + if (queue_id > dev_info.max_rx_queues) { + RTE_ETHDEV_LOG(ERR, + "Set queue LWM:port %u: invalid queue ID=%u.\n", + port_id, queue_id); + return -EINVAL; + } + + if (lwm > 99) + return -EINVAL; + RTE_FUNC_PTR_OR_ERR_RET(*dev->dev_ops->rx_queue_lwm_set, -ENOTSUP); + return eth_err(port_id, (*dev->dev_ops->rx_queue_lwm_set)(dev, + queue_id, lwm)); +} + +int rte_eth_rx_lwm_query(uint16_t port_id, uint16_t *queue_id, + uint8_t *lwm) +{ + struct rte_eth_dev_info dev_info; + struct rte_eth_dev *dev; + int ret; + + RTE_ETH_VALID_PORTID_OR_ERR_RET(port_id, -ENODEV); + dev = &rte_eth_devices[port_id]; + + ret = rte_eth_dev_info_get(port_id, &dev_info); + if (ret != 0) + return ret; + + if (queue_id == NULL) + return -EINVAL; + if (*queue_id >= dev_info.max_rx_queues) + *queue_id = 0; + + RTE_FUNC_PTR_OR_ERR_RET(*dev->dev_ops->rx_queue_lwm_query, -ENOTSUP); + return eth_err(port_id, (*dev->dev_ops->rx_queue_lwm_query)(dev, + queue_id, lwm)); +} + RTE_INIT(eth_dev_init_fp_ops) { uint32_t i; diff --git a/lib/ethdev/rte_ethdev.h b/lib/ethdev/rte_ethdev.h index 04cff8ee10..687ae5ff29 100644 --- a/lib/ethdev/rte_ethdev.h +++ b/lib/ethdev/rte_ethdev.h @@ -1249,7 +1249,16 @@ struct rte_eth_rxconf { */ union rte_eth_rxseg *rx_seg; - uint64_t reserved_64s[2]; /**< Reserved for future fields */ + /** + * Per-queue Rx limit watermark defined as percentage of Rx queue + * size. If Rx queue receives traffic higher than this percentage, + * the event RTE_ETH_EVENT_RX_LWM is triggered. + */ + uint8_t lwm; + + uint8_t reserved_bits[3]; + uint32_t reserved_32s; + uint64_t reserved_64s; void *reserved_ptrs[2]; /**< Reserved for future fields */ }; @@ -3668,6 +3677,64 @@ int rte_eth_dev_get_vlan_offload(uint16_t port_id); */ int rte_eth_dev_set_vlan_pvid(uint16_t port_id, uint16_t pvid, int on); +/** + * @warning + * @b EXPERIMENTAL: this API may change without prior notice. + * + * Set Rx queue based limit watermark. + * + * @param port_id + * The port identifier of the Ethernet device. + * @param queue_id + * The index of the receive queue. + * @param lwm + * The limit watermark percentage of Rx queue size which describes + * the fullness of Rx queue. If the Rx queue fullness is above LWM, + * the device will trigger the event RTE_ETH_EVENT_RX_LWM. + * [1-99] to set a new LWM. + * 0 to disable watermark monitoring. + * + * @return + * - 0 if successful. + * - negative if failed. + */ +__rte_experimental +int rte_eth_rx_lwm_set(uint16_t port_id, uint16_t queue_id, uint8_t lwm); + +/** + * @warning + * @b EXPERIMENTAL: this API may change without prior notice. + * + * Query Rx queue based limit watermark. + * The function queries all queues in the port circularly until one + * pending LWM event is found or no pending LWM event is found. + * + * @param port_id + * The port identifier of the Ethernet device. + * @param queue_id + * The API caller sets the starting Rx queue id in the pointer. + * If the queue_id is bigger than maximum queue id of the port, + * it's rewinded to 0 so that application can keep calling + * this function to handle all pending LWM events in the queues + * with a simple increment between calls. + * If a Rx queue has pending lwm event, the pointer is updated + * with this Rx queue id; otherwise this pointer's content is + * unchanged. + * @param lwm + * The pointer to the limit watermark percentage of Rx queue. + * If Rx queue with pending lwm event is found, the queue's LWM + * percentage is stored in this pointer, otherwise the pointer's + * content is unchanged. + * + * @return + * - 1 if a Rx queue with pending lwm event is found. + * - 0 if no Rx queue with pending lwm event is found. + * - -EINVAL if queue_id is NULL. + */ +__rte_experimental +int rte_eth_rx_lwm_query(uint16_t port_id, uint16_t *queue_id, + uint8_t *lwm); + typedef void (*buffer_tx_error_fn)(struct rte_mbuf **unsent, uint16_t count, void *userdata); @@ -3873,6 +3940,11 @@ enum rte_eth_event_type { RTE_ETH_EVENT_DESTROY, /**< port is released */ RTE_ETH_EVENT_IPSEC, /**< IPsec offload related event */ RTE_ETH_EVENT_FLOW_AGED,/**< New aged-out flows is detected */ + /** + * watermark value is exceeded in a queue. + * see @rte_eth_rx_lwm_set() + */ + RTE_ETH_EVENT_RX_LWM, RTE_ETH_EVENT_MAX /**< max value of this enum */ }; diff --git a/lib/ethdev/version.map b/lib/ethdev/version.map index 20391ab29e..5cf44c5d1d 100644 --- a/lib/ethdev/version.map +++ b/lib/ethdev/version.map @@ -279,6 +279,10 @@ EXPERIMENTAL { rte_flow_async_action_handle_create; rte_flow_async_action_handle_destroy; rte_flow_async_action_handle_update; + + # added in 22.07 + rte_eth_rx_lwm_set; + rte_eth_rx_lwm_query; }; INTERNAL { -- 2.27.0