From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 291AD42665; Thu, 28 Sep 2023 22:59:55 +0200 (CEST) Received: from mails.dpdk.org (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id C9ED34025F; Thu, 28 Sep 2023 22:59:53 +0200 (CEST) Received: from NAM10-MW2-obe.outbound.protection.outlook.com (mail-mw2nam10on2070.outbound.protection.outlook.com [40.107.94.70]) by mails.dpdk.org (Postfix) with ESMTP id 5C7EF4021E for ; Thu, 28 Sep 2023 22:59:52 +0200 (CEST) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=kawfw3jLaMGN/x73sSj/D++xagA5NcA1PRG+tQhloFKBExGpP4Jumv4F9LXZ/FNUTfZ+3/ekMdEavmNm32oRjmNbHXVMy1ss7Y0xlNKVZc2c6SXImwhjeUZGQcsmfz6faAE3a3tqS6R4dLoo7SH0xSMxdXzOQqTF9a3KYvN73hknrCeWO2yMgfVDK3cb3VRMzxJ4gHFulJ+u3dxgRrHsZKv2s91W5YhxSutFdY9lRcJTm1QBQ8Q0zbv4EWuU4XiZuGivC9nIZ4qB8wvhm7SuQqAtzpTgwRa9x4N+B9It5bWrHmqMB8WG/f28t4ub1EqS3RvKk4crwy2vUV6eXslF+A== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=dNkUqs8eD4uIbM7FezBpuugei1OjI874oqnwH1n94eg=; b=FoEJSCKlWeQJEOZfXtiTIiFjCJ4w/aRLlbMqtdnwA64ws1agP6o61fRC4+phOGIiBuQU6is3FftXkyKgOcIGXrjFZhj0MAZiCmZU++iM9MDw8c+rbaOdddkDVbLl0mDqrl6hMnoxh11P/U01mBNri1uTuQz5QHT720qyYnhDHO1TcP0OQ96m4F7nkb1BFBRDz9WuVkLRuDfydmF0n62xpbUR5FRUUyBj5AwIhZqY3Am6Y9Fhnu64SBndQ3Se0RHTk3Jsb65Uhm2gTwpDgH1cLve6BUr8Zw02QgWmv16RvL0c5uK55g5AbSFcTaKbS9X+cAoHx/OQ1MohbuzFsXFBrA== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 165.204.84.17) smtp.rcpttodomain=monjalon.net smtp.mailfrom=amd.com; dmarc=pass (p=quarantine sp=quarantine pct=100) action=none header.from=amd.com; dkim=none (message not signed); arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amd.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=dNkUqs8eD4uIbM7FezBpuugei1OjI874oqnwH1n94eg=; b=P6oHDDjJ1bpUR4wV4CV6x+ZWVM6Klpew5AtykoZnBpJIpfgbyOwiHtedjWv+3o2etVWk1dVz2YwmY4052E67fngKxMaiAJxs1jPCQKR42f5QYl/3oVY3SNDI0RLlPD1o6D/vLqXf6iH6D3KG2ANeYLxfcXuwhdZmXpxCBAWDXIo= Received: from BL0PR01CA0006.prod.exchangelabs.com (2603:10b6:208:71::19) by DS7PR12MB8420.namprd12.prod.outlook.com (2603:10b6:8:e9::14) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.6838.22; Thu, 28 Sep 2023 20:59:49 +0000 Received: from MN1PEPF0000F0E1.namprd04.prod.outlook.com (2603:10b6:208:71:cafe::89) by BL0PR01CA0006.outlook.office365.com (2603:10b6:208:71::19) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.6838.25 via Frontend Transport; Thu, 28 Sep 2023 20:59:49 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 165.204.84.17) smtp.mailfrom=amd.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=amd.com; Received-SPF: Pass (protection.outlook.com: domain of amd.com designates 165.204.84.17 as permitted sender) receiver=protection.outlook.com; client-ip=165.204.84.17; helo=SATLEXMB04.amd.com; pr=C Received: from SATLEXMB04.amd.com (165.204.84.17) by MN1PEPF0000F0E1.mail.protection.outlook.com (10.167.242.39) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.20.6838.21 via Frontend Transport; Thu, 28 Sep 2023 20:59:49 +0000 Received: from telcodpdk.amd.com (10.180.168.240) by SATLEXMB04.amd.com (10.181.40.145) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.27; Thu, 28 Sep 2023 15:59:48 -0500 From: Ferruh Yigit To: Thomas Monjalon , Andrew Rybchenko CC: , David Marchand , Jie Hai , Song Jiale , Yuan Peng , Raslan Darawsheh , Qiming Yang , Ivan Malov , Huisong Li Subject: [PATCH] ethdev: clarify device queue state after start and stop Date: Thu, 28 Sep 2023 21:59:29 +0100 Message-ID: <20230928205930.2619353-1-ferruh.yigit@amd.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20230721160422.3848154-1-ferruh.yigit@amd.com> References: <20230721160422.3848154-1-ferruh.yigit@amd.com> MIME-Version: 1.0 Content-Transfer-Encoding: 8bit Content-Type: text/plain X-Originating-IP: [10.180.168.240] X-ClientProxiedBy: SATLEXMB03.amd.com (10.181.40.144) To SATLEXMB04.amd.com (10.181.40.145) X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: MN1PEPF0000F0E1:EE_|DS7PR12MB8420:EE_ X-MS-Office365-Filtering-Correlation-Id: ef72e951-ffeb-43c6-275d-08dbc065da94 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: sVFYafjrL6v7x/eS5gQRob8hbYzkMDZ6MURPqIgYhcm4NVt5aqwpv3pAljMA/+HTXlvtQ4fMEmm7o57ClEmEl4zVT0p064ZnB2pWWNVZFyI/YjkmTqhBxB1Fp8udtQaJtxf/PglEVRWLGRZ8Kpnh3ZKtKOz5UhtOsugKKLes2Yb2KiyJUGj7bZyYtDCD/G5+ZWq5aDUuVZ0Mv49Xub3i/fPKPHgbhuLymySKaITVKIHDGVP9JStwVzEUmDDSFMQBCCkbKK7v47Ij/ba4QsWb9BOEbiRzL2AO0CjMq14MoLfAP7UAY44hXqoVp5eF1Il5xS7wG8GajuAMEceitC4TIub1jCsDNntjGXyo7TJ+TT/vlZ3tiDbv2HISZZxF4qyiBsi+wkHJUdahFdGc5basrGyGdSPKVulYfDHUtkmIW+4z5Xigrh2HCGczW8e5O0CIBK+YKB88XuJfyyGjjOEeycenYAnKzfpuoe2Smsvs3W2V2jeE+MPGKrm/C/KgurMDj6H34wkBrpU+37DhRRwdmDqNKvZRbX8bN51FdQp4//q95l67L7WAFgwiEvg4JMo10xFoi5KLugMJyzwpMHR9KM+1mIo/7fARMGtgMyPmMmirkdRbs4XTLdCQRqfxeBQFy+d8isrYnUZ4rvIs6uBfkaqJnTtPJjMTDmJ92Q0oOwYNPAyjYYfHZ2IHB3nqdXPxH372G9rcHK/GL/PZICNq1PTFsbx1YMiXzZYG+LYnuMWHMZUZUTZ+8O8EssOUD49Bl3DiPpNmilIaWdfKEKj0wQ== X-Forefront-Antispam-Report: CIP:165.204.84.17; CTRY:US; LANG:en; SCL:1; SRV:; IPV:CAL; SFV:NSPM; H:SATLEXMB04.amd.com; PTR:InfoDomainNonexistent; CAT:NONE; SFS:(13230031)(4636009)(376002)(346002)(396003)(39860400002)(136003)(230922051799003)(451199024)(1800799009)(64100799003)(186009)(82310400011)(40470700004)(36840700001)(46966006)(41300700001)(316002)(70586007)(40480700001)(70206006)(54906003)(110136005)(478600001)(6666004)(86362001)(40460700003)(7416002)(2906002)(36756003)(44832011)(8676002)(4326008)(336012)(5660300002)(8936002)(426003)(26005)(16526019)(83380400001)(1076003)(2616005)(36860700001)(47076005)(82740400003)(7696005)(356005)(81166007)(36900700001); DIR:OUT; SFP:1101; X-OriginatorOrg: amd.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 28 Sep 2023 20:59:49.3960 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: ef72e951-ffeb-43c6-275d-08dbc065da94 X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=3dd8961f-e488-4e60-8e11-a82d994e183d; Ip=[165.204.84.17]; Helo=[SATLEXMB04.amd.com] X-MS-Exchange-CrossTenant-AuthSource: MN1PEPF0000F0E1.namprd04.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: DS7PR12MB8420 X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Drivers start/stop device queues on port start/stop, but not all drivers update queue state accordingly. This becomes more visible if a specific queue stopped explicitly and port stopped/started later, in this case although all queues are started, the state of that specific queue is stopped and it is misleading. Misrepresentation of queue state became a defect with commit [1] that does forwarding decision based on queue state and commit [2] that gets up to date queue state from ethdev/device before forwarding. [1] commit 3c4426db54fc ("app/testpmd: do not poll stopped queues") [2] commit 5028f207a4fa ("app/testpmd: fix secondary process packet forwarding") This patch documents that status of all queues of a device should be `RTE_ETH_QUEUE_STATE_STOPPED` after port stop and their status should be`RTE_ETH_QUEUE_STATE_STARTED` after port start. Also an unit test added to verify drivers. Signed-off-by: Ferruh Yigit --- Cc: Jie Hai Cc: Song Jiale Cc: Yuan Peng Cc: Raslan Darawsheh Cc: Qiming Yang Cc: Ivan Malov Cc: Huisong Li v1: * fix memset * remove commented out code * update unit test to skip queue state if rte_eth_[rt]x_queue_info_get() is not supported --- app/test/meson.build | 1 + app/test/test_ethdev_api.c | 184 +++++++++++++++++++++++++++++++++++++ lib/ethdev/rte_ethdev.h | 5 + 3 files changed, 190 insertions(+) create mode 100644 app/test/test_ethdev_api.c diff --git a/app/test/meson.build b/app/test/meson.build index 05bae9216dbc..05bbe84868f6 100644 --- a/app/test/meson.build +++ b/app/test/meson.build @@ -65,6 +65,7 @@ source_file_deps = { 'test_efd_perf.c': ['efd', 'hash'], 'test_errno.c': [], 'test_ethdev_link.c': ['ethdev'], + 'test_ethdev_api.c': ['ethdev'], 'test_event_crypto_adapter.c': ['cryptodev', 'eventdev', 'bus_vdev'], 'test_event_eth_rx_adapter.c': ['ethdev', 'eventdev', 'bus_vdev'], 'test_event_eth_tx_adapter.c': ['bus_vdev', 'ethdev', 'net_ring', 'eventdev'], diff --git a/app/test/test_ethdev_api.c b/app/test/test_ethdev_api.c new file mode 100644 index 000000000000..68239f82ff33 --- /dev/null +++ b/app/test/test_ethdev_api.c @@ -0,0 +1,184 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright (C) 2023, Advanced Micro Devices, Inc. + */ + +#include +#include + +#include +#include "test.h" + +#define NUM_RXQ 2 +#define NUM_TXQ 2 +#define NUM_RXD 512 +#define NUM_TXD 512 +#define NUM_MBUF 1024 +#define MBUF_CACHE_SIZE 256 + +static int32_t +ethdev_api_queue_status(void) +{ + struct rte_eth_dev_info dev_info; + struct rte_eth_rxq_info rx_qinfo; + struct rte_eth_txq_info tx_qinfo; + struct rte_mempool *mbuf_pool; + struct rte_eth_conf eth_conf; + uint16_t port_id; + int ret; + + mbuf_pool = rte_pktmbuf_pool_create("MBUF_POOL", NUM_MBUF, MBUF_CACHE_SIZE, 0, + RTE_MBUF_DEFAULT_BUF_SIZE, rte_socket_id()); + + RTE_ETH_FOREACH_DEV(port_id) { + memset(ð_conf, 0, sizeof(eth_conf)); + ret = rte_eth_dev_configure(port_id, NUM_RXQ, NUM_TXQ, ð_conf); + TEST_ASSERT(ret == 0, + "Port(%u) failed to configure.\n", port_id); + + /* RxQ setup */ + for (uint16_t queue_id = 0; queue_id < NUM_RXQ; queue_id++) { + ret = rte_eth_rx_queue_setup(port_id, queue_id, NUM_RXD, + rte_socket_id(), NULL, mbuf_pool); + TEST_ASSERT(ret == 0, + "Port(%u), queue(%u) failed to setup RxQ.\n", + port_id, queue_id); + } + + /* TxQ setup */ + for (uint16_t queue_id = 0; queue_id < NUM_TXQ; queue_id++) { + ret = rte_eth_tx_queue_setup(port_id, queue_id, NUM_TXD, + rte_socket_id(), NULL); + TEST_ASSERT(ret == 0, + "Port(%u), queue(%u) failed to setup TxQ.\n", + port_id, queue_id); + } + + ret = rte_eth_dev_info_get(port_id, &dev_info); + TEST_ASSERT(ret == 0, + "Port(%u) failed to get dev info.\n", port_id); + + /* Initial RxQ */ + for (uint16_t queue_id = 0; queue_id < dev_info.nb_rx_queues; queue_id++) { + ret = rte_eth_rx_queue_info_get(port_id, queue_id, &rx_qinfo); + if (ret == -ENOTSUP) + continue; + + TEST_ASSERT(ret == 0, + "Port(%u), queue(%u) failed to get RxQ info.\n", + port_id, queue_id); + + TEST_ASSERT(rx_qinfo.queue_state == RTE_ETH_QUEUE_STATE_STOPPED, + "Wrong initial Rx queue(%u) state(%d)\n", + queue_id, rx_qinfo.queue_state); + } + + /* Initial TxQ */ + for (uint16_t queue_id = 0; queue_id < dev_info.nb_tx_queues; queue_id++) { + ret = rte_eth_tx_queue_info_get(port_id, queue_id, &tx_qinfo); + if (ret == -ENOTSUP) + continue; + + TEST_ASSERT(ret == 0, + "Port(%u), queue(%u) failed to get TxQ info.\n", + port_id, queue_id); + + TEST_ASSERT(tx_qinfo.queue_state == RTE_ETH_QUEUE_STATE_STOPPED, + "Wrong initial Tx queue(%u) state(%d)\n", + queue_id, tx_qinfo.queue_state); + } + + + ret = rte_eth_dev_start(port_id); + TEST_ASSERT(ret == 0, + "Port(%u) failed to start.\n", port_id); + + /* Started RxQ */ + for (uint16_t queue_id = 0; queue_id < dev_info.nb_rx_queues; queue_id++) { + ret = rte_eth_rx_queue_info_get(port_id, queue_id, &rx_qinfo); + if (ret == -ENOTSUP) + continue; + + TEST_ASSERT(ret == 0, + "Port(%u), queue(%u) failed to get RxQ info.\n", + port_id, queue_id); + + TEST_ASSERT(rx_qinfo.queue_state == RTE_ETH_QUEUE_STATE_STARTED, + "Wrong started Rx queue(%u) state(%d)\n", + queue_id, rx_qinfo.queue_state); + } + + /* Started TxQ */ + for (uint16_t queue_id = 0; queue_id < dev_info.nb_tx_queues; queue_id++) { + ret = rte_eth_tx_queue_info_get(port_id, queue_id, &tx_qinfo); + if (ret == -ENOTSUP) + continue; + + TEST_ASSERT(ret == 0, + "Port(%u), queue(%u) failed to get TxQ info.\n", + port_id, queue_id); + + TEST_ASSERT(tx_qinfo.queue_state == RTE_ETH_QUEUE_STATE_STARTED, + "Wrong started Tx queue(%u) state(%d)\n", + queue_id, tx_qinfo.queue_state); + } + + + ret = rte_eth_dev_stop(port_id); + TEST_ASSERT(ret == 0, + "Port(%u) failed to stop.\n", port_id); + + /* Stopped RxQ */ + for (uint16_t queue_id = 0; queue_id < dev_info.nb_rx_queues; queue_id++) { + ret = rte_eth_rx_queue_info_get(port_id, queue_id, &rx_qinfo); + if (ret == -ENOTSUP) + continue; + + TEST_ASSERT(ret == 0, + "Port(%u), queue(%u) failed to get RxQ info.\n", + port_id, queue_id); + + TEST_ASSERT(rx_qinfo.queue_state == RTE_ETH_QUEUE_STATE_STOPPED, + "Wrong stopped Rx queue(%u) state(%d)\n", + queue_id, rx_qinfo.queue_state); + } + + /* Stopped TxQ */ + for (uint16_t queue_id = 0; queue_id < dev_info.nb_tx_queues; queue_id++) { + ret = rte_eth_tx_queue_info_get(port_id, queue_id, &tx_qinfo); + if (ret == -ENOTSUP) + continue; + + TEST_ASSERT(ret == 0, + "Port(%u), queue(%u) failed to get TxQ info.\n", + port_id, queue_id); + + TEST_ASSERT(tx_qinfo.queue_state == RTE_ETH_QUEUE_STATE_STOPPED, + "Wrong stopped Tx queue(%u) state(%d)\n", + queue_id, tx_qinfo.queue_state); + } + } + + return TEST_SUCCESS; +} + +static struct unit_test_suite ethdev_api_testsuite = { + .suite_name = "ethdev API tests", + .setup = NULL, + .teardown = NULL, + .unit_test_cases = { + TEST_CASE(ethdev_api_queue_status), + /* TODO: Add deferred_start queue status test */ + TEST_CASES_END() /**< NULL terminate unit test array */ + } +}; + +static int +test_ethdev_api(void) +{ + rte_log_set_global_level(RTE_LOG_DEBUG); + rte_log_set_level(RTE_LOGTYPE_EAL, RTE_LOG_DEBUG); + + return unit_test_suite_runner(ðdev_api_testsuite); +} + +REGISTER_TEST_COMMAND(ethdev_api, test_ethdev_api); diff --git a/lib/ethdev/rte_ethdev.h b/lib/ethdev/rte_ethdev.h index 8542257721c9..6441fe049b06 100644 --- a/lib/ethdev/rte_ethdev.h +++ b/lib/ethdev/rte_ethdev.h @@ -2812,6 +2812,9 @@ int rte_eth_dev_tx_queue_stop(uint16_t port_id, uint16_t tx_queue_id); * Device RTE_ETH_DEV_NOLIVE_MAC_ADDR flag causes MAC address to be set before * PMD port start callback function is invoked. * + * All device queues (except form deferred start queues) status should be + * `RTE_ETH_QUEUE_STATE_STARTED` after start. + * * On success, all basic functions exported by the Ethernet API (link status, * receive/transmit, and so on) can be invoked. * @@ -2828,6 +2831,8 @@ int rte_eth_dev_start(uint16_t port_id); * Stop an Ethernet device. The device can be restarted with a call to * rte_eth_dev_start() * + * All device queues status should be `RTE_ETH_QUEUE_STATE_STOPPED` after stop. + * * @param port_id * The port identifier of the Ethernet device. * @return -- 2.34.1