From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 94E06A034C; Wed, 21 Dec 2022 10:00:58 +0100 (CET) Received: from mails.dpdk.org (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 24A8342D10; Wed, 21 Dec 2022 10:00:55 +0100 (CET) Received: from NAM10-MW2-obe.outbound.protection.outlook.com (mail-mw2nam10on2066.outbound.protection.outlook.com [40.107.94.66]) by mails.dpdk.org (Postfix) with ESMTP id E0F4042D10 for ; Wed, 21 Dec 2022 10:00:52 +0100 (CET) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=jjgecdEYBBE10LJnkc8UD04BQnmHohg2nZxp35eddT3n3hsLaH05cKI5Qycyc9jLMwS79gAYo9NqSv7S8ydAhff/ct09YE+VyFy8PxU2ltTX48i2dh/T7mqiU9uofcTVlNDsBXT3NMCTSHPZloxcnkPV8ZrE1RmFj/Cz1NhFZ9mfXEgh462Rz+3X6mZV1HrPKdaldZmjZuB8FnJDgGYNCGQTuWpo6ySpkRSVNXx6MzFjwWKyLTP6/cxCeqPmWpE5UA3uWXSeFG3zMcgqjXWX45rXqqql/CO3CroZDkVouDMEVnBlFAQXL4tbtXoVITLb4PAJ9Rc9vfdb3kzXr0Xesw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=l0qbtQ0epaOezn9TYHLrOG2w9IV4ou27wQ0NVLsxRbY=; b=Dta3pRdFvLBQ3BPhI+0jW+5XBUF2ELVVFZOQUKZm3RT8uzxN4f2nJiY2BMTwrFnb4nIeMVJm7bNihdXiCgunOlc180mihS6PlKK/3azhbxQxDydgTHzCryX+SYqER/fnRyU1ljGrXevAaiyrPRCSrkiFn202IYccJy8naoazoT2TtoApPc2ugWLH/6rDG6NTvAwNpgVK8z+RgzTEpCt67JQmSXxCoPeyQ5iubdtyZoi3rWRWmvDhjaQ/sqVLsb6gOXCTw3r3t/Jvw4r8pPrqETm2YadFfvdGP2O69rtaneyWNi/Ev8ecBuMvZBw+ma2LnxOrwNrcMdak5Ma7rZ2pwA== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 216.228.117.160) smtp.rcpttodomain=monjalon.net smtp.mailfrom=nvidia.com; dmarc=pass (p=reject sp=reject pct=100) action=none header.from=nvidia.com; dkim=none (message not signed); arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=l0qbtQ0epaOezn9TYHLrOG2w9IV4ou27wQ0NVLsxRbY=; b=cFK8PFTSDj2Q1PQ7KVfBOOX+sDOzrNMM9CaB1pb+w9sVh8s2gS+8orvj99iUWTTH9t3+lYzKNAew9Q4k+RDGMmHAyp6XNCX2AkkJ6K/smuCM5JWUuVGdZB7mr1B0igNyy0rAekY5fcazuP/73imTkEbMdTPxIC2xtbJzRwLrP4k0mfV5OJdgw969DGyr9ClZT6N+6DQ8v79jZsxkBHenTyFr8Kq6dud/HYAKG+LWpL/BBkJNVzhDd5tv8RQo+6c31ay2Pml34NKDrcRdCsocXYvICa9t4coxuHIETKBSP+S+vzJgtIZXA01awiMQWveFwa7m7NDzYVOiM5aNZsMdWA== Received: from MW4PR03CA0069.namprd03.prod.outlook.com (2603:10b6:303:b6::14) by PH8PR12MB7304.namprd12.prod.outlook.com (2603:10b6:510:217::20) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5924.16; Wed, 21 Dec 2022 09:00:51 +0000 Received: from CO1NAM11FT042.eop-nam11.prod.protection.outlook.com (2603:10b6:303:b6:cafe::6c) by MW4PR03CA0069.outlook.office365.com (2603:10b6:303:b6::14) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5924.21 via Frontend Transport; Wed, 21 Dec 2022 09:00:50 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 216.228.117.160) smtp.mailfrom=nvidia.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=nvidia.com; Received-SPF: Pass (protection.outlook.com: domain of nvidia.com designates 216.228.117.160 as permitted sender) receiver=protection.outlook.com; client-ip=216.228.117.160; helo=mail.nvidia.com; pr=C Received: from mail.nvidia.com (216.228.117.160) by CO1NAM11FT042.mail.protection.outlook.com (10.13.174.250) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5944.6 via Frontend Transport; Wed, 21 Dec 2022 09:00:50 +0000 Received: from rnnvmail201.nvidia.com (10.129.68.8) by mail.nvidia.com (10.129.200.66) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.986.36; Wed, 21 Dec 2022 01:00:38 -0800 Received: from nvidia.com (10.126.231.37) by rnnvmail201.nvidia.com (10.129.68.8) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.986.36; Wed, 21 Dec 2022 01:00:35 -0800 From: Rongwei Liu To: , , , , Ferruh Yigit , Andrew Rybchenko CC: , Subject: [RFC v3 2/2] ethdev: add API to set process to active or standby Date: Wed, 21 Dec 2022 11:00:17 +0200 Message-ID: <20221221090017.3715030-3-rongweil@nvidia.com> X-Mailer: git-send-email 2.27.0 In-Reply-To: <20221221090017.3715030-1-rongweil@nvidia.com> References: <20221205215416.7ac53a55@hermes.local> <20221221090017.3715030-1-rongweil@nvidia.com> MIME-Version: 1.0 Content-Transfer-Encoding: 8bit Content-Type: text/plain X-Originating-IP: [10.126.231.37] X-ClientProxiedBy: rnnvmail201.nvidia.com (10.129.68.8) To rnnvmail201.nvidia.com (10.129.68.8) X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: CO1NAM11FT042:EE_|PH8PR12MB7304:EE_ X-MS-Office365-Filtering-Correlation-Id: 26437a40-5ec4-487c-2de3-08dae331dbab X-LD-Processed: 43083d15-7273-40c1-b7db-39efd9ccc17a,ExtAddr X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: GMWPQAUyxauKHC33Q+aHiJbo5olH5IQ2v0gKUBHj1QjNuV9g0jGUB9uwsyyM3JK2/Xtj332A8uMIrJ3X4loE+Pt+jTouKqTiLso+aBTwsi0LPOZIuAHfqTggSO7xM4FaeChN0+RAlj/m3YSknimTdQ8MnUo818wcahlQseBV4P4qYbeScNaMBYxhfSdYXnhVvGvlR987PzztgZSE2DztQ3ZOYgW+UvaBG7NQMvl5nnjuBX6hsp6O33S7aEyBsRFvf2g7zG509gcqYPQuf52OHijgn4uskmd+A5cl/JxTDN3cABSebfMEXmXltU/VqJAH5+V3yfl/5uk9TaSc3IrE/hXOTtBT3Yo3D+MHd7Sq2/sXb58v6jSovH0X5edORoMKfB2AjdlKiHpukQSmHmHZCGS5J+Px1jZpAXKy6y0v/kMUbAI/c7mkUEJVuZEKHWy2zB9cESvO8tjmnAn6Jsva524K74Ni18kZuiQ1vXfHW/3tjCpAxtlevow4admCUlumjs1RSAmgvMMCCA6IDRnn7wIZCRTAEQCuiX5Jcj+D0deIYgZe+Bt556f403rKZ7PZsEw/n6TD8HMA4CTOZ1kitb82lwhg+aHiCdcfNf6ska3ZpQRwu5EPd9jeJNiUeImyUL0Riy7jw3GUFTf9jJURvKGvmDcFH9xqZYnMiUCmdyOdtlJrd58AvPq7ivkpqlrOYi7NjD/Ae+ogAt6hEqbr/g== X-Forefront-Antispam-Report: CIP:216.228.117.160; CTRY:US; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:mail.nvidia.com; PTR:dc6edge1.nvidia.com; CAT:NONE; SFS:(13230022)(4636009)(136003)(39860400002)(346002)(396003)(376002)(451199015)(36840700001)(40470700004)(46966006)(336012)(1076003)(8936002)(82740400003)(2616005)(83380400001)(82310400005)(107886003)(6666004)(6286002)(40460700003)(26005)(16526019)(7696005)(186003)(36860700001)(36756003)(478600001)(426003)(47076005)(4326008)(40480700001)(5660300002)(8676002)(41300700001)(7636003)(356005)(86362001)(316002)(110136005)(54906003)(2906002)(55016003)(70586007)(70206006); DIR:OUT; SFP:1101; X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 21 Dec 2022 09:00:50.3740 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: 26437a40-5ec4-487c-2de3-08dae331dbab X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=43083d15-7273-40c1-b7db-39efd9ccc17a; Ip=[216.228.117.160]; Helo=[mail.nvidia.com] X-MS-Exchange-CrossTenant-AuthSource: CO1NAM11FT042.eop-nam11.prod.protection.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: PH8PR12MB7304 X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Users may want to change the DPDK process to different versions such as hot upgrade. There is a strong requirement to simplify the logic and shorten the traffic downtime as much as possible. This update introduces new rte_eth process role definitions: active or standby. The active role means rules are programmed to HW immediately, and no behavior changed. This is the default state. The standby role means rules are queued in the HW. If no active roles alive or back to active, the rules are effective immediately. Signed-off-by: Rongwei Liu --- doc/guides/nics/mlx5.rst | 10 ++++ doc/guides/rel_notes/release_22_03.rst | 5 ++ lib/ethdev/ethdev_driver.h | 63 ++++++++++++++++++++++++++ lib/ethdev/rte_ethdev.c | 41 +++++++++++++++++ lib/ethdev/rte_ethdev.h | 7 ++- lib/ethdev/version.map | 3 ++ 6 files changed, 128 insertions(+), 1 deletion(-) diff --git a/doc/guides/nics/mlx5.rst b/doc/guides/nics/mlx5.rst index 51f51259e3..de1fdac0a1 100644 --- a/doc/guides/nics/mlx5.rst +++ b/doc/guides/nics/mlx5.rst @@ -2001,3 +2001,13 @@ where: * ``sw_queue_id``: queue index in range [64536, 65535]. This range is the highest 1000 numbers. * ``hw_queue_id``: queue index given by HW in queue creation. + +ethdev set process active or standby +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +User should only program group 0 (fdb_def_rule_en=0) when ``rte_eth_process_set_active`` +has been called and set to a standby role. +Group 0 is shared across different DPDK processes while the other groups are limited +to the current process scope. +The process can't move from active to standby role if preceding active application's +rules are still present and vice versa. diff --git a/doc/guides/rel_notes/release_22_03.rst b/doc/guides/rel_notes/release_22_03.rst index 0923707cb8..6fa48106c4 100644 --- a/doc/guides/rel_notes/release_22_03.rst +++ b/doc/guides/rel_notes/release_22_03.rst @@ -207,6 +207,11 @@ API Changes * ethdev: Old public macros and enumeration constants without ``RTE_ETH_`` prefix, which are kept for backward compatibility, are marked as deprecated. +* ethdev: added a new experimental api: + + The new API ``rte_eth_process_set_active()`` was added. + If ``RTE_ETH_CAPA_PROCESS_SET_ROLE`` is not advertised, this new api is not supported. + * cryptodev: The asymmetric session handling was modified to use a single mempool object. An API ``rte_cryptodev_asym_session_pool_create`` was added to create a mempool with element size big enough to hold the generic asymmetric diff --git a/lib/ethdev/ethdev_driver.h b/lib/ethdev/ethdev_driver.h index 6a550cfc83..3c583bc39d 100644 --- a/lib/ethdev/ethdev_driver.h +++ b/lib/ethdev/ethdev_driver.h @@ -179,6 +179,16 @@ struct rte_eth_dev_data { pthread_mutex_t flow_ops_mutex; /**< rte_flow ops mutex */ } __rte_cache_aligned; +/**@{@name Different rte_eth role flag definitions which will be used + * when miagrating DPDK to a different version. + */ +/* + * Traffic coming from NIC domain rules will reach + * both active and standby processes. + */ +#define RTE_ETH_PROCESS_NIC_DUP_WITH_STANDBY RTE_BIT32(0), +/**@}*/ + /** * @internal * The pool of *rte_eth_dev* structures. The size of the pool @@ -1087,6 +1097,22 @@ typedef const uint32_t *(*eth_buffer_split_supported_hdr_ptypes_get_t)(struct rt */ typedef int (*eth_dev_priv_dump_t)(struct rte_eth_dev *dev, FILE *file); +/** + * @internal + * Set rte_eth process to active or standby role. + * + * @param dev + * Port (ethdev) handle. + * @param active + * Device (role) active or not (standby). + * @param flag + * Role specific flag. + * + * @return + * Negative value on error, 0 on success. + */ +typedef int (*eth_process_set_active_t)(struct rte_eth_dev *dev, bool active, uint32_t flag); + /** * @internal Set Rx queue available descriptors threshold. * @see rte_eth_rx_avail_thresh_set() @@ -1403,6 +1429,8 @@ struct eth_dev_ops { eth_cman_config_set_t cman_config_set; /** Retrieve congestion management configuration */ eth_cman_config_get_t cman_config_get; + /** Set the whole rte_eth process to active or standby role. */ + eth_process_set_active_t eth_process_set_active; }; /** @@ -2046,6 +2074,41 @@ struct rte_eth_fdir_conf { struct rte_eth_fdir_flex_conf flex_conf; }; +/** + * @warning + * @b EXPERIMENTAL: this API may change without prior notice + * + * Set the rte_eth process to the active or standby role which affects + * the flow rules offloading. It doesn't allow multiple processes to be the + * same role unless no offload rules are set. + * The active process flow rules are effective immediately while the standby + * process rules will be matched (active) when the process becomes active or + * when the traffic is not matched by the active process rules. + * The active application will always receive traffic while the standby + * application will receive traffic when no matching rules are present from + * the active application. + * + * The application is active by default if this API is not called. + * + * When a process transforms from a standby to a active role, all preceding + * flow rules which are queued by hardware will be effective immediately. + * Before role transition, all the rules set by the active process should be + * flushed first. + * + * When role flag "RTE_ETH_PROCESS_NIC_DUP_WITH_STANDBY" is set, NIC domain + * flow rules are effective immediately even if a process is standby role. + * + * @param active + * Process active (role) or not (standby). + * @param flag + * The role flag. + * @return + * - (>=0) Number of rte devices which have been switched successfully. + * - (-EINVAL) if bad parameter. + */ +__rte_experimental +int rte_eth_process_set_active(bool active, uint32_t flag); + #ifdef __cplusplus } #endif diff --git a/lib/ethdev/rte_ethdev.c b/lib/ethdev/rte_ethdev.c index 5d5e18db1e..f19da75bfe 100644 --- a/lib/ethdev/rte_ethdev.c +++ b/lib/ethdev/rte_ethdev.c @@ -6318,6 +6318,47 @@ rte_eth_buffer_split_get_supported_hdr_ptypes(uint16_t port_id, uint32_t *ptypes return j; } +int rte_eth_process_set_active(bool active, uint32_t flag) +{ + struct rte_eth_dev_info dev_info = {0}; + uint32_t flags[RTE_MAX_ETHPORTS]; + struct rte_eth_dev *dev; + uint16_t port_id; + int ret = 0; + + /* Check if all devices support. */ + RTE_ETH_FOREACH_DEV(port_id) { + dev = &rte_eth_devices[port_id]; + if (*dev->dev_ops->dev_infos_get == NULL || + *dev->dev_ops->eth_process_set_active == NULL) + return -ENOTSUP; + if ((*dev->dev_ops->dev_infos_get)(dev, &dev_info)) + return -EINVAL; + if (!(dev_info.dev_capa & RTE_ETH_CAPA_PROCESS_SET_ROLE)) + return -ENOTSUP; + } + RTE_ETH_FOREACH_DEV(port_id) { + dev = &rte_eth_devices[port_id]; + if ((*dev->dev_ops->dev_infos_get)(dev, &dev_info)) + goto err; + flags[port_id] = dev_info.eth_process_flag; + if ((*dev->dev_ops->eth_process_set_active)(dev, active, flag) < 0) + goto err; + ret++; + } + return ret; +err: + if (!ret) + return 0; + RTE_ETH_FOREACH_DEV(port_id) { + dev = &rte_eth_devices[port_id]; + (*dev->dev_ops->eth_process_set_active)(dev, !active, flags[port_id]); + if (--ret == 0) + break; + } + return 0; +} + RTE_LOG_REGISTER_DEFAULT(rte_eth_dev_logtype, INFO); RTE_INIT(ethdev_init_telemetry) diff --git a/lib/ethdev/rte_ethdev.h b/lib/ethdev/rte_ethdev.h index c129ca1eaf..d29f051d6f 100644 --- a/lib/ethdev/rte_ethdev.h +++ b/lib/ethdev/rte_ethdev.h @@ -1606,6 +1606,8 @@ struct rte_eth_conf { #define RTE_ETH_DEV_CAPA_FLOW_RULE_KEEP RTE_BIT64(3) /** Device supports keeping shared flow objects across restart. */ #define RTE_ETH_DEV_CAPA_FLOW_SHARED_OBJECT_KEEP RTE_BIT64(4) +/**Device supports "rte_eth_process_set_active" callback. */ +#define RTE_ETH_CAPA_PROCESS_SET_ROLE RTE_BIT64(5) /**@}*/ /* @@ -1777,8 +1779,11 @@ struct rte_eth_dev_info { struct rte_eth_switch_info switch_info; /** Supported error handling mode. */ enum rte_eth_err_handle_mode err_handle_mode; + /** Process specific role flag. */ + uint32_t eth_process_flag; - uint64_t reserved_64s[2]; /**< Reserved for future fields */ + uint32_t reserved_32s[1]; /**< Reserved for future fields */ + uint64_t reserved_64s[1]; /**< Reserved for future fields */ void *reserved_ptrs[2]; /**< Reserved for future fields */ }; diff --git a/lib/ethdev/version.map b/lib/ethdev/version.map index 17201fbe0f..a5503f6fde 100644 --- a/lib/ethdev/version.map +++ b/lib/ethdev/version.map @@ -298,6 +298,9 @@ EXPERIMENTAL { rte_flow_get_q_aged_flows; rte_mtr_meter_policy_get; rte_mtr_meter_profile_get; + + # added in 23.03 + rte_eth_process_set_active; }; INTERNAL { -- 2.27.0