From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 8A03DA00C5; Thu, 1 Dec 2022 09:20:41 +0100 (CET) Received: from mails.dpdk.org (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 7F3B742D18; Thu, 1 Dec 2022 09:20:41 +0100 (CET) Received: from NAM11-CO1-obe.outbound.protection.outlook.com (mail-co1nam11on2062.outbound.protection.outlook.com [40.107.220.62]) by mails.dpdk.org (Postfix) with ESMTP id 09FFF42D0D for ; Thu, 1 Dec 2022 09:20:40 +0100 (CET) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=FK7Vhh81Rnm2/NXbS6iW/SbtMrA/RImnjM6FuoyAx2Pvizcz4wpvUcg62wArA6uuMW7U5PThvEkU/aK4v7cm8P+47QwpvP7WYsGeHnQ9k/xVAZPio5ZbI7VsDU2PCwYvYaPxfjyD5g3pYevjayml7WadvtzCkJ47mVHkXIot52SiCSu5/Xwi4x4UDn2mAr4yZW0m852y4pYk0WutQCpBSYknsA/pKonMbN/6EeR+vdsHELhOW2QIO2Re89UAhBmDoVAgZxyOP0Ou2GKvEtuK6+3onXBUa890b72DKQp+zdnkPMdKXduOWwp+VOhT94YiUO1x9VB5uo6Pq3er1Z7Wgw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=lexcSEG5TZ31Wj+U+S0YrDEvy9LtNZyDO4VTIrPp1Lo=; b=oX8gsO5butEQDZX3bWD8ruwjXE5jvokSE08THd3+y45P4e5XO8xjSgkaFYaZfWTzLvTGmsDlX8js0R54rkkfdqDHJOZs5L+b4pJCxDxhudUs3ykYPBiPbz/s7qPOiTcrT7DU6aCnfZJ7xVloft61BU6a5GMY727Y1KjWgr/ijBN8ACFtyULHjloX+VEXeWvk+V0fxKVnupNuDq1jLuwxWKwGZHEJXFNxn6/J2/f8VUUgC3p0r4U5DiMtOYr4KlDH0+fIk7hGlQiZLuU+mSGK71gJOMPoHNDSItMRe4hDmJqW4R9iWNVLNPPvf77HvzkQaS1r3UqUUs3r0MV04XnhMA== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 216.228.117.161) smtp.rcpttodomain=monjalon.net smtp.mailfrom=nvidia.com; dmarc=pass (p=reject sp=reject pct=100) action=none header.from=nvidia.com; dkim=none (message not signed); arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=lexcSEG5TZ31Wj+U+S0YrDEvy9LtNZyDO4VTIrPp1Lo=; b=pTg1ui3/ht1YyAC75R6t+nOkEiQpiTKKZTKr2lfFR2sLZjkZKxOfxAh6EI70yjiPFwlHvr2MJgLUW2Hj19muA028d1h78Z3gD9SQkor5caDAht8hrRVqPQfzbE3WFIDph+YbIsMU25nQsQEHAu2+kqxq7oMVgr9HDYk8/gvLB1QGlh+3wOQdIzWuzIzV7TSCYDSRV4XcL6ScXY2GMO3oWL1oWlO3EnqKSnpYrcpb9LB2MzN18vo+UqLp0NjmKzexRSGcEg2bkL3fWIbstljsXHfYy/2cZkEeChP+Uc9IF6qsxith48czN1ygzO/AlCHs5G+ZhM6x35EBO6Kf1w6AHA== Received: from DS7PR03CA0284.namprd03.prod.outlook.com (2603:10b6:5:3ad::19) by SA1PR12MB6946.namprd12.prod.outlook.com (2603:10b6:806:24d::17) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5857.23; Thu, 1 Dec 2022 08:20:37 +0000 Received: from DM6NAM11FT086.eop-nam11.prod.protection.outlook.com (2603:10b6:5:3ad:cafe::3c) by DS7PR03CA0284.outlook.office365.com (2603:10b6:5:3ad::19) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5857.23 via Frontend Transport; Thu, 1 Dec 2022 08:20:37 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 216.228.117.161) smtp.mailfrom=nvidia.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=nvidia.com; Received-SPF: Pass (protection.outlook.com: domain of nvidia.com designates 216.228.117.161 as permitted sender) receiver=protection.outlook.com; client-ip=216.228.117.161; helo=mail.nvidia.com; pr=C Received: from mail.nvidia.com (216.228.117.161) by DM6NAM11FT086.mail.protection.outlook.com (10.13.173.75) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5880.8 via Frontend Transport; Thu, 1 Dec 2022 08:20:37 +0000 Received: from rnnvmail201.nvidia.com (10.129.68.8) by mail.nvidia.com (10.129.200.67) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.986.36; Thu, 1 Dec 2022 00:20:27 -0800 Received: from nvidia.com (10.126.230.35) by rnnvmail201.nvidia.com (10.129.68.8) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.986.36; Thu, 1 Dec 2022 00:20:24 -0800 From: Rongwei Liu To: , , , , Ferruh Yigit , Andrew Rybchenko CC: , Subject: [RFC 2/2] ethdev: add API to set process to primary or secondary Date: Thu, 1 Dec 2022 10:20:05 +0200 Message-ID: <20221201082005.732252-3-rongweil@nvidia.com> X-Mailer: git-send-email 2.27.0 In-Reply-To: <20221201082005.732252-1-rongweil@nvidia.com> References: <20221201082005.732252-1-rongweil@nvidia.com> MIME-Version: 1.0 Content-Transfer-Encoding: 8bit Content-Type: text/plain X-Originating-IP: [10.126.230.35] X-ClientProxiedBy: rnnvmail203.nvidia.com (10.129.68.9) To rnnvmail201.nvidia.com (10.129.68.8) X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: DM6NAM11FT086:EE_|SA1PR12MB6946:EE_ X-MS-Office365-Filtering-Correlation-Id: 782aa066-a5fb-4083-615b-08dad374ed43 X-LD-Processed: 43083d15-7273-40c1-b7db-39efd9ccc17a,ExtAddr X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: 9AlPHHMSC0YxEUk05qpxAhYGcpof53wGDM7iBFyxn9Ok71q6hgCuI3fcnyHBgjS1BtjyJ06Cusp058XLJiQQhmSaquDH1c0Rv3VF42B+P01dYFSSmiq/N3jqNOqStajdaBb6AvZCoeTnPHD4QJxjzNZZWpYpnkpM5TJ05SBBF1v4/UjXMWSkSe+w14ZND0mV2/J52geI+Zwz+ZCNbwunLvWhn6ifYLhjrcTdqOItBTJD/NhywaxX9axG+6YFpWswHFX1e2oE/E8OaOClvO+WDBc7MjI69SROVYB/1z8V0xPNCMtf1muu0Dsr8iyABOEPB5U500mVvWBXDk8WMTv3eMQiyeiaZUPkvNaTyfc5PIkOdLDF2965ImCVB4W/TAM4eWhGbVjpvMYirUxGQyws8Bl6C6MayTink0STUzrVl+vnuWwOErLhD8ZRNQWuoXsB7nAnILgKavAE2U6x0y8ha4c9ktE0hC/A9xqwhG8BKNmyy1owExNSWmMiTUif9TAauKj/oa81EjbPzN2H0qx3oqGyOS4YLjOzId9VxH3e/luSHN90sKP1SEFJb/Yv+1NNA+5bEm7/SdSecwnEtipHc2zrXmEBJ+1sDpJ78KaEzwJX1w27kmG6A/sif7SY5/ac1DKLIhQ+kEccOBjBLofPvMWKqvLq5qPqXU7xyqzZISJx9zk1cSfebvBdo8q7oC2hmxxXjWaA4WzXDuWolMNaIg== X-Forefront-Antispam-Report: CIP:216.228.117.161; CTRY:US; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:mail.nvidia.com; PTR:dc6edge2.nvidia.com; CAT:NONE; SFS:(13230022)(4636009)(39860400002)(346002)(396003)(376002)(136003)(451199015)(36840700001)(46966006)(40470700004)(36860700001)(478600001)(2906002)(83380400001)(86362001)(6666004)(107886003)(356005)(5660300002)(54906003)(82740400003)(110136005)(7636003)(40460700003)(6286002)(426003)(82310400005)(47076005)(36756003)(8936002)(41300700001)(1076003)(7696005)(2616005)(8676002)(336012)(4326008)(26005)(316002)(40480700001)(55016003)(16526019)(186003)(70586007)(70206006); DIR:OUT; SFP:1101; X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 01 Dec 2022 08:20:37.5067 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: 782aa066-a5fb-4083-615b-08dad374ed43 X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=43083d15-7273-40c1-b7db-39efd9ccc17a; Ip=[216.228.117.161]; Helo=[mail.nvidia.com] X-MS-Exchange-CrossTenant-AuthSource: DM6NAM11FT086.eop-nam11.prod.protection.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: SA1PR12MB6946 X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Users may want to change the DPDK process to different versions such as hot upgrade. There is a strong requirement to simplify the logic and shorten the traffic downtime as much as possible. This update introduces new rte_eth process role definitions: primary or secondary. The primary role means rules are programmed to HW immediately, and no behavior changed. This is the default state. The secondary role means rules are queued in the HW. If no primary roles alive or back to primary, the rules are effective immediately. Signed-off-by: Rongwei Liu --- doc/guides/nics/mlx5.rst | 10 ++++++ lib/ethdev/ethdev_driver.h | 63 ++++++++++++++++++++++++++++++++++++++ lib/ethdev/rte_ethdev.c | 20 ++++++++++++ lib/ethdev/version.map | 3 ++ 4 files changed, 96 insertions(+) diff --git a/doc/guides/nics/mlx5.rst b/doc/guides/nics/mlx5.rst index 51f51259e3..ea78e14b80 100644 --- a/doc/guides/nics/mlx5.rst +++ b/doc/guides/nics/mlx5.rst @@ -2001,3 +2001,13 @@ where: * ``sw_queue_id``: queue index in range [64536, 65535]. This range is the highest 1000 numbers. * ``hw_queue_id``: queue index given by HW in queue creation. + +ethdev set process primary or secondary +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +User should only program group 0 (fdb_def_rule_en=0) when ``rte_eth_process_set_primary`` +has been called and set to a secondary role. +Group 0 is shared across different DPDK processes while the other groups are limited +to the current process scope. +The process can't move from primary to secondary role if preceding primary application's +rules are still present and vice versa. diff --git a/lib/ethdev/ethdev_driver.h b/lib/ethdev/ethdev_driver.h index 6a550cfc83..67b53840c8 100644 --- a/lib/ethdev/ethdev_driver.h +++ b/lib/ethdev/ethdev_driver.h @@ -179,6 +179,16 @@ struct rte_eth_dev_data { pthread_mutex_t flow_ops_mutex; /**< rte_flow ops mutex */ } __rte_cache_aligned; +/**@{@name Different rte_eth role flag definitions which will be used + * when miagrating DPDK to a different version. + */ +/* + * Traffic coming from NIC domain rules will reach + * both primary and secondary processes. + */ +#define RTE_ETH_PROCESS_NIC_DUP_WITH_SECONDARY RTE_BIT32(0), +/**@}*/ + /** * @internal * The pool of *rte_eth_dev* structures. The size of the pool @@ -1087,6 +1097,22 @@ typedef const uint32_t *(*eth_buffer_split_supported_hdr_ptypes_get_t)(struct rt */ typedef int (*eth_dev_priv_dump_t)(struct rte_eth_dev *dev, FILE *file); +/** + * @internal + * Set rte_eth process to primary or secondary role. + * + * @param dev + * Port (ethdev) handle. + * @param active + * Device (role) primary or not (secondary). + * @param flag + * Role specific flag. + * + * @return + * Negative value on error, 0 on success. + */ +typedef int (*eth_process_set_primary_t)(struct rte_eth_dev *dev, bool primary, uint32_t flag); + /** * @internal Set Rx queue available descriptors threshold. * @see rte_eth_rx_avail_thresh_set() @@ -1403,6 +1429,8 @@ struct eth_dev_ops { eth_cman_config_set_t cman_config_set; /** Retrieve congestion management configuration */ eth_cman_config_get_t cman_config_get; + /** Set the whole rte_eth process to primary or secondary role. */ + eth_process_set_primary_t eth_process_set_primary; }; /** @@ -2046,6 +2074,41 @@ struct rte_eth_fdir_conf { struct rte_eth_fdir_flex_conf flex_conf; }; +/** + * @warning + * @b EXPERIMENTAL: this API may change without prior notice + * + * Set the rte_eth process to the primary or secondary role which affects + * the flow rules offloading. It doesn't allow multiple processes to be the + * same role unless no offload rules are set. + * The primary process's flow rules are effective immediately while the secondary + * process's rules will be queued in hardware until it becomes primary or no + * primary process is alive. + * The primary application will always receive traffic while the secondary + * application will receive traffic when no matching rules are present from + * the primary application. + * + * The application is primary by default if this API is not called. + * + * When a process transforms from a secondary to a primary role, all preceding + * flow rules which are queued by hardware will be effective immediately. + * Before role transition, all the rules set by the primary process should be + * flushed first. + * + * When role flag "RTE_ETH_PROCESS_NIC_DUP_WITH_SECONDARY" is set, NIC domain + * flow rules are effective immediately even if a process is secondary. + * + * @param active + * Process primary (role) or not (secondary). + * @param flag + * The role flag. + * @return + * - (>=0) Number of rte devices which have been switched successfully. + * - (-EINVAL) if bad parameter. + */ +__rte_experimental +int rte_eth_process_set_primary(bool primary, uint32_t flag); + #ifdef __cplusplus } #endif diff --git a/lib/ethdev/rte_ethdev.c b/lib/ethdev/rte_ethdev.c index 5d5e18db1e..251908b4e3 100644 --- a/lib/ethdev/rte_ethdev.c +++ b/lib/ethdev/rte_ethdev.c @@ -6318,6 +6318,26 @@ rte_eth_buffer_split_get_supported_hdr_ptypes(uint16_t port_id, uint32_t *ptypes return j; } +int rte_eth_process_set_primary(bool primary, uint32_t flag) +{ + struct rte_eth_dev *dev; + uint16_t port_id; + int ret, val; + + ret = 0; + RTE_ETH_FOREACH_DEV(port_id) { + dev = &rte_eth_devices[port_id]; + if (*dev->dev_ops->eth_process_set_primary == NULL) + return -ENOTSUP; + val = (*dev->dev_ops->eth_process_set_primary)(dev, primary, flag); + + if (val < 0) + return -EINVAL; + ret++; + } + return ret; +} + RTE_LOG_REGISTER_DEFAULT(rte_eth_dev_logtype, INFO); RTE_INIT(ethdev_init_telemetry) diff --git a/lib/ethdev/version.map b/lib/ethdev/version.map index 17201fbe0f..1823869e73 100644 --- a/lib/ethdev/version.map +++ b/lib/ethdev/version.map @@ -298,6 +298,9 @@ EXPERIMENTAL { rte_flow_get_q_aged_flows; rte_mtr_meter_policy_get; rte_mtr_meter_profile_get; + + # added in 23.03 + rte_eth_process_set_primary; }; INTERNAL { -- 2.27.0