From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 4F522A034C; Wed, 21 Dec 2022 10:13:28 +0100 (CET) Received: from mails.dpdk.org (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 3F1AB41141; Wed, 21 Dec 2022 10:13:28 +0100 (CET) Received: from mail-vk1-f173.google.com (mail-vk1-f173.google.com [209.85.221.173]) by mails.dpdk.org (Postfix) with ESMTP id 6B94641141 for ; Wed, 21 Dec 2022 10:13:26 +0100 (CET) Received: by mail-vk1-f173.google.com with SMTP id x65so6936655vkg.11 for ; Wed, 21 Dec 2022 01:13:26 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=cc:to:subject:message-id:date:from:in-reply-to:references :mime-version:from:to:cc:subject:date:message-id:reply-to; bh=lw4nY1squ7Zr3Xdv0d1CBk3A5ut6zRouVjrkNny3QXc=; b=geh/irGWZmpILfEQqOzjruygA9IXTSwN+6xkW+rnyZ+Nwm0xmDt4pkRVEG4DbhYZwG KPOclEw9shLh5Cw0melQNJTpYpeB03+1QzsEHbpnKMscmsnPbtHuADwlYLzQY9/DYaeG RQPRveUnaV0CUG8CDSndGgNSK7e6N9FKOk9veU7mH3p+Ry7OcbiRoYxx2b2RY/QyfS1N CLp9oQzo1cqFF3cvQylhRZhT2wIQkd6f0CPyF5fiF5PXfqIWRGwzwpGynP5DR+eQS2nH zx8dAj8Ut6nsLSoN5ZQJyLcURF/l4b826sQED9I/Kb03T3sDDF7eFe97mhH6Hc1jP5hR GDKw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=cc:to:subject:message-id:date:from:in-reply-to:references :mime-version:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=lw4nY1squ7Zr3Xdv0d1CBk3A5ut6zRouVjrkNny3QXc=; b=ZgGrpqZ9Gk4SNEniUfeKMPPQJkjXtrBck2lZfC2OvCwE5/eCit6jO1JgHLRO+2zg5v AFUz3q2wHKp3MptBDeOOCPzY2cOW8tsqRpd+OyZHultZjFKzXY88baIJEJWJVn82CoH7 9gf1i+fUW5dXVv73rxTvflrR/G29mV/v2f88HVVDOGJQO4E0pFdViP9/qWWTEadyTrNb Y/W1oPXyPD66u1E8nCeBF5vsFI/fD1Cvcewx7z4/yHRXR3UPXXGd9vZqdoqP08RyvEvu l1QcxJPiE9cEymcG6YwguDLdxWW6Nx+0G6dmR8Lg3NqjaWWkSpzlaV7PlR07N9T7aLXM k5gg== X-Gm-Message-State: AFqh2kryXrpPDcWkgydw7AWlwqtAwFro3XPcD33U5tDMRY9ord2gZ+zm zcGXiTU8MDIwdzpHLrFt66yM4ogP2iJmWUTrbqE= X-Google-Smtp-Source: AMrXdXulYPPuGVxr8Cgrxa0Sr+u73BNGcJG1mbam4a8waEVSBn5nnYYxtohELQ73nbDBCsp/UvNYJpjac/U10iIbLrg= X-Received: by 2002:ac5:cb62:0:b0:3d2:1350:bdb0 with SMTP id l2-20020ac5cb62000000b003d21350bdb0mr102252vkn.20.1671614004243; Wed, 21 Dec 2022 01:13:24 -0800 (PST) MIME-Version: 1.0 References: <20221205215416.7ac53a55@hermes.local> <20221221090017.3715030-1-rongweil@nvidia.com> <20221221090017.3715030-3-rongweil@nvidia.com> In-Reply-To: <20221221090017.3715030-3-rongweil@nvidia.com> From: Jerin Jacob Date: Wed, 21 Dec 2022 14:42:58 +0530 Message-ID: Subject: Re: [RFC v3 2/2] ethdev: add API to set process to active or standby To: Rongwei Liu Cc: matan@nvidia.com, viacheslavo@nvidia.com, orika@nvidia.com, thomas@monjalon.net, Ferruh Yigit , Andrew Rybchenko , dev@dpdk.org, rasland@nvidia.com Content-Type: text/plain; charset="UTF-8" X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org On Wed, Dec 21, 2022 at 2:31 PM Rongwei Liu wrote: > > Users may want to change the DPDK process to different versions Different version of DPDK? If there is any ABI change how to support this? > such as hot upgrade. > There is a strong requirement to simplify the logic and shorten the > traffic downtime as much as possible. > > This update introduces new rte_eth process role definitions: active > or standby. > > The active role means rules are programmed to HW immediately, and no Why it has to be specific only to rte_flow rule? If it spedieic to rte_flow, why it is in rte_eth_process_ name space? Also, if we are moving the standby, What about the rule whose ABI is changed between versions? > behavior changed. This is the default state. > The standby role means rules are queued in the HW. If no active roles > alive or back to active, the rules are effective immediately. > > Signed-off-by: Rongwei Liu > --- > doc/guides/nics/mlx5.rst | 10 ++++ > doc/guides/rel_notes/release_22_03.rst | 5 ++ > lib/ethdev/ethdev_driver.h | 63 ++++++++++++++++++++++++++ > lib/ethdev/rte_ethdev.c | 41 +++++++++++++++++ > lib/ethdev/rte_ethdev.h | 7 ++- > lib/ethdev/version.map | 3 ++ > 6 files changed, 128 insertions(+), 1 deletion(-) > > diff --git a/doc/guides/nics/mlx5.rst b/doc/guides/nics/mlx5.rst > index 51f51259e3..de1fdac0a1 100644 > --- a/doc/guides/nics/mlx5.rst > +++ b/doc/guides/nics/mlx5.rst > @@ -2001,3 +2001,13 @@ where: > * ``sw_queue_id``: queue index in range [64536, 65535]. > This range is the highest 1000 numbers. > * ``hw_queue_id``: queue index given by HW in queue creation. > + > +ethdev set process active or standby > +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ > + > +User should only program group 0 (fdb_def_rule_en=0) when ``rte_eth_process_set_active`` > +has been called and set to a standby role. > +Group 0 is shared across different DPDK processes while the other groups are limited > +to the current process scope. > +The process can't move from active to standby role if preceding active application's > +rules are still present and vice versa. > diff --git a/doc/guides/rel_notes/release_22_03.rst b/doc/guides/rel_notes/release_22_03.rst > index 0923707cb8..6fa48106c4 100644 > --- a/doc/guides/rel_notes/release_22_03.rst > +++ b/doc/guides/rel_notes/release_22_03.rst > @@ -207,6 +207,11 @@ API Changes > * ethdev: Old public macros and enumeration constants without ``RTE_ETH_`` prefix, > which are kept for backward compatibility, are marked as deprecated. > > +* ethdev: added a new experimental api: > + > + The new API ``rte_eth_process_set_active()`` was added. > + If ``RTE_ETH_CAPA_PROCESS_SET_ROLE`` is not advertised, this new api is not supported. > + > * cryptodev: The asymmetric session handling was modified to use a single > mempool object. An API ``rte_cryptodev_asym_session_pool_create`` was added > to create a mempool with element size big enough to hold the generic asymmetric > diff --git a/lib/ethdev/ethdev_driver.h b/lib/ethdev/ethdev_driver.h > index 6a550cfc83..3c583bc39d 100644 > --- a/lib/ethdev/ethdev_driver.h > +++ b/lib/ethdev/ethdev_driver.h > @@ -179,6 +179,16 @@ struct rte_eth_dev_data { > pthread_mutex_t flow_ops_mutex; /**< rte_flow ops mutex */ > } __rte_cache_aligned; > > +/**@{@name Different rte_eth role flag definitions which will be used > + * when miagrating DPDK to a different version. > + */ > +/* > + * Traffic coming from NIC domain rules will reach > + * both active and standby processes. > + */ > +#define RTE_ETH_PROCESS_NIC_DUP_WITH_STANDBY RTE_BIT32(0), > +/**@}*/ > + > /** > * @internal > * The pool of *rte_eth_dev* structures. The size of the pool > @@ -1087,6 +1097,22 @@ typedef const uint32_t *(*eth_buffer_split_supported_hdr_ptypes_get_t)(struct rt > */ > typedef int (*eth_dev_priv_dump_t)(struct rte_eth_dev *dev, FILE *file); > > +/** > + * @internal > + * Set rte_eth process to active or standby role. > + * > + * @param dev > + * Port (ethdev) handle. > + * @param active > + * Device (role) active or not (standby). > + * @param flag > + * Role specific flag. > + * > + * @return > + * Negative value on error, 0 on success. > + */ > +typedef int (*eth_process_set_active_t)(struct rte_eth_dev *dev, bool active, uint32_t flag); > + > /** > * @internal Set Rx queue available descriptors threshold. > * @see rte_eth_rx_avail_thresh_set() > @@ -1403,6 +1429,8 @@ struct eth_dev_ops { > eth_cman_config_set_t cman_config_set; > /** Retrieve congestion management configuration */ > eth_cman_config_get_t cman_config_get; > + /** Set the whole rte_eth process to active or standby role. */ > + eth_process_set_active_t eth_process_set_active; > }; > > /** > @@ -2046,6 +2074,41 @@ struct rte_eth_fdir_conf { > struct rte_eth_fdir_flex_conf flex_conf; > }; > > +/** > + * @warning > + * @b EXPERIMENTAL: this API may change without prior notice > + * > + * Set the rte_eth process to the active or standby role which affects > + * the flow rules offloading. It doesn't allow multiple processes to be the > + * same role unless no offload rules are set. > + * The active process flow rules are effective immediately while the standby > + * process rules will be matched (active) when the process becomes active or > + * when the traffic is not matched by the active process rules. > + * The active application will always receive traffic while the standby > + * application will receive traffic when no matching rules are present from > + * the active application. > + * > + * The application is active by default if this API is not called. > + * > + * When a process transforms from a standby to a active role, all preceding > + * flow rules which are queued by hardware will be effective immediately. > + * Before role transition, all the rules set by the active process should be > + * flushed first. > + * > + * When role flag "RTE_ETH_PROCESS_NIC_DUP_WITH_STANDBY" is set, NIC domain > + * flow rules are effective immediately even if a process is standby role. > + * > + * @param active > + * Process active (role) or not (standby). > + * @param flag > + * The role flag. > + * @return > + * - (>=0) Number of rte devices which have been switched successfully. > + * - (-EINVAL) if bad parameter. > + */ > +__rte_experimental > +int rte_eth_process_set_active(bool active, uint32_t flag); > + > #ifdef __cplusplus > } > #endif > diff --git a/lib/ethdev/rte_ethdev.c b/lib/ethdev/rte_ethdev.c > index 5d5e18db1e..f19da75bfe 100644 > --- a/lib/ethdev/rte_ethdev.c > +++ b/lib/ethdev/rte_ethdev.c > @@ -6318,6 +6318,47 @@ rte_eth_buffer_split_get_supported_hdr_ptypes(uint16_t port_id, uint32_t *ptypes > return j; > } > > +int rte_eth_process_set_active(bool active, uint32_t flag) > +{ > + struct rte_eth_dev_info dev_info = {0}; > + uint32_t flags[RTE_MAX_ETHPORTS]; > + struct rte_eth_dev *dev; > + uint16_t port_id; > + int ret = 0; > + > + /* Check if all devices support. */ > + RTE_ETH_FOREACH_DEV(port_id) { > + dev = &rte_eth_devices[port_id]; > + if (*dev->dev_ops->dev_infos_get == NULL || > + *dev->dev_ops->eth_process_set_active == NULL) > + return -ENOTSUP; > + if ((*dev->dev_ops->dev_infos_get)(dev, &dev_info)) > + return -EINVAL; > + if (!(dev_info.dev_capa & RTE_ETH_CAPA_PROCESS_SET_ROLE)) > + return -ENOTSUP; > + } > + RTE_ETH_FOREACH_DEV(port_id) { > + dev = &rte_eth_devices[port_id]; > + if ((*dev->dev_ops->dev_infos_get)(dev, &dev_info)) > + goto err; > + flags[port_id] = dev_info.eth_process_flag; > + if ((*dev->dev_ops->eth_process_set_active)(dev, active, flag) < 0) > + goto err; > + ret++; > + } > + return ret; > +err: > + if (!ret) > + return 0; > + RTE_ETH_FOREACH_DEV(port_id) { > + dev = &rte_eth_devices[port_id]; > + (*dev->dev_ops->eth_process_set_active)(dev, !active, flags[port_id]); > + if (--ret == 0) > + break; > + } > + return 0; > +} > + > RTE_LOG_REGISTER_DEFAULT(rte_eth_dev_logtype, INFO); > > RTE_INIT(ethdev_init_telemetry) > diff --git a/lib/ethdev/rte_ethdev.h b/lib/ethdev/rte_ethdev.h > index c129ca1eaf..d29f051d6f 100644 > --- a/lib/ethdev/rte_ethdev.h > +++ b/lib/ethdev/rte_ethdev.h > @@ -1606,6 +1606,8 @@ struct rte_eth_conf { > #define RTE_ETH_DEV_CAPA_FLOW_RULE_KEEP RTE_BIT64(3) > /** Device supports keeping shared flow objects across restart. */ > #define RTE_ETH_DEV_CAPA_FLOW_SHARED_OBJECT_KEEP RTE_BIT64(4) > +/**Device supports "rte_eth_process_set_active" callback. */ > +#define RTE_ETH_CAPA_PROCESS_SET_ROLE RTE_BIT64(5) > /**@}*/ > > /* > @@ -1777,8 +1779,11 @@ struct rte_eth_dev_info { > struct rte_eth_switch_info switch_info; > /** Supported error handling mode. */ > enum rte_eth_err_handle_mode err_handle_mode; > + /** Process specific role flag. */ > + uint32_t eth_process_flag; > > - uint64_t reserved_64s[2]; /**< Reserved for future fields */ > + uint32_t reserved_32s[1]; /**< Reserved for future fields */ > + uint64_t reserved_64s[1]; /**< Reserved for future fields */ > void *reserved_ptrs[2]; /**< Reserved for future fields */ > }; > > diff --git a/lib/ethdev/version.map b/lib/ethdev/version.map > index 17201fbe0f..a5503f6fde 100644 > --- a/lib/ethdev/version.map > +++ b/lib/ethdev/version.map > @@ -298,6 +298,9 @@ EXPERIMENTAL { > rte_flow_get_q_aged_flows; > rte_mtr_meter_policy_get; > rte_mtr_meter_profile_get; > + > + # added in 23.03 > + rte_eth_process_set_active; > }; > > INTERNAL { > -- > 2.27.0 >