From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id F3628A00C2; Thu, 6 Oct 2022 10:36:28 +0200 (CEST) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 72E7142BAE; Thu, 6 Oct 2022 10:36:28 +0200 (CEST) Received: from shelob.oktetlabs.ru (shelob.oktetlabs.ru [91.220.146.113]) by mails.dpdk.org (Postfix) with ESMTP id 134B442B98 for ; Thu, 6 Oct 2022 10:36:27 +0200 (CEST) Received: from [192.168.38.17] (aros.oktetlabs.ru [192.168.38.17]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by shelob.oktetlabs.ru (Postfix) with ESMTPSA id 72FF186; Thu, 6 Oct 2022 11:36:26 +0300 (MSK) DKIM-Filter: OpenDKIM Filter v2.11.0 shelob.oktetlabs.ru 72FF186 DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple; d=oktetlabs.ru; s=default; t=1665045386; bh=kCEcyeQiOXjc+Gc3pgyuXZfV0gGzB0u0zV4brXDJuj8=; h=Date:Subject:From:To:Cc:References:In-Reply-To:From; b=QNVKe4wezcj5/7JSwOTHTq8gH4ba0AbH5XLvt/yhSYfqwPaoe/AqUwxFYsjjcw+l4 NCtgrKm98tsP+JrASyydfpxpX3EsZJ4fwJriaoGTfZRiMH4o9G1AAoNUEvmIZFa3vc KUvi7wvLd2Wl2YLGtK188nZCMmRoSWk3RgAGFMkA= Message-ID: <9ed695c4-15c3-0e50-05fb-7235915ae70d@oktetlabs.ru> Date: Thu, 6 Oct 2022 11:36:26 +0300 MIME-Version: 1.0 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Thunderbird/102.3.0 Subject: Re: [PATCH v4] ethdev: support congestion management Content-Language: en-US From: Andrew Rybchenko To: Ferruh Yigit , Thomas Monjalon , Ray Kinsella Cc: dev@dpdk.org, Jerin Jacob , Sunil Kumar Kori References: <20221004090258.2472727-1-andrew.rybchenko@oktetlabs.ru> Organization: OKTET Labs In-Reply-To: <20221004090258.2472727-1-andrew.rybchenko@oktetlabs.ru> Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 7bit X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org @Jerin, could you confirm if my minor changes are OK for you and, if so, I'll proceed with applying the patch. On 10/4/22 12:02, Andrew Rybchenko wrote: > From: Jerin Jacob > > NIC HW controllers often come with congestion management support on > various HW objects such as Rx queue depth or mempool queue depth. > > Also, it can support various modes of operation such as RED > (Random early discard), WRED etc on those HW objects. > > Add a framework to express such modes(enum rte_cman_mode) and > introduce (enum rte_eth_cman_obj) to enumerate the different > objects where the modes can operate on. > > Add RTE_CMAN_RED mode of operation and RTE_ETH_CMAN_OBJ_RX_QUEUE, > RTE_ETH_CMAN_OBJ_RX_QUEUE_MEMPOOL objects. > > Introduce reserved fields in configuration structure > backed by rte_eth_cman_config_init() to add new configuration > parameters without ABI breakage. > > Add rte_eth_cman_info_get() API to get the information such as > supported modes and objects. > > Add rte_eth_cman_config_init(), rte_eth_cman_config_set() APIs > to configure congestion management on those object with associated mode. > > Finally, add rte_eth_cman_config_get() API to retrieve the > applied configuration. > > Signed-off-by: Jerin Jacob > Signed-off-by: Sunil Kumar Kori > Signed-off-by: Andrew Rybchenko > --- > v3..v4: Andrew Rybchenko > - rebase > - remove eth_check_err() and use eth_err() instead > - minor fixes in description to avoid "This patch" and "Added". > - correct position in release notes > v2..v3: > - Rename rte_cman.c to rte_ethdev_cman.c > - Move lib/eal/include/rte_cman.h to lib/ethdev/rte_cman.h > - Fix review comments (Andrew Rybchenko) > - Add release notes > > v1..v2: > - Fix review comments (Akhil Goyal) > > rfc..v1: > - Added RED specification (http://www.aciri.org/floyd/papers/red/red.html) link > - Fixed doxygen comment issue (Min Hu) > > doc/guides/nics/features.rst | 12 ++ > doc/guides/nics/features/default.ini | 1 + > doc/guides/rel_notes/release_22_11.rst | 6 + > lib/ethdev/ethdev_driver.h | 25 ++++ > lib/ethdev/ethdev_private.h | 3 + > lib/ethdev/meson.build | 2 + > lib/ethdev/rte_cman.h | 55 +++++++++ > lib/ethdev/rte_ethdev.c | 2 +- > lib/ethdev/rte_ethdev.h | 164 +++++++++++++++++++++++++ > lib/ethdev/rte_ethdev_cman.c | 101 +++++++++++++++ > lib/ethdev/version.map | 4 + > 11 files changed, 374 insertions(+), 1 deletion(-) > create mode 100644 lib/ethdev/rte_cman.h > create mode 100644 lib/ethdev/rte_ethdev_cman.c > > diff --git a/doc/guides/nics/features.rst b/doc/guides/nics/features.rst > index b4a8e9881c..70ca46e651 100644 > --- a/doc/guides/nics/features.rst > +++ b/doc/guides/nics/features.rst > @@ -727,6 +727,18 @@ Supports configuring per-queue stat counter mapping. > ``rte_eth_dev_set_tx_queue_stats_mapping()``. > > > +.. _nic_features_congestion_management: > + > +Congestion management > +--------------------- > + > +Supports congestion management. > + > +* **[implements] eth_dev_ops**: ``cman_info_get``, ``cman_config_set``, ``cman_config_get``. > +* **[related] API**: ``rte_eth_cman_info_get()``, ``rte_eth_cman_config_init()``, > + ``rte_eth_cman_config_set()``, ``rte_eth_cman_config_get()``. > + > + > .. _nic_features_fw_version: > > FW version > diff --git a/doc/guides/nics/features/default.ini b/doc/guides/nics/features/default.ini > index f7192cb0da..a9c0008ebd 100644 > --- a/doc/guides/nics/features/default.ini > +++ b/doc/guides/nics/features/default.ini > @@ -60,6 +60,7 @@ Tx descriptor status = > Basic stats = > Extended stats = > Stats per queue = > +Congestion management = > FW version = > EEPROM dump = > Module EEPROM dump = > diff --git a/doc/guides/rel_notes/release_22_11.rst b/doc/guides/rel_notes/release_22_11.rst > index 44f9a30c6a..0ffa004a9e 100644 > --- a/doc/guides/rel_notes/release_22_11.rst > +++ b/doc/guides/rel_notes/release_22_11.rst > @@ -78,6 +78,12 @@ New Features > Added new rte_flow action which allows application to re-route packets > directly to the kernel without software involvement. > > +* **Added support for congestion management for ethdev.** > + > + Added new APIs ``rte_eth_cman_config_init()``, ``rte_eth_cman_config_get()``, > + ``rte_eth_cman_config_set()``, ``rte_eth_cman_info_get()`` > + to support congestion management. > + > * **Updated Intel iavf driver.** > > * Added flow subscription support. > diff --git a/lib/ethdev/ethdev_driver.h b/lib/ethdev/ethdev_driver.h > index 8cd8eb8685..e1e2d10a35 100644 > --- a/lib/ethdev/ethdev_driver.h > +++ b/lib/ethdev/ethdev_driver.h > @@ -1094,6 +1094,22 @@ typedef int (*eth_rx_queue_avail_thresh_query_t)(struct rte_eth_dev *dev, > uint16_t *rx_queue_id, > uint8_t *avail_thresh); > > +/** @internal Get congestion management information. */ > +typedef int (*eth_cman_info_get_t)(struct rte_eth_dev *dev, > + struct rte_eth_cman_info *info); > + > +/** @internal Init congestion management structure with default values. */ > +typedef int (*eth_cman_config_init_t)(struct rte_eth_dev *dev, > + struct rte_eth_cman_config *config); > + > +/** @internal Configure congestion management on a port. */ > +typedef int (*eth_cman_config_set_t)(struct rte_eth_dev *dev, > + const struct rte_eth_cman_config *config); > + > +/** @internal Retrieve congestion management configuration of a port. */ > +typedef int (*eth_cman_config_get_t)(struct rte_eth_dev *dev, > + struct rte_eth_cman_config *config); > + > /** > * @internal A structure containing the functions exported by an Ethernet driver. > */ > @@ -1309,6 +1325,15 @@ struct eth_dev_ops { > eth_rx_queue_avail_thresh_set_t rx_queue_avail_thresh_set; > /** Query Rx queue available descriptors threshold event */ > eth_rx_queue_avail_thresh_query_t rx_queue_avail_thresh_query; > + > + /** Get congestion management information */ > + eth_cman_info_get_t cman_info_get; > + /** Initialize congestion management structure with default values */ > + eth_cman_config_init_t cman_config_init; > + /** Configure congestion management */ > + eth_cman_config_set_t cman_config_set; > + /** Retrieve congestion management configuration */ > + eth_cman_config_get_t cman_config_get; > }; > > /** > diff --git a/lib/ethdev/ethdev_private.h b/lib/ethdev/ethdev_private.h > index cc9879907c..acb4b335c8 100644 > --- a/lib/ethdev/ethdev_private.h > +++ b/lib/ethdev/ethdev_private.h > @@ -37,6 +37,9 @@ struct rte_eth_dev_callback { > > extern rte_spinlock_t eth_dev_cb_lock; > > +/* Convert all error to -EIO if device is removed. */ > +int eth_err(uint16_t port_id, int ret); > + > /* > * Convert rte_eth_dev pointer to port ID. > * NULL will be translated to RTE_MAX_ETHPORTS. > diff --git a/lib/ethdev/meson.build b/lib/ethdev/meson.build > index 47bb2625b0..39250b5da1 100644 > --- a/lib/ethdev/meson.build > +++ b/lib/ethdev/meson.build > @@ -8,6 +8,7 @@ sources = files( > 'ethdev_trace_points.c', > 'rte_class_eth.c', > 'rte_ethdev.c', > + 'rte_ethdev_cman.c', > 'rte_flow.c', > 'rte_mtr.c', > 'rte_tm.c', > @@ -19,6 +20,7 @@ sources = files( > ) > > headers = files( > + 'rte_cman.h', > 'rte_ethdev.h', > 'rte_ethdev_trace.h', > 'rte_ethdev_trace_fp.h', > diff --git a/lib/ethdev/rte_cman.h b/lib/ethdev/rte_cman.h > new file mode 100644 > index 0000000000..297db8e095 > --- /dev/null > +++ b/lib/ethdev/rte_cman.h > @@ -0,0 +1,55 @@ > +/* SPDX-License-Identifier: BSD-3-Clause > + * Copyright(C) 2022 Marvell International Ltd. > + */ > + > +#ifndef RTE_CMAN_H > +#define RTE_CMAN_H > + > +#ifdef __cplusplus > +extern "C" { > +#endif > + > +#include > + > +/** > + * @file > + * Congestion management related parameters for DPDK. > + */ > + > +/** Congestion management modes */ > +enum rte_cman_mode { > + /** > + * Congestion based on Random Early Detection. > + * > + * https://en.wikipedia.org/wiki/Random_early_detection > + * http://www.aciri.org/floyd/papers/red/red.html > + * @see struct rte_cman_red_params > + */ > + RTE_CMAN_RED = RTE_BIT32(0), > +}; > + > +/** > + * RED based congestion management configuration parameters. > + */ > +struct rte_cman_red_params { > + /** > + * Minimum threshold (min_th) value > + * > + * Value expressed as percentage. Value must be in 0 to 100(inclusive). > + */ > + uint8_t min_th; > + /** > + * Maximum threshold (max_th) value > + * > + * Value expressed as percentage. Value must be in 0 to 100(inclusive). > + */ > + uint8_t max_th; > + /** Inverse of packet marking probability maximum value (maxp = 1 / maxp_inv) */ > + uint16_t maxp_inv; > +}; > + > +#ifdef __cplusplus > +} > +#endif > + > +#endif /* RTE_CMAN_H */ > diff --git a/lib/ethdev/rte_ethdev.c b/lib/ethdev/rte_ethdev.c > index 0c2c1088c0..e4eb17221b 100644 > --- a/lib/ethdev/rte_ethdev.c > +++ b/lib/ethdev/rte_ethdev.c > @@ -642,7 +642,7 @@ rte_eth_dev_get_port_by_name(const char *name, uint16_t *port_id) > return -ENODEV; > } > > -static int > +int > eth_err(uint16_t port_id, int ret) > { > if (ret == 0) > diff --git a/lib/ethdev/rte_ethdev.h b/lib/ethdev/rte_ethdev.h > index a21f58b9cd..8df1cdfad0 100644 > --- a/lib/ethdev/rte_ethdev.h > +++ b/lib/ethdev/rte_ethdev.h > @@ -160,6 +160,7 @@ extern "C" { > #define RTE_ETHDEV_DEBUG_TX > #endif > > +#include > #include > #include > #include > @@ -5314,6 +5315,169 @@ typedef struct { > __rte_experimental > int rte_eth_dev_priv_dump(uint16_t port_id, FILE *file); > > +/* Congestion management */ > + > +/** Enumerate list of ethdev congestion management objects */ > +enum rte_eth_cman_obj { > + /** Congestion management based on Rx queue depth */ > + RTE_ETH_CMAN_OBJ_RX_QUEUE = RTE_BIT32(0), > + /** > + * Congestion management based on mempool depth associated with Rx queue > + * @see rte_eth_rx_queue_setup() > + */ > + RTE_ETH_CMAN_OBJ_RX_QUEUE_MEMPOOL = RTE_BIT32(1), > +}; > + > +/** > + * @warning > + * @b EXPERIMENTAL: this structure may change, or be removed, without prior notice > + * > + * A structure used to retrieve information of ethdev congestion management. > + */ > +struct rte_eth_cman_info { > + /** > + * Set of supported congestion management modes > + * @see enum rte_cman_mode > + */ > + uint64_t modes_supported; > + /** > + * Set of supported congestion management objects > + * @see enum rte_eth_cman_obj > + */ > + uint64_t objs_supported; > + /** > + * Reserved for future fields. Always returned as 0 when > + * rte_eth_cman_info_get() is invoked > + */ > + uint8_t rsvd[8]; > +}; > + > +/** > + * @warning > + * @b EXPERIMENTAL: this structure may change, or be removed, without prior notice > + * > + * A structure used to configure the ethdev congestion management. > + */ > +struct rte_eth_cman_config { > + /** Congestion management object */ > + enum rte_eth_cman_obj obj; > + /** Congestion management mode */ > + enum rte_cman_mode mode; > + union { > + /** > + * Rx queue to configure congestion management. > + * > + * Valid when object is RTE_ETH_CMAN_OBJ_RX_QUEUE or > + * RTE_ETH_CMAN_OBJ_RX_QUEUE_MEMPOOL. > + */ > + uint16_t rx_queue; > + /** > + * Reserved for future fields. > + * It must be set to 0 when rte_eth_cman_config_set() is invoked > + * and will be returned as 0 when rte_eth_cman_config_get() is > + * invoked. > + */ > + uint8_t rsvd_obj_params[4]; > + } obj_param; > + union { > + /** > + * RED configuration parameters. > + * > + * Valid when mode is RTE_CMAN_RED. > + */ > + struct rte_cman_red_params red; > + /** > + * Reserved for future fields. > + * It must be set to 0 when rte_eth_cman_config_set() is invoked > + * and will be returned as 0 when rte_eth_cman_config_get() is > + * invoked. > + */ > + uint8_t rsvd_mode_params[4]; > + } mode_param; > +}; > + > +/** > + * @warning > + * @b EXPERIMENTAL: this API may change, or be removed, without prior notice > + * > + * Retrieve the information for ethdev congestion management > + * > + * @param port_id > + * The port identifier of the Ethernet device. > + * @param info > + * A pointer to a structure of type *rte_eth_cman_info* to be filled with > + * the information about congestion management. > + * @return > + * - (0) if successful. > + * - (-ENOTSUP) if support for cman_info_get does not exist. > + * - (-ENODEV) if *port_id* invalid. > + * - (-EINVAL) if bad parameter. > + */ > +__rte_experimental > +int rte_eth_cman_info_get(uint16_t port_id, struct rte_eth_cman_info *info); > + > +/** > + * @warning > + * @b EXPERIMENTAL: this API may change, or be removed, without prior notice > + * > + * Initialize the ethdev congestion management configuration structure with default values. > + * > + * @param port_id > + * The port identifier of the Ethernet device. > + * @param config > + * A pointer to a structure of type *rte_eth_cman_config* to be initialized > + * with default value. > + * @return > + * - (0) if successful. > + * - (-ENOTSUP) if support for cman_config_init does not exist. > + * - (-ENODEV) if *port_id* invalid. > + * - (-EINVAL) if bad parameter. > + */ > +__rte_experimental > +int rte_eth_cman_config_init(uint16_t port_id, struct rte_eth_cman_config *config); > + > +/** > + * @warning > + * @b EXPERIMENTAL: this API may change, or be removed, without prior notice > + * > + * Configure ethdev congestion management > + * > + * @param port_id > + * The port identifier of the Ethernet device. > + * @param config > + * A pointer to a structure of type *rte_eth_cman_config* to be configured. > + * @return > + * - (0) if successful. > + * - (-ENOTSUP) if support for cman_config_set does not exist. > + * - (-ENODEV) if *port_id* invalid. > + * - (-EINVAL) if bad parameter. > + */ > +__rte_experimental > +int rte_eth_cman_config_set(uint16_t port_id, const struct rte_eth_cman_config *config); > + > +/** > + * @warning > + * @b EXPERIMENTAL: this API may change, or be removed, without prior notice > + * > + * Retrieve the applied ethdev congestion management parameters for the given port. > + * > + * @param port_id > + * The port identifier of the Ethernet device. > + * @param config > + * A pointer to a structure of type *rte_eth_cman_config* to retrieve > + * congestion management parameters for the given object. > + * Application must fill all parameters except mode_param parameter in > + * struct rte_eth_cman_config. > + * > + * @return > + * - (0) if successful. > + * - (-ENOTSUP) if support for cman_config_get does not exist. > + * - (-ENODEV) if *port_id* invalid. > + * - (-EINVAL) if bad parameter. > + */ > +__rte_experimental > +int rte_eth_cman_config_get(uint16_t port_id, struct rte_eth_cman_config *config); > + > #include > > /** > diff --git a/lib/ethdev/rte_ethdev_cman.c b/lib/ethdev/rte_ethdev_cman.c > new file mode 100644 > index 0000000000..4a1bdd7bd0 > --- /dev/null > +++ b/lib/ethdev/rte_ethdev_cman.c > @@ -0,0 +1,101 @@ > +/* SPDX-License-Identifier: BSD-3-Clause > + * Copyright(C) 2022 Marvell International Ltd. > + */ > + > +#include > + > +#include > +#include "rte_ethdev.h" > +#include "ethdev_driver.h" > +#include "ethdev_private.h" > + > +/* Get congestion management information for a port */ > +int > +rte_eth_cman_info_get(uint16_t port_id, struct rte_eth_cman_info *info) > +{ > + struct rte_eth_dev *dev; > + > + RTE_ETH_VALID_PORTID_OR_ERR_RET(port_id, -ENODEV); > + dev = &rte_eth_devices[port_id]; > + > + if (info == NULL) { > + RTE_ETHDEV_LOG(ERR, "congestion management info is NULL\n"); > + return -EINVAL; > + } > + > + if (dev->dev_ops->cman_info_get == NULL) { > + RTE_ETHDEV_LOG(ERR, "Function not implemented\n"); > + return -ENOTSUP; > + } > + > + memset(info, 0, sizeof(struct rte_eth_cman_info)); > + return eth_err(port_id, (*dev->dev_ops->cman_info_get)(dev, info)); > +} > + > +/* Initialize congestion management structure with default values */ > +int > +rte_eth_cman_config_init(uint16_t port_id, struct rte_eth_cman_config *config) > +{ > + struct rte_eth_dev *dev; > + > + RTE_ETH_VALID_PORTID_OR_ERR_RET(port_id, -ENODEV); > + dev = &rte_eth_devices[port_id]; > + > + if (config == NULL) { > + RTE_ETHDEV_LOG(ERR, "congestion management config is NULL\n"); > + return -EINVAL; > + } > + > + if (dev->dev_ops->cman_config_init == NULL) { > + RTE_ETHDEV_LOG(ERR, "Function not implemented\n"); > + return -ENOTSUP; > + } > + > + memset(config, 0, sizeof(struct rte_eth_cman_config)); > + return eth_err(port_id, (*dev->dev_ops->cman_config_init)(dev, config)); > +} > + > +/* Configure congestion management on a port */ > +int > +rte_eth_cman_config_set(uint16_t port_id, const struct rte_eth_cman_config *config) > +{ > + struct rte_eth_dev *dev; > + > + RTE_ETH_VALID_PORTID_OR_ERR_RET(port_id, -ENODEV); > + dev = &rte_eth_devices[port_id]; > + > + if (config == NULL) { > + RTE_ETHDEV_LOG(ERR, "congestion management config is NULL\n"); > + return -EINVAL; > + } > + > + if (dev->dev_ops->cman_config_set == NULL) { > + RTE_ETHDEV_LOG(ERR, "Function not implemented\n"); > + return -ENOTSUP; > + } > + > + return eth_err(port_id, (*dev->dev_ops->cman_config_set)(dev, config)); > +} > + > +/* Retrieve congestion management configuration of a port */ > +int > +rte_eth_cman_config_get(uint16_t port_id, struct rte_eth_cman_config *config) > +{ > + struct rte_eth_dev *dev; > + > + RTE_ETH_VALID_PORTID_OR_ERR_RET(port_id, -ENODEV); > + dev = &rte_eth_devices[port_id]; > + > + if (config == NULL) { > + RTE_ETHDEV_LOG(ERR, "congestion management config is NULL\n"); > + return -EINVAL; > + } > + > + if (dev->dev_ops->cman_config_get == NULL) { > + RTE_ETHDEV_LOG(ERR, "Function not implemented\n"); > + return -ENOTSUP; > + } > + > + memset(config, 0, sizeof(struct rte_eth_cman_config)); > + return eth_err(port_id, (*dev->dev_ops->cman_config_get)(dev, config)); > +} > diff --git a/lib/ethdev/version.map b/lib/ethdev/version.map > index 3651ceb234..857d21be1c 100644 > --- a/lib/ethdev/version.map > +++ b/lib/ethdev/version.map > @@ -287,6 +287,10 @@ EXPERIMENTAL { > rte_mtr_meter_vlan_table_update; > > # added in 22.11 > + rte_eth_cman_config_get; > + rte_eth_cman_config_init; > + rte_eth_cman_config_set; > + rte_eth_cman_info_get; > rte_flow_async_action_handle_query; > rte_mtr_meter_policy_get; > rte_mtr_meter_profile_get;