* [PATCH 0/8] Enhance the bond framework to support offload
@ 2023-10-05 2:40 Chaoyong He
2023-10-05 2:40 ` [PATCH 1/8] ethdev: add member notification for bonding port Chaoyong He
` (8 more replies)
0 siblings, 9 replies; 80+ messages in thread
From: Chaoyong He @ 2023-10-05 2:40 UTC (permalink / raw)
To: dev; +Cc: oss-drivers, Chaoyong He
This patch series try to enhance the bond framework to support the
offload feature better:
* Add new API to make the member port can access some information of the
bond port which belongs.
* Add new API to get the result of whether bond port is created by the
member port.
* Add two command line argument to control if enable member port
notification and dedicated queue features.
* Add logic to support add ports which share the same PCI address into
bond port.
* Also modify the testpmd application to test the new APIs and logics
added by this patch series.
Long Wu (8):
ethdev: add member notification for bonding port
ethdev: add API to get hardware creation of bonding port
net/bonding: modify interface comment format
net/bonding: add bonding port arguments
net/bonding: support add port by data name
net/bonding: create new rte flow header file
net/bonding: support checking valid bonding port ID
net/bonding: add commands for bonding port notification
.../link_bonding_poll_mode_drv_lib.rst | 19 ++
drivers/net/bonding/bonding_testpmd.c | 128 +++++++++
drivers/net/bonding/eth_bond_8023ad_private.h | 52 ++--
drivers/net/bonding/eth_bond_private.h | 25 +-
drivers/net/bonding/rte_eth_bond.h | 252 ++++++++++++++----
drivers/net/bonding/rte_eth_bond_8023ad.h | 76 ++++--
drivers/net/bonding/rte_eth_bond_alb.h | 34 ++-
drivers/net/bonding/rte_eth_bond_api.c | 123 +++++++++
drivers/net/bonding/rte_eth_bond_args.c | 47 ++++
drivers/net/bonding/rte_eth_bond_flow.c | 1 +
drivers/net/bonding/rte_eth_bond_flow.h | 22 ++
drivers/net/bonding/rte_eth_bond_pmd.c | 89 ++++++-
drivers/net/bonding/version.map | 5 +
lib/ethdev/ethdev_driver.h | 38 +++
14 files changed, 781 insertions(+), 130 deletions(-)
create mode 100644 drivers/net/bonding/rte_eth_bond_flow.h
--
2.39.1
^ permalink raw reply [flat|nested] 80+ messages in thread* [PATCH 1/8] ethdev: add member notification for bonding port 2023-10-05 2:40 [PATCH 0/8] Enhance the bond framework to support offload Chaoyong He @ 2023-10-05 2:40 ` Chaoyong He 2023-10-05 2:40 ` [PATCH 2/8] ethdev: add API to get hardware creation of " Chaoyong He ` (7 subsequent siblings) 8 siblings, 0 replies; 80+ messages in thread From: Chaoyong He @ 2023-10-05 2:40 UTC (permalink / raw) To: dev; +Cc: oss-drivers, Long Wu, James Hershaw, Chaoyong He From: Long Wu <long.wu@corigine.com> Bonding PMD does not let member ports know the bonding port's information, like how many member ports the bonding port has, what mode the bonding port is in and so on. Add the notification interface for bonding port to let member port know it is added to a bonding port and what the bonding port's configuration is. If so the member ports have chance to achieve its bond-flow-offlod or other private bonding functions. Signed-off-by: Long Wu <long.wu@corigine.com> Reviewed-by: James Hershaw <james.hershaw@corigine.com> Reviewed-by: Chaoyong He <chaoyong.he@corigine.com> --- drivers/net/bonding/eth_bond_private.h | 2 + drivers/net/bonding/rte_eth_bond.h | 46 ++++++++++++++++ drivers/net/bonding/rte_eth_bond_api.c | 73 ++++++++++++++++++++++++++ drivers/net/bonding/rte_eth_bond_pmd.c | 27 ++++++++-- drivers/net/bonding/version.map | 3 ++ lib/ethdev/ethdev_driver.h | 18 +++++++ 6 files changed, 166 insertions(+), 3 deletions(-) diff --git a/drivers/net/bonding/eth_bond_private.h b/drivers/net/bonding/eth_bond_private.h index e688894210..1344f8c002 100644 --- a/drivers/net/bonding/eth_bond_private.h +++ b/drivers/net/bonding/eth_bond_private.h @@ -186,6 +186,8 @@ struct bond_dev_private { void *vlan_filter_bmpmem; /* enabled vlan filter bitmap */ struct rte_bitmap *vlan_filter_bmp; + + bool notify_member; /**< Enable member notification of bonding port. */ }; extern const struct eth_dev_ops default_dev_ops; diff --git a/drivers/net/bonding/rte_eth_bond.h b/drivers/net/bonding/rte_eth_bond.h index f10165f2c6..737beca446 100644 --- a/drivers/net/bonding/rte_eth_bond.h +++ b/drivers/net/bonding/rte_eth_bond.h @@ -351,6 +351,52 @@ rte_eth_bond_link_up_prop_delay_set(uint16_t bonding_port_id, int rte_eth_bond_link_up_prop_delay_get(uint16_t bonding_port_id); +/** + * Set the flag of whether bonding port notifies member ports. + * + * @param bonding_port_id + * Port ID of bonding device. + * @param notify_member + * Flag of whether bonding port notifies member ports. + * + * @return + * 0 on success, negative value otherwise. + */ +__rte_experimental +int +rte_eth_bond_notify_member_flag_set(uint16_t bonding_port_id, bool notify_member); + +/** + * Get the flag of whether bonding port notifies member ports. + * + * @param bonding_port_id + * Port ID of bonding device. + * @param notify_member + * Flag of whether bonding port notifies member ports. + * + * @return + * 0 on success, negative value otherwise. + */ +__rte_experimental +int +rte_eth_bond_notify_member_flag_get(uint16_t bonding_port_id, bool *notify_member); + +/** + * Notify the member ports of bonding port's information. + * + * This interface is called in the following functions: + * - bond_ethdev_lsc_event_callback() + * - bond_ethdev_configure() + * + * @param bonding_port_id + * Port ID of bonding device. + * + * @return + * 0 on success, negative value otherwise. + */ +__rte_experimental +int +rte_eth_bond_notify_members(uint16_t bonding_port_id); #ifdef __cplusplus } diff --git a/drivers/net/bonding/rte_eth_bond_api.c b/drivers/net/bonding/rte_eth_bond_api.c index 99e496556a..48c1d050fd 100644 --- a/drivers/net/bonding/rte_eth_bond_api.c +++ b/drivers/net/bonding/rte_eth_bond_api.c @@ -627,6 +627,17 @@ __eth_bond_member_add_lock_free(uint16_t bonding_port_id, uint16_t member_port_i member_vlan_filter_set(bonding_port_id, member_port_id); + if (internals->notify_member && + *member_eth_dev->dev_ops->bond_notify_member != NULL) { + ret = member_eth_dev->dev_ops->bond_notify_member(member_eth_dev, + bonding_eth_dev); + if (ret < 0) { + RTE_BOND_LOG(ERR, "Add member (port %u) notify failed!", + member_port_id); + return -1; + } + } + return 0; } @@ -733,6 +744,10 @@ __eth_bond_member_remove_lock_free(uint16_t bonding_port_id, member_eth_dev = &rte_eth_devices[member_port_id]; member_remove(internals, member_eth_dev); member_eth_dev->data->dev_flags &= (~RTE_ETH_DEV_BONDING_MEMBER); + if (internals->notify_member && + *member_eth_dev->dev_ops->bond_notify_member != NULL) + member_eth_dev->dev_ops->bond_notify_member(member_eth_dev, + bonding_eth_dev); /* first member in the active list will be the primary by default, * otherwise use first device in list */ @@ -1098,3 +1113,61 @@ rte_eth_bond_link_up_prop_delay_get(uint16_t bonding_port_id) return internals->link_up_delay_ms; } + +int +rte_eth_bond_notify_member_flag_set(uint16_t bonding_port_id, bool notify_member) +{ + struct bond_dev_private *internals; + + if (valid_bonding_port_id(bonding_port_id) != 0) + return -EINVAL; + + internals = rte_eth_devices[bonding_port_id].data->dev_private; + + internals->notify_member = notify_member; + + return 0; +} + +int +rte_eth_bond_notify_member_flag_get(uint16_t bonding_port_id, bool *notify_member) +{ + struct bond_dev_private *internals; + + if (valid_bonding_port_id(bonding_port_id) != 0) + return -EINVAL; + + internals = rte_eth_devices[bonding_port_id].data->dev_private; + + *notify_member = internals->notify_member; + + return 0; +} + +int +rte_eth_bond_notify_members(uint16_t bonding_port_id) +{ + uint32_t i; + uint16_t member_port_id; + struct rte_eth_dev *bond_dev; + struct bond_dev_private *internals; + struct rte_eth_dev *member_dev[RTE_MAX_ETHPORTS]; + + if (valid_bonding_port_id(bonding_port_id) != 0) + return -EINVAL; + + bond_dev = &rte_eth_devices[bonding_port_id]; + internals = bond_dev->data->dev_private; + + for (i = 0; i < internals->member_count; i++) { + member_port_id = internals->members[i].port_id; + member_dev[i] = &rte_eth_devices[member_port_id]; + if (*member_dev[i]->dev_ops->bond_notify_member == NULL) + return -ENOTSUP; + } + + for (i = 0; i < internals->member_count; i++) + member_dev[i]->dev_ops->bond_notify_member(member_dev[i], bond_dev); + + return 0; +} diff --git a/drivers/net/bonding/rte_eth_bond_pmd.c b/drivers/net/bonding/rte_eth_bond_pmd.c index 122b1187fd..b99b8b8938 100644 --- a/drivers/net/bonding/rte_eth_bond_pmd.c +++ b/drivers/net/bonding/rte_eth_bond_pmd.c @@ -2968,11 +2968,13 @@ bond_ethdev_lsc_event_callback(uint16_t port_id, enum rte_eth_event_type type, int valid_member = 0; uint16_t active_pos, member_idx; uint16_t i; + uint16_t bonding_port_id; if (type != RTE_ETH_EVENT_INTR_LSC || param == NULL) return rc; - bonding_eth_dev = &rte_eth_devices[*(uint16_t *)param]; + bonding_port_id = *(uint16_t *)param; + bonding_eth_dev = &rte_eth_devices[bonding_port_id]; if (check_for_bonding_ethdev(bonding_eth_dev)) return rc; @@ -3044,8 +3046,10 @@ bond_ethdev_lsc_event_callback(uint16_t port_id, enum rte_eth_event_type type, * using it. */ if (internals->user_defined_primary_port && - internals->primary_port == port_id) + internals->primary_port == port_id) { bond_ethdev_primary_set(internals, port_id); + rte_eth_bond_notify_members(bonding_port_id); + } } else { if (active_pos == internals->active_member_count) goto link_update; @@ -3064,6 +3068,7 @@ bond_ethdev_lsc_event_callback(uint16_t port_id, enum rte_eth_event_type type, internals->active_members[0]); else internals->current_primary_port = internals->primary_port; + rte_eth_bond_notify_members(bonding_port_id); mac_address_members_update(bonding_eth_dev); bond_ethdev_promiscuous_update(bonding_eth_dev); bond_ethdev_allmulticast_update(bonding_eth_dev); @@ -3362,6 +3367,7 @@ dump_basic(const struct rte_eth_dev *dev, FILE *f) struct bond_dev_private instant_priv; const struct bond_dev_private *internals = &instant_priv; int mode, i; + bool notify_member; /* Obtain a instance of dev_private to prevent data from being modified. */ memcpy(&instant_priv, dev->data->dev_private, sizeof(struct bond_dev_private)); @@ -3431,6 +3437,13 @@ dump_basic(const struct rte_eth_dev *dev, FILE *f) fprintf(f, "\tUser Defined Primary: [%u]\n", internals->primary_port); if (internals->member_count > 0) fprintf(f, "\tCurrent Primary: [%u]\n", internals->current_primary_port); + + if (rte_eth_bond_notify_member_flag_get(internals->port_id, ¬ify_member) == 0) + fprintf(f, "\tNotify Member Ports Flag: %s\n", + notify_member ? "enable" : "disable"); + else + fprintf(f, "\tFailed to get notify member ports flag for bonding port %d\n", + internals->port_id); } static void @@ -3983,8 +3996,12 @@ bond_ethdev_configure(struct rte_eth_dev *dev) * if no kvlist, it means that this bonding device has been created * through the bonding api. */ - if (!kvlist || internals->kvargs_processing_is_done) + if (!kvlist || internals->kvargs_processing_is_done) { + if (internals->notify_member && rte_eth_bond_notify_members(port_id) != 0) + RTE_BOND_LOG(ERR, "Notify member ports failed"); + return 0; + } internals->kvargs_processing_is_done = true; @@ -4222,6 +4239,10 @@ bond_ethdev_configure(struct rte_eth_dev *dev) return -1; } } + + if (internals->notify_member && rte_eth_bond_notify_members(port_id) != 0) + RTE_BOND_LOG(ERR, "Notify member ports failed"); + return 0; } diff --git a/drivers/net/bonding/version.map b/drivers/net/bonding/version.map index 09ee21c55f..3bd5e8ad11 100644 --- a/drivers/net/bonding/version.map +++ b/drivers/net/bonding/version.map @@ -35,4 +35,7 @@ EXPERIMENTAL { rte_eth_bond_member_add; rte_eth_bond_member_remove; rte_eth_bond_members_get; + rte_eth_bond_notify_member_flag_get; + rte_eth_bond_notify_member_flag_set; + rte_eth_bond_notify_members; }; diff --git a/lib/ethdev/ethdev_driver.h b/lib/ethdev/ethdev_driver.h index 980f837ab6..5e73579f22 100644 --- a/lib/ethdev/ethdev_driver.h +++ b/lib/ethdev/ethdev_driver.h @@ -1206,6 +1206,21 @@ typedef int (*eth_count_aggr_ports_t)(struct rte_eth_dev *dev); typedef int (*eth_map_aggr_tx_affinity_t)(struct rte_eth_dev *dev, uint16_t tx_queue_id, uint8_t affinity); +/** + * @internal + * Bonding port notifies the member ports. + * + * @param dev + * Member port (ethdev) handle. + * @param bonding_dev + * Bonding port (ethdev) handle. + * + * @return + * Negative on error, 0 on success. + */ +typedef int (*eth_bond_notify_member)(struct rte_eth_dev *dev, + struct rte_eth_dev *bonding_dev); + /** * @internal A structure containing the functions exported by an Ethernet driver. */ @@ -1443,6 +1458,9 @@ struct eth_dev_ops { eth_count_aggr_ports_t count_aggr_ports; /** Map a Tx queue with an aggregated port of the DPDK port */ eth_map_aggr_tx_affinity_t map_aggr_tx_affinity; + + /** Notify the member port of bonding port information */ + eth_bond_notify_member bond_notify_member; }; /** -- 2.39.1 ^ permalink raw reply [flat|nested] 80+ messages in thread
* [PATCH 2/8] ethdev: add API to get hardware creation of bonding port 2023-10-05 2:40 [PATCH 0/8] Enhance the bond framework to support offload Chaoyong He 2023-10-05 2:40 ` [PATCH 1/8] ethdev: add member notification for bonding port Chaoyong He @ 2023-10-05 2:40 ` Chaoyong He 2023-10-05 2:40 ` [PATCH 3/8] net/bonding: modify interface comment format Chaoyong He ` (6 subsequent siblings) 8 siblings, 0 replies; 80+ messages in thread From: Chaoyong He @ 2023-10-05 2:40 UTC (permalink / raw) To: dev; +Cc: oss-drivers, Long Wu, James Hershaw, Chaoyong He From: Long Wu <long.wu@corigine.com> After bonding port notification, member port hardware may create the bonding port. We want to get the result of creatition, so we add this API to do the getting action. Signed-off-by: Long Wu <long.wu@corigine.com> Reviewed-by: James Hershaw <james.hershaw@corigine.com> Reviewed-by: Chaoyong He <chaoyong.he@corigine.com> --- drivers/net/bonding/rte_eth_bond.h | 15 ++++++++++++++ drivers/net/bonding/rte_eth_bond_api.c | 28 ++++++++++++++++++++++++++ drivers/net/bonding/version.map | 1 + lib/ethdev/ethdev_driver.h | 20 ++++++++++++++++++ 4 files changed, 64 insertions(+) diff --git a/drivers/net/bonding/rte_eth_bond.h b/drivers/net/bonding/rte_eth_bond.h index 737beca446..6be5e46deb 100644 --- a/drivers/net/bonding/rte_eth_bond.h +++ b/drivers/net/bonding/rte_eth_bond.h @@ -398,6 +398,21 @@ __rte_experimental int rte_eth_bond_notify_members(uint16_t bonding_port_id); +/** + * Get the status of specified bonding port created by member port hardware. + * + * @param bonding_port_id + * Port ID of bonding device. + * @param member_port_id + * Port ID of member device. + * + * @return + * 0 on success, negative value otherwise. + */ +__rte_experimental +int +rte_eth_bond_hw_create_get(uint16_t bonding_port_id, uint16_t member_port_id); + #ifdef __cplusplus } #endif diff --git a/drivers/net/bonding/rte_eth_bond_api.c b/drivers/net/bonding/rte_eth_bond_api.c index 48c1d050fd..0be580b19b 100644 --- a/drivers/net/bonding/rte_eth_bond_api.c +++ b/drivers/net/bonding/rte_eth_bond_api.c @@ -1171,3 +1171,31 @@ rte_eth_bond_notify_members(uint16_t bonding_port_id) return 0; } + +int +rte_eth_bond_hw_create_get(uint16_t bonding_port_id, uint16_t member_port_id) +{ + uint32_t i; + struct rte_eth_dev *bonding_dev; + struct rte_eth_dev *member_dev; + struct bond_dev_private *internals; + + if (valid_bonding_port_id(bonding_port_id) != 0) + return -EINVAL; + + bonding_dev = &rte_eth_devices[bonding_port_id]; + internals = bonding_dev->data->dev_private; + for (i = 0; i < internals->member_count; i++) { + if (internals->members[i].port_id == member_port_id) + break; + } + + if (i == internals->member_count) + return -EINVAL; + + member_dev = &rte_eth_devices[member_port_id]; + if (*member_dev->dev_ops->bond_hw_create_get == NULL) + return -ENOTSUP; + + return member_dev->dev_ops->bond_hw_create_get(member_dev, bonding_dev); +} diff --git a/drivers/net/bonding/version.map b/drivers/net/bonding/version.map index 3bd5e8ad11..3cfff51269 100644 --- a/drivers/net/bonding/version.map +++ b/drivers/net/bonding/version.map @@ -32,6 +32,7 @@ EXPERIMENTAL { global: rte_eth_bond_8023ad_member_info; rte_eth_bond_active_members_get; + rte_eth_bond_hw_create_get; rte_eth_bond_member_add; rte_eth_bond_member_remove; rte_eth_bond_members_get; diff --git a/lib/ethdev/ethdev_driver.h b/lib/ethdev/ethdev_driver.h index 5e73579f22..8bb7210fa9 100644 --- a/lib/ethdev/ethdev_driver.h +++ b/lib/ethdev/ethdev_driver.h @@ -1221,6 +1221,21 @@ typedef int (*eth_map_aggr_tx_affinity_t)(struct rte_eth_dev *dev, uint16_t tx_q typedef int (*eth_bond_notify_member)(struct rte_eth_dev *dev, struct rte_eth_dev *bonding_dev); +/** + * @internal + * Get the status of specified bonding port created by member port hardware. + * + * @param dev + * Member port (ethdev) handle. + * @param bonding_dev + * Bonding port (ethdev) handle. + * + * @return + * Negative on error, 0 on success. + */ +typedef int (*eth_bond_hw_create_get)(struct rte_eth_dev *dev, + struct rte_eth_dev *bonding_dev); + /** * @internal A structure containing the functions exported by an Ethernet driver. */ @@ -1461,6 +1476,11 @@ struct eth_dev_ops { /** Notify the member port of bonding port information */ eth_bond_notify_member bond_notify_member; + /** + * Get the status of whether bonding port is successfully created by + * the member port hardware. + */ + eth_bond_hw_create_get bond_hw_create_get; }; /** -- 2.39.1 ^ permalink raw reply [flat|nested] 80+ messages in thread
* [PATCH 3/8] net/bonding: modify interface comment format 2023-10-05 2:40 [PATCH 0/8] Enhance the bond framework to support offload Chaoyong He 2023-10-05 2:40 ` [PATCH 1/8] ethdev: add member notification for bonding port Chaoyong He 2023-10-05 2:40 ` [PATCH 2/8] ethdev: add API to get hardware creation of " Chaoyong He @ 2023-10-05 2:40 ` Chaoyong He 2023-10-05 2:40 ` [PATCH 4/8] net/bonding: add bonding port arguments Chaoyong He ` (5 subsequent siblings) 8 siblings, 0 replies; 80+ messages in thread From: Chaoyong He @ 2023-10-05 2:40 UTC (permalink / raw) To: dev; +Cc: oss-drivers, Long Wu, James Hershaw, Chaoyong He From: Long Wu <long.wu@corigine.com> Most of the previous interface comment format does not meet the current standards and were not uniform. Modify them to meet current standards. Signed-off-by: Long Wu <long.wu@corigine.com> Reviewed-by: James Hershaw <james.hershaw@corigine.com> Reviewed-by: Chaoyong He <chaoyong.he@corigine.com> --- drivers/net/bonding/eth_bond_8023ad_private.h | 52 +++--- drivers/net/bonding/rte_eth_bond.h | 150 +++++++++++------- drivers/net/bonding/rte_eth_bond_8023ad.h | 76 ++++++--- drivers/net/bonding/rte_eth_bond_alb.h | 34 ++-- 4 files changed, 199 insertions(+), 113 deletions(-) diff --git a/drivers/net/bonding/eth_bond_8023ad_private.h b/drivers/net/bonding/eth_bond_8023ad_private.h index ab7d15f81a..29ee8700c3 100644 --- a/drivers/net/bonding/eth_bond_8023ad_private.h +++ b/drivers/net/bonding/eth_bond_8023ad_private.h @@ -196,10 +196,13 @@ struct bond_dev_private; * * Set mode 4 configuration of bonding interface. * - * @pre Bonding interface must be stopped. + * @pre + * Bonding interface must be stopped. * - * @param dev Bonding interface - * @param conf new configuration. If NULL set default configuration. + * @param dev + * Bonding interface + * @param conf + * new configuration. If NULL set default configuration. */ void bond_mode_8023ad_setup(struct rte_eth_dev *dev, @@ -210,9 +213,11 @@ bond_mode_8023ad_setup(struct rte_eth_dev *dev, * * Enables 802.1AX mode and all active members on bonding interface. * - * @param dev Bonding interface + * @param dev + * Bonding interface + * * @return - * 0 on success, negative value otherwise. + * 0 on success, negative value otherwise. */ int bond_mode_8023ad_enable(struct rte_eth_dev *dev); @@ -222,7 +227,8 @@ bond_mode_8023ad_enable(struct rte_eth_dev *dev); * * Disables 802.1AX mode of the bonding interface and members. * - * @param dev Bonding interface + * @param dev + * Bonding interface * @return * 0 on success, negative value otherwise. */ @@ -232,7 +238,8 @@ int bond_mode_8023ad_disable(struct rte_eth_dev *dev); * @internal * * Starts 802.3AX state machines management logic. - * @param dev Bonding interface + * @param dev + * Bonding interface * @return * 0 if machines was started, 1 if machines was already running, * negative value otherwise. @@ -244,7 +251,8 @@ bond_mode_8023ad_start(struct rte_eth_dev *dev); * @internal * * Stops 802.3AX state machines management logic. - * @param dev Bonding interface + * @param dev + * Bonding interface * @return * 0 if this call stopped state machines, -ENOENT if alarm was not set. */ @@ -255,9 +263,12 @@ bond_mode_8023ad_stop(struct rte_eth_dev *dev); * @internal * * Passes given slow packet to state machines management logic. - * @param internals Bonding device private data. - * @param member_id Member port id. - * @param slot_pkt Slow packet. + * @param internals + * Bonding device private data. + * @param member_id + * Member port id. + * @param slot_pkt + * Slow packet. */ void bond_mode_8023ad_handle_slow_pkt(struct bond_dev_private *internals, @@ -268,11 +279,13 @@ bond_mode_8023ad_handle_slow_pkt(struct bond_dev_private *internals, * * Appends given member used member * - * @param dev Bonding interface. - * @param port_id Member port ID to be added + * @param dev + * Bonding interface. + * @param port_id + * Member port ID to be added * * @return - * 0 on success, negative value otherwise. + * 0 on success, negative value otherwise. */ void bond_mode_8023ad_activate_member(struct rte_eth_dev *dev, uint16_t port_id); @@ -282,18 +295,21 @@ bond_mode_8023ad_activate_member(struct rte_eth_dev *dev, uint16_t port_id); * * Denitializes and removes given member from 802.1AX mode. * - * @param dev Bonding interface. - * @param member_num Position of member in active_members array + * @param dev + * Bonding interface. + * @param member_num + * Position of member in active_members array * * @return - * 0 on success, negative value otherwise. + * 0 on success, negative value otherwise. */ int bond_mode_8023ad_deactivate_member(struct rte_eth_dev *dev, uint16_t member_pos); /** * Updates state when MAC was changed on bonding device or one of its members. - * @param bond_dev Bonding device + * @param bond_dev + * Bonding device */ void bond_mode_8023ad_mac_address_update(struct rte_eth_dev *bond_dev); diff --git a/drivers/net/bonding/rte_eth_bond.h b/drivers/net/bonding/rte_eth_bond.h index 6be5e46deb..28aa341d2f 100644 --- a/drivers/net/bonding/rte_eth_bond.h +++ b/drivers/net/bonding/rte_eth_bond.h @@ -95,12 +95,15 @@ extern "C" { /** * Create a bonding rte_eth_dev device * - * @param name Name of new link bonding device. - * @param mode Mode to initialize bonding device in. - * @param socket_id Socket Id on which to allocate eth_dev resources. + * @param name + * Name of new link bonding device. + * @param mode + * Mode to initialize bonding device in. + * @param socket_id + * Socket Id on which to allocate eth_dev resources. * * @return - * Port Id of created rte_eth_dev on success, negative value otherwise + * Port Id of created rte_eth_dev on success, negative value otherwise */ int rte_eth_bond_create(const char *name, uint8_t mode, uint8_t socket_id); @@ -108,10 +111,11 @@ rte_eth_bond_create(const char *name, uint8_t mode, uint8_t socket_id); /** * Free a bonding rte_eth_dev device * - * @param name Name of the link bonding device. + * @param name + * Name of the link bonding device. * * @return - * 0 on success, negative value otherwise + * 0 on success, negative value otherwise */ int rte_eth_bond_free(const char *name); @@ -119,11 +123,13 @@ rte_eth_bond_free(const char *name); /** * Add a rte_eth_dev device as a member to the bonding device * - * @param bonding_port_id Port ID of bonding device. - * @param member_port_id Port ID of member device. + * @param bonding_port_id + * Port ID of bonding device. + * @param member_port_id + * Port ID of member device. * * @return - * 0 on success, negative value otherwise + * 0 on success, negative value otherwise */ __rte_experimental int @@ -132,11 +138,13 @@ rte_eth_bond_member_add(uint16_t bonding_port_id, uint16_t member_port_id); /** * Remove a member rte_eth_dev device from the bonding device * - * @param bonding_port_id Port ID of bonding device. - * @param member_port_id Port ID of member device. + * @param bonding_port_id + * Port ID of bonding device. + * @param member_port_id + * Port ID of member device. * * @return - * 0 on success, negative value otherwise + * 0 on success, negative value otherwise */ __rte_experimental int @@ -145,11 +153,13 @@ rte_eth_bond_member_remove(uint16_t bonding_port_id, uint16_t member_port_id); /** * Set link bonding mode of bonding device * - * @param bonding_port_id Port ID of bonding device. - * @param mode Bonding mode to set + * @param bonding_port_id + * Port ID of bonding device. + * @param mode + * Bonding mode to set * * @return - * 0 on success, negative value otherwise + * 0 on success, negative value otherwise */ int rte_eth_bond_mode_set(uint16_t bonding_port_id, uint8_t mode); @@ -157,10 +167,11 @@ rte_eth_bond_mode_set(uint16_t bonding_port_id, uint8_t mode); /** * Get link bonding mode of bonding device * - * @param bonding_port_id Port ID of bonding device. + * @param bonding_port_id + * Port ID of bonding device. * * @return - * link bonding mode on success, negative value otherwise + * link bonding mode on success, negative value otherwise */ int rte_eth_bond_mode_get(uint16_t bonding_port_id); @@ -168,11 +179,13 @@ rte_eth_bond_mode_get(uint16_t bonding_port_id); /** * Set member rte_eth_dev as primary member of bonding device * - * @param bonding_port_id Port ID of bonding device. - * @param member_port_id Port ID of member device. + * @param bonding_port_id + * Port ID of bonding device. + * @param member_port_id + * Port ID of member device. * * @return - * 0 on success, negative value otherwise + * 0 on success, negative value otherwise */ int rte_eth_bond_primary_set(uint16_t bonding_port_id, uint16_t member_port_id); @@ -180,10 +193,11 @@ rte_eth_bond_primary_set(uint16_t bonding_port_id, uint16_t member_port_id); /** * Get primary member of bonding device * - * @param bonding_port_id Port ID of bonding device. + * @param bonding_port_id + * Port ID of bonding device. * * @return - * Port Id of primary member on success, -1 on failure + * Port Id of primary member on success, -1 on failure */ int rte_eth_bond_primary_get(uint16_t bonding_port_id); @@ -191,13 +205,16 @@ rte_eth_bond_primary_get(uint16_t bonding_port_id); /** * Populate an array with list of the members port id's of the bonding device * - * @param bonding_port_id Port ID of bonding eth_dev to interrogate - * @param members Array to be populated with the current active members - * @param len Length of members array + * @param bonding_port_id + * Port ID of bonding eth_dev to interrogate + * @param members + * Array to be populated with the current active members + * @param len + * Length of members array * * @return - * Number of members associated with bonding device on success, - * negative value otherwise + * Number of members associated with bonding device on success, + * negative value otherwise */ __rte_experimental int @@ -208,13 +225,16 @@ rte_eth_bond_members_get(uint16_t bonding_port_id, uint16_t members[], * Populate an array with list of the active members port id's of the bonding * device. * - * @param bonding_port_id Port ID of bonding eth_dev to interrogate - * @param members Array to be populated with the current active members - * @param len Length of members array + * @param bonding_port_id + * Port ID of bonding eth_dev to interrogate + * @param members + * Array to be populated with the current active members + * @param len + * Length of members array * * @return - * Number of active members associated with bonding device on success, - * negative value otherwise + * Number of active members associated with bonding device on success, + * negative value otherwise */ __rte_experimental int @@ -224,12 +244,13 @@ rte_eth_bond_active_members_get(uint16_t bonding_port_id, uint16_t members[], /** * Set explicit MAC address to use on bonding device and it's members. * - * @param bonding_port_id Port ID of bonding device. - * @param mac_addr MAC Address to use on bonding device overriding - * members MAC addresses + * @param bonding_port_id + * Port ID of bonding device. + * @param mac_addr + * MAC Address to use on bonding device overriding members MAC addresses * * @return - * 0 on success, negative value otherwise + * 0 on success, negative value otherwise */ int rte_eth_bond_mac_address_set(uint16_t bonding_port_id, @@ -239,10 +260,11 @@ rte_eth_bond_mac_address_set(uint16_t bonding_port_id, * Reset bonding device to use MAC from primary member on bonding device and it's * members. * - * @param bonding_port_id Port ID of bonding device. + * @param bonding_port_id + * Port ID of bonding device. * * @return - * 0 on success, negative value otherwise + * 0 on success, negative value otherwise */ int rte_eth_bond_mac_address_reset(uint16_t bonding_port_id); @@ -252,11 +274,13 @@ rte_eth_bond_mac_address_reset(uint16_t bonding_port_id); * balance mode, this parameter is otherwise ignored in other modes of * operation. * - * @param bonding_port_id Port ID of bonding device. - * @param policy Balance mode transmission policy. + * @param bonding_port_id + * Port ID of bonding device. + * @param policy + * Balance mode transmission policy. * * @return - * 0 on success, negative value otherwise. + * 0 on success, negative value otherwise. */ int rte_eth_bond_xmit_policy_set(uint16_t bonding_port_id, uint8_t policy); @@ -264,10 +288,11 @@ rte_eth_bond_xmit_policy_set(uint16_t bonding_port_id, uint8_t policy); /** * Get the transmit policy set on bonding device for balance mode operation * - * @param bonding_port_id Port ID of bonding device. + * @param bonding_port_id + * Port ID of bonding device. * * @return - * Balance transmit policy on success, negative value otherwise. + * Balance transmit policy on success, negative value otherwise. */ int rte_eth_bond_xmit_policy_get(uint16_t bonding_port_id); @@ -276,11 +301,13 @@ rte_eth_bond_xmit_policy_get(uint16_t bonding_port_id); * Set the link monitoring frequency (in ms) for monitoring the link status of * member devices * - * @param bonding_port_id Port ID of bonding device. - * @param internal_ms Monitoring interval in milliseconds + * @param bonding_port_id + * Port ID of bonding device. + * @param internal_ms + * Monitoring interval in milliseconds * * @return - * 0 on success, negative value otherwise. + * 0 on success, negative value otherwise. */ int @@ -290,10 +317,11 @@ rte_eth_bond_link_monitoring_set(uint16_t bonding_port_id, uint32_t internal_ms) * Get the current link monitoring frequency (in ms) for monitoring of the link * status of member devices * - * @param bonding_port_id Port ID of bonding device. + * @param bonding_port_id + * Port ID of bonding device. * * @return - * Monitoring interval on success, negative value otherwise. + * Monitoring interval on success, negative value otherwise. */ int rte_eth_bond_link_monitoring_get(uint16_t bonding_port_id); @@ -303,11 +331,13 @@ rte_eth_bond_link_monitoring_get(uint16_t bonding_port_id); * Set the period in milliseconds for delaying the disabling of a bonding link * when the link down status has been detected * - * @param bonding_port_id Port ID of bonding device. - * @param delay_ms Delay period in milliseconds. + * @param bonding_port_id + * Port ID of bonding device. + * @param delay_ms + * Delay period in milliseconds. * * @return - * 0 on success, negative value otherwise. + * 0 on success, negative value otherwise. */ int rte_eth_bond_link_down_prop_delay_set(uint16_t bonding_port_id, @@ -317,10 +347,11 @@ rte_eth_bond_link_down_prop_delay_set(uint16_t bonding_port_id, * Get the period in milliseconds set for delaying the disabling of a bonding * link when the link down status has been detected * - * @param bonding_port_id Port ID of bonding device. + * @param bonding_port_id + * Port ID of bonding device. * * @return - * Delay period on success, negative value otherwise. + * Delay period on success, negative value otherwise. */ int rte_eth_bond_link_down_prop_delay_get(uint16_t bonding_port_id); @@ -329,11 +360,13 @@ rte_eth_bond_link_down_prop_delay_get(uint16_t bonding_port_id); * Set the period in milliseconds for delaying the enabling of a bonding link * when the link up status has been detected * - * @param bonding_port_id Port ID of bonding device. - * @param delay_ms Delay period in milliseconds. + * @param bonding_port_id + * Port ID of bonding device. + * @param delay_ms + * Delay period in milliseconds. * * @return - * 0 on success, negative value otherwise. + * 0 on success, negative value otherwise. */ int rte_eth_bond_link_up_prop_delay_set(uint16_t bonding_port_id, @@ -343,10 +376,11 @@ rte_eth_bond_link_up_prop_delay_set(uint16_t bonding_port_id, * Get the period in milliseconds set for delaying the enabling of a bonding * link when the link up status has been detected * - * @param bonding_port_id Port ID of bonding device. + * @param bonding_port_id + * Port ID of bonding device. * * @return - * Delay period on success, negative value otherwise. + * Delay period on success, negative value otherwise. */ int rte_eth_bond_link_up_prop_delay_get(uint16_t bonding_port_id); diff --git a/drivers/net/bonding/rte_eth_bond_8023ad.h b/drivers/net/bonding/rte_eth_bond_8023ad.h index 4c280c7565..4936d7fc0c 100644 --- a/drivers/net/bonding/rte_eth_bond_8023ad.h +++ b/drivers/net/bonding/rte_eth_bond_8023ad.h @@ -155,8 +155,10 @@ struct rte_eth_bond_8023ad_member_info { * * Function returns current configuration of 802.3AX mode. * - * @param port_id Bonding device id - * @param conf Pointer to timeout structure. + * @param port_id + * Bonding device id + * @param conf + * Pointer to timeout structure. * * @return * 0 - if ok @@ -171,8 +173,10 @@ rte_eth_bond_8023ad_conf_get(uint16_t port_id, * * Function set new configuration of 802.3AX mode. * - * @param port_id Bonding device id - * @param conf Configuration, if NULL set default configuration. + * @param port_id + * Bonding device id + * @param conf + * Configuration, if NULL set default configuration. * @return * 0 - if ok * -EINVAL if configuration is invalid. @@ -186,8 +190,10 @@ rte_eth_bond_8023ad_setup(uint16_t port_id, * * Function returns current state of given member device. * - * @param member_id Port id of valid member. - * @param conf buffer for configuration + * @param member_id + * Port id of valid member. + * @param conf + * buffer for configuration * @return * 0 - if ok * -EINVAL if conf is NULL or member id is invalid (not a member of given @@ -201,9 +207,12 @@ rte_eth_bond_8023ad_member_info(uint16_t port_id, uint16_t member_id, /** * Configure a member port to start collecting. * - * @param port_id Bonding device id - * @param member_id Port id of valid member. - * @param enabled Non-zero when collection enabled. + * @param port_id + * Bonding device id + * @param member_id + * Port id of valid member. + * @param enabled + * Non-zero when collection enabled. * @return * 0 - if ok * -EINVAL if member is not valid. @@ -215,8 +224,10 @@ rte_eth_bond_8023ad_ext_collect(uint16_t port_id, uint16_t member_id, /** * Get COLLECTING flag from member port actor state. * - * @param port_id Bonding device id - * @param member_id Port id of valid member. + * @param port_id + * Bonding device id + * @param member_id + * Port id of valid member. * @return * 0 - if not set * 1 - if set @@ -228,9 +239,12 @@ rte_eth_bond_8023ad_ext_collect_get(uint16_t port_id, uint16_t member_id); /** * Configure a member port to start distributing. * - * @param port_id Bonding device id - * @param member_id Port id of valid member. - * @param enabled Non-zero when distribution enabled. + * @param port_id + * Bonding device id + * @param member_id + * Port id of valid member. + * @param enabled + * Non-zero when distribution enabled. * @return * 0 - if ok * -EINVAL if member is not valid. @@ -242,8 +256,10 @@ rte_eth_bond_8023ad_ext_distrib(uint16_t port_id, uint16_t member_id, /** * Get DISTRIBUTING flag from member port actor state. * - * @param port_id Bonding device id - * @param member_id Port id of valid member. + * @param port_id + * Bonding device id + * @param member_id + * Port id of valid member. * @return * 0 - if not set * 1 - if set @@ -256,9 +272,12 @@ rte_eth_bond_8023ad_ext_distrib_get(uint16_t port_id, uint16_t member_id); * LACPDU transmit path for external 802.3ad state machine. Caller retains * ownership of the packet on failure. * - * @param port_id Bonding device id - * @param member_id Port ID of valid member device. - * @param lacp_pkt mbuf containing LACPDU. + * @param port_id + * Bonding device id + * @param member_id + * Port ID of valid member device. + * @param lacp_pkt + * mbuf containing LACPDU. * * @return * 0 on success, negative value otherwise. @@ -284,7 +303,8 @@ rte_eth_bond_8023ad_ext_slowtx(uint16_t port_id, uint16_t member_id, * * Bonding port must be stopped to change this configuration. * - * @param port_id Bonding device id + * @param port_id + * Bonding device id * * @return * 0 on success, negative value otherwise. @@ -299,9 +319,11 @@ rte_eth_bond_8023ad_dedicated_queues_enable(uint16_t port_id); * * Bonding port must be stopped to change this configuration. * - * @see rte_eth_bond_8023ad_slow_pkt_hw_filter_enable + * @see + * rte_eth_bond_8023ad_slow_pkt_hw_filter_enable * - * @param port_id Bonding device id + * @param port_id + * Bonding device id * @return * 0 on success, negative value otherwise. * @@ -309,9 +331,11 @@ rte_eth_bond_8023ad_dedicated_queues_enable(uint16_t port_id); int rte_eth_bond_8023ad_dedicated_queues_disable(uint16_t port_id); -/* +/** * Get aggregator mode for 8023ad - * @param port_id Bonding device id + * + * @param port_id + * Bonding device id * * @return * aggregator mode on success, negative value otherwise @@ -321,7 +345,9 @@ rte_eth_bond_8023ad_agg_selection_get(uint16_t port_id); /** * Set aggregator mode for 8023ad - * @param port_id Bonding device id + * @param port_id + * Bonding device id + * * @return * 0 on success, negative value otherwise */ diff --git a/drivers/net/bonding/rte_eth_bond_alb.h b/drivers/net/bonding/rte_eth_bond_alb.h index f51c355d36..3cf0d87750 100644 --- a/drivers/net/bonding/rte_eth_bond_alb.h +++ b/drivers/net/bonding/rte_eth_bond_alb.h @@ -63,9 +63,12 @@ bond_mode_alb_enable(struct rte_eth_dev *bond_dev); * forwarded to application without changes. If it is ARP reply, client table * is updated. * - * @param eth_h ETH header of received packet. - * @param offset Vlan header offset. - * @param internals Bonding data. + * @param eth_h + * ETH header of received packet. + * @param offset + * Vlan header offset. + * @param internals + * Bonding data. */ void bond_mode_alb_arp_recv(struct rte_ether_hdr *eth_h, uint16_t offset, @@ -77,12 +80,15 @@ bond_mode_alb_arp_recv(struct rte_ether_hdr *eth_h, uint16_t offset, * If it is ARP Reply, it is send on member stored in client table for that * connection. On Reply function also updates data in client table. * - * @param eth_h ETH header of transmitted packet. - * @param offset Vlan header offset. - * @param internals Bonding data. + * @param eth_h + * ETH header of transmitted packet. + * @param offset + * Vlan header offset. + * @param internals + * Bonding data. * * @return - * Index of member on which packet should be sent. + * Index of member on which packet should be sent. */ uint16_t bond_mode_alb_arp_xmit(struct rte_ether_hdr *eth_h, uint16_t offset, @@ -91,12 +97,15 @@ bond_mode_alb_arp_xmit(struct rte_ether_hdr *eth_h, uint16_t offset, /** * Function fills packet with ARP data from client_info. * - * @param client_info Data of client to which packet is sent. - * @param pkt Pointer to packet which is sent. - * @param internals Bonding data. + * @param client_info + * Data of client to which packet is sent. + * @param pkt + * Pointer to packet which is sent. + * @param internals + * Bonding data. * * @return - * Index of member on which packet should be sent. + * Index of member on which packet should be sent. */ uint16_t bond_mode_alb_arp_upd(struct client_data *client_info, @@ -105,7 +114,8 @@ bond_mode_alb_arp_upd(struct client_data *client_info, /** * Function updates member indexes of active connections. * - * @param bond_dev Pointer to bonding device struct. + * @param bond_dev + * Pointer to bonding device struct. */ void bond_mode_alb_client_list_upd(struct rte_eth_dev *bond_dev); -- 2.39.1 ^ permalink raw reply [flat|nested] 80+ messages in thread
* [PATCH 4/8] net/bonding: add bonding port arguments 2023-10-05 2:40 [PATCH 0/8] Enhance the bond framework to support offload Chaoyong He ` (2 preceding siblings ...) 2023-10-05 2:40 ` [PATCH 3/8] net/bonding: modify interface comment format Chaoyong He @ 2023-10-05 2:40 ` Chaoyong He 2023-10-05 2:40 ` [PATCH 5/8] net/bonding: support add port by data name Chaoyong He ` (4 subsequent siblings) 8 siblings, 0 replies; 80+ messages in thread From: Chaoyong He @ 2023-10-05 2:40 UTC (permalink / raw) To: dev; +Cc: oss-drivers, Long Wu, James Hershaw, Chaoyong He From: Long Wu <long.wu@corigine.com> Include the following new arguments for bonding ports: - "notify_member" to enable/disable member notification. - "dedicated_queue" to enable/disable dedicated queue. Add these two arguments in initial argument. Signed-off-by: Long Wu <long.wu@corigine.com> Reviewed-by: James Hershaw <james.hershaw@corigine.com> Reviewed-by: Chaoyong He <chaoyong.he@corigine.com> --- drivers/net/bonding/eth_bond_private.h | 10 ++++ drivers/net/bonding/rte_eth_bond.h | 28 ++++++++++++ drivers/net/bonding/rte_eth_bond_api.c | 14 ++++++ drivers/net/bonding/rte_eth_bond_args.c | 44 ++++++++++++++++++ drivers/net/bonding/rte_eth_bond_pmd.c | 61 ++++++++++++++++++++++++- 5 files changed, 156 insertions(+), 1 deletion(-) diff --git a/drivers/net/bonding/eth_bond_private.h b/drivers/net/bonding/eth_bond_private.h index 1344f8c002..b217d98c31 100644 --- a/drivers/net/bonding/eth_bond_private.h +++ b/drivers/net/bonding/eth_bond_private.h @@ -28,6 +28,8 @@ #define PMD_BOND_LSC_POLL_PERIOD_KVARG ("lsc_poll_period_ms") #define PMD_BOND_LINK_UP_PROP_DELAY_KVARG ("up_delay") #define PMD_BOND_LINK_DOWN_PROP_DELAY_KVARG ("down_delay") +#define PMD_BOND_NOTIFY_MEMBER_KVARG ("notify_member") +#define PMD_BOND_DEDICATED_QUEUE_KVARG ("dedicated_queue") #define PMD_BOND_XMIT_POLICY_LAYER2_KVARG ("l2") #define PMD_BOND_XMIT_POLICY_LAYER23_KVARG ("l23") @@ -320,6 +322,14 @@ int bond_ethdev_parse_time_ms_kvarg(const char *key, const char *value, void *extra_args); +int +bond_ethdev_parse_notify_member_kvarg(const char *key __rte_unused, + const char *value, void *extra_args); + +int +bond_ethdev_parse_dedicated_queue_kvarg(const char *key __rte_unused, + const char *value, void *extra_args); + void bond_tlb_disable(struct bond_dev_private *internals); diff --git a/drivers/net/bonding/rte_eth_bond.h b/drivers/net/bonding/rte_eth_bond.h index 28aa341d2f..2ae2f28ec1 100644 --- a/drivers/net/bonding/rte_eth_bond.h +++ b/drivers/net/bonding/rte_eth_bond.h @@ -385,6 +385,34 @@ rte_eth_bond_link_up_prop_delay_set(uint16_t bonding_port_id, int rte_eth_bond_link_up_prop_delay_get(uint16_t bonding_port_id); +/** + * Set the flag that whether bonding device enable member notification. + * + * @param bonding_port_id + * Port ID of bonding device. + * @param notify_member + * The flag of enable bond member notification + * + * @return + * 0 on success, negative value otherwise. + */ +int +rte_eth_bond_notify_member_flag_set(uint16_t bonding_port_id, bool notify_member); + +/** + * Set the flag that whether bonding device enable dedicated queue. + * + * @param bonding_port_id + * Port ID of bonding device. + * @param queue_flag + * The flag of enable bond dedicated queue + * + * @return + * 0 on success, negative value otherwise. + */ +int +rte_eth_bond_dedicated_queue_flag_set(uint16_t bonding_port_id, bool queue_flag); + /** * Set the flag of whether bonding port notifies member ports. * diff --git a/drivers/net/bonding/rte_eth_bond_api.c b/drivers/net/bonding/rte_eth_bond_api.c index 0be580b19b..a042f05a4c 100644 --- a/drivers/net/bonding/rte_eth_bond_api.c +++ b/drivers/net/bonding/rte_eth_bond_api.c @@ -1114,6 +1114,20 @@ rte_eth_bond_link_up_prop_delay_get(uint16_t bonding_port_id) return internals->link_up_delay_ms; } +int +rte_eth_bond_dedicated_queue_flag_set(uint16_t bonding_port_id, bool queue_flag) +{ + struct bond_dev_private *internals; + + if (valid_bonding_port_id(bonding_port_id) != 0) + return -1; + + internals = rte_eth_devices[bonding_port_id].data->dev_private; + internals->mode4.dedicated_queues.enabled = queue_flag; + + return 0; +} + int rte_eth_bond_notify_member_flag_set(uint16_t bonding_port_id, bool notify_member) { diff --git a/drivers/net/bonding/rte_eth_bond_args.c b/drivers/net/bonding/rte_eth_bond_args.c index bdec5d61d4..8a3e4656ef 100644 --- a/drivers/net/bonding/rte_eth_bond_args.c +++ b/drivers/net/bonding/rte_eth_bond_args.c @@ -20,6 +20,8 @@ const char *pmd_bond_init_valid_arguments[] = { PMD_BOND_MAC_ADDR_KVARG, PMD_BOND_AGG_MODE_KVARG, RTE_DEVARGS_KEY_DRIVER, + PMD_BOND_NOTIFY_MEMBER_KVARG, + PMD_BOND_DEDICATED_QUEUE_KVARG, NULL }; @@ -297,3 +299,45 @@ bond_ethdev_parse_time_ms_kvarg(const char *key __rte_unused, return 0; } + +int +bond_ethdev_parse_notify_member_kvarg(const char *key __rte_unused, + const char *value, void *extra_args) +{ + bool *notify_member; + + if (value == NULL || extra_args == NULL) + return -1; + + notify_member = extra_args; + + if (strcmp("enable", value) == 0) + *notify_member = true; + else if (strcmp("disable", value) == 0) + *notify_member = false; + else + return -1; + + return 0; +} + +int +bond_ethdev_parse_dedicated_queue_kvarg(const char *key __rte_unused, + const char *value, void *extra_args) +{ + bool *dedicated_queue; + + if (value == NULL || extra_args == NULL) + return -1; + + dedicated_queue = extra_args; + + if (strcmp("enable", value) == 0) + *dedicated_queue = true; + else if (strcmp("disable", value) == 0) + *dedicated_queue = false; + else + return -1; + + return 0; +} diff --git a/drivers/net/bonding/rte_eth_bond_pmd.c b/drivers/net/bonding/rte_eth_bond_pmd.c index b99b8b8938..1ebeb270c8 100644 --- a/drivers/net/bonding/rte_eth_bond_pmd.c +++ b/drivers/net/bonding/rte_eth_bond_pmd.c @@ -4227,6 +4227,63 @@ bond_ethdev_configure(struct rte_eth_dev *dev) return -1; } + /* Parse/set notify member flag */ + arg_count = rte_kvargs_count(kvlist, PMD_BOND_NOTIFY_MEMBER_KVARG); + if (arg_count == 1) { + bool notify_member; + if (rte_kvargs_process(kvlist, + PMD_BOND_NOTIFY_MEMBER_KVARG, + &bond_ethdev_parse_notify_member_kvarg, + ¬ify_member) < 0) { + RTE_BOND_LOG(ERR, + "Invalid notify member value specified" + " for bonding device %s", name); + return -1; + } + + if (rte_eth_bond_notify_member_flag_set(port_id, notify_member) != 0) { + RTE_BOND_LOG(ERR, + "Failed to set notify member (%u) on" + " bonding device %s", notify_member, name); + return -1; + } + } else if (arg_count > 1) { + RTE_BOND_LOG(ERR, + "notify member flag can be specified only once" + " for bonding device %s", name); + return -1; + } + + /* Parse/set dedicated queue flag */ + arg_count = rte_kvargs_count(kvlist, PMD_BOND_DEDICATED_QUEUE_KVARG); + if (arg_count == 1) { + bool dedicated_queue; + if (rte_kvargs_process(kvlist, + PMD_BOND_DEDICATED_QUEUE_KVARG, + &bond_ethdev_parse_dedicated_queue_kvarg, + &dedicated_queue) < 0) { + RTE_BOND_LOG(ERR, + "Invalid dedicated queue flag specified" + " for bonding device %s", name); + return -1; + } + + if (internals->mode == BONDING_MODE_8023AD) { + if (rte_eth_bond_dedicated_queue_flag_set(port_id, dedicated_queue) != 0) { + RTE_BOND_LOG(ERR, + "Failed to enable/disable dedicated" + " queue flag on bonding device %s", + name); + return -1; + } + } + } else if (arg_count > 1) { + RTE_BOND_LOG(ERR, + "dedicated queue flag can be specified only once" + " for bonding device %s", name); + return -1; + } + /* configure members so we can pass mtu setting */ for (i = 0; i < internals->member_count; i++) { struct rte_eth_dev *member_ethdev = @@ -4264,7 +4321,9 @@ RTE_PMD_REGISTER_PARAM_STRING(net_bonding, "mac=<mac addr> " "lsc_poll_period_ms=<int> " "up_delay=<int> " - "down_delay=<int>"); + "down_delay=<int>" + "notify_member=[enable | disable] " + "dedicated_queue=[enable | disable] "); /* We can't use RTE_LOG_REGISTER_DEFAULT because of the forced name for * this library, see meson.build. -- 2.39.1 ^ permalink raw reply [flat|nested] 80+ messages in thread
* [PATCH 5/8] net/bonding: support add port by data name 2023-10-05 2:40 [PATCH 0/8] Enhance the bond framework to support offload Chaoyong He ` (3 preceding siblings ...) 2023-10-05 2:40 ` [PATCH 4/8] net/bonding: add bonding port arguments Chaoyong He @ 2023-10-05 2:40 ` Chaoyong He 2023-10-05 2:40 ` [PATCH 6/8] net/bonding: create new rte flow header file Chaoyong He ` (3 subsequent siblings) 8 siblings, 0 replies; 80+ messages in thread From: Chaoyong He @ 2023-10-05 2:40 UTC (permalink / raw) To: dev; +Cc: oss-drivers, Long Wu, James Hershaw, Chaoyong He From: Long Wu <long.wu@corigine.com> Several ports may share the same PCI address, like nfp representor. So we cannot add this type of ports to bonding port by "--vdev" argument in dpdk-testpmd. But the port's data name is unique between them, we include an option to add such ports to the bonding port. After adding this feature, we can create a bonding port that member port is this type of port by "--vdev" in dpdk-testpmd start command. For example: dpdk-testpmd -l 2-10 -s 0x8 -a ca:00.0,representor=[0-2] --vdev 'net_bonding0,member=flower_repr_p0,member=flower_repr_p1, mode=4,socket_id=1,xmit_policy=l34' -- -i Note: 1. "ca:00.0" is nfp 4000 card. 2. "flower_repr_p0" and "flower_repr_p1" are nfp phy representor's data name. Signed-off-by: Long Wu <long.wu@corigine.com> Reviewed-by: James Hershaw <james.hershaw@corigine.com> Reviewed-by: Chaoyong He <chaoyong.he@corigine.com> --- drivers/net/bonding/rte_eth_bond_args.c | 3 +++ 1 file changed, 3 insertions(+) diff --git a/drivers/net/bonding/rte_eth_bond_args.c b/drivers/net/bonding/rte_eth_bond_args.c index 8a3e4656ef..b320eb3038 100644 --- a/drivers/net/bonding/rte_eth_bond_args.c +++ b/drivers/net/bonding/rte_eth_bond_args.c @@ -70,6 +70,9 @@ find_port_id_by_dev_name(const char *name) if (strcmp(rte_eth_devices[i].device->name, name) == 0) return i; + + if (strcmp(rte_eth_devices[i].data->name, name) == 0) + return i; } return -1; } -- 2.39.1 ^ permalink raw reply [flat|nested] 80+ messages in thread
* [PATCH 6/8] net/bonding: create new rte flow header file 2023-10-05 2:40 [PATCH 0/8] Enhance the bond framework to support offload Chaoyong He ` (4 preceding siblings ...) 2023-10-05 2:40 ` [PATCH 5/8] net/bonding: support add port by data name Chaoyong He @ 2023-10-05 2:40 ` Chaoyong He 2023-10-05 2:40 ` [PATCH 7/8] net/bonding: support checking valid bonding port ID Chaoyong He ` (2 subsequent siblings) 8 siblings, 0 replies; 80+ messages in thread From: Chaoyong He @ 2023-10-05 2:40 UTC (permalink / raw) To: dev; +Cc: oss-drivers, Long Wu, James Hershaw, Chaoyong He From: Long Wu <long.wu@corigine.com> Move the flow code to a new head file to make flow related code more clean and make the code architecture more reasonable in the future. There is no functional change, just moving verbatim code around. Signed-off-by: Long Wu <long.wu@corigine.com> Reviewed-by: James Hershaw <james.hershaw@corigine.com> Reviewed-by: Chaoyong He <chaoyong.he@corigine.com> --- drivers/net/bonding/eth_bond_private.h | 13 ------------- drivers/net/bonding/rte_eth_bond_api.c | 1 + drivers/net/bonding/rte_eth_bond_flow.c | 1 + drivers/net/bonding/rte_eth_bond_flow.h | 22 ++++++++++++++++++++++ drivers/net/bonding/rte_eth_bond_pmd.c | 1 + 5 files changed, 25 insertions(+), 13 deletions(-) create mode 100644 drivers/net/bonding/rte_eth_bond_flow.h diff --git a/drivers/net/bonding/eth_bond_private.h b/drivers/net/bonding/eth_bond_private.h index b217d98c31..aaab521eb9 100644 --- a/drivers/net/bonding/eth_bond_private.h +++ b/drivers/net/bonding/eth_bond_private.h @@ -9,10 +9,8 @@ #include <sys/queue.h> #include <ethdev_driver.h> -#include <rte_flow.h> #include <rte_spinlock.h> #include <rte_bitmap.h> -#include <rte_flow_driver.h> #include "rte_eth_bond.h" #include "eth_bond_8023ad_private.h" @@ -47,8 +45,6 @@ extern const char *pmd_bond_init_valid_arguments[]; extern struct rte_vdev_driver pmd_bond_drv; -extern const struct rte_flow_ops bond_flow_ops; - /** Port Queue Mapping Structure */ struct bond_rx_queue { uint16_t queue_id; @@ -94,15 +90,6 @@ struct bond_member_details { uint16_t reta_size; }; -struct rte_flow { - TAILQ_ENTRY(rte_flow) next; - /* Members flows */ - struct rte_flow *flows[RTE_MAX_ETHPORTS]; - /* Flow description for synchronization */ - struct rte_flow_conv_rule rule; - uint8_t rule_data[]; -}; - typedef void (*burst_xmit_hash_t)(struct rte_mbuf **buf, uint16_t nb_pkts, uint16_t member_count, uint16_t *members); diff --git a/drivers/net/bonding/rte_eth_bond_api.c b/drivers/net/bonding/rte_eth_bond_api.c index a042f05a4c..0113dfdc16 100644 --- a/drivers/net/bonding/rte_eth_bond_api.c +++ b/drivers/net/bonding/rte_eth_bond_api.c @@ -12,6 +12,7 @@ #include <rte_kvargs.h> #include "rte_eth_bond.h" +#include "rte_eth_bond_flow.h" #include "eth_bond_private.h" #include "eth_bond_8023ad_private.h" diff --git a/drivers/net/bonding/rte_eth_bond_flow.c b/drivers/net/bonding/rte_eth_bond_flow.c index 71a91675f7..e6c7ce5362 100644 --- a/drivers/net/bonding/rte_eth_bond_flow.c +++ b/drivers/net/bonding/rte_eth_bond_flow.c @@ -1,6 +1,7 @@ /* SPDX-License-Identifier: BSD-3-Clause * Copyright 2018 Mellanox Technologies, Ltd */ +#include "rte_eth_bond_flow.h" #include <stddef.h> #include <string.h> diff --git a/drivers/net/bonding/rte_eth_bond_flow.h b/drivers/net/bonding/rte_eth_bond_flow.h new file mode 100644 index 0000000000..7394e0e2e1 --- /dev/null +++ b/drivers/net/bonding/rte_eth_bond_flow.h @@ -0,0 +1,22 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright (c) 2023 Corigine, Inc. + */ + +#ifndef _RTE_ETH_BOND_FLOW_H_ +#define _RTE_ETH_BOND_FLOW_H_ + +#include <rte_flow.h> +#include <rte_flow_driver.h> + +extern const struct rte_flow_ops bond_flow_ops; + +struct rte_flow { + TAILQ_ENTRY(rte_flow) next; + struct rte_flow *flows[RTE_MAX_ETHPORTS]; + /**< Member ports flows */ + struct rte_flow_conv_rule rule; + /**< Flow description for synchronization */ + uint8_t rule_data[]; +}; + +#endif /* _RTE_ETH_BOND_FLOW_H_ */ diff --git a/drivers/net/bonding/rte_eth_bond_pmd.c b/drivers/net/bonding/rte_eth_bond_pmd.c index 1ebeb270c8..630afc3740 100644 --- a/drivers/net/bonding/rte_eth_bond_pmd.c +++ b/drivers/net/bonding/rte_eth_bond_pmd.c @@ -21,6 +21,7 @@ #include <rte_string_fns.h> #include "rte_eth_bond.h" +#include "rte_eth_bond_flow.h" #include "eth_bond_private.h" #include "eth_bond_8023ad_private.h" -- 2.39.1 ^ permalink raw reply [flat|nested] 80+ messages in thread
* [PATCH 7/8] net/bonding: support checking valid bonding port ID 2023-10-05 2:40 [PATCH 0/8] Enhance the bond framework to support offload Chaoyong He ` (5 preceding siblings ...) 2023-10-05 2:40 ` [PATCH 6/8] net/bonding: create new rte flow header file Chaoyong He @ 2023-10-05 2:40 ` Chaoyong He 2023-10-05 2:40 ` [PATCH 8/8] net/bonding: add commands for bonding port notification Chaoyong He 2023-10-07 1:34 ` [PATCH v2 0/8] Enhance the bond framework to support offload Chaoyong He 8 siblings, 0 replies; 80+ messages in thread From: Chaoyong He @ 2023-10-05 2:40 UTC (permalink / raw) To: dev; +Cc: oss-drivers, Long Wu, James Hershaw, Chaoyong He From: Long Wu <long.wu@corigine.com> Add API to support checking if the port id is a bonding port id. Signed-off-by: Long Wu <long.wu@corigine.com> Reviewed-by: James Hershaw <james.hershaw@corigine.com> Reviewed-by: Chaoyong He <chaoyong.he@corigine.com> --- drivers/net/bonding/rte_eth_bond.h | 13 +++++++++++++ drivers/net/bonding/rte_eth_bond_api.c | 7 +++++++ drivers/net/bonding/version.map | 1 + 3 files changed, 21 insertions(+) diff --git a/drivers/net/bonding/rte_eth_bond.h b/drivers/net/bonding/rte_eth_bond.h index 2ae2f28ec1..8fa7b0aa01 100644 --- a/drivers/net/bonding/rte_eth_bond.h +++ b/drivers/net/bonding/rte_eth_bond.h @@ -475,6 +475,19 @@ __rte_experimental int rte_eth_bond_hw_create_get(uint16_t bonding_port_id, uint16_t member_port_id); +/** + * Check whether bonding port id is valid. + * + * @param bonding_port_id + * Port ID of bonding device. + * + * @return + * 0 on success, negative value otherwise. + */ +__rte_experimental +int +rte_eth_bond_valid_bonding_port_id(uint16_t bonding_port_id); + #ifdef __cplusplus } #endif diff --git a/drivers/net/bonding/rte_eth_bond_api.c b/drivers/net/bonding/rte_eth_bond_api.c index 0113dfdc16..80d71529cc 100644 --- a/drivers/net/bonding/rte_eth_bond_api.c +++ b/drivers/net/bonding/rte_eth_bond_api.c @@ -1214,3 +1214,10 @@ rte_eth_bond_hw_create_get(uint16_t bonding_port_id, uint16_t member_port_id) return member_dev->dev_ops->bond_hw_create_get(member_dev, bonding_dev); } + + +int +rte_eth_bond_valid_bonding_port_id(uint16_t port_id) +{ + return valid_bonding_port_id(port_id); +} diff --git a/drivers/net/bonding/version.map b/drivers/net/bonding/version.map index 3cfff51269..bf5e50521e 100644 --- a/drivers/net/bonding/version.map +++ b/drivers/net/bonding/version.map @@ -39,4 +39,5 @@ EXPERIMENTAL { rte_eth_bond_notify_member_flag_get; rte_eth_bond_notify_member_flag_set; rte_eth_bond_notify_members; + rte_eth_bond_valid_bonding_port_id; }; -- 2.39.1 ^ permalink raw reply [flat|nested] 80+ messages in thread
* [PATCH 8/8] net/bonding: add commands for bonding port notification 2023-10-05 2:40 [PATCH 0/8] Enhance the bond framework to support offload Chaoyong He ` (6 preceding siblings ...) 2023-10-05 2:40 ` [PATCH 7/8] net/bonding: support checking valid bonding port ID Chaoyong He @ 2023-10-05 2:40 ` Chaoyong He 2023-10-07 1:34 ` [PATCH v2 0/8] Enhance the bond framework to support offload Chaoyong He 8 siblings, 0 replies; 80+ messages in thread From: Chaoyong He @ 2023-10-05 2:40 UTC (permalink / raw) To: dev; +Cc: oss-drivers, Long Wu, James Hershaw, Chaoyong He From: Long Wu <long.wu@corigine.com> Add some commands to support bonding port notification in dpdk-testpmd. 1. We can enable the notification by command: "set bonding notify_member (port_id) (enable|disable)" 2. If member port hardware try to create the bonding port after notification we can get the status by command: "get bonding member hardware create (member_port_id) (bonding_port_id)" Signed-off-by: Long Wu <long.wu@corigine.com> Reviewed-by: James Hershaw <james.hershaw@corigine.com> Reviewed-by: Chaoyong He <chaoyong.he@corigine.com> --- .../link_bonding_poll_mode_drv_lib.rst | 19 +++ drivers/net/bonding/bonding_testpmd.c | 128 ++++++++++++++++++ 2 files changed, 147 insertions(+) diff --git a/doc/guides/prog_guide/link_bonding_poll_mode_drv_lib.rst b/doc/guides/prog_guide/link_bonding_poll_mode_drv_lib.rst index 60717a3587..9f6443ebd8 100644 --- a/doc/guides/prog_guide/link_bonding_poll_mode_drv_lib.rst +++ b/doc/guides/prog_guide/link_bonding_poll_mode_drv_lib.rst @@ -637,3 +637,22 @@ in balance mode with a transmission policy of layer 2+3:: Members (3): [1 3 4] Active Members (3): [1 3 4] Primary: [3] + +set bonding notify_member +~~~~~~~~~~~~~~~~~~~~~~~~~ + +Set the notify member flag of bonding port:: + + testpmd> set bonding notify_member (port_id) (enable|disable) + +This command just set the flag of notification. +If we enable it, bonding PMD will notify member ports when its some +configurations changed. So member ports can do some private things, maybe hardware +bonding creation and etc. + +get bonding member hardware create +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +Get the status of member port hardware creating the bonding port:: + + testpmd> get bonding member hardware create (member_port_id) (bonding_port_id) diff --git a/drivers/net/bonding/bonding_testpmd.c b/drivers/net/bonding/bonding_testpmd.c index 8fcd6cadd0..da7d9cc58f 100644 --- a/drivers/net/bonding/bonding_testpmd.c +++ b/drivers/net/bonding/bonding_testpmd.c @@ -692,6 +692,124 @@ static cmdline_parse_inst_t cmd_set_bonding_agg_mode_policy = { } }; +struct cmd_set_bonding_notify_member_result { + cmdline_fixed_string_t set; + cmdline_fixed_string_t bonding; + cmdline_fixed_string_t notify_member; + uint16_t port_num; + cmdline_fixed_string_t mode; +}; + +static void +cmd_set_bonding_notify_member_parsed(void *parsed_result, + __rte_unused struct cmdline *cl, __rte_unused void *data) +{ + struct cmd_set_bonding_notify_member_result *res = parsed_result; + bool notify_member = false; + + if (strcmp(res->notify_member, "enable") == 0) + notify_member = true; + else if (strcmp(res->notify_member, "disable") == 0) + notify_member = false; + + rte_eth_bond_notify_member_flag_set(res->port_num, notify_member); +} + +static cmdline_parse_token_string_t cmd_set_bonding_notify_member_set = + TOKEN_STRING_INITIALIZER(struct cmd_set_bonding_notify_member_result, + set, "set"); +static cmdline_parse_token_string_t cmd_set_bonding_notify_member_bonding = + TOKEN_STRING_INITIALIZER(struct cmd_set_bonding_notify_member_result, + bonding, "bonding"); +static cmdline_parse_token_string_t cmd_set_bonding_notify_member = + TOKEN_STRING_INITIALIZER(struct cmd_set_bonding_notify_member_result, + notify_member, "notify_member"); +static cmdline_parse_token_num_t cmd_set_bonding_notify_member_portnum = + TOKEN_NUM_INITIALIZER(struct cmd_set_bonding_notify_member_result, + port_num, RTE_UINT16); +static cmdline_parse_token_string_t cmd_set_bonding_notify_member_mode_string = + TOKEN_STRING_INITIALIZER(struct cmd_set_bonding_notify_member_result, + mode, "enable#disable"); + +static cmdline_parse_inst_t cmd_set_bonding_notify_member_ports = { + .f = cmd_set_bonding_notify_member_parsed, + .data = NULL, + .help_str = "set bonding notify_member (port_id) (enable|disable)", + .tokens = { + (void *)&cmd_set_bonding_notify_member_set, + (void *)&cmd_set_bonding_notify_member_bonding, + (void *)&cmd_set_bonding_notify_member, + (void *)&cmd_set_bonding_notify_member_portnum, + (void *)&cmd_set_bonding_notify_member_mode_string, + NULL + } +}; + +struct cmd_get_bonding_member_hw_create_result { + cmdline_fixed_string_t get; + cmdline_fixed_string_t bonding; + cmdline_fixed_string_t member; + cmdline_fixed_string_t hardware; + cmdline_fixed_string_t create; + uint16_t member_port_id; + uint16_t bonding_port_id; +}; + +static void +cmd_get_bonding_member_hw_create_parsed(void *parsed_result, + __rte_unused struct cmdline *cl, __rte_unused void *data) +{ + struct cmd_get_bonding_member_hw_create_result *res = parsed_result; + int ret; + + ret = rte_eth_bond_hw_create_get(res->bonding_port_id, res->member_port_id); + if (ret == 0) + printf("Member port %u hardware creates bonding port %u successfully\n", + res->member_port_id, res->bonding_port_id); + else + printf("Failed to get status of member port %u hardware creating" + " bonding port %u, %d\n", + res->member_port_id, res->bonding_port_id, ret); +} + +static cmdline_parse_token_string_t cmd_get_bonding_member_hw_create_get = + TOKEN_STRING_INITIALIZER(struct cmd_get_bonding_member_hw_create_result, + get, "get"); +static cmdline_parse_token_string_t cmd_get_bonding_member_hw_create_bonding = + TOKEN_STRING_INITIALIZER(struct cmd_get_bonding_member_hw_create_result, + bonding, "bonding"); +static cmdline_parse_token_string_t cmd_get_bonding_member_hw_create_member = + TOKEN_STRING_INITIALIZER(struct cmd_get_bonding_member_hw_create_result, + member, "member"); +static cmdline_parse_token_string_t cmd_get_bonding_member_hw_create_hardware = + TOKEN_STRING_INITIALIZER(struct cmd_get_bonding_member_hw_create_result, + create, "hardware"); +static cmdline_parse_token_string_t cmd_get_bonding_member_hw_create_create = + TOKEN_STRING_INITIALIZER(struct cmd_get_bonding_member_hw_create_result, + create, "create"); +static cmdline_parse_token_num_t cmd_get_bonding_member_hw_create_memberportid = + TOKEN_NUM_INITIALIZER(struct cmd_get_bonding_member_hw_create_result, + member_port_id, RTE_UINT16); +static cmdline_parse_token_num_t cmd_get_bonding_member_hw_create_bondingportid = + TOKEN_NUM_INITIALIZER(struct cmd_get_bonding_member_hw_create_result, + bonding_port_id, RTE_UINT16); + +static cmdline_parse_inst_t cmd_get_member_hw_create_bonding = { + .f = cmd_get_bonding_member_hw_create_parsed, + .data = NULL, + .help_str = "get bonding member hardware create (member_port_id) (bonding_port_id)", + .tokens = { + (void *)&cmd_get_bonding_member_hw_create_get, + (void *)&cmd_get_bonding_member_hw_create_bonding, + (void *)&cmd_get_bonding_member_hw_create_member, + (void *)&cmd_get_bonding_member_hw_create_hardware, + (void *)&cmd_get_bonding_member_hw_create_create, + (void *)&cmd_get_bonding_member_hw_create_memberportid, + (void *)&cmd_get_bonding_member_hw_create_bondingportid, + NULL + } +}; + static struct testpmd_driver_commands bonding_cmds = { .commands = { { @@ -749,6 +867,16 @@ static struct testpmd_driver_commands bonding_cmds = { "set bonding mode IEEE802.3AD aggregator policy (port_id) (agg_name)\n" " Set Aggregation mode for IEEE802.3AD (mode 4)\n", }, + { + &cmd_set_bonding_notify_member_ports, + "set bonding notify_member (port_id) (enable|disable)\n" + " Enable/disable the notify member flag of bonding port\n", + }, + { + &cmd_get_member_hw_create_bonding, + "get bonding member hardware create (member_port_id) (bonding_port_id)\n" + " Get the status of member port hardware creating the bonding port\n", + }, { NULL, NULL }, }, }; -- 2.39.1 ^ permalink raw reply [flat|nested] 80+ messages in thread
* [PATCH v2 0/8] Enhance the bond framework to support offload 2023-10-05 2:40 [PATCH 0/8] Enhance the bond framework to support offload Chaoyong He ` (7 preceding siblings ...) 2023-10-05 2:40 ` [PATCH 8/8] net/bonding: add commands for bonding port notification Chaoyong He @ 2023-10-07 1:34 ` Chaoyong He 2023-10-07 1:34 ` [PATCH v2 1/8] ethdev: add member notification for bonding port Chaoyong He ` (8 more replies) 8 siblings, 9 replies; 80+ messages in thread From: Chaoyong He @ 2023-10-07 1:34 UTC (permalink / raw) To: dev; +Cc: oss-drivers, Chaoyong He This patch series try to enhance the bond framework to support the offload feature better: * Add new API to make the member port can access some information of the bond port which belongs. * Add new API to get the result of whether bond port is created by the member port. * Add two command line argument to control if enable member port notification and dedicated queue features. * Add logic to support add ports which share the same PCI address into bond port. * Also modify the testpmd application to test the new APIs and logics added by this patch series. --- v2: * Fix compile error on github-robot by removing the redundancy function declaration in the header file. --- Long Wu (8): ethdev: add member notification for bonding port ethdev: add API to get hardware creation of bonding port net/bonding: modify interface comment format net/bonding: add bonding port arguments net/bonding: support add port by data name net/bonding: create new rte flow header file net/bonding: support checking valid bonding port ID net/bonding: add commands for bonding port notification .../link_bonding_poll_mode_drv_lib.rst | 19 ++ drivers/net/bonding/bonding_testpmd.c | 128 ++++++++++ drivers/net/bonding/eth_bond_8023ad_private.h | 52 ++-- drivers/net/bonding/eth_bond_private.h | 25 +- drivers/net/bonding/rte_eth_bond.h | 238 +++++++++++++----- drivers/net/bonding/rte_eth_bond_8023ad.h | 76 ++++-- drivers/net/bonding/rte_eth_bond_alb.h | 34 ++- drivers/net/bonding/rte_eth_bond_api.c | 123 +++++++++ drivers/net/bonding/rte_eth_bond_args.c | 47 ++++ drivers/net/bonding/rte_eth_bond_flow.c | 1 + drivers/net/bonding/rte_eth_bond_flow.h | 22 ++ drivers/net/bonding/rte_eth_bond_pmd.c | 89 ++++++- drivers/net/bonding/version.map | 5 + lib/ethdev/ethdev_driver.h | 38 +++ 14 files changed, 767 insertions(+), 130 deletions(-) create mode 100644 drivers/net/bonding/rte_eth_bond_flow.h -- 2.39.1 ^ permalink raw reply [flat|nested] 80+ messages in thread
* [PATCH v2 1/8] ethdev: add member notification for bonding port 2023-10-07 1:34 ` [PATCH v2 0/8] Enhance the bond framework to support offload Chaoyong He @ 2023-10-07 1:34 ` Chaoyong He 2023-10-07 15:49 ` Stephen Hemminger 2023-10-07 1:34 ` [PATCH v2 2/8] ethdev: add API to get hardware creation of " Chaoyong He ` (7 subsequent siblings) 8 siblings, 1 reply; 80+ messages in thread From: Chaoyong He @ 2023-10-07 1:34 UTC (permalink / raw) To: dev; +Cc: oss-drivers, Long Wu, James Hershaw, Chaoyong He From: Long Wu <long.wu@corigine.com> Bonding PMD does not let member ports know the bonding port's information, like how many member ports the bonding port has, what mode the bonding port is in and so on. Add the notification interface for bonding port to let member port know it is added to a bonding port and what the bonding port's configuration is. If so the member ports have chance to achieve its bond-flow-offlod or other private bonding functions. Signed-off-by: Long Wu <long.wu@corigine.com> Reviewed-by: James Hershaw <james.hershaw@corigine.com> Reviewed-by: Chaoyong He <chaoyong.he@corigine.com> --- drivers/net/bonding/eth_bond_private.h | 2 + drivers/net/bonding/rte_eth_bond.h | 46 ++++++++++++++++ drivers/net/bonding/rte_eth_bond_api.c | 73 ++++++++++++++++++++++++++ drivers/net/bonding/rte_eth_bond_pmd.c | 27 ++++++++-- drivers/net/bonding/version.map | 3 ++ lib/ethdev/ethdev_driver.h | 18 +++++++ 6 files changed, 166 insertions(+), 3 deletions(-) diff --git a/drivers/net/bonding/eth_bond_private.h b/drivers/net/bonding/eth_bond_private.h index e688894210..1344f8c002 100644 --- a/drivers/net/bonding/eth_bond_private.h +++ b/drivers/net/bonding/eth_bond_private.h @@ -186,6 +186,8 @@ struct bond_dev_private { void *vlan_filter_bmpmem; /* enabled vlan filter bitmap */ struct rte_bitmap *vlan_filter_bmp; + + bool notify_member; /**< Enable member notification of bonding port. */ }; extern const struct eth_dev_ops default_dev_ops; diff --git a/drivers/net/bonding/rte_eth_bond.h b/drivers/net/bonding/rte_eth_bond.h index f10165f2c6..737beca446 100644 --- a/drivers/net/bonding/rte_eth_bond.h +++ b/drivers/net/bonding/rte_eth_bond.h @@ -351,6 +351,52 @@ rte_eth_bond_link_up_prop_delay_set(uint16_t bonding_port_id, int rte_eth_bond_link_up_prop_delay_get(uint16_t bonding_port_id); +/** + * Set the flag of whether bonding port notifies member ports. + * + * @param bonding_port_id + * Port ID of bonding device. + * @param notify_member + * Flag of whether bonding port notifies member ports. + * + * @return + * 0 on success, negative value otherwise. + */ +__rte_experimental +int +rte_eth_bond_notify_member_flag_set(uint16_t bonding_port_id, bool notify_member); + +/** + * Get the flag of whether bonding port notifies member ports. + * + * @param bonding_port_id + * Port ID of bonding device. + * @param notify_member + * Flag of whether bonding port notifies member ports. + * + * @return + * 0 on success, negative value otherwise. + */ +__rte_experimental +int +rte_eth_bond_notify_member_flag_get(uint16_t bonding_port_id, bool *notify_member); + +/** + * Notify the member ports of bonding port's information. + * + * This interface is called in the following functions: + * - bond_ethdev_lsc_event_callback() + * - bond_ethdev_configure() + * + * @param bonding_port_id + * Port ID of bonding device. + * + * @return + * 0 on success, negative value otherwise. + */ +__rte_experimental +int +rte_eth_bond_notify_members(uint16_t bonding_port_id); #ifdef __cplusplus } diff --git a/drivers/net/bonding/rte_eth_bond_api.c b/drivers/net/bonding/rte_eth_bond_api.c index 99e496556a..48c1d050fd 100644 --- a/drivers/net/bonding/rte_eth_bond_api.c +++ b/drivers/net/bonding/rte_eth_bond_api.c @@ -627,6 +627,17 @@ __eth_bond_member_add_lock_free(uint16_t bonding_port_id, uint16_t member_port_i member_vlan_filter_set(bonding_port_id, member_port_id); + if (internals->notify_member && + *member_eth_dev->dev_ops->bond_notify_member != NULL) { + ret = member_eth_dev->dev_ops->bond_notify_member(member_eth_dev, + bonding_eth_dev); + if (ret < 0) { + RTE_BOND_LOG(ERR, "Add member (port %u) notify failed!", + member_port_id); + return -1; + } + } + return 0; } @@ -733,6 +744,10 @@ __eth_bond_member_remove_lock_free(uint16_t bonding_port_id, member_eth_dev = &rte_eth_devices[member_port_id]; member_remove(internals, member_eth_dev); member_eth_dev->data->dev_flags &= (~RTE_ETH_DEV_BONDING_MEMBER); + if (internals->notify_member && + *member_eth_dev->dev_ops->bond_notify_member != NULL) + member_eth_dev->dev_ops->bond_notify_member(member_eth_dev, + bonding_eth_dev); /* first member in the active list will be the primary by default, * otherwise use first device in list */ @@ -1098,3 +1113,61 @@ rte_eth_bond_link_up_prop_delay_get(uint16_t bonding_port_id) return internals->link_up_delay_ms; } + +int +rte_eth_bond_notify_member_flag_set(uint16_t bonding_port_id, bool notify_member) +{ + struct bond_dev_private *internals; + + if (valid_bonding_port_id(bonding_port_id) != 0) + return -EINVAL; + + internals = rte_eth_devices[bonding_port_id].data->dev_private; + + internals->notify_member = notify_member; + + return 0; +} + +int +rte_eth_bond_notify_member_flag_get(uint16_t bonding_port_id, bool *notify_member) +{ + struct bond_dev_private *internals; + + if (valid_bonding_port_id(bonding_port_id) != 0) + return -EINVAL; + + internals = rte_eth_devices[bonding_port_id].data->dev_private; + + *notify_member = internals->notify_member; + + return 0; +} + +int +rte_eth_bond_notify_members(uint16_t bonding_port_id) +{ + uint32_t i; + uint16_t member_port_id; + struct rte_eth_dev *bond_dev; + struct bond_dev_private *internals; + struct rte_eth_dev *member_dev[RTE_MAX_ETHPORTS]; + + if (valid_bonding_port_id(bonding_port_id) != 0) + return -EINVAL; + + bond_dev = &rte_eth_devices[bonding_port_id]; + internals = bond_dev->data->dev_private; + + for (i = 0; i < internals->member_count; i++) { + member_port_id = internals->members[i].port_id; + member_dev[i] = &rte_eth_devices[member_port_id]; + if (*member_dev[i]->dev_ops->bond_notify_member == NULL) + return -ENOTSUP; + } + + for (i = 0; i < internals->member_count; i++) + member_dev[i]->dev_ops->bond_notify_member(member_dev[i], bond_dev); + + return 0; +} diff --git a/drivers/net/bonding/rte_eth_bond_pmd.c b/drivers/net/bonding/rte_eth_bond_pmd.c index 122b1187fd..b99b8b8938 100644 --- a/drivers/net/bonding/rte_eth_bond_pmd.c +++ b/drivers/net/bonding/rte_eth_bond_pmd.c @@ -2968,11 +2968,13 @@ bond_ethdev_lsc_event_callback(uint16_t port_id, enum rte_eth_event_type type, int valid_member = 0; uint16_t active_pos, member_idx; uint16_t i; + uint16_t bonding_port_id; if (type != RTE_ETH_EVENT_INTR_LSC || param == NULL) return rc; - bonding_eth_dev = &rte_eth_devices[*(uint16_t *)param]; + bonding_port_id = *(uint16_t *)param; + bonding_eth_dev = &rte_eth_devices[bonding_port_id]; if (check_for_bonding_ethdev(bonding_eth_dev)) return rc; @@ -3044,8 +3046,10 @@ bond_ethdev_lsc_event_callback(uint16_t port_id, enum rte_eth_event_type type, * using it. */ if (internals->user_defined_primary_port && - internals->primary_port == port_id) + internals->primary_port == port_id) { bond_ethdev_primary_set(internals, port_id); + rte_eth_bond_notify_members(bonding_port_id); + } } else { if (active_pos == internals->active_member_count) goto link_update; @@ -3064,6 +3068,7 @@ bond_ethdev_lsc_event_callback(uint16_t port_id, enum rte_eth_event_type type, internals->active_members[0]); else internals->current_primary_port = internals->primary_port; + rte_eth_bond_notify_members(bonding_port_id); mac_address_members_update(bonding_eth_dev); bond_ethdev_promiscuous_update(bonding_eth_dev); bond_ethdev_allmulticast_update(bonding_eth_dev); @@ -3362,6 +3367,7 @@ dump_basic(const struct rte_eth_dev *dev, FILE *f) struct bond_dev_private instant_priv; const struct bond_dev_private *internals = &instant_priv; int mode, i; + bool notify_member; /* Obtain a instance of dev_private to prevent data from being modified. */ memcpy(&instant_priv, dev->data->dev_private, sizeof(struct bond_dev_private)); @@ -3431,6 +3437,13 @@ dump_basic(const struct rte_eth_dev *dev, FILE *f) fprintf(f, "\tUser Defined Primary: [%u]\n", internals->primary_port); if (internals->member_count > 0) fprintf(f, "\tCurrent Primary: [%u]\n", internals->current_primary_port); + + if (rte_eth_bond_notify_member_flag_get(internals->port_id, ¬ify_member) == 0) + fprintf(f, "\tNotify Member Ports Flag: %s\n", + notify_member ? "enable" : "disable"); + else + fprintf(f, "\tFailed to get notify member ports flag for bonding port %d\n", + internals->port_id); } static void @@ -3983,8 +3996,12 @@ bond_ethdev_configure(struct rte_eth_dev *dev) * if no kvlist, it means that this bonding device has been created * through the bonding api. */ - if (!kvlist || internals->kvargs_processing_is_done) + if (!kvlist || internals->kvargs_processing_is_done) { + if (internals->notify_member && rte_eth_bond_notify_members(port_id) != 0) + RTE_BOND_LOG(ERR, "Notify member ports failed"); + return 0; + } internals->kvargs_processing_is_done = true; @@ -4222,6 +4239,10 @@ bond_ethdev_configure(struct rte_eth_dev *dev) return -1; } } + + if (internals->notify_member && rte_eth_bond_notify_members(port_id) != 0) + RTE_BOND_LOG(ERR, "Notify member ports failed"); + return 0; } diff --git a/drivers/net/bonding/version.map b/drivers/net/bonding/version.map index 09ee21c55f..3bd5e8ad11 100644 --- a/drivers/net/bonding/version.map +++ b/drivers/net/bonding/version.map @@ -35,4 +35,7 @@ EXPERIMENTAL { rte_eth_bond_member_add; rte_eth_bond_member_remove; rte_eth_bond_members_get; + rte_eth_bond_notify_member_flag_get; + rte_eth_bond_notify_member_flag_set; + rte_eth_bond_notify_members; }; diff --git a/lib/ethdev/ethdev_driver.h b/lib/ethdev/ethdev_driver.h index deb23ada18..f626f971e5 100644 --- a/lib/ethdev/ethdev_driver.h +++ b/lib/ethdev/ethdev_driver.h @@ -1216,6 +1216,21 @@ typedef int (*eth_count_aggr_ports_t)(struct rte_eth_dev *dev); typedef int (*eth_map_aggr_tx_affinity_t)(struct rte_eth_dev *dev, uint16_t tx_queue_id, uint8_t affinity); +/** + * @internal + * Bonding port notifies the member ports. + * + * @param dev + * Member port (ethdev) handle. + * @param bonding_dev + * Bonding port (ethdev) handle. + * + * @return + * Negative on error, 0 on success. + */ +typedef int (*eth_bond_notify_member)(struct rte_eth_dev *dev, + struct rte_eth_dev *bonding_dev); + /** * @internal A structure containing the functions exported by an Ethernet driver. */ @@ -1455,6 +1470,9 @@ struct eth_dev_ops { eth_count_aggr_ports_t count_aggr_ports; /** Map a Tx queue with an aggregated port of the DPDK port */ eth_map_aggr_tx_affinity_t map_aggr_tx_affinity; + + /** Notify the member port of bonding port information */ + eth_bond_notify_member bond_notify_member; }; /** -- 2.39.1 ^ permalink raw reply [flat|nested] 80+ messages in thread
* Re: [PATCH v2 1/8] ethdev: add member notification for bonding port 2023-10-07 1:34 ` [PATCH v2 1/8] ethdev: add member notification for bonding port Chaoyong He @ 2023-10-07 15:49 ` Stephen Hemminger 2023-10-08 1:37 ` Chaoyong He 0 siblings, 1 reply; 80+ messages in thread From: Stephen Hemminger @ 2023-10-07 15:49 UTC (permalink / raw) To: Chaoyong He; +Cc: dev, oss-drivers, Long Wu, James Hershaw On Sat, 7 Oct 2023 09:34:33 +0800 Chaoyong He <chaoyong.he@corigine.com> wrote: > diff --git a/drivers/net/bonding/eth_bond_private.h b/drivers/net/bonding/eth_bond_private.h > index e688894210..1344f8c002 100644 > --- a/drivers/net/bonding/eth_bond_private.h > +++ b/drivers/net/bonding/eth_bond_private.h > @@ -186,6 +186,8 @@ struct bond_dev_private { > > void *vlan_filter_bmpmem; /* enabled vlan filter bitmap */ > struct rte_bitmap *vlan_filter_bmp; > + > + bool notify_member; /**< Enable member notification of bonding port. */ > }; > > extern const struct eth_dev_ops default_dev_ops; There are holes in existing data structure, use one of them for your new flag. ^ permalink raw reply [flat|nested] 80+ messages in thread
* RE: [PATCH v2 1/8] ethdev: add member notification for bonding port 2023-10-07 15:49 ` Stephen Hemminger @ 2023-10-08 1:37 ` Chaoyong He 0 siblings, 0 replies; 80+ messages in thread From: Chaoyong He @ 2023-10-08 1:37 UTC (permalink / raw) To: Stephen Hemminger; +Cc: dev, oss-drivers, Long Wu, James Hershaw > Chaoyong He <chaoyong.he@corigine.com> wrote: > > > diff --git a/drivers/net/bonding/eth_bond_private.h > b/drivers/net/bonding/eth_bond_private.h > > index e688894210..1344f8c002 100644 > > --- a/drivers/net/bonding/eth_bond_private.h > > +++ b/drivers/net/bonding/eth_bond_private.h > > @@ -186,6 +186,8 @@ struct bond_dev_private { > > > > void *vlan_filter_bmpmem; /* enabled vlan filter bitmap > */ > > struct rte_bitmap *vlan_filter_bmp; > > + > > + bool notify_member; /**< Enable member notification of bonding > port. */ > > }; > > > > extern const struct eth_dev_ops default_dev_ops; > > There are holes in existing data structure, use one of them for your new flag. Okay, I will do that in the next version, thanks! ^ permalink raw reply [flat|nested] 80+ messages in thread
* [PATCH v2 2/8] ethdev: add API to get hardware creation of bonding port 2023-10-07 1:34 ` [PATCH v2 0/8] Enhance the bond framework to support offload Chaoyong He 2023-10-07 1:34 ` [PATCH v2 1/8] ethdev: add member notification for bonding port Chaoyong He @ 2023-10-07 1:34 ` Chaoyong He 2023-10-07 1:34 ` [PATCH v2 3/8] net/bonding: modify interface comment format Chaoyong He ` (6 subsequent siblings) 8 siblings, 0 replies; 80+ messages in thread From: Chaoyong He @ 2023-10-07 1:34 UTC (permalink / raw) To: dev; +Cc: oss-drivers, Long Wu, James Hershaw, Chaoyong He From: Long Wu <long.wu@corigine.com> After bonding port notification, member port hardware may create the bonding port. We want to get the result of creatition, so we add this API to do the getting action. Signed-off-by: Long Wu <long.wu@corigine.com> Reviewed-by: James Hershaw <james.hershaw@corigine.com> Reviewed-by: Chaoyong He <chaoyong.he@corigine.com> --- drivers/net/bonding/rte_eth_bond.h | 15 ++++++++++++++ drivers/net/bonding/rte_eth_bond_api.c | 28 ++++++++++++++++++++++++++ drivers/net/bonding/version.map | 1 + lib/ethdev/ethdev_driver.h | 20 ++++++++++++++++++ 4 files changed, 64 insertions(+) diff --git a/drivers/net/bonding/rte_eth_bond.h b/drivers/net/bonding/rte_eth_bond.h index 737beca446..6be5e46deb 100644 --- a/drivers/net/bonding/rte_eth_bond.h +++ b/drivers/net/bonding/rte_eth_bond.h @@ -398,6 +398,21 @@ __rte_experimental int rte_eth_bond_notify_members(uint16_t bonding_port_id); +/** + * Get the status of specified bonding port created by member port hardware. + * + * @param bonding_port_id + * Port ID of bonding device. + * @param member_port_id + * Port ID of member device. + * + * @return + * 0 on success, negative value otherwise. + */ +__rte_experimental +int +rte_eth_bond_hw_create_get(uint16_t bonding_port_id, uint16_t member_port_id); + #ifdef __cplusplus } #endif diff --git a/drivers/net/bonding/rte_eth_bond_api.c b/drivers/net/bonding/rte_eth_bond_api.c index 48c1d050fd..0be580b19b 100644 --- a/drivers/net/bonding/rte_eth_bond_api.c +++ b/drivers/net/bonding/rte_eth_bond_api.c @@ -1171,3 +1171,31 @@ rte_eth_bond_notify_members(uint16_t bonding_port_id) return 0; } + +int +rte_eth_bond_hw_create_get(uint16_t bonding_port_id, uint16_t member_port_id) +{ + uint32_t i; + struct rte_eth_dev *bonding_dev; + struct rte_eth_dev *member_dev; + struct bond_dev_private *internals; + + if (valid_bonding_port_id(bonding_port_id) != 0) + return -EINVAL; + + bonding_dev = &rte_eth_devices[bonding_port_id]; + internals = bonding_dev->data->dev_private; + for (i = 0; i < internals->member_count; i++) { + if (internals->members[i].port_id == member_port_id) + break; + } + + if (i == internals->member_count) + return -EINVAL; + + member_dev = &rte_eth_devices[member_port_id]; + if (*member_dev->dev_ops->bond_hw_create_get == NULL) + return -ENOTSUP; + + return member_dev->dev_ops->bond_hw_create_get(member_dev, bonding_dev); +} diff --git a/drivers/net/bonding/version.map b/drivers/net/bonding/version.map index 3bd5e8ad11..3cfff51269 100644 --- a/drivers/net/bonding/version.map +++ b/drivers/net/bonding/version.map @@ -32,6 +32,7 @@ EXPERIMENTAL { global: rte_eth_bond_8023ad_member_info; rte_eth_bond_active_members_get; + rte_eth_bond_hw_create_get; rte_eth_bond_member_add; rte_eth_bond_member_remove; rte_eth_bond_members_get; diff --git a/lib/ethdev/ethdev_driver.h b/lib/ethdev/ethdev_driver.h index f626f971e5..18ff5db969 100644 --- a/lib/ethdev/ethdev_driver.h +++ b/lib/ethdev/ethdev_driver.h @@ -1231,6 +1231,21 @@ typedef int (*eth_map_aggr_tx_affinity_t)(struct rte_eth_dev *dev, uint16_t tx_q typedef int (*eth_bond_notify_member)(struct rte_eth_dev *dev, struct rte_eth_dev *bonding_dev); +/** + * @internal + * Get the status of specified bonding port created by member port hardware. + * + * @param dev + * Member port (ethdev) handle. + * @param bonding_dev + * Bonding port (ethdev) handle. + * + * @return + * Negative on error, 0 on success. + */ +typedef int (*eth_bond_hw_create_get)(struct rte_eth_dev *dev, + struct rte_eth_dev *bonding_dev); + /** * @internal A structure containing the functions exported by an Ethernet driver. */ @@ -1473,6 +1488,11 @@ struct eth_dev_ops { /** Notify the member port of bonding port information */ eth_bond_notify_member bond_notify_member; + /** + * Get the status of whether bonding port is successfully created by + * the member port hardware. + */ + eth_bond_hw_create_get bond_hw_create_get; }; /** -- 2.39.1 ^ permalink raw reply [flat|nested] 80+ messages in thread
* [PATCH v2 3/8] net/bonding: modify interface comment format 2023-10-07 1:34 ` [PATCH v2 0/8] Enhance the bond framework to support offload Chaoyong He 2023-10-07 1:34 ` [PATCH v2 1/8] ethdev: add member notification for bonding port Chaoyong He 2023-10-07 1:34 ` [PATCH v2 2/8] ethdev: add API to get hardware creation of " Chaoyong He @ 2023-10-07 1:34 ` Chaoyong He 2023-10-07 1:34 ` [PATCH v2 4/8] net/bonding: add bonding port arguments Chaoyong He ` (5 subsequent siblings) 8 siblings, 0 replies; 80+ messages in thread From: Chaoyong He @ 2023-10-07 1:34 UTC (permalink / raw) To: dev; +Cc: oss-drivers, Long Wu, James Hershaw, Chaoyong He From: Long Wu <long.wu@corigine.com> Most of the previous interface comment format does not meet the current standards and were not uniform. Modify them to meet current standards. Signed-off-by: Long Wu <long.wu@corigine.com> Reviewed-by: James Hershaw <james.hershaw@corigine.com> Reviewed-by: Chaoyong He <chaoyong.he@corigine.com> --- drivers/net/bonding/eth_bond_8023ad_private.h | 52 +++--- drivers/net/bonding/rte_eth_bond.h | 150 +++++++++++------- drivers/net/bonding/rte_eth_bond_8023ad.h | 76 ++++++--- drivers/net/bonding/rte_eth_bond_alb.h | 34 ++-- 4 files changed, 199 insertions(+), 113 deletions(-) diff --git a/drivers/net/bonding/eth_bond_8023ad_private.h b/drivers/net/bonding/eth_bond_8023ad_private.h index ab7d15f81a..29ee8700c3 100644 --- a/drivers/net/bonding/eth_bond_8023ad_private.h +++ b/drivers/net/bonding/eth_bond_8023ad_private.h @@ -196,10 +196,13 @@ struct bond_dev_private; * * Set mode 4 configuration of bonding interface. * - * @pre Bonding interface must be stopped. + * @pre + * Bonding interface must be stopped. * - * @param dev Bonding interface - * @param conf new configuration. If NULL set default configuration. + * @param dev + * Bonding interface + * @param conf + * new configuration. If NULL set default configuration. */ void bond_mode_8023ad_setup(struct rte_eth_dev *dev, @@ -210,9 +213,11 @@ bond_mode_8023ad_setup(struct rte_eth_dev *dev, * * Enables 802.1AX mode and all active members on bonding interface. * - * @param dev Bonding interface + * @param dev + * Bonding interface + * * @return - * 0 on success, negative value otherwise. + * 0 on success, negative value otherwise. */ int bond_mode_8023ad_enable(struct rte_eth_dev *dev); @@ -222,7 +227,8 @@ bond_mode_8023ad_enable(struct rte_eth_dev *dev); * * Disables 802.1AX mode of the bonding interface and members. * - * @param dev Bonding interface + * @param dev + * Bonding interface * @return * 0 on success, negative value otherwise. */ @@ -232,7 +238,8 @@ int bond_mode_8023ad_disable(struct rte_eth_dev *dev); * @internal * * Starts 802.3AX state machines management logic. - * @param dev Bonding interface + * @param dev + * Bonding interface * @return * 0 if machines was started, 1 if machines was already running, * negative value otherwise. @@ -244,7 +251,8 @@ bond_mode_8023ad_start(struct rte_eth_dev *dev); * @internal * * Stops 802.3AX state machines management logic. - * @param dev Bonding interface + * @param dev + * Bonding interface * @return * 0 if this call stopped state machines, -ENOENT if alarm was not set. */ @@ -255,9 +263,12 @@ bond_mode_8023ad_stop(struct rte_eth_dev *dev); * @internal * * Passes given slow packet to state machines management logic. - * @param internals Bonding device private data. - * @param member_id Member port id. - * @param slot_pkt Slow packet. + * @param internals + * Bonding device private data. + * @param member_id + * Member port id. + * @param slot_pkt + * Slow packet. */ void bond_mode_8023ad_handle_slow_pkt(struct bond_dev_private *internals, @@ -268,11 +279,13 @@ bond_mode_8023ad_handle_slow_pkt(struct bond_dev_private *internals, * * Appends given member used member * - * @param dev Bonding interface. - * @param port_id Member port ID to be added + * @param dev + * Bonding interface. + * @param port_id + * Member port ID to be added * * @return - * 0 on success, negative value otherwise. + * 0 on success, negative value otherwise. */ void bond_mode_8023ad_activate_member(struct rte_eth_dev *dev, uint16_t port_id); @@ -282,18 +295,21 @@ bond_mode_8023ad_activate_member(struct rte_eth_dev *dev, uint16_t port_id); * * Denitializes and removes given member from 802.1AX mode. * - * @param dev Bonding interface. - * @param member_num Position of member in active_members array + * @param dev + * Bonding interface. + * @param member_num + * Position of member in active_members array * * @return - * 0 on success, negative value otherwise. + * 0 on success, negative value otherwise. */ int bond_mode_8023ad_deactivate_member(struct rte_eth_dev *dev, uint16_t member_pos); /** * Updates state when MAC was changed on bonding device or one of its members. - * @param bond_dev Bonding device + * @param bond_dev + * Bonding device */ void bond_mode_8023ad_mac_address_update(struct rte_eth_dev *bond_dev); diff --git a/drivers/net/bonding/rte_eth_bond.h b/drivers/net/bonding/rte_eth_bond.h index 6be5e46deb..28aa341d2f 100644 --- a/drivers/net/bonding/rte_eth_bond.h +++ b/drivers/net/bonding/rte_eth_bond.h @@ -95,12 +95,15 @@ extern "C" { /** * Create a bonding rte_eth_dev device * - * @param name Name of new link bonding device. - * @param mode Mode to initialize bonding device in. - * @param socket_id Socket Id on which to allocate eth_dev resources. + * @param name + * Name of new link bonding device. + * @param mode + * Mode to initialize bonding device in. + * @param socket_id + * Socket Id on which to allocate eth_dev resources. * * @return - * Port Id of created rte_eth_dev on success, negative value otherwise + * Port Id of created rte_eth_dev on success, negative value otherwise */ int rte_eth_bond_create(const char *name, uint8_t mode, uint8_t socket_id); @@ -108,10 +111,11 @@ rte_eth_bond_create(const char *name, uint8_t mode, uint8_t socket_id); /** * Free a bonding rte_eth_dev device * - * @param name Name of the link bonding device. + * @param name + * Name of the link bonding device. * * @return - * 0 on success, negative value otherwise + * 0 on success, negative value otherwise */ int rte_eth_bond_free(const char *name); @@ -119,11 +123,13 @@ rte_eth_bond_free(const char *name); /** * Add a rte_eth_dev device as a member to the bonding device * - * @param bonding_port_id Port ID of bonding device. - * @param member_port_id Port ID of member device. + * @param bonding_port_id + * Port ID of bonding device. + * @param member_port_id + * Port ID of member device. * * @return - * 0 on success, negative value otherwise + * 0 on success, negative value otherwise */ __rte_experimental int @@ -132,11 +138,13 @@ rte_eth_bond_member_add(uint16_t bonding_port_id, uint16_t member_port_id); /** * Remove a member rte_eth_dev device from the bonding device * - * @param bonding_port_id Port ID of bonding device. - * @param member_port_id Port ID of member device. + * @param bonding_port_id + * Port ID of bonding device. + * @param member_port_id + * Port ID of member device. * * @return - * 0 on success, negative value otherwise + * 0 on success, negative value otherwise */ __rte_experimental int @@ -145,11 +153,13 @@ rte_eth_bond_member_remove(uint16_t bonding_port_id, uint16_t member_port_id); /** * Set link bonding mode of bonding device * - * @param bonding_port_id Port ID of bonding device. - * @param mode Bonding mode to set + * @param bonding_port_id + * Port ID of bonding device. + * @param mode + * Bonding mode to set * * @return - * 0 on success, negative value otherwise + * 0 on success, negative value otherwise */ int rte_eth_bond_mode_set(uint16_t bonding_port_id, uint8_t mode); @@ -157,10 +167,11 @@ rte_eth_bond_mode_set(uint16_t bonding_port_id, uint8_t mode); /** * Get link bonding mode of bonding device * - * @param bonding_port_id Port ID of bonding device. + * @param bonding_port_id + * Port ID of bonding device. * * @return - * link bonding mode on success, negative value otherwise + * link bonding mode on success, negative value otherwise */ int rte_eth_bond_mode_get(uint16_t bonding_port_id); @@ -168,11 +179,13 @@ rte_eth_bond_mode_get(uint16_t bonding_port_id); /** * Set member rte_eth_dev as primary member of bonding device * - * @param bonding_port_id Port ID of bonding device. - * @param member_port_id Port ID of member device. + * @param bonding_port_id + * Port ID of bonding device. + * @param member_port_id + * Port ID of member device. * * @return - * 0 on success, negative value otherwise + * 0 on success, negative value otherwise */ int rte_eth_bond_primary_set(uint16_t bonding_port_id, uint16_t member_port_id); @@ -180,10 +193,11 @@ rte_eth_bond_primary_set(uint16_t bonding_port_id, uint16_t member_port_id); /** * Get primary member of bonding device * - * @param bonding_port_id Port ID of bonding device. + * @param bonding_port_id + * Port ID of bonding device. * * @return - * Port Id of primary member on success, -1 on failure + * Port Id of primary member on success, -1 on failure */ int rte_eth_bond_primary_get(uint16_t bonding_port_id); @@ -191,13 +205,16 @@ rte_eth_bond_primary_get(uint16_t bonding_port_id); /** * Populate an array with list of the members port id's of the bonding device * - * @param bonding_port_id Port ID of bonding eth_dev to interrogate - * @param members Array to be populated with the current active members - * @param len Length of members array + * @param bonding_port_id + * Port ID of bonding eth_dev to interrogate + * @param members + * Array to be populated with the current active members + * @param len + * Length of members array * * @return - * Number of members associated with bonding device on success, - * negative value otherwise + * Number of members associated with bonding device on success, + * negative value otherwise */ __rte_experimental int @@ -208,13 +225,16 @@ rte_eth_bond_members_get(uint16_t bonding_port_id, uint16_t members[], * Populate an array with list of the active members port id's of the bonding * device. * - * @param bonding_port_id Port ID of bonding eth_dev to interrogate - * @param members Array to be populated with the current active members - * @param len Length of members array + * @param bonding_port_id + * Port ID of bonding eth_dev to interrogate + * @param members + * Array to be populated with the current active members + * @param len + * Length of members array * * @return - * Number of active members associated with bonding device on success, - * negative value otherwise + * Number of active members associated with bonding device on success, + * negative value otherwise */ __rte_experimental int @@ -224,12 +244,13 @@ rte_eth_bond_active_members_get(uint16_t bonding_port_id, uint16_t members[], /** * Set explicit MAC address to use on bonding device and it's members. * - * @param bonding_port_id Port ID of bonding device. - * @param mac_addr MAC Address to use on bonding device overriding - * members MAC addresses + * @param bonding_port_id + * Port ID of bonding device. + * @param mac_addr + * MAC Address to use on bonding device overriding members MAC addresses * * @return - * 0 on success, negative value otherwise + * 0 on success, negative value otherwise */ int rte_eth_bond_mac_address_set(uint16_t bonding_port_id, @@ -239,10 +260,11 @@ rte_eth_bond_mac_address_set(uint16_t bonding_port_id, * Reset bonding device to use MAC from primary member on bonding device and it's * members. * - * @param bonding_port_id Port ID of bonding device. + * @param bonding_port_id + * Port ID of bonding device. * * @return - * 0 on success, negative value otherwise + * 0 on success, negative value otherwise */ int rte_eth_bond_mac_address_reset(uint16_t bonding_port_id); @@ -252,11 +274,13 @@ rte_eth_bond_mac_address_reset(uint16_t bonding_port_id); * balance mode, this parameter is otherwise ignored in other modes of * operation. * - * @param bonding_port_id Port ID of bonding device. - * @param policy Balance mode transmission policy. + * @param bonding_port_id + * Port ID of bonding device. + * @param policy + * Balance mode transmission policy. * * @return - * 0 on success, negative value otherwise. + * 0 on success, negative value otherwise. */ int rte_eth_bond_xmit_policy_set(uint16_t bonding_port_id, uint8_t policy); @@ -264,10 +288,11 @@ rte_eth_bond_xmit_policy_set(uint16_t bonding_port_id, uint8_t policy); /** * Get the transmit policy set on bonding device for balance mode operation * - * @param bonding_port_id Port ID of bonding device. + * @param bonding_port_id + * Port ID of bonding device. * * @return - * Balance transmit policy on success, negative value otherwise. + * Balance transmit policy on success, negative value otherwise. */ int rte_eth_bond_xmit_policy_get(uint16_t bonding_port_id); @@ -276,11 +301,13 @@ rte_eth_bond_xmit_policy_get(uint16_t bonding_port_id); * Set the link monitoring frequency (in ms) for monitoring the link status of * member devices * - * @param bonding_port_id Port ID of bonding device. - * @param internal_ms Monitoring interval in milliseconds + * @param bonding_port_id + * Port ID of bonding device. + * @param internal_ms + * Monitoring interval in milliseconds * * @return - * 0 on success, negative value otherwise. + * 0 on success, negative value otherwise. */ int @@ -290,10 +317,11 @@ rte_eth_bond_link_monitoring_set(uint16_t bonding_port_id, uint32_t internal_ms) * Get the current link monitoring frequency (in ms) for monitoring of the link * status of member devices * - * @param bonding_port_id Port ID of bonding device. + * @param bonding_port_id + * Port ID of bonding device. * * @return - * Monitoring interval on success, negative value otherwise. + * Monitoring interval on success, negative value otherwise. */ int rte_eth_bond_link_monitoring_get(uint16_t bonding_port_id); @@ -303,11 +331,13 @@ rte_eth_bond_link_monitoring_get(uint16_t bonding_port_id); * Set the period in milliseconds for delaying the disabling of a bonding link * when the link down status has been detected * - * @param bonding_port_id Port ID of bonding device. - * @param delay_ms Delay period in milliseconds. + * @param bonding_port_id + * Port ID of bonding device. + * @param delay_ms + * Delay period in milliseconds. * * @return - * 0 on success, negative value otherwise. + * 0 on success, negative value otherwise. */ int rte_eth_bond_link_down_prop_delay_set(uint16_t bonding_port_id, @@ -317,10 +347,11 @@ rte_eth_bond_link_down_prop_delay_set(uint16_t bonding_port_id, * Get the period in milliseconds set for delaying the disabling of a bonding * link when the link down status has been detected * - * @param bonding_port_id Port ID of bonding device. + * @param bonding_port_id + * Port ID of bonding device. * * @return - * Delay period on success, negative value otherwise. + * Delay period on success, negative value otherwise. */ int rte_eth_bond_link_down_prop_delay_get(uint16_t bonding_port_id); @@ -329,11 +360,13 @@ rte_eth_bond_link_down_prop_delay_get(uint16_t bonding_port_id); * Set the period in milliseconds for delaying the enabling of a bonding link * when the link up status has been detected * - * @param bonding_port_id Port ID of bonding device. - * @param delay_ms Delay period in milliseconds. + * @param bonding_port_id + * Port ID of bonding device. + * @param delay_ms + * Delay period in milliseconds. * * @return - * 0 on success, negative value otherwise. + * 0 on success, negative value otherwise. */ int rte_eth_bond_link_up_prop_delay_set(uint16_t bonding_port_id, @@ -343,10 +376,11 @@ rte_eth_bond_link_up_prop_delay_set(uint16_t bonding_port_id, * Get the period in milliseconds set for delaying the enabling of a bonding * link when the link up status has been detected * - * @param bonding_port_id Port ID of bonding device. + * @param bonding_port_id + * Port ID of bonding device. * * @return - * Delay period on success, negative value otherwise. + * Delay period on success, negative value otherwise. */ int rte_eth_bond_link_up_prop_delay_get(uint16_t bonding_port_id); diff --git a/drivers/net/bonding/rte_eth_bond_8023ad.h b/drivers/net/bonding/rte_eth_bond_8023ad.h index 4c280c7565..4936d7fc0c 100644 --- a/drivers/net/bonding/rte_eth_bond_8023ad.h +++ b/drivers/net/bonding/rte_eth_bond_8023ad.h @@ -155,8 +155,10 @@ struct rte_eth_bond_8023ad_member_info { * * Function returns current configuration of 802.3AX mode. * - * @param port_id Bonding device id - * @param conf Pointer to timeout structure. + * @param port_id + * Bonding device id + * @param conf + * Pointer to timeout structure. * * @return * 0 - if ok @@ -171,8 +173,10 @@ rte_eth_bond_8023ad_conf_get(uint16_t port_id, * * Function set new configuration of 802.3AX mode. * - * @param port_id Bonding device id - * @param conf Configuration, if NULL set default configuration. + * @param port_id + * Bonding device id + * @param conf + * Configuration, if NULL set default configuration. * @return * 0 - if ok * -EINVAL if configuration is invalid. @@ -186,8 +190,10 @@ rte_eth_bond_8023ad_setup(uint16_t port_id, * * Function returns current state of given member device. * - * @param member_id Port id of valid member. - * @param conf buffer for configuration + * @param member_id + * Port id of valid member. + * @param conf + * buffer for configuration * @return * 0 - if ok * -EINVAL if conf is NULL or member id is invalid (not a member of given @@ -201,9 +207,12 @@ rte_eth_bond_8023ad_member_info(uint16_t port_id, uint16_t member_id, /** * Configure a member port to start collecting. * - * @param port_id Bonding device id - * @param member_id Port id of valid member. - * @param enabled Non-zero when collection enabled. + * @param port_id + * Bonding device id + * @param member_id + * Port id of valid member. + * @param enabled + * Non-zero when collection enabled. * @return * 0 - if ok * -EINVAL if member is not valid. @@ -215,8 +224,10 @@ rte_eth_bond_8023ad_ext_collect(uint16_t port_id, uint16_t member_id, /** * Get COLLECTING flag from member port actor state. * - * @param port_id Bonding device id - * @param member_id Port id of valid member. + * @param port_id + * Bonding device id + * @param member_id + * Port id of valid member. * @return * 0 - if not set * 1 - if set @@ -228,9 +239,12 @@ rte_eth_bond_8023ad_ext_collect_get(uint16_t port_id, uint16_t member_id); /** * Configure a member port to start distributing. * - * @param port_id Bonding device id - * @param member_id Port id of valid member. - * @param enabled Non-zero when distribution enabled. + * @param port_id + * Bonding device id + * @param member_id + * Port id of valid member. + * @param enabled + * Non-zero when distribution enabled. * @return * 0 - if ok * -EINVAL if member is not valid. @@ -242,8 +256,10 @@ rte_eth_bond_8023ad_ext_distrib(uint16_t port_id, uint16_t member_id, /** * Get DISTRIBUTING flag from member port actor state. * - * @param port_id Bonding device id - * @param member_id Port id of valid member. + * @param port_id + * Bonding device id + * @param member_id + * Port id of valid member. * @return * 0 - if not set * 1 - if set @@ -256,9 +272,12 @@ rte_eth_bond_8023ad_ext_distrib_get(uint16_t port_id, uint16_t member_id); * LACPDU transmit path for external 802.3ad state machine. Caller retains * ownership of the packet on failure. * - * @param port_id Bonding device id - * @param member_id Port ID of valid member device. - * @param lacp_pkt mbuf containing LACPDU. + * @param port_id + * Bonding device id + * @param member_id + * Port ID of valid member device. + * @param lacp_pkt + * mbuf containing LACPDU. * * @return * 0 on success, negative value otherwise. @@ -284,7 +303,8 @@ rte_eth_bond_8023ad_ext_slowtx(uint16_t port_id, uint16_t member_id, * * Bonding port must be stopped to change this configuration. * - * @param port_id Bonding device id + * @param port_id + * Bonding device id * * @return * 0 on success, negative value otherwise. @@ -299,9 +319,11 @@ rte_eth_bond_8023ad_dedicated_queues_enable(uint16_t port_id); * * Bonding port must be stopped to change this configuration. * - * @see rte_eth_bond_8023ad_slow_pkt_hw_filter_enable + * @see + * rte_eth_bond_8023ad_slow_pkt_hw_filter_enable * - * @param port_id Bonding device id + * @param port_id + * Bonding device id * @return * 0 on success, negative value otherwise. * @@ -309,9 +331,11 @@ rte_eth_bond_8023ad_dedicated_queues_enable(uint16_t port_id); int rte_eth_bond_8023ad_dedicated_queues_disable(uint16_t port_id); -/* +/** * Get aggregator mode for 8023ad - * @param port_id Bonding device id + * + * @param port_id + * Bonding device id * * @return * aggregator mode on success, negative value otherwise @@ -321,7 +345,9 @@ rte_eth_bond_8023ad_agg_selection_get(uint16_t port_id); /** * Set aggregator mode for 8023ad - * @param port_id Bonding device id + * @param port_id + * Bonding device id + * * @return * 0 on success, negative value otherwise */ diff --git a/drivers/net/bonding/rte_eth_bond_alb.h b/drivers/net/bonding/rte_eth_bond_alb.h index f51c355d36..3cf0d87750 100644 --- a/drivers/net/bonding/rte_eth_bond_alb.h +++ b/drivers/net/bonding/rte_eth_bond_alb.h @@ -63,9 +63,12 @@ bond_mode_alb_enable(struct rte_eth_dev *bond_dev); * forwarded to application without changes. If it is ARP reply, client table * is updated. * - * @param eth_h ETH header of received packet. - * @param offset Vlan header offset. - * @param internals Bonding data. + * @param eth_h + * ETH header of received packet. + * @param offset + * Vlan header offset. + * @param internals + * Bonding data. */ void bond_mode_alb_arp_recv(struct rte_ether_hdr *eth_h, uint16_t offset, @@ -77,12 +80,15 @@ bond_mode_alb_arp_recv(struct rte_ether_hdr *eth_h, uint16_t offset, * If it is ARP Reply, it is send on member stored in client table for that * connection. On Reply function also updates data in client table. * - * @param eth_h ETH header of transmitted packet. - * @param offset Vlan header offset. - * @param internals Bonding data. + * @param eth_h + * ETH header of transmitted packet. + * @param offset + * Vlan header offset. + * @param internals + * Bonding data. * * @return - * Index of member on which packet should be sent. + * Index of member on which packet should be sent. */ uint16_t bond_mode_alb_arp_xmit(struct rte_ether_hdr *eth_h, uint16_t offset, @@ -91,12 +97,15 @@ bond_mode_alb_arp_xmit(struct rte_ether_hdr *eth_h, uint16_t offset, /** * Function fills packet with ARP data from client_info. * - * @param client_info Data of client to which packet is sent. - * @param pkt Pointer to packet which is sent. - * @param internals Bonding data. + * @param client_info + * Data of client to which packet is sent. + * @param pkt + * Pointer to packet which is sent. + * @param internals + * Bonding data. * * @return - * Index of member on which packet should be sent. + * Index of member on which packet should be sent. */ uint16_t bond_mode_alb_arp_upd(struct client_data *client_info, @@ -105,7 +114,8 @@ bond_mode_alb_arp_upd(struct client_data *client_info, /** * Function updates member indexes of active connections. * - * @param bond_dev Pointer to bonding device struct. + * @param bond_dev + * Pointer to bonding device struct. */ void bond_mode_alb_client_list_upd(struct rte_eth_dev *bond_dev); -- 2.39.1 ^ permalink raw reply [flat|nested] 80+ messages in thread
* [PATCH v2 4/8] net/bonding: add bonding port arguments 2023-10-07 1:34 ` [PATCH v2 0/8] Enhance the bond framework to support offload Chaoyong He ` (2 preceding siblings ...) 2023-10-07 1:34 ` [PATCH v2 3/8] net/bonding: modify interface comment format Chaoyong He @ 2023-10-07 1:34 ` Chaoyong He 2023-10-07 1:34 ` [PATCH v2 5/8] net/bonding: support add port by data name Chaoyong He ` (4 subsequent siblings) 8 siblings, 0 replies; 80+ messages in thread From: Chaoyong He @ 2023-10-07 1:34 UTC (permalink / raw) To: dev; +Cc: oss-drivers, Long Wu, James Hershaw, Chaoyong He From: Long Wu <long.wu@corigine.com> Include the following new arguments for bonding ports: - "notify_member" to enable/disable member notification. - "dedicated_queue" to enable/disable dedicated queue. Add these two arguments in initial argument. Signed-off-by: Long Wu <long.wu@corigine.com> Reviewed-by: James Hershaw <james.hershaw@corigine.com> Reviewed-by: Chaoyong He <chaoyong.he@corigine.com> --- drivers/net/bonding/eth_bond_private.h | 10 ++++ drivers/net/bonding/rte_eth_bond.h | 14 ++++++ drivers/net/bonding/rte_eth_bond_api.c | 14 ++++++ drivers/net/bonding/rte_eth_bond_args.c | 44 ++++++++++++++++++ drivers/net/bonding/rte_eth_bond_pmd.c | 61 ++++++++++++++++++++++++- 5 files changed, 142 insertions(+), 1 deletion(-) diff --git a/drivers/net/bonding/eth_bond_private.h b/drivers/net/bonding/eth_bond_private.h index 1344f8c002..b217d98c31 100644 --- a/drivers/net/bonding/eth_bond_private.h +++ b/drivers/net/bonding/eth_bond_private.h @@ -28,6 +28,8 @@ #define PMD_BOND_LSC_POLL_PERIOD_KVARG ("lsc_poll_period_ms") #define PMD_BOND_LINK_UP_PROP_DELAY_KVARG ("up_delay") #define PMD_BOND_LINK_DOWN_PROP_DELAY_KVARG ("down_delay") +#define PMD_BOND_NOTIFY_MEMBER_KVARG ("notify_member") +#define PMD_BOND_DEDICATED_QUEUE_KVARG ("dedicated_queue") #define PMD_BOND_XMIT_POLICY_LAYER2_KVARG ("l2") #define PMD_BOND_XMIT_POLICY_LAYER23_KVARG ("l23") @@ -320,6 +322,14 @@ int bond_ethdev_parse_time_ms_kvarg(const char *key, const char *value, void *extra_args); +int +bond_ethdev_parse_notify_member_kvarg(const char *key __rte_unused, + const char *value, void *extra_args); + +int +bond_ethdev_parse_dedicated_queue_kvarg(const char *key __rte_unused, + const char *value, void *extra_args); + void bond_tlb_disable(struct bond_dev_private *internals); diff --git a/drivers/net/bonding/rte_eth_bond.h b/drivers/net/bonding/rte_eth_bond.h index 28aa341d2f..3f427b6bab 100644 --- a/drivers/net/bonding/rte_eth_bond.h +++ b/drivers/net/bonding/rte_eth_bond.h @@ -385,6 +385,20 @@ rte_eth_bond_link_up_prop_delay_set(uint16_t bonding_port_id, int rte_eth_bond_link_up_prop_delay_get(uint16_t bonding_port_id); +/** + * Set the flag that whether bonding device enable dedicated queue. + * + * @param bonding_port_id + * Port ID of bonding device. + * @param queue_flag + * The flag of enable bond dedicated queue + * + * @return + * 0 on success, negative value otherwise. + */ +int +rte_eth_bond_dedicated_queue_flag_set(uint16_t bonding_port_id, bool queue_flag); + /** * Set the flag of whether bonding port notifies member ports. * diff --git a/drivers/net/bonding/rte_eth_bond_api.c b/drivers/net/bonding/rte_eth_bond_api.c index 0be580b19b..a042f05a4c 100644 --- a/drivers/net/bonding/rte_eth_bond_api.c +++ b/drivers/net/bonding/rte_eth_bond_api.c @@ -1114,6 +1114,20 @@ rte_eth_bond_link_up_prop_delay_get(uint16_t bonding_port_id) return internals->link_up_delay_ms; } +int +rte_eth_bond_dedicated_queue_flag_set(uint16_t bonding_port_id, bool queue_flag) +{ + struct bond_dev_private *internals; + + if (valid_bonding_port_id(bonding_port_id) != 0) + return -1; + + internals = rte_eth_devices[bonding_port_id].data->dev_private; + internals->mode4.dedicated_queues.enabled = queue_flag; + + return 0; +} + int rte_eth_bond_notify_member_flag_set(uint16_t bonding_port_id, bool notify_member) { diff --git a/drivers/net/bonding/rte_eth_bond_args.c b/drivers/net/bonding/rte_eth_bond_args.c index bdec5d61d4..8a3e4656ef 100644 --- a/drivers/net/bonding/rte_eth_bond_args.c +++ b/drivers/net/bonding/rte_eth_bond_args.c @@ -20,6 +20,8 @@ const char *pmd_bond_init_valid_arguments[] = { PMD_BOND_MAC_ADDR_KVARG, PMD_BOND_AGG_MODE_KVARG, RTE_DEVARGS_KEY_DRIVER, + PMD_BOND_NOTIFY_MEMBER_KVARG, + PMD_BOND_DEDICATED_QUEUE_KVARG, NULL }; @@ -297,3 +299,45 @@ bond_ethdev_parse_time_ms_kvarg(const char *key __rte_unused, return 0; } + +int +bond_ethdev_parse_notify_member_kvarg(const char *key __rte_unused, + const char *value, void *extra_args) +{ + bool *notify_member; + + if (value == NULL || extra_args == NULL) + return -1; + + notify_member = extra_args; + + if (strcmp("enable", value) == 0) + *notify_member = true; + else if (strcmp("disable", value) == 0) + *notify_member = false; + else + return -1; + + return 0; +} + +int +bond_ethdev_parse_dedicated_queue_kvarg(const char *key __rte_unused, + const char *value, void *extra_args) +{ + bool *dedicated_queue; + + if (value == NULL || extra_args == NULL) + return -1; + + dedicated_queue = extra_args; + + if (strcmp("enable", value) == 0) + *dedicated_queue = true; + else if (strcmp("disable", value) == 0) + *dedicated_queue = false; + else + return -1; + + return 0; +} diff --git a/drivers/net/bonding/rte_eth_bond_pmd.c b/drivers/net/bonding/rte_eth_bond_pmd.c index b99b8b8938..1ebeb270c8 100644 --- a/drivers/net/bonding/rte_eth_bond_pmd.c +++ b/drivers/net/bonding/rte_eth_bond_pmd.c @@ -4227,6 +4227,63 @@ bond_ethdev_configure(struct rte_eth_dev *dev) return -1; } + /* Parse/set notify member flag */ + arg_count = rte_kvargs_count(kvlist, PMD_BOND_NOTIFY_MEMBER_KVARG); + if (arg_count == 1) { + bool notify_member; + if (rte_kvargs_process(kvlist, + PMD_BOND_NOTIFY_MEMBER_KVARG, + &bond_ethdev_parse_notify_member_kvarg, + ¬ify_member) < 0) { + RTE_BOND_LOG(ERR, + "Invalid notify member value specified" + " for bonding device %s", name); + return -1; + } + + if (rte_eth_bond_notify_member_flag_set(port_id, notify_member) != 0) { + RTE_BOND_LOG(ERR, + "Failed to set notify member (%u) on" + " bonding device %s", notify_member, name); + return -1; + } + } else if (arg_count > 1) { + RTE_BOND_LOG(ERR, + "notify member flag can be specified only once" + " for bonding device %s", name); + return -1; + } + + /* Parse/set dedicated queue flag */ + arg_count = rte_kvargs_count(kvlist, PMD_BOND_DEDICATED_QUEUE_KVARG); + if (arg_count == 1) { + bool dedicated_queue; + if (rte_kvargs_process(kvlist, + PMD_BOND_DEDICATED_QUEUE_KVARG, + &bond_ethdev_parse_dedicated_queue_kvarg, + &dedicated_queue) < 0) { + RTE_BOND_LOG(ERR, + "Invalid dedicated queue flag specified" + " for bonding device %s", name); + return -1; + } + + if (internals->mode == BONDING_MODE_8023AD) { + if (rte_eth_bond_dedicated_queue_flag_set(port_id, dedicated_queue) != 0) { + RTE_BOND_LOG(ERR, + "Failed to enable/disable dedicated" + " queue flag on bonding device %s", + name); + return -1; + } + } + } else if (arg_count > 1) { + RTE_BOND_LOG(ERR, + "dedicated queue flag can be specified only once" + " for bonding device %s", name); + return -1; + } + /* configure members so we can pass mtu setting */ for (i = 0; i < internals->member_count; i++) { struct rte_eth_dev *member_ethdev = @@ -4264,7 +4321,9 @@ RTE_PMD_REGISTER_PARAM_STRING(net_bonding, "mac=<mac addr> " "lsc_poll_period_ms=<int> " "up_delay=<int> " - "down_delay=<int>"); + "down_delay=<int>" + "notify_member=[enable | disable] " + "dedicated_queue=[enable | disable] "); /* We can't use RTE_LOG_REGISTER_DEFAULT because of the forced name for * this library, see meson.build. -- 2.39.1 ^ permalink raw reply [flat|nested] 80+ messages in thread
* [PATCH v2 5/8] net/bonding: support add port by data name 2023-10-07 1:34 ` [PATCH v2 0/8] Enhance the bond framework to support offload Chaoyong He ` (3 preceding siblings ...) 2023-10-07 1:34 ` [PATCH v2 4/8] net/bonding: add bonding port arguments Chaoyong He @ 2023-10-07 1:34 ` Chaoyong He 2023-10-07 1:34 ` [PATCH v2 6/8] net/bonding: create new rte flow header file Chaoyong He ` (3 subsequent siblings) 8 siblings, 0 replies; 80+ messages in thread From: Chaoyong He @ 2023-10-07 1:34 UTC (permalink / raw) To: dev; +Cc: oss-drivers, Long Wu, James Hershaw, Chaoyong He From: Long Wu <long.wu@corigine.com> Several ports may share the same PCI address, like nfp representor. So we cannot add this type of ports to bonding port by "--vdev" argument in dpdk-testpmd. But the port's data name is unique between them, we include an option to add such ports to the bonding port. After adding this feature, we can create a bonding port that member port is this type of port by "--vdev" in dpdk-testpmd start command. For example: dpdk-testpmd -l 2-10 -s 0x8 -a ca:00.0,representor=[0-2] --vdev 'net_bonding0,member=flower_repr_p0,member=flower_repr_p1, mode=4,socket_id=1,xmit_policy=l34' -- -i Note: 1. "ca:00.0" is nfp 4000 card. 2. "flower_repr_p0" and "flower_repr_p1" are nfp phy representor's data name. Signed-off-by: Long Wu <long.wu@corigine.com> Reviewed-by: James Hershaw <james.hershaw@corigine.com> Reviewed-by: Chaoyong He <chaoyong.he@corigine.com> --- drivers/net/bonding/rte_eth_bond_args.c | 3 +++ 1 file changed, 3 insertions(+) diff --git a/drivers/net/bonding/rte_eth_bond_args.c b/drivers/net/bonding/rte_eth_bond_args.c index 8a3e4656ef..b320eb3038 100644 --- a/drivers/net/bonding/rte_eth_bond_args.c +++ b/drivers/net/bonding/rte_eth_bond_args.c @@ -70,6 +70,9 @@ find_port_id_by_dev_name(const char *name) if (strcmp(rte_eth_devices[i].device->name, name) == 0) return i; + + if (strcmp(rte_eth_devices[i].data->name, name) == 0) + return i; } return -1; } -- 2.39.1 ^ permalink raw reply [flat|nested] 80+ messages in thread
* [PATCH v2 6/8] net/bonding: create new rte flow header file 2023-10-07 1:34 ` [PATCH v2 0/8] Enhance the bond framework to support offload Chaoyong He ` (4 preceding siblings ...) 2023-10-07 1:34 ` [PATCH v2 5/8] net/bonding: support add port by data name Chaoyong He @ 2023-10-07 1:34 ` Chaoyong He 2023-10-07 1:34 ` [PATCH v2 7/8] net/bonding: support checking valid bonding port ID Chaoyong He ` (2 subsequent siblings) 8 siblings, 0 replies; 80+ messages in thread From: Chaoyong He @ 2023-10-07 1:34 UTC (permalink / raw) To: dev; +Cc: oss-drivers, Long Wu, James Hershaw, Chaoyong He From: Long Wu <long.wu@corigine.com> Move the flow code to a new head file to make flow related code more clean and make the code architecture more reasonable in the future. There is no functional change, just moving verbatim code around. Signed-off-by: Long Wu <long.wu@corigine.com> Reviewed-by: James Hershaw <james.hershaw@corigine.com> Reviewed-by: Chaoyong He <chaoyong.he@corigine.com> --- drivers/net/bonding/eth_bond_private.h | 13 ------------- drivers/net/bonding/rte_eth_bond_api.c | 1 + drivers/net/bonding/rte_eth_bond_flow.c | 1 + drivers/net/bonding/rte_eth_bond_flow.h | 22 ++++++++++++++++++++++ drivers/net/bonding/rte_eth_bond_pmd.c | 1 + 5 files changed, 25 insertions(+), 13 deletions(-) create mode 100644 drivers/net/bonding/rte_eth_bond_flow.h diff --git a/drivers/net/bonding/eth_bond_private.h b/drivers/net/bonding/eth_bond_private.h index b217d98c31..aaab521eb9 100644 --- a/drivers/net/bonding/eth_bond_private.h +++ b/drivers/net/bonding/eth_bond_private.h @@ -9,10 +9,8 @@ #include <sys/queue.h> #include <ethdev_driver.h> -#include <rte_flow.h> #include <rte_spinlock.h> #include <rte_bitmap.h> -#include <rte_flow_driver.h> #include "rte_eth_bond.h" #include "eth_bond_8023ad_private.h" @@ -47,8 +45,6 @@ extern const char *pmd_bond_init_valid_arguments[]; extern struct rte_vdev_driver pmd_bond_drv; -extern const struct rte_flow_ops bond_flow_ops; - /** Port Queue Mapping Structure */ struct bond_rx_queue { uint16_t queue_id; @@ -94,15 +90,6 @@ struct bond_member_details { uint16_t reta_size; }; -struct rte_flow { - TAILQ_ENTRY(rte_flow) next; - /* Members flows */ - struct rte_flow *flows[RTE_MAX_ETHPORTS]; - /* Flow description for synchronization */ - struct rte_flow_conv_rule rule; - uint8_t rule_data[]; -}; - typedef void (*burst_xmit_hash_t)(struct rte_mbuf **buf, uint16_t nb_pkts, uint16_t member_count, uint16_t *members); diff --git a/drivers/net/bonding/rte_eth_bond_api.c b/drivers/net/bonding/rte_eth_bond_api.c index a042f05a4c..0113dfdc16 100644 --- a/drivers/net/bonding/rte_eth_bond_api.c +++ b/drivers/net/bonding/rte_eth_bond_api.c @@ -12,6 +12,7 @@ #include <rte_kvargs.h> #include "rte_eth_bond.h" +#include "rte_eth_bond_flow.h" #include "eth_bond_private.h" #include "eth_bond_8023ad_private.h" diff --git a/drivers/net/bonding/rte_eth_bond_flow.c b/drivers/net/bonding/rte_eth_bond_flow.c index 71a91675f7..e6c7ce5362 100644 --- a/drivers/net/bonding/rte_eth_bond_flow.c +++ b/drivers/net/bonding/rte_eth_bond_flow.c @@ -1,6 +1,7 @@ /* SPDX-License-Identifier: BSD-3-Clause * Copyright 2018 Mellanox Technologies, Ltd */ +#include "rte_eth_bond_flow.h" #include <stddef.h> #include <string.h> diff --git a/drivers/net/bonding/rte_eth_bond_flow.h b/drivers/net/bonding/rte_eth_bond_flow.h new file mode 100644 index 0000000000..7394e0e2e1 --- /dev/null +++ b/drivers/net/bonding/rte_eth_bond_flow.h @@ -0,0 +1,22 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright (c) 2023 Corigine, Inc. + */ + +#ifndef _RTE_ETH_BOND_FLOW_H_ +#define _RTE_ETH_BOND_FLOW_H_ + +#include <rte_flow.h> +#include <rte_flow_driver.h> + +extern const struct rte_flow_ops bond_flow_ops; + +struct rte_flow { + TAILQ_ENTRY(rte_flow) next; + struct rte_flow *flows[RTE_MAX_ETHPORTS]; + /**< Member ports flows */ + struct rte_flow_conv_rule rule; + /**< Flow description for synchronization */ + uint8_t rule_data[]; +}; + +#endif /* _RTE_ETH_BOND_FLOW_H_ */ diff --git a/drivers/net/bonding/rte_eth_bond_pmd.c b/drivers/net/bonding/rte_eth_bond_pmd.c index 1ebeb270c8..630afc3740 100644 --- a/drivers/net/bonding/rte_eth_bond_pmd.c +++ b/drivers/net/bonding/rte_eth_bond_pmd.c @@ -21,6 +21,7 @@ #include <rte_string_fns.h> #include "rte_eth_bond.h" +#include "rte_eth_bond_flow.h" #include "eth_bond_private.h" #include "eth_bond_8023ad_private.h" -- 2.39.1 ^ permalink raw reply [flat|nested] 80+ messages in thread
* [PATCH v2 7/8] net/bonding: support checking valid bonding port ID 2023-10-07 1:34 ` [PATCH v2 0/8] Enhance the bond framework to support offload Chaoyong He ` (5 preceding siblings ...) 2023-10-07 1:34 ` [PATCH v2 6/8] net/bonding: create new rte flow header file Chaoyong He @ 2023-10-07 1:34 ` Chaoyong He 2023-10-07 1:34 ` [PATCH v2 8/8] net/bonding: add commands for bonding port notification Chaoyong He 2023-10-08 1:50 ` [PATCH v3 0/8] Enhance the bond framework to support offload Chaoyong He 8 siblings, 0 replies; 80+ messages in thread From: Chaoyong He @ 2023-10-07 1:34 UTC (permalink / raw) To: dev; +Cc: oss-drivers, Long Wu, James Hershaw, Chaoyong He From: Long Wu <long.wu@corigine.com> Add API to support checking if the port id is a bonding port id. Signed-off-by: Long Wu <long.wu@corigine.com> Reviewed-by: James Hershaw <james.hershaw@corigine.com> Reviewed-by: Chaoyong He <chaoyong.he@corigine.com> --- drivers/net/bonding/rte_eth_bond.h | 13 +++++++++++++ drivers/net/bonding/rte_eth_bond_api.c | 7 +++++++ drivers/net/bonding/version.map | 1 + 3 files changed, 21 insertions(+) diff --git a/drivers/net/bonding/rte_eth_bond.h b/drivers/net/bonding/rte_eth_bond.h index 3f427b6bab..e8152a155f 100644 --- a/drivers/net/bonding/rte_eth_bond.h +++ b/drivers/net/bonding/rte_eth_bond.h @@ -461,6 +461,19 @@ __rte_experimental int rte_eth_bond_hw_create_get(uint16_t bonding_port_id, uint16_t member_port_id); +/** + * Check whether bonding port id is valid. + * + * @param bonding_port_id + * Port ID of bonding device. + * + * @return + * 0 on success, negative value otherwise. + */ +__rte_experimental +int +rte_eth_bond_valid_bonding_port_id(uint16_t bonding_port_id); + #ifdef __cplusplus } #endif diff --git a/drivers/net/bonding/rte_eth_bond_api.c b/drivers/net/bonding/rte_eth_bond_api.c index 0113dfdc16..80d71529cc 100644 --- a/drivers/net/bonding/rte_eth_bond_api.c +++ b/drivers/net/bonding/rte_eth_bond_api.c @@ -1214,3 +1214,10 @@ rte_eth_bond_hw_create_get(uint16_t bonding_port_id, uint16_t member_port_id) return member_dev->dev_ops->bond_hw_create_get(member_dev, bonding_dev); } + + +int +rte_eth_bond_valid_bonding_port_id(uint16_t port_id) +{ + return valid_bonding_port_id(port_id); +} diff --git a/drivers/net/bonding/version.map b/drivers/net/bonding/version.map index 3cfff51269..bf5e50521e 100644 --- a/drivers/net/bonding/version.map +++ b/drivers/net/bonding/version.map @@ -39,4 +39,5 @@ EXPERIMENTAL { rte_eth_bond_notify_member_flag_get; rte_eth_bond_notify_member_flag_set; rte_eth_bond_notify_members; + rte_eth_bond_valid_bonding_port_id; }; -- 2.39.1 ^ permalink raw reply [flat|nested] 80+ messages in thread
* [PATCH v2 8/8] net/bonding: add commands for bonding port notification 2023-10-07 1:34 ` [PATCH v2 0/8] Enhance the bond framework to support offload Chaoyong He ` (6 preceding siblings ...) 2023-10-07 1:34 ` [PATCH v2 7/8] net/bonding: support checking valid bonding port ID Chaoyong He @ 2023-10-07 1:34 ` Chaoyong He 2023-10-08 1:50 ` [PATCH v3 0/8] Enhance the bond framework to support offload Chaoyong He 8 siblings, 0 replies; 80+ messages in thread From: Chaoyong He @ 2023-10-07 1:34 UTC (permalink / raw) To: dev; +Cc: oss-drivers, Long Wu, James Hershaw, Chaoyong He From: Long Wu <long.wu@corigine.com> Add some commands to support bonding port notification in dpdk-testpmd. 1. We can enable the notification by command: "set bonding notify_member (port_id) (enable|disable)" 2. If member port hardware try to create the bonding port after notification we can get the status by command: "get bonding member hardware create (member_port_id) (bonding_port_id)" Signed-off-by: Long Wu <long.wu@corigine.com> Reviewed-by: James Hershaw <james.hershaw@corigine.com> Reviewed-by: Chaoyong He <chaoyong.he@corigine.com> --- .../link_bonding_poll_mode_drv_lib.rst | 19 +++ drivers/net/bonding/bonding_testpmd.c | 128 ++++++++++++++++++ 2 files changed, 147 insertions(+) diff --git a/doc/guides/prog_guide/link_bonding_poll_mode_drv_lib.rst b/doc/guides/prog_guide/link_bonding_poll_mode_drv_lib.rst index 60717a3587..9f6443ebd8 100644 --- a/doc/guides/prog_guide/link_bonding_poll_mode_drv_lib.rst +++ b/doc/guides/prog_guide/link_bonding_poll_mode_drv_lib.rst @@ -637,3 +637,22 @@ in balance mode with a transmission policy of layer 2+3:: Members (3): [1 3 4] Active Members (3): [1 3 4] Primary: [3] + +set bonding notify_member +~~~~~~~~~~~~~~~~~~~~~~~~~ + +Set the notify member flag of bonding port:: + + testpmd> set bonding notify_member (port_id) (enable|disable) + +This command just set the flag of notification. +If we enable it, bonding PMD will notify member ports when its some +configurations changed. So member ports can do some private things, maybe hardware +bonding creation and etc. + +get bonding member hardware create +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +Get the status of member port hardware creating the bonding port:: + + testpmd> get bonding member hardware create (member_port_id) (bonding_port_id) diff --git a/drivers/net/bonding/bonding_testpmd.c b/drivers/net/bonding/bonding_testpmd.c index 8fcd6cadd0..da7d9cc58f 100644 --- a/drivers/net/bonding/bonding_testpmd.c +++ b/drivers/net/bonding/bonding_testpmd.c @@ -692,6 +692,124 @@ static cmdline_parse_inst_t cmd_set_bonding_agg_mode_policy = { } }; +struct cmd_set_bonding_notify_member_result { + cmdline_fixed_string_t set; + cmdline_fixed_string_t bonding; + cmdline_fixed_string_t notify_member; + uint16_t port_num; + cmdline_fixed_string_t mode; +}; + +static void +cmd_set_bonding_notify_member_parsed(void *parsed_result, + __rte_unused struct cmdline *cl, __rte_unused void *data) +{ + struct cmd_set_bonding_notify_member_result *res = parsed_result; + bool notify_member = false; + + if (strcmp(res->notify_member, "enable") == 0) + notify_member = true; + else if (strcmp(res->notify_member, "disable") == 0) + notify_member = false; + + rte_eth_bond_notify_member_flag_set(res->port_num, notify_member); +} + +static cmdline_parse_token_string_t cmd_set_bonding_notify_member_set = + TOKEN_STRING_INITIALIZER(struct cmd_set_bonding_notify_member_result, + set, "set"); +static cmdline_parse_token_string_t cmd_set_bonding_notify_member_bonding = + TOKEN_STRING_INITIALIZER(struct cmd_set_bonding_notify_member_result, + bonding, "bonding"); +static cmdline_parse_token_string_t cmd_set_bonding_notify_member = + TOKEN_STRING_INITIALIZER(struct cmd_set_bonding_notify_member_result, + notify_member, "notify_member"); +static cmdline_parse_token_num_t cmd_set_bonding_notify_member_portnum = + TOKEN_NUM_INITIALIZER(struct cmd_set_bonding_notify_member_result, + port_num, RTE_UINT16); +static cmdline_parse_token_string_t cmd_set_bonding_notify_member_mode_string = + TOKEN_STRING_INITIALIZER(struct cmd_set_bonding_notify_member_result, + mode, "enable#disable"); + +static cmdline_parse_inst_t cmd_set_bonding_notify_member_ports = { + .f = cmd_set_bonding_notify_member_parsed, + .data = NULL, + .help_str = "set bonding notify_member (port_id) (enable|disable)", + .tokens = { + (void *)&cmd_set_bonding_notify_member_set, + (void *)&cmd_set_bonding_notify_member_bonding, + (void *)&cmd_set_bonding_notify_member, + (void *)&cmd_set_bonding_notify_member_portnum, + (void *)&cmd_set_bonding_notify_member_mode_string, + NULL + } +}; + +struct cmd_get_bonding_member_hw_create_result { + cmdline_fixed_string_t get; + cmdline_fixed_string_t bonding; + cmdline_fixed_string_t member; + cmdline_fixed_string_t hardware; + cmdline_fixed_string_t create; + uint16_t member_port_id; + uint16_t bonding_port_id; +}; + +static void +cmd_get_bonding_member_hw_create_parsed(void *parsed_result, + __rte_unused struct cmdline *cl, __rte_unused void *data) +{ + struct cmd_get_bonding_member_hw_create_result *res = parsed_result; + int ret; + + ret = rte_eth_bond_hw_create_get(res->bonding_port_id, res->member_port_id); + if (ret == 0) + printf("Member port %u hardware creates bonding port %u successfully\n", + res->member_port_id, res->bonding_port_id); + else + printf("Failed to get status of member port %u hardware creating" + " bonding port %u, %d\n", + res->member_port_id, res->bonding_port_id, ret); +} + +static cmdline_parse_token_string_t cmd_get_bonding_member_hw_create_get = + TOKEN_STRING_INITIALIZER(struct cmd_get_bonding_member_hw_create_result, + get, "get"); +static cmdline_parse_token_string_t cmd_get_bonding_member_hw_create_bonding = + TOKEN_STRING_INITIALIZER(struct cmd_get_bonding_member_hw_create_result, + bonding, "bonding"); +static cmdline_parse_token_string_t cmd_get_bonding_member_hw_create_member = + TOKEN_STRING_INITIALIZER(struct cmd_get_bonding_member_hw_create_result, + member, "member"); +static cmdline_parse_token_string_t cmd_get_bonding_member_hw_create_hardware = + TOKEN_STRING_INITIALIZER(struct cmd_get_bonding_member_hw_create_result, + create, "hardware"); +static cmdline_parse_token_string_t cmd_get_bonding_member_hw_create_create = + TOKEN_STRING_INITIALIZER(struct cmd_get_bonding_member_hw_create_result, + create, "create"); +static cmdline_parse_token_num_t cmd_get_bonding_member_hw_create_memberportid = + TOKEN_NUM_INITIALIZER(struct cmd_get_bonding_member_hw_create_result, + member_port_id, RTE_UINT16); +static cmdline_parse_token_num_t cmd_get_bonding_member_hw_create_bondingportid = + TOKEN_NUM_INITIALIZER(struct cmd_get_bonding_member_hw_create_result, + bonding_port_id, RTE_UINT16); + +static cmdline_parse_inst_t cmd_get_member_hw_create_bonding = { + .f = cmd_get_bonding_member_hw_create_parsed, + .data = NULL, + .help_str = "get bonding member hardware create (member_port_id) (bonding_port_id)", + .tokens = { + (void *)&cmd_get_bonding_member_hw_create_get, + (void *)&cmd_get_bonding_member_hw_create_bonding, + (void *)&cmd_get_bonding_member_hw_create_member, + (void *)&cmd_get_bonding_member_hw_create_hardware, + (void *)&cmd_get_bonding_member_hw_create_create, + (void *)&cmd_get_bonding_member_hw_create_memberportid, + (void *)&cmd_get_bonding_member_hw_create_bondingportid, + NULL + } +}; + static struct testpmd_driver_commands bonding_cmds = { .commands = { { @@ -749,6 +867,16 @@ static struct testpmd_driver_commands bonding_cmds = { "set bonding mode IEEE802.3AD aggregator policy (port_id) (agg_name)\n" " Set Aggregation mode for IEEE802.3AD (mode 4)\n", }, + { + &cmd_set_bonding_notify_member_ports, + "set bonding notify_member (port_id) (enable|disable)\n" + " Enable/disable the notify member flag of bonding port\n", + }, + { + &cmd_get_member_hw_create_bonding, + "get bonding member hardware create (member_port_id) (bonding_port_id)\n" + " Get the status of member port hardware creating the bonding port\n", + }, { NULL, NULL }, }, }; -- 2.39.1 ^ permalink raw reply [flat|nested] 80+ messages in thread
* [PATCH v3 0/8] Enhance the bond framework to support offload 2023-10-07 1:34 ` [PATCH v2 0/8] Enhance the bond framework to support offload Chaoyong He ` (7 preceding siblings ...) 2023-10-07 1:34 ` [PATCH v2 8/8] net/bonding: add commands for bonding port notification Chaoyong He @ 2023-10-08 1:50 ` Chaoyong He 2023-10-08 1:50 ` [PATCH v3 1/8] ethdev: add member notification for bonding port Chaoyong He ` (9 more replies) 8 siblings, 10 replies; 80+ messages in thread From: Chaoyong He @ 2023-10-08 1:50 UTC (permalink / raw) To: dev; +Cc: oss-drivers, Chaoyong He This patch series try to enhance the bond framework to support the offload feature better: * Add new API to make the member port can access some information of the bond port which belongs. * Add new API to get the result of whether bond port is created by the member port. * Add two command line argument to control if enable member port notification and dedicated queue features. * Add logic to support add ports which share the same PCI address into bond port. * Also modify the testpmd application to test the new APIs and logics added by this patch series. --- v2: * Fix compile error on github-robot by removing the redundancy function declaration in the header file. v3: * Use the hole in the structure for the new added flag data field. --- Long Wu (8): ethdev: add member notification for bonding port ethdev: add API to get hardware creation of bonding port net/bonding: modify interface comment format net/bonding: add bonding port arguments net/bonding: support add port by data name net/bonding: create new rte flow header file net/bonding: support checking valid bonding port ID net/bonding: add commands for bonding port notification .../link_bonding_poll_mode_drv_lib.rst | 19 ++ drivers/net/bonding/bonding_testpmd.c | 128 ++++++++++ drivers/net/bonding/eth_bond_8023ad_private.h | 52 ++-- drivers/net/bonding/eth_bond_private.h | 24 +- drivers/net/bonding/rte_eth_bond.h | 238 +++++++++++++----- drivers/net/bonding/rte_eth_bond_8023ad.h | 76 ++++-- drivers/net/bonding/rte_eth_bond_alb.h | 34 ++- drivers/net/bonding/rte_eth_bond_api.c | 123 +++++++++ drivers/net/bonding/rte_eth_bond_args.c | 47 ++++ drivers/net/bonding/rte_eth_bond_flow.c | 1 + drivers/net/bonding/rte_eth_bond_flow.h | 22 ++ drivers/net/bonding/rte_eth_bond_pmd.c | 89 ++++++- drivers/net/bonding/version.map | 5 + lib/ethdev/ethdev_driver.h | 38 +++ 14 files changed, 766 insertions(+), 130 deletions(-) create mode 100644 drivers/net/bonding/rte_eth_bond_flow.h -- 2.39.1 ^ permalink raw reply [flat|nested] 80+ messages in thread
* [PATCH v3 1/8] ethdev: add member notification for bonding port 2023-10-08 1:50 ` [PATCH v3 0/8] Enhance the bond framework to support offload Chaoyong He @ 2023-10-08 1:50 ` Chaoyong He 2023-10-08 2:49 ` lihuisong (C) 2023-10-08 1:50 ` [PATCH v3 2/8] ethdev: add API to get hardware creation of " Chaoyong He ` (8 subsequent siblings) 9 siblings, 1 reply; 80+ messages in thread From: Chaoyong He @ 2023-10-08 1:50 UTC (permalink / raw) To: dev; +Cc: oss-drivers, Long Wu, James Hershaw, Chaoyong He From: Long Wu <long.wu@corigine.com> Bonding PMD does not let member ports know the bonding port's information, like how many member ports the bonding port has, what mode the bonding port is in and so on. Add the notification interface for bonding port to let member port know it is added to a bonding port and what the bonding port's configuration is. If so the member ports have chance to achieve its bond-flow-offlod or other private bonding functions. Signed-off-by: Long Wu <long.wu@corigine.com> Reviewed-by: James Hershaw <james.hershaw@corigine.com> Reviewed-by: Chaoyong He <chaoyong.he@corigine.com> --- drivers/net/bonding/eth_bond_private.h | 1 + drivers/net/bonding/rte_eth_bond.h | 46 ++++++++++++++++ drivers/net/bonding/rte_eth_bond_api.c | 73 ++++++++++++++++++++++++++ drivers/net/bonding/rte_eth_bond_pmd.c | 27 ++++++++-- drivers/net/bonding/version.map | 3 ++ lib/ethdev/ethdev_driver.h | 18 +++++++ 6 files changed, 165 insertions(+), 3 deletions(-) diff --git a/drivers/net/bonding/eth_bond_private.h b/drivers/net/bonding/eth_bond_private.h index e688894210..f69e85c199 100644 --- a/drivers/net/bonding/eth_bond_private.h +++ b/drivers/net/bonding/eth_bond_private.h @@ -180,6 +180,7 @@ struct bond_dev_private { uint8_t member_update_idx; bool kvargs_processing_is_done; + bool notify_member; /**< Enable member notification of bonding port. */ uint32_t candidate_max_rx_pktlen; uint32_t max_rx_pktlen; diff --git a/drivers/net/bonding/rte_eth_bond.h b/drivers/net/bonding/rte_eth_bond.h index f10165f2c6..737beca446 100644 --- a/drivers/net/bonding/rte_eth_bond.h +++ b/drivers/net/bonding/rte_eth_bond.h @@ -351,6 +351,52 @@ rte_eth_bond_link_up_prop_delay_set(uint16_t bonding_port_id, int rte_eth_bond_link_up_prop_delay_get(uint16_t bonding_port_id); +/** + * Set the flag of whether bonding port notifies member ports. + * + * @param bonding_port_id + * Port ID of bonding device. + * @param notify_member + * Flag of whether bonding port notifies member ports. + * + * @return + * 0 on success, negative value otherwise. + */ +__rte_experimental +int +rte_eth_bond_notify_member_flag_set(uint16_t bonding_port_id, bool notify_member); + +/** + * Get the flag of whether bonding port notifies member ports. + * + * @param bonding_port_id + * Port ID of bonding device. + * @param notify_member + * Flag of whether bonding port notifies member ports. + * + * @return + * 0 on success, negative value otherwise. + */ +__rte_experimental +int +rte_eth_bond_notify_member_flag_get(uint16_t bonding_port_id, bool *notify_member); + +/** + * Notify the member ports of bonding port's information. + * + * This interface is called in the following functions: + * - bond_ethdev_lsc_event_callback() + * - bond_ethdev_configure() + * + * @param bonding_port_id + * Port ID of bonding device. + * + * @return + * 0 on success, negative value otherwise. + */ +__rte_experimental +int +rte_eth_bond_notify_members(uint16_t bonding_port_id); #ifdef __cplusplus } diff --git a/drivers/net/bonding/rte_eth_bond_api.c b/drivers/net/bonding/rte_eth_bond_api.c index 99e496556a..48c1d050fd 100644 --- a/drivers/net/bonding/rte_eth_bond_api.c +++ b/drivers/net/bonding/rte_eth_bond_api.c @@ -627,6 +627,17 @@ __eth_bond_member_add_lock_free(uint16_t bonding_port_id, uint16_t member_port_i member_vlan_filter_set(bonding_port_id, member_port_id); + if (internals->notify_member && + *member_eth_dev->dev_ops->bond_notify_member != NULL) { + ret = member_eth_dev->dev_ops->bond_notify_member(member_eth_dev, + bonding_eth_dev); + if (ret < 0) { + RTE_BOND_LOG(ERR, "Add member (port %u) notify failed!", + member_port_id); + return -1; + } + } + return 0; } @@ -733,6 +744,10 @@ __eth_bond_member_remove_lock_free(uint16_t bonding_port_id, member_eth_dev = &rte_eth_devices[member_port_id]; member_remove(internals, member_eth_dev); member_eth_dev->data->dev_flags &= (~RTE_ETH_DEV_BONDING_MEMBER); + if (internals->notify_member && + *member_eth_dev->dev_ops->bond_notify_member != NULL) + member_eth_dev->dev_ops->bond_notify_member(member_eth_dev, + bonding_eth_dev); /* first member in the active list will be the primary by default, * otherwise use first device in list */ @@ -1098,3 +1113,61 @@ rte_eth_bond_link_up_prop_delay_get(uint16_t bonding_port_id) return internals->link_up_delay_ms; } + +int +rte_eth_bond_notify_member_flag_set(uint16_t bonding_port_id, bool notify_member) +{ + struct bond_dev_private *internals; + + if (valid_bonding_port_id(bonding_port_id) != 0) + return -EINVAL; + + internals = rte_eth_devices[bonding_port_id].data->dev_private; + + internals->notify_member = notify_member; + + return 0; +} + +int +rte_eth_bond_notify_member_flag_get(uint16_t bonding_port_id, bool *notify_member) +{ + struct bond_dev_private *internals; + + if (valid_bonding_port_id(bonding_port_id) != 0) + return -EINVAL; + + internals = rte_eth_devices[bonding_port_id].data->dev_private; + + *notify_member = internals->notify_member; + + return 0; +} + +int +rte_eth_bond_notify_members(uint16_t bonding_port_id) +{ + uint32_t i; + uint16_t member_port_id; + struct rte_eth_dev *bond_dev; + struct bond_dev_private *internals; + struct rte_eth_dev *member_dev[RTE_MAX_ETHPORTS]; + + if (valid_bonding_port_id(bonding_port_id) != 0) + return -EINVAL; + + bond_dev = &rte_eth_devices[bonding_port_id]; + internals = bond_dev->data->dev_private; + + for (i = 0; i < internals->member_count; i++) { + member_port_id = internals->members[i].port_id; + member_dev[i] = &rte_eth_devices[member_port_id]; + if (*member_dev[i]->dev_ops->bond_notify_member == NULL) + return -ENOTSUP; + } + + for (i = 0; i < internals->member_count; i++) + member_dev[i]->dev_ops->bond_notify_member(member_dev[i], bond_dev); + + return 0; +} diff --git a/drivers/net/bonding/rte_eth_bond_pmd.c b/drivers/net/bonding/rte_eth_bond_pmd.c index 122b1187fd..b99b8b8938 100644 --- a/drivers/net/bonding/rte_eth_bond_pmd.c +++ b/drivers/net/bonding/rte_eth_bond_pmd.c @@ -2968,11 +2968,13 @@ bond_ethdev_lsc_event_callback(uint16_t port_id, enum rte_eth_event_type type, int valid_member = 0; uint16_t active_pos, member_idx; uint16_t i; + uint16_t bonding_port_id; if (type != RTE_ETH_EVENT_INTR_LSC || param == NULL) return rc; - bonding_eth_dev = &rte_eth_devices[*(uint16_t *)param]; + bonding_port_id = *(uint16_t *)param; + bonding_eth_dev = &rte_eth_devices[bonding_port_id]; if (check_for_bonding_ethdev(bonding_eth_dev)) return rc; @@ -3044,8 +3046,10 @@ bond_ethdev_lsc_event_callback(uint16_t port_id, enum rte_eth_event_type type, * using it. */ if (internals->user_defined_primary_port && - internals->primary_port == port_id) + internals->primary_port == port_id) { bond_ethdev_primary_set(internals, port_id); + rte_eth_bond_notify_members(bonding_port_id); + } } else { if (active_pos == internals->active_member_count) goto link_update; @@ -3064,6 +3068,7 @@ bond_ethdev_lsc_event_callback(uint16_t port_id, enum rte_eth_event_type type, internals->active_members[0]); else internals->current_primary_port = internals->primary_port; + rte_eth_bond_notify_members(bonding_port_id); mac_address_members_update(bonding_eth_dev); bond_ethdev_promiscuous_update(bonding_eth_dev); bond_ethdev_allmulticast_update(bonding_eth_dev); @@ -3362,6 +3367,7 @@ dump_basic(const struct rte_eth_dev *dev, FILE *f) struct bond_dev_private instant_priv; const struct bond_dev_private *internals = &instant_priv; int mode, i; + bool notify_member; /* Obtain a instance of dev_private to prevent data from being modified. */ memcpy(&instant_priv, dev->data->dev_private, sizeof(struct bond_dev_private)); @@ -3431,6 +3437,13 @@ dump_basic(const struct rte_eth_dev *dev, FILE *f) fprintf(f, "\tUser Defined Primary: [%u]\n", internals->primary_port); if (internals->member_count > 0) fprintf(f, "\tCurrent Primary: [%u]\n", internals->current_primary_port); + + if (rte_eth_bond_notify_member_flag_get(internals->port_id, ¬ify_member) == 0) + fprintf(f, "\tNotify Member Ports Flag: %s\n", + notify_member ? "enable" : "disable"); + else + fprintf(f, "\tFailed to get notify member ports flag for bonding port %d\n", + internals->port_id); } static void @@ -3983,8 +3996,12 @@ bond_ethdev_configure(struct rte_eth_dev *dev) * if no kvlist, it means that this bonding device has been created * through the bonding api. */ - if (!kvlist || internals->kvargs_processing_is_done) + if (!kvlist || internals->kvargs_processing_is_done) { + if (internals->notify_member && rte_eth_bond_notify_members(port_id) != 0) + RTE_BOND_LOG(ERR, "Notify member ports failed"); + return 0; + } internals->kvargs_processing_is_done = true; @@ -4222,6 +4239,10 @@ bond_ethdev_configure(struct rte_eth_dev *dev) return -1; } } + + if (internals->notify_member && rte_eth_bond_notify_members(port_id) != 0) + RTE_BOND_LOG(ERR, "Notify member ports failed"); + return 0; } diff --git a/drivers/net/bonding/version.map b/drivers/net/bonding/version.map index 09ee21c55f..3bd5e8ad11 100644 --- a/drivers/net/bonding/version.map +++ b/drivers/net/bonding/version.map @@ -35,4 +35,7 @@ EXPERIMENTAL { rte_eth_bond_member_add; rte_eth_bond_member_remove; rte_eth_bond_members_get; + rte_eth_bond_notify_member_flag_get; + rte_eth_bond_notify_member_flag_set; + rte_eth_bond_notify_members; }; diff --git a/lib/ethdev/ethdev_driver.h b/lib/ethdev/ethdev_driver.h index deb23ada18..f626f971e5 100644 --- a/lib/ethdev/ethdev_driver.h +++ b/lib/ethdev/ethdev_driver.h @@ -1216,6 +1216,21 @@ typedef int (*eth_count_aggr_ports_t)(struct rte_eth_dev *dev); typedef int (*eth_map_aggr_tx_affinity_t)(struct rte_eth_dev *dev, uint16_t tx_queue_id, uint8_t affinity); +/** + * @internal + * Bonding port notifies the member ports. + * + * @param dev + * Member port (ethdev) handle. + * @param bonding_dev + * Bonding port (ethdev) handle. + * + * @return + * Negative on error, 0 on success. + */ +typedef int (*eth_bond_notify_member)(struct rte_eth_dev *dev, + struct rte_eth_dev *bonding_dev); + /** * @internal A structure containing the functions exported by an Ethernet driver. */ @@ -1455,6 +1470,9 @@ struct eth_dev_ops { eth_count_aggr_ports_t count_aggr_ports; /** Map a Tx queue with an aggregated port of the DPDK port */ eth_map_aggr_tx_affinity_t map_aggr_tx_affinity; + + /** Notify the member port of bonding port information */ + eth_bond_notify_member bond_notify_member; }; /** -- 2.39.1 ^ permalink raw reply [flat|nested] 80+ messages in thread
* Re: [PATCH v3 1/8] ethdev: add member notification for bonding port 2023-10-08 1:50 ` [PATCH v3 1/8] ethdev: add member notification for bonding port Chaoyong He @ 2023-10-08 2:49 ` lihuisong (C) 2023-10-09 3:11 ` 回复: " Long Wu 0 siblings, 1 reply; 80+ messages in thread From: lihuisong (C) @ 2023-10-08 2:49 UTC (permalink / raw) To: Chaoyong He, dev; +Cc: oss-drivers, Long Wu, James Hershaw Hi Chaoyong, some comments as below. 在 2023/10/8 9:50, Chaoyong He 写道: > From: Long Wu <long.wu@corigine.com> > > Bonding PMD does not let member ports know the bonding port's > information, like how many member ports the bonding port has, > what mode the bonding port is in and so on. > > Add the notification interface for bonding port to let member > port know it is added to a bonding port and what the bonding > port's configuration is. If so the member ports have chance to > achieve its bond-flow-offlod or other private bonding functions. "its bond-flow-offlod or other private bonding functions" I wonder that what PMDs can do with this. Can you give an example in PMD to help others review? > > Signed-off-by: Long Wu <long.wu@corigine.com> > Reviewed-by: James Hershaw <james.hershaw@corigine.com> > Reviewed-by: Chaoyong He <chaoyong.he@corigine.com> > --- > drivers/net/bonding/eth_bond_private.h | 1 + > drivers/net/bonding/rte_eth_bond.h | 46 ++++++++++++++++ > drivers/net/bonding/rte_eth_bond_api.c | 73 ++++++++++++++++++++++++++ > drivers/net/bonding/rte_eth_bond_pmd.c | 27 ++++++++-- > drivers/net/bonding/version.map | 3 ++ > lib/ethdev/ethdev_driver.h | 18 +++++++ > 6 files changed, 165 insertions(+), 3 deletions(-) > > diff --git a/drivers/net/bonding/eth_bond_private.h b/drivers/net/bonding/eth_bond_private.h > index e688894210..f69e85c199 100644 > --- a/drivers/net/bonding/eth_bond_private.h > +++ b/drivers/net/bonding/eth_bond_private.h > @@ -180,6 +180,7 @@ struct bond_dev_private { > uint8_t member_update_idx; > > bool kvargs_processing_is_done; > + bool notify_member; /**< Enable member notification of bonding port. */ > > uint32_t candidate_max_rx_pktlen; > uint32_t max_rx_pktlen; > diff --git a/drivers/net/bonding/rte_eth_bond.h b/drivers/net/bonding/rte_eth_bond.h > index f10165f2c6..737beca446 100644 > --- a/drivers/net/bonding/rte_eth_bond.h > +++ b/drivers/net/bonding/rte_eth_bond.h > @@ -351,6 +351,52 @@ rte_eth_bond_link_up_prop_delay_set(uint16_t bonding_port_id, > int > rte_eth_bond_link_up_prop_delay_get(uint16_t bonding_port_id); > > +/** > + * Set the flag of whether bonding port notifies member ports. > + * > + * @param bonding_port_id > + * Port ID of bonding device. > + * @param notify_member > + * Flag of whether bonding port notifies member ports. > + * > + * @return > + * 0 on success, negative value otherwise. > + */ > +__rte_experimental > +int > +rte_eth_bond_notify_member_flag_set(uint16_t bonding_port_id, bool notify_member); s/notify_membe/notify in input? because function name reveals the meaning already. > + > +/** > + * Get the flag of whether bonding port notifies member ports. > + * > + * @param bonding_port_id > + * Port ID of bonding device. > + * @param notify_member > + * Flag of whether bonding port notifies member ports. > + * > + * @return > + * 0 on success, negative value otherwise. > + */ > +__rte_experimental > +int > +rte_eth_bond_notify_member_flag_get(uint16_t bonding_port_id, bool *notify_member); > + > +/** > + * Notify the member ports of bonding port's information. > + * > + * This interface is called in the following functions: > + * - bond_ethdev_lsc_event_callback() > + * - bond_ethdev_configure() Is this interface used just in these cases? If so I don't think it should be a API in eth_dev_op. > + * > + * @param bonding_port_id > + * Port ID of bonding device. > + * > + * @return > + * 0 on success, negative value otherwise. > + */ > +__rte_experimental > +int > +rte_eth_bond_notify_members(uint16_t bonding_port_id); > > #ifdef __cplusplus > } > diff --git a/drivers/net/bonding/rte_eth_bond_api.c b/drivers/net/bonding/rte_eth_bond_api.c > index 99e496556a..48c1d050fd 100644 > --- a/drivers/net/bonding/rte_eth_bond_api.c > +++ b/drivers/net/bonding/rte_eth_bond_api.c > @@ -627,6 +627,17 @@ __eth_bond_member_add_lock_free(uint16_t bonding_port_id, uint16_t member_port_i > > member_vlan_filter_set(bonding_port_id, member_port_id); > > + if (internals->notify_member && > + *member_eth_dev->dev_ops->bond_notify_member != NULL) { > + ret = member_eth_dev->dev_ops->bond_notify_member(member_eth_dev, > + bonding_eth_dev); > + if (ret < 0) { > + RTE_BOND_LOG(ERR, "Add member (port %u) notify failed!", > + member_port_id); > + return -1; > + } > + } > + > return 0; > > } > @@ -733,6 +744,10 @@ __eth_bond_member_remove_lock_free(uint16_t bonding_port_id, > member_eth_dev = &rte_eth_devices[member_port_id]; > member_remove(internals, member_eth_dev); > member_eth_dev->data->dev_flags &= (~RTE_ETH_DEV_BONDING_MEMBER); > + if (internals->notify_member && > + *member_eth_dev->dev_ops->bond_notify_member != NULL) > + member_eth_dev->dev_ops->bond_notify_member(member_eth_dev, > + bonding_eth_dev); > > /* first member in the active list will be the primary by default, > * otherwise use first device in list */ > @@ -1098,3 +1113,61 @@ rte_eth_bond_link_up_prop_delay_get(uint16_t bonding_port_id) > > return internals->link_up_delay_ms; > } > + > +int > +rte_eth_bond_notify_member_flag_set(uint16_t bonding_port_id, bool notify_member) > +{ > + struct bond_dev_private *internals; > + > + if (valid_bonding_port_id(bonding_port_id) != 0) > + return -EINVAL; > + > + internals = rte_eth_devices[bonding_port_id].data->dev_private; > + > + internals->notify_member = notify_member; > + > + return 0; > +} > + > +int > +rte_eth_bond_notify_member_flag_get(uint16_t bonding_port_id, bool *notify_member) > +{ > + struct bond_dev_private *internals; > + > + if (valid_bonding_port_id(bonding_port_id) != 0) > + return -EINVAL; > + > + internals = rte_eth_devices[bonding_port_id].data->dev_private; > + > + *notify_member = internals->notify_member; > + > + return 0; > +} > + > +int > +rte_eth_bond_notify_members(uint16_t bonding_port_id) > +{ > + uint32_t i; > + uint16_t member_port_id; > + struct rte_eth_dev *bond_dev; > + struct bond_dev_private *internals; > + struct rte_eth_dev *member_dev[RTE_MAX_ETHPORTS]; > + > + if (valid_bonding_port_id(bonding_port_id) != 0) > + return -EINVAL; > + > + bond_dev = &rte_eth_devices[bonding_port_id]; > + internals = bond_dev->data->dev_private; > + > + for (i = 0; i < internals->member_count; i++) { > + member_port_id = internals->members[i].port_id; > + member_dev[i] = &rte_eth_devices[member_port_id]; > + if (*member_dev[i]->dev_ops->bond_notify_member == NULL) > + return -ENOTSUP; > + } Do we need all member ports support bond_notify_member api? If allow user to select one member port to notify? This might be more general. In addition, this action here is inconsistent with above handle(do not notify member if doesn't supportbond_notify_member API). > + > + for (i = 0; i < internals->member_count; i++) > + member_dev[i]->dev_ops->bond_notify_member(member_dev[i], bond_dev); > + > + return 0; > +} > diff --git a/drivers/net/bonding/rte_eth_bond_pmd.c b/drivers/net/bonding/rte_eth_bond_pmd.c > index 122b1187fd..b99b8b8938 100644 > --- a/drivers/net/bonding/rte_eth_bond_pmd.c > +++ b/drivers/net/bonding/rte_eth_bond_pmd.c > @@ -2968,11 +2968,13 @@ bond_ethdev_lsc_event_callback(uint16_t port_id, enum rte_eth_event_type type, > int valid_member = 0; > uint16_t active_pos, member_idx; > uint16_t i; > + uint16_t bonding_port_id; > > if (type != RTE_ETH_EVENT_INTR_LSC || param == NULL) > return rc; > > - bonding_eth_dev = &rte_eth_devices[*(uint16_t *)param]; > + bonding_port_id = *(uint16_t *)param; > + bonding_eth_dev = &rte_eth_devices[bonding_port_id]; > > if (check_for_bonding_ethdev(bonding_eth_dev)) > return rc; > @@ -3044,8 +3046,10 @@ bond_ethdev_lsc_event_callback(uint16_t port_id, enum rte_eth_event_type type, > * using it. > */ > if (internals->user_defined_primary_port && > - internals->primary_port == port_id) > + internals->primary_port == port_id) { > bond_ethdev_primary_set(internals, port_id); > + rte_eth_bond_notify_members(bonding_port_id); > + } > } else { > if (active_pos == internals->active_member_count) > goto link_update; > @@ -3064,6 +3068,7 @@ bond_ethdev_lsc_event_callback(uint16_t port_id, enum rte_eth_event_type type, > internals->active_members[0]); > else > internals->current_primary_port = internals->primary_port; > + rte_eth_bond_notify_members(bonding_port_id); > mac_address_members_update(bonding_eth_dev); > bond_ethdev_promiscuous_update(bonding_eth_dev); > bond_ethdev_allmulticast_update(bonding_eth_dev); > @@ -3362,6 +3367,7 @@ dump_basic(const struct rte_eth_dev *dev, FILE *f) > struct bond_dev_private instant_priv; > const struct bond_dev_private *internals = &instant_priv; > int mode, i; > + bool notify_member; > > /* Obtain a instance of dev_private to prevent data from being modified. */ > memcpy(&instant_priv, dev->data->dev_private, sizeof(struct bond_dev_private)); > @@ -3431,6 +3437,13 @@ dump_basic(const struct rte_eth_dev *dev, FILE *f) > fprintf(f, "\tUser Defined Primary: [%u]\n", internals->primary_port); > if (internals->member_count > 0) > fprintf(f, "\tCurrent Primary: [%u]\n", internals->current_primary_port); > + > + if (rte_eth_bond_notify_member_flag_get(internals->port_id, ¬ify_member) == 0) > + fprintf(f, "\tNotify Member Ports Flag: %s\n", > + notify_member ? "enable" : "disable"); > + else > + fprintf(f, "\tFailed to get notify member ports flag for bonding port %d\n", > + internals->port_id); > } > > static void > @@ -3983,8 +3996,12 @@ bond_ethdev_configure(struct rte_eth_dev *dev) > * if no kvlist, it means that this bonding device has been created > * through the bonding api. > */ > - if (!kvlist || internals->kvargs_processing_is_done) > + if (!kvlist || internals->kvargs_processing_is_done) { > + if (internals->notify_member && rte_eth_bond_notify_members(port_id) != 0) > + RTE_BOND_LOG(ERR, "Notify member ports failed"); > + > return 0; > + } > > internals->kvargs_processing_is_done = true; > > @@ -4222,6 +4239,10 @@ bond_ethdev_configure(struct rte_eth_dev *dev) > return -1; > } > } > + > + if (internals->notify_member && rte_eth_bond_notify_members(port_id) != 0) > + RTE_BOND_LOG(ERR, "Notify member ports failed"); > + > return 0; > } > > diff --git a/drivers/net/bonding/version.map b/drivers/net/bonding/version.map > index 09ee21c55f..3bd5e8ad11 100644 > --- a/drivers/net/bonding/version.map > +++ b/drivers/net/bonding/version.map > @@ -35,4 +35,7 @@ EXPERIMENTAL { > rte_eth_bond_member_add; > rte_eth_bond_member_remove; > rte_eth_bond_members_get; > + rte_eth_bond_notify_member_flag_get; > + rte_eth_bond_notify_member_flag_set; > + rte_eth_bond_notify_members; > }; > diff --git a/lib/ethdev/ethdev_driver.h b/lib/ethdev/ethdev_driver.h > index deb23ada18..f626f971e5 100644 > --- a/lib/ethdev/ethdev_driver.h > +++ b/lib/ethdev/ethdev_driver.h > @@ -1216,6 +1216,21 @@ typedef int (*eth_count_aggr_ports_t)(struct rte_eth_dev *dev); > typedef int (*eth_map_aggr_tx_affinity_t)(struct rte_eth_dev *dev, uint16_t tx_queue_id, > uint8_t affinity); > > +/** > + * @internal > + * Bonding port notifies the member ports. > + * > + * @param dev > + * Member port (ethdev) handle. > + * @param bonding_dev > + * Bonding port (ethdev) handle. > + * > + * @return > + * Negative on error, 0 on success. > + */ > +typedef int (*eth_bond_notify_member)(struct rte_eth_dev *dev, > + struct rte_eth_dev *bonding_dev); > + > /** > * @internal A structure containing the functions exported by an Ethernet driver. > */ > @@ -1455,6 +1470,9 @@ struct eth_dev_ops { > eth_count_aggr_ports_t count_aggr_ports; > /** Map a Tx queue with an aggregated port of the DPDK port */ > eth_map_aggr_tx_affinity_t map_aggr_tx_affinity; > + > + /** Notify the member port of bonding port information */ > + eth_bond_notify_member bond_notify_member; > }; > > /** ^ permalink raw reply [flat|nested] 80+ messages in thread
* 回复: [PATCH v3 1/8] ethdev: add member notification for bonding port 2023-10-08 2:49 ` lihuisong (C) @ 2023-10-09 3:11 ` Long Wu 2023-10-17 8:27 ` lihuisong (C) 0 siblings, 1 reply; 80+ messages in thread From: Long Wu @ 2023-10-09 3:11 UTC (permalink / raw) To: lihuisong (C), Chaoyong He, dev; +Cc: oss-drivers, James Hershaw > Hi Chaoyong, > > some comments as below. > > > 在 2023/10/8 9:50, Chaoyong He 写道: > > From: Long Wu <long.wu@corigine.com> > > > > Bonding PMD does not let member ports know the bonding port's > > information, like how many member ports the bonding port has, what > > mode the bonding port is in and so on. > > > > Add the notification interface for bonding port to let member port > > know it is added to a bonding port and what the bonding port's > > configuration is. If so the member ports have chance to achieve its > > bond-flow-offlod or other private bonding functions. > "its bond-flow-offlod or other private bonding functions" > I wonder that what PMDs can do with this. > Can you give an example in PMD to help others review? After this patch series, I will send out nfp PMD code about "its bond-flow-offlod or other private bonding functions ". I can explain here: "bond-flow" means the flow rule's destination port is a bonding port. Now DPDK can use bonding port as the source port in a flow rule and member ports can offload this flow. But member ports cannot offload a flow that its destination port is a bonding port. Because the member ports don't know the bonding port. (Of course, some PMDs has its self-function to let member ports know the bonding port but it doesn't a general "DPDK" way). After this "notify patch", DPDK can do "bond-flow-offload", also "other private bonding functions"(like hardware balance policy, primary port changing and so on) can work. > > > > Signed-off-by: Long Wu <long.wu@corigine.com> > > Reviewed-by: James Hershaw <james.hershaw@corigine.com> > > Reviewed-by: Chaoyong He <chaoyong.he@corigine.com> > > --- > > drivers/net/bonding/eth_bond_private.h | 1 + > > drivers/net/bonding/rte_eth_bond.h | 46 ++++++++++++++++ > > drivers/net/bonding/rte_eth_bond_api.c | 73 > ++++++++++++++++++++++++++ > > drivers/net/bonding/rte_eth_bond_pmd.c | 27 ++++++++-- > > drivers/net/bonding/version.map | 3 ++ > > lib/ethdev/ethdev_driver.h | 18 +++++++ > > 6 files changed, 165 insertions(+), 3 deletions(-) > > > > diff --git a/drivers/net/bonding/eth_bond_private.h > > b/drivers/net/bonding/eth_bond_private.h > > index e688894210..f69e85c199 100644 > > --- a/drivers/net/bonding/eth_bond_private.h > > +++ b/drivers/net/bonding/eth_bond_private.h > > @@ -180,6 +180,7 @@ struct bond_dev_private { > > uint8_t member_update_idx; > > > > bool kvargs_processing_is_done; > > + bool notify_member; /**< Enable member notification of bonding port. > > +*/ > > > > uint32_t candidate_max_rx_pktlen; > > uint32_t max_rx_pktlen; > > diff --git a/drivers/net/bonding/rte_eth_bond.h > > b/drivers/net/bonding/rte_eth_bond.h > > index f10165f2c6..737beca446 100644 > > --- a/drivers/net/bonding/rte_eth_bond.h > > +++ b/drivers/net/bonding/rte_eth_bond.h > > @@ -351,6 +351,52 @@ rte_eth_bond_link_up_prop_delay_set(uint16_t > bonding_port_id, > > int > > rte_eth_bond_link_up_prop_delay_get(uint16_t bonding_port_id); > > > > +/** > > + * Set the flag of whether bonding port notifies member ports. > > + * > > + * @param bonding_port_id > > + * Port ID of bonding device. > > + * @param notify_member > > + * Flag of whether bonding port notifies member ports. > > + * > > + * @return > > + * 0 on success, negative value otherwise. > > + */ > > +__rte_experimental > > +int > > +rte_eth_bond_notify_member_flag_set(uint16_t bonding_port_id, bool > > +notify_member); > s/notify_membe/notify in input? > because function name reveals the meaning already. Thank you, you are right. > > + > > +/** > > + * Get the flag of whether bonding port notifies member ports. > > + * > > + * @param bonding_port_id > > + * Port ID of bonding device. > > + * @param notify_member > > + * Flag of whether bonding port notifies member ports. > > + * > > + * @return > > + * 0 on success, negative value otherwise. > > + */ > > +__rte_experimental > > +int > > +rte_eth_bond_notify_member_flag_get(uint16_t bonding_port_id, bool > > +*notify_member); > > + > > +/** > > + * Notify the member ports of bonding port's information. > > + * > > + * This interface is called in the following functions: > > + * - bond_ethdev_lsc_event_callback() > > + * - bond_ethdev_configure() > Is this interface used just in these cases? > If so I don't think it should be a API in eth_dev_op. Sorry I'm a bit confused. Do you mean "rte_eth_bond_notify_members" this interface? This interface is called in 3 functions. > > ... > > + struct rte_eth_dev *member_dev[RTE_MAX_ETHPORTS]; > > + > > + if (valid_bonding_port_id(bonding_port_id) != 0) > > + return -EINVAL; > > + > > + bond_dev = &rte_eth_devices[bonding_port_id]; > > + internals = bond_dev->data->dev_private; > > + > > + for (i = 0; i < internals->member_count; i++) { > > + member_port_id = internals->members[i].port_id; > > + member_dev[i] = &rte_eth_devices[member_port_id]; > > + if (*member_dev[i]->dev_ops->bond_notify_member == NULL) > > + return -ENOTSUP; > > + } > Do we need all member ports support bond_notify_member api? > If allow user to select one member port to notify? This might be more general. > In addition, this action here is inconsistent with above handle(do not > notify member if doesn't supportbond_notify_member API). Yes, we do not need all member ports support this API. I just want to have a stricter restriction on this feature before, but I think your suggestion is better. > > ... ^ permalink raw reply [flat|nested] 80+ messages in thread
* Re: [PATCH v3 1/8] ethdev: add member notification for bonding port 2023-10-09 3:11 ` 回复: " Long Wu @ 2023-10-17 8:27 ` lihuisong (C) 2023-10-18 2:16 ` Chaoyong He 0 siblings, 1 reply; 80+ messages in thread From: lihuisong (C) @ 2023-10-17 8:27 UTC (permalink / raw) To: Long Wu, Chaoyong He, dev, Ferruh Yigit; +Cc: oss-drivers, James Hershaw 在 2023/10/9 11:11, Long Wu 写道: >> Hi Chaoyong, >> >> some comments as below. >> >> >> 在 2023/10/8 9:50, Chaoyong He 写道: >>> From: Long Wu <long.wu@corigine.com> >>> >>> Bonding PMD does not let member ports know the bonding port's >>> information, like how many member ports the bonding port has, what >>> mode the bonding port is in and so on. >>> >>> Add the notification interface for bonding port to let member port >>> know it is added to a bonding port and what the bonding port's >>> configuration is. If so the member ports have chance to achieve its >>> bond-flow-offlod or other private bonding functions. >> "its bond-flow-offlod or other private bonding functions" >> I wonder that what PMDs can do with this. >> Can you give an example in PMD to help others review? > After this patch series, I will send out nfp PMD code about "its bond-flow-offlod or other private bonding functions ". > I can explain here: > "bond-flow" means the flow rule's destination port is a bonding port. > Now DPDK can use bonding port as the source port in a flow rule and member ports can offload this flow. > But member ports cannot offload a flow that its destination port is a bonding port. > Because the member ports don't know the bonding port. (Of course, some PMDs has its self-function to let member ports know the bonding port but it doesn't a general "DPDK" way). > After this "notify patch", DPDK can do "bond-flow-offload", also "other private bonding functions"(like hardware balance policy, primary port changing and so on) can work. I think what you said is more like "bonding offload", right? It seems that you cannot do it just based on current these API in this series. You probably have other works to upstream for like "bond-flow" and "other private bonding functions". > >>> Signed-off-by: Long Wu <long.wu@corigine.com> >>> Reviewed-by: James Hershaw <james.hershaw@corigine.com> >>> Reviewed-by: Chaoyong He <chaoyong.he@corigine.com> >>> --- >>> drivers/net/bonding/eth_bond_private.h | 1 + >>> drivers/net/bonding/rte_eth_bond.h | 46 ++++++++++++++++ >>> drivers/net/bonding/rte_eth_bond_api.c | 73 >> ++++++++++++++++++++++++++ >>> drivers/net/bonding/rte_eth_bond_pmd.c | 27 ++++++++-- >>> drivers/net/bonding/version.map | 3 ++ >>> lib/ethdev/ethdev_driver.h | 18 +++++++ >>> 6 files changed, 165 insertions(+), 3 deletions(-) >>> >>> diff --git a/drivers/net/bonding/eth_bond_private.h >>> b/drivers/net/bonding/eth_bond_private.h >>> index e688894210..f69e85c199 100644 >>> --- a/drivers/net/bonding/eth_bond_private.h >>> +++ b/drivers/net/bonding/eth_bond_private.h >>> @@ -180,6 +180,7 @@ struct bond_dev_private { >>> uint8_t member_update_idx; >>> >>> bool kvargs_processing_is_done; >>> + bool notify_member; /**< Enable member notification of bonding port. >>> +*/ >>> >>> uint32_t candidate_max_rx_pktlen; >>> uint32_t max_rx_pktlen; >>> diff --git a/drivers/net/bonding/rte_eth_bond.h >>> b/drivers/net/bonding/rte_eth_bond.h >>> index f10165f2c6..737beca446 100644 >>> --- a/drivers/net/bonding/rte_eth_bond.h >>> +++ b/drivers/net/bonding/rte_eth_bond.h >>> @@ -351,6 +351,52 @@ rte_eth_bond_link_up_prop_delay_set(uint16_t >> bonding_port_id, >>> int >>> rte_eth_bond_link_up_prop_delay_get(uint16_t bonding_port_id); >>> >>> +/** >>> + * Set the flag of whether bonding port notifies member ports. >>> + * >>> + * @param bonding_port_id >>> + * Port ID of bonding device. >>> + * @param notify_member >>> + * Flag of whether bonding port notifies member ports. >>> + * >>> + * @return >>> + * 0 on success, negative value otherwise. >>> + */ >>> +__rte_experimental >>> +int >>> +rte_eth_bond_notify_member_flag_set(uint16_t bonding_port_id, bool >>> +notify_member); >> s/notify_membe/notify in input? >> because function name reveals the meaning already. > Thank you, you are right. > >>> + >>> +/** >>> + * Get the flag of whether bonding port notifies member ports. >>> + * >>> + * @param bonding_port_id >>> + * Port ID of bonding device. >>> + * @param notify_member >>> + * Flag of whether bonding port notifies member ports. >>> + * >>> + * @return >>> + * 0 on success, negative value otherwise. >>> + */ >>> +__rte_experimental >>> +int >>> +rte_eth_bond_notify_member_flag_get(uint16_t bonding_port_id, bool >>> +*notify_member); >>> + >>> +/** >>> + * Notify the member ports of bonding port's information. >>> + * >>> + * This interface is called in the following functions: >>> + * - bond_ethdev_lsc_event_callback() >>> + * - bond_ethdev_configure() >> Is this interface used just in these cases? >> If so I don't think it should be a API in eth_dev_op. > Sorry I'm a bit confused. > Do you mean "rte_eth_bond_notify_members" this interface? This interface is called in 3 functions. User can also call this API, right? How should the user use it? Is it when update bonding port information? I feel like to know what the time of the notice is. > >>> ... >>> + struct rte_eth_dev *member_dev[RTE_MAX_ETHPORTS]; >>> + >>> + if (valid_bonding_port_id(bonding_port_id) != 0) >>> + return -EINVAL; >>> + >>> + bond_dev = &rte_eth_devices[bonding_port_id]; >>> + internals = bond_dev->data->dev_private; >>> + >>> + for (i = 0; i < internals->member_count; i++) { >>> + member_port_id = internals->members[i].port_id; >>> + member_dev[i] = &rte_eth_devices[member_port_id]; >>> + if (*member_dev[i]->dev_ops->bond_notify_member == NULL) >>> + return -ENOTSUP; >>> + } >> Do we need all member ports support bond_notify_member api? >> If allow user to select one member port to notify? This might be more general. >> In addition, this action here is inconsistent with above handle(do not >> notify member if doesn't supportbond_notify_member API). > Yes, we do not need all member ports support this API. > I just want to have a stricter restriction on this feature before, but I think your suggestion is better. anyway, we need to have a clearly comments about this. > >>> ... ^ permalink raw reply [flat|nested] 80+ messages in thread
* RE: [PATCH v3 1/8] ethdev: add member notification for bonding port 2023-10-17 8:27 ` lihuisong (C) @ 2023-10-18 2:16 ` Chaoyong He 0 siblings, 0 replies; 80+ messages in thread From: Chaoyong He @ 2023-10-18 2:16 UTC (permalink / raw) To: lihuisong (C), Long Wu, dev, Ferruh Yigit; +Cc: oss-drivers, James Hershaw > 在 2023/10/9 11:11, Long Wu 写道: > >> Hi Chaoyong, > >> > >> some comments as below. > >> > >> > >> 在 2023/10/8 9:50, Chaoyong He 写道: > >>> From: Long Wu <long.wu@corigine.com> > >>> > >>> Bonding PMD does not let member ports know the bonding port's > >>> information, like how many member ports the bonding port has, what > >>> mode the bonding port is in and so on. > >>> > >>> Add the notification interface for bonding port to let member port > >>> know it is added to a bonding port and what the bonding port's > >>> configuration is. If so the member ports have chance to achieve its > >>> bond-flow-offlod or other private bonding functions. > >> "its bond-flow-offlod or other private bonding functions" > >> I wonder that what PMDs can do with this. > >> Can you give an example in PMD to help others review? > > After this patch series, I will send out nfp PMD code about "its bond-flow- > offlod or other private bonding functions ". > > I can explain here: > > "bond-flow" means the flow rule's destination port is a bonding port. > > Now DPDK can use bonding port as the source port in a flow rule and > member ports can offload this flow. > > But member ports cannot offload a flow that its destination port is a > bonding port. > > Because the member ports don't know the bonding port. (Of course, some > PMDs has its self-function to let member ports know the bonding port but it > doesn't a general "DPDK" way). > > After this "notify patch", DPDK can do "bond-flow-offload", also "other > private bonding functions"(like hardware balance policy, primary port > changing and so on) can work. > I think what you said is more like "bonding offload", right? > It seems that you cannot do it just based on current these API in this series. > You probably have other works to upstream for like "bond-flow" and "other > private bonding functions". > All the efforts are to make the NIC hardware aware that port has been added to the bonding port and then create a bonding port on hardware. Based on this(Bonding port notification and member port hardware create bonding port), we can do "bond-flow-offload" or "other private bonding functions". Taking "bond-flow-offload" as an example: Step1: bonding port configure process --> bonding port notify member ports --> member ports know bonding port configuration --> member ports hardware creates bonding port. Step2: app adds a flow rule about "bond-flow"(See the explanation above) --> PMD analyzes "bond-flow" then the hardware offloads this flow. (In the next patch series, and not affair the bonding PMD) > > > >>> Signed-off-by: Long Wu <long.wu@corigine.com> > >>> Reviewed-by: James Hershaw <james.hershaw@corigine.com> > >>> Reviewed-by: Chaoyong He <chaoyong.he@corigine.com> > >>> --- > >>> drivers/net/bonding/eth_bond_private.h | 1 + > >>> drivers/net/bonding/rte_eth_bond.h | 46 ++++++++++++++++ > >>> drivers/net/bonding/rte_eth_bond_api.c | 73 > >> ++++++++++++++++++++++++++ > >>> drivers/net/bonding/rte_eth_bond_pmd.c | 27 ++++++++-- > >>> drivers/net/bonding/version.map | 3 ++ > >>> lib/ethdev/ethdev_driver.h | 18 +++++++ > >>> 6 files changed, 165 insertions(+), 3 deletions(-) > >>> > >>> diff --git a/drivers/net/bonding/eth_bond_private.h > >>> b/drivers/net/bonding/eth_bond_private.h > >>> index e688894210..f69e85c199 100644 > >>> --- a/drivers/net/bonding/eth_bond_private.h > >>> +++ b/drivers/net/bonding/eth_bond_private.h > >>> @@ -180,6 +180,7 @@ struct bond_dev_private { > >>> uint8_t member_update_idx; > >>> > >>> bool kvargs_processing_is_done; > >>> + bool notify_member; /**< Enable member notification of bonding port. > >>> +*/ > >>> > >>> uint32_t candidate_max_rx_pktlen; > >>> uint32_t max_rx_pktlen; > >>> diff --git a/drivers/net/bonding/rte_eth_bond.h > >>> b/drivers/net/bonding/rte_eth_bond.h > >>> index f10165f2c6..737beca446 100644 > >>> --- a/drivers/net/bonding/rte_eth_bond.h > >>> +++ b/drivers/net/bonding/rte_eth_bond.h > >>> @@ -351,6 +351,52 @@ > rte_eth_bond_link_up_prop_delay_set(uint16_t > >> bonding_port_id, > >>> int > >>> rte_eth_bond_link_up_prop_delay_get(uint16_t bonding_port_id); > >>> > >>> +/** > >>> + * Set the flag of whether bonding port notifies member ports. > >>> + * > >>> + * @param bonding_port_id > >>> + * Port ID of bonding device. > >>> + * @param notify_member > >>> + * Flag of whether bonding port notifies member ports. > >>> + * > >>> + * @return > >>> + * 0 on success, negative value otherwise. > >>> + */ > >>> +__rte_experimental > >>> +int > >>> +rte_eth_bond_notify_member_flag_set(uint16_t bonding_port_id, bool > >>> +notify_member); > >> s/notify_membe/notify in input? > >> because function name reveals the meaning already. > > Thank you, you are right. > > > >>> + > >>> +/** > >>> + * Get the flag of whether bonding port notifies member ports. > >>> + * > >>> + * @param bonding_port_id > >>> + * Port ID of bonding device. > >>> + * @param notify_member > >>> + * Flag of whether bonding port notifies member ports. > >>> + * > >>> + * @return > >>> + * 0 on success, negative value otherwise. > >>> + */ > >>> +__rte_experimental > >>> +int > >>> +rte_eth_bond_notify_member_flag_get(uint16_t bonding_port_id, bool > >>> +*notify_member); > >>> + > >>> +/** > >>> + * Notify the member ports of bonding port's information. > >>> + * > >>> + * This interface is called in the following functions: > >>> + * - bond_ethdev_lsc_event_callback() > >>> + * - bond_ethdev_configure() > >> Is this interface used just in these cases? > >> If so I don't think it should be a API in eth_dev_op. > > Sorry I'm a bit confused. > > Do you mean "rte_eth_bond_notify_members" this interface? This interface > is called in 3 functions. > User can also call this API, right? Sorry, I think the user should not call API 'rte_eth_bond_notify_members()', it's the bonding PMD call it. There exist a so-called `notify_member_flag` in the bonding PMD, which control if the 'rte_eth_bond_notify_members()' will be invoked. And this `notify_member_flag` is controlled by ` rte_eth_bond_notify_member_flag_set()` and ` rte_eth_bond_notify_member_flag_get()`. > How should the user use it? Is it when update bonding port information? > I feel like to know what the time of the notice is. > > My commit "net/bonding: add commands for bonding port notification " in this patch series is for app to set the "notify flag". For example: testpmd> port stop all testpmd> create bonded device 4 1 testpmd> set bonding notify_member 2 enable. testpmd> set bonding mode 4 2 testpmd> set bonding balance_xmit_policy 2 l34. testpmd> add bonding member 0 2 testpmd> add bonding member 1 2 testpmd> port start 2 ("set bonding notify_member 2 enable " is a new command to set the "notify_member_flag".) I also add this setting in "vdev", like: --vdev 'net_bonding0,member=<port 1>,member=<port 2>,socket_id=1,xmit_policy=l34,notify_member=enable' > >>> ... > >>> + struct rte_eth_dev *member_dev[RTE_MAX_ETHPORTS]; > >>> + > >>> + if (valid_bonding_port_id(bonding_port_id) != 0) > >>> + return -EINVAL; > >>> + > >>> + bond_dev = &rte_eth_devices[bonding_port_id]; > >>> + internals = bond_dev->data->dev_private; > >>> + > >>> + for (i = 0; i < internals->member_count; i++) { > >>> + member_port_id = internals->members[i].port_id; > >>> + member_dev[i] = &rte_eth_devices[member_port_id]; > >>> + if (*member_dev[i]->dev_ops->bond_notify_member == NULL) > >>> + return -ENOTSUP; > >>> + } > >> Do we need all member ports support bond_notify_member api? > >> If allow user to select one member port to notify? This might be more > general. > >> In addition, this action here is inconsistent with above handle(do > >> not notify member if doesn't supportbond_notify_member API). > > Yes, we do not need all member ports support this API. > > I just want to have a stricter restriction on this feature before, but I think > your suggestion is better. > anyway, we need to have a clearly comments about this. Agree, I will add comments in next version. Thanks > > > >>> ... ^ permalink raw reply [flat|nested] 80+ messages in thread
* [PATCH v3 2/8] ethdev: add API to get hardware creation of bonding port 2023-10-08 1:50 ` [PATCH v3 0/8] Enhance the bond framework to support offload Chaoyong He 2023-10-08 1:50 ` [PATCH v3 1/8] ethdev: add member notification for bonding port Chaoyong He @ 2023-10-08 1:50 ` Chaoyong He 2023-10-08 1:50 ` [PATCH v3 3/8] net/bonding: modify interface comment format Chaoyong He ` (7 subsequent siblings) 9 siblings, 0 replies; 80+ messages in thread From: Chaoyong He @ 2023-10-08 1:50 UTC (permalink / raw) To: dev; +Cc: oss-drivers, Long Wu, James Hershaw, Chaoyong He From: Long Wu <long.wu@corigine.com> After bonding port notification, member port hardware may create the bonding port. We want to get the result of creatition, so we add this API to do the getting action. Signed-off-by: Long Wu <long.wu@corigine.com> Reviewed-by: James Hershaw <james.hershaw@corigine.com> Reviewed-by: Chaoyong He <chaoyong.he@corigine.com> --- drivers/net/bonding/rte_eth_bond.h | 15 ++++++++++++++ drivers/net/bonding/rte_eth_bond_api.c | 28 ++++++++++++++++++++++++++ drivers/net/bonding/version.map | 1 + lib/ethdev/ethdev_driver.h | 20 ++++++++++++++++++ 4 files changed, 64 insertions(+) diff --git a/drivers/net/bonding/rte_eth_bond.h b/drivers/net/bonding/rte_eth_bond.h index 737beca446..6be5e46deb 100644 --- a/drivers/net/bonding/rte_eth_bond.h +++ b/drivers/net/bonding/rte_eth_bond.h @@ -398,6 +398,21 @@ __rte_experimental int rte_eth_bond_notify_members(uint16_t bonding_port_id); +/** + * Get the status of specified bonding port created by member port hardware. + * + * @param bonding_port_id + * Port ID of bonding device. + * @param member_port_id + * Port ID of member device. + * + * @return + * 0 on success, negative value otherwise. + */ +__rte_experimental +int +rte_eth_bond_hw_create_get(uint16_t bonding_port_id, uint16_t member_port_id); + #ifdef __cplusplus } #endif diff --git a/drivers/net/bonding/rte_eth_bond_api.c b/drivers/net/bonding/rte_eth_bond_api.c index 48c1d050fd..0be580b19b 100644 --- a/drivers/net/bonding/rte_eth_bond_api.c +++ b/drivers/net/bonding/rte_eth_bond_api.c @@ -1171,3 +1171,31 @@ rte_eth_bond_notify_members(uint16_t bonding_port_id) return 0; } + +int +rte_eth_bond_hw_create_get(uint16_t bonding_port_id, uint16_t member_port_id) +{ + uint32_t i; + struct rte_eth_dev *bonding_dev; + struct rte_eth_dev *member_dev; + struct bond_dev_private *internals; + + if (valid_bonding_port_id(bonding_port_id) != 0) + return -EINVAL; + + bonding_dev = &rte_eth_devices[bonding_port_id]; + internals = bonding_dev->data->dev_private; + for (i = 0; i < internals->member_count; i++) { + if (internals->members[i].port_id == member_port_id) + break; + } + + if (i == internals->member_count) + return -EINVAL; + + member_dev = &rte_eth_devices[member_port_id]; + if (*member_dev->dev_ops->bond_hw_create_get == NULL) + return -ENOTSUP; + + return member_dev->dev_ops->bond_hw_create_get(member_dev, bonding_dev); +} diff --git a/drivers/net/bonding/version.map b/drivers/net/bonding/version.map index 3bd5e8ad11..3cfff51269 100644 --- a/drivers/net/bonding/version.map +++ b/drivers/net/bonding/version.map @@ -32,6 +32,7 @@ EXPERIMENTAL { global: rte_eth_bond_8023ad_member_info; rte_eth_bond_active_members_get; + rte_eth_bond_hw_create_get; rte_eth_bond_member_add; rte_eth_bond_member_remove; rte_eth_bond_members_get; diff --git a/lib/ethdev/ethdev_driver.h b/lib/ethdev/ethdev_driver.h index f626f971e5..18ff5db969 100644 --- a/lib/ethdev/ethdev_driver.h +++ b/lib/ethdev/ethdev_driver.h @@ -1231,6 +1231,21 @@ typedef int (*eth_map_aggr_tx_affinity_t)(struct rte_eth_dev *dev, uint16_t tx_q typedef int (*eth_bond_notify_member)(struct rte_eth_dev *dev, struct rte_eth_dev *bonding_dev); +/** + * @internal + * Get the status of specified bonding port created by member port hardware. + * + * @param dev + * Member port (ethdev) handle. + * @param bonding_dev + * Bonding port (ethdev) handle. + * + * @return + * Negative on error, 0 on success. + */ +typedef int (*eth_bond_hw_create_get)(struct rte_eth_dev *dev, + struct rte_eth_dev *bonding_dev); + /** * @internal A structure containing the functions exported by an Ethernet driver. */ @@ -1473,6 +1488,11 @@ struct eth_dev_ops { /** Notify the member port of bonding port information */ eth_bond_notify_member bond_notify_member; + /** + * Get the status of whether bonding port is successfully created by + * the member port hardware. + */ + eth_bond_hw_create_get bond_hw_create_get; }; /** -- 2.39.1 ^ permalink raw reply [flat|nested] 80+ messages in thread
* [PATCH v3 3/8] net/bonding: modify interface comment format 2023-10-08 1:50 ` [PATCH v3 0/8] Enhance the bond framework to support offload Chaoyong He 2023-10-08 1:50 ` [PATCH v3 1/8] ethdev: add member notification for bonding port Chaoyong He 2023-10-08 1:50 ` [PATCH v3 2/8] ethdev: add API to get hardware creation of " Chaoyong He @ 2023-10-08 1:50 ` Chaoyong He 2023-10-08 1:50 ` [PATCH v3 4/8] net/bonding: add bonding port arguments Chaoyong He ` (6 subsequent siblings) 9 siblings, 0 replies; 80+ messages in thread From: Chaoyong He @ 2023-10-08 1:50 UTC (permalink / raw) To: dev; +Cc: oss-drivers, Long Wu, James Hershaw, Chaoyong He From: Long Wu <long.wu@corigine.com> Most of the previous interface comment format does not meet the current standards and were not uniform. Modify them to meet current standards. Signed-off-by: Long Wu <long.wu@corigine.com> Reviewed-by: James Hershaw <james.hershaw@corigine.com> Reviewed-by: Chaoyong He <chaoyong.he@corigine.com> --- drivers/net/bonding/eth_bond_8023ad_private.h | 52 +++--- drivers/net/bonding/rte_eth_bond.h | 150 +++++++++++------- drivers/net/bonding/rte_eth_bond_8023ad.h | 76 ++++++--- drivers/net/bonding/rte_eth_bond_alb.h | 34 ++-- 4 files changed, 199 insertions(+), 113 deletions(-) diff --git a/drivers/net/bonding/eth_bond_8023ad_private.h b/drivers/net/bonding/eth_bond_8023ad_private.h index ab7d15f81a..29ee8700c3 100644 --- a/drivers/net/bonding/eth_bond_8023ad_private.h +++ b/drivers/net/bonding/eth_bond_8023ad_private.h @@ -196,10 +196,13 @@ struct bond_dev_private; * * Set mode 4 configuration of bonding interface. * - * @pre Bonding interface must be stopped. + * @pre + * Bonding interface must be stopped. * - * @param dev Bonding interface - * @param conf new configuration. If NULL set default configuration. + * @param dev + * Bonding interface + * @param conf + * new configuration. If NULL set default configuration. */ void bond_mode_8023ad_setup(struct rte_eth_dev *dev, @@ -210,9 +213,11 @@ bond_mode_8023ad_setup(struct rte_eth_dev *dev, * * Enables 802.1AX mode and all active members on bonding interface. * - * @param dev Bonding interface + * @param dev + * Bonding interface + * * @return - * 0 on success, negative value otherwise. + * 0 on success, negative value otherwise. */ int bond_mode_8023ad_enable(struct rte_eth_dev *dev); @@ -222,7 +227,8 @@ bond_mode_8023ad_enable(struct rte_eth_dev *dev); * * Disables 802.1AX mode of the bonding interface and members. * - * @param dev Bonding interface + * @param dev + * Bonding interface * @return * 0 on success, negative value otherwise. */ @@ -232,7 +238,8 @@ int bond_mode_8023ad_disable(struct rte_eth_dev *dev); * @internal * * Starts 802.3AX state machines management logic. - * @param dev Bonding interface + * @param dev + * Bonding interface * @return * 0 if machines was started, 1 if machines was already running, * negative value otherwise. @@ -244,7 +251,8 @@ bond_mode_8023ad_start(struct rte_eth_dev *dev); * @internal * * Stops 802.3AX state machines management logic. - * @param dev Bonding interface + * @param dev + * Bonding interface * @return * 0 if this call stopped state machines, -ENOENT if alarm was not set. */ @@ -255,9 +263,12 @@ bond_mode_8023ad_stop(struct rte_eth_dev *dev); * @internal * * Passes given slow packet to state machines management logic. - * @param internals Bonding device private data. - * @param member_id Member port id. - * @param slot_pkt Slow packet. + * @param internals + * Bonding device private data. + * @param member_id + * Member port id. + * @param slot_pkt + * Slow packet. */ void bond_mode_8023ad_handle_slow_pkt(struct bond_dev_private *internals, @@ -268,11 +279,13 @@ bond_mode_8023ad_handle_slow_pkt(struct bond_dev_private *internals, * * Appends given member used member * - * @param dev Bonding interface. - * @param port_id Member port ID to be added + * @param dev + * Bonding interface. + * @param port_id + * Member port ID to be added * * @return - * 0 on success, negative value otherwise. + * 0 on success, negative value otherwise. */ void bond_mode_8023ad_activate_member(struct rte_eth_dev *dev, uint16_t port_id); @@ -282,18 +295,21 @@ bond_mode_8023ad_activate_member(struct rte_eth_dev *dev, uint16_t port_id); * * Denitializes and removes given member from 802.1AX mode. * - * @param dev Bonding interface. - * @param member_num Position of member in active_members array + * @param dev + * Bonding interface. + * @param member_num + * Position of member in active_members array * * @return - * 0 on success, negative value otherwise. + * 0 on success, negative value otherwise. */ int bond_mode_8023ad_deactivate_member(struct rte_eth_dev *dev, uint16_t member_pos); /** * Updates state when MAC was changed on bonding device or one of its members. - * @param bond_dev Bonding device + * @param bond_dev + * Bonding device */ void bond_mode_8023ad_mac_address_update(struct rte_eth_dev *bond_dev); diff --git a/drivers/net/bonding/rte_eth_bond.h b/drivers/net/bonding/rte_eth_bond.h index 6be5e46deb..28aa341d2f 100644 --- a/drivers/net/bonding/rte_eth_bond.h +++ b/drivers/net/bonding/rte_eth_bond.h @@ -95,12 +95,15 @@ extern "C" { /** * Create a bonding rte_eth_dev device * - * @param name Name of new link bonding device. - * @param mode Mode to initialize bonding device in. - * @param socket_id Socket Id on which to allocate eth_dev resources. + * @param name + * Name of new link bonding device. + * @param mode + * Mode to initialize bonding device in. + * @param socket_id + * Socket Id on which to allocate eth_dev resources. * * @return - * Port Id of created rte_eth_dev on success, negative value otherwise + * Port Id of created rte_eth_dev on success, negative value otherwise */ int rte_eth_bond_create(const char *name, uint8_t mode, uint8_t socket_id); @@ -108,10 +111,11 @@ rte_eth_bond_create(const char *name, uint8_t mode, uint8_t socket_id); /** * Free a bonding rte_eth_dev device * - * @param name Name of the link bonding device. + * @param name + * Name of the link bonding device. * * @return - * 0 on success, negative value otherwise + * 0 on success, negative value otherwise */ int rte_eth_bond_free(const char *name); @@ -119,11 +123,13 @@ rte_eth_bond_free(const char *name); /** * Add a rte_eth_dev device as a member to the bonding device * - * @param bonding_port_id Port ID of bonding device. - * @param member_port_id Port ID of member device. + * @param bonding_port_id + * Port ID of bonding device. + * @param member_port_id + * Port ID of member device. * * @return - * 0 on success, negative value otherwise + * 0 on success, negative value otherwise */ __rte_experimental int @@ -132,11 +138,13 @@ rte_eth_bond_member_add(uint16_t bonding_port_id, uint16_t member_port_id); /** * Remove a member rte_eth_dev device from the bonding device * - * @param bonding_port_id Port ID of bonding device. - * @param member_port_id Port ID of member device. + * @param bonding_port_id + * Port ID of bonding device. + * @param member_port_id + * Port ID of member device. * * @return - * 0 on success, negative value otherwise + * 0 on success, negative value otherwise */ __rte_experimental int @@ -145,11 +153,13 @@ rte_eth_bond_member_remove(uint16_t bonding_port_id, uint16_t member_port_id); /** * Set link bonding mode of bonding device * - * @param bonding_port_id Port ID of bonding device. - * @param mode Bonding mode to set + * @param bonding_port_id + * Port ID of bonding device. + * @param mode + * Bonding mode to set * * @return - * 0 on success, negative value otherwise + * 0 on success, negative value otherwise */ int rte_eth_bond_mode_set(uint16_t bonding_port_id, uint8_t mode); @@ -157,10 +167,11 @@ rte_eth_bond_mode_set(uint16_t bonding_port_id, uint8_t mode); /** * Get link bonding mode of bonding device * - * @param bonding_port_id Port ID of bonding device. + * @param bonding_port_id + * Port ID of bonding device. * * @return - * link bonding mode on success, negative value otherwise + * link bonding mode on success, negative value otherwise */ int rte_eth_bond_mode_get(uint16_t bonding_port_id); @@ -168,11 +179,13 @@ rte_eth_bond_mode_get(uint16_t bonding_port_id); /** * Set member rte_eth_dev as primary member of bonding device * - * @param bonding_port_id Port ID of bonding device. - * @param member_port_id Port ID of member device. + * @param bonding_port_id + * Port ID of bonding device. + * @param member_port_id + * Port ID of member device. * * @return - * 0 on success, negative value otherwise + * 0 on success, negative value otherwise */ int rte_eth_bond_primary_set(uint16_t bonding_port_id, uint16_t member_port_id); @@ -180,10 +193,11 @@ rte_eth_bond_primary_set(uint16_t bonding_port_id, uint16_t member_port_id); /** * Get primary member of bonding device * - * @param bonding_port_id Port ID of bonding device. + * @param bonding_port_id + * Port ID of bonding device. * * @return - * Port Id of primary member on success, -1 on failure + * Port Id of primary member on success, -1 on failure */ int rte_eth_bond_primary_get(uint16_t bonding_port_id); @@ -191,13 +205,16 @@ rte_eth_bond_primary_get(uint16_t bonding_port_id); /** * Populate an array with list of the members port id's of the bonding device * - * @param bonding_port_id Port ID of bonding eth_dev to interrogate - * @param members Array to be populated with the current active members - * @param len Length of members array + * @param bonding_port_id + * Port ID of bonding eth_dev to interrogate + * @param members + * Array to be populated with the current active members + * @param len + * Length of members array * * @return - * Number of members associated with bonding device on success, - * negative value otherwise + * Number of members associated with bonding device on success, + * negative value otherwise */ __rte_experimental int @@ -208,13 +225,16 @@ rte_eth_bond_members_get(uint16_t bonding_port_id, uint16_t members[], * Populate an array with list of the active members port id's of the bonding * device. * - * @param bonding_port_id Port ID of bonding eth_dev to interrogate - * @param members Array to be populated with the current active members - * @param len Length of members array + * @param bonding_port_id + * Port ID of bonding eth_dev to interrogate + * @param members + * Array to be populated with the current active members + * @param len + * Length of members array * * @return - * Number of active members associated with bonding device on success, - * negative value otherwise + * Number of active members associated with bonding device on success, + * negative value otherwise */ __rte_experimental int @@ -224,12 +244,13 @@ rte_eth_bond_active_members_get(uint16_t bonding_port_id, uint16_t members[], /** * Set explicit MAC address to use on bonding device and it's members. * - * @param bonding_port_id Port ID of bonding device. - * @param mac_addr MAC Address to use on bonding device overriding - * members MAC addresses + * @param bonding_port_id + * Port ID of bonding device. + * @param mac_addr + * MAC Address to use on bonding device overriding members MAC addresses * * @return - * 0 on success, negative value otherwise + * 0 on success, negative value otherwise */ int rte_eth_bond_mac_address_set(uint16_t bonding_port_id, @@ -239,10 +260,11 @@ rte_eth_bond_mac_address_set(uint16_t bonding_port_id, * Reset bonding device to use MAC from primary member on bonding device and it's * members. * - * @param bonding_port_id Port ID of bonding device. + * @param bonding_port_id + * Port ID of bonding device. * * @return - * 0 on success, negative value otherwise + * 0 on success, negative value otherwise */ int rte_eth_bond_mac_address_reset(uint16_t bonding_port_id); @@ -252,11 +274,13 @@ rte_eth_bond_mac_address_reset(uint16_t bonding_port_id); * balance mode, this parameter is otherwise ignored in other modes of * operation. * - * @param bonding_port_id Port ID of bonding device. - * @param policy Balance mode transmission policy. + * @param bonding_port_id + * Port ID of bonding device. + * @param policy + * Balance mode transmission policy. * * @return - * 0 on success, negative value otherwise. + * 0 on success, negative value otherwise. */ int rte_eth_bond_xmit_policy_set(uint16_t bonding_port_id, uint8_t policy); @@ -264,10 +288,11 @@ rte_eth_bond_xmit_policy_set(uint16_t bonding_port_id, uint8_t policy); /** * Get the transmit policy set on bonding device for balance mode operation * - * @param bonding_port_id Port ID of bonding device. + * @param bonding_port_id + * Port ID of bonding device. * * @return - * Balance transmit policy on success, negative value otherwise. + * Balance transmit policy on success, negative value otherwise. */ int rte_eth_bond_xmit_policy_get(uint16_t bonding_port_id); @@ -276,11 +301,13 @@ rte_eth_bond_xmit_policy_get(uint16_t bonding_port_id); * Set the link monitoring frequency (in ms) for monitoring the link status of * member devices * - * @param bonding_port_id Port ID of bonding device. - * @param internal_ms Monitoring interval in milliseconds + * @param bonding_port_id + * Port ID of bonding device. + * @param internal_ms + * Monitoring interval in milliseconds * * @return - * 0 on success, negative value otherwise. + * 0 on success, negative value otherwise. */ int @@ -290,10 +317,11 @@ rte_eth_bond_link_monitoring_set(uint16_t bonding_port_id, uint32_t internal_ms) * Get the current link monitoring frequency (in ms) for monitoring of the link * status of member devices * - * @param bonding_port_id Port ID of bonding device. + * @param bonding_port_id + * Port ID of bonding device. * * @return - * Monitoring interval on success, negative value otherwise. + * Monitoring interval on success, negative value otherwise. */ int rte_eth_bond_link_monitoring_get(uint16_t bonding_port_id); @@ -303,11 +331,13 @@ rte_eth_bond_link_monitoring_get(uint16_t bonding_port_id); * Set the period in milliseconds for delaying the disabling of a bonding link * when the link down status has been detected * - * @param bonding_port_id Port ID of bonding device. - * @param delay_ms Delay period in milliseconds. + * @param bonding_port_id + * Port ID of bonding device. + * @param delay_ms + * Delay period in milliseconds. * * @return - * 0 on success, negative value otherwise. + * 0 on success, negative value otherwise. */ int rte_eth_bond_link_down_prop_delay_set(uint16_t bonding_port_id, @@ -317,10 +347,11 @@ rte_eth_bond_link_down_prop_delay_set(uint16_t bonding_port_id, * Get the period in milliseconds set for delaying the disabling of a bonding * link when the link down status has been detected * - * @param bonding_port_id Port ID of bonding device. + * @param bonding_port_id + * Port ID of bonding device. * * @return - * Delay period on success, negative value otherwise. + * Delay period on success, negative value otherwise. */ int rte_eth_bond_link_down_prop_delay_get(uint16_t bonding_port_id); @@ -329,11 +360,13 @@ rte_eth_bond_link_down_prop_delay_get(uint16_t bonding_port_id); * Set the period in milliseconds for delaying the enabling of a bonding link * when the link up status has been detected * - * @param bonding_port_id Port ID of bonding device. - * @param delay_ms Delay period in milliseconds. + * @param bonding_port_id + * Port ID of bonding device. + * @param delay_ms + * Delay period in milliseconds. * * @return - * 0 on success, negative value otherwise. + * 0 on success, negative value otherwise. */ int rte_eth_bond_link_up_prop_delay_set(uint16_t bonding_port_id, @@ -343,10 +376,11 @@ rte_eth_bond_link_up_prop_delay_set(uint16_t bonding_port_id, * Get the period in milliseconds set for delaying the enabling of a bonding * link when the link up status has been detected * - * @param bonding_port_id Port ID of bonding device. + * @param bonding_port_id + * Port ID of bonding device. * * @return - * Delay period on success, negative value otherwise. + * Delay period on success, negative value otherwise. */ int rte_eth_bond_link_up_prop_delay_get(uint16_t bonding_port_id); diff --git a/drivers/net/bonding/rte_eth_bond_8023ad.h b/drivers/net/bonding/rte_eth_bond_8023ad.h index 4c280c7565..4936d7fc0c 100644 --- a/drivers/net/bonding/rte_eth_bond_8023ad.h +++ b/drivers/net/bonding/rte_eth_bond_8023ad.h @@ -155,8 +155,10 @@ struct rte_eth_bond_8023ad_member_info { * * Function returns current configuration of 802.3AX mode. * - * @param port_id Bonding device id - * @param conf Pointer to timeout structure. + * @param port_id + * Bonding device id + * @param conf + * Pointer to timeout structure. * * @return * 0 - if ok @@ -171,8 +173,10 @@ rte_eth_bond_8023ad_conf_get(uint16_t port_id, * * Function set new configuration of 802.3AX mode. * - * @param port_id Bonding device id - * @param conf Configuration, if NULL set default configuration. + * @param port_id + * Bonding device id + * @param conf + * Configuration, if NULL set default configuration. * @return * 0 - if ok * -EINVAL if configuration is invalid. @@ -186,8 +190,10 @@ rte_eth_bond_8023ad_setup(uint16_t port_id, * * Function returns current state of given member device. * - * @param member_id Port id of valid member. - * @param conf buffer for configuration + * @param member_id + * Port id of valid member. + * @param conf + * buffer for configuration * @return * 0 - if ok * -EINVAL if conf is NULL or member id is invalid (not a member of given @@ -201,9 +207,12 @@ rte_eth_bond_8023ad_member_info(uint16_t port_id, uint16_t member_id, /** * Configure a member port to start collecting. * - * @param port_id Bonding device id - * @param member_id Port id of valid member. - * @param enabled Non-zero when collection enabled. + * @param port_id + * Bonding device id + * @param member_id + * Port id of valid member. + * @param enabled + * Non-zero when collection enabled. * @return * 0 - if ok * -EINVAL if member is not valid. @@ -215,8 +224,10 @@ rte_eth_bond_8023ad_ext_collect(uint16_t port_id, uint16_t member_id, /** * Get COLLECTING flag from member port actor state. * - * @param port_id Bonding device id - * @param member_id Port id of valid member. + * @param port_id + * Bonding device id + * @param member_id + * Port id of valid member. * @return * 0 - if not set * 1 - if set @@ -228,9 +239,12 @@ rte_eth_bond_8023ad_ext_collect_get(uint16_t port_id, uint16_t member_id); /** * Configure a member port to start distributing. * - * @param port_id Bonding device id - * @param member_id Port id of valid member. - * @param enabled Non-zero when distribution enabled. + * @param port_id + * Bonding device id + * @param member_id + * Port id of valid member. + * @param enabled + * Non-zero when distribution enabled. * @return * 0 - if ok * -EINVAL if member is not valid. @@ -242,8 +256,10 @@ rte_eth_bond_8023ad_ext_distrib(uint16_t port_id, uint16_t member_id, /** * Get DISTRIBUTING flag from member port actor state. * - * @param port_id Bonding device id - * @param member_id Port id of valid member. + * @param port_id + * Bonding device id + * @param member_id + * Port id of valid member. * @return * 0 - if not set * 1 - if set @@ -256,9 +272,12 @@ rte_eth_bond_8023ad_ext_distrib_get(uint16_t port_id, uint16_t member_id); * LACPDU transmit path for external 802.3ad state machine. Caller retains * ownership of the packet on failure. * - * @param port_id Bonding device id - * @param member_id Port ID of valid member device. - * @param lacp_pkt mbuf containing LACPDU. + * @param port_id + * Bonding device id + * @param member_id + * Port ID of valid member device. + * @param lacp_pkt + * mbuf containing LACPDU. * * @return * 0 on success, negative value otherwise. @@ -284,7 +303,8 @@ rte_eth_bond_8023ad_ext_slowtx(uint16_t port_id, uint16_t member_id, * * Bonding port must be stopped to change this configuration. * - * @param port_id Bonding device id + * @param port_id + * Bonding device id * * @return * 0 on success, negative value otherwise. @@ -299,9 +319,11 @@ rte_eth_bond_8023ad_dedicated_queues_enable(uint16_t port_id); * * Bonding port must be stopped to change this configuration. * - * @see rte_eth_bond_8023ad_slow_pkt_hw_filter_enable + * @see + * rte_eth_bond_8023ad_slow_pkt_hw_filter_enable * - * @param port_id Bonding device id + * @param port_id + * Bonding device id * @return * 0 on success, negative value otherwise. * @@ -309,9 +331,11 @@ rte_eth_bond_8023ad_dedicated_queues_enable(uint16_t port_id); int rte_eth_bond_8023ad_dedicated_queues_disable(uint16_t port_id); -/* +/** * Get aggregator mode for 8023ad - * @param port_id Bonding device id + * + * @param port_id + * Bonding device id * * @return * aggregator mode on success, negative value otherwise @@ -321,7 +345,9 @@ rte_eth_bond_8023ad_agg_selection_get(uint16_t port_id); /** * Set aggregator mode for 8023ad - * @param port_id Bonding device id + * @param port_id + * Bonding device id + * * @return * 0 on success, negative value otherwise */ diff --git a/drivers/net/bonding/rte_eth_bond_alb.h b/drivers/net/bonding/rte_eth_bond_alb.h index f51c355d36..3cf0d87750 100644 --- a/drivers/net/bonding/rte_eth_bond_alb.h +++ b/drivers/net/bonding/rte_eth_bond_alb.h @@ -63,9 +63,12 @@ bond_mode_alb_enable(struct rte_eth_dev *bond_dev); * forwarded to application without changes. If it is ARP reply, client table * is updated. * - * @param eth_h ETH header of received packet. - * @param offset Vlan header offset. - * @param internals Bonding data. + * @param eth_h + * ETH header of received packet. + * @param offset + * Vlan header offset. + * @param internals + * Bonding data. */ void bond_mode_alb_arp_recv(struct rte_ether_hdr *eth_h, uint16_t offset, @@ -77,12 +80,15 @@ bond_mode_alb_arp_recv(struct rte_ether_hdr *eth_h, uint16_t offset, * If it is ARP Reply, it is send on member stored in client table for that * connection. On Reply function also updates data in client table. * - * @param eth_h ETH header of transmitted packet. - * @param offset Vlan header offset. - * @param internals Bonding data. + * @param eth_h + * ETH header of transmitted packet. + * @param offset + * Vlan header offset. + * @param internals + * Bonding data. * * @return - * Index of member on which packet should be sent. + * Index of member on which packet should be sent. */ uint16_t bond_mode_alb_arp_xmit(struct rte_ether_hdr *eth_h, uint16_t offset, @@ -91,12 +97,15 @@ bond_mode_alb_arp_xmit(struct rte_ether_hdr *eth_h, uint16_t offset, /** * Function fills packet with ARP data from client_info. * - * @param client_info Data of client to which packet is sent. - * @param pkt Pointer to packet which is sent. - * @param internals Bonding data. + * @param client_info + * Data of client to which packet is sent. + * @param pkt + * Pointer to packet which is sent. + * @param internals + * Bonding data. * * @return - * Index of member on which packet should be sent. + * Index of member on which packet should be sent. */ uint16_t bond_mode_alb_arp_upd(struct client_data *client_info, @@ -105,7 +114,8 @@ bond_mode_alb_arp_upd(struct client_data *client_info, /** * Function updates member indexes of active connections. * - * @param bond_dev Pointer to bonding device struct. + * @param bond_dev + * Pointer to bonding device struct. */ void bond_mode_alb_client_list_upd(struct rte_eth_dev *bond_dev); -- 2.39.1 ^ permalink raw reply [flat|nested] 80+ messages in thread
* [PATCH v3 4/8] net/bonding: add bonding port arguments 2023-10-08 1:50 ` [PATCH v3 0/8] Enhance the bond framework to support offload Chaoyong He ` (2 preceding siblings ...) 2023-10-08 1:50 ` [PATCH v3 3/8] net/bonding: modify interface comment format Chaoyong He @ 2023-10-08 1:50 ` Chaoyong He 2023-10-08 1:50 ` [PATCH v3 5/8] net/bonding: support add port by data name Chaoyong He ` (5 subsequent siblings) 9 siblings, 0 replies; 80+ messages in thread From: Chaoyong He @ 2023-10-08 1:50 UTC (permalink / raw) To: dev; +Cc: oss-drivers, Long Wu, James Hershaw, Chaoyong He From: Long Wu <long.wu@corigine.com> Include the following new arguments for bonding ports: - "notify_member" to enable/disable member notification. - "dedicated_queue" to enable/disable dedicated queue. Add these two arguments in initial argument. Signed-off-by: Long Wu <long.wu@corigine.com> Reviewed-by: James Hershaw <james.hershaw@corigine.com> Reviewed-by: Chaoyong He <chaoyong.he@corigine.com> --- drivers/net/bonding/eth_bond_private.h | 10 ++++ drivers/net/bonding/rte_eth_bond.h | 14 ++++++ drivers/net/bonding/rte_eth_bond_api.c | 14 ++++++ drivers/net/bonding/rte_eth_bond_args.c | 44 ++++++++++++++++++ drivers/net/bonding/rte_eth_bond_pmd.c | 61 ++++++++++++++++++++++++- 5 files changed, 142 insertions(+), 1 deletion(-) diff --git a/drivers/net/bonding/eth_bond_private.h b/drivers/net/bonding/eth_bond_private.h index f69e85c199..f9603a0f6b 100644 --- a/drivers/net/bonding/eth_bond_private.h +++ b/drivers/net/bonding/eth_bond_private.h @@ -28,6 +28,8 @@ #define PMD_BOND_LSC_POLL_PERIOD_KVARG ("lsc_poll_period_ms") #define PMD_BOND_LINK_UP_PROP_DELAY_KVARG ("up_delay") #define PMD_BOND_LINK_DOWN_PROP_DELAY_KVARG ("down_delay") +#define PMD_BOND_NOTIFY_MEMBER_KVARG ("notify_member") +#define PMD_BOND_DEDICATED_QUEUE_KVARG ("dedicated_queue") #define PMD_BOND_XMIT_POLICY_LAYER2_KVARG ("l2") #define PMD_BOND_XMIT_POLICY_LAYER23_KVARG ("l23") @@ -319,6 +321,14 @@ int bond_ethdev_parse_time_ms_kvarg(const char *key, const char *value, void *extra_args); +int +bond_ethdev_parse_notify_member_kvarg(const char *key __rte_unused, + const char *value, void *extra_args); + +int +bond_ethdev_parse_dedicated_queue_kvarg(const char *key __rte_unused, + const char *value, void *extra_args); + void bond_tlb_disable(struct bond_dev_private *internals); diff --git a/drivers/net/bonding/rte_eth_bond.h b/drivers/net/bonding/rte_eth_bond.h index 28aa341d2f..3f427b6bab 100644 --- a/drivers/net/bonding/rte_eth_bond.h +++ b/drivers/net/bonding/rte_eth_bond.h @@ -385,6 +385,20 @@ rte_eth_bond_link_up_prop_delay_set(uint16_t bonding_port_id, int rte_eth_bond_link_up_prop_delay_get(uint16_t bonding_port_id); +/** + * Set the flag that whether bonding device enable dedicated queue. + * + * @param bonding_port_id + * Port ID of bonding device. + * @param queue_flag + * The flag of enable bond dedicated queue + * + * @return + * 0 on success, negative value otherwise. + */ +int +rte_eth_bond_dedicated_queue_flag_set(uint16_t bonding_port_id, bool queue_flag); + /** * Set the flag of whether bonding port notifies member ports. * diff --git a/drivers/net/bonding/rte_eth_bond_api.c b/drivers/net/bonding/rte_eth_bond_api.c index 0be580b19b..a042f05a4c 100644 --- a/drivers/net/bonding/rte_eth_bond_api.c +++ b/drivers/net/bonding/rte_eth_bond_api.c @@ -1114,6 +1114,20 @@ rte_eth_bond_link_up_prop_delay_get(uint16_t bonding_port_id) return internals->link_up_delay_ms; } +int +rte_eth_bond_dedicated_queue_flag_set(uint16_t bonding_port_id, bool queue_flag) +{ + struct bond_dev_private *internals; + + if (valid_bonding_port_id(bonding_port_id) != 0) + return -1; + + internals = rte_eth_devices[bonding_port_id].data->dev_private; + internals->mode4.dedicated_queues.enabled = queue_flag; + + return 0; +} + int rte_eth_bond_notify_member_flag_set(uint16_t bonding_port_id, bool notify_member) { diff --git a/drivers/net/bonding/rte_eth_bond_args.c b/drivers/net/bonding/rte_eth_bond_args.c index bdec5d61d4..8a3e4656ef 100644 --- a/drivers/net/bonding/rte_eth_bond_args.c +++ b/drivers/net/bonding/rte_eth_bond_args.c @@ -20,6 +20,8 @@ const char *pmd_bond_init_valid_arguments[] = { PMD_BOND_MAC_ADDR_KVARG, PMD_BOND_AGG_MODE_KVARG, RTE_DEVARGS_KEY_DRIVER, + PMD_BOND_NOTIFY_MEMBER_KVARG, + PMD_BOND_DEDICATED_QUEUE_KVARG, NULL }; @@ -297,3 +299,45 @@ bond_ethdev_parse_time_ms_kvarg(const char *key __rte_unused, return 0; } + +int +bond_ethdev_parse_notify_member_kvarg(const char *key __rte_unused, + const char *value, void *extra_args) +{ + bool *notify_member; + + if (value == NULL || extra_args == NULL) + return -1; + + notify_member = extra_args; + + if (strcmp("enable", value) == 0) + *notify_member = true; + else if (strcmp("disable", value) == 0) + *notify_member = false; + else + return -1; + + return 0; +} + +int +bond_ethdev_parse_dedicated_queue_kvarg(const char *key __rte_unused, + const char *value, void *extra_args) +{ + bool *dedicated_queue; + + if (value == NULL || extra_args == NULL) + return -1; + + dedicated_queue = extra_args; + + if (strcmp("enable", value) == 0) + *dedicated_queue = true; + else if (strcmp("disable", value) == 0) + *dedicated_queue = false; + else + return -1; + + return 0; +} diff --git a/drivers/net/bonding/rte_eth_bond_pmd.c b/drivers/net/bonding/rte_eth_bond_pmd.c index b99b8b8938..1ebeb270c8 100644 --- a/drivers/net/bonding/rte_eth_bond_pmd.c +++ b/drivers/net/bonding/rte_eth_bond_pmd.c @@ -4227,6 +4227,63 @@ bond_ethdev_configure(struct rte_eth_dev *dev) return -1; } + /* Parse/set notify member flag */ + arg_count = rte_kvargs_count(kvlist, PMD_BOND_NOTIFY_MEMBER_KVARG); + if (arg_count == 1) { + bool notify_member; + if (rte_kvargs_process(kvlist, + PMD_BOND_NOTIFY_MEMBER_KVARG, + &bond_ethdev_parse_notify_member_kvarg, + ¬ify_member) < 0) { + RTE_BOND_LOG(ERR, + "Invalid notify member value specified" + " for bonding device %s", name); + return -1; + } + + if (rte_eth_bond_notify_member_flag_set(port_id, notify_member) != 0) { + RTE_BOND_LOG(ERR, + "Failed to set notify member (%u) on" + " bonding device %s", notify_member, name); + return -1; + } + } else if (arg_count > 1) { + RTE_BOND_LOG(ERR, + "notify member flag can be specified only once" + " for bonding device %s", name); + return -1; + } + + /* Parse/set dedicated queue flag */ + arg_count = rte_kvargs_count(kvlist, PMD_BOND_DEDICATED_QUEUE_KVARG); + if (arg_count == 1) { + bool dedicated_queue; + if (rte_kvargs_process(kvlist, + PMD_BOND_DEDICATED_QUEUE_KVARG, + &bond_ethdev_parse_dedicated_queue_kvarg, + &dedicated_queue) < 0) { + RTE_BOND_LOG(ERR, + "Invalid dedicated queue flag specified" + " for bonding device %s", name); + return -1; + } + + if (internals->mode == BONDING_MODE_8023AD) { + if (rte_eth_bond_dedicated_queue_flag_set(port_id, dedicated_queue) != 0) { + RTE_BOND_LOG(ERR, + "Failed to enable/disable dedicated" + " queue flag on bonding device %s", + name); + return -1; + } + } + } else if (arg_count > 1) { + RTE_BOND_LOG(ERR, + "dedicated queue flag can be specified only once" + " for bonding device %s", name); + return -1; + } + /* configure members so we can pass mtu setting */ for (i = 0; i < internals->member_count; i++) { struct rte_eth_dev *member_ethdev = @@ -4264,7 +4321,9 @@ RTE_PMD_REGISTER_PARAM_STRING(net_bonding, "mac=<mac addr> " "lsc_poll_period_ms=<int> " "up_delay=<int> " - "down_delay=<int>"); + "down_delay=<int>" + "notify_member=[enable | disable] " + "dedicated_queue=[enable | disable] "); /* We can't use RTE_LOG_REGISTER_DEFAULT because of the forced name for * this library, see meson.build. -- 2.39.1 ^ permalink raw reply [flat|nested] 80+ messages in thread
* [PATCH v3 5/8] net/bonding: support add port by data name 2023-10-08 1:50 ` [PATCH v3 0/8] Enhance the bond framework to support offload Chaoyong He ` (3 preceding siblings ...) 2023-10-08 1:50 ` [PATCH v3 4/8] net/bonding: add bonding port arguments Chaoyong He @ 2023-10-08 1:50 ` Chaoyong He 2023-10-08 1:50 ` [PATCH v3 6/8] net/bonding: create new rte flow header file Chaoyong He ` (4 subsequent siblings) 9 siblings, 0 replies; 80+ messages in thread From: Chaoyong He @ 2023-10-08 1:50 UTC (permalink / raw) To: dev; +Cc: oss-drivers, Long Wu, James Hershaw, Chaoyong He From: Long Wu <long.wu@corigine.com> Several ports may share the same PCI address, like nfp representor. So we cannot add this type of ports to bonding port by "--vdev" argument in dpdk-testpmd. But the port's data name is unique between them, we include an option to add such ports to the bonding port. After adding this feature, we can create a bonding port that member port is this type of port by "--vdev" in dpdk-testpmd start command. For example: dpdk-testpmd -l 2-10 -s 0x8 -a ca:00.0,representor=[0-2] --vdev 'net_bonding0,member=flower_repr_p0,member=flower_repr_p1, mode=4,socket_id=1,xmit_policy=l34' -- -i Note: 1. "ca:00.0" is nfp 4000 card. 2. "flower_repr_p0" and "flower_repr_p1" are nfp phy representor's data name. Signed-off-by: Long Wu <long.wu@corigine.com> Reviewed-by: James Hershaw <james.hershaw@corigine.com> Reviewed-by: Chaoyong He <chaoyong.he@corigine.com> --- drivers/net/bonding/rte_eth_bond_args.c | 3 +++ 1 file changed, 3 insertions(+) diff --git a/drivers/net/bonding/rte_eth_bond_args.c b/drivers/net/bonding/rte_eth_bond_args.c index 8a3e4656ef..b320eb3038 100644 --- a/drivers/net/bonding/rte_eth_bond_args.c +++ b/drivers/net/bonding/rte_eth_bond_args.c @@ -70,6 +70,9 @@ find_port_id_by_dev_name(const char *name) if (strcmp(rte_eth_devices[i].device->name, name) == 0) return i; + + if (strcmp(rte_eth_devices[i].data->name, name) == 0) + return i; } return -1; } -- 2.39.1 ^ permalink raw reply [flat|nested] 80+ messages in thread
* [PATCH v3 6/8] net/bonding: create new rte flow header file 2023-10-08 1:50 ` [PATCH v3 0/8] Enhance the bond framework to support offload Chaoyong He ` (4 preceding siblings ...) 2023-10-08 1:50 ` [PATCH v3 5/8] net/bonding: support add port by data name Chaoyong He @ 2023-10-08 1:50 ` Chaoyong He 2023-10-08 1:50 ` [PATCH v3 7/8] net/bonding: support checking valid bonding port ID Chaoyong He ` (3 subsequent siblings) 9 siblings, 0 replies; 80+ messages in thread From: Chaoyong He @ 2023-10-08 1:50 UTC (permalink / raw) To: dev; +Cc: oss-drivers, Long Wu, James Hershaw, Chaoyong He From: Long Wu <long.wu@corigine.com> Move the flow code to a new head file to make flow related code more clean and make the code architecture more reasonable in the future. There is no functional change, just moving verbatim code around. Signed-off-by: Long Wu <long.wu@corigine.com> Reviewed-by: James Hershaw <james.hershaw@corigine.com> Reviewed-by: Chaoyong He <chaoyong.he@corigine.com> --- drivers/net/bonding/eth_bond_private.h | 13 ------------- drivers/net/bonding/rte_eth_bond_api.c | 1 + drivers/net/bonding/rte_eth_bond_flow.c | 1 + drivers/net/bonding/rte_eth_bond_flow.h | 22 ++++++++++++++++++++++ drivers/net/bonding/rte_eth_bond_pmd.c | 1 + 5 files changed, 25 insertions(+), 13 deletions(-) create mode 100644 drivers/net/bonding/rte_eth_bond_flow.h diff --git a/drivers/net/bonding/eth_bond_private.h b/drivers/net/bonding/eth_bond_private.h index f9603a0f6b..4373465d8d 100644 --- a/drivers/net/bonding/eth_bond_private.h +++ b/drivers/net/bonding/eth_bond_private.h @@ -9,10 +9,8 @@ #include <sys/queue.h> #include <ethdev_driver.h> -#include <rte_flow.h> #include <rte_spinlock.h> #include <rte_bitmap.h> -#include <rte_flow_driver.h> #include "rte_eth_bond.h" #include "eth_bond_8023ad_private.h" @@ -47,8 +45,6 @@ extern const char *pmd_bond_init_valid_arguments[]; extern struct rte_vdev_driver pmd_bond_drv; -extern const struct rte_flow_ops bond_flow_ops; - /** Port Queue Mapping Structure */ struct bond_rx_queue { uint16_t queue_id; @@ -94,15 +90,6 @@ struct bond_member_details { uint16_t reta_size; }; -struct rte_flow { - TAILQ_ENTRY(rte_flow) next; - /* Members flows */ - struct rte_flow *flows[RTE_MAX_ETHPORTS]; - /* Flow description for synchronization */ - struct rte_flow_conv_rule rule; - uint8_t rule_data[]; -}; - typedef void (*burst_xmit_hash_t)(struct rte_mbuf **buf, uint16_t nb_pkts, uint16_t member_count, uint16_t *members); diff --git a/drivers/net/bonding/rte_eth_bond_api.c b/drivers/net/bonding/rte_eth_bond_api.c index a042f05a4c..0113dfdc16 100644 --- a/drivers/net/bonding/rte_eth_bond_api.c +++ b/drivers/net/bonding/rte_eth_bond_api.c @@ -12,6 +12,7 @@ #include <rte_kvargs.h> #include "rte_eth_bond.h" +#include "rte_eth_bond_flow.h" #include "eth_bond_private.h" #include "eth_bond_8023ad_private.h" diff --git a/drivers/net/bonding/rte_eth_bond_flow.c b/drivers/net/bonding/rte_eth_bond_flow.c index 71a91675f7..e6c7ce5362 100644 --- a/drivers/net/bonding/rte_eth_bond_flow.c +++ b/drivers/net/bonding/rte_eth_bond_flow.c @@ -1,6 +1,7 @@ /* SPDX-License-Identifier: BSD-3-Clause * Copyright 2018 Mellanox Technologies, Ltd */ +#include "rte_eth_bond_flow.h" #include <stddef.h> #include <string.h> diff --git a/drivers/net/bonding/rte_eth_bond_flow.h b/drivers/net/bonding/rte_eth_bond_flow.h new file mode 100644 index 0000000000..7394e0e2e1 --- /dev/null +++ b/drivers/net/bonding/rte_eth_bond_flow.h @@ -0,0 +1,22 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright (c) 2023 Corigine, Inc. + */ + +#ifndef _RTE_ETH_BOND_FLOW_H_ +#define _RTE_ETH_BOND_FLOW_H_ + +#include <rte_flow.h> +#include <rte_flow_driver.h> + +extern const struct rte_flow_ops bond_flow_ops; + +struct rte_flow { + TAILQ_ENTRY(rte_flow) next; + struct rte_flow *flows[RTE_MAX_ETHPORTS]; + /**< Member ports flows */ + struct rte_flow_conv_rule rule; + /**< Flow description for synchronization */ + uint8_t rule_data[]; +}; + +#endif /* _RTE_ETH_BOND_FLOW_H_ */ diff --git a/drivers/net/bonding/rte_eth_bond_pmd.c b/drivers/net/bonding/rte_eth_bond_pmd.c index 1ebeb270c8..630afc3740 100644 --- a/drivers/net/bonding/rte_eth_bond_pmd.c +++ b/drivers/net/bonding/rte_eth_bond_pmd.c @@ -21,6 +21,7 @@ #include <rte_string_fns.h> #include "rte_eth_bond.h" +#include "rte_eth_bond_flow.h" #include "eth_bond_private.h" #include "eth_bond_8023ad_private.h" -- 2.39.1 ^ permalink raw reply [flat|nested] 80+ messages in thread
* [PATCH v3 7/8] net/bonding: support checking valid bonding port ID 2023-10-08 1:50 ` [PATCH v3 0/8] Enhance the bond framework to support offload Chaoyong He ` (5 preceding siblings ...) 2023-10-08 1:50 ` [PATCH v3 6/8] net/bonding: create new rte flow header file Chaoyong He @ 2023-10-08 1:50 ` Chaoyong He 2023-10-17 8:33 ` lihuisong (C) 2023-10-08 1:50 ` [PATCH v3 8/8] net/bonding: add commands for bonding port notification Chaoyong He ` (2 subsequent siblings) 9 siblings, 1 reply; 80+ messages in thread From: Chaoyong He @ 2023-10-08 1:50 UTC (permalink / raw) To: dev; +Cc: oss-drivers, Long Wu, James Hershaw, Chaoyong He From: Long Wu <long.wu@corigine.com> Add API to support checking if the port id is a bonding port id. Signed-off-by: Long Wu <long.wu@corigine.com> Reviewed-by: James Hershaw <james.hershaw@corigine.com> Reviewed-by: Chaoyong He <chaoyong.he@corigine.com> --- drivers/net/bonding/rte_eth_bond.h | 13 +++++++++++++ drivers/net/bonding/rte_eth_bond_api.c | 7 +++++++ drivers/net/bonding/version.map | 1 + 3 files changed, 21 insertions(+) diff --git a/drivers/net/bonding/rte_eth_bond.h b/drivers/net/bonding/rte_eth_bond.h index 3f427b6bab..e8152a155f 100644 --- a/drivers/net/bonding/rte_eth_bond.h +++ b/drivers/net/bonding/rte_eth_bond.h @@ -461,6 +461,19 @@ __rte_experimental int rte_eth_bond_hw_create_get(uint16_t bonding_port_id, uint16_t member_port_id); +/** + * Check whether bonding port id is valid. + * + * @param bonding_port_id + * Port ID of bonding device. + * + * @return + * 0 on success, negative value otherwise. + */ +__rte_experimental +int +rte_eth_bond_valid_bonding_port_id(uint16_t bonding_port_id); + #ifdef __cplusplus } #endif diff --git a/drivers/net/bonding/rte_eth_bond_api.c b/drivers/net/bonding/rte_eth_bond_api.c index 0113dfdc16..80d71529cc 100644 --- a/drivers/net/bonding/rte_eth_bond_api.c +++ b/drivers/net/bonding/rte_eth_bond_api.c @@ -1214,3 +1214,10 @@ rte_eth_bond_hw_create_get(uint16_t bonding_port_id, uint16_t member_port_id) return member_dev->dev_ops->bond_hw_create_get(member_dev, bonding_dev); } + + +int +rte_eth_bond_valid_bonding_port_id(uint16_t port_id) +{ + return valid_bonding_port_id(port_id); +} diff --git a/drivers/net/bonding/version.map b/drivers/net/bonding/version.map index 3cfff51269..bf5e50521e 100644 --- a/drivers/net/bonding/version.map +++ b/drivers/net/bonding/version.map @@ -39,4 +39,5 @@ EXPERIMENTAL { rte_eth_bond_notify_member_flag_get; rte_eth_bond_notify_member_flag_set; rte_eth_bond_notify_members; + rte_eth_bond_valid_bonding_port_id; }; -- 2.39.1 ^ permalink raw reply [flat|nested] 80+ messages in thread
* Re: [PATCH v3 7/8] net/bonding: support checking valid bonding port ID 2023-10-08 1:50 ` [PATCH v3 7/8] net/bonding: support checking valid bonding port ID Chaoyong He @ 2023-10-17 8:33 ` lihuisong (C) 2023-10-17 9:25 ` Chaoyong He 2023-10-17 15:56 ` Stephen Hemminger 0 siblings, 2 replies; 80+ messages in thread From: lihuisong (C) @ 2023-10-17 8:33 UTC (permalink / raw) To: Chaoyong He, dev; +Cc: oss-drivers, Long Wu, James Hershaw Hi Chaoyong, It is better to separate patch 3/8 and patch 6/8 from this series. 在 2023/10/8 9:50, Chaoyong He 写道: > From: Long Wu <long.wu@corigine.com> > > Add API to support checking if the port id is a bonding > port id. > > Signed-off-by: Long Wu <long.wu@corigine.com> > Reviewed-by: James Hershaw <james.hershaw@corigine.com> > Reviewed-by: Chaoyong He <chaoyong.he@corigine.com> > --- > drivers/net/bonding/rte_eth_bond.h | 13 +++++++++++++ > drivers/net/bonding/rte_eth_bond_api.c | 7 +++++++ > drivers/net/bonding/version.map | 1 + > 3 files changed, 21 insertions(+) > > diff --git a/drivers/net/bonding/rte_eth_bond.h b/drivers/net/bonding/rte_eth_bond.h > index 3f427b6bab..e8152a155f 100644 > --- a/drivers/net/bonding/rte_eth_bond.h > +++ b/drivers/net/bonding/rte_eth_bond.h > @@ -461,6 +461,19 @@ __rte_experimental > int > rte_eth_bond_hw_create_get(uint16_t bonding_port_id, uint16_t member_port_id); > > +/** > + * Check whether bonding port id is valid. > + * > + * @param bonding_port_id > + * Port ID of bonding device. > + * > + * @return > + * 0 on success, negative value otherwise. > + */ > +__rte_experimental > +int > +rte_eth_bond_valid_bonding_port_id(uint16_t bonding_port_id); > + > #ifdef __cplusplus > } > #endif > diff --git a/drivers/net/bonding/rte_eth_bond_api.c b/drivers/net/bonding/rte_eth_bond_api.c > index 0113dfdc16..80d71529cc 100644 > --- a/drivers/net/bonding/rte_eth_bond_api.c > +++ b/drivers/net/bonding/rte_eth_bond_api.c > @@ -1214,3 +1214,10 @@ rte_eth_bond_hw_create_get(uint16_t bonding_port_id, uint16_t member_port_id) > > return member_dev->dev_ops->bond_hw_create_get(member_dev, bonding_dev); > } > + > + > +int > +rte_eth_bond_valid_bonding_port_id(uint16_t port_id) > +{ > + return valid_bonding_port_id(port_id); > +} > diff --git a/drivers/net/bonding/version.map b/drivers/net/bonding/version.map > index 3cfff51269..bf5e50521e 100644 > --- a/drivers/net/bonding/version.map > +++ b/drivers/net/bonding/version.map > @@ -39,4 +39,5 @@ EXPERIMENTAL { > rte_eth_bond_notify_member_flag_get; > rte_eth_bond_notify_member_flag_set; > rte_eth_bond_notify_members; > + rte_eth_bond_valid_bonding_port_id; > }; ^ permalink raw reply [flat|nested] 80+ messages in thread
* RE: [PATCH v3 7/8] net/bonding: support checking valid bonding port ID 2023-10-17 8:33 ` lihuisong (C) @ 2023-10-17 9:25 ` Chaoyong He 2023-10-17 11:34 ` lihuisong (C) 2023-10-17 15:56 ` Stephen Hemminger 1 sibling, 1 reply; 80+ messages in thread From: Chaoyong He @ 2023-10-17 9:25 UTC (permalink / raw) To: lihuisong (C), dev; +Cc: oss-drivers, Long Wu > Hi Chaoyong, > > It is better to separate patch 3/8 and patch 6/8 from this series. The patch 3/8 is okay to separate, there is no problem. But if patch 6/8 be a separate patch, this patch series will need depend on it. I'm not sure if that is okay? > > 在 2023/10/8 9:50, Chaoyong He 写道: > > From: Long Wu <long.wu@corigine.com> > > > > Add API to support checking if the port id is a bonding port id. > > > > Signed-off-by: Long Wu <long.wu@corigine.com> > > Reviewed-by: James Hershaw <james.hershaw@corigine.com> > > Reviewed-by: Chaoyong He <chaoyong.he@corigine.com> > > --- > > drivers/net/bonding/rte_eth_bond.h | 13 +++++++++++++ > > drivers/net/bonding/rte_eth_bond_api.c | 7 +++++++ > > drivers/net/bonding/version.map | 1 + > > 3 files changed, 21 insertions(+) > > > > diff --git a/drivers/net/bonding/rte_eth_bond.h > > b/drivers/net/bonding/rte_eth_bond.h > > index 3f427b6bab..e8152a155f 100644 > > --- a/drivers/net/bonding/rte_eth_bond.h > > +++ b/drivers/net/bonding/rte_eth_bond.h > > @@ -461,6 +461,19 @@ __rte_experimental > > int > > rte_eth_bond_hw_create_get(uint16_t bonding_port_id, uint16_t > > member_port_id); > > > > +/** > > + * Check whether bonding port id is valid. > > + * > > + * @param bonding_port_id > > + * Port ID of bonding device. > > + * > > + * @return > > + * 0 on success, negative value otherwise. > > + */ > > +__rte_experimental > > +int > > +rte_eth_bond_valid_bonding_port_id(uint16_t bonding_port_id); > > + > > #ifdef __cplusplus > > } > > #endif > > diff --git a/drivers/net/bonding/rte_eth_bond_api.c > > b/drivers/net/bonding/rte_eth_bond_api.c > > index 0113dfdc16..80d71529cc 100644 > > --- a/drivers/net/bonding/rte_eth_bond_api.c > > +++ b/drivers/net/bonding/rte_eth_bond_api.c > > @@ -1214,3 +1214,10 @@ rte_eth_bond_hw_create_get(uint16_t > > bonding_port_id, uint16_t member_port_id) > > > > return member_dev->dev_ops->bond_hw_create_get(member_dev, > bonding_dev); > > } > > + > > + > > +int > > +rte_eth_bond_valid_bonding_port_id(uint16_t port_id) { > > + return valid_bonding_port_id(port_id); } > > diff --git a/drivers/net/bonding/version.map > > b/drivers/net/bonding/version.map index 3cfff51269..bf5e50521e 100644 > > --- a/drivers/net/bonding/version.map > > +++ b/drivers/net/bonding/version.map > > @@ -39,4 +39,5 @@ EXPERIMENTAL { > > rte_eth_bond_notify_member_flag_get; > > rte_eth_bond_notify_member_flag_set; > > rte_eth_bond_notify_members; > > + rte_eth_bond_valid_bonding_port_id; > > }; ^ permalink raw reply [flat|nested] 80+ messages in thread
* Re: [PATCH v3 7/8] net/bonding: support checking valid bonding port ID 2023-10-17 9:25 ` Chaoyong He @ 2023-10-17 11:34 ` lihuisong (C) 2023-10-18 1:53 ` Chaoyong He 0 siblings, 1 reply; 80+ messages in thread From: lihuisong (C) @ 2023-10-17 11:34 UTC (permalink / raw) To: Chaoyong He, dev; +Cc: oss-drivers, Long Wu 在 2023/10/17 17:25, Chaoyong He 写道: >> Hi Chaoyong, >> >> It is better to separate patch 3/8 and patch 6/8 from this series. > The patch 3/8 is okay to separate, there is no problem. > But if patch 6/8 be a separate patch, this patch series will need depend on it. > I'm not sure if that is okay? I doesn't see the dependency of patch 7/8 and 8/8 on it. If you remove the patches which isn't releated with the subject of this series, it is helpful for reviewing. > >> 在 2023/10/8 9:50, Chaoyong He 写道: >>> From: Long Wu <long.wu@corigine.com> >>> >>> Add API to support checking if the port id is a bonding port id. >>> >>> Signed-off-by: Long Wu <long.wu@corigine.com> >>> Reviewed-by: James Hershaw <james.hershaw@corigine.com> >>> Reviewed-by: Chaoyong He <chaoyong.he@corigine.com> >>> --- >>> drivers/net/bonding/rte_eth_bond.h | 13 +++++++++++++ >>> drivers/net/bonding/rte_eth_bond_api.c | 7 +++++++ >>> drivers/net/bonding/version.map | 1 + >>> 3 files changed, 21 insertions(+) >>> >>> diff --git a/drivers/net/bonding/rte_eth_bond.h >>> b/drivers/net/bonding/rte_eth_bond.h >>> index 3f427b6bab..e8152a155f 100644 >>> --- a/drivers/net/bonding/rte_eth_bond.h >>> +++ b/drivers/net/bonding/rte_eth_bond.h >>> @@ -461,6 +461,19 @@ __rte_experimental >>> int >>> rte_eth_bond_hw_create_get(uint16_t bonding_port_id, uint16_t >>> member_port_id); >>> >>> +/** >>> + * Check whether bonding port id is valid. >>> + * >>> + * @param bonding_port_id >>> + * Port ID of bonding device. >>> + * >>> + * @return >>> + * 0 on success, negative value otherwise. >>> + */ >>> +__rte_experimental >>> +int >>> +rte_eth_bond_valid_bonding_port_id(uint16_t bonding_port_id); >>> + >>> #ifdef __cplusplus >>> } >>> #endif >>> diff --git a/drivers/net/bonding/rte_eth_bond_api.c >>> b/drivers/net/bonding/rte_eth_bond_api.c >>> index 0113dfdc16..80d71529cc 100644 >>> --- a/drivers/net/bonding/rte_eth_bond_api.c >>> +++ b/drivers/net/bonding/rte_eth_bond_api.c >>> @@ -1214,3 +1214,10 @@ rte_eth_bond_hw_create_get(uint16_t >>> bonding_port_id, uint16_t member_port_id) >>> >>> return member_dev->dev_ops->bond_hw_create_get(member_dev, >> bonding_dev); >>> } >>> + >>> + >>> +int >>> +rte_eth_bond_valid_bonding_port_id(uint16_t port_id) { >>> + return valid_bonding_port_id(port_id); } >>> diff --git a/drivers/net/bonding/version.map >>> b/drivers/net/bonding/version.map index 3cfff51269..bf5e50521e 100644 >>> --- a/drivers/net/bonding/version.map >>> +++ b/drivers/net/bonding/version.map >>> @@ -39,4 +39,5 @@ EXPERIMENTAL { >>> rte_eth_bond_notify_member_flag_get; >>> rte_eth_bond_notify_member_flag_set; >>> rte_eth_bond_notify_members; >>> + rte_eth_bond_valid_bonding_port_id; >>> }; ^ permalink raw reply [flat|nested] 80+ messages in thread
* RE: [PATCH v3 7/8] net/bonding: support checking valid bonding port ID 2023-10-17 11:34 ` lihuisong (C) @ 2023-10-18 1:53 ` Chaoyong He 0 siblings, 0 replies; 80+ messages in thread From: Chaoyong He @ 2023-10-18 1:53 UTC (permalink / raw) To: lihuisong (C), dev; +Cc: oss-drivers, Long Wu > 在 2023/10/17 17:25, Chaoyong He 写道: > >> Hi Chaoyong, > >> > >> It is better to separate patch 3/8 and patch 6/8 from this series. > > The patch 3/8 is okay to separate, there is no problem. > > But if patch 6/8 be a separate patch, this patch series will need depend on it. > > I'm not sure if that is okay? > I doesn't see the dependency of patch 7/8 and 8/8 on it. > If you remove the patches which isn't releated with the subject of this series, it > is helpful for reviewing. After discussing with Long Wu again, we confirmed that what you said is right. Both patch 3/8 and 6/8 can be separated from this series, and we will do it in the next version. Thanks. > > > >> 在 2023/10/8 9:50, Chaoyong He 写道: > >>> From: Long Wu <long.wu@corigine.com> > >>> > >>> Add API to support checking if the port id is a bonding port id. > >>> > >>> Signed-off-by: Long Wu <long.wu@corigine.com> > >>> Reviewed-by: James Hershaw <james.hershaw@corigine.com> > >>> Reviewed-by: Chaoyong He <chaoyong.he@corigine.com> > >>> --- > >>> drivers/net/bonding/rte_eth_bond.h | 13 +++++++++++++ > >>> drivers/net/bonding/rte_eth_bond_api.c | 7 +++++++ > >>> drivers/net/bonding/version.map | 1 + > >>> 3 files changed, 21 insertions(+) > >>> > >>> diff --git a/drivers/net/bonding/rte_eth_bond.h > >>> b/drivers/net/bonding/rte_eth_bond.h > >>> index 3f427b6bab..e8152a155f 100644 > >>> --- a/drivers/net/bonding/rte_eth_bond.h > >>> +++ b/drivers/net/bonding/rte_eth_bond.h > >>> @@ -461,6 +461,19 @@ __rte_experimental > >>> int > >>> rte_eth_bond_hw_create_get(uint16_t bonding_port_id, uint16_t > >>> member_port_id); > >>> > >>> +/** > >>> + * Check whether bonding port id is valid. > >>> + * > >>> + * @param bonding_port_id > >>> + * Port ID of bonding device. > >>> + * > >>> + * @return > >>> + * 0 on success, negative value otherwise. > >>> + */ > >>> +__rte_experimental > >>> +int > >>> +rte_eth_bond_valid_bonding_port_id(uint16_t bonding_port_id); > >>> + > >>> #ifdef __cplusplus > >>> } > >>> #endif > >>> diff --git a/drivers/net/bonding/rte_eth_bond_api.c > >>> b/drivers/net/bonding/rte_eth_bond_api.c > >>> index 0113dfdc16..80d71529cc 100644 > >>> --- a/drivers/net/bonding/rte_eth_bond_api.c > >>> +++ b/drivers/net/bonding/rte_eth_bond_api.c > >>> @@ -1214,3 +1214,10 @@ rte_eth_bond_hw_create_get(uint16_t > >>> bonding_port_id, uint16_t member_port_id) > >>> > >>> return member_dev->dev_ops->bond_hw_create_get(member_dev, > >> bonding_dev); > >>> } > >>> + > >>> + > >>> +int > >>> +rte_eth_bond_valid_bonding_port_id(uint16_t port_id) { > >>> + return valid_bonding_port_id(port_id); } > >>> diff --git a/drivers/net/bonding/version.map > >>> b/drivers/net/bonding/version.map index 3cfff51269..bf5e50521e > >>> 100644 > >>> --- a/drivers/net/bonding/version.map > >>> +++ b/drivers/net/bonding/version.map > >>> @@ -39,4 +39,5 @@ EXPERIMENTAL { > >>> rte_eth_bond_notify_member_flag_get; > >>> rte_eth_bond_notify_member_flag_set; > >>> rte_eth_bond_notify_members; > >>> + rte_eth_bond_valid_bonding_port_id; > >>> }; ^ permalink raw reply [flat|nested] 80+ messages in thread
* Re: [PATCH v3 7/8] net/bonding: support checking valid bonding port ID 2023-10-17 8:33 ` lihuisong (C) 2023-10-17 9:25 ` Chaoyong He @ 2023-10-17 15:56 ` Stephen Hemminger 1 sibling, 0 replies; 80+ messages in thread From: Stephen Hemminger @ 2023-10-17 15:56 UTC (permalink / raw) To: lihuisong (C); +Cc: Chaoyong He, dev, oss-drivers, Long Wu, James Hershaw On Tue, 17 Oct 2023 16:33:26 +0800 "lihuisong (C)" <lihuisong@huawei.com> wrote: > > +__rte_experimental > > +int > > +rte_eth_bond_valid_bonding_port_id(uint16_t bonding_port_id); Ok, but my personal preference is to use shorter variable names to avoid typos and overly long lines. Maybe: __rte_experimental bool rte_eth_bond_is_valid_port(uint16_t port_id) ^ permalink raw reply [flat|nested] 80+ messages in thread
* [PATCH v3 8/8] net/bonding: add commands for bonding port notification 2023-10-08 1:50 ` [PATCH v3 0/8] Enhance the bond framework to support offload Chaoyong He ` (6 preceding siblings ...) 2023-10-08 1:50 ` [PATCH v3 7/8] net/bonding: support checking valid bonding port ID Chaoyong He @ 2023-10-08 1:50 ` Chaoyong He 2023-10-13 2:22 ` [PATCH v3 0/8] Enhance the bond framework to support offload Chaoyong He 2023-10-18 7:48 ` [PATCH v4 0/6] " Chaoyong He 9 siblings, 0 replies; 80+ messages in thread From: Chaoyong He @ 2023-10-08 1:50 UTC (permalink / raw) To: dev; +Cc: oss-drivers, Long Wu, James Hershaw, Chaoyong He From: Long Wu <long.wu@corigine.com> Add some commands to support bonding port notification in dpdk-testpmd. 1. We can enable the notification by command: "set bonding notify_member (port_id) (enable|disable)" 2. If member port hardware try to create the bonding port after notification we can get the status by command: "get bonding member hardware create (member_port_id) (bonding_port_id)" Signed-off-by: Long Wu <long.wu@corigine.com> Reviewed-by: James Hershaw <james.hershaw@corigine.com> Reviewed-by: Chaoyong He <chaoyong.he@corigine.com> --- .../link_bonding_poll_mode_drv_lib.rst | 19 +++ drivers/net/bonding/bonding_testpmd.c | 128 ++++++++++++++++++ 2 files changed, 147 insertions(+) diff --git a/doc/guides/prog_guide/link_bonding_poll_mode_drv_lib.rst b/doc/guides/prog_guide/link_bonding_poll_mode_drv_lib.rst index 60717a3587..9f6443ebd8 100644 --- a/doc/guides/prog_guide/link_bonding_poll_mode_drv_lib.rst +++ b/doc/guides/prog_guide/link_bonding_poll_mode_drv_lib.rst @@ -637,3 +637,22 @@ in balance mode with a transmission policy of layer 2+3:: Members (3): [1 3 4] Active Members (3): [1 3 4] Primary: [3] + +set bonding notify_member +~~~~~~~~~~~~~~~~~~~~~~~~~ + +Set the notify member flag of bonding port:: + + testpmd> set bonding notify_member (port_id) (enable|disable) + +This command just set the flag of notification. +If we enable it, bonding PMD will notify member ports when its some +configurations changed. So member ports can do some private things, maybe hardware +bonding creation and etc. + +get bonding member hardware create +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +Get the status of member port hardware creating the bonding port:: + + testpmd> get bonding member hardware create (member_port_id) (bonding_port_id) diff --git a/drivers/net/bonding/bonding_testpmd.c b/drivers/net/bonding/bonding_testpmd.c index 8fcd6cadd0..da7d9cc58f 100644 --- a/drivers/net/bonding/bonding_testpmd.c +++ b/drivers/net/bonding/bonding_testpmd.c @@ -692,6 +692,124 @@ static cmdline_parse_inst_t cmd_set_bonding_agg_mode_policy = { } }; +struct cmd_set_bonding_notify_member_result { + cmdline_fixed_string_t set; + cmdline_fixed_string_t bonding; + cmdline_fixed_string_t notify_member; + uint16_t port_num; + cmdline_fixed_string_t mode; +}; + +static void +cmd_set_bonding_notify_member_parsed(void *parsed_result, + __rte_unused struct cmdline *cl, __rte_unused void *data) +{ + struct cmd_set_bonding_notify_member_result *res = parsed_result; + bool notify_member = false; + + if (strcmp(res->notify_member, "enable") == 0) + notify_member = true; + else if (strcmp(res->notify_member, "disable") == 0) + notify_member = false; + + rte_eth_bond_notify_member_flag_set(res->port_num, notify_member); +} + +static cmdline_parse_token_string_t cmd_set_bonding_notify_member_set = + TOKEN_STRING_INITIALIZER(struct cmd_set_bonding_notify_member_result, + set, "set"); +static cmdline_parse_token_string_t cmd_set_bonding_notify_member_bonding = + TOKEN_STRING_INITIALIZER(struct cmd_set_bonding_notify_member_result, + bonding, "bonding"); +static cmdline_parse_token_string_t cmd_set_bonding_notify_member = + TOKEN_STRING_INITIALIZER(struct cmd_set_bonding_notify_member_result, + notify_member, "notify_member"); +static cmdline_parse_token_num_t cmd_set_bonding_notify_member_portnum = + TOKEN_NUM_INITIALIZER(struct cmd_set_bonding_notify_member_result, + port_num, RTE_UINT16); +static cmdline_parse_token_string_t cmd_set_bonding_notify_member_mode_string = + TOKEN_STRING_INITIALIZER(struct cmd_set_bonding_notify_member_result, + mode, "enable#disable"); + +static cmdline_parse_inst_t cmd_set_bonding_notify_member_ports = { + .f = cmd_set_bonding_notify_member_parsed, + .data = NULL, + .help_str = "set bonding notify_member (port_id) (enable|disable)", + .tokens = { + (void *)&cmd_set_bonding_notify_member_set, + (void *)&cmd_set_bonding_notify_member_bonding, + (void *)&cmd_set_bonding_notify_member, + (void *)&cmd_set_bonding_notify_member_portnum, + (void *)&cmd_set_bonding_notify_member_mode_string, + NULL + } +}; + +struct cmd_get_bonding_member_hw_create_result { + cmdline_fixed_string_t get; + cmdline_fixed_string_t bonding; + cmdline_fixed_string_t member; + cmdline_fixed_string_t hardware; + cmdline_fixed_string_t create; + uint16_t member_port_id; + uint16_t bonding_port_id; +}; + +static void +cmd_get_bonding_member_hw_create_parsed(void *parsed_result, + __rte_unused struct cmdline *cl, __rte_unused void *data) +{ + struct cmd_get_bonding_member_hw_create_result *res = parsed_result; + int ret; + + ret = rte_eth_bond_hw_create_get(res->bonding_port_id, res->member_port_id); + if (ret == 0) + printf("Member port %u hardware creates bonding port %u successfully\n", + res->member_port_id, res->bonding_port_id); + else + printf("Failed to get status of member port %u hardware creating" + " bonding port %u, %d\n", + res->member_port_id, res->bonding_port_id, ret); +} + +static cmdline_parse_token_string_t cmd_get_bonding_member_hw_create_get = + TOKEN_STRING_INITIALIZER(struct cmd_get_bonding_member_hw_create_result, + get, "get"); +static cmdline_parse_token_string_t cmd_get_bonding_member_hw_create_bonding = + TOKEN_STRING_INITIALIZER(struct cmd_get_bonding_member_hw_create_result, + bonding, "bonding"); +static cmdline_parse_token_string_t cmd_get_bonding_member_hw_create_member = + TOKEN_STRING_INITIALIZER(struct cmd_get_bonding_member_hw_create_result, + member, "member"); +static cmdline_parse_token_string_t cmd_get_bonding_member_hw_create_hardware = + TOKEN_STRING_INITIALIZER(struct cmd_get_bonding_member_hw_create_result, + create, "hardware"); +static cmdline_parse_token_string_t cmd_get_bonding_member_hw_create_create = + TOKEN_STRING_INITIALIZER(struct cmd_get_bonding_member_hw_create_result, + create, "create"); +static cmdline_parse_token_num_t cmd_get_bonding_member_hw_create_memberportid = + TOKEN_NUM_INITIALIZER(struct cmd_get_bonding_member_hw_create_result, + member_port_id, RTE_UINT16); +static cmdline_parse_token_num_t cmd_get_bonding_member_hw_create_bondingportid = + TOKEN_NUM_INITIALIZER(struct cmd_get_bonding_member_hw_create_result, + bonding_port_id, RTE_UINT16); + +static cmdline_parse_inst_t cmd_get_member_hw_create_bonding = { + .f = cmd_get_bonding_member_hw_create_parsed, + .data = NULL, + .help_str = "get bonding member hardware create (member_port_id) (bonding_port_id)", + .tokens = { + (void *)&cmd_get_bonding_member_hw_create_get, + (void *)&cmd_get_bonding_member_hw_create_bonding, + (void *)&cmd_get_bonding_member_hw_create_member, + (void *)&cmd_get_bonding_member_hw_create_hardware, + (void *)&cmd_get_bonding_member_hw_create_create, + (void *)&cmd_get_bonding_member_hw_create_memberportid, + (void *)&cmd_get_bonding_member_hw_create_bondingportid, + NULL + } +}; + static struct testpmd_driver_commands bonding_cmds = { .commands = { { @@ -749,6 +867,16 @@ static struct testpmd_driver_commands bonding_cmds = { "set bonding mode IEEE802.3AD aggregator policy (port_id) (agg_name)\n" " Set Aggregation mode for IEEE802.3AD (mode 4)\n", }, + { + &cmd_set_bonding_notify_member_ports, + "set bonding notify_member (port_id) (enable|disable)\n" + " Enable/disable the notify member flag of bonding port\n", + }, + { + &cmd_get_member_hw_create_bonding, + "get bonding member hardware create (member_port_id) (bonding_port_id)\n" + " Get the status of member port hardware creating the bonding port\n", + }, { NULL, NULL }, }, }; -- 2.39.1 ^ permalink raw reply [flat|nested] 80+ messages in thread
* RE: [PATCH v3 0/8] Enhance the bond framework to support offload 2023-10-08 1:50 ` [PATCH v3 0/8] Enhance the bond framework to support offload Chaoyong He ` (7 preceding siblings ...) 2023-10-08 1:50 ` [PATCH v3 8/8] net/bonding: add commands for bonding port notification Chaoyong He @ 2023-10-13 2:22 ` Chaoyong He 2023-10-13 12:53 ` Ferruh Yigit 2023-10-18 7:48 ` [PATCH v4 0/6] " Chaoyong He 9 siblings, 1 reply; 80+ messages in thread From: Chaoyong He @ 2023-10-13 2:22 UTC (permalink / raw) To: dev; +Cc: oss-drivers A gentle ping ~ As this patch series add new APIs, hoping it will catch up deadline of 23.11-RC1 (API freeze). Thanks. > -----Original Message----- > From: Chaoyong He > Sent: Sunday, October 8, 2023 9:51 AM > To: dev@dpdk.org > Cc: oss-drivers <oss-drivers@corigine.com>; Chaoyong He > <chaoyong.he@corigine.com> > Subject: [PATCH v3 0/8] Enhance the bond framework to support offload > > This patch series try to enhance the bond framework to support the offload > feature better: > * Add new API to make the member port can access some information of the > bond port which belongs. > * Add new API to get the result of whether bond port is created by the > member port. > * Add two command line argument to control if enable member port > notification and dedicated queue features. > * Add logic to support add ports which share the same PCI address into > bond port. > * Also modify the testpmd application to test the new APIs and logics > added by this patch series. > > --- > v2: > * Fix compile error on github-robot by removing the redundancy function > declaration in the header file. > v3: > * Use the hole in the structure for the new added flag data field. > --- > > > Long Wu (8): > ethdev: add member notification for bonding port > ethdev: add API to get hardware creation of bonding port > net/bonding: modify interface comment format > net/bonding: add bonding port arguments > net/bonding: support add port by data name > net/bonding: create new rte flow header file > net/bonding: support checking valid bonding port ID > net/bonding: add commands for bonding port notification > > .../link_bonding_poll_mode_drv_lib.rst | 19 ++ > drivers/net/bonding/bonding_testpmd.c | 128 ++++++++++ > drivers/net/bonding/eth_bond_8023ad_private.h | 52 ++-- > drivers/net/bonding/eth_bond_private.h | 24 +- > drivers/net/bonding/rte_eth_bond.h | 238 +++++++++++++----- > drivers/net/bonding/rte_eth_bond_8023ad.h | 76 ++++-- > drivers/net/bonding/rte_eth_bond_alb.h | 34 ++- > drivers/net/bonding/rte_eth_bond_api.c | 123 +++++++++ > drivers/net/bonding/rte_eth_bond_args.c | 47 ++++ > drivers/net/bonding/rte_eth_bond_flow.c | 1 + > drivers/net/bonding/rte_eth_bond_flow.h | 22 ++ > drivers/net/bonding/rte_eth_bond_pmd.c | 89 ++++++- > drivers/net/bonding/version.map | 5 + > lib/ethdev/ethdev_driver.h | 38 +++ > 14 files changed, 766 insertions(+), 130 deletions(-) create mode 100644 > drivers/net/bonding/rte_eth_bond_flow.h > > -- > 2.39.1 ^ permalink raw reply [flat|nested] 80+ messages in thread
* Re: [PATCH v3 0/8] Enhance the bond framework to support offload 2023-10-13 2:22 ` [PATCH v3 0/8] Enhance the bond framework to support offload Chaoyong He @ 2023-10-13 12:53 ` Ferruh Yigit 0 siblings, 0 replies; 80+ messages in thread From: Ferruh Yigit @ 2023-10-13 12:53 UTC (permalink / raw) To: Chaoyong He, dev, Chas Williams, Min Hu (Connor) Cc: oss-drivers, techboard, David Marchand On 10/13/2023 3:22 AM, Chaoyong He wrote: > A gentle ping ~ > > As this patch series add new APIs, hoping it will catch up deadline of 23.11-RC1 (API freeze). > Hi Chaoyong, I did a quick scan of the patch, followings are red flag to me - adding an ethdev dev_ops for a specific driver (bonding), - adding more PMD specific API for bondig I will try to spend more time to understand the logic behind it but meanwhile I cc'ed more people for comment/review. > Thanks. > >> -----Original Message----- >> From: Chaoyong He >> Sent: Sunday, October 8, 2023 9:51 AM >> To: dev@dpdk.org >> Cc: oss-drivers <oss-drivers@corigine.com>; Chaoyong He >> <chaoyong.he@corigine.com> >> Subject: [PATCH v3 0/8] Enhance the bond framework to support offload >> >> This patch series try to enhance the bond framework to support the offload >> feature better: >> * Add new API to make the member port can access some information of the >> bond port which belongs. >> * Add new API to get the result of whether bond port is created by the >> member port. >> * Add two command line argument to control if enable member port >> notification and dedicated queue features. >> * Add logic to support add ports which share the same PCI address into >> bond port. >> * Also modify the testpmd application to test the new APIs and logics >> added by this patch series. >> >> --- >> v2: >> * Fix compile error on github-robot by removing the redundancy function >> declaration in the header file. >> v3: >> * Use the hole in the structure for the new added flag data field. >> --- >> >> >> Long Wu (8): >> ethdev: add member notification for bonding port >> ethdev: add API to get hardware creation of bonding port >> net/bonding: modify interface comment format >> net/bonding: add bonding port arguments >> net/bonding: support add port by data name >> net/bonding: create new rte flow header file >> net/bonding: support checking valid bonding port ID >> net/bonding: add commands for bonding port notification >> >> .../link_bonding_poll_mode_drv_lib.rst | 19 ++ >> drivers/net/bonding/bonding_testpmd.c | 128 ++++++++++ >> drivers/net/bonding/eth_bond_8023ad_private.h | 52 ++-- >> drivers/net/bonding/eth_bond_private.h | 24 +- >> drivers/net/bonding/rte_eth_bond.h | 238 +++++++++++++----- >> drivers/net/bonding/rte_eth_bond_8023ad.h | 76 ++++-- >> drivers/net/bonding/rte_eth_bond_alb.h | 34 ++- >> drivers/net/bonding/rte_eth_bond_api.c | 123 +++++++++ >> drivers/net/bonding/rte_eth_bond_args.c | 47 ++++ >> drivers/net/bonding/rte_eth_bond_flow.c | 1 + >> drivers/net/bonding/rte_eth_bond_flow.h | 22 ++ >> drivers/net/bonding/rte_eth_bond_pmd.c | 89 ++++++- >> drivers/net/bonding/version.map | 5 + >> lib/ethdev/ethdev_driver.h | 38 +++ >> 14 files changed, 766 insertions(+), 130 deletions(-) create mode 100644 >> drivers/net/bonding/rte_eth_bond_flow.h >> >> -- >> 2.39.1 > ^ permalink raw reply [flat|nested] 80+ messages in thread
* [PATCH v4 0/6] Enhance the bond framework to support offload 2023-10-08 1:50 ` [PATCH v3 0/8] Enhance the bond framework to support offload Chaoyong He ` (8 preceding siblings ...) 2023-10-13 2:22 ` [PATCH v3 0/8] Enhance the bond framework to support offload Chaoyong He @ 2023-10-18 7:48 ` Chaoyong He 2023-10-18 7:48 ` [PATCH v4 1/6] ethdev: add member notification for bonding port Chaoyong He ` (7 more replies) 9 siblings, 8 replies; 80+ messages in thread From: Chaoyong He @ 2023-10-18 7:48 UTC (permalink / raw) To: dev; +Cc: oss-drivers, Chaoyong He This patch series try to enhance the bond framework to support the offload feature better: * Add new API to make the member port can access some information of the bond port which belongs. * Add new API to get the result of whether bond port is created by the member port. * Add two command line argument to control if enable member port notification and dedicated queue features. * Add logic to support add ports which share the same PCI address into bond port. * Also modify the testpmd application to test the new APIs and logics added by this patch series. --- v2: * Fix compile error on github-robot by removing the redundancy function declaration in the header file. v3: * Use the hole in the structure for the new added flag data field. v4: * Drop two commits not necessary for this series. * Modify some logic as the review comments from reviewers. --- Long Wu (6): ethdev: add member notification for bonding port ethdev: add API to get hardware creation of bonding port net/bonding: add bonding port arguments net/bonding: support add port by data name net/bonding: support checking valid bonding port ID net/bonding: add commands for bonding port notification .../link_bonding_poll_mode_drv_lib.rst | 19 +++ drivers/net/bonding/bonding_testpmd.c | 128 ++++++++++++++++++ drivers/net/bonding/eth_bond_private.h | 11 ++ drivers/net/bonding/rte_eth_bond.h | 88 ++++++++++++ drivers/net/bonding/rte_eth_bond_api.c | 121 +++++++++++++++++ drivers/net/bonding/rte_eth_bond_args.c | 47 +++++++ drivers/net/bonding/rte_eth_bond_pmd.c | 93 ++++++++++++- drivers/net/bonding/version.map | 5 + lib/ethdev/ethdev_driver.h | 38 ++++++ 9 files changed, 546 insertions(+), 4 deletions(-) -- 2.39.1 ^ permalink raw reply [flat|nested] 80+ messages in thread
* [PATCH v4 1/6] ethdev: add member notification for bonding port 2023-10-18 7:48 ` [PATCH v4 0/6] " Chaoyong He @ 2023-10-18 7:48 ` Chaoyong He 2023-10-18 7:48 ` [PATCH v4 2/6] ethdev: add API to get hardware creation of " Chaoyong He ` (6 subsequent siblings) 7 siblings, 0 replies; 80+ messages in thread From: Chaoyong He @ 2023-10-18 7:48 UTC (permalink / raw) To: dev; +Cc: oss-drivers, Long Wu, James Hershaw, Chaoyong He From: Long Wu <long.wu@corigine.com> Bonding PMD does not let member ports know the bonding port's information, like how many member ports the bonding port has, what mode the bonding port is in and so on. Add the notification interface for bonding port to let member port know it is added to a bonding port and what the bonding port's configuration is. If so the member ports have chance to achieve its bond-flow-offlod or other private bonding functions. Signed-off-by: Long Wu <long.wu@corigine.com> Reviewed-by: James Hershaw <james.hershaw@corigine.com> Reviewed-by: Chaoyong He <chaoyong.he@corigine.com> --- drivers/net/bonding/eth_bond_private.h | 1 + drivers/net/bonding/rte_eth_bond.h | 46 ++++++++++++++++ drivers/net/bonding/rte_eth_bond_api.c | 72 ++++++++++++++++++++++++++ drivers/net/bonding/rte_eth_bond_pmd.c | 32 ++++++++++-- drivers/net/bonding/version.map | 3 ++ lib/ethdev/ethdev_driver.h | 18 +++++++ 6 files changed, 169 insertions(+), 3 deletions(-) diff --git a/drivers/net/bonding/eth_bond_private.h b/drivers/net/bonding/eth_bond_private.h index e688894210..f69e85c199 100644 --- a/drivers/net/bonding/eth_bond_private.h +++ b/drivers/net/bonding/eth_bond_private.h @@ -180,6 +180,7 @@ struct bond_dev_private { uint8_t member_update_idx; bool kvargs_processing_is_done; + bool notify_member; /**< Enable member notification of bonding port. */ uint32_t candidate_max_rx_pktlen; uint32_t max_rx_pktlen; diff --git a/drivers/net/bonding/rte_eth_bond.h b/drivers/net/bonding/rte_eth_bond.h index f10165f2c6..f6c773615c 100644 --- a/drivers/net/bonding/rte_eth_bond.h +++ b/drivers/net/bonding/rte_eth_bond.h @@ -351,6 +351,52 @@ rte_eth_bond_link_up_prop_delay_set(uint16_t bonding_port_id, int rte_eth_bond_link_up_prop_delay_get(uint16_t bonding_port_id); +/** + * Set the flag of whether bonding port notifies member ports. + * + * @param bonding_port_id + * Port ID of bonding device. + * @param notify + * Flag of whether bonding port notifies member ports. + * + * @return + * 0 on success, negative value otherwise. + */ +__rte_experimental +int +rte_eth_bond_notify_member_flag_set(uint16_t bonding_port_id, bool notify); + +/** + * Get the flag of whether bonding port notifies member ports. + * + * @param bonding_port_id + * Port ID of bonding device. + * @param notify + * Flag of whether bonding port notifies member ports. + * + * @return + * 0 on success, negative value otherwise. + */ +__rte_experimental +int +rte_eth_bond_notify_member_flag_get(uint16_t bonding_port_id, bool *notify); + +/** + * Notify the member ports of bonding port's information. + * + * This interface is called in the following functions: + * - bond_ethdev_lsc_event_callback() + * - bond_ethdev_configure() + * + * @param bonding_port_id + * Port ID of bonding device. + * + * @return + * 0 on success, negative value otherwise. + */ +__rte_experimental +int +rte_eth_bond_notify_members(uint16_t bonding_port_id); #ifdef __cplusplus } diff --git a/drivers/net/bonding/rte_eth_bond_api.c b/drivers/net/bonding/rte_eth_bond_api.c index 99e496556a..239f86ee92 100644 --- a/drivers/net/bonding/rte_eth_bond_api.c +++ b/drivers/net/bonding/rte_eth_bond_api.c @@ -627,6 +627,17 @@ __eth_bond_member_add_lock_free(uint16_t bonding_port_id, uint16_t member_port_i member_vlan_filter_set(bonding_port_id, member_port_id); + if (internals->notify_member && + *member_eth_dev->dev_ops->bond_notify_member != NULL) { + ret = member_eth_dev->dev_ops->bond_notify_member(member_eth_dev, + bonding_eth_dev); + if (ret < 0) { + RTE_BOND_LOG(ERR, "Add member (port %u) notify failed!", + member_port_id); + return -1; + } + } + return 0; } @@ -733,6 +744,10 @@ __eth_bond_member_remove_lock_free(uint16_t bonding_port_id, member_eth_dev = &rte_eth_devices[member_port_id]; member_remove(internals, member_eth_dev); member_eth_dev->data->dev_flags &= (~RTE_ETH_DEV_BONDING_MEMBER); + if (internals->notify_member && + *member_eth_dev->dev_ops->bond_notify_member != NULL) + member_eth_dev->dev_ops->bond_notify_member(member_eth_dev, + bonding_eth_dev); /* first member in the active list will be the primary by default, * otherwise use first device in list */ @@ -1098,3 +1113,60 @@ rte_eth_bond_link_up_prop_delay_get(uint16_t bonding_port_id) return internals->link_up_delay_ms; } + +int +rte_eth_bond_notify_member_flag_set(uint16_t bonding_port_id, bool notify) +{ + struct bond_dev_private *internals; + + if (valid_bonding_port_id(bonding_port_id) != 0) + return -EINVAL; + + internals = rte_eth_devices[bonding_port_id].data->dev_private; + + internals->notify_member = notify; + + return 0; +} + +int +rte_eth_bond_notify_member_flag_get(uint16_t bonding_port_id, bool *notify) +{ + struct bond_dev_private *internals; + + if (valid_bonding_port_id(bonding_port_id) != 0) + return -EINVAL; + + internals = rte_eth_devices[bonding_port_id].data->dev_private; + + *notify = internals->notify_member; + + return 0; +} + +int +rte_eth_bond_notify_members(uint16_t bonding_port_id) +{ + uint32_t i; + uint16_t member_port_id; + struct rte_eth_dev *bond_dev; + struct rte_eth_dev *member_dev; + struct bond_dev_private *internals; + + if (valid_bonding_port_id(bonding_port_id) != 0) + return -EINVAL; + + bond_dev = &rte_eth_devices[bonding_port_id]; + internals = bond_dev->data->dev_private; + + for (i = 0; i < internals->member_count; i++) { + member_port_id = internals->members[i].port_id; + member_dev = &rte_eth_devices[member_port_id]; + /* Notify member port if it supports. */ + if (*member_dev->dev_ops->bond_notify_member != NULL) + member_dev->dev_ops->bond_notify_member(member_dev, + bond_dev); + } + + return 0; +} diff --git a/drivers/net/bonding/rte_eth_bond_pmd.c b/drivers/net/bonding/rte_eth_bond_pmd.c index 122b1187fd..fc7f0f364f 100644 --- a/drivers/net/bonding/rte_eth_bond_pmd.c +++ b/drivers/net/bonding/rte_eth_bond_pmd.c @@ -2968,11 +2968,13 @@ bond_ethdev_lsc_event_callback(uint16_t port_id, enum rte_eth_event_type type, int valid_member = 0; uint16_t active_pos, member_idx; uint16_t i; + uint16_t bonding_port_id; if (type != RTE_ETH_EVENT_INTR_LSC || param == NULL) return rc; - bonding_eth_dev = &rte_eth_devices[*(uint16_t *)param]; + bonding_port_id = *(uint16_t *)param; + bonding_eth_dev = &rte_eth_devices[bonding_port_id]; if (check_for_bonding_ethdev(bonding_eth_dev)) return rc; @@ -3044,8 +3046,12 @@ bond_ethdev_lsc_event_callback(uint16_t port_id, enum rte_eth_event_type type, * using it. */ if (internals->user_defined_primary_port && - internals->primary_port == port_id) + internals->primary_port == port_id) { bond_ethdev_primary_set(internals, port_id); + + if (internals->notify_member) + rte_eth_bond_notify_members(bonding_port_id); + } } else { if (active_pos == internals->active_member_count) goto link_update; @@ -3064,6 +3070,10 @@ bond_ethdev_lsc_event_callback(uint16_t port_id, enum rte_eth_event_type type, internals->active_members[0]); else internals->current_primary_port = internals->primary_port; + + if (internals->notify_member) + rte_eth_bond_notify_members(bonding_port_id); + mac_address_members_update(bonding_eth_dev); bond_ethdev_promiscuous_update(bonding_eth_dev); bond_ethdev_allmulticast_update(bonding_eth_dev); @@ -3362,6 +3372,7 @@ dump_basic(const struct rte_eth_dev *dev, FILE *f) struct bond_dev_private instant_priv; const struct bond_dev_private *internals = &instant_priv; int mode, i; + bool notify_member; /* Obtain a instance of dev_private to prevent data from being modified. */ memcpy(&instant_priv, dev->data->dev_private, sizeof(struct bond_dev_private)); @@ -3431,6 +3442,13 @@ dump_basic(const struct rte_eth_dev *dev, FILE *f) fprintf(f, "\tUser Defined Primary: [%u]\n", internals->primary_port); if (internals->member_count > 0) fprintf(f, "\tCurrent Primary: [%u]\n", internals->current_primary_port); + + if (rte_eth_bond_notify_member_flag_get(internals->port_id, ¬ify_member) == 0) + fprintf(f, "\tNotify Member Ports Flag: %s\n", + notify_member ? "enable" : "disable"); + else + fprintf(f, "\tFailed to get notify member ports flag for bonding port %d\n", + internals->port_id); } static void @@ -3983,8 +4001,12 @@ bond_ethdev_configure(struct rte_eth_dev *dev) * if no kvlist, it means that this bonding device has been created * through the bonding api. */ - if (!kvlist || internals->kvargs_processing_is_done) + if (!kvlist || internals->kvargs_processing_is_done) { + if (internals->notify_member && rte_eth_bond_notify_members(port_id) != 0) + RTE_BOND_LOG(ERR, "Notify member ports failed"); + return 0; + } internals->kvargs_processing_is_done = true; @@ -4222,6 +4244,10 @@ bond_ethdev_configure(struct rte_eth_dev *dev) return -1; } } + + if (internals->notify_member && rte_eth_bond_notify_members(port_id) != 0) + RTE_BOND_LOG(ERR, "Notify member ports failed"); + return 0; } diff --git a/drivers/net/bonding/version.map b/drivers/net/bonding/version.map index 09ee21c55f..3bd5e8ad11 100644 --- a/drivers/net/bonding/version.map +++ b/drivers/net/bonding/version.map @@ -35,4 +35,7 @@ EXPERIMENTAL { rte_eth_bond_member_add; rte_eth_bond_member_remove; rte_eth_bond_members_get; + rte_eth_bond_notify_member_flag_get; + rte_eth_bond_notify_member_flag_set; + rte_eth_bond_notify_members; }; diff --git a/lib/ethdev/ethdev_driver.h b/lib/ethdev/ethdev_driver.h index deb23ada18..f626f971e5 100644 --- a/lib/ethdev/ethdev_driver.h +++ b/lib/ethdev/ethdev_driver.h @@ -1216,6 +1216,21 @@ typedef int (*eth_count_aggr_ports_t)(struct rte_eth_dev *dev); typedef int (*eth_map_aggr_tx_affinity_t)(struct rte_eth_dev *dev, uint16_t tx_queue_id, uint8_t affinity); +/** + * @internal + * Bonding port notifies the member ports. + * + * @param dev + * Member port (ethdev) handle. + * @param bonding_dev + * Bonding port (ethdev) handle. + * + * @return + * Negative on error, 0 on success. + */ +typedef int (*eth_bond_notify_member)(struct rte_eth_dev *dev, + struct rte_eth_dev *bonding_dev); + /** * @internal A structure containing the functions exported by an Ethernet driver. */ @@ -1455,6 +1470,9 @@ struct eth_dev_ops { eth_count_aggr_ports_t count_aggr_ports; /** Map a Tx queue with an aggregated port of the DPDK port */ eth_map_aggr_tx_affinity_t map_aggr_tx_affinity; + + /** Notify the member port of bonding port information */ + eth_bond_notify_member bond_notify_member; }; /** -- 2.39.1 ^ permalink raw reply [flat|nested] 80+ messages in thread
* [PATCH v4 2/6] ethdev: add API to get hardware creation of bonding port 2023-10-18 7:48 ` [PATCH v4 0/6] " Chaoyong He 2023-10-18 7:48 ` [PATCH v4 1/6] ethdev: add member notification for bonding port Chaoyong He @ 2023-10-18 7:48 ` Chaoyong He 2023-10-18 7:48 ` [PATCH v4 3/6] net/bonding: add bonding port arguments Chaoyong He ` (5 subsequent siblings) 7 siblings, 0 replies; 80+ messages in thread From: Chaoyong He @ 2023-10-18 7:48 UTC (permalink / raw) To: dev; +Cc: oss-drivers, Long Wu, James Hershaw, Chaoyong He From: Long Wu <long.wu@corigine.com> After bonding port notification, member port hardware may create the bonding port. We want to get the result of creatition, so we add this API to do the getting action. Signed-off-by: Long Wu <long.wu@corigine.com> Reviewed-by: James Hershaw <james.hershaw@corigine.com> Reviewed-by: Chaoyong He <chaoyong.he@corigine.com> --- drivers/net/bonding/rte_eth_bond.h | 15 ++++++++++++++ drivers/net/bonding/rte_eth_bond_api.c | 28 ++++++++++++++++++++++++++ drivers/net/bonding/version.map | 1 + lib/ethdev/ethdev_driver.h | 20 ++++++++++++++++++ 4 files changed, 64 insertions(+) diff --git a/drivers/net/bonding/rte_eth_bond.h b/drivers/net/bonding/rte_eth_bond.h index f6c773615c..987269b323 100644 --- a/drivers/net/bonding/rte_eth_bond.h +++ b/drivers/net/bonding/rte_eth_bond.h @@ -398,6 +398,21 @@ __rte_experimental int rte_eth_bond_notify_members(uint16_t bonding_port_id); +/** + * Get the status of specified bonding port created by member port hardware. + * + * @param bonding_port_id + * Port ID of bonding device. + * @param member_port_id + * Port ID of member device. + * + * @return + * 0 on success, negative value otherwise. + */ +__rte_experimental +int +rte_eth_bond_hw_create_get(uint16_t bonding_port_id, uint16_t member_port_id); + #ifdef __cplusplus } #endif diff --git a/drivers/net/bonding/rte_eth_bond_api.c b/drivers/net/bonding/rte_eth_bond_api.c index 239f86ee92..317c3c1542 100644 --- a/drivers/net/bonding/rte_eth_bond_api.c +++ b/drivers/net/bonding/rte_eth_bond_api.c @@ -1170,3 +1170,31 @@ rte_eth_bond_notify_members(uint16_t bonding_port_id) return 0; } + +int +rte_eth_bond_hw_create_get(uint16_t bonding_port_id, uint16_t member_port_id) +{ + uint32_t i; + struct rte_eth_dev *bonding_dev; + struct rte_eth_dev *member_dev; + struct bond_dev_private *internals; + + if (valid_bonding_port_id(bonding_port_id) != 0) + return -EINVAL; + + bonding_dev = &rte_eth_devices[bonding_port_id]; + internals = bonding_dev->data->dev_private; + for (i = 0; i < internals->member_count; i++) { + if (internals->members[i].port_id == member_port_id) + break; + } + + if (i == internals->member_count) + return -EINVAL; + + member_dev = &rte_eth_devices[member_port_id]; + if (*member_dev->dev_ops->bond_hw_create_get == NULL) + return -ENOTSUP; + + return member_dev->dev_ops->bond_hw_create_get(member_dev, bonding_dev); +} diff --git a/drivers/net/bonding/version.map b/drivers/net/bonding/version.map index 3bd5e8ad11..3cfff51269 100644 --- a/drivers/net/bonding/version.map +++ b/drivers/net/bonding/version.map @@ -32,6 +32,7 @@ EXPERIMENTAL { global: rte_eth_bond_8023ad_member_info; rte_eth_bond_active_members_get; + rte_eth_bond_hw_create_get; rte_eth_bond_member_add; rte_eth_bond_member_remove; rte_eth_bond_members_get; diff --git a/lib/ethdev/ethdev_driver.h b/lib/ethdev/ethdev_driver.h index f626f971e5..18ff5db969 100644 --- a/lib/ethdev/ethdev_driver.h +++ b/lib/ethdev/ethdev_driver.h @@ -1231,6 +1231,21 @@ typedef int (*eth_map_aggr_tx_affinity_t)(struct rte_eth_dev *dev, uint16_t tx_q typedef int (*eth_bond_notify_member)(struct rte_eth_dev *dev, struct rte_eth_dev *bonding_dev); +/** + * @internal + * Get the status of specified bonding port created by member port hardware. + * + * @param dev + * Member port (ethdev) handle. + * @param bonding_dev + * Bonding port (ethdev) handle. + * + * @return + * Negative on error, 0 on success. + */ +typedef int (*eth_bond_hw_create_get)(struct rte_eth_dev *dev, + struct rte_eth_dev *bonding_dev); + /** * @internal A structure containing the functions exported by an Ethernet driver. */ @@ -1473,6 +1488,11 @@ struct eth_dev_ops { /** Notify the member port of bonding port information */ eth_bond_notify_member bond_notify_member; + /** + * Get the status of whether bonding port is successfully created by + * the member port hardware. + */ + eth_bond_hw_create_get bond_hw_create_get; }; /** -- 2.39.1 ^ permalink raw reply [flat|nested] 80+ messages in thread
* [PATCH v4 3/6] net/bonding: add bonding port arguments 2023-10-18 7:48 ` [PATCH v4 0/6] " Chaoyong He 2023-10-18 7:48 ` [PATCH v4 1/6] ethdev: add member notification for bonding port Chaoyong He 2023-10-18 7:48 ` [PATCH v4 2/6] ethdev: add API to get hardware creation of " Chaoyong He @ 2023-10-18 7:48 ` Chaoyong He 2023-10-18 7:48 ` [PATCH v4 4/6] net/bonding: support add port by data name Chaoyong He ` (4 subsequent siblings) 7 siblings, 0 replies; 80+ messages in thread From: Chaoyong He @ 2023-10-18 7:48 UTC (permalink / raw) To: dev; +Cc: oss-drivers, Long Wu, James Hershaw, Chaoyong He From: Long Wu <long.wu@corigine.com> Include the following new arguments for bonding ports: - "notify_member" to enable/disable member notification. - "dedicated_queue" to enable/disable dedicated queue. Add these two arguments in initial argument. Signed-off-by: Long Wu <long.wu@corigine.com> Reviewed-by: James Hershaw <james.hershaw@corigine.com> Reviewed-by: Chaoyong He <chaoyong.he@corigine.com> --- drivers/net/bonding/eth_bond_private.h | 10 ++++ drivers/net/bonding/rte_eth_bond.h | 14 ++++++ drivers/net/bonding/rte_eth_bond_api.c | 14 ++++++ drivers/net/bonding/rte_eth_bond_args.c | 44 ++++++++++++++++++ drivers/net/bonding/rte_eth_bond_pmd.c | 61 ++++++++++++++++++++++++- 5 files changed, 142 insertions(+), 1 deletion(-) diff --git a/drivers/net/bonding/eth_bond_private.h b/drivers/net/bonding/eth_bond_private.h index f69e85c199..f9603a0f6b 100644 --- a/drivers/net/bonding/eth_bond_private.h +++ b/drivers/net/bonding/eth_bond_private.h @@ -28,6 +28,8 @@ #define PMD_BOND_LSC_POLL_PERIOD_KVARG ("lsc_poll_period_ms") #define PMD_BOND_LINK_UP_PROP_DELAY_KVARG ("up_delay") #define PMD_BOND_LINK_DOWN_PROP_DELAY_KVARG ("down_delay") +#define PMD_BOND_NOTIFY_MEMBER_KVARG ("notify_member") +#define PMD_BOND_DEDICATED_QUEUE_KVARG ("dedicated_queue") #define PMD_BOND_XMIT_POLICY_LAYER2_KVARG ("l2") #define PMD_BOND_XMIT_POLICY_LAYER23_KVARG ("l23") @@ -319,6 +321,14 @@ int bond_ethdev_parse_time_ms_kvarg(const char *key, const char *value, void *extra_args); +int +bond_ethdev_parse_notify_member_kvarg(const char *key __rte_unused, + const char *value, void *extra_args); + +int +bond_ethdev_parse_dedicated_queue_kvarg(const char *key __rte_unused, + const char *value, void *extra_args); + void bond_tlb_disable(struct bond_dev_private *internals); diff --git a/drivers/net/bonding/rte_eth_bond.h b/drivers/net/bonding/rte_eth_bond.h index 987269b323..936ab8c3a0 100644 --- a/drivers/net/bonding/rte_eth_bond.h +++ b/drivers/net/bonding/rte_eth_bond.h @@ -351,6 +351,20 @@ rte_eth_bond_link_up_prop_delay_set(uint16_t bonding_port_id, int rte_eth_bond_link_up_prop_delay_get(uint16_t bonding_port_id); +/** + * Set the flag that whether bonding device enable dedicated queue. + * + * @param bonding_port_id + * Port ID of bonding device. + * @param queue_flag + * The flag of enable bond dedicated queue + * + * @return + * 0 on success, negative value otherwise. + */ +int +rte_eth_bond_dedicated_queue_flag_set(uint16_t bonding_port_id, bool queue_flag); + /** * Set the flag of whether bonding port notifies member ports. * diff --git a/drivers/net/bonding/rte_eth_bond_api.c b/drivers/net/bonding/rte_eth_bond_api.c index 317c3c1542..656ddd35a7 100644 --- a/drivers/net/bonding/rte_eth_bond_api.c +++ b/drivers/net/bonding/rte_eth_bond_api.c @@ -1114,6 +1114,20 @@ rte_eth_bond_link_up_prop_delay_get(uint16_t bonding_port_id) return internals->link_up_delay_ms; } +int +rte_eth_bond_dedicated_queue_flag_set(uint16_t bonding_port_id, bool queue_flag) +{ + struct bond_dev_private *internals; + + if (valid_bonding_port_id(bonding_port_id) != 0) + return -1; + + internals = rte_eth_devices[bonding_port_id].data->dev_private; + internals->mode4.dedicated_queues.enabled = queue_flag; + + return 0; +} + int rte_eth_bond_notify_member_flag_set(uint16_t bonding_port_id, bool notify) { diff --git a/drivers/net/bonding/rte_eth_bond_args.c b/drivers/net/bonding/rte_eth_bond_args.c index bdec5d61d4..8a3e4656ef 100644 --- a/drivers/net/bonding/rte_eth_bond_args.c +++ b/drivers/net/bonding/rte_eth_bond_args.c @@ -20,6 +20,8 @@ const char *pmd_bond_init_valid_arguments[] = { PMD_BOND_MAC_ADDR_KVARG, PMD_BOND_AGG_MODE_KVARG, RTE_DEVARGS_KEY_DRIVER, + PMD_BOND_NOTIFY_MEMBER_KVARG, + PMD_BOND_DEDICATED_QUEUE_KVARG, NULL }; @@ -297,3 +299,45 @@ bond_ethdev_parse_time_ms_kvarg(const char *key __rte_unused, return 0; } + +int +bond_ethdev_parse_notify_member_kvarg(const char *key __rte_unused, + const char *value, void *extra_args) +{ + bool *notify_member; + + if (value == NULL || extra_args == NULL) + return -1; + + notify_member = extra_args; + + if (strcmp("enable", value) == 0) + *notify_member = true; + else if (strcmp("disable", value) == 0) + *notify_member = false; + else + return -1; + + return 0; +} + +int +bond_ethdev_parse_dedicated_queue_kvarg(const char *key __rte_unused, + const char *value, void *extra_args) +{ + bool *dedicated_queue; + + if (value == NULL || extra_args == NULL) + return -1; + + dedicated_queue = extra_args; + + if (strcmp("enable", value) == 0) + *dedicated_queue = true; + else if (strcmp("disable", value) == 0) + *dedicated_queue = false; + else + return -1; + + return 0; +} diff --git a/drivers/net/bonding/rte_eth_bond_pmd.c b/drivers/net/bonding/rte_eth_bond_pmd.c index fc7f0f364f..4bede0aa71 100644 --- a/drivers/net/bonding/rte_eth_bond_pmd.c +++ b/drivers/net/bonding/rte_eth_bond_pmd.c @@ -4232,6 +4232,63 @@ bond_ethdev_configure(struct rte_eth_dev *dev) return -1; } + /* Parse/set notify member flag */ + arg_count = rte_kvargs_count(kvlist, PMD_BOND_NOTIFY_MEMBER_KVARG); + if (arg_count == 1) { + bool notify_member; + if (rte_kvargs_process(kvlist, + PMD_BOND_NOTIFY_MEMBER_KVARG, + &bond_ethdev_parse_notify_member_kvarg, + ¬ify_member) < 0) { + RTE_BOND_LOG(ERR, + "Invalid notify member value specified" + " for bonding device %s", name); + return -1; + } + + if (rte_eth_bond_notify_member_flag_set(port_id, notify_member) != 0) { + RTE_BOND_LOG(ERR, + "Failed to set notify member (%u) on" + " bonding device %s", notify_member, name); + return -1; + } + } else if (arg_count > 1) { + RTE_BOND_LOG(ERR, + "notify member flag can be specified only once" + " for bonding device %s", name); + return -1; + } + + /* Parse/set dedicated queue flag */ + arg_count = rte_kvargs_count(kvlist, PMD_BOND_DEDICATED_QUEUE_KVARG); + if (arg_count == 1) { + bool dedicated_queue; + if (rte_kvargs_process(kvlist, + PMD_BOND_DEDICATED_QUEUE_KVARG, + &bond_ethdev_parse_dedicated_queue_kvarg, + &dedicated_queue) < 0) { + RTE_BOND_LOG(ERR, + "Invalid dedicated queue flag specified" + " for bonding device %s", name); + return -1; + } + + if (internals->mode == BONDING_MODE_8023AD) { + if (rte_eth_bond_dedicated_queue_flag_set(port_id, dedicated_queue) != 0) { + RTE_BOND_LOG(ERR, + "Failed to enable/disable dedicated" + " queue flag on bonding device %s", + name); + return -1; + } + } + } else if (arg_count > 1) { + RTE_BOND_LOG(ERR, + "dedicated queue flag can be specified only once" + " for bonding device %s", name); + return -1; + } + /* configure members so we can pass mtu setting */ for (i = 0; i < internals->member_count; i++) { struct rte_eth_dev *member_ethdev = @@ -4269,7 +4326,9 @@ RTE_PMD_REGISTER_PARAM_STRING(net_bonding, "mac=<mac addr> " "lsc_poll_period_ms=<int> " "up_delay=<int> " - "down_delay=<int>"); + "down_delay=<int>" + "notify_member=[enable | disable] " + "dedicated_queue=[enable | disable] "); /* We can't use RTE_LOG_REGISTER_DEFAULT because of the forced name for * this library, see meson.build. -- 2.39.1 ^ permalink raw reply [flat|nested] 80+ messages in thread
* [PATCH v4 4/6] net/bonding: support add port by data name 2023-10-18 7:48 ` [PATCH v4 0/6] " Chaoyong He ` (2 preceding siblings ...) 2023-10-18 7:48 ` [PATCH v4 3/6] net/bonding: add bonding port arguments Chaoyong He @ 2023-10-18 7:48 ` Chaoyong He 2023-10-18 7:48 ` [PATCH v4 5/6] net/bonding: support checking valid bonding port ID Chaoyong He ` (3 subsequent siblings) 7 siblings, 0 replies; 80+ messages in thread From: Chaoyong He @ 2023-10-18 7:48 UTC (permalink / raw) To: dev; +Cc: oss-drivers, Long Wu, James Hershaw, Chaoyong He From: Long Wu <long.wu@corigine.com> Several ports may share the same PCI address, like nfp representor. So we cannot add this type of ports to bonding port by "--vdev" argument in dpdk-testpmd. But the port's data name is unique between them, we include an option to add such ports to the bonding port. After adding this feature, we can create a bonding port that member port is this type of port by "--vdev" in dpdk-testpmd start command. For example: dpdk-testpmd -l 2-10 -s 0x8 -a ca:00.0,representor=[0-2] --vdev 'net_bonding0,member=flower_repr_p0,member=flower_repr_p1, mode=4,socket_id=1,xmit_policy=l34' -- -i Note: 1. "ca:00.0" is nfp 4000 card. 2. "flower_repr_p0" and "flower_repr_p1" are nfp phy representor's data name. Signed-off-by: Long Wu <long.wu@corigine.com> Reviewed-by: James Hershaw <james.hershaw@corigine.com> Reviewed-by: Chaoyong He <chaoyong.he@corigine.com> --- drivers/net/bonding/rte_eth_bond_args.c | 3 +++ 1 file changed, 3 insertions(+) diff --git a/drivers/net/bonding/rte_eth_bond_args.c b/drivers/net/bonding/rte_eth_bond_args.c index 8a3e4656ef..b320eb3038 100644 --- a/drivers/net/bonding/rte_eth_bond_args.c +++ b/drivers/net/bonding/rte_eth_bond_args.c @@ -70,6 +70,9 @@ find_port_id_by_dev_name(const char *name) if (strcmp(rte_eth_devices[i].device->name, name) == 0) return i; + + if (strcmp(rte_eth_devices[i].data->name, name) == 0) + return i; } return -1; } -- 2.39.1 ^ permalink raw reply [flat|nested] 80+ messages in thread
* [PATCH v4 5/6] net/bonding: support checking valid bonding port ID 2023-10-18 7:48 ` [PATCH v4 0/6] " Chaoyong He ` (3 preceding siblings ...) 2023-10-18 7:48 ` [PATCH v4 4/6] net/bonding: support add port by data name Chaoyong He @ 2023-10-18 7:48 ` Chaoyong He 2023-10-18 7:48 ` [PATCH v4 6/6] net/bonding: add commands for bonding port notification Chaoyong He ` (2 subsequent siblings) 7 siblings, 0 replies; 80+ messages in thread From: Chaoyong He @ 2023-10-18 7:48 UTC (permalink / raw) To: dev; +Cc: oss-drivers, Long Wu, James Hershaw, Chaoyong He From: Long Wu <long.wu@corigine.com> Add API to support checking if the port id is a bonding port id. Signed-off-by: Long Wu <long.wu@corigine.com> Reviewed-by: James Hershaw <james.hershaw@corigine.com> Reviewed-by: Chaoyong He <chaoyong.he@corigine.com> --- drivers/net/bonding/rte_eth_bond.h | 13 +++++++++++++ drivers/net/bonding/rte_eth_bond_api.c | 7 +++++++ drivers/net/bonding/version.map | 1 + 3 files changed, 21 insertions(+) diff --git a/drivers/net/bonding/rte_eth_bond.h b/drivers/net/bonding/rte_eth_bond.h index 936ab8c3a0..02ddb496bb 100644 --- a/drivers/net/bonding/rte_eth_bond.h +++ b/drivers/net/bonding/rte_eth_bond.h @@ -427,6 +427,19 @@ __rte_experimental int rte_eth_bond_hw_create_get(uint16_t bonding_port_id, uint16_t member_port_id); +/** + * Check whether bonding port id is valid. + * + * @param port_id + * Port ID of bonding device. + * + * @return + * true means the port is a bonding device, false means not. + */ +__rte_experimental +bool +rte_eth_bond_is_valid_port(uint16_t port_id); + #ifdef __cplusplus } #endif diff --git a/drivers/net/bonding/rte_eth_bond_api.c b/drivers/net/bonding/rte_eth_bond_api.c index 656ddd35a7..07efc7a7e5 100644 --- a/drivers/net/bonding/rte_eth_bond_api.c +++ b/drivers/net/bonding/rte_eth_bond_api.c @@ -1212,3 +1212,10 @@ rte_eth_bond_hw_create_get(uint16_t bonding_port_id, uint16_t member_port_id) return member_dev->dev_ops->bond_hw_create_get(member_dev, bonding_dev); } + + +bool +rte_eth_bond_is_valid_port(uint16_t port_id) +{ + return (valid_bonding_port_id(port_id) == 0); +} diff --git a/drivers/net/bonding/version.map b/drivers/net/bonding/version.map index 3cfff51269..97ef24dcdb 100644 --- a/drivers/net/bonding/version.map +++ b/drivers/net/bonding/version.map @@ -33,6 +33,7 @@ EXPERIMENTAL { rte_eth_bond_8023ad_member_info; rte_eth_bond_active_members_get; rte_eth_bond_hw_create_get; + rte_eth_bond_is_valid_port; rte_eth_bond_member_add; rte_eth_bond_member_remove; rte_eth_bond_members_get; -- 2.39.1 ^ permalink raw reply [flat|nested] 80+ messages in thread
* [PATCH v4 6/6] net/bonding: add commands for bonding port notification 2023-10-18 7:48 ` [PATCH v4 0/6] " Chaoyong He ` (4 preceding siblings ...) 2023-10-18 7:48 ` [PATCH v4 5/6] net/bonding: support checking valid bonding port ID Chaoyong He @ 2023-10-18 7:48 ` Chaoyong He 2023-11-15 16:01 ` [PATCH v4 0/6] Enhance the bond framework to support offload Ferruh Yigit 2023-12-26 2:37 ` [PATCH v5 00/14] " Chaoyong He 7 siblings, 0 replies; 80+ messages in thread From: Chaoyong He @ 2023-10-18 7:48 UTC (permalink / raw) To: dev; +Cc: oss-drivers, Long Wu, James Hershaw, Chaoyong He From: Long Wu <long.wu@corigine.com> Add some commands to support bonding port notification in dpdk-testpmd. 1. We can enable the notification by command: "set bonding notify_member (port_id) (enable|disable)" 2. If member port hardware try to create the bonding port after notification we can get the status by command: "get bonding member hardware create (member_port_id) (bonding_port_id)" Signed-off-by: Long Wu <long.wu@corigine.com> Reviewed-by: James Hershaw <james.hershaw@corigine.com> Reviewed-by: Chaoyong He <chaoyong.he@corigine.com> --- .../link_bonding_poll_mode_drv_lib.rst | 19 +++ drivers/net/bonding/bonding_testpmd.c | 128 ++++++++++++++++++ 2 files changed, 147 insertions(+) diff --git a/doc/guides/prog_guide/link_bonding_poll_mode_drv_lib.rst b/doc/guides/prog_guide/link_bonding_poll_mode_drv_lib.rst index 60717a3587..9f6443ebd8 100644 --- a/doc/guides/prog_guide/link_bonding_poll_mode_drv_lib.rst +++ b/doc/guides/prog_guide/link_bonding_poll_mode_drv_lib.rst @@ -637,3 +637,22 @@ in balance mode with a transmission policy of layer 2+3:: Members (3): [1 3 4] Active Members (3): [1 3 4] Primary: [3] + +set bonding notify_member +~~~~~~~~~~~~~~~~~~~~~~~~~ + +Set the notify member flag of bonding port:: + + testpmd> set bonding notify_member (port_id) (enable|disable) + +This command just set the flag of notification. +If we enable it, bonding PMD will notify member ports when its some +configurations changed. So member ports can do some private things, maybe hardware +bonding creation and etc. + +get bonding member hardware create +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +Get the status of member port hardware creating the bonding port:: + + testpmd> get bonding member hardware create (member_port_id) (bonding_port_id) diff --git a/drivers/net/bonding/bonding_testpmd.c b/drivers/net/bonding/bonding_testpmd.c index 8fcd6cadd0..da7d9cc58f 100644 --- a/drivers/net/bonding/bonding_testpmd.c +++ b/drivers/net/bonding/bonding_testpmd.c @@ -692,6 +692,124 @@ static cmdline_parse_inst_t cmd_set_bonding_agg_mode_policy = { } }; +struct cmd_set_bonding_notify_member_result { + cmdline_fixed_string_t set; + cmdline_fixed_string_t bonding; + cmdline_fixed_string_t notify_member; + uint16_t port_num; + cmdline_fixed_string_t mode; +}; + +static void +cmd_set_bonding_notify_member_parsed(void *parsed_result, + __rte_unused struct cmdline *cl, __rte_unused void *data) +{ + struct cmd_set_bonding_notify_member_result *res = parsed_result; + bool notify_member = false; + + if (strcmp(res->notify_member, "enable") == 0) + notify_member = true; + else if (strcmp(res->notify_member, "disable") == 0) + notify_member = false; + + rte_eth_bond_notify_member_flag_set(res->port_num, notify_member); +} + +static cmdline_parse_token_string_t cmd_set_bonding_notify_member_set = + TOKEN_STRING_INITIALIZER(struct cmd_set_bonding_notify_member_result, + set, "set"); +static cmdline_parse_token_string_t cmd_set_bonding_notify_member_bonding = + TOKEN_STRING_INITIALIZER(struct cmd_set_bonding_notify_member_result, + bonding, "bonding"); +static cmdline_parse_token_string_t cmd_set_bonding_notify_member = + TOKEN_STRING_INITIALIZER(struct cmd_set_bonding_notify_member_result, + notify_member, "notify_member"); +static cmdline_parse_token_num_t cmd_set_bonding_notify_member_portnum = + TOKEN_NUM_INITIALIZER(struct cmd_set_bonding_notify_member_result, + port_num, RTE_UINT16); +static cmdline_parse_token_string_t cmd_set_bonding_notify_member_mode_string = + TOKEN_STRING_INITIALIZER(struct cmd_set_bonding_notify_member_result, + mode, "enable#disable"); + +static cmdline_parse_inst_t cmd_set_bonding_notify_member_ports = { + .f = cmd_set_bonding_notify_member_parsed, + .data = NULL, + .help_str = "set bonding notify_member (port_id) (enable|disable)", + .tokens = { + (void *)&cmd_set_bonding_notify_member_set, + (void *)&cmd_set_bonding_notify_member_bonding, + (void *)&cmd_set_bonding_notify_member, + (void *)&cmd_set_bonding_notify_member_portnum, + (void *)&cmd_set_bonding_notify_member_mode_string, + NULL + } +}; + +struct cmd_get_bonding_member_hw_create_result { + cmdline_fixed_string_t get; + cmdline_fixed_string_t bonding; + cmdline_fixed_string_t member; + cmdline_fixed_string_t hardware; + cmdline_fixed_string_t create; + uint16_t member_port_id; + uint16_t bonding_port_id; +}; + +static void +cmd_get_bonding_member_hw_create_parsed(void *parsed_result, + __rte_unused struct cmdline *cl, __rte_unused void *data) +{ + struct cmd_get_bonding_member_hw_create_result *res = parsed_result; + int ret; + + ret = rte_eth_bond_hw_create_get(res->bonding_port_id, res->member_port_id); + if (ret == 0) + printf("Member port %u hardware creates bonding port %u successfully\n", + res->member_port_id, res->bonding_port_id); + else + printf("Failed to get status of member port %u hardware creating" + " bonding port %u, %d\n", + res->member_port_id, res->bonding_port_id, ret); +} + +static cmdline_parse_token_string_t cmd_get_bonding_member_hw_create_get = + TOKEN_STRING_INITIALIZER(struct cmd_get_bonding_member_hw_create_result, + get, "get"); +static cmdline_parse_token_string_t cmd_get_bonding_member_hw_create_bonding = + TOKEN_STRING_INITIALIZER(struct cmd_get_bonding_member_hw_create_result, + bonding, "bonding"); +static cmdline_parse_token_string_t cmd_get_bonding_member_hw_create_member = + TOKEN_STRING_INITIALIZER(struct cmd_get_bonding_member_hw_create_result, + member, "member"); +static cmdline_parse_token_string_t cmd_get_bonding_member_hw_create_hardware = + TOKEN_STRING_INITIALIZER(struct cmd_get_bonding_member_hw_create_result, + create, "hardware"); +static cmdline_parse_token_string_t cmd_get_bonding_member_hw_create_create = + TOKEN_STRING_INITIALIZER(struct cmd_get_bonding_member_hw_create_result, + create, "create"); +static cmdline_parse_token_num_t cmd_get_bonding_member_hw_create_memberportid = + TOKEN_NUM_INITIALIZER(struct cmd_get_bonding_member_hw_create_result, + member_port_id, RTE_UINT16); +static cmdline_parse_token_num_t cmd_get_bonding_member_hw_create_bondingportid = + TOKEN_NUM_INITIALIZER(struct cmd_get_bonding_member_hw_create_result, + bonding_port_id, RTE_UINT16); + +static cmdline_parse_inst_t cmd_get_member_hw_create_bonding = { + .f = cmd_get_bonding_member_hw_create_parsed, + .data = NULL, + .help_str = "get bonding member hardware create (member_port_id) (bonding_port_id)", + .tokens = { + (void *)&cmd_get_bonding_member_hw_create_get, + (void *)&cmd_get_bonding_member_hw_create_bonding, + (void *)&cmd_get_bonding_member_hw_create_member, + (void *)&cmd_get_bonding_member_hw_create_hardware, + (void *)&cmd_get_bonding_member_hw_create_create, + (void *)&cmd_get_bonding_member_hw_create_memberportid, + (void *)&cmd_get_bonding_member_hw_create_bondingportid, + NULL + } +}; + static struct testpmd_driver_commands bonding_cmds = { .commands = { { @@ -749,6 +867,16 @@ static struct testpmd_driver_commands bonding_cmds = { "set bonding mode IEEE802.3AD aggregator policy (port_id) (agg_name)\n" " Set Aggregation mode for IEEE802.3AD (mode 4)\n", }, + { + &cmd_set_bonding_notify_member_ports, + "set bonding notify_member (port_id) (enable|disable)\n" + " Enable/disable the notify member flag of bonding port\n", + }, + { + &cmd_get_member_hw_create_bonding, + "get bonding member hardware create (member_port_id) (bonding_port_id)\n" + " Get the status of member port hardware creating the bonding port\n", + }, { NULL, NULL }, }, }; -- 2.39.1 ^ permalink raw reply [flat|nested] 80+ messages in thread
* Re: [PATCH v4 0/6] Enhance the bond framework to support offload 2023-10-18 7:48 ` [PATCH v4 0/6] " Chaoyong He ` (5 preceding siblings ...) 2023-10-18 7:48 ` [PATCH v4 6/6] net/bonding: add commands for bonding port notification Chaoyong He @ 2023-11-15 16:01 ` Ferruh Yigit 2023-11-16 1:45 ` Chaoyong He 2023-12-26 2:37 ` [PATCH v5 00/14] " Chaoyong He 7 siblings, 1 reply; 80+ messages in thread From: Ferruh Yigit @ 2023-11-15 16:01 UTC (permalink / raw) To: Chaoyong He, dev; +Cc: oss-drivers On 10/18/2023 8:48 AM, Chaoyong He wrote: > This patch series try to enhance the bond framework to support the > offload feature better: > * Add new API to make the member port can access some information of the > bond port which belongs. > * Add new API to get the result of whether bond port is created by the > member port. > * Add two command line argument to control if enable member port > notification and dedicated queue features. > * Add logic to support add ports which share the same PCI address into > bond port. > * Also modify the testpmd application to test the new APIs and logics > added by this patch series. > > --- > v2: > * Fix compile error on github-robot by removing the redundancy function > declaration in the header file. > v3: > * Use the hole in the structure for the new added flag data field. > v4: > * Drop two commits not necessary for this series. > * Modify some logic as the review comments from reviewers. > --- > > Long Wu (6): > ethdev: add member notification for bonding port > ethdev: add API to get hardware creation of bonding port > net/bonding: add bonding port arguments > net/bonding: support add port by data name > net/bonding: support checking valid bonding port ID > net/bonding: add commands for bonding port notification > > Hi Chaoyong, This patchset discussed in the technical board, to be able to discuss in more details, can you please provide more details in the usecase? Also there can be a gap on how to test this new additions, again knowing the intended usecase may help. There are a few things not ideal in the set, like adding bonding specific APIs, ethdev additions specific to bonding, port not working on generic capabilities/configuration approach but app needs to know that it is bonding port etc ... I will postpone the patchset to next release, please clarify more the usecase and we can continue discussions. Thanks, ferruh ^ permalink raw reply [flat|nested] 80+ messages in thread
* RE: [PATCH v4 0/6] Enhance the bond framework to support offload 2023-11-15 16:01 ` [PATCH v4 0/6] Enhance the bond framework to support offload Ferruh Yigit @ 2023-11-16 1:45 ` Chaoyong He 0 siblings, 0 replies; 80+ messages in thread From: Chaoyong He @ 2023-11-16 1:45 UTC (permalink / raw) To: Ferruh Yigit, dev; +Cc: oss-drivers > On 10/18/2023 8:48 AM, Chaoyong He wrote: > > This patch series try to enhance the bond framework to support the > > offload feature better: > > * Add new API to make the member port can access some information of the > > bond port which belongs. > > * Add new API to get the result of whether bond port is created by the > > member port. > > * Add two command line argument to control if enable member port > > notification and dedicated queue features. > > * Add logic to support add ports which share the same PCI address into > > bond port. > > * Also modify the testpmd application to test the new APIs and logics > > added by this patch series. > > > > --- > > v2: > > * Fix compile error on github-robot by removing the redundancy function > > declaration in the header file. > > v3: > > * Use the hole in the structure for the new added flag data field. > > v4: > > * Drop two commits not necessary for this series. > > * Modify some logic as the review comments from reviewers. > > --- > > > > Long Wu (6): > > ethdev: add member notification for bonding port > > ethdev: add API to get hardware creation of bonding port > > net/bonding: add bonding port arguments > > net/bonding: support add port by data name > > net/bonding: support checking valid bonding port ID > > net/bonding: add commands for bonding port notification > > > > > > Hi Chaoyong, > > This patchset discussed in the technical board, to be able to discuss in more > details, can you please provide more details in the usecase? > > Also there can be a gap on how to test this new additions, again knowing the > intended usecase may help. > > > There are a few things not ideal in the set, like adding bonding specific APIs, > ethdev additions specific to bonding, port not working on generic > capabilities/configuration approach but app needs to know that it is bonding > port etc ... > > I will postpone the patchset to next release, please clarify more the usecase > and we can continue discussions. > > > Thanks, > Ferruh We will also send out the modifications of the NFP PMD as an usecase in the next release, and I think that will provide some details for current problems. Thanks. ^ permalink raw reply [flat|nested] 80+ messages in thread
* [PATCH v5 00/14] Enhance the bond framework to support offload 2023-10-18 7:48 ` [PATCH v4 0/6] " Chaoyong He ` (6 preceding siblings ...) 2023-11-15 16:01 ` [PATCH v4 0/6] Enhance the bond framework to support offload Ferruh Yigit @ 2023-12-26 2:37 ` Chaoyong He 2023-12-26 2:37 ` [PATCH v5 01/14] ethdev: add member notification for bonding port Chaoyong He ` (14 more replies) 7 siblings, 15 replies; 80+ messages in thread From: Chaoyong He @ 2023-12-26 2:37 UTC (permalink / raw) To: dev; +Cc: oss-drivers, Chaoyong He This patch series try to enhance the bond framework to support the offload feature better: * Add new API to make the member port can access some information of the bond port which belongs. * Add new API to get the result of whether bond port is created by the member port. * Add two command line argument to control if enable member port notification and dedicated queue features. * Add logic to support add ports which share the same PCI address into bond port. * Also modify the testpmd application to test the new APIs and logics added by this patch series. --- v2: * Fix compile error on github-robot by removing the redundancy function declaration in the header file. v3: * Use the hole in the structure for the new added flag data field. v4: * Drop two commits not necessary for this series. * Modify some logic as the review comments from reviewers. v5: * Add a new 'rte_eth_bond_flow.h' header file. * Add the patches of NFP PMD as the example of support bond flow offload. --- Long Wu (14): ethdev: add member notification for bonding port ethdev: add API to get firmware creation of bonding port net/bonding: add bonding port arguments net/bonding: support add port by data name net/bonding: support checking valid bonding port ID net/bonding: add commands for bonding port notification net/bonding: create new rte flow header file net/nfp: add bond firmware creation initialization net/nfp: reset bond configuration of firmware net/nfp: handle link event of bond firmware creation net/nfp: support bond member notification net/nfp: handle bond packets from firmware net/nfp: support getting bond firmware creation net/nfp: support offloading bond-flow .../link_bonding_poll_mode_drv_lib.rst | 18 + drivers/net/bonding/bonding_testpmd.c | 128 ++++ drivers/net/bonding/eth_bond_private.h | 24 +- drivers/net/bonding/rte_eth_bond.h | 88 +++ drivers/net/bonding/rte_eth_bond_api.c | 122 ++++ drivers/net/bonding/rte_eth_bond_args.c | 47 ++ drivers/net/bonding/rte_eth_bond_flow.c | 1 + drivers/net/bonding/rte_eth_bond_flow.h | 22 + drivers/net/bonding/rte_eth_bond_pmd.c | 94 ++- drivers/net/bonding/version.map | 5 + drivers/net/nfp/flower/nfp_flower.c | 87 ++- drivers/net/nfp/flower/nfp_flower.h | 13 + drivers/net/nfp/flower/nfp_flower_bond.c | 691 ++++++++++++++++++ drivers/net/nfp/flower/nfp_flower_bond.h | 165 +++++ drivers/net/nfp/flower/nfp_flower_cmsg.c | 35 + drivers/net/nfp/flower/nfp_flower_cmsg.h | 12 + drivers/net/nfp/flower/nfp_flower_ctrl.c | 106 ++- drivers/net/nfp/flower/nfp_flower_flow.c | 232 +++++- .../net/nfp/flower/nfp_flower_representor.c | 35 + .../net/nfp/flower/nfp_flower_representor.h | 2 + drivers/net/nfp/meson.build | 3 +- drivers/net/nfp/nfp_net_common.c | 25 + drivers/net/nfp/nfp_net_common.h | 1 - lib/ethdev/ethdev_driver.h | 38 + 24 files changed, 1951 insertions(+), 43 deletions(-) create mode 100644 drivers/net/bonding/rte_eth_bond_flow.h create mode 100644 drivers/net/nfp/flower/nfp_flower_bond.c create mode 100644 drivers/net/nfp/flower/nfp_flower_bond.h -- 2.39.1 ^ permalink raw reply [flat|nested] 80+ messages in thread
* [PATCH v5 01/14] ethdev: add member notification for bonding port 2023-12-26 2:37 ` [PATCH v5 00/14] " Chaoyong He @ 2023-12-26 2:37 ` Chaoyong He 2023-12-26 2:37 ` [PATCH v5 02/14] ethdev: add API to get firmware creation of " Chaoyong He ` (13 subsequent siblings) 14 siblings, 0 replies; 80+ messages in thread From: Chaoyong He @ 2023-12-26 2:37 UTC (permalink / raw) To: dev; +Cc: oss-drivers, Long Wu, James Hershaw, Chaoyong He, Peng Zhang From: Long Wu <long.wu@corigine.com> Bonding PMD does not let member ports know the bonding port's information, like how many member ports the bonding port has, what mode the bonding port is in and so on. Add the notification interface for bonding port to let member port know it is added to a bonding port and what the bonding port's configuration is. If so the member ports have chance to offload bond-flow that its destination port is a bonding port. Signed-off-by: Long Wu <long.wu@corigine.com> Reviewed-by: James Hershaw <james.hershaw@corigine.com> Reviewed-by: Chaoyong He <chaoyong.he@corigine.com> Reviewed-by: Peng Zhang <peng.zhang@corigine.com> --- drivers/net/bonding/eth_bond_private.h | 1 + drivers/net/bonding/rte_eth_bond.h | 46 ++++++++++++++++ drivers/net/bonding/rte_eth_bond_api.c | 72 ++++++++++++++++++++++++++ drivers/net/bonding/rte_eth_bond_pmd.c | 32 ++++++++++-- drivers/net/bonding/version.map | 3 ++ lib/ethdev/ethdev_driver.h | 18 +++++++ 6 files changed, 169 insertions(+), 3 deletions(-) diff --git a/drivers/net/bonding/eth_bond_private.h b/drivers/net/bonding/eth_bond_private.h index e688894210..f69e85c199 100644 --- a/drivers/net/bonding/eth_bond_private.h +++ b/drivers/net/bonding/eth_bond_private.h @@ -180,6 +180,7 @@ struct bond_dev_private { uint8_t member_update_idx; bool kvargs_processing_is_done; + bool notify_member; /**< Enable member notification of bonding port. */ uint32_t candidate_max_rx_pktlen; uint32_t max_rx_pktlen; diff --git a/drivers/net/bonding/rte_eth_bond.h b/drivers/net/bonding/rte_eth_bond.h index f10165f2c6..f6c773615c 100644 --- a/drivers/net/bonding/rte_eth_bond.h +++ b/drivers/net/bonding/rte_eth_bond.h @@ -351,6 +351,52 @@ rte_eth_bond_link_up_prop_delay_set(uint16_t bonding_port_id, int rte_eth_bond_link_up_prop_delay_get(uint16_t bonding_port_id); +/** + * Set the flag of whether bonding port notifies member ports. + * + * @param bonding_port_id + * Port ID of bonding device. + * @param notify + * Flag of whether bonding port notifies member ports. + * + * @return + * 0 on success, negative value otherwise. + */ +__rte_experimental +int +rte_eth_bond_notify_member_flag_set(uint16_t bonding_port_id, bool notify); + +/** + * Get the flag of whether bonding port notifies member ports. + * + * @param bonding_port_id + * Port ID of bonding device. + * @param notify + * Flag of whether bonding port notifies member ports. + * + * @return + * 0 on success, negative value otherwise. + */ +__rte_experimental +int +rte_eth_bond_notify_member_flag_get(uint16_t bonding_port_id, bool *notify); + +/** + * Notify the member ports of bonding port's information. + * + * This interface is called in the following functions: + * - bond_ethdev_lsc_event_callback() + * - bond_ethdev_configure() + * + * @param bonding_port_id + * Port ID of bonding device. + * + * @return + * 0 on success, negative value otherwise. + */ +__rte_experimental +int +rte_eth_bond_notify_members(uint16_t bonding_port_id); #ifdef __cplusplus } diff --git a/drivers/net/bonding/rte_eth_bond_api.c b/drivers/net/bonding/rte_eth_bond_api.c index 99e496556a..239f86ee92 100644 --- a/drivers/net/bonding/rte_eth_bond_api.c +++ b/drivers/net/bonding/rte_eth_bond_api.c @@ -627,6 +627,17 @@ __eth_bond_member_add_lock_free(uint16_t bonding_port_id, uint16_t member_port_i member_vlan_filter_set(bonding_port_id, member_port_id); + if (internals->notify_member && + *member_eth_dev->dev_ops->bond_notify_member != NULL) { + ret = member_eth_dev->dev_ops->bond_notify_member(member_eth_dev, + bonding_eth_dev); + if (ret < 0) { + RTE_BOND_LOG(ERR, "Add member (port %u) notify failed!", + member_port_id); + return -1; + } + } + return 0; } @@ -733,6 +744,10 @@ __eth_bond_member_remove_lock_free(uint16_t bonding_port_id, member_eth_dev = &rte_eth_devices[member_port_id]; member_remove(internals, member_eth_dev); member_eth_dev->data->dev_flags &= (~RTE_ETH_DEV_BONDING_MEMBER); + if (internals->notify_member && + *member_eth_dev->dev_ops->bond_notify_member != NULL) + member_eth_dev->dev_ops->bond_notify_member(member_eth_dev, + bonding_eth_dev); /* first member in the active list will be the primary by default, * otherwise use first device in list */ @@ -1098,3 +1113,60 @@ rte_eth_bond_link_up_prop_delay_get(uint16_t bonding_port_id) return internals->link_up_delay_ms; } + +int +rte_eth_bond_notify_member_flag_set(uint16_t bonding_port_id, bool notify) +{ + struct bond_dev_private *internals; + + if (valid_bonding_port_id(bonding_port_id) != 0) + return -EINVAL; + + internals = rte_eth_devices[bonding_port_id].data->dev_private; + + internals->notify_member = notify; + + return 0; +} + +int +rte_eth_bond_notify_member_flag_get(uint16_t bonding_port_id, bool *notify) +{ + struct bond_dev_private *internals; + + if (valid_bonding_port_id(bonding_port_id) != 0) + return -EINVAL; + + internals = rte_eth_devices[bonding_port_id].data->dev_private; + + *notify = internals->notify_member; + + return 0; +} + +int +rte_eth_bond_notify_members(uint16_t bonding_port_id) +{ + uint32_t i; + uint16_t member_port_id; + struct rte_eth_dev *bond_dev; + struct rte_eth_dev *member_dev; + struct bond_dev_private *internals; + + if (valid_bonding_port_id(bonding_port_id) != 0) + return -EINVAL; + + bond_dev = &rte_eth_devices[bonding_port_id]; + internals = bond_dev->data->dev_private; + + for (i = 0; i < internals->member_count; i++) { + member_port_id = internals->members[i].port_id; + member_dev = &rte_eth_devices[member_port_id]; + /* Notify member port if it supports. */ + if (*member_dev->dev_ops->bond_notify_member != NULL) + member_dev->dev_ops->bond_notify_member(member_dev, + bond_dev); + } + + return 0; +} diff --git a/drivers/net/bonding/rte_eth_bond_pmd.c b/drivers/net/bonding/rte_eth_bond_pmd.c index c40d18d128..d2c890075a 100644 --- a/drivers/net/bonding/rte_eth_bond_pmd.c +++ b/drivers/net/bonding/rte_eth_bond_pmd.c @@ -2982,11 +2982,13 @@ bond_ethdev_lsc_event_callback(uint16_t port_id, enum rte_eth_event_type type, int valid_member = 0; uint16_t active_pos, member_idx; uint16_t i; + uint16_t bonding_port_id; if (type != RTE_ETH_EVENT_INTR_LSC || param == NULL) return rc; - bonding_eth_dev = &rte_eth_devices[*(uint16_t *)param]; + bonding_port_id = *(uint16_t *)param; + bonding_eth_dev = &rte_eth_devices[bonding_port_id]; if (check_for_bonding_ethdev(bonding_eth_dev)) return rc; @@ -3058,8 +3060,12 @@ bond_ethdev_lsc_event_callback(uint16_t port_id, enum rte_eth_event_type type, * using it. */ if (internals->user_defined_primary_port && - internals->primary_port == port_id) + internals->primary_port == port_id) { bond_ethdev_primary_set(internals, port_id); + + if (internals->notify_member) + rte_eth_bond_notify_members(bonding_port_id); + } } else { if (active_pos == internals->active_member_count) goto link_update; @@ -3078,6 +3084,10 @@ bond_ethdev_lsc_event_callback(uint16_t port_id, enum rte_eth_event_type type, internals->active_members[0]); else internals->current_primary_port = internals->primary_port; + + if (internals->notify_member) + rte_eth_bond_notify_members(bonding_port_id); + mac_address_members_update(bonding_eth_dev); bond_ethdev_promiscuous_update(bonding_eth_dev); bond_ethdev_allmulticast_update(bonding_eth_dev); @@ -3376,6 +3386,7 @@ dump_basic(const struct rte_eth_dev *dev, FILE *f) struct bond_dev_private instant_priv; const struct bond_dev_private *internals = &instant_priv; int mode, i; + bool notify_member; /* Obtain a instance of dev_private to prevent data from being modified. */ memcpy(&instant_priv, dev->data->dev_private, sizeof(struct bond_dev_private)); @@ -3445,6 +3456,13 @@ dump_basic(const struct rte_eth_dev *dev, FILE *f) fprintf(f, "\tUser Defined Primary: [%u]\n", internals->primary_port); if (internals->member_count > 0) fprintf(f, "\tCurrent Primary: [%u]\n", internals->current_primary_port); + + if (rte_eth_bond_notify_member_flag_get(internals->port_id, ¬ify_member) == 0) + fprintf(f, "\tNotify Member Ports Flag: %s\n", + notify_member ? "enable" : "disable"); + else + fprintf(f, "\tFailed to get notify member ports flag for bonding port %d\n", + internals->port_id); } static void @@ -3997,8 +4015,12 @@ bond_ethdev_configure(struct rte_eth_dev *dev) * if no kvlist, it means that this bonding device has been created * through the bonding api. */ - if (!kvlist || internals->kvargs_processing_is_done) + if (!kvlist || internals->kvargs_processing_is_done) { + if (internals->notify_member && rte_eth_bond_notify_members(port_id) != 0) + RTE_BOND_LOG(ERR, "Notify member ports failed"); + return 0; + } internals->kvargs_processing_is_done = true; @@ -4236,6 +4258,10 @@ bond_ethdev_configure(struct rte_eth_dev *dev) return -1; } } + + if (internals->notify_member && rte_eth_bond_notify_members(port_id) != 0) + RTE_BOND_LOG(ERR, "Notify member ports failed"); + return 0; } diff --git a/drivers/net/bonding/version.map b/drivers/net/bonding/version.map index 09ee21c55f..3bd5e8ad11 100644 --- a/drivers/net/bonding/version.map +++ b/drivers/net/bonding/version.map @@ -35,4 +35,7 @@ EXPERIMENTAL { rte_eth_bond_member_add; rte_eth_bond_member_remove; rte_eth_bond_members_get; + rte_eth_bond_notify_member_flag_get; + rte_eth_bond_notify_member_flag_set; + rte_eth_bond_notify_members; }; diff --git a/lib/ethdev/ethdev_driver.h b/lib/ethdev/ethdev_driver.h index b482cd12bb..39316a7a29 100644 --- a/lib/ethdev/ethdev_driver.h +++ b/lib/ethdev/ethdev_driver.h @@ -1216,6 +1216,21 @@ typedef int (*eth_count_aggr_ports_t)(struct rte_eth_dev *dev); typedef int (*eth_map_aggr_tx_affinity_t)(struct rte_eth_dev *dev, uint16_t tx_queue_id, uint8_t affinity); +/** + * @internal + * Bonding port notifies the member ports. + * + * @param dev + * Member port (ethdev) handle. + * @param bonding_dev + * Bonding port (ethdev) handle. + * + * @return + * Negative on error, 0 on success. + */ +typedef int (*eth_bond_notify_member)(struct rte_eth_dev *dev, + struct rte_eth_dev *bonding_dev); + /** * @internal A structure containing the functions exported by an Ethernet driver. */ @@ -1455,6 +1470,9 @@ struct eth_dev_ops { eth_count_aggr_ports_t count_aggr_ports; /** Map a Tx queue with an aggregated port of the DPDK port */ eth_map_aggr_tx_affinity_t map_aggr_tx_affinity; + + /** Notify the member port of bonding port information */ + eth_bond_notify_member bond_notify_member; }; /** -- 2.39.1 ^ permalink raw reply [flat|nested] 80+ messages in thread
* [PATCH v5 02/14] ethdev: add API to get firmware creation of bonding port 2023-12-26 2:37 ` [PATCH v5 00/14] " Chaoyong He 2023-12-26 2:37 ` [PATCH v5 01/14] ethdev: add member notification for bonding port Chaoyong He @ 2023-12-26 2:37 ` Chaoyong He 2023-12-26 2:37 ` [PATCH v5 03/14] net/bonding: add bonding port arguments Chaoyong He ` (12 subsequent siblings) 14 siblings, 0 replies; 80+ messages in thread From: Chaoyong He @ 2023-12-26 2:37 UTC (permalink / raw) To: dev; +Cc: oss-drivers, Long Wu, Chaoyong He, James Hershaw, Peng Zhang From: Long Wu <long.wu@corigine.com> After bonding port notification, member port firmware may create the bonding port. We want to get the result of creatition, so we add this API to do the getting action. Signed-off-by: Long Wu <long.wu@corigine.com> Reviewed-by: Chaoyong He <chaoyong.he@corigine.com> Reviewed-by: James Hershaw <james.hershaw@corigine.com> Reviewed-by: Peng Zhang <peng.zhang@corigine.com> --- drivers/net/bonding/rte_eth_bond.h | 15 ++++++++++++++ drivers/net/bonding/rte_eth_bond_api.c | 28 ++++++++++++++++++++++++++ drivers/net/bonding/version.map | 1 + lib/ethdev/ethdev_driver.h | 20 ++++++++++++++++++ 4 files changed, 64 insertions(+) diff --git a/drivers/net/bonding/rte_eth_bond.h b/drivers/net/bonding/rte_eth_bond.h index f6c773615c..9a8bed0346 100644 --- a/drivers/net/bonding/rte_eth_bond.h +++ b/drivers/net/bonding/rte_eth_bond.h @@ -398,6 +398,21 @@ __rte_experimental int rte_eth_bond_notify_members(uint16_t bonding_port_id); +/** + * Get the status of specified bonding port created by member port firmware. + * + * @param bonding_port_id + * Port ID of bonding device. + * @param member_port_id + * Port ID of member device. + * + * @return + * 0 on success, negative value otherwise. + */ +__rte_experimental +int +rte_eth_bond_fw_create_get(uint16_t bonding_port_id, uint16_t member_port_id); + #ifdef __cplusplus } #endif diff --git a/drivers/net/bonding/rte_eth_bond_api.c b/drivers/net/bonding/rte_eth_bond_api.c index 239f86ee92..83957830a9 100644 --- a/drivers/net/bonding/rte_eth_bond_api.c +++ b/drivers/net/bonding/rte_eth_bond_api.c @@ -1170,3 +1170,31 @@ rte_eth_bond_notify_members(uint16_t bonding_port_id) return 0; } + +int +rte_eth_bond_fw_create_get(uint16_t bonding_port_id, uint16_t member_port_id) +{ + uint32_t i; + struct rte_eth_dev *bonding_dev; + struct rte_eth_dev *member_dev; + struct bond_dev_private *internals; + + if (valid_bonding_port_id(bonding_port_id) != 0) + return -EINVAL; + + bonding_dev = &rte_eth_devices[bonding_port_id]; + internals = bonding_dev->data->dev_private; + for (i = 0; i < internals->member_count; i++) { + if (internals->members[i].port_id == member_port_id) + break; + } + + if (i == internals->member_count) + return -EINVAL; + + member_dev = &rte_eth_devices[member_port_id]; + if (*member_dev->dev_ops->bond_fw_create_get == NULL) + return -ENOTSUP; + + return member_dev->dev_ops->bond_fw_create_get(member_dev, bonding_dev); +} diff --git a/drivers/net/bonding/version.map b/drivers/net/bonding/version.map index 3bd5e8ad11..12589f61b0 100644 --- a/drivers/net/bonding/version.map +++ b/drivers/net/bonding/version.map @@ -32,6 +32,7 @@ EXPERIMENTAL { global: rte_eth_bond_8023ad_member_info; rte_eth_bond_active_members_get; + rte_eth_bond_fw_create_get; rte_eth_bond_member_add; rte_eth_bond_member_remove; rte_eth_bond_members_get; diff --git a/lib/ethdev/ethdev_driver.h b/lib/ethdev/ethdev_driver.h index 39316a7a29..09d85ad101 100644 --- a/lib/ethdev/ethdev_driver.h +++ b/lib/ethdev/ethdev_driver.h @@ -1231,6 +1231,21 @@ typedef int (*eth_map_aggr_tx_affinity_t)(struct rte_eth_dev *dev, uint16_t tx_q typedef int (*eth_bond_notify_member)(struct rte_eth_dev *dev, struct rte_eth_dev *bonding_dev); +/** + * @internal + * Get the status of specified bonding port created by member port firmware. + * + * @param dev + * Member port (ethdev) handle. + * @param bonding_dev + * Bonding port (ethdev) handle. + * + * @return + * Negative on error, 0 on success. + */ +typedef int (*eth_bond_fw_create_get)(struct rte_eth_dev *dev, + struct rte_eth_dev *bonding_dev); + /** * @internal A structure containing the functions exported by an Ethernet driver. */ @@ -1473,6 +1488,11 @@ struct eth_dev_ops { /** Notify the member port of bonding port information */ eth_bond_notify_member bond_notify_member; + /** + * Get the status of whether bonding port is successfully created by + * the member port firmware. + */ + eth_bond_fw_create_get bond_fw_create_get; }; /** -- 2.39.1 ^ permalink raw reply [flat|nested] 80+ messages in thread
* [PATCH v5 03/14] net/bonding: add bonding port arguments 2023-12-26 2:37 ` [PATCH v5 00/14] " Chaoyong He 2023-12-26 2:37 ` [PATCH v5 01/14] ethdev: add member notification for bonding port Chaoyong He 2023-12-26 2:37 ` [PATCH v5 02/14] ethdev: add API to get firmware creation of " Chaoyong He @ 2023-12-26 2:37 ` Chaoyong He 2023-12-26 2:37 ` [PATCH v5 04/14] net/bonding: support add port by data name Chaoyong He ` (11 subsequent siblings) 14 siblings, 0 replies; 80+ messages in thread From: Chaoyong He @ 2023-12-26 2:37 UTC (permalink / raw) To: dev; +Cc: oss-drivers, Long Wu, Chaoyong He, James Hershaw, Peng Zhang From: Long Wu <long.wu@corigine.com> Include the following new arguments for bonding ports: - "notify_member" to enable/disable member notification. - "dedicated_queue" to enable/disable dedicated queue. Add these two arguments in initial argument. Signed-off-by: Long Wu <long.wu@corigine.com> Reviewed-by: Chaoyong He <chaoyong.he@corigine.com> Reviewed-by: James Hershaw <james.hershaw@corigine.com> Reviewed-by: Peng Zhang <peng.zhang@corigine.com> --- drivers/net/bonding/eth_bond_private.h | 10 ++++ drivers/net/bonding/rte_eth_bond.h | 14 ++++++ drivers/net/bonding/rte_eth_bond_api.c | 14 ++++++ drivers/net/bonding/rte_eth_bond_args.c | 44 ++++++++++++++++++ drivers/net/bonding/rte_eth_bond_pmd.c | 61 ++++++++++++++++++++++++- 5 files changed, 142 insertions(+), 1 deletion(-) diff --git a/drivers/net/bonding/eth_bond_private.h b/drivers/net/bonding/eth_bond_private.h index f69e85c199..f9603a0f6b 100644 --- a/drivers/net/bonding/eth_bond_private.h +++ b/drivers/net/bonding/eth_bond_private.h @@ -28,6 +28,8 @@ #define PMD_BOND_LSC_POLL_PERIOD_KVARG ("lsc_poll_period_ms") #define PMD_BOND_LINK_UP_PROP_DELAY_KVARG ("up_delay") #define PMD_BOND_LINK_DOWN_PROP_DELAY_KVARG ("down_delay") +#define PMD_BOND_NOTIFY_MEMBER_KVARG ("notify_member") +#define PMD_BOND_DEDICATED_QUEUE_KVARG ("dedicated_queue") #define PMD_BOND_XMIT_POLICY_LAYER2_KVARG ("l2") #define PMD_BOND_XMIT_POLICY_LAYER23_KVARG ("l23") @@ -319,6 +321,14 @@ int bond_ethdev_parse_time_ms_kvarg(const char *key, const char *value, void *extra_args); +int +bond_ethdev_parse_notify_member_kvarg(const char *key __rte_unused, + const char *value, void *extra_args); + +int +bond_ethdev_parse_dedicated_queue_kvarg(const char *key __rte_unused, + const char *value, void *extra_args); + void bond_tlb_disable(struct bond_dev_private *internals); diff --git a/drivers/net/bonding/rte_eth_bond.h b/drivers/net/bonding/rte_eth_bond.h index 9a8bed0346..84dd39b12f 100644 --- a/drivers/net/bonding/rte_eth_bond.h +++ b/drivers/net/bonding/rte_eth_bond.h @@ -351,6 +351,20 @@ rte_eth_bond_link_up_prop_delay_set(uint16_t bonding_port_id, int rte_eth_bond_link_up_prop_delay_get(uint16_t bonding_port_id); +/** + * Set the flag that whether bonding device enable dedicated queue. + * + * @param bonding_port_id + * Port ID of bonding device. + * @param queue_flag + * The flag of enable bond dedicated queue + * + * @return + * 0 on success, negative value otherwise. + */ +int +rte_eth_bond_dedicated_queue_flag_set(uint16_t bonding_port_id, bool queue_flag); + /** * Set the flag of whether bonding port notifies member ports. * diff --git a/drivers/net/bonding/rte_eth_bond_api.c b/drivers/net/bonding/rte_eth_bond_api.c index 83957830a9..32fa761baf 100644 --- a/drivers/net/bonding/rte_eth_bond_api.c +++ b/drivers/net/bonding/rte_eth_bond_api.c @@ -1114,6 +1114,20 @@ rte_eth_bond_link_up_prop_delay_get(uint16_t bonding_port_id) return internals->link_up_delay_ms; } +int +rte_eth_bond_dedicated_queue_flag_set(uint16_t bonding_port_id, bool queue_flag) +{ + struct bond_dev_private *internals; + + if (valid_bonding_port_id(bonding_port_id) != 0) + return -1; + + internals = rte_eth_devices[bonding_port_id].data->dev_private; + internals->mode4.dedicated_queues.enabled = queue_flag; + + return 0; +} + int rte_eth_bond_notify_member_flag_set(uint16_t bonding_port_id, bool notify) { diff --git a/drivers/net/bonding/rte_eth_bond_args.c b/drivers/net/bonding/rte_eth_bond_args.c index bdec5d61d4..8a3e4656ef 100644 --- a/drivers/net/bonding/rte_eth_bond_args.c +++ b/drivers/net/bonding/rte_eth_bond_args.c @@ -20,6 +20,8 @@ const char *pmd_bond_init_valid_arguments[] = { PMD_BOND_MAC_ADDR_KVARG, PMD_BOND_AGG_MODE_KVARG, RTE_DEVARGS_KEY_DRIVER, + PMD_BOND_NOTIFY_MEMBER_KVARG, + PMD_BOND_DEDICATED_QUEUE_KVARG, NULL }; @@ -297,3 +299,45 @@ bond_ethdev_parse_time_ms_kvarg(const char *key __rte_unused, return 0; } + +int +bond_ethdev_parse_notify_member_kvarg(const char *key __rte_unused, + const char *value, void *extra_args) +{ + bool *notify_member; + + if (value == NULL || extra_args == NULL) + return -1; + + notify_member = extra_args; + + if (strcmp("enable", value) == 0) + *notify_member = true; + else if (strcmp("disable", value) == 0) + *notify_member = false; + else + return -1; + + return 0; +} + +int +bond_ethdev_parse_dedicated_queue_kvarg(const char *key __rte_unused, + const char *value, void *extra_args) +{ + bool *dedicated_queue; + + if (value == NULL || extra_args == NULL) + return -1; + + dedicated_queue = extra_args; + + if (strcmp("enable", value) == 0) + *dedicated_queue = true; + else if (strcmp("disable", value) == 0) + *dedicated_queue = false; + else + return -1; + + return 0; +} diff --git a/drivers/net/bonding/rte_eth_bond_pmd.c b/drivers/net/bonding/rte_eth_bond_pmd.c index d2c890075a..b3a37a36c8 100644 --- a/drivers/net/bonding/rte_eth_bond_pmd.c +++ b/drivers/net/bonding/rte_eth_bond_pmd.c @@ -4246,6 +4246,63 @@ bond_ethdev_configure(struct rte_eth_dev *dev) return -1; } + /* Parse/set notify member flag */ + arg_count = rte_kvargs_count(kvlist, PMD_BOND_NOTIFY_MEMBER_KVARG); + if (arg_count == 1) { + bool notify_member; + if (rte_kvargs_process(kvlist, + PMD_BOND_NOTIFY_MEMBER_KVARG, + &bond_ethdev_parse_notify_member_kvarg, + ¬ify_member) < 0) { + RTE_BOND_LOG(ERR, + "Invalid notify member value specified" + " for bonding device %s", name); + return -1; + } + + if (rte_eth_bond_notify_member_flag_set(port_id, notify_member) != 0) { + RTE_BOND_LOG(ERR, + "Failed to set notify member (%u) on" + " bonding device %s", notify_member, name); + return -1; + } + } else if (arg_count > 1) { + RTE_BOND_LOG(ERR, + "notify member flag can be specified only once" + " for bonding device %s", name); + return -1; + } + + /* Parse/set dedicated queue flag */ + arg_count = rte_kvargs_count(kvlist, PMD_BOND_DEDICATED_QUEUE_KVARG); + if (arg_count == 1) { + bool dedicated_queue; + if (rte_kvargs_process(kvlist, + PMD_BOND_DEDICATED_QUEUE_KVARG, + &bond_ethdev_parse_dedicated_queue_kvarg, + &dedicated_queue) < 0) { + RTE_BOND_LOG(ERR, + "Invalid dedicated queue flag specified" + " for bonding device %s", name); + return -1; + } + + if (internals->mode == BONDING_MODE_8023AD) { + if (rte_eth_bond_dedicated_queue_flag_set(port_id, dedicated_queue) != 0) { + RTE_BOND_LOG(ERR, + "Failed to enable/disable dedicated" + " queue flag on bonding device %s", + name); + return -1; + } + } + } else if (arg_count > 1) { + RTE_BOND_LOG(ERR, + "dedicated queue flag can be specified only once" + " for bonding device %s", name); + return -1; + } + /* configure members so we can pass mtu setting */ for (i = 0; i < internals->member_count; i++) { struct rte_eth_dev *member_ethdev = @@ -4283,7 +4340,9 @@ RTE_PMD_REGISTER_PARAM_STRING(net_bonding, "mac=<mac addr> " "lsc_poll_period_ms=<int> " "up_delay=<int> " - "down_delay=<int>"); + "down_delay=<int>" + "notify_member=[enable | disable] " + "dedicated_queue=[enable | disable] "); /* We can't use RTE_LOG_REGISTER_DEFAULT because of the forced name for * this library, see meson.build. -- 2.39.1 ^ permalink raw reply [flat|nested] 80+ messages in thread
* [PATCH v5 04/14] net/bonding: support add port by data name 2023-12-26 2:37 ` [PATCH v5 00/14] " Chaoyong He ` (2 preceding siblings ...) 2023-12-26 2:37 ` [PATCH v5 03/14] net/bonding: add bonding port arguments Chaoyong He @ 2023-12-26 2:37 ` Chaoyong He 2023-12-26 2:37 ` [PATCH v5 05/14] net/bonding: support checking valid bonding port ID Chaoyong He ` (10 subsequent siblings) 14 siblings, 0 replies; 80+ messages in thread From: Chaoyong He @ 2023-12-26 2:37 UTC (permalink / raw) To: dev; +Cc: oss-drivers, Long Wu, Chaoyong He, James Hershaw, Peng Zhang From: Long Wu <long.wu@corigine.com> Several ports may share the same PCI address, like NFP representor port. So users cannot add this type of ports to bonding port by "--vdev" argument in dpdk-testpmd. But the port's data name is unique between them, include an option to add such ports to the bonding port. After adding this feature, users can create a bonding port that member port is this type of port by "--vdev" in dpdk-testpmd start command. For example: dpdk-testpmd -l 2-10 -s 0x8 -a ca:00.0,representor=[0-2] --vdev 'net_bonding0,member=flower_repr_p0,member=flower_repr_p1, mode=4,socket_id=1,xmit_policy=l34' -- -i Note: 1. "ca:00.0" is NFP 4000 card. 2. "flower_repr_p0" and "flower_repr_p1" are NFP phy representor port's data name. Signed-off-by: Long Wu <long.wu@corigine.com> Reviewed-by: Chaoyong He <chaoyong.he@corigine.com> Reviewed-by: James Hershaw <james.hershaw@corigine.com> Reviewed-by: Peng Zhang <peng.zhang@corigine.com> --- drivers/net/bonding/rte_eth_bond_args.c | 3 +++ 1 file changed, 3 insertions(+) diff --git a/drivers/net/bonding/rte_eth_bond_args.c b/drivers/net/bonding/rte_eth_bond_args.c index 8a3e4656ef..b320eb3038 100644 --- a/drivers/net/bonding/rte_eth_bond_args.c +++ b/drivers/net/bonding/rte_eth_bond_args.c @@ -70,6 +70,9 @@ find_port_id_by_dev_name(const char *name) if (strcmp(rte_eth_devices[i].device->name, name) == 0) return i; + + if (strcmp(rte_eth_devices[i].data->name, name) == 0) + return i; } return -1; } -- 2.39.1 ^ permalink raw reply [flat|nested] 80+ messages in thread
* [PATCH v5 05/14] net/bonding: support checking valid bonding port ID 2023-12-26 2:37 ` [PATCH v5 00/14] " Chaoyong He ` (3 preceding siblings ...) 2023-12-26 2:37 ` [PATCH v5 04/14] net/bonding: support add port by data name Chaoyong He @ 2023-12-26 2:37 ` Chaoyong He 2023-12-26 2:37 ` [PATCH v5 06/14] net/bonding: add commands for bonding port notification Chaoyong He ` (9 subsequent siblings) 14 siblings, 0 replies; 80+ messages in thread From: Chaoyong He @ 2023-12-26 2:37 UTC (permalink / raw) To: dev; +Cc: oss-drivers, Long Wu, Chaoyong He, James Hershaw, Peng Zhang From: Long Wu <long.wu@corigine.com> Add API to support checking if the port id is a bonding port id. Signed-off-by: Long Wu <long.wu@corigine.com> Reviewed-by: Chaoyong He <chaoyong.he@corigine.com> Reviewed-by: James Hershaw <james.hershaw@corigine.com> Reviewed-by: Peng Zhang <peng.zhang@corigine.com> --- drivers/net/bonding/rte_eth_bond.h | 13 +++++++++++++ drivers/net/bonding/rte_eth_bond_api.c | 7 +++++++ drivers/net/bonding/version.map | 1 + 3 files changed, 21 insertions(+) diff --git a/drivers/net/bonding/rte_eth_bond.h b/drivers/net/bonding/rte_eth_bond.h index 84dd39b12f..62af9bbd99 100644 --- a/drivers/net/bonding/rte_eth_bond.h +++ b/drivers/net/bonding/rte_eth_bond.h @@ -427,6 +427,19 @@ __rte_experimental int rte_eth_bond_fw_create_get(uint16_t bonding_port_id, uint16_t member_port_id); +/** + * Check whether bonding port id is valid. + * + * @param port_id + * Port ID of bonding device. + * + * @return + * true means the port is a bonding device, false means not. + */ +__rte_experimental +bool +rte_eth_bond_is_valid_port(uint16_t port_id); + #ifdef __cplusplus } #endif diff --git a/drivers/net/bonding/rte_eth_bond_api.c b/drivers/net/bonding/rte_eth_bond_api.c index 32fa761baf..5e9437c786 100644 --- a/drivers/net/bonding/rte_eth_bond_api.c +++ b/drivers/net/bonding/rte_eth_bond_api.c @@ -1212,3 +1212,10 @@ rte_eth_bond_fw_create_get(uint16_t bonding_port_id, uint16_t member_port_id) return member_dev->dev_ops->bond_fw_create_get(member_dev, bonding_dev); } + + +bool +rte_eth_bond_is_valid_port(uint16_t port_id) +{ + return (valid_bonding_port_id(port_id) == 0); +} diff --git a/drivers/net/bonding/version.map b/drivers/net/bonding/version.map index 12589f61b0..cd27c43092 100644 --- a/drivers/net/bonding/version.map +++ b/drivers/net/bonding/version.map @@ -33,6 +33,7 @@ EXPERIMENTAL { rte_eth_bond_8023ad_member_info; rte_eth_bond_active_members_get; rte_eth_bond_fw_create_get; + rte_eth_bond_is_valid_port; rte_eth_bond_member_add; rte_eth_bond_member_remove; rte_eth_bond_members_get; -- 2.39.1 ^ permalink raw reply [flat|nested] 80+ messages in thread
* [PATCH v5 06/14] net/bonding: add commands for bonding port notification 2023-12-26 2:37 ` [PATCH v5 00/14] " Chaoyong He ` (4 preceding siblings ...) 2023-12-26 2:37 ` [PATCH v5 05/14] net/bonding: support checking valid bonding port ID Chaoyong He @ 2023-12-26 2:37 ` Chaoyong He 2023-12-26 2:37 ` [PATCH v5 07/14] net/bonding: create new rte flow header file Chaoyong He ` (8 subsequent siblings) 14 siblings, 0 replies; 80+ messages in thread From: Chaoyong He @ 2023-12-26 2:37 UTC (permalink / raw) To: dev; +Cc: oss-drivers, Long Wu, Chaoyong He, James Hershaw, Peng Zhang From: Long Wu <long.wu@corigine.com> Add some commands to support bonding port notification in dpdk-testpmd. 1. Users can enable the notification by command: "set bonding notify_member (port_id) (enable|disable)" 2. If member port firmware try to create the bonding port after notification users can get the status by command: "get bonding member firmware create (member_port_id) (bonding_port_id)" Signed-off-by: Long Wu <long.wu@corigine.com> Reviewed-by: Chaoyong He <chaoyong.he@corigine.com> Reviewed-by: James Hershaw <james.hershaw@corigine.com> Reviewed-by: Peng Zhang <peng.zhang@corigine.com> --- .../link_bonding_poll_mode_drv_lib.rst | 18 +++ drivers/net/bonding/bonding_testpmd.c | 128 ++++++++++++++++++ 2 files changed, 146 insertions(+) diff --git a/doc/guides/prog_guide/link_bonding_poll_mode_drv_lib.rst b/doc/guides/prog_guide/link_bonding_poll_mode_drv_lib.rst index 60717a3587..653d2f850d 100644 --- a/doc/guides/prog_guide/link_bonding_poll_mode_drv_lib.rst +++ b/doc/guides/prog_guide/link_bonding_poll_mode_drv_lib.rst @@ -637,3 +637,21 @@ in balance mode with a transmission policy of layer 2+3:: Members (3): [1 3 4] Active Members (3): [1 3 4] Primary: [3] + +set bonding notify_member +~~~~~~~~~~~~~~~~~~~~~~~~~ + +Set the notify member flag of bonding port:: + + testpmd> set bonding notify_member (port_id) (enable|disable) + +This command just set the flag of notification. +If we enable it, bonding PMD will notify member ports when its some +configurations changed. + +get bonding member firmware create +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +Get the status of member port hardware creating the bonding port:: + + testpmd> get bonding member firmware create (member_port_id) (bonding_port_id) diff --git a/drivers/net/bonding/bonding_testpmd.c b/drivers/net/bonding/bonding_testpmd.c index 8fcd6cadd0..48f71da3a7 100644 --- a/drivers/net/bonding/bonding_testpmd.c +++ b/drivers/net/bonding/bonding_testpmd.c @@ -692,6 +692,124 @@ static cmdline_parse_inst_t cmd_set_bonding_agg_mode_policy = { } }; +struct cmd_set_bonding_notify_member_result { + cmdline_fixed_string_t set; + cmdline_fixed_string_t bonding; + cmdline_fixed_string_t notify_member; + uint16_t port_num; + cmdline_fixed_string_t mode; +}; + +static void +cmd_set_bonding_notify_member_parsed(void *parsed_result, + __rte_unused struct cmdline *cl, __rte_unused void *data) +{ + struct cmd_set_bonding_notify_member_result *res = parsed_result; + bool notify_member = false; + + if (strcmp(res->notify_member, "enable") == 0) + notify_member = true; + else if (strcmp(res->notify_member, "disable") == 0) + notify_member = false; + + rte_eth_bond_notify_member_flag_set(res->port_num, notify_member); +} + +static cmdline_parse_token_string_t cmd_set_bonding_notify_member_set = + TOKEN_STRING_INITIALIZER(struct cmd_set_bonding_notify_member_result, + set, "set"); +static cmdline_parse_token_string_t cmd_set_bonding_notify_member_bonding = + TOKEN_STRING_INITIALIZER(struct cmd_set_bonding_notify_member_result, + bonding, "bonding"); +static cmdline_parse_token_string_t cmd_set_bonding_notify_member = + TOKEN_STRING_INITIALIZER(struct cmd_set_bonding_notify_member_result, + notify_member, "notify_member"); +static cmdline_parse_token_num_t cmd_set_bonding_notify_member_portnum = + TOKEN_NUM_INITIALIZER(struct cmd_set_bonding_notify_member_result, + port_num, RTE_UINT16); +static cmdline_parse_token_string_t cmd_set_bonding_notify_member_mode_string = + TOKEN_STRING_INITIALIZER(struct cmd_set_bonding_notify_member_result, + mode, "enable#disable"); + +static cmdline_parse_inst_t cmd_set_bonding_notify_member_ports = { + .f = cmd_set_bonding_notify_member_parsed, + .data = NULL, + .help_str = "set bonding notify_member (port_id) (enable|disable)", + .tokens = { + (void *)&cmd_set_bonding_notify_member_set, + (void *)&cmd_set_bonding_notify_member_bonding, + (void *)&cmd_set_bonding_notify_member, + (void *)&cmd_set_bonding_notify_member_portnum, + (void *)&cmd_set_bonding_notify_member_mode_string, + NULL + } +}; + +struct cmd_get_bonding_member_hw_create_result { + cmdline_fixed_string_t get; + cmdline_fixed_string_t bonding; + cmdline_fixed_string_t member; + cmdline_fixed_string_t firmware; + cmdline_fixed_string_t create; + uint16_t member_port_id; + uint16_t bonding_port_id; +}; + +static void +cmd_get_bonding_member_hw_create_parsed(void *parsed_result, + __rte_unused struct cmdline *cl, __rte_unused void *data) +{ + struct cmd_get_bonding_member_hw_create_result *res = parsed_result; + int ret; + + ret = rte_eth_bond_fw_create_get(res->bonding_port_id, res->member_port_id); + if (ret == 0) + printf("Member port %u firmware creates bonding port %u successfully\n", + res->member_port_id, res->bonding_port_id); + else + printf("Failed to get status of member port %u firmware creating" + " bonding port %u, %d\n", + res->member_port_id, res->bonding_port_id, ret); +} + +static cmdline_parse_token_string_t cmd_get_bonding_member_hw_create_get = + TOKEN_STRING_INITIALIZER(struct cmd_get_bonding_member_hw_create_result, + get, "get"); +static cmdline_parse_token_string_t cmd_get_bonding_member_hw_create_bonding = + TOKEN_STRING_INITIALIZER(struct cmd_get_bonding_member_hw_create_result, + bonding, "bonding"); +static cmdline_parse_token_string_t cmd_get_bonding_member_hw_create_member = + TOKEN_STRING_INITIALIZER(struct cmd_get_bonding_member_hw_create_result, + member, "member"); +static cmdline_parse_token_string_t cmd_get_bonding_member_hw_create_firmware = + TOKEN_STRING_INITIALIZER(struct cmd_get_bonding_member_hw_create_result, + create, "firmware"); +static cmdline_parse_token_string_t cmd_get_bonding_member_hw_create_create = + TOKEN_STRING_INITIALIZER(struct cmd_get_bonding_member_hw_create_result, + create, "create"); +static cmdline_parse_token_num_t cmd_get_bonding_member_hw_create_memberportid = + TOKEN_NUM_INITIALIZER(struct cmd_get_bonding_member_hw_create_result, + member_port_id, RTE_UINT16); +static cmdline_parse_token_num_t cmd_get_bonding_member_hw_create_bondingportid = + TOKEN_NUM_INITIALIZER(struct cmd_get_bonding_member_hw_create_result, + bonding_port_id, RTE_UINT16); + +static cmdline_parse_inst_t cmd_get_member_hw_create_bonding = { + .f = cmd_get_bonding_member_hw_create_parsed, + .data = NULL, + .help_str = "get bonding member firmware create (member_port_id) (bonding_port_id)", + .tokens = { + (void *)&cmd_get_bonding_member_hw_create_get, + (void *)&cmd_get_bonding_member_hw_create_bonding, + (void *)&cmd_get_bonding_member_hw_create_member, + (void *)&cmd_get_bonding_member_hw_create_firmware, + (void *)&cmd_get_bonding_member_hw_create_create, + (void *)&cmd_get_bonding_member_hw_create_memberportid, + (void *)&cmd_get_bonding_member_hw_create_bondingportid, + NULL + } +}; + static struct testpmd_driver_commands bonding_cmds = { .commands = { { @@ -749,6 +867,16 @@ static struct testpmd_driver_commands bonding_cmds = { "set bonding mode IEEE802.3AD aggregator policy (port_id) (agg_name)\n" " Set Aggregation mode for IEEE802.3AD (mode 4)\n", }, + { + &cmd_set_bonding_notify_member_ports, + "set bonding notify_member (port_id) (enable|disable)\n" + " Enable/disable the notify member flag of bonding port\n", + }, + { + &cmd_get_member_hw_create_bonding, + "get bonding member firmware create (member_port_id) (bonding_port_id)\n" + " Get the status of member port firmware creating the bonding port\n", + }, { NULL, NULL }, }, }; -- 2.39.1 ^ permalink raw reply [flat|nested] 80+ messages in thread
* [PATCH v5 07/14] net/bonding: create new rte flow header file 2023-12-26 2:37 ` [PATCH v5 00/14] " Chaoyong He ` (5 preceding siblings ...) 2023-12-26 2:37 ` [PATCH v5 06/14] net/bonding: add commands for bonding port notification Chaoyong He @ 2023-12-26 2:37 ` Chaoyong He 2023-12-26 2:37 ` [PATCH v5 08/14] net/nfp: add bond firmware creation initialization Chaoyong He ` (7 subsequent siblings) 14 siblings, 0 replies; 80+ messages in thread From: Chaoyong He @ 2023-12-26 2:37 UTC (permalink / raw) To: dev; +Cc: oss-drivers, Long Wu, James Hershaw, Chaoyong He, Peng Zhang From: Long Wu <long.wu@corigine.com> Move the flow code to a new head file to make flow related code more clean and make the code architecture more reasonable in the future. There is no functional change, just moving verbatim code around. Signed-off-by: Long Wu <long.wu@corigine.com> Reviewed-by: James Hershaw <james.hershaw@corigine.com> Reviewed-by: Chaoyong He <chaoyong.he@corigine.com> Reviewed-by: Peng Zhang <peng.zhang@corigine.com> --- drivers/net/bonding/eth_bond_private.h | 13 ------------- drivers/net/bonding/rte_eth_bond_api.c | 1 + drivers/net/bonding/rte_eth_bond_flow.c | 1 + drivers/net/bonding/rte_eth_bond_flow.h | 22 ++++++++++++++++++++++ drivers/net/bonding/rte_eth_bond_pmd.c | 1 + 5 files changed, 25 insertions(+), 13 deletions(-) create mode 100644 drivers/net/bonding/rte_eth_bond_flow.h diff --git a/drivers/net/bonding/eth_bond_private.h b/drivers/net/bonding/eth_bond_private.h index f9603a0f6b..4373465d8d 100644 --- a/drivers/net/bonding/eth_bond_private.h +++ b/drivers/net/bonding/eth_bond_private.h @@ -9,10 +9,8 @@ #include <sys/queue.h> #include <ethdev_driver.h> -#include <rte_flow.h> #include <rte_spinlock.h> #include <rte_bitmap.h> -#include <rte_flow_driver.h> #include "rte_eth_bond.h" #include "eth_bond_8023ad_private.h" @@ -47,8 +45,6 @@ extern const char *pmd_bond_init_valid_arguments[]; extern struct rte_vdev_driver pmd_bond_drv; -extern const struct rte_flow_ops bond_flow_ops; - /** Port Queue Mapping Structure */ struct bond_rx_queue { uint16_t queue_id; @@ -94,15 +90,6 @@ struct bond_member_details { uint16_t reta_size; }; -struct rte_flow { - TAILQ_ENTRY(rte_flow) next; - /* Members flows */ - struct rte_flow *flows[RTE_MAX_ETHPORTS]; - /* Flow description for synchronization */ - struct rte_flow_conv_rule rule; - uint8_t rule_data[]; -}; - typedef void (*burst_xmit_hash_t)(struct rte_mbuf **buf, uint16_t nb_pkts, uint16_t member_count, uint16_t *members); diff --git a/drivers/net/bonding/rte_eth_bond_api.c b/drivers/net/bonding/rte_eth_bond_api.c index 5e9437c786..0a296d3894 100644 --- a/drivers/net/bonding/rte_eth_bond_api.c +++ b/drivers/net/bonding/rte_eth_bond_api.c @@ -12,6 +12,7 @@ #include <rte_kvargs.h> #include "rte_eth_bond.h" +#include "rte_eth_bond_flow.h" #include "eth_bond_private.h" #include "eth_bond_8023ad_private.h" diff --git a/drivers/net/bonding/rte_eth_bond_flow.c b/drivers/net/bonding/rte_eth_bond_flow.c index 71a91675f7..e6c7ce5362 100644 --- a/drivers/net/bonding/rte_eth_bond_flow.c +++ b/drivers/net/bonding/rte_eth_bond_flow.c @@ -1,6 +1,7 @@ /* SPDX-License-Identifier: BSD-3-Clause * Copyright 2018 Mellanox Technologies, Ltd */ +#include "rte_eth_bond_flow.h" #include <stddef.h> #include <string.h> diff --git a/drivers/net/bonding/rte_eth_bond_flow.h b/drivers/net/bonding/rte_eth_bond_flow.h new file mode 100644 index 0000000000..7394e0e2e1 --- /dev/null +++ b/drivers/net/bonding/rte_eth_bond_flow.h @@ -0,0 +1,22 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright (c) 2023 Corigine, Inc. + */ + +#ifndef _RTE_ETH_BOND_FLOW_H_ +#define _RTE_ETH_BOND_FLOW_H_ + +#include <rte_flow.h> +#include <rte_flow_driver.h> + +extern const struct rte_flow_ops bond_flow_ops; + +struct rte_flow { + TAILQ_ENTRY(rte_flow) next; + struct rte_flow *flows[RTE_MAX_ETHPORTS]; + /**< Member ports flows */ + struct rte_flow_conv_rule rule; + /**< Flow description for synchronization */ + uint8_t rule_data[]; +}; + +#endif /* _RTE_ETH_BOND_FLOW_H_ */ diff --git a/drivers/net/bonding/rte_eth_bond_pmd.c b/drivers/net/bonding/rte_eth_bond_pmd.c index b3a37a36c8..650f8061a2 100644 --- a/drivers/net/bonding/rte_eth_bond_pmd.c +++ b/drivers/net/bonding/rte_eth_bond_pmd.c @@ -21,6 +21,7 @@ #include <rte_string_fns.h> #include "rte_eth_bond.h" +#include "rte_eth_bond_flow.h" #include "eth_bond_private.h" #include "eth_bond_8023ad_private.h" -- 2.39.1 ^ permalink raw reply [flat|nested] 80+ messages in thread
* [PATCH v5 08/14] net/nfp: add bond firmware creation initialization 2023-12-26 2:37 ` [PATCH v5 00/14] " Chaoyong He ` (6 preceding siblings ...) 2023-12-26 2:37 ` [PATCH v5 07/14] net/bonding: create new rte flow header file Chaoyong He @ 2023-12-26 2:37 ` Chaoyong He 2023-12-26 2:37 ` [PATCH v5 09/14] net/nfp: reset bond configuration of firmware Chaoyong He ` (6 subsequent siblings) 14 siblings, 0 replies; 80+ messages in thread From: Chaoyong He @ 2023-12-26 2:37 UTC (permalink / raw) To: dev; +Cc: oss-drivers, Long Wu, Peng Zhang, Chaoyong He From: Long Wu <long.wu@corigine.com> Firmware supports several features and bond firmware creation is one of the features. Driver notifies firmware that driver supports bond firmware creation feature by CPP bus write. If write successfully, initialize driver configuration. Signed-off-by: Long Wu <long.wu@corigine.com> Reviewed-by: Peng Zhang <peng.zhang@corigine.com> Reviewed-by: Chaoyong He <chaoyong.he@corigine.com> --- drivers/net/nfp/flower/nfp_flower.c | 64 ++++++++++++++++++++++- drivers/net/nfp/flower/nfp_flower.h | 13 +++++ drivers/net/nfp/flower/nfp_flower_bond.c | 59 +++++++++++++++++++++ drivers/net/nfp/flower/nfp_flower_bond.h | 65 ++++++++++++++++++++++++ drivers/net/nfp/meson.build | 1 + 5 files changed, 200 insertions(+), 2 deletions(-) create mode 100644 drivers/net/nfp/flower/nfp_flower_bond.c create mode 100644 drivers/net/nfp/flower/nfp_flower_bond.h diff --git a/drivers/net/nfp/flower/nfp_flower.c b/drivers/net/nfp/flower/nfp_flower.c index 94b50611f0..195960e00d 100644 --- a/drivers/net/nfp/flower/nfp_flower.c +++ b/drivers/net/nfp/flower/nfp_flower.c @@ -16,6 +16,7 @@ #include "../nfp_cpp_bridge.h" #include "../nfp_logs.h" #include "../nfp_mtr.h" +#include "nfp_flower_bond.h" #include "nfp_flower_ctrl.h" #include "nfp_flower_representor.h" @@ -228,6 +229,42 @@ nfp_flower_init_vnic_common(struct nfp_net_hw *hw, return 0; } +static int +nfp_flower_bond_feature_init(struct nfp_app_fw_flower *app_fw_flower) +{ + int ret; + + /* Notify hardware that driver supports hardware creation of bonding port */ + ret = nfp_rtsym_write_le(app_fw_flower->pf_hw->pf_dev->sym_tbl, + "_abi_flower_balance_sync_enable", 1); + if (ret != 0) { + PMD_INIT_LOG(ERR, "Enable bonding port hardware creation failed."); + return ret; + } + + ret = nfp_flower_bond_init(app_fw_flower); + if (ret != 0) { + PMD_INIT_LOG(ERR, "Initialize bonding structure failed."); + return ret; + } + + app_fw_flower->flower_en_feats |= NFP_FL_ENABLE_BOND; + + return 0; +} + +static int +nfp_flower_sync_feature_bits(struct nfp_app_fw_flower *app_fw_flower) +{ + int ret; + + ret = nfp_flower_bond_feature_init(app_fw_flower); + if (ret != 0) + return ret; + + return 0; +} + static int nfp_flower_init_ctrl_vnic(struct nfp_net_hw *hw) { @@ -538,6 +575,20 @@ nfp_flower_cleanup_ctrl_vnic(struct nfp_net_hw *hw) rte_free(eth_dev); } +static void +nfp_flower_bond_feature_cleanup(struct nfp_app_fw_flower *app_fw_flower) +{ + nfp_flower_bond_cleanup(app_fw_flower); + app_fw_flower->flower_en_feats &= ~NFP_FL_ENABLE_BOND; +} + +static void +nfp_flower_sync_feature_cleanup(struct nfp_app_fw_flower *app_fw_flower) +{ + if (nfp_flower_support_bond_offload(app_fw_flower)) + nfp_flower_bond_feature_cleanup(app_fw_flower); +} + static int nfp_flower_start_ctrl_vnic(struct nfp_net_hw *net_hw) { @@ -768,22 +819,31 @@ nfp_init_app_fw_flower(struct nfp_pf_dev *pf_dev, goto ctrl_vnic_cleanup; } + /* Synchronize the features of driver and hardware */ + ret = nfp_flower_sync_feature_bits(app_fw_flower); + if (ret != 0) { + PMD_INIT_LOG(ERR, "Sync feature bits failed"); + goto ctrl_vnic_cleanup; + } + /* Start up flower services */ ret = nfp_flower_enable_services(app_fw_flower); if (ret != 0) { PMD_INIT_LOG(ERR, "Could not enable flower services"); ret = -ESRCH; - goto ctrl_vnic_cleanup; + goto sync_feature_cleanup; } ret = nfp_flower_repr_create(app_fw_flower); if (ret != 0) { PMD_INIT_LOG(ERR, "Could not create representor ports"); - goto ctrl_vnic_cleanup; + goto sync_feature_cleanup; } return 0; +sync_feature_cleanup: + nfp_flower_sync_feature_cleanup(app_fw_flower); ctrl_vnic_cleanup: nfp_flower_cleanup_ctrl_vnic(app_fw_flower->ctrl_hw); ctrl_cpp_area_cleanup: diff --git a/drivers/net/nfp/flower/nfp_flower.h b/drivers/net/nfp/flower/nfp_flower.h index 8393de66c5..30443cd568 100644 --- a/drivers/net/nfp/flower/nfp_flower.h +++ b/drivers/net/nfp/flower/nfp_flower.h @@ -7,6 +7,7 @@ #define __NFP_FLOWER_H__ #include "../nfp_net_common.h" +#include "nfp_flower_bond.h" /* Extra features bitmap. */ #define NFP_FL_FEATS_GENEVE RTE_BIT64(0) @@ -61,6 +62,9 @@ struct nfp_app_fw_flower { /** Number of phyport representors */ uint8_t num_phyport_reprs; + /** Bitmap of features enabled by HW */ + uint8_t flower_en_feats; + /** Pointer to the PF vNIC */ struct nfp_net_hw *pf_hw; @@ -96,6 +100,9 @@ struct nfp_app_fw_flower { /** Function pointers for different NFD version */ struct nfp_flower_nfd_func nfd_func; + + /** Link bond data block */ + struct nfp_flower_bond *nfp_bond; }; static inline bool @@ -104,6 +111,12 @@ nfp_flower_support_decap_v2(const struct nfp_app_fw_flower *app_fw_flower) return app_fw_flower->ext_features & NFP_FL_FEATS_DECAP_V2; } +static inline bool +nfp_flower_support_bond_offload(const struct nfp_app_fw_flower *app_fw_flower) +{ + return app_fw_flower->flower_en_feats & NFP_FL_ENABLE_BOND; +} + int nfp_init_app_fw_flower(struct nfp_pf_dev *pf_dev, const struct nfp_dev_info *dev_info); void nfp_uninit_app_fw_flower(struct nfp_pf_dev *pf_dev); diff --git a/drivers/net/nfp/flower/nfp_flower_bond.c b/drivers/net/nfp/flower/nfp_flower_bond.c new file mode 100644 index 0000000000..bbd2818e68 --- /dev/null +++ b/drivers/net/nfp/flower/nfp_flower_bond.c @@ -0,0 +1,59 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright (c) 2023 Corigine, Inc. + * All rights reserved. + */ +#include "nfp_flower_bond.h" + +#include <rte_malloc.h> + +#include "nfp_flower_representor.h" + +static void +nfp_flower_bond_increment_version(struct nfp_flower_bond *nfp_bond) +{ + /* LSB is not considered by firmware so add 2 for each increment. */ + nfp_bond->batch_ver += 2; + nfp_bond->batch_ver &= NFP_FL_BOND_VERSION_MASK; + + /* Zero is reserved by firmware. */ + if (nfp_bond->batch_ver == 0) + nfp_bond->batch_ver += 2; +} + +int +nfp_flower_bond_init(struct nfp_app_fw_flower *app_fw_flower) +{ + struct nfp_flower_bond *nfp_bond; + + app_fw_flower->nfp_bond = rte_zmalloc("nfp_bond", + sizeof(struct nfp_flower_bond), RTE_CACHE_LINE_SIZE); + if (app_fw_flower->nfp_bond == NULL) + return -ENOMEM; + + nfp_bond = app_fw_flower->nfp_bond; + pthread_mutex_init(&nfp_bond->mutex, NULL); + LIST_INIT(&nfp_bond->group_list); + nfp_flower_bond_increment_version(nfp_bond); + nfp_bond->app_fw_flower = app_fw_flower; + + return 0; +} + +void +nfp_flower_bond_cleanup(struct nfp_app_fw_flower *app_fw_flower) +{ + struct nfp_bond_group *entry; + struct nfp_flower_bond *nfp_bond = app_fw_flower->nfp_bond; + + pthread_mutex_lock(&nfp_bond->mutex); + LIST_FOREACH(entry, &nfp_bond->group_list, next) { + LIST_REMOVE(entry, next); + rte_free(entry); + } + pthread_mutex_unlock(&nfp_bond->mutex); + + pthread_mutex_destroy(&nfp_bond->mutex); + + rte_free(nfp_bond); + app_fw_flower->nfp_bond = NULL; +} diff --git a/drivers/net/nfp/flower/nfp_flower_bond.h b/drivers/net/nfp/flower/nfp_flower_bond.h new file mode 100644 index 0000000000..9612b2c6cf --- /dev/null +++ b/drivers/net/nfp/flower/nfp_flower_bond.h @@ -0,0 +1,65 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright(c) 2023 Corigine, Inc. + * All rights reserved. + */ + +#ifndef __NFP_FLOWER_BOND_H__ +#define __NFP_FLOWER_BOND_H__ + +#include <rte_spinlock.h> +#include <stdbool.h> +#include <sys/queue.h> + +/* The batch version of bond offload packets between firmware and driver */ +#define NFP_FL_BOND_VERSION_MASK 0x007fffff /* [0, 22] */ + +#define NFP_FL_ENABLE_BOND RTE_BIT32(1) + +/* ID 0 reserved and IDs 1 to 31 are valid */ +#define NFP_FL_BOND_GROUP_MIN 1 +#define NFP_FL_BOND_GROUP_MAX 32 + +/* List entry for each bond group */ +struct nfp_bond_group { + /** List entry */ + LIST_ENTRY(nfp_bond_group) next; + /** Marked if the group needs synced to HW */ + bool dirty; + /** Marked if the group is currently offloaded to NIC */ + bool offloaded; + /** Marked if the group should be removed from NIC */ + bool to_remove; + /** Marked if the group should be removed from driver */ + bool to_destroy; + /** Assigned group ID for host/kernel sync */ + uint32_t group_id; + /** Number of members in group */ + uint32_t member_cnt; + /** Group instance in case of ID reuse */ + uint32_t group_inst; + /** Group main Netdev */ + struct rte_eth_dev *main_dev; +}; + +/* Flower APP priv data for bond offload */ +struct nfp_flower_bond { + /** Marker to reset firmware bond config */ + bool rst_cfg; + /** List of all main/member groups offloaded */ + LIST_HEAD(, nfp_bond_group) group_list; + /** Lock to protect bond group_list */ + pthread_mutex_t mutex; + /** Incremented for each batch of config packets */ + uint32_t batch_ver; + /** Instance allocator for groups */ + uint32_t global_inst; + /** Incremented for each config packet sent */ + uint32_t pkt_num; + /** Pointer to the flower app */ + struct nfp_app_fw_flower *app_fw_flower; +}; + +int nfp_flower_bond_init(struct nfp_app_fw_flower *app_fw_flower); +void nfp_flower_bond_cleanup(struct nfp_app_fw_flower *app_fw_flower); + +#endif /* __NFP_FLOWER_BOND_H__ */ diff --git a/drivers/net/nfp/meson.build b/drivers/net/nfp/meson.build index 46be6f60cd..f0d4bbcecb 100644 --- a/drivers/net/nfp/meson.build +++ b/drivers/net/nfp/meson.build @@ -13,6 +13,7 @@ sources = files( 'flower/nfp_flower_ctrl.c', 'flower/nfp_flower_flow.c', 'flower/nfp_flower_representor.c', + 'flower/nfp_flower_bond.c', 'nfd3/nfp_nfd3_dp.c', 'nfdk/nfp_nfdk_dp.c', 'nfpcore/nfp_cppcore.c', -- 2.39.1 ^ permalink raw reply [flat|nested] 80+ messages in thread
* [PATCH v5 09/14] net/nfp: reset bond configuration of firmware 2023-12-26 2:37 ` [PATCH v5 00/14] " Chaoyong He ` (7 preceding siblings ...) 2023-12-26 2:37 ` [PATCH v5 08/14] net/nfp: add bond firmware creation initialization Chaoyong He @ 2023-12-26 2:37 ` Chaoyong He 2023-12-26 2:37 ` [PATCH v5 10/14] net/nfp: handle link event of bond firmware creation Chaoyong He ` (5 subsequent siblings) 14 siblings, 0 replies; 80+ messages in thread From: Chaoyong He @ 2023-12-26 2:37 UTC (permalink / raw) To: dev; +Cc: oss-drivers, Long Wu, Peng Zhang, Chaoyong He From: Long Wu <long.wu@corigine.com> Driver sends control message to reset the bond firmware configuration in flower NIC initialization. Firmware should reset bond configuration to avoid strange problems caused by residues. Signed-off-by: Long Wu <long.wu@corigine.com> Reviewed-by: Peng Zhang <peng.zhang@corigine.com> Reviewed-by: Chaoyong He <chaoyong.he@corigine.com> --- drivers/net/nfp/flower/nfp_flower.c | 23 ++++++ drivers/net/nfp/flower/nfp_flower_bond.c | 90 ++++++++++++++++++++++++ drivers/net/nfp/flower/nfp_flower_bond.h | 54 ++++++++++++++ drivers/net/nfp/flower/nfp_flower_cmsg.c | 35 +++++++++ drivers/net/nfp/flower/nfp_flower_cmsg.h | 3 + 5 files changed, 205 insertions(+) diff --git a/drivers/net/nfp/flower/nfp_flower.c b/drivers/net/nfp/flower/nfp_flower.c index 195960e00d..fc3ea84828 100644 --- a/drivers/net/nfp/flower/nfp_flower.c +++ b/drivers/net/nfp/flower/nfp_flower.c @@ -589,6 +589,22 @@ nfp_flower_sync_feature_cleanup(struct nfp_app_fw_flower *app_fw_flower) nfp_flower_bond_feature_cleanup(app_fw_flower); } +static int +nfp_flower_start_features(struct nfp_app_fw_flower *app_flower) +{ + int ret; + + if (nfp_flower_support_bond_offload(app_flower)) { + ret = nfp_flower_bond_reset(app_flower->nfp_bond); + if (ret != 0) { + PMD_INIT_LOG(ERR, "Reset bond feature failed"); + return ret; + } + } + + return 0; +} + static int nfp_flower_start_ctrl_vnic(struct nfp_net_hw *net_hw) { @@ -826,6 +842,13 @@ nfp_init_app_fw_flower(struct nfp_pf_dev *pf_dev, goto ctrl_vnic_cleanup; } + /* Start up some features */ + ret = nfp_flower_start_features(app_fw_flower); + if (ret != 0) { + PMD_INIT_LOG(ERR, "Failed to start features"); + goto sync_feature_cleanup; + } + /* Start up flower services */ ret = nfp_flower_enable_services(app_fw_flower); if (ret != 0) { diff --git a/drivers/net/nfp/flower/nfp_flower_bond.c b/drivers/net/nfp/flower/nfp_flower_bond.c index bbd2818e68..4ac27f117c 100644 --- a/drivers/net/nfp/flower/nfp_flower_bond.c +++ b/drivers/net/nfp/flower/nfp_flower_bond.c @@ -6,8 +6,31 @@ #include <rte_malloc.h> +#include "nfp_flower_cmsg.h" #include "nfp_flower_representor.h" +static void +nfp_fl_bond_cmsg_args_init(struct nfp_flower_bond_cmsg_args *cmsg_args, + struct nfp_bond_group *group, + struct rte_eth_dev **active_members, + uint32_t member_cnt, + enum nfp_flower_bond_batch batch) +{ + cmsg_args->group = group; + cmsg_args->active_members = active_members; + cmsg_args->member_cnt = member_cnt; + cmsg_args->batch = batch; +} + +static uint32_t +nfp_fl_get_next_pkt_number(struct nfp_flower_bond *nfp_bond) +{ + nfp_bond->pkt_num++; + nfp_bond->pkt_num &= NFP_FL_BOND_PKT_NUMBER_MASK; + + return nfp_bond->pkt_num; +} + static void nfp_flower_bond_increment_version(struct nfp_flower_bond *nfp_bond) { @@ -57,3 +80,70 @@ nfp_flower_bond_cleanup(struct nfp_app_fw_flower *app_fw_flower) rte_free(nfp_bond); app_fw_flower->nfp_bond = NULL; } + +int +nfp_flower_bond_reset(struct nfp_flower_bond *nfp_bond) +{ + struct nfp_app_fw_flower *app_flower; + enum nfp_flower_bond_batch batch = NFP_FLOWER_BOND_BATCH_FIRST; + struct nfp_flower_bond_cmsg_args init_args; + + app_flower = nfp_bond->app_fw_flower; + app_flower->nfp_bond->rst_cfg = true; + + nfp_fl_bond_cmsg_args_init(&init_args, NULL, NULL, 0, batch); + + return nfp_flower_cmsg_bond_config_group(app_flower, &init_args, &batch); +} + +enum nfp_flower_bond_batch +nfp_flower_bond_cmsg_payload(struct nfp_flower_bond *nfp_bond, + struct nfp_flower_cmsg_bond_config *msg, + struct nfp_flower_bond_cmsg_args *init_args) +{ + uint32_t i; + uint8_t flags = 0; + struct nfp_flower_representor *repr; + enum nfp_flower_bond_batch batch = init_args->batch; + + /* Increment batch version for each new batch of config messages. */ + if (batch == NFP_FLOWER_BOND_BATCH_FIRST) { + flags |= NFP_FL_BOND_FIRST; + nfp_flower_bond_increment_version(nfp_bond); + batch = NFP_FLOWER_BOND_BATCH_MEMBER; + } + + /* If it is a reset msg then it is also the end of the batch. */ + if (nfp_bond->rst_cfg) { + flags |= NFP_FL_BOND_RESET; + batch = NFP_FLOWER_BOND_BATCH_FINISHED; + } + + /* + * To signal the end of a batch, both the switch and last flags are set + * and the reserved SYNC group ID is used. + */ + if (batch == NFP_FLOWER_BOND_BATCH_FINISHED) { + flags |= NFP_FL_BOND_SWITCH | NFP_FL_BOND_LAST; + nfp_bond->rst_cfg = false; + msg->group_id = rte_cpu_to_be_32(NFP_FL_BOND_SYNC_ID); + msg->group_inst = 0; + } else { + msg->group_id = rte_cpu_to_be_32(init_args->group->group_id); + msg->group_inst = rte_cpu_to_be_32(init_args->group->group_inst); + } + + msg->reserved[0] = 0; + msg->reserved[1] = 0; + msg->ttl = NFP_FL_BOND_HOST_TTL; + msg->ctrl_flags = flags; + msg->batch_ver = rte_cpu_to_be_32(nfp_bond->batch_ver); + msg->pkt_number = rte_cpu_to_be_32(nfp_fl_get_next_pkt_number(nfp_bond)); + + for (i = 0; i < init_args->member_cnt; i++) { + repr = init_args->active_members[i]->data->dev_private; + msg->members[i] = rte_cpu_to_be_32(repr->port_id); + } + + return batch; +} diff --git a/drivers/net/nfp/flower/nfp_flower_bond.h b/drivers/net/nfp/flower/nfp_flower_bond.h index 9612b2c6cf..bb7ba8a1ad 100644 --- a/drivers/net/nfp/flower/nfp_flower_bond.h +++ b/drivers/net/nfp/flower/nfp_flower_bond.h @@ -6,6 +6,7 @@ #ifndef __NFP_FLOWER_BOND_H__ #define __NFP_FLOWER_BOND_H__ +#include <rte_byteorder.h> #include <rte_spinlock.h> #include <stdbool.h> #include <sys/queue.h> @@ -13,12 +14,54 @@ /* The batch version of bond offload packets between firmware and driver */ #define NFP_FL_BOND_VERSION_MASK 0x007fffff /* [0, 22] */ +#define NFP_FL_BOND_PKT_NUMBER_MASK 0x7fffffff /* [0, 30] */ + #define NFP_FL_ENABLE_BOND RTE_BIT32(1) /* ID 0 reserved and IDs 1 to 31 are valid */ #define NFP_FL_BOND_GROUP_MIN 1 #define NFP_FL_BOND_GROUP_MAX 32 +#define NFP_FL_BOND_HOST_TTL 0xff + +/* Use this ID with zero members to ack a batch config */ +#define NFP_FL_BOND_SYNC_ID 0 + +/* BOND group config flags */ +#define NFP_FL_BOND_LAST RTE_BIT32(1) +#define NFP_FL_BOND_FIRST RTE_BIT32(2) +#define NFP_FL_BOND_DATA RTE_BIT32(3) +#define NFP_FL_BOND_XON RTE_BIT32(4) +#define NFP_FL_BOND_SYNC RTE_BIT32(5) +#define NFP_FL_BOND_SWITCH RTE_BIT32(6) +#define NFP_FL_BOND_RESET RTE_BIT32(7) + +enum nfp_flower_bond_batch { + NFP_FLOWER_BOND_BATCH_FIRST, + NFP_FLOWER_BOND_BATCH_MEMBER, + NFP_FLOWER_BOND_BATCH_FINISHED +}; + +/* Control message payload for bond config */ +struct nfp_flower_cmsg_bond_config { + /** Configuration flags */ + uint8_t ctrl_flags; + /** Reserved for future use */ + uint8_t reserved[2]; + /** Time to live of packet - host always sets to 0xff */ + uint8_t ttl; + /** Config message packet number - increment for each message */ + rte_be32_t pkt_number; + /** Batch version of messages - increment for each batch of messages */ + rte_be32_t batch_ver; + /** Group ID applicable */ + rte_be32_t group_id; + /** Group instance number - increment when group is reused */ + rte_be32_t group_inst; + /** Array of 32-bit words listing all active group members */ + rte_be32_t members[]; +}; + /* List entry for each bond group */ struct nfp_bond_group { /** List entry */ @@ -59,7 +102,18 @@ struct nfp_flower_bond { struct nfp_app_fw_flower *app_fw_flower; }; +struct nfp_flower_bond_cmsg_args { + struct nfp_bond_group *group; + struct rte_eth_dev **active_members; + uint32_t member_cnt; + enum nfp_flower_bond_batch batch; +}; + int nfp_flower_bond_init(struct nfp_app_fw_flower *app_fw_flower); void nfp_flower_bond_cleanup(struct nfp_app_fw_flower *app_fw_flower); +int nfp_flower_bond_reset(struct nfp_flower_bond *nfp_bond); +enum nfp_flower_bond_batch nfp_flower_bond_cmsg_payload(struct nfp_flower_bond *nfp_bond, + struct nfp_flower_cmsg_bond_config *msg, + struct nfp_flower_bond_cmsg_args *init_args); #endif /* __NFP_FLOWER_BOND_H__ */ diff --git a/drivers/net/nfp/flower/nfp_flower_cmsg.c b/drivers/net/nfp/flower/nfp_flower_cmsg.c index 8effe9474d..0cf1bf2281 100644 --- a/drivers/net/nfp/flower/nfp_flower_cmsg.c +++ b/drivers/net/nfp/flower/nfp_flower_cmsg.c @@ -567,3 +567,38 @@ nfp_flower_cmsg_qos_stats(struct nfp_app_fw_flower *app_fw_flower, return 0; } + +int +nfp_flower_cmsg_bond_config_group(struct nfp_app_fw_flower *app_flower, + struct nfp_flower_bond_cmsg_args *init_args, + enum nfp_flower_bond_batch *batch_out) +{ + uint16_t cnt; + uint32_t size; + struct rte_mbuf *mbuf; + struct nfp_flower_bond *nfp_bond; + struct nfp_flower_cmsg_bond_config *msg; + + mbuf = rte_pktmbuf_alloc(app_flower->ctrl_pktmbuf_pool); + if (mbuf == NULL) { + PMD_DRV_LOG(DEBUG, "Alloc mbuf for bond config failed"); + return -ENOMEM; + } + + size = sizeof(*msg) + sizeof(rte_be32_t) * init_args->member_cnt; + msg = nfp_flower_cmsg_init(app_flower, mbuf, + NFP_FLOWER_CMSG_TYPE_LAG_CONFIG, size); + + nfp_bond = app_flower->nfp_bond; + + *batch_out = nfp_flower_bond_cmsg_payload(nfp_bond, msg, init_args); + + cnt = nfp_flower_ctrl_vnic_xmit(app_flower, mbuf); + if (cnt == 0) { + PMD_DRV_LOG(ERR, "Send cmsg through ctrl vnic failed."); + rte_pktmbuf_free(mbuf); + return -EIO; + } + + return 0; +} diff --git a/drivers/net/nfp/flower/nfp_flower_cmsg.h b/drivers/net/nfp/flower/nfp_flower_cmsg.h index 45543816ae..60ab58a3b1 100644 --- a/drivers/net/nfp/flower/nfp_flower_cmsg.h +++ b/drivers/net/nfp/flower/nfp_flower_cmsg.h @@ -988,5 +988,8 @@ int nfp_flower_cmsg_qos_delete(struct nfp_app_fw_flower *app_fw_flower, struct nfp_profile_conf *conf); int nfp_flower_cmsg_qos_stats(struct nfp_app_fw_flower *app_fw_flower, struct nfp_cfg_head *head); +int nfp_flower_cmsg_bond_config_group(struct nfp_app_fw_flower *app_flower, + struct nfp_flower_bond_cmsg_args *init_args, + enum nfp_flower_bond_batch *batch_out); #endif /* __NFP_CMSG_H__ */ -- 2.39.1 ^ permalink raw reply [flat|nested] 80+ messages in thread
* [PATCH v5 10/14] net/nfp: handle link event of bond firmware creation 2023-12-26 2:37 ` [PATCH v5 00/14] " Chaoyong He ` (8 preceding siblings ...) 2023-12-26 2:37 ` [PATCH v5 09/14] net/nfp: reset bond configuration of firmware Chaoyong He @ 2023-12-26 2:37 ` Chaoyong He 2023-12-26 2:37 ` [PATCH v5 11/14] net/nfp: support bond member notification Chaoyong He ` (4 subsequent siblings) 14 siblings, 0 replies; 80+ messages in thread From: Chaoyong He @ 2023-12-26 2:37 UTC (permalink / raw) To: dev; +Cc: oss-drivers, Long Wu, Peng Zhang, Chaoyong He From: Long Wu <long.wu@corigine.com> If NFP physical representor port is a member port of bonding port, its link status changed and firmware has created the bonding port, driver will record the link status and send control message to notify firmware. Signed-off-by: Long Wu <long.wu@corigine.com> Reviewed-by: Peng Zhang <peng.zhang@corigine.com> Reviewed-by: Chaoyong He <chaoyong.he@corigine.com> --- drivers/net/nfp/flower/nfp_flower_bond.c | 259 ++++++++++++++++++ drivers/net/nfp/flower/nfp_flower_bond.h | 22 ++ .../net/nfp/flower/nfp_flower_representor.c | 13 + .../net/nfp/flower/nfp_flower_representor.h | 2 + drivers/net/nfp/meson.build | 2 +- drivers/net/nfp/nfp_net_common.c | 25 ++ drivers/net/nfp/nfp_net_common.h | 1 - 7 files changed, 322 insertions(+), 2 deletions(-) diff --git a/drivers/net/nfp/flower/nfp_flower_bond.c b/drivers/net/nfp/flower/nfp_flower_bond.c index 4ac27f117c..c814c0d4ea 100644 --- a/drivers/net/nfp/flower/nfp_flower_bond.c +++ b/drivers/net/nfp/flower/nfp_flower_bond.c @@ -4,8 +4,11 @@ */ #include "nfp_flower_bond.h" +#include <eth_bond_private.h> +#include <rte_eth_bond.h> #include <rte_malloc.h> +#include "../nfp_logs.h" #include "nfp_flower_cmsg.h" #include "nfp_flower_representor.h" @@ -43,6 +46,72 @@ nfp_flower_bond_increment_version(struct nfp_flower_bond *nfp_bond) nfp_bond->batch_ver += 2; } +static enum nfp_flower_bond_batch +nfp_flower_bond_remove_node(struct nfp_flower_bond *nfp_bond, + struct nfp_bond_group *entry, + enum nfp_flower_bond_batch batch) +{ + enum nfp_flower_bond_batch batch_out; + struct nfp_flower_bond_cmsg_args cmsg_args; + struct nfp_app_fw_flower *app_flower = nfp_bond->app_fw_flower; + + nfp_fl_bond_cmsg_args_init(&cmsg_args, entry, NULL, 0, batch); + if (nfp_flower_cmsg_bond_config_group(app_flower, &cmsg_args, &batch_out) != 0) { + PMD_DRV_LOG(ERR, "group remove failed."); + return batch_out; + } + + entry->to_remove = false; + entry->offloaded = false; + + if (entry->to_destroy) { + LIST_REMOVE(entry, next); + rte_free(entry); + } + + return batch_out; +} + +static void +nfp_flower_bond_member_work_status(struct nfp_bond_group *entry, + struct rte_eth_dev **active_eth_devs, + uint32_t *active) +{ + uint32_t i; + uint32_t *flags; + uint32_t active_count; + uint16_t member_port_id; + struct rte_eth_dev *eth_dev; + struct bond_dev_private *internals; + struct nfp_flower_representor *repr; + + active_count = 0; + + internals = entry->main_dev->data->dev_private; + for (i = 0; i < internals->member_count; i++) { + member_port_id = internals->members[i].port_id; + if (internals->current_primary_port == member_port_id || + internals->mode != BONDING_MODE_ACTIVE_BACKUP) { + eth_dev = &rte_eth_devices[member_port_id]; + repr = eth_dev->data->dev_private; + flags = &repr->bond_port_flags; + + if ((*flags & NFP_FL_BOND_PORT_CHANGED) != 0) { + *flags &= ~NFP_FL_BOND_PORT_CHANGED; + entry->dirty = true; + } + + if ((*flags & NFP_FL_BOND_PORT_TX_ENABLED) != 0 && + (*flags & NFP_FL_BOND_PORT_LINK_UP) != 0) { + active_eth_devs[active_count] = eth_dev; + active_count++; + } + } + } + + *active = active_count; +} + int nfp_flower_bond_init(struct nfp_app_fw_flower *app_fw_flower) { @@ -96,6 +165,132 @@ nfp_flower_bond_reset(struct nfp_flower_bond *nfp_bond) return nfp_flower_cmsg_bond_config_group(app_flower, &init_args, &batch); } +struct nfp_bond_group * +nfp_flower_bond_find_group(struct nfp_flower_bond *nfp_bond, + const struct rte_eth_dev *bond_dev) +{ + struct nfp_bond_group *group; + + LIST_FOREACH(group, &nfp_bond->group_list, next) { + if (group->main_dev == bond_dev) + return group; + } + + return NULL; +} + +void +nfp_flower_bond_do_work(struct nfp_flower_bond *nfp_bond) +{ + int ret; + uint32_t active_count; + struct nfp_bond_group *entry; + struct nfp_app_fw_flower *app_flower; + struct nfp_flower_bond_cmsg_args cmsg_args; + struct rte_eth_dev *active_eth_devs[RTE_MAX_ETHPORTS]; + enum nfp_flower_bond_batch batch = NFP_FLOWER_BOND_BATCH_FIRST; + + app_flower = nfp_bond->app_fw_flower; + + pthread_mutex_lock(&nfp_bond->mutex); + + LIST_FOREACH(entry, &nfp_bond->group_list, next) { + if (entry->to_remove) { + batch = nfp_flower_bond_remove_node(nfp_bond, entry, batch); + continue; + } + + nfp_flower_bond_member_work_status(entry, active_eth_devs, + &active_count); + + if (!entry->dirty) + continue; + + nfp_fl_bond_cmsg_args_init(&cmsg_args, entry, active_eth_devs, + active_count, batch); + + ret = nfp_flower_cmsg_bond_config_group(app_flower, &cmsg_args, &batch); + if (ret == 0) { + entry->offloaded = true; + entry->dirty = false; + } else { + PMD_DRV_LOG(DEBUG, "Group config failed."); + } + } + + /* End the config batch if at least one packet has been batched. */ + if (batch == NFP_FLOWER_BOND_BATCH_MEMBER) { + batch = NFP_FLOWER_BOND_BATCH_FINISHED; + nfp_fl_bond_cmsg_args_init(&cmsg_args, NULL, NULL, 0, batch); + ret = nfp_flower_cmsg_bond_config_group(app_flower, &cmsg_args, &batch); + if (ret != 0) + PMD_DRV_LOG(DEBUG, "Group batch end cmsg failed"); + } + + pthread_mutex_unlock(&nfp_bond->mutex); + + /* + * A batch of packets rx in firmware has been set for at least two timing + * units, so trigger copying at next opportunity to limit latency from + * receiving data to being available for use. + */ + rte_delay_ms(1); +} + +static void +nfp_flower_bond_change_linkstatus_event(struct nfp_flower_bond *nfp_bond, + struct rte_eth_dev *bond_dev, + struct rte_eth_dev *nfp_dev) +{ + uint32_t *bond_port_flags; + struct nfp_bond_group *group; + struct nfp_flower_representor *repr; + + if (!nfp_flower_bond_is_member_port(bond_dev, nfp_dev)) + return; + + pthread_mutex_lock(&nfp_bond->mutex); + + group = nfp_flower_bond_find_group(nfp_bond, bond_dev); + if (group == NULL) { + pthread_mutex_unlock(&nfp_bond->mutex); + return; + } + + pthread_mutex_unlock(&nfp_bond->mutex); + + repr = nfp_dev->data->dev_private; + bond_port_flags = &repr->bond_port_flags; + + if (repr->link.link_status == RTE_ETH_LINK_UP) + *bond_port_flags |= NFP_FL_BOND_PORT_LINK_UP; + else + *bond_port_flags &= ~NFP_FL_BOND_PORT_LINK_UP; + + *bond_port_flags |= NFP_FL_BOND_PORT_TX_ENABLED; + *bond_port_flags |= NFP_FL_BOND_PORT_CHANGED; +} + +int +nfp_flower_bond_event_handle(struct nfp_flower_bond *nfp_bond, + struct rte_eth_dev *bond_dev, + struct rte_eth_dev *nfp_dev, + enum nfp_flower_bond_event event) +{ + switch (event) { + case NFP_FLOWER_CHANGELINKSTATUS: + nfp_flower_bond_change_linkstatus_event(nfp_bond, bond_dev, nfp_dev); + break; + default: + PMD_DRV_LOG(ERR, "Invalid bond offload event."); + return -1; + } + + nfp_flower_bond_do_work(nfp_bond); + + return 0; +} + enum nfp_flower_bond_batch nfp_flower_bond_cmsg_payload(struct nfp_flower_bond *nfp_bond, struct nfp_flower_cmsg_bond_config *msg, @@ -147,3 +342,67 @@ nfp_flower_bond_cmsg_payload(struct nfp_flower_bond *nfp_bond, return batch; } + +bool +nfp_flower_bond_is_member_port(struct rte_eth_dev *bond_dev, + struct rte_eth_dev *nfp_dev) +{ + uint32_t i; + struct bond_dev_private *internals; + + if (!nfp_flower_is_phy_repr(nfp_dev)) + return false; + + internals = bond_dev->data->dev_private; + + for (i = 0; i < internals->member_count; i++) { + if (nfp_dev == &rte_eth_devices[internals->members[i].port_id]) + return true; + } + + return false; +} + +struct rte_eth_dev * +nfp_flower_bond_find_bond_device(struct rte_eth_dev *nfp_dev) +{ + struct nfp_bond_group *group; + struct nfp_flower_representor *repr; + struct rte_eth_dev *bond_dev = NULL; + + if (!nfp_flower_is_phy_repr(nfp_dev)) + return NULL; + + repr = nfp_dev->data->dev_private; + + pthread_mutex_lock(&repr->app_fw_flower->nfp_bond->mutex); + LIST_FOREACH(group, &repr->app_fw_flower->nfp_bond->group_list, next) { + if (nfp_flower_bond_is_member_port(group->main_dev, nfp_dev)) { + bond_dev = group->main_dev; + break; + } + } + pthread_mutex_unlock(&repr->app_fw_flower->nfp_bond->mutex); + + return bond_dev; +} + +bool +nfp_flower_bond_all_member_are_phyrepr(struct rte_eth_dev *bond_dev) +{ + uint32_t i; + struct rte_eth_dev *eth_dev; + struct bond_dev_private *internals; + + if (bond_dev == NULL) + return false; + + internals = bond_dev->data->dev_private; + for (i = 0; i < internals->member_count; i++) { + eth_dev = &rte_eth_devices[internals->members[i].port_id]; + if (!nfp_flower_is_phy_repr(eth_dev)) + return false; + } + + return true; +} diff --git a/drivers/net/nfp/flower/nfp_flower_bond.h b/drivers/net/nfp/flower/nfp_flower_bond.h index bb7ba8a1ad..80f56a6780 100644 --- a/drivers/net/nfp/flower/nfp_flower_bond.h +++ b/drivers/net/nfp/flower/nfp_flower_bond.h @@ -36,12 +36,23 @@ #define NFP_FL_BOND_SWITCH RTE_BIT32(6) #define NFP_FL_BOND_RESET RTE_BIT32(7) +/* BOND port state flags. */ +#define NFP_FL_BOND_PORT_LINK_UP RTE_BIT32(0) +#define NFP_FL_BOND_PORT_TX_ENABLED RTE_BIT32(1) +#define NFP_FL_BOND_PORT_CHANGED RTE_BIT32(2) + enum nfp_flower_bond_batch { NFP_FLOWER_BOND_BATCH_FIRST, NFP_FLOWER_BOND_BATCH_MEMBER, NFP_FLOWER_BOND_BATCH_FINISHED }; +enum nfp_flower_bond_event { + NFP_FLOWER_CHANGEUPPER, + NFP_FLOWER_CHANGELINKSTATUS, + NFP_FLOWER_UNREGISTER +}; + /* Control message payload for bond config */ struct nfp_flower_cmsg_bond_config { /** Configuration flags */ @@ -115,5 +126,16 @@ int nfp_flower_bond_reset(struct nfp_flower_bond *nfp_bond); enum nfp_flower_bond_batch nfp_flower_bond_cmsg_payload(struct nfp_flower_bond *nfp_bond, struct nfp_flower_cmsg_bond_config *msg, struct nfp_flower_bond_cmsg_args *init_args); +struct rte_eth_dev *nfp_flower_bond_find_bond_device(struct rte_eth_dev *nfp_dev); +bool nfp_flower_bond_is_member_port(struct rte_eth_dev *bond_dev, + struct rte_eth_dev *nfp_dev); +struct nfp_bond_group *nfp_flower_bond_find_group(struct nfp_flower_bond *nfp_bond, + const struct rte_eth_dev *bond_dev); +int nfp_flower_bond_event_handle(struct nfp_flower_bond *nfp_bond, + struct rte_eth_dev *bond_dev, + struct rte_eth_dev *nfp_dev, + enum nfp_flower_bond_event event); +void nfp_flower_bond_do_work(struct nfp_flower_bond *nfp_bond); +bool nfp_flower_bond_all_member_are_phyrepr(struct rte_eth_dev *bond_dev); #endif /* __NFP_FLOWER_BOND_H__ */ diff --git a/drivers/net/nfp/flower/nfp_flower_representor.c b/drivers/net/nfp/flower/nfp_flower_representor.c index 7d8c055b80..2810a7a271 100644 --- a/drivers/net/nfp/flower/nfp_flower_representor.c +++ b/drivers/net/nfp/flower/nfp_flower_representor.c @@ -827,3 +827,16 @@ nfp_flower_repr_create(struct nfp_app_fw_flower *app_fw_flower) return ret; } + +bool +nfp_flower_is_phy_repr(struct rte_eth_dev *eth_dev) +{ + struct nfp_flower_representor *repr; + + if ((eth_dev->data->dev_flags & RTE_ETH_DEV_REPRESENTOR) != 0) { + repr = eth_dev->data->dev_private; + return repr->repr_type == NFP_REPR_TYPE_PHYS_PORT; + } + + return false; +} diff --git a/drivers/net/nfp/flower/nfp_flower_representor.h b/drivers/net/nfp/flower/nfp_flower_representor.h index 8053617562..164e2f1322 100644 --- a/drivers/net/nfp/flower/nfp_flower_representor.h +++ b/drivers/net/nfp/flower/nfp_flower_representor.h @@ -14,6 +14,7 @@ struct nfp_flower_representor { uint32_t repr_type; uint32_t port_id; uint32_t nfp_idx; /**< Only valid for the repr of physical port */ + uint32_t bond_port_flags; char name[RTE_ETH_NAME_MAX_LEN]; struct rte_ether_addr mac_addr; struct nfp_app_fw_flower *app_fw_flower; @@ -24,5 +25,6 @@ struct nfp_flower_representor { }; int nfp_flower_repr_create(struct nfp_app_fw_flower *app_fw_flower); +bool nfp_flower_is_phy_repr(struct rte_eth_dev *eth_dev); #endif /* __NFP_FLOWER_REPRESENTOR_H__ */ diff --git a/drivers/net/nfp/meson.build b/drivers/net/nfp/meson.build index f0d4bbcecb..c309505c83 100644 --- a/drivers/net/nfp/meson.build +++ b/drivers/net/nfp/meson.build @@ -42,4 +42,4 @@ sources = files( 'nfp_rxtx.c', ) -deps += ['hash', 'security', 'common_nfp'] +deps += ['hash', 'security', 'common_nfp', 'net_bond'] diff --git a/drivers/net/nfp/nfp_net_common.c b/drivers/net/nfp/nfp_net_common.c index a438eb5871..810903c033 100644 --- a/drivers/net/nfp/nfp_net_common.c +++ b/drivers/net/nfp/nfp_net_common.c @@ -9,6 +9,7 @@ #include <rte_alarm.h> +#include "flower/nfp_flower.h" #include "flower/nfp_flower_representor.h" #include "nfd3/nfp_nfd3.h" #include "nfdk/nfp_nfdk.h" @@ -719,6 +720,23 @@ nfp_net_speed_aneg_update(struct rte_eth_dev *dev, return 0; } +static void +nfp_net_bond_link_event_notify(struct rte_eth_dev *dev) +{ + struct rte_eth_dev *bond_dev; + struct nfp_flower_representor *repr = dev->data->dev_private; + + if (!nfp_flower_support_bond_offload(repr->app_fw_flower)) + return; + + bond_dev = nfp_flower_bond_find_bond_device(dev); + if (!nfp_flower_bond_all_member_are_phyrepr(bond_dev)) + return; + + nfp_flower_bond_event_handle(repr->app_fw_flower->nfp_bond, bond_dev, + dev, NFP_FLOWER_CHANGELINKSTATUS); +} + int nfp_net_link_update_common(struct rte_eth_dev *dev, struct nfp_net_hw *hw, @@ -753,6 +771,13 @@ nfp_net_link_update_common(struct rte_eth_dev *dev, PMD_DRV_LOG(INFO, "NIC Link is Up"); else PMD_DRV_LOG(INFO, "NIC Link is Down"); + + /* + * Link status changed and if repr is a member port in bond device, + * we need to call func to do something special. + */ + if ((dev->data->dev_flags & RTE_ETH_DEV_REPRESENTOR) != 0) + nfp_net_bond_link_event_notify(dev); } return ret; diff --git a/drivers/net/nfp/nfp_net_common.h b/drivers/net/nfp/nfp_net_common.h index 66c900e3b8..9d6cf4b016 100644 --- a/drivers/net/nfp/nfp_net_common.h +++ b/drivers/net/nfp/nfp_net_common.h @@ -295,7 +295,6 @@ int nfp_net_fec_get(struct rte_eth_dev *dev, uint32_t *fec_capa); int nfp_net_fec_set(struct rte_eth_dev *dev, uint32_t fec_capa); - #define NFP_PRIV_TO_APP_FW_NIC(app_fw_priv)\ ((struct nfp_app_fw_nic *)app_fw_priv) -- 2.39.1 ^ permalink raw reply [flat|nested] 80+ messages in thread
* [PATCH v5 11/14] net/nfp: support bond member notification 2023-12-26 2:37 ` [PATCH v5 00/14] " Chaoyong He ` (9 preceding siblings ...) 2023-12-26 2:37 ` [PATCH v5 10/14] net/nfp: handle link event of bond firmware creation Chaoyong He @ 2023-12-26 2:37 ` Chaoyong He 2023-12-26 2:37 ` [PATCH v5 12/14] net/nfp: handle bond packets from firmware Chaoyong He ` (3 subsequent siblings) 14 siblings, 0 replies; 80+ messages in thread From: Chaoyong He @ 2023-12-26 2:37 UTC (permalink / raw) To: dev; +Cc: oss-drivers, Long Wu, Peng Zhang, Chaoyong He From: Long Wu <long.wu@corigine.com> A NFP representor port can receive a bond notification. Then driver will parse this notification into one of these two events: 1. Bonding port configuration may have changed. This includes creation of a bonding port, removal/addition of a member port, changing the bond mode, etc. 2. Bonding port is deleted. Signed-off-by: Long Wu <long.wu@corigine.com> Reviewed-by: Peng Zhang <peng.zhang@corigine.com> Reviewed-by: Chaoyong He <chaoyong.he@corigine.com> --- drivers/net/nfp/flower/nfp_flower_bond.c | 183 +++++++++++++++++- drivers/net/nfp/flower/nfp_flower_bond.h | 2 + .../net/nfp/flower/nfp_flower_representor.c | 21 ++ 3 files changed, 205 insertions(+), 1 deletion(-) diff --git a/drivers/net/nfp/flower/nfp_flower_bond.c b/drivers/net/nfp/flower/nfp_flower_bond.c index c814c0d4ea..523e0025ad 100644 --- a/drivers/net/nfp/flower/nfp_flower_bond.c +++ b/drivers/net/nfp/flower/nfp_flower_bond.c @@ -46,6 +46,39 @@ nfp_flower_bond_increment_version(struct nfp_flower_bond *nfp_bond) nfp_bond->batch_ver += 2; } +static void +nfp_flower_bond_group_id_clear(struct nfp_flower_bond *nfp_bond, + uint32_t id_clear) +{ + bool *group_id_map = nfp_bond->group_id_map; + + if (id_clear >= NFP_FL_BOND_GROUP_MAX || id_clear < NFP_FL_BOND_GROUP_MIN) { + PMD_DRV_LOG(ERR, "Try to clear invalid group id %u.", id_clear); + return; + } + + if (group_id_map[id_clear]) + group_id_map[id_clear] = false; +} + +static int +nfp_flower_bond_group_id_get(struct nfp_flower_bond *nfp_bond, + uint32_t *id_ret) +{ + uint32_t id; + bool *group_id_map = nfp_bond->group_id_map; + + for (id = NFP_FL_BOND_GROUP_MIN; id < NFP_FL_BOND_GROUP_MAX; id++) { + if (!group_id_map[id]) { + group_id_map[id] = true; + *id_ret = id; + return 0; + } + } + + return -ENOSPC; +} + static enum nfp_flower_bond_batch nfp_flower_bond_remove_node(struct nfp_flower_bond *nfp_bond, struct nfp_bond_group *entry, @@ -65,6 +98,7 @@ nfp_flower_bond_remove_node(struct nfp_flower_bond *nfp_bond, entry->offloaded = false; if (entry->to_destroy) { + nfp_flower_bond_group_id_clear(nfp_bond, entry->group_id); LIST_REMOVE(entry, next); rte_free(entry); } @@ -139,6 +173,7 @@ nfp_flower_bond_cleanup(struct nfp_app_fw_flower *app_fw_flower) pthread_mutex_lock(&nfp_bond->mutex); LIST_FOREACH(entry, &nfp_bond->group_list, next) { + nfp_flower_bond_group_id_clear(nfp_bond, entry->group_id); LIST_REMOVE(entry, next); rte_free(entry); } @@ -271,16 +306,162 @@ nfp_flower_bond_change_linkstatus_event(struct nfp_flower_bond *nfp_bond, *bond_port_flags |= NFP_FL_BOND_PORT_CHANGED; } +static struct nfp_bond_group * +nfp_flower_bond_group_create(struct nfp_flower_bond *nfp_bond, + struct rte_eth_dev *bond_dev) +{ + uint32_t id; + unsigned int numa_node; + struct nfp_bond_group *group = NULL; + + if (nfp_flower_bond_group_id_get(nfp_bond, &id) < 0) + return NULL; + + numa_node = rte_socket_id(); + + group = rte_zmalloc_socket(NULL, sizeof(struct nfp_bond_group), + RTE_CACHE_LINE_SIZE, numa_node); + if (group == NULL) { + PMD_DRV_LOG(ERR, "Unable malloc memory for nfp bond group"); + nfp_flower_bond_group_id_clear(nfp_bond, id); + return NULL; + } + + group->group_id = id; + group->main_dev = bond_dev; + group->dirty = true; + group->offloaded = false; + group->to_remove = false; + group->to_destroy = false; + group->member_cnt = 0; + group->group_inst = ++nfp_bond->global_inst; + LIST_INSERT_HEAD(&nfp_bond->group_list, group, next); + + return group; +} + +static int +nfp_flower_bond_changeupper_event(struct nfp_flower_bond *nfp_bond, + struct rte_eth_dev *bond_dev) +{ + uint32_t i; + uint16_t port_id; + bool can_offload = true; + uint16_t nfp_member_count; + struct rte_eth_dev *eth_dev; + struct nfp_bond_group *group; + struct bond_dev_private *internals; + struct nfp_flower_representor *repr; + struct nfp_app_fw_flower *app_flower; + + internals = bond_dev->data->dev_private; + app_flower = nfp_bond->app_fw_flower; + nfp_member_count = 0; + + for (i = 0; i < internals->member_count; i++) { + port_id = internals->members[i].port_id; + eth_dev = &rte_eth_devices[port_id]; + if (!nfp_flower_is_phy_repr(eth_dev)) { + can_offload = false; + break; + } + + repr = eth_dev->data->dev_private; + + if (repr->app_fw_flower != app_flower) { + can_offload = false; + break; + } + + if (internals->current_primary_port == port_id || + internals->mode != BONDING_MODE_ACTIVE_BACKUP) + nfp_member_count++; + } + + if (internals != NULL && + internals->mode != BONDING_MODE_ACTIVE_BACKUP && + ((internals->mode != BONDING_MODE_BALANCE && + internals->mode != BONDING_MODE_8023AD) || + internals->balance_xmit_policy != BALANCE_XMIT_POLICY_LAYER34)) { + can_offload = false; + PMD_DRV_LOG(WARNING, "Unable to offload mode %u hash %u.", + internals->mode, + internals->balance_xmit_policy); + } + + pthread_mutex_lock(&nfp_bond->mutex); + + group = nfp_flower_bond_find_group(nfp_bond, bond_dev); + if (nfp_member_count == 0 || !can_offload) { + if (group != NULL && group->offloaded) + /* Set remove flag */ + group->to_remove = true; + + pthread_mutex_unlock(&nfp_bond->mutex); + return 0; + } + + if (group == NULL) { + group = nfp_flower_bond_group_create(nfp_bond, bond_dev); + if (group == NULL) { + pthread_mutex_unlock(&nfp_bond->mutex); + return -1; + } + } + + group->dirty = true; + group->member_cnt = nfp_member_count; + group->to_remove = false; + + pthread_mutex_unlock(&nfp_bond->mutex); + + return 0; +} + +static void +nfp_flower_bond_group_delete(struct nfp_flower_bond *nfp_bond, + struct rte_eth_dev *bond_dev) +{ + struct nfp_bond_group *group; + + pthread_mutex_lock(&nfp_bond->mutex); + + group = nfp_flower_bond_find_group(nfp_bond, bond_dev); + if (group == NULL) { + pthread_mutex_unlock(&nfp_bond->mutex); + PMD_DRV_LOG(WARNING, "Untracked bond got unregistered %s", + bond_dev->device->name); + return; + } + + group->to_remove = true; + group->to_destroy = true; + + pthread_mutex_unlock(&nfp_bond->mutex); +} + int nfp_flower_bond_event_handle(struct nfp_flower_bond *nfp_bond, struct rte_eth_dev *bond_dev, struct rte_eth_dev *nfp_dev, enum nfp_flower_bond_event event) { + int ret = 0; + switch (event) { + case NFP_FLOWER_CHANGEUPPER: + ret = nfp_flower_bond_changeupper_event(nfp_bond, bond_dev); + if (ret != 0) { + PMD_DRV_LOG(ERR, "Change upper event can not work."); + return ret; + } + break; case NFP_FLOWER_CHANGELINKSTATUS: nfp_flower_bond_change_linkstatus_event(nfp_bond, bond_dev, nfp_dev); break; + case NFP_FLOWER_UNREGISTER: + nfp_flower_bond_group_delete(nfp_bond, bond_dev); + break; default: PMD_DRV_LOG(ERR, "Invalid bond offload event."); return -1; @@ -288,7 +469,7 @@ nfp_flower_bond_event_handle(struct nfp_flower_bond *nfp_bond, nfp_flower_bond_do_work(nfp_bond); - return 0; + return ret; } enum nfp_flower_bond_batch diff --git a/drivers/net/nfp/flower/nfp_flower_bond.h b/drivers/net/nfp/flower/nfp_flower_bond.h index 80f56a6780..f73589265f 100644 --- a/drivers/net/nfp/flower/nfp_flower_bond.h +++ b/drivers/net/nfp/flower/nfp_flower_bond.h @@ -109,6 +109,8 @@ struct nfp_flower_bond { uint32_t global_inst; /** Incremented for each config packet sent */ uint32_t pkt_num; + /** ID of bond group in driver, true is used */ + bool group_id_map[NFP_FL_BOND_GROUP_MAX]; /** Pointer to the flower app */ struct nfp_app_fw_flower *app_fw_flower; }; diff --git a/drivers/net/nfp/flower/nfp_flower_representor.c b/drivers/net/nfp/flower/nfp_flower_representor.c index 2810a7a271..43106da90e 100644 --- a/drivers/net/nfp/flower/nfp_flower_representor.c +++ b/drivers/net/nfp/flower/nfp_flower_representor.c @@ -390,6 +390,25 @@ nfp_flower_repr_dev_close(struct rte_eth_dev *dev) return 0; } +static int +nfp_flower_repr_bond_notify_member(struct rte_eth_dev *dev, + struct rte_eth_dev *bond_dev) +{ + struct nfp_flower_representor *repr; + enum nfp_flower_bond_event event = NFP_FLOWER_CHANGEUPPER; + + if (!nfp_flower_bond_all_member_are_phyrepr(bond_dev)) + event = NFP_FLOWER_UNREGISTER; + + repr = dev->data->dev_private; + + if (nfp_flower_support_bond_offload(repr->app_fw_flower)) + return nfp_flower_bond_event_handle(repr->app_fw_flower->nfp_bond, + bond_dev, dev, event); + + return 0; +} + static const struct eth_dev_ops nfp_flower_pf_repr_dev_ops = { .dev_infos_get = nfp_flower_repr_dev_infos_get, @@ -437,6 +456,8 @@ static const struct eth_dev_ops nfp_flower_repr_dev_ops = { .flow_ops_get = nfp_flow_ops_get, .mtr_ops_get = nfp_net_mtr_ops_get, + + .bond_notify_member = nfp_flower_repr_bond_notify_member, }; static uint32_t -- 2.39.1 ^ permalink raw reply [flat|nested] 80+ messages in thread
* [PATCH v5 12/14] net/nfp: handle bond packets from firmware 2023-12-26 2:37 ` [PATCH v5 00/14] " Chaoyong He ` (10 preceding siblings ...) 2023-12-26 2:37 ` [PATCH v5 11/14] net/nfp: support bond member notification Chaoyong He @ 2023-12-26 2:37 ` Chaoyong He 2023-12-26 2:37 ` [PATCH v5 13/14] net/nfp: support getting bond firmware creation Chaoyong He ` (2 subsequent siblings) 14 siblings, 0 replies; 80+ messages in thread From: Chaoyong He @ 2023-12-26 2:37 UTC (permalink / raw) To: dev; +Cc: oss-drivers, Long Wu, Peng Zhang, Chaoyong He From: Long Wu <long.wu@corigine.com> Firmware sends bond firmware creation packets to driver, driver needs to handle it in ctrl VNIC service. Signed-off-by: Long Wu <long.wu@corigine.com> Reviewed-by: Peng Zhang <peng.zhang@corigine.com> Reviewed-by: Chaoyong He <chaoyong.he@corigine.com> --- drivers/net/nfp/flower/nfp_flower_bond.c | 53 ++++++++++++ drivers/net/nfp/flower/nfp_flower_bond.h | 14 +++ drivers/net/nfp/flower/nfp_flower_ctrl.c | 106 +++++++++++++++++++++-- 3 files changed, 168 insertions(+), 5 deletions(-) diff --git a/drivers/net/nfp/flower/nfp_flower_bond.c b/drivers/net/nfp/flower/nfp_flower_bond.c index 523e0025ad..73b616360a 100644 --- a/drivers/net/nfp/flower/nfp_flower_bond.c +++ b/drivers/net/nfp/flower/nfp_flower_bond.c @@ -161,6 +161,8 @@ nfp_flower_bond_init(struct nfp_app_fw_flower *app_fw_flower) LIST_INIT(&nfp_bond->group_list); nfp_flower_bond_increment_version(nfp_bond); nfp_bond->app_fw_flower = app_fw_flower; + nfp_bond->retrans.head = 0; + nfp_bond->retrans.rear = 0; return 0; } @@ -168,6 +170,7 @@ nfp_flower_bond_init(struct nfp_app_fw_flower *app_fw_flower) void nfp_flower_bond_cleanup(struct nfp_app_fw_flower *app_fw_flower) { + struct rte_mbuf *mbuf_tmp; struct nfp_bond_group *entry; struct nfp_flower_bond *nfp_bond = app_fw_flower->nfp_bond; @@ -177,6 +180,12 @@ nfp_flower_bond_cleanup(struct nfp_app_fw_flower *app_fw_flower) LIST_REMOVE(entry, next); rte_free(entry); } + + mbuf_tmp = nfp_fl_bond_get_unprocessed(nfp_bond); + for (; mbuf_tmp != NULL; + mbuf_tmp = nfp_fl_bond_get_unprocessed(nfp_bond)) + rte_pktmbuf_free(mbuf_tmp); + pthread_mutex_unlock(&nfp_bond->mutex); pthread_mutex_destroy(&nfp_bond->mutex); @@ -587,3 +596,47 @@ nfp_flower_bond_all_member_are_phyrepr(struct rte_eth_dev *bond_dev) return true; } + +int +nfp_fl_bond_put_unprocessed(struct nfp_flower_bond *nfp_bond, + struct rte_mbuf *mbuf) +{ + uint16_t rear_real; + struct nfp_flower_bond_retrans *retrans; + struct nfp_flower_cmsg_bond_config *cmsg_payload; + + cmsg_payload = rte_pktmbuf_mtod_offset(mbuf, + struct nfp_flower_cmsg_bond_config *, + NFP_FLOWER_CMSG_HLEN); + + if (rte_be_to_cpu_32(cmsg_payload->group_id) >= NFP_FL_BOND_GROUP_MAX) + return -EINVAL; + + retrans = &nfp_bond->retrans; + rear_real = (retrans->rear + 1) % NFP_FL_BOND_RETRANS_LIMIT; + if (rear_real == retrans->head) + return -ENOSPC; + + retrans->mbufs[retrans->rear] = mbuf; + + retrans->rear = rear_real; + + return 0; +} + +struct rte_mbuf * +nfp_fl_bond_get_unprocessed(struct nfp_flower_bond *nfp_bond) +{ + struct rte_mbuf *mbuf; + struct nfp_flower_bond_retrans *retrans; + + retrans = &nfp_bond->retrans; + if (retrans->rear == retrans->head) + return NULL; + + mbuf = retrans->mbufs[retrans->head]; + + retrans->head = (retrans->head + 1) % NFP_FL_BOND_RETRANS_LIMIT; + + return mbuf; +} diff --git a/drivers/net/nfp/flower/nfp_flower_bond.h b/drivers/net/nfp/flower/nfp_flower_bond.h index f73589265f..59d1c211f2 100644 --- a/drivers/net/nfp/flower/nfp_flower_bond.h +++ b/drivers/net/nfp/flower/nfp_flower_bond.h @@ -41,6 +41,9 @@ #define NFP_FL_BOND_PORT_TX_ENABLED RTE_BIT32(1) #define NFP_FL_BOND_PORT_CHANGED RTE_BIT32(2) +/** We store 100(101-1) mbufs but queue needs 101 */ +#define NFP_FL_BOND_RETRANS_LIMIT 101 + enum nfp_flower_bond_batch { NFP_FLOWER_BOND_BATCH_FIRST, NFP_FLOWER_BOND_BATCH_MEMBER, @@ -53,6 +56,12 @@ enum nfp_flower_bond_event { NFP_FLOWER_UNREGISTER }; +struct nfp_flower_bond_retrans { + struct rte_mbuf *mbufs[NFP_FL_BOND_RETRANS_LIMIT]; + uint16_t head; + uint16_t rear; +}; + /* Control message payload for bond config */ struct nfp_flower_cmsg_bond_config { /** Configuration flags */ @@ -113,6 +122,8 @@ struct nfp_flower_bond { bool group_id_map[NFP_FL_BOND_GROUP_MAX]; /** Pointer to the flower app */ struct nfp_app_fw_flower *app_fw_flower; + /** Store bond offload packets from firmware */ + struct nfp_flower_bond_retrans retrans; }; struct nfp_flower_bond_cmsg_args { @@ -139,5 +150,8 @@ int nfp_flower_bond_event_handle(struct nfp_flower_bond *nfp_bond, enum nfp_flower_bond_event event); void nfp_flower_bond_do_work(struct nfp_flower_bond *nfp_bond); bool nfp_flower_bond_all_member_are_phyrepr(struct rte_eth_dev *bond_dev); +struct rte_mbuf *nfp_fl_bond_get_unprocessed(struct nfp_flower_bond *nfp_bond); +int nfp_fl_bond_put_unprocessed(struct nfp_flower_bond *nfp_bond, + struct rte_mbuf *mbuf); #endif /* __NFP_FLOWER_BOND_H__ */ diff --git a/drivers/net/nfp/flower/nfp_flower_ctrl.c b/drivers/net/nfp/flower/nfp_flower_ctrl.c index c25487c277..75c8c0b20a 100644 --- a/drivers/net/nfp/flower/nfp_flower_ctrl.c +++ b/drivers/net/nfp/flower/nfp_flower_ctrl.c @@ -15,6 +15,36 @@ #define MAX_PKT_BURST 32 +static void +nfp_flower_bond_recover_status(struct nfp_flower_bond *nfp_bond) +{ + int err; + struct rte_mbuf *mbuf_tmp; + struct nfp_bond_group *entry; + + pthread_mutex_lock(&nfp_bond->mutex); + + /* Clear all unprocessed messages */ + mbuf_tmp = nfp_fl_bond_get_unprocessed(nfp_bond); + for (; mbuf_tmp != NULL; + mbuf_tmp = nfp_fl_bond_get_unprocessed(nfp_bond)) + rte_pktmbuf_free(mbuf_tmp); + + /* Mark all groups dirty */ + LIST_FOREACH(entry, &nfp_bond->group_list, next) + entry->dirty = true; + + /* Reset NFP group config */ + err = nfp_flower_bond_reset(nfp_bond); + if (err != 0) + PMD_DRV_LOG(ERR, "Mem err in group reset msg"); + + pthread_mutex_unlock(&nfp_bond->mutex); + + /* Schedule a LAG/BOND config update */ + nfp_flower_bond_do_work(nfp_bond); +} + static uint16_t nfp_flower_ctrl_vnic_recv(void *rx_queue, struct rte_mbuf **rx_pkts, @@ -450,6 +480,55 @@ nfp_flower_cmsg_port_mod_rx(struct nfp_app_fw_flower *app_fw_flower, return 0; } +static bool +nfp_flower_bond_unprocessed_msg(void *app_flower, + struct rte_mbuf *mbuf) +{ + uint16_t cnt; + uint8_t flags; + bool store_tag = false; + struct rte_mbuf *mbuf_tmp; + struct nfp_flower_bond *nfp_bond; + struct nfp_flower_cmsg_bond_config *cmsg_payload; + struct nfp_app_fw_flower *app_fw_flower = app_flower; + + cmsg_payload = rte_pktmbuf_mtod_offset(mbuf, + struct nfp_flower_cmsg_bond_config *, + NFP_FLOWER_CMSG_HLEN); + + flags = cmsg_payload->ctrl_flags; + + /* + * Note the intentional fall through below. If DATA and XON are both + * set, the message will stored and sent again with the rest of the + * unprocessed messages list. + */ + nfp_bond = app_fw_flower->nfp_bond; + + /* Store */ + if ((flags & NFP_FL_BOND_DATA) != 0) { + if (nfp_fl_bond_put_unprocessed(nfp_bond, mbuf) == 0) + store_tag = true; + } + + /* Send stored */ + if ((flags & NFP_FL_BOND_XON) != 0) { + mbuf_tmp = nfp_fl_bond_get_unprocessed(nfp_bond); + for (; mbuf_tmp != NULL; + mbuf_tmp = nfp_fl_bond_get_unprocessed(nfp_bond)) { + cnt = nfp_flower_ctrl_vnic_xmit(app_fw_flower, mbuf_tmp); + if (cnt == 0) + rte_pktmbuf_free(mbuf_tmp); + } + } + + /* Resend all */ + if ((flags & NFP_FL_BOND_SYNC) != 0) + nfp_flower_bond_recover_status(nfp_bond); + + return store_tag; +} + static void nfp_flower_cmsg_rx(struct nfp_app_fw_flower *app_fw_flower, struct rte_mbuf **pkts_burst, @@ -457,6 +536,7 @@ nfp_flower_cmsg_rx(struct nfp_app_fw_flower *app_fw_flower, { uint16_t i; char *meta; + bool mbuf_stored; uint32_t meta_type; uint32_t meta_info; struct nfp_mtr_priv *mtr_priv; @@ -486,18 +566,34 @@ nfp_flower_cmsg_rx(struct nfp_app_fw_flower *app_fw_flower, continue; } - if (cmsg_hdr->type == NFP_FLOWER_CMSG_TYPE_FLOW_STATS) { + mbuf_stored = false; + + switch (cmsg_hdr->type) { + case NFP_FLOWER_CMSG_TYPE_FLOW_STATS: /* We need to deal with stats updates from HW asap */ nfp_flower_cmsg_rx_stats(flow_priv, pkts_burst[i]); - } else if (cmsg_hdr->type == NFP_FLOWER_CMSG_TYPE_QOS_STATS) { + break; + case NFP_FLOWER_CMSG_TYPE_QOS_STATS: /* Handle meter stats */ nfp_flower_cmsg_rx_qos_stats(mtr_priv, pkts_burst[i]); - } else if (cmsg_hdr->type == NFP_FLOWER_CMSG_TYPE_PORT_MOD) { - /* Handle changes to port configuration/status */ + break; + case NFP_FLOWER_CMSG_TYPE_PORT_MOD: + /* Handle meter stats */ nfp_flower_cmsg_port_mod_rx(app_fw_flower, pkts_burst[i]); + break; + case NFP_FLOWER_CMSG_TYPE_LAG_CONFIG: + /* Handle LAG/BOND related packets */ + if (nfp_flower_support_bond_offload(app_fw_flower)) + mbuf_stored = nfp_flower_bond_unprocessed_msg(app_fw_flower, + pkts_burst[i]); + break; + default: + PMD_DRV_LOG(INFO, "Unmatched repr message type: %u", + cmsg_hdr->type); } - rte_pktmbuf_free(pkts_burst[i]); + if (!mbuf_stored) + rte_pktmbuf_free(pkts_burst[i]); } } -- 2.39.1 ^ permalink raw reply [flat|nested] 80+ messages in thread
* [PATCH v5 13/14] net/nfp: support getting bond firmware creation 2023-12-26 2:37 ` [PATCH v5 00/14] " Chaoyong He ` (11 preceding siblings ...) 2023-12-26 2:37 ` [PATCH v5 12/14] net/nfp: handle bond packets from firmware Chaoyong He @ 2023-12-26 2:37 ` Chaoyong He 2023-12-26 2:37 ` [PATCH v5 14/14] net/nfp: support offloading bond-flow Chaoyong He 2023-12-26 7:28 ` [PATCH v6 00/14] Enhance the bond framework to support offload Chaoyong He 14 siblings, 0 replies; 80+ messages in thread From: Chaoyong He @ 2023-12-26 2:37 UTC (permalink / raw) To: dev; +Cc: oss-drivers, Long Wu, Peng Zhang, Chaoyong He From: Long Wu <long.wu@corigine.com> Support getting the status that whether the bonding port is created by the nfp firmware. Signed-off-by: Long Wu <long.wu@corigine.com> Reviewed-by: Peng Zhang <peng.zhang@corigine.com> Reviewed-by: Chaoyong He <chaoyong.he@corigine.com> --- drivers/net/nfp/flower/nfp_flower_bond.c | 27 +++++++++++++++++++ drivers/net/nfp/flower/nfp_flower_bond.h | 2 ++ .../net/nfp/flower/nfp_flower_representor.c | 1 + 3 files changed, 30 insertions(+) diff --git a/drivers/net/nfp/flower/nfp_flower_bond.c b/drivers/net/nfp/flower/nfp_flower_bond.c index 73b616360a..c809465ffc 100644 --- a/drivers/net/nfp/flower/nfp_flower_bond.c +++ b/drivers/net/nfp/flower/nfp_flower_bond.c @@ -640,3 +640,30 @@ nfp_fl_bond_get_unprocessed(struct nfp_flower_bond *nfp_bond) return mbuf; } + +int +nfp_flower_bond_fw_create_get(struct rte_eth_dev *bond_dev, + struct rte_eth_dev *nfp_dev) +{ + int ret = -EINVAL; + struct nfp_bond_group *group; + struct nfp_flower_bond *nfp_bond; + struct nfp_flower_representor *repr; + + if (!nfp_flower_bond_is_member_port(bond_dev, nfp_dev)) + return ret; + + repr = nfp_dev->data->dev_private; + nfp_bond = repr->app_fw_flower->nfp_bond; + + pthread_mutex_lock(&nfp_bond->mutex); + LIST_FOREACH(group, &nfp_bond->group_list, next) { + if (group->main_dev == bond_dev && group->offloaded == true) { + ret = 0; + break; + } + } + pthread_mutex_unlock(&nfp_bond->mutex); + + return ret; +} diff --git a/drivers/net/nfp/flower/nfp_flower_bond.h b/drivers/net/nfp/flower/nfp_flower_bond.h index 59d1c211f2..91d7d88b8c 100644 --- a/drivers/net/nfp/flower/nfp_flower_bond.h +++ b/drivers/net/nfp/flower/nfp_flower_bond.h @@ -153,5 +153,7 @@ bool nfp_flower_bond_all_member_are_phyrepr(struct rte_eth_dev *bond_dev); struct rte_mbuf *nfp_fl_bond_get_unprocessed(struct nfp_flower_bond *nfp_bond); int nfp_fl_bond_put_unprocessed(struct nfp_flower_bond *nfp_bond, struct rte_mbuf *mbuf); +int nfp_flower_bond_fw_create_get(struct rte_eth_dev *bond_dev, + struct rte_eth_dev *nfp_dev); #endif /* __NFP_FLOWER_BOND_H__ */ diff --git a/drivers/net/nfp/flower/nfp_flower_representor.c b/drivers/net/nfp/flower/nfp_flower_representor.c index 43106da90e..8a1b2eff66 100644 --- a/drivers/net/nfp/flower/nfp_flower_representor.c +++ b/drivers/net/nfp/flower/nfp_flower_representor.c @@ -458,6 +458,7 @@ static const struct eth_dev_ops nfp_flower_repr_dev_ops = { .mtr_ops_get = nfp_net_mtr_ops_get, .bond_notify_member = nfp_flower_repr_bond_notify_member, + .bond_fw_create_get = nfp_flower_bond_fw_create_get, }; static uint32_t -- 2.39.1 ^ permalink raw reply [flat|nested] 80+ messages in thread
* [PATCH v5 14/14] net/nfp: support offloading bond-flow 2023-12-26 2:37 ` [PATCH v5 00/14] " Chaoyong He ` (12 preceding siblings ...) 2023-12-26 2:37 ` [PATCH v5 13/14] net/nfp: support getting bond firmware creation Chaoyong He @ 2023-12-26 2:37 ` Chaoyong He 2023-12-26 7:28 ` [PATCH v6 00/14] Enhance the bond framework to support offload Chaoyong He 14 siblings, 0 replies; 80+ messages in thread From: Chaoyong He @ 2023-12-26 2:37 UTC (permalink / raw) To: dev; +Cc: oss-drivers, Long Wu, Peng Zhang, Chaoyong He From: Long Wu <long.wu@corigine.com> After bonding port notification and firmware created the bonding port, NFP rte_flow rule adds support for destination port to bonding port. Signed-off-by: Long Wu <long.wu@corigine.com> Reviewed-by: Peng Zhang <peng.zhang@corigine.com> Reviewed-by: Chaoyong He <chaoyong.he@corigine.com> --- drivers/net/nfp/flower/nfp_flower_bond.c | 22 +++ drivers/net/nfp/flower/nfp_flower_bond.h | 6 + drivers/net/nfp/flower/nfp_flower_cmsg.h | 9 + drivers/net/nfp/flower/nfp_flower_flow.c | 232 +++++++++++++++++++++-- 4 files changed, 252 insertions(+), 17 deletions(-) diff --git a/drivers/net/nfp/flower/nfp_flower_bond.c b/drivers/net/nfp/flower/nfp_flower_bond.c index c809465ffc..6e4d60659a 100644 --- a/drivers/net/nfp/flower/nfp_flower_bond.c +++ b/drivers/net/nfp/flower/nfp_flower_bond.c @@ -667,3 +667,25 @@ nfp_flower_bond_fw_create_get(struct rte_eth_dev *bond_dev, return ret; } + +int +nfp_flower_bond_get_output_id(struct nfp_flower_bond *nfp_bond, + const struct rte_eth_dev *bond_dev, + uint32_t *group_id) +{ + struct nfp_bond_group *group; + + pthread_mutex_lock(&nfp_bond->mutex); + + group = nfp_flower_bond_find_group(nfp_bond, bond_dev); + if (group == NULL) { + pthread_mutex_unlock(&nfp_bond->mutex); + return -ENOENT; + } + + *group_id = group->group_id; + + pthread_mutex_unlock(&nfp_bond->mutex); + + return 0; +} diff --git a/drivers/net/nfp/flower/nfp_flower_bond.h b/drivers/net/nfp/flower/nfp_flower_bond.h index 91d7d88b8c..6debe32549 100644 --- a/drivers/net/nfp/flower/nfp_flower_bond.h +++ b/drivers/net/nfp/flower/nfp_flower_bond.h @@ -44,6 +44,9 @@ /** We store 100(101-1) mbufs but queue needs 101 */ #define NFP_FL_BOND_RETRANS_LIMIT 101 +/* Flow output action of bonding port */ +#define NFP_FL_BOND_OUT 0xC0DE0000 + enum nfp_flower_bond_batch { NFP_FLOWER_BOND_BATCH_FIRST, NFP_FLOWER_BOND_BATCH_MEMBER, @@ -155,5 +158,8 @@ int nfp_fl_bond_put_unprocessed(struct nfp_flower_bond *nfp_bond, struct rte_mbuf *mbuf); int nfp_flower_bond_fw_create_get(struct rte_eth_dev *bond_dev, struct rte_eth_dev *nfp_dev); +int nfp_flower_bond_get_output_id(struct nfp_flower_bond *nfp_bond, + const struct rte_eth_dev *bond_dev, + uint32_t *group_id); #endif /* __NFP_FLOWER_BOND_H__ */ diff --git a/drivers/net/nfp/flower/nfp_flower_cmsg.h b/drivers/net/nfp/flower/nfp_flower_cmsg.h index 60ab58a3b1..17ef943855 100644 --- a/drivers/net/nfp/flower/nfp_flower_cmsg.h +++ b/drivers/net/nfp/flower/nfp_flower_cmsg.h @@ -712,6 +712,15 @@ struct nfp_fl_act_head { uint8_t len_lw; }; +#define NFP_FL_PRE_BOND_VER_OFF 8 + +struct nfp_fl_pre_bond { + struct nfp_fl_act_head head; + rte_be16_t group_id; + uint8_t lag_version[3]; + uint8_t instance; +}; + struct nfp_fl_act_output { struct nfp_fl_act_head head; rte_be16_t flags; diff --git a/drivers/net/nfp/flower/nfp_flower_flow.c b/drivers/net/nfp/flower/nfp_flower_flow.c index e26be30d18..6e21ad8c26 100644 --- a/drivers/net/nfp/flower/nfp_flower_flow.c +++ b/drivers/net/nfp/flower/nfp_flower_flow.c @@ -5,12 +5,14 @@ #include "nfp_flower_flow.h" +#include <rte_eth_bond.h> #include <rte_flow_driver.h> #include <rte_hash.h> #include <rte_jhash.h> #include <rte_malloc.h> #include "flower/nfp_conntrack.h" +#include "flower/nfp_flower_bond.h" #include "flower/nfp_flower_representor.h" #include "nfpcore/nfp_rtsym.h" #include "nfp_logs.h" @@ -969,9 +971,23 @@ nfp_flow_key_layers_calculate_items(const struct rte_flow_item items[], return 0; } +static bool +nfp_flow_check_bond_action(struct nfp_flower_representor *representor, + uint32_t port_id) +{ + struct rte_eth_dev *bond_dev = &rte_eth_devices[port_id]; + + if (nfp_flower_support_bond_offload(representor->app_fw_flower) || + !nfp_flower_bond_all_member_are_phyrepr(bond_dev)) + return false; + + return true; +} + static int nfp_flow_key_layers_calculate_actions(const struct rte_flow_action actions[], - struct nfp_fl_key_ls *key_ls) + struct nfp_fl_key_ls *key_ls, + struct nfp_flower_representor *representor) { int ret = 0; bool meter_flag = false; @@ -981,6 +997,8 @@ nfp_flow_key_layers_calculate_actions(const struct rte_flow_action actions[], bool mac_set_flag = false; bool ttl_tos_flag = false; const struct rte_flow_action *action; + const struct rte_flow_action_port_id *port_id; + const struct rte_flow_action_ethdev *action_ethdev; for (action = actions; action->type != RTE_FLOW_ACTION_TYPE_END; ++action) { /* Make sure actions length no longer than NFP_FL_MAX_A_SIZ */ @@ -1006,10 +1024,36 @@ nfp_flow_key_layers_calculate_actions(const struct rte_flow_action actions[], case RTE_FLOW_ACTION_TYPE_PORT_ID: PMD_DRV_LOG(DEBUG, "RTE_FLOW_ACTION_TYPE_PORT_ID detected"); key_ls->act_size += sizeof(struct nfp_fl_act_output); + + port_id = action->conf; + if (port_id == NULL) + return -ERANGE; + + if (!rte_eth_bond_is_valid_port(port_id->id)) + break; + + if (!nfp_flow_check_bond_action(representor, port_id->id)) + return -ENOTSUP; + + /* We only add when all checks have passed */ + key_ls->act_size += sizeof(struct nfp_fl_pre_bond); break; case RTE_FLOW_ACTION_TYPE_REPRESENTED_PORT: PMD_DRV_LOG(DEBUG, "RTE_FLOW_ACTION_TYPE_REPRESENTED_PORT detected"); key_ls->act_size += sizeof(struct nfp_fl_act_output); + + action_ethdev = action->conf; + if (action_ethdev == NULL) + return -ERANGE; + + if (!rte_eth_bond_is_valid_port(action_ethdev->port_id)) + break; + + if (!nfp_flow_check_bond_action(representor, action_ethdev->port_id)) + return -ENOTSUP; + + /* We only add when all checks have passed */ + key_ls->act_size += sizeof(struct nfp_fl_pre_bond); break; case RTE_FLOW_ACTION_TYPE_SET_MAC_SRC: PMD_DRV_LOG(DEBUG, "RTE_FLOW_ACTION_TYPE_SET_MAC_SRC detected"); @@ -1150,7 +1194,8 @@ nfp_flow_key_layers_calculate_actions(const struct rte_flow_action actions[], static int nfp_flow_key_layers_calculate(const struct rte_flow_item items[], const struct rte_flow_action actions[], - struct nfp_fl_key_ls *key_ls) + struct nfp_fl_key_ls *key_ls, + struct nfp_flower_representor *representor) { int ret = 0; @@ -1164,7 +1209,7 @@ nfp_flow_key_layers_calculate(const struct rte_flow_item items[], key_ls->tun_type = NFP_FL_TUN_NONE; ret |= nfp_flow_key_layers_calculate_items(items, key_ls); - ret |= nfp_flow_key_layers_calculate_actions(actions, key_ls); + ret |= nfp_flow_key_layers_calculate_actions(actions, key_ls, representor); return ret; } @@ -2160,12 +2205,45 @@ nfp_flow_compile_items(struct nfp_flower_representor *representor, return 0; } +static int +nfp_flower_bond_populate_pre_action(struct nfp_flower_representor *repr, + struct rte_eth_dev *bond_dev, + struct nfp_fl_pre_bond *pre_bond) +{ + rte_be32_t version; + struct nfp_bond_group *group; + struct nfp_flower_bond *nfp_bond; + + nfp_bond = repr->app_fw_flower->nfp_bond; + + pthread_mutex_lock(&nfp_bond->mutex); + + group = nfp_flower_bond_find_group(nfp_bond, bond_dev); + if (group == NULL) { + pthread_mutex_unlock(&nfp_bond->mutex); + PMD_DRV_LOG(ERR, "Invalid entry: group does not exist for bond action"); + return -ENOENT; + } + + pre_bond->group_id = rte_cpu_to_be_16(group->group_id); + version = rte_cpu_to_be_32(nfp_bond->batch_ver << NFP_FL_PRE_BOND_VER_OFF); + memcpy(pre_bond->lag_version, &version, 3); + pre_bond->instance = group->group_inst; + + pthread_mutex_unlock(&nfp_bond->mutex); + + return 0; +} + static int nfp_flow_action_output(char *act_data, const struct rte_flow_action *action, struct nfp_fl_rule_metadata *nfp_flow_meta, - uint32_t output_cnt) + uint32_t output_cnt, + struct nfp_flower_bond *nfp_bond, + bool *bonding_port) { + uint32_t gid; size_t act_size; struct rte_eth_dev *ethdev; struct nfp_fl_act_output *output; @@ -2177,18 +2255,31 @@ nfp_flow_action_output(char *act_data, return -ERANGE; ethdev = &rte_eth_devices[port_id->id]; - representor = ethdev->data->dev_private; + output = (struct nfp_fl_act_output *)act_data; + if (rte_eth_bond_is_valid_port(port_id->id)) { + /* Output port is bonding port */ + if (nfp_flower_bond_get_output_id(nfp_bond, ethdev, &gid) != 0) { + PMD_DRV_LOG(ERR, "Cannot find group id for bond action"); + return -EINVAL; + } + + output->port = rte_cpu_to_be_32(NFP_FL_BOND_OUT | gid); + nfp_flow_meta->shortcut = rte_cpu_to_be_32(NFP_FL_SC_ACT_NULL); + *bonding_port = true; + } else { + representor = ethdev->data->dev_private; + + output->port = rte_cpu_to_be_32(representor->port_id); + nfp_flow_meta->shortcut = rte_cpu_to_be_32(representor->port_id); + } + act_size = sizeof(struct nfp_fl_act_output); - output = (struct nfp_fl_act_output *)act_data; output->head.jump_id = NFP_FL_ACTION_OPCODE_OUTPUT; output->head.len_lw = act_size >> NFP_FL_LW_SIZ; - output->port = rte_cpu_to_be_32(representor->port_id); if (output_cnt == 0) output->flags = rte_cpu_to_be_16(NFP_FL_OUT_FLAGS_LAST); - nfp_flow_meta->shortcut = rte_cpu_to_be_32(representor->port_id); - return 0; } @@ -2196,8 +2287,11 @@ static int nfp_flow_action_output_stage(char *act_data, const struct rte_flow_action *action, struct nfp_fl_rule_metadata *nfp_flow_meta, - uint32_t output_cnt) + uint32_t output_cnt, + struct nfp_flower_bond *nfp_bond, + bool *bonding_port) { + uint32_t gid; size_t act_size; struct rte_eth_dev *ethdev; struct nfp_fl_act_output *output; @@ -2209,17 +2303,70 @@ nfp_flow_action_output_stage(char *act_data, return -ERANGE; ethdev = &rte_eth_devices[action_ethdev->port_id]; - representor = ethdev->data->dev_private; + output = (struct nfp_fl_act_output *)act_data; + if (rte_eth_bond_is_valid_port(action_ethdev->port_id)) { + /* Output port is bonding port */ + if (nfp_flower_bond_get_output_id(nfp_bond, ethdev, &gid) != 0) { + PMD_DRV_LOG(ERR, "Cannot find group id for bond action"); + return -EINVAL; + } + + output->port = rte_cpu_to_be_32(NFP_FL_BOND_OUT | gid); + nfp_flow_meta->shortcut = rte_cpu_to_be_32(NFP_FL_SC_ACT_NULL); + *bonding_port = true; + } else { + representor = ethdev->data->dev_private; + + output->port = rte_cpu_to_be_32(representor->port_id); + nfp_flow_meta->shortcut = rte_cpu_to_be_32(representor->port_id); + } + act_size = sizeof(struct nfp_fl_act_output); - output = (struct nfp_fl_act_output *)act_data; output->head.jump_id = NFP_FL_ACTION_OPCODE_OUTPUT; output->head.len_lw = act_size >> NFP_FL_LW_SIZ; - output->port = rte_cpu_to_be_32(representor->port_id); if (output_cnt == 0) output->flags = rte_cpu_to_be_16(NFP_FL_OUT_FLAGS_LAST); - nfp_flow_meta->shortcut = rte_cpu_to_be_32(representor->port_id); + return 0; +} + +static int +nfp_flow_pre_bond(struct nfp_flower_representor *representor, + char *act_data, + char *actions, + const struct rte_flow_action *action) +{ + int err; + int act_len; + uint32_t port_id; + struct rte_eth_dev *ethdev; + struct nfp_fl_pre_bond *pre_bond; + uint32_t pre_bond_size = sizeof(struct nfp_fl_pre_bond); + + act_len = act_data - actions; + + if (act_len + pre_bond_size > NFP_FL_MAX_A_SIZ) { + PMD_DRV_LOG(ERR, "Maximum allowed action list size exceeded at bond action"); + return -EOPNOTSUPP; + } + + if (act_len != 0) + memmove(actions + pre_bond_size, actions, act_len); + + if (action->type == RTE_FLOW_ACTION_TYPE_REPRESENTED_PORT) + port_id = ((const struct rte_flow_action_ethdev *)action->conf)->port_id; + else + port_id = ((const struct rte_flow_action_port_id *)action->conf)->id; + + ethdev = &rte_eth_devices[port_id]; + pre_bond = (struct nfp_fl_pre_bond *)actions; + err = nfp_flower_bond_populate_pre_action(representor, ethdev, pre_bond); + if (err != 0) + return err; + + pre_bond->head.jump_id = NFP_FL_ACTION_OPCODE_PRE_LAG; + pre_bond->head.len_lw = pre_bond_size >> NFP_FL_LW_SIZ; return 0; } @@ -3502,6 +3649,13 @@ nfp_flow_count_output(const struct rte_flow_action actions[]) return count; } +static bool +nfp_flow_check_action_first(const struct rte_flow_action actions[], + uint32_t count) +{ + return ((nfp_flow_count_output(actions) - count) == 1); +} + static int nfp_flow_compile_action(struct nfp_flower_representor *representor, const struct rte_flow_action actions[], @@ -3517,7 +3671,9 @@ nfp_flow_compile_action(struct nfp_flower_representor *representor, bool tp_set_flag = false; bool mac_set_flag = false; bool ttl_tos_flag = false; + bool bonding_port = false; uint32_t total_actions = 0; + struct nfp_flower_bond *nfp_bond; const struct rte_flow_action *action; struct nfp_flower_meta_tci *meta_tci; struct nfp_fl_rule_metadata *nfp_flow_meta; @@ -3546,7 +3702,10 @@ nfp_flow_compile_action(struct nfp_flower_representor *representor, case RTE_FLOW_ACTION_TYPE_REPRESENTED_PORT: PMD_DRV_LOG(DEBUG, "Process RTE_FLOW_ACTION_TYPE_REPRESENTED_PORT"); count--; - ret = nfp_flow_action_output_stage(position, action, nfp_flow_meta, count); + + nfp_bond = representor->app_fw_flower->nfp_bond; + ret = nfp_flow_action_output_stage(position, action, + nfp_flow_meta, count, nfp_bond, &bonding_port); if (ret != 0) { PMD_DRV_LOG(ERR, "Failed when process" " RTE_FLOW_ACTION_TYPE_REPRESENTED_PORT"); @@ -3554,11 +3713,32 @@ nfp_flow_compile_action(struct nfp_flower_representor *representor, } position += sizeof(struct nfp_fl_act_output); + + /* Output port is not a bonding port */ + if (!bonding_port) + break; + + if (!nfp_flow_check_action_first(actions, count)) { + PMD_DRV_LOG(ERR, "Bond action should be first"); + return -EOPNOTSUPP; + } + + ret = nfp_flow_pre_bond(representor, position, + action_data, action); + if (ret != 0) { + PMD_DRV_LOG(ERR, "Bond action special move failed"); + return ret; + } + + position += sizeof(struct nfp_fl_pre_bond); break; case RTE_FLOW_ACTION_TYPE_PORT_ID: PMD_DRV_LOG(DEBUG, "Process RTE_FLOW_ACTION_TYPE_PORT_ID"); count--; - ret = nfp_flow_action_output(position, action, nfp_flow_meta, count); + + nfp_bond = representor->app_fw_flower->nfp_bond; + ret = nfp_flow_action_output(position, action, + nfp_flow_meta, count, nfp_bond, &bonding_port); if (ret != 0) { PMD_DRV_LOG(ERR, "Failed when process" " RTE_FLOW_ACTION_TYPE_PORT_ID"); @@ -3566,6 +3746,24 @@ nfp_flow_compile_action(struct nfp_flower_representor *representor, } position += sizeof(struct nfp_fl_act_output); + + /* Output port is not a bonding port */ + if (!bonding_port) + break; + + if (!nfp_flow_check_action_first(actions, count)) { + PMD_DRV_LOG(ERR, "Bond action should be first"); + return -EOPNOTSUPP; + } + + ret = nfp_flow_pre_bond(representor, position, + action_data, action); + if (ret != 0) { + PMD_DRV_LOG(ERR, "Bond action special move failed"); + return ret; + } + + position += sizeof(struct nfp_fl_pre_bond); break; case RTE_FLOW_ACTION_TYPE_SET_MAC_SRC: PMD_DRV_LOG(DEBUG, "Process RTE_FLOW_ACTION_TYPE_SET_MAC_SRC"); @@ -3770,7 +3968,7 @@ nfp_flow_process(struct nfp_flower_representor *representor, struct nfp_fl_key_ls key_layer; struct nfp_fl_rule_metadata *nfp_flow_meta; - ret = nfp_flow_key_layers_calculate(items, actions, &key_layer); + ret = nfp_flow_key_layers_calculate(items, actions, &key_layer, representor); if (ret != 0) { PMD_DRV_LOG(ERR, "Key layers calculate failed."); return NULL; -- 2.39.1 ^ permalink raw reply [flat|nested] 80+ messages in thread
* [PATCH v6 00/14] Enhance the bond framework to support offload 2023-12-26 2:37 ` [PATCH v5 00/14] " Chaoyong He ` (13 preceding siblings ...) 2023-12-26 2:37 ` [PATCH v5 14/14] net/nfp: support offloading bond-flow Chaoyong He @ 2023-12-26 7:28 ` Chaoyong He 2023-12-26 7:28 ` [PATCH v6 01/14] ethdev: add member notification for bonding port Chaoyong He ` (14 more replies) 14 siblings, 15 replies; 80+ messages in thread From: Chaoyong He @ 2023-12-26 7:28 UTC (permalink / raw) To: dev; +Cc: oss-drivers, Chaoyong He This patch series try to enhance the bond framework to support the offload feature better: * Add new API to make the member port can access some information of the bond port which belongs. * Add new API to get the result of whether bond port is created by the member port. * Add two command line argument to control if enable member port notification and dedicated queue features. * Add logic to support add ports which share the same PCI address into bond port. * Also modify the testpmd application to test the new APIs and logics added by this patch series. --- v2: * Fix compile error on github-robot by removing the redundancy function declaration in the header file. v3: * Use the hole in the structure for the new added flag data field. v4: * Drop two commits not necessary for this series. * Modify some logic as the review comments from reviewers. v5: * Add a new 'rte_eth_bond_flow.h' header file. * Add the patches of NFP PMD as the example of support bond flow offload. v6: * Try to solve the CI build error. --- Long Wu (14): ethdev: add member notification for bonding port ethdev: add API to get firmware creation of bonding port net/bonding: add bonding port arguments net/bonding: support add port by data name net/bonding: support checking valid bonding port ID net/bonding: add commands for bonding port notification net/bonding: create new rte flow header file net/nfp: add bond firmware creation initialization net/nfp: reset bond configuration of firmware net/nfp: handle link event of bond firmware creation net/nfp: support bond member notification net/nfp: handle bond packets from firmware net/nfp: support getting bond firmware creation net/nfp: support offloading bond-flow .../link_bonding_poll_mode_drv_lib.rst | 18 + drivers/net/bonding/bonding_testpmd.c | 128 ++++ drivers/net/bonding/eth_bond_private.h | 24 +- drivers/net/bonding/rte_eth_bond.h | 88 +++ drivers/net/bonding/rte_eth_bond_api.c | 122 ++++ drivers/net/bonding/rte_eth_bond_args.c | 47 ++ drivers/net/bonding/rte_eth_bond_flow.c | 1 + drivers/net/bonding/rte_eth_bond_flow.h | 22 + drivers/net/bonding/rte_eth_bond_pmd.c | 94 ++- drivers/net/bonding/version.map | 5 + drivers/net/nfp/flower/nfp_flower.c | 87 ++- drivers/net/nfp/flower/nfp_flower.h | 13 + drivers/net/nfp/flower/nfp_flower_bond.c | 691 ++++++++++++++++++ drivers/net/nfp/flower/nfp_flower_bond.h | 165 +++++ drivers/net/nfp/flower/nfp_flower_cmsg.c | 35 + drivers/net/nfp/flower/nfp_flower_cmsg.h | 12 + drivers/net/nfp/flower/nfp_flower_ctrl.c | 106 ++- drivers/net/nfp/flower/nfp_flower_flow.c | 232 +++++- .../net/nfp/flower/nfp_flower_representor.c | 35 + .../net/nfp/flower/nfp_flower_representor.h | 2 + drivers/net/nfp/meson.build | 3 +- drivers/net/nfp/nfp_net_common.c | 25 + drivers/net/nfp/nfp_net_common.h | 1 - lib/ethdev/ethdev_driver.h | 38 + 24 files changed, 1951 insertions(+), 43 deletions(-) create mode 100644 drivers/net/bonding/rte_eth_bond_flow.h create mode 100644 drivers/net/nfp/flower/nfp_flower_bond.c create mode 100644 drivers/net/nfp/flower/nfp_flower_bond.h -- 2.39.1 ^ permalink raw reply [flat|nested] 80+ messages in thread
* [PATCH v6 01/14] ethdev: add member notification for bonding port 2023-12-26 7:28 ` [PATCH v6 00/14] Enhance the bond framework to support offload Chaoyong He @ 2023-12-26 7:28 ` Chaoyong He 2023-12-26 7:28 ` [PATCH v6 02/14] ethdev: add API to get firmware creation of " Chaoyong He ` (13 subsequent siblings) 14 siblings, 0 replies; 80+ messages in thread From: Chaoyong He @ 2023-12-26 7:28 UTC (permalink / raw) To: dev; +Cc: oss-drivers, Long Wu, James Hershaw, Chaoyong He, Peng Zhang From: Long Wu <long.wu@corigine.com> Bonding PMD does not let member ports know the bonding port's information, like how many member ports the bonding port has, what mode the bonding port is in and so on. Add the notification interface for bonding port to let member port know it is added to a bonding port and what the bonding port's configuration is. If so the member ports have chance to offload bond-flow that its destination port is a bonding port. Signed-off-by: Long Wu <long.wu@corigine.com> Reviewed-by: James Hershaw <james.hershaw@corigine.com> Reviewed-by: Chaoyong He <chaoyong.he@corigine.com> Reviewed-by: Peng Zhang <peng.zhang@corigine.com> --- drivers/net/bonding/eth_bond_private.h | 1 + drivers/net/bonding/rte_eth_bond.h | 46 ++++++++++++++++ drivers/net/bonding/rte_eth_bond_api.c | 72 ++++++++++++++++++++++++++ drivers/net/bonding/rte_eth_bond_pmd.c | 32 ++++++++++-- drivers/net/bonding/version.map | 3 ++ lib/ethdev/ethdev_driver.h | 18 +++++++ 6 files changed, 169 insertions(+), 3 deletions(-) diff --git a/drivers/net/bonding/eth_bond_private.h b/drivers/net/bonding/eth_bond_private.h index e688894210..f69e85c199 100644 --- a/drivers/net/bonding/eth_bond_private.h +++ b/drivers/net/bonding/eth_bond_private.h @@ -180,6 +180,7 @@ struct bond_dev_private { uint8_t member_update_idx; bool kvargs_processing_is_done; + bool notify_member; /**< Enable member notification of bonding port. */ uint32_t candidate_max_rx_pktlen; uint32_t max_rx_pktlen; diff --git a/drivers/net/bonding/rte_eth_bond.h b/drivers/net/bonding/rte_eth_bond.h index f10165f2c6..f6c773615c 100644 --- a/drivers/net/bonding/rte_eth_bond.h +++ b/drivers/net/bonding/rte_eth_bond.h @@ -351,6 +351,52 @@ rte_eth_bond_link_up_prop_delay_set(uint16_t bonding_port_id, int rte_eth_bond_link_up_prop_delay_get(uint16_t bonding_port_id); +/** + * Set the flag of whether bonding port notifies member ports. + * + * @param bonding_port_id + * Port ID of bonding device. + * @param notify + * Flag of whether bonding port notifies member ports. + * + * @return + * 0 on success, negative value otherwise. + */ +__rte_experimental +int +rte_eth_bond_notify_member_flag_set(uint16_t bonding_port_id, bool notify); + +/** + * Get the flag of whether bonding port notifies member ports. + * + * @param bonding_port_id + * Port ID of bonding device. + * @param notify + * Flag of whether bonding port notifies member ports. + * + * @return + * 0 on success, negative value otherwise. + */ +__rte_experimental +int +rte_eth_bond_notify_member_flag_get(uint16_t bonding_port_id, bool *notify); + +/** + * Notify the member ports of bonding port's information. + * + * This interface is called in the following functions: + * - bond_ethdev_lsc_event_callback() + * - bond_ethdev_configure() + * + * @param bonding_port_id + * Port ID of bonding device. + * + * @return + * 0 on success, negative value otherwise. + */ +__rte_experimental +int +rte_eth_bond_notify_members(uint16_t bonding_port_id); #ifdef __cplusplus } diff --git a/drivers/net/bonding/rte_eth_bond_api.c b/drivers/net/bonding/rte_eth_bond_api.c index 99e496556a..239f86ee92 100644 --- a/drivers/net/bonding/rte_eth_bond_api.c +++ b/drivers/net/bonding/rte_eth_bond_api.c @@ -627,6 +627,17 @@ __eth_bond_member_add_lock_free(uint16_t bonding_port_id, uint16_t member_port_i member_vlan_filter_set(bonding_port_id, member_port_id); + if (internals->notify_member && + *member_eth_dev->dev_ops->bond_notify_member != NULL) { + ret = member_eth_dev->dev_ops->bond_notify_member(member_eth_dev, + bonding_eth_dev); + if (ret < 0) { + RTE_BOND_LOG(ERR, "Add member (port %u) notify failed!", + member_port_id); + return -1; + } + } + return 0; } @@ -733,6 +744,10 @@ __eth_bond_member_remove_lock_free(uint16_t bonding_port_id, member_eth_dev = &rte_eth_devices[member_port_id]; member_remove(internals, member_eth_dev); member_eth_dev->data->dev_flags &= (~RTE_ETH_DEV_BONDING_MEMBER); + if (internals->notify_member && + *member_eth_dev->dev_ops->bond_notify_member != NULL) + member_eth_dev->dev_ops->bond_notify_member(member_eth_dev, + bonding_eth_dev); /* first member in the active list will be the primary by default, * otherwise use first device in list */ @@ -1098,3 +1113,60 @@ rte_eth_bond_link_up_prop_delay_get(uint16_t bonding_port_id) return internals->link_up_delay_ms; } + +int +rte_eth_bond_notify_member_flag_set(uint16_t bonding_port_id, bool notify) +{ + struct bond_dev_private *internals; + + if (valid_bonding_port_id(bonding_port_id) != 0) + return -EINVAL; + + internals = rte_eth_devices[bonding_port_id].data->dev_private; + + internals->notify_member = notify; + + return 0; +} + +int +rte_eth_bond_notify_member_flag_get(uint16_t bonding_port_id, bool *notify) +{ + struct bond_dev_private *internals; + + if (valid_bonding_port_id(bonding_port_id) != 0) + return -EINVAL; + + internals = rte_eth_devices[bonding_port_id].data->dev_private; + + *notify = internals->notify_member; + + return 0; +} + +int +rte_eth_bond_notify_members(uint16_t bonding_port_id) +{ + uint32_t i; + uint16_t member_port_id; + struct rte_eth_dev *bond_dev; + struct rte_eth_dev *member_dev; + struct bond_dev_private *internals; + + if (valid_bonding_port_id(bonding_port_id) != 0) + return -EINVAL; + + bond_dev = &rte_eth_devices[bonding_port_id]; + internals = bond_dev->data->dev_private; + + for (i = 0; i < internals->member_count; i++) { + member_port_id = internals->members[i].port_id; + member_dev = &rte_eth_devices[member_port_id]; + /* Notify member port if it supports. */ + if (*member_dev->dev_ops->bond_notify_member != NULL) + member_dev->dev_ops->bond_notify_member(member_dev, + bond_dev); + } + + return 0; +} diff --git a/drivers/net/bonding/rte_eth_bond_pmd.c b/drivers/net/bonding/rte_eth_bond_pmd.c index c40d18d128..d2c890075a 100644 --- a/drivers/net/bonding/rte_eth_bond_pmd.c +++ b/drivers/net/bonding/rte_eth_bond_pmd.c @@ -2982,11 +2982,13 @@ bond_ethdev_lsc_event_callback(uint16_t port_id, enum rte_eth_event_type type, int valid_member = 0; uint16_t active_pos, member_idx; uint16_t i; + uint16_t bonding_port_id; if (type != RTE_ETH_EVENT_INTR_LSC || param == NULL) return rc; - bonding_eth_dev = &rte_eth_devices[*(uint16_t *)param]; + bonding_port_id = *(uint16_t *)param; + bonding_eth_dev = &rte_eth_devices[bonding_port_id]; if (check_for_bonding_ethdev(bonding_eth_dev)) return rc; @@ -3058,8 +3060,12 @@ bond_ethdev_lsc_event_callback(uint16_t port_id, enum rte_eth_event_type type, * using it. */ if (internals->user_defined_primary_port && - internals->primary_port == port_id) + internals->primary_port == port_id) { bond_ethdev_primary_set(internals, port_id); + + if (internals->notify_member) + rte_eth_bond_notify_members(bonding_port_id); + } } else { if (active_pos == internals->active_member_count) goto link_update; @@ -3078,6 +3084,10 @@ bond_ethdev_lsc_event_callback(uint16_t port_id, enum rte_eth_event_type type, internals->active_members[0]); else internals->current_primary_port = internals->primary_port; + + if (internals->notify_member) + rte_eth_bond_notify_members(bonding_port_id); + mac_address_members_update(bonding_eth_dev); bond_ethdev_promiscuous_update(bonding_eth_dev); bond_ethdev_allmulticast_update(bonding_eth_dev); @@ -3376,6 +3386,7 @@ dump_basic(const struct rte_eth_dev *dev, FILE *f) struct bond_dev_private instant_priv; const struct bond_dev_private *internals = &instant_priv; int mode, i; + bool notify_member; /* Obtain a instance of dev_private to prevent data from being modified. */ memcpy(&instant_priv, dev->data->dev_private, sizeof(struct bond_dev_private)); @@ -3445,6 +3456,13 @@ dump_basic(const struct rte_eth_dev *dev, FILE *f) fprintf(f, "\tUser Defined Primary: [%u]\n", internals->primary_port); if (internals->member_count > 0) fprintf(f, "\tCurrent Primary: [%u]\n", internals->current_primary_port); + + if (rte_eth_bond_notify_member_flag_get(internals->port_id, ¬ify_member) == 0) + fprintf(f, "\tNotify Member Ports Flag: %s\n", + notify_member ? "enable" : "disable"); + else + fprintf(f, "\tFailed to get notify member ports flag for bonding port %d\n", + internals->port_id); } static void @@ -3997,8 +4015,12 @@ bond_ethdev_configure(struct rte_eth_dev *dev) * if no kvlist, it means that this bonding device has been created * through the bonding api. */ - if (!kvlist || internals->kvargs_processing_is_done) + if (!kvlist || internals->kvargs_processing_is_done) { + if (internals->notify_member && rte_eth_bond_notify_members(port_id) != 0) + RTE_BOND_LOG(ERR, "Notify member ports failed"); + return 0; + } internals->kvargs_processing_is_done = true; @@ -4236,6 +4258,10 @@ bond_ethdev_configure(struct rte_eth_dev *dev) return -1; } } + + if (internals->notify_member && rte_eth_bond_notify_members(port_id) != 0) + RTE_BOND_LOG(ERR, "Notify member ports failed"); + return 0; } diff --git a/drivers/net/bonding/version.map b/drivers/net/bonding/version.map index 09ee21c55f..3bd5e8ad11 100644 --- a/drivers/net/bonding/version.map +++ b/drivers/net/bonding/version.map @@ -35,4 +35,7 @@ EXPERIMENTAL { rte_eth_bond_member_add; rte_eth_bond_member_remove; rte_eth_bond_members_get; + rte_eth_bond_notify_member_flag_get; + rte_eth_bond_notify_member_flag_set; + rte_eth_bond_notify_members; }; diff --git a/lib/ethdev/ethdev_driver.h b/lib/ethdev/ethdev_driver.h index b482cd12bb..39316a7a29 100644 --- a/lib/ethdev/ethdev_driver.h +++ b/lib/ethdev/ethdev_driver.h @@ -1216,6 +1216,21 @@ typedef int (*eth_count_aggr_ports_t)(struct rte_eth_dev *dev); typedef int (*eth_map_aggr_tx_affinity_t)(struct rte_eth_dev *dev, uint16_t tx_queue_id, uint8_t affinity); +/** + * @internal + * Bonding port notifies the member ports. + * + * @param dev + * Member port (ethdev) handle. + * @param bonding_dev + * Bonding port (ethdev) handle. + * + * @return + * Negative on error, 0 on success. + */ +typedef int (*eth_bond_notify_member)(struct rte_eth_dev *dev, + struct rte_eth_dev *bonding_dev); + /** * @internal A structure containing the functions exported by an Ethernet driver. */ @@ -1455,6 +1470,9 @@ struct eth_dev_ops { eth_count_aggr_ports_t count_aggr_ports; /** Map a Tx queue with an aggregated port of the DPDK port */ eth_map_aggr_tx_affinity_t map_aggr_tx_affinity; + + /** Notify the member port of bonding port information */ + eth_bond_notify_member bond_notify_member; }; /** -- 2.39.1 ^ permalink raw reply [flat|nested] 80+ messages in thread
* [PATCH v6 02/14] ethdev: add API to get firmware creation of bonding port 2023-12-26 7:28 ` [PATCH v6 00/14] Enhance the bond framework to support offload Chaoyong He 2023-12-26 7:28 ` [PATCH v6 01/14] ethdev: add member notification for bonding port Chaoyong He @ 2023-12-26 7:28 ` Chaoyong He 2023-12-26 7:28 ` [PATCH v6 03/14] net/bonding: add bonding port arguments Chaoyong He ` (12 subsequent siblings) 14 siblings, 0 replies; 80+ messages in thread From: Chaoyong He @ 2023-12-26 7:28 UTC (permalink / raw) To: dev; +Cc: oss-drivers, Long Wu, Chaoyong He, James Hershaw, Peng Zhang From: Long Wu <long.wu@corigine.com> After bonding port notification, member port firmware may create the bonding port. We want to get the result of creatition, so we add this API to do the getting action. Signed-off-by: Long Wu <long.wu@corigine.com> Reviewed-by: Chaoyong He <chaoyong.he@corigine.com> Reviewed-by: James Hershaw <james.hershaw@corigine.com> Reviewed-by: Peng Zhang <peng.zhang@corigine.com> --- drivers/net/bonding/rte_eth_bond.h | 15 ++++++++++++++ drivers/net/bonding/rte_eth_bond_api.c | 28 ++++++++++++++++++++++++++ drivers/net/bonding/version.map | 1 + lib/ethdev/ethdev_driver.h | 20 ++++++++++++++++++ 4 files changed, 64 insertions(+) diff --git a/drivers/net/bonding/rte_eth_bond.h b/drivers/net/bonding/rte_eth_bond.h index f6c773615c..9a8bed0346 100644 --- a/drivers/net/bonding/rte_eth_bond.h +++ b/drivers/net/bonding/rte_eth_bond.h @@ -398,6 +398,21 @@ __rte_experimental int rte_eth_bond_notify_members(uint16_t bonding_port_id); +/** + * Get the status of specified bonding port created by member port firmware. + * + * @param bonding_port_id + * Port ID of bonding device. + * @param member_port_id + * Port ID of member device. + * + * @return + * 0 on success, negative value otherwise. + */ +__rte_experimental +int +rte_eth_bond_fw_create_get(uint16_t bonding_port_id, uint16_t member_port_id); + #ifdef __cplusplus } #endif diff --git a/drivers/net/bonding/rte_eth_bond_api.c b/drivers/net/bonding/rte_eth_bond_api.c index 239f86ee92..83957830a9 100644 --- a/drivers/net/bonding/rte_eth_bond_api.c +++ b/drivers/net/bonding/rte_eth_bond_api.c @@ -1170,3 +1170,31 @@ rte_eth_bond_notify_members(uint16_t bonding_port_id) return 0; } + +int +rte_eth_bond_fw_create_get(uint16_t bonding_port_id, uint16_t member_port_id) +{ + uint32_t i; + struct rte_eth_dev *bonding_dev; + struct rte_eth_dev *member_dev; + struct bond_dev_private *internals; + + if (valid_bonding_port_id(bonding_port_id) != 0) + return -EINVAL; + + bonding_dev = &rte_eth_devices[bonding_port_id]; + internals = bonding_dev->data->dev_private; + for (i = 0; i < internals->member_count; i++) { + if (internals->members[i].port_id == member_port_id) + break; + } + + if (i == internals->member_count) + return -EINVAL; + + member_dev = &rte_eth_devices[member_port_id]; + if (*member_dev->dev_ops->bond_fw_create_get == NULL) + return -ENOTSUP; + + return member_dev->dev_ops->bond_fw_create_get(member_dev, bonding_dev); +} diff --git a/drivers/net/bonding/version.map b/drivers/net/bonding/version.map index 3bd5e8ad11..12589f61b0 100644 --- a/drivers/net/bonding/version.map +++ b/drivers/net/bonding/version.map @@ -32,6 +32,7 @@ EXPERIMENTAL { global: rte_eth_bond_8023ad_member_info; rte_eth_bond_active_members_get; + rte_eth_bond_fw_create_get; rte_eth_bond_member_add; rte_eth_bond_member_remove; rte_eth_bond_members_get; diff --git a/lib/ethdev/ethdev_driver.h b/lib/ethdev/ethdev_driver.h index 39316a7a29..09d85ad101 100644 --- a/lib/ethdev/ethdev_driver.h +++ b/lib/ethdev/ethdev_driver.h @@ -1231,6 +1231,21 @@ typedef int (*eth_map_aggr_tx_affinity_t)(struct rte_eth_dev *dev, uint16_t tx_q typedef int (*eth_bond_notify_member)(struct rte_eth_dev *dev, struct rte_eth_dev *bonding_dev); +/** + * @internal + * Get the status of specified bonding port created by member port firmware. + * + * @param dev + * Member port (ethdev) handle. + * @param bonding_dev + * Bonding port (ethdev) handle. + * + * @return + * Negative on error, 0 on success. + */ +typedef int (*eth_bond_fw_create_get)(struct rte_eth_dev *dev, + struct rte_eth_dev *bonding_dev); + /** * @internal A structure containing the functions exported by an Ethernet driver. */ @@ -1473,6 +1488,11 @@ struct eth_dev_ops { /** Notify the member port of bonding port information */ eth_bond_notify_member bond_notify_member; + /** + * Get the status of whether bonding port is successfully created by + * the member port firmware. + */ + eth_bond_fw_create_get bond_fw_create_get; }; /** -- 2.39.1 ^ permalink raw reply [flat|nested] 80+ messages in thread
* [PATCH v6 03/14] net/bonding: add bonding port arguments 2023-12-26 7:28 ` [PATCH v6 00/14] Enhance the bond framework to support offload Chaoyong He 2023-12-26 7:28 ` [PATCH v6 01/14] ethdev: add member notification for bonding port Chaoyong He 2023-12-26 7:28 ` [PATCH v6 02/14] ethdev: add API to get firmware creation of " Chaoyong He @ 2023-12-26 7:28 ` Chaoyong He 2023-12-26 7:28 ` [PATCH v6 04/14] net/bonding: support add port by data name Chaoyong He ` (11 subsequent siblings) 14 siblings, 0 replies; 80+ messages in thread From: Chaoyong He @ 2023-12-26 7:28 UTC (permalink / raw) To: dev; +Cc: oss-drivers, Long Wu, Chaoyong He, James Hershaw, Peng Zhang From: Long Wu <long.wu@corigine.com> Include the following new arguments for bonding ports: - "notify_member" to enable/disable member notification. - "dedicated_queue" to enable/disable dedicated queue. Add these two arguments in initial argument. Signed-off-by: Long Wu <long.wu@corigine.com> Reviewed-by: Chaoyong He <chaoyong.he@corigine.com> Reviewed-by: James Hershaw <james.hershaw@corigine.com> Reviewed-by: Peng Zhang <peng.zhang@corigine.com> --- drivers/net/bonding/eth_bond_private.h | 10 ++++ drivers/net/bonding/rte_eth_bond.h | 14 ++++++ drivers/net/bonding/rte_eth_bond_api.c | 14 ++++++ drivers/net/bonding/rte_eth_bond_args.c | 44 ++++++++++++++++++ drivers/net/bonding/rte_eth_bond_pmd.c | 61 ++++++++++++++++++++++++- 5 files changed, 142 insertions(+), 1 deletion(-) diff --git a/drivers/net/bonding/eth_bond_private.h b/drivers/net/bonding/eth_bond_private.h index f69e85c199..f9603a0f6b 100644 --- a/drivers/net/bonding/eth_bond_private.h +++ b/drivers/net/bonding/eth_bond_private.h @@ -28,6 +28,8 @@ #define PMD_BOND_LSC_POLL_PERIOD_KVARG ("lsc_poll_period_ms") #define PMD_BOND_LINK_UP_PROP_DELAY_KVARG ("up_delay") #define PMD_BOND_LINK_DOWN_PROP_DELAY_KVARG ("down_delay") +#define PMD_BOND_NOTIFY_MEMBER_KVARG ("notify_member") +#define PMD_BOND_DEDICATED_QUEUE_KVARG ("dedicated_queue") #define PMD_BOND_XMIT_POLICY_LAYER2_KVARG ("l2") #define PMD_BOND_XMIT_POLICY_LAYER23_KVARG ("l23") @@ -319,6 +321,14 @@ int bond_ethdev_parse_time_ms_kvarg(const char *key, const char *value, void *extra_args); +int +bond_ethdev_parse_notify_member_kvarg(const char *key __rte_unused, + const char *value, void *extra_args); + +int +bond_ethdev_parse_dedicated_queue_kvarg(const char *key __rte_unused, + const char *value, void *extra_args); + void bond_tlb_disable(struct bond_dev_private *internals); diff --git a/drivers/net/bonding/rte_eth_bond.h b/drivers/net/bonding/rte_eth_bond.h index 9a8bed0346..84dd39b12f 100644 --- a/drivers/net/bonding/rte_eth_bond.h +++ b/drivers/net/bonding/rte_eth_bond.h @@ -351,6 +351,20 @@ rte_eth_bond_link_up_prop_delay_set(uint16_t bonding_port_id, int rte_eth_bond_link_up_prop_delay_get(uint16_t bonding_port_id); +/** + * Set the flag that whether bonding device enable dedicated queue. + * + * @param bonding_port_id + * Port ID of bonding device. + * @param queue_flag + * The flag of enable bond dedicated queue + * + * @return + * 0 on success, negative value otherwise. + */ +int +rte_eth_bond_dedicated_queue_flag_set(uint16_t bonding_port_id, bool queue_flag); + /** * Set the flag of whether bonding port notifies member ports. * diff --git a/drivers/net/bonding/rte_eth_bond_api.c b/drivers/net/bonding/rte_eth_bond_api.c index 83957830a9..32fa761baf 100644 --- a/drivers/net/bonding/rte_eth_bond_api.c +++ b/drivers/net/bonding/rte_eth_bond_api.c @@ -1114,6 +1114,20 @@ rte_eth_bond_link_up_prop_delay_get(uint16_t bonding_port_id) return internals->link_up_delay_ms; } +int +rte_eth_bond_dedicated_queue_flag_set(uint16_t bonding_port_id, bool queue_flag) +{ + struct bond_dev_private *internals; + + if (valid_bonding_port_id(bonding_port_id) != 0) + return -1; + + internals = rte_eth_devices[bonding_port_id].data->dev_private; + internals->mode4.dedicated_queues.enabled = queue_flag; + + return 0; +} + int rte_eth_bond_notify_member_flag_set(uint16_t bonding_port_id, bool notify) { diff --git a/drivers/net/bonding/rte_eth_bond_args.c b/drivers/net/bonding/rte_eth_bond_args.c index bdec5d61d4..8a3e4656ef 100644 --- a/drivers/net/bonding/rte_eth_bond_args.c +++ b/drivers/net/bonding/rte_eth_bond_args.c @@ -20,6 +20,8 @@ const char *pmd_bond_init_valid_arguments[] = { PMD_BOND_MAC_ADDR_KVARG, PMD_BOND_AGG_MODE_KVARG, RTE_DEVARGS_KEY_DRIVER, + PMD_BOND_NOTIFY_MEMBER_KVARG, + PMD_BOND_DEDICATED_QUEUE_KVARG, NULL }; @@ -297,3 +299,45 @@ bond_ethdev_parse_time_ms_kvarg(const char *key __rte_unused, return 0; } + +int +bond_ethdev_parse_notify_member_kvarg(const char *key __rte_unused, + const char *value, void *extra_args) +{ + bool *notify_member; + + if (value == NULL || extra_args == NULL) + return -1; + + notify_member = extra_args; + + if (strcmp("enable", value) == 0) + *notify_member = true; + else if (strcmp("disable", value) == 0) + *notify_member = false; + else + return -1; + + return 0; +} + +int +bond_ethdev_parse_dedicated_queue_kvarg(const char *key __rte_unused, + const char *value, void *extra_args) +{ + bool *dedicated_queue; + + if (value == NULL || extra_args == NULL) + return -1; + + dedicated_queue = extra_args; + + if (strcmp("enable", value) == 0) + *dedicated_queue = true; + else if (strcmp("disable", value) == 0) + *dedicated_queue = false; + else + return -1; + + return 0; +} diff --git a/drivers/net/bonding/rte_eth_bond_pmd.c b/drivers/net/bonding/rte_eth_bond_pmd.c index d2c890075a..b3a37a36c8 100644 --- a/drivers/net/bonding/rte_eth_bond_pmd.c +++ b/drivers/net/bonding/rte_eth_bond_pmd.c @@ -4246,6 +4246,63 @@ bond_ethdev_configure(struct rte_eth_dev *dev) return -1; } + /* Parse/set notify member flag */ + arg_count = rte_kvargs_count(kvlist, PMD_BOND_NOTIFY_MEMBER_KVARG); + if (arg_count == 1) { + bool notify_member; + if (rte_kvargs_process(kvlist, + PMD_BOND_NOTIFY_MEMBER_KVARG, + &bond_ethdev_parse_notify_member_kvarg, + ¬ify_member) < 0) { + RTE_BOND_LOG(ERR, + "Invalid notify member value specified" + " for bonding device %s", name); + return -1; + } + + if (rte_eth_bond_notify_member_flag_set(port_id, notify_member) != 0) { + RTE_BOND_LOG(ERR, + "Failed to set notify member (%u) on" + " bonding device %s", notify_member, name); + return -1; + } + } else if (arg_count > 1) { + RTE_BOND_LOG(ERR, + "notify member flag can be specified only once" + " for bonding device %s", name); + return -1; + } + + /* Parse/set dedicated queue flag */ + arg_count = rte_kvargs_count(kvlist, PMD_BOND_DEDICATED_QUEUE_KVARG); + if (arg_count == 1) { + bool dedicated_queue; + if (rte_kvargs_process(kvlist, + PMD_BOND_DEDICATED_QUEUE_KVARG, + &bond_ethdev_parse_dedicated_queue_kvarg, + &dedicated_queue) < 0) { + RTE_BOND_LOG(ERR, + "Invalid dedicated queue flag specified" + " for bonding device %s", name); + return -1; + } + + if (internals->mode == BONDING_MODE_8023AD) { + if (rte_eth_bond_dedicated_queue_flag_set(port_id, dedicated_queue) != 0) { + RTE_BOND_LOG(ERR, + "Failed to enable/disable dedicated" + " queue flag on bonding device %s", + name); + return -1; + } + } + } else if (arg_count > 1) { + RTE_BOND_LOG(ERR, + "dedicated queue flag can be specified only once" + " for bonding device %s", name); + return -1; + } + /* configure members so we can pass mtu setting */ for (i = 0; i < internals->member_count; i++) { struct rte_eth_dev *member_ethdev = @@ -4283,7 +4340,9 @@ RTE_PMD_REGISTER_PARAM_STRING(net_bonding, "mac=<mac addr> " "lsc_poll_period_ms=<int> " "up_delay=<int> " - "down_delay=<int>"); + "down_delay=<int>" + "notify_member=[enable | disable] " + "dedicated_queue=[enable | disable] "); /* We can't use RTE_LOG_REGISTER_DEFAULT because of the forced name for * this library, see meson.build. -- 2.39.1 ^ permalink raw reply [flat|nested] 80+ messages in thread
* [PATCH v6 04/14] net/bonding: support add port by data name 2023-12-26 7:28 ` [PATCH v6 00/14] Enhance the bond framework to support offload Chaoyong He ` (2 preceding siblings ...) 2023-12-26 7:28 ` [PATCH v6 03/14] net/bonding: add bonding port arguments Chaoyong He @ 2023-12-26 7:28 ` Chaoyong He 2023-12-26 7:28 ` [PATCH v6 05/14] net/bonding: support checking valid bonding port ID Chaoyong He ` (10 subsequent siblings) 14 siblings, 0 replies; 80+ messages in thread From: Chaoyong He @ 2023-12-26 7:28 UTC (permalink / raw) To: dev; +Cc: oss-drivers, Long Wu, Chaoyong He, James Hershaw, Peng Zhang From: Long Wu <long.wu@corigine.com> Several ports may share the same PCI address, like NFP representor port. So users cannot add this type of ports to bonding port by "--vdev" argument in dpdk-testpmd. But the port's data name is unique between them, include an option to add such ports to the bonding port. After adding this feature, users can create a bonding port that member port is this type of port by "--vdev" in dpdk-testpmd start command. For example: dpdk-testpmd -l 2-10 -s 0x8 -a ca:00.0,representor=[0-2] --vdev 'net_bonding0,member=flower_repr_p0,member=flower_repr_p1, mode=4,socket_id=1,xmit_policy=l34' -- -i Note: 1. "ca:00.0" is NFP 4000 card. 2. "flower_repr_p0" and "flower_repr_p1" are NFP phy representor port's data name. Signed-off-by: Long Wu <long.wu@corigine.com> Reviewed-by: Chaoyong He <chaoyong.he@corigine.com> Reviewed-by: James Hershaw <james.hershaw@corigine.com> Reviewed-by: Peng Zhang <peng.zhang@corigine.com> --- drivers/net/bonding/rte_eth_bond_args.c | 3 +++ 1 file changed, 3 insertions(+) diff --git a/drivers/net/bonding/rte_eth_bond_args.c b/drivers/net/bonding/rte_eth_bond_args.c index 8a3e4656ef..b320eb3038 100644 --- a/drivers/net/bonding/rte_eth_bond_args.c +++ b/drivers/net/bonding/rte_eth_bond_args.c @@ -70,6 +70,9 @@ find_port_id_by_dev_name(const char *name) if (strcmp(rte_eth_devices[i].device->name, name) == 0) return i; + + if (strcmp(rte_eth_devices[i].data->name, name) == 0) + return i; } return -1; } -- 2.39.1 ^ permalink raw reply [flat|nested] 80+ messages in thread
* [PATCH v6 05/14] net/bonding: support checking valid bonding port ID 2023-12-26 7:28 ` [PATCH v6 00/14] Enhance the bond framework to support offload Chaoyong He ` (3 preceding siblings ...) 2023-12-26 7:28 ` [PATCH v6 04/14] net/bonding: support add port by data name Chaoyong He @ 2023-12-26 7:28 ` Chaoyong He 2023-12-26 7:28 ` [PATCH v6 06/14] net/bonding: add commands for bonding port notification Chaoyong He ` (9 subsequent siblings) 14 siblings, 0 replies; 80+ messages in thread From: Chaoyong He @ 2023-12-26 7:28 UTC (permalink / raw) To: dev; +Cc: oss-drivers, Long Wu, Chaoyong He, James Hershaw, Peng Zhang From: Long Wu <long.wu@corigine.com> Add API to support checking if the port id is a bonding port id. Signed-off-by: Long Wu <long.wu@corigine.com> Reviewed-by: Chaoyong He <chaoyong.he@corigine.com> Reviewed-by: James Hershaw <james.hershaw@corigine.com> Reviewed-by: Peng Zhang <peng.zhang@corigine.com> --- drivers/net/bonding/rte_eth_bond.h | 13 +++++++++++++ drivers/net/bonding/rte_eth_bond_api.c | 7 +++++++ drivers/net/bonding/version.map | 1 + 3 files changed, 21 insertions(+) diff --git a/drivers/net/bonding/rte_eth_bond.h b/drivers/net/bonding/rte_eth_bond.h index 84dd39b12f..62af9bbd99 100644 --- a/drivers/net/bonding/rte_eth_bond.h +++ b/drivers/net/bonding/rte_eth_bond.h @@ -427,6 +427,19 @@ __rte_experimental int rte_eth_bond_fw_create_get(uint16_t bonding_port_id, uint16_t member_port_id); +/** + * Check whether bonding port id is valid. + * + * @param port_id + * Port ID of bonding device. + * + * @return + * true means the port is a bonding device, false means not. + */ +__rte_experimental +bool +rte_eth_bond_is_valid_port(uint16_t port_id); + #ifdef __cplusplus } #endif diff --git a/drivers/net/bonding/rte_eth_bond_api.c b/drivers/net/bonding/rte_eth_bond_api.c index 32fa761baf..5e9437c786 100644 --- a/drivers/net/bonding/rte_eth_bond_api.c +++ b/drivers/net/bonding/rte_eth_bond_api.c @@ -1212,3 +1212,10 @@ rte_eth_bond_fw_create_get(uint16_t bonding_port_id, uint16_t member_port_id) return member_dev->dev_ops->bond_fw_create_get(member_dev, bonding_dev); } + + +bool +rte_eth_bond_is_valid_port(uint16_t port_id) +{ + return (valid_bonding_port_id(port_id) == 0); +} diff --git a/drivers/net/bonding/version.map b/drivers/net/bonding/version.map index 12589f61b0..cd27c43092 100644 --- a/drivers/net/bonding/version.map +++ b/drivers/net/bonding/version.map @@ -33,6 +33,7 @@ EXPERIMENTAL { rte_eth_bond_8023ad_member_info; rte_eth_bond_active_members_get; rte_eth_bond_fw_create_get; + rte_eth_bond_is_valid_port; rte_eth_bond_member_add; rte_eth_bond_member_remove; rte_eth_bond_members_get; -- 2.39.1 ^ permalink raw reply [flat|nested] 80+ messages in thread
* [PATCH v6 06/14] net/bonding: add commands for bonding port notification 2023-12-26 7:28 ` [PATCH v6 00/14] Enhance the bond framework to support offload Chaoyong He ` (4 preceding siblings ...) 2023-12-26 7:28 ` [PATCH v6 05/14] net/bonding: support checking valid bonding port ID Chaoyong He @ 2023-12-26 7:28 ` Chaoyong He 2023-12-26 7:28 ` [PATCH v6 07/14] net/bonding: create new rte flow header file Chaoyong He ` (8 subsequent siblings) 14 siblings, 0 replies; 80+ messages in thread From: Chaoyong He @ 2023-12-26 7:28 UTC (permalink / raw) To: dev; +Cc: oss-drivers, Long Wu, Chaoyong He, James Hershaw, Peng Zhang From: Long Wu <long.wu@corigine.com> Add some commands to support bonding port notification in dpdk-testpmd. 1. Users can enable the notification by command: "set bonding notify_member (port_id) (enable|disable)" 2. If member port firmware try to create the bonding port after notification users can get the status by command: "get bonding member firmware create (member_port_id) (bonding_port_id)" Signed-off-by: Long Wu <long.wu@corigine.com> Reviewed-by: Chaoyong He <chaoyong.he@corigine.com> Reviewed-by: James Hershaw <james.hershaw@corigine.com> Reviewed-by: Peng Zhang <peng.zhang@corigine.com> --- .../link_bonding_poll_mode_drv_lib.rst | 18 +++ drivers/net/bonding/bonding_testpmd.c | 128 ++++++++++++++++++ 2 files changed, 146 insertions(+) diff --git a/doc/guides/prog_guide/link_bonding_poll_mode_drv_lib.rst b/doc/guides/prog_guide/link_bonding_poll_mode_drv_lib.rst index 60717a3587..653d2f850d 100644 --- a/doc/guides/prog_guide/link_bonding_poll_mode_drv_lib.rst +++ b/doc/guides/prog_guide/link_bonding_poll_mode_drv_lib.rst @@ -637,3 +637,21 @@ in balance mode with a transmission policy of layer 2+3:: Members (3): [1 3 4] Active Members (3): [1 3 4] Primary: [3] + +set bonding notify_member +~~~~~~~~~~~~~~~~~~~~~~~~~ + +Set the notify member flag of bonding port:: + + testpmd> set bonding notify_member (port_id) (enable|disable) + +This command just set the flag of notification. +If we enable it, bonding PMD will notify member ports when its some +configurations changed. + +get bonding member firmware create +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +Get the status of member port hardware creating the bonding port:: + + testpmd> get bonding member firmware create (member_port_id) (bonding_port_id) diff --git a/drivers/net/bonding/bonding_testpmd.c b/drivers/net/bonding/bonding_testpmd.c index 8fcd6cadd0..48f71da3a7 100644 --- a/drivers/net/bonding/bonding_testpmd.c +++ b/drivers/net/bonding/bonding_testpmd.c @@ -692,6 +692,124 @@ static cmdline_parse_inst_t cmd_set_bonding_agg_mode_policy = { } }; +struct cmd_set_bonding_notify_member_result { + cmdline_fixed_string_t set; + cmdline_fixed_string_t bonding; + cmdline_fixed_string_t notify_member; + uint16_t port_num; + cmdline_fixed_string_t mode; +}; + +static void +cmd_set_bonding_notify_member_parsed(void *parsed_result, + __rte_unused struct cmdline *cl, __rte_unused void *data) +{ + struct cmd_set_bonding_notify_member_result *res = parsed_result; + bool notify_member = false; + + if (strcmp(res->notify_member, "enable") == 0) + notify_member = true; + else if (strcmp(res->notify_member, "disable") == 0) + notify_member = false; + + rte_eth_bond_notify_member_flag_set(res->port_num, notify_member); +} + +static cmdline_parse_token_string_t cmd_set_bonding_notify_member_set = + TOKEN_STRING_INITIALIZER(struct cmd_set_bonding_notify_member_result, + set, "set"); +static cmdline_parse_token_string_t cmd_set_bonding_notify_member_bonding = + TOKEN_STRING_INITIALIZER(struct cmd_set_bonding_notify_member_result, + bonding, "bonding"); +static cmdline_parse_token_string_t cmd_set_bonding_notify_member = + TOKEN_STRING_INITIALIZER(struct cmd_set_bonding_notify_member_result, + notify_member, "notify_member"); +static cmdline_parse_token_num_t cmd_set_bonding_notify_member_portnum = + TOKEN_NUM_INITIALIZER(struct cmd_set_bonding_notify_member_result, + port_num, RTE_UINT16); +static cmdline_parse_token_string_t cmd_set_bonding_notify_member_mode_string = + TOKEN_STRING_INITIALIZER(struct cmd_set_bonding_notify_member_result, + mode, "enable#disable"); + +static cmdline_parse_inst_t cmd_set_bonding_notify_member_ports = { + .f = cmd_set_bonding_notify_member_parsed, + .data = NULL, + .help_str = "set bonding notify_member (port_id) (enable|disable)", + .tokens = { + (void *)&cmd_set_bonding_notify_member_set, + (void *)&cmd_set_bonding_notify_member_bonding, + (void *)&cmd_set_bonding_notify_member, + (void *)&cmd_set_bonding_notify_member_portnum, + (void *)&cmd_set_bonding_notify_member_mode_string, + NULL + } +}; + +struct cmd_get_bonding_member_hw_create_result { + cmdline_fixed_string_t get; + cmdline_fixed_string_t bonding; + cmdline_fixed_string_t member; + cmdline_fixed_string_t firmware; + cmdline_fixed_string_t create; + uint16_t member_port_id; + uint16_t bonding_port_id; +}; + +static void +cmd_get_bonding_member_hw_create_parsed(void *parsed_result, + __rte_unused struct cmdline *cl, __rte_unused void *data) +{ + struct cmd_get_bonding_member_hw_create_result *res = parsed_result; + int ret; + + ret = rte_eth_bond_fw_create_get(res->bonding_port_id, res->member_port_id); + if (ret == 0) + printf("Member port %u firmware creates bonding port %u successfully\n", + res->member_port_id, res->bonding_port_id); + else + printf("Failed to get status of member port %u firmware creating" + " bonding port %u, %d\n", + res->member_port_id, res->bonding_port_id, ret); +} + +static cmdline_parse_token_string_t cmd_get_bonding_member_hw_create_get = + TOKEN_STRING_INITIALIZER(struct cmd_get_bonding_member_hw_create_result, + get, "get"); +static cmdline_parse_token_string_t cmd_get_bonding_member_hw_create_bonding = + TOKEN_STRING_INITIALIZER(struct cmd_get_bonding_member_hw_create_result, + bonding, "bonding"); +static cmdline_parse_token_string_t cmd_get_bonding_member_hw_create_member = + TOKEN_STRING_INITIALIZER(struct cmd_get_bonding_member_hw_create_result, + member, "member"); +static cmdline_parse_token_string_t cmd_get_bonding_member_hw_create_firmware = + TOKEN_STRING_INITIALIZER(struct cmd_get_bonding_member_hw_create_result, + create, "firmware"); +static cmdline_parse_token_string_t cmd_get_bonding_member_hw_create_create = + TOKEN_STRING_INITIALIZER(struct cmd_get_bonding_member_hw_create_result, + create, "create"); +static cmdline_parse_token_num_t cmd_get_bonding_member_hw_create_memberportid = + TOKEN_NUM_INITIALIZER(struct cmd_get_bonding_member_hw_create_result, + member_port_id, RTE_UINT16); +static cmdline_parse_token_num_t cmd_get_bonding_member_hw_create_bondingportid = + TOKEN_NUM_INITIALIZER(struct cmd_get_bonding_member_hw_create_result, + bonding_port_id, RTE_UINT16); + +static cmdline_parse_inst_t cmd_get_member_hw_create_bonding = { + .f = cmd_get_bonding_member_hw_create_parsed, + .data = NULL, + .help_str = "get bonding member firmware create (member_port_id) (bonding_port_id)", + .tokens = { + (void *)&cmd_get_bonding_member_hw_create_get, + (void *)&cmd_get_bonding_member_hw_create_bonding, + (void *)&cmd_get_bonding_member_hw_create_member, + (void *)&cmd_get_bonding_member_hw_create_firmware, + (void *)&cmd_get_bonding_member_hw_create_create, + (void *)&cmd_get_bonding_member_hw_create_memberportid, + (void *)&cmd_get_bonding_member_hw_create_bondingportid, + NULL + } +}; + static struct testpmd_driver_commands bonding_cmds = { .commands = { { @@ -749,6 +867,16 @@ static struct testpmd_driver_commands bonding_cmds = { "set bonding mode IEEE802.3AD aggregator policy (port_id) (agg_name)\n" " Set Aggregation mode for IEEE802.3AD (mode 4)\n", }, + { + &cmd_set_bonding_notify_member_ports, + "set bonding notify_member (port_id) (enable|disable)\n" + " Enable/disable the notify member flag of bonding port\n", + }, + { + &cmd_get_member_hw_create_bonding, + "get bonding member firmware create (member_port_id) (bonding_port_id)\n" + " Get the status of member port firmware creating the bonding port\n", + }, { NULL, NULL }, }, }; -- 2.39.1 ^ permalink raw reply [flat|nested] 80+ messages in thread
* [PATCH v6 07/14] net/bonding: create new rte flow header file 2023-12-26 7:28 ` [PATCH v6 00/14] Enhance the bond framework to support offload Chaoyong He ` (5 preceding siblings ...) 2023-12-26 7:28 ` [PATCH v6 06/14] net/bonding: add commands for bonding port notification Chaoyong He @ 2023-12-26 7:28 ` Chaoyong He 2023-12-26 7:28 ` [PATCH v6 08/14] net/nfp: add bond firmware creation initialization Chaoyong He ` (7 subsequent siblings) 14 siblings, 0 replies; 80+ messages in thread From: Chaoyong He @ 2023-12-26 7:28 UTC (permalink / raw) To: dev; +Cc: oss-drivers, Long Wu, James Hershaw, Chaoyong He, Peng Zhang From: Long Wu <long.wu@corigine.com> Move the flow code to a new head file to make flow related code more clean and make the code architecture more reasonable in the future. There is no functional change, just moving verbatim code around. Signed-off-by: Long Wu <long.wu@corigine.com> Reviewed-by: James Hershaw <james.hershaw@corigine.com> Reviewed-by: Chaoyong He <chaoyong.he@corigine.com> Reviewed-by: Peng Zhang <peng.zhang@corigine.com> --- drivers/net/bonding/eth_bond_private.h | 13 ------------- drivers/net/bonding/rte_eth_bond_api.c | 1 + drivers/net/bonding/rte_eth_bond_flow.c | 1 + drivers/net/bonding/rte_eth_bond_flow.h | 22 ++++++++++++++++++++++ drivers/net/bonding/rte_eth_bond_pmd.c | 1 + 5 files changed, 25 insertions(+), 13 deletions(-) create mode 100644 drivers/net/bonding/rte_eth_bond_flow.h diff --git a/drivers/net/bonding/eth_bond_private.h b/drivers/net/bonding/eth_bond_private.h index f9603a0f6b..4373465d8d 100644 --- a/drivers/net/bonding/eth_bond_private.h +++ b/drivers/net/bonding/eth_bond_private.h @@ -9,10 +9,8 @@ #include <sys/queue.h> #include <ethdev_driver.h> -#include <rte_flow.h> #include <rte_spinlock.h> #include <rte_bitmap.h> -#include <rte_flow_driver.h> #include "rte_eth_bond.h" #include "eth_bond_8023ad_private.h" @@ -47,8 +45,6 @@ extern const char *pmd_bond_init_valid_arguments[]; extern struct rte_vdev_driver pmd_bond_drv; -extern const struct rte_flow_ops bond_flow_ops; - /** Port Queue Mapping Structure */ struct bond_rx_queue { uint16_t queue_id; @@ -94,15 +90,6 @@ struct bond_member_details { uint16_t reta_size; }; -struct rte_flow { - TAILQ_ENTRY(rte_flow) next; - /* Members flows */ - struct rte_flow *flows[RTE_MAX_ETHPORTS]; - /* Flow description for synchronization */ - struct rte_flow_conv_rule rule; - uint8_t rule_data[]; -}; - typedef void (*burst_xmit_hash_t)(struct rte_mbuf **buf, uint16_t nb_pkts, uint16_t member_count, uint16_t *members); diff --git a/drivers/net/bonding/rte_eth_bond_api.c b/drivers/net/bonding/rte_eth_bond_api.c index 5e9437c786..0a296d3894 100644 --- a/drivers/net/bonding/rte_eth_bond_api.c +++ b/drivers/net/bonding/rte_eth_bond_api.c @@ -12,6 +12,7 @@ #include <rte_kvargs.h> #include "rte_eth_bond.h" +#include "rte_eth_bond_flow.h" #include "eth_bond_private.h" #include "eth_bond_8023ad_private.h" diff --git a/drivers/net/bonding/rte_eth_bond_flow.c b/drivers/net/bonding/rte_eth_bond_flow.c index 71a91675f7..e6c7ce5362 100644 --- a/drivers/net/bonding/rte_eth_bond_flow.c +++ b/drivers/net/bonding/rte_eth_bond_flow.c @@ -1,6 +1,7 @@ /* SPDX-License-Identifier: BSD-3-Clause * Copyright 2018 Mellanox Technologies, Ltd */ +#include "rte_eth_bond_flow.h" #include <stddef.h> #include <string.h> diff --git a/drivers/net/bonding/rte_eth_bond_flow.h b/drivers/net/bonding/rte_eth_bond_flow.h new file mode 100644 index 0000000000..7394e0e2e1 --- /dev/null +++ b/drivers/net/bonding/rte_eth_bond_flow.h @@ -0,0 +1,22 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright (c) 2023 Corigine, Inc. + */ + +#ifndef _RTE_ETH_BOND_FLOW_H_ +#define _RTE_ETH_BOND_FLOW_H_ + +#include <rte_flow.h> +#include <rte_flow_driver.h> + +extern const struct rte_flow_ops bond_flow_ops; + +struct rte_flow { + TAILQ_ENTRY(rte_flow) next; + struct rte_flow *flows[RTE_MAX_ETHPORTS]; + /**< Member ports flows */ + struct rte_flow_conv_rule rule; + /**< Flow description for synchronization */ + uint8_t rule_data[]; +}; + +#endif /* _RTE_ETH_BOND_FLOW_H_ */ diff --git a/drivers/net/bonding/rte_eth_bond_pmd.c b/drivers/net/bonding/rte_eth_bond_pmd.c index b3a37a36c8..650f8061a2 100644 --- a/drivers/net/bonding/rte_eth_bond_pmd.c +++ b/drivers/net/bonding/rte_eth_bond_pmd.c @@ -21,6 +21,7 @@ #include <rte_string_fns.h> #include "rte_eth_bond.h" +#include "rte_eth_bond_flow.h" #include "eth_bond_private.h" #include "eth_bond_8023ad_private.h" -- 2.39.1 ^ permalink raw reply [flat|nested] 80+ messages in thread
* [PATCH v6 08/14] net/nfp: add bond firmware creation initialization 2023-12-26 7:28 ` [PATCH v6 00/14] Enhance the bond framework to support offload Chaoyong He ` (6 preceding siblings ...) 2023-12-26 7:28 ` [PATCH v6 07/14] net/bonding: create new rte flow header file Chaoyong He @ 2023-12-26 7:28 ` Chaoyong He 2023-12-26 7:28 ` [PATCH v6 09/14] net/nfp: reset bond configuration of firmware Chaoyong He ` (6 subsequent siblings) 14 siblings, 0 replies; 80+ messages in thread From: Chaoyong He @ 2023-12-26 7:28 UTC (permalink / raw) To: dev; +Cc: oss-drivers, Long Wu, Peng Zhang, Chaoyong He From: Long Wu <long.wu@corigine.com> Firmware supports several features and bond firmware creation is one of the features. Driver notifies firmware that driver supports bond firmware creation feature by CPP bus write. If write successfully, initialize driver configuration. Signed-off-by: Long Wu <long.wu@corigine.com> Reviewed-by: Peng Zhang <peng.zhang@corigine.com> Reviewed-by: Chaoyong He <chaoyong.he@corigine.com> --- drivers/net/nfp/flower/nfp_flower.c | 64 ++++++++++++++++++++++- drivers/net/nfp/flower/nfp_flower.h | 13 +++++ drivers/net/nfp/flower/nfp_flower_bond.c | 59 +++++++++++++++++++++ drivers/net/nfp/flower/nfp_flower_bond.h | 65 ++++++++++++++++++++++++ drivers/net/nfp/meson.build | 1 + 5 files changed, 200 insertions(+), 2 deletions(-) create mode 100644 drivers/net/nfp/flower/nfp_flower_bond.c create mode 100644 drivers/net/nfp/flower/nfp_flower_bond.h diff --git a/drivers/net/nfp/flower/nfp_flower.c b/drivers/net/nfp/flower/nfp_flower.c index 94b50611f0..195960e00d 100644 --- a/drivers/net/nfp/flower/nfp_flower.c +++ b/drivers/net/nfp/flower/nfp_flower.c @@ -16,6 +16,7 @@ #include "../nfp_cpp_bridge.h" #include "../nfp_logs.h" #include "../nfp_mtr.h" +#include "nfp_flower_bond.h" #include "nfp_flower_ctrl.h" #include "nfp_flower_representor.h" @@ -228,6 +229,42 @@ nfp_flower_init_vnic_common(struct nfp_net_hw *hw, return 0; } +static int +nfp_flower_bond_feature_init(struct nfp_app_fw_flower *app_fw_flower) +{ + int ret; + + /* Notify hardware that driver supports hardware creation of bonding port */ + ret = nfp_rtsym_write_le(app_fw_flower->pf_hw->pf_dev->sym_tbl, + "_abi_flower_balance_sync_enable", 1); + if (ret != 0) { + PMD_INIT_LOG(ERR, "Enable bonding port hardware creation failed."); + return ret; + } + + ret = nfp_flower_bond_init(app_fw_flower); + if (ret != 0) { + PMD_INIT_LOG(ERR, "Initialize bonding structure failed."); + return ret; + } + + app_fw_flower->flower_en_feats |= NFP_FL_ENABLE_BOND; + + return 0; +} + +static int +nfp_flower_sync_feature_bits(struct nfp_app_fw_flower *app_fw_flower) +{ + int ret; + + ret = nfp_flower_bond_feature_init(app_fw_flower); + if (ret != 0) + return ret; + + return 0; +} + static int nfp_flower_init_ctrl_vnic(struct nfp_net_hw *hw) { @@ -538,6 +575,20 @@ nfp_flower_cleanup_ctrl_vnic(struct nfp_net_hw *hw) rte_free(eth_dev); } +static void +nfp_flower_bond_feature_cleanup(struct nfp_app_fw_flower *app_fw_flower) +{ + nfp_flower_bond_cleanup(app_fw_flower); + app_fw_flower->flower_en_feats &= ~NFP_FL_ENABLE_BOND; +} + +static void +nfp_flower_sync_feature_cleanup(struct nfp_app_fw_flower *app_fw_flower) +{ + if (nfp_flower_support_bond_offload(app_fw_flower)) + nfp_flower_bond_feature_cleanup(app_fw_flower); +} + static int nfp_flower_start_ctrl_vnic(struct nfp_net_hw *net_hw) { @@ -768,22 +819,31 @@ nfp_init_app_fw_flower(struct nfp_pf_dev *pf_dev, goto ctrl_vnic_cleanup; } + /* Synchronize the features of driver and hardware */ + ret = nfp_flower_sync_feature_bits(app_fw_flower); + if (ret != 0) { + PMD_INIT_LOG(ERR, "Sync feature bits failed"); + goto ctrl_vnic_cleanup; + } + /* Start up flower services */ ret = nfp_flower_enable_services(app_fw_flower); if (ret != 0) { PMD_INIT_LOG(ERR, "Could not enable flower services"); ret = -ESRCH; - goto ctrl_vnic_cleanup; + goto sync_feature_cleanup; } ret = nfp_flower_repr_create(app_fw_flower); if (ret != 0) { PMD_INIT_LOG(ERR, "Could not create representor ports"); - goto ctrl_vnic_cleanup; + goto sync_feature_cleanup; } return 0; +sync_feature_cleanup: + nfp_flower_sync_feature_cleanup(app_fw_flower); ctrl_vnic_cleanup: nfp_flower_cleanup_ctrl_vnic(app_fw_flower->ctrl_hw); ctrl_cpp_area_cleanup: diff --git a/drivers/net/nfp/flower/nfp_flower.h b/drivers/net/nfp/flower/nfp_flower.h index 8393de66c5..30443cd568 100644 --- a/drivers/net/nfp/flower/nfp_flower.h +++ b/drivers/net/nfp/flower/nfp_flower.h @@ -7,6 +7,7 @@ #define __NFP_FLOWER_H__ #include "../nfp_net_common.h" +#include "nfp_flower_bond.h" /* Extra features bitmap. */ #define NFP_FL_FEATS_GENEVE RTE_BIT64(0) @@ -61,6 +62,9 @@ struct nfp_app_fw_flower { /** Number of phyport representors */ uint8_t num_phyport_reprs; + /** Bitmap of features enabled by HW */ + uint8_t flower_en_feats; + /** Pointer to the PF vNIC */ struct nfp_net_hw *pf_hw; @@ -96,6 +100,9 @@ struct nfp_app_fw_flower { /** Function pointers for different NFD version */ struct nfp_flower_nfd_func nfd_func; + + /** Link bond data block */ + struct nfp_flower_bond *nfp_bond; }; static inline bool @@ -104,6 +111,12 @@ nfp_flower_support_decap_v2(const struct nfp_app_fw_flower *app_fw_flower) return app_fw_flower->ext_features & NFP_FL_FEATS_DECAP_V2; } +static inline bool +nfp_flower_support_bond_offload(const struct nfp_app_fw_flower *app_fw_flower) +{ + return app_fw_flower->flower_en_feats & NFP_FL_ENABLE_BOND; +} + int nfp_init_app_fw_flower(struct nfp_pf_dev *pf_dev, const struct nfp_dev_info *dev_info); void nfp_uninit_app_fw_flower(struct nfp_pf_dev *pf_dev); diff --git a/drivers/net/nfp/flower/nfp_flower_bond.c b/drivers/net/nfp/flower/nfp_flower_bond.c new file mode 100644 index 0000000000..bbd2818e68 --- /dev/null +++ b/drivers/net/nfp/flower/nfp_flower_bond.c @@ -0,0 +1,59 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright (c) 2023 Corigine, Inc. + * All rights reserved. + */ +#include "nfp_flower_bond.h" + +#include <rte_malloc.h> + +#include "nfp_flower_representor.h" + +static void +nfp_flower_bond_increment_version(struct nfp_flower_bond *nfp_bond) +{ + /* LSB is not considered by firmware so add 2 for each increment. */ + nfp_bond->batch_ver += 2; + nfp_bond->batch_ver &= NFP_FL_BOND_VERSION_MASK; + + /* Zero is reserved by firmware. */ + if (nfp_bond->batch_ver == 0) + nfp_bond->batch_ver += 2; +} + +int +nfp_flower_bond_init(struct nfp_app_fw_flower *app_fw_flower) +{ + struct nfp_flower_bond *nfp_bond; + + app_fw_flower->nfp_bond = rte_zmalloc("nfp_bond", + sizeof(struct nfp_flower_bond), RTE_CACHE_LINE_SIZE); + if (app_fw_flower->nfp_bond == NULL) + return -ENOMEM; + + nfp_bond = app_fw_flower->nfp_bond; + pthread_mutex_init(&nfp_bond->mutex, NULL); + LIST_INIT(&nfp_bond->group_list); + nfp_flower_bond_increment_version(nfp_bond); + nfp_bond->app_fw_flower = app_fw_flower; + + return 0; +} + +void +nfp_flower_bond_cleanup(struct nfp_app_fw_flower *app_fw_flower) +{ + struct nfp_bond_group *entry; + struct nfp_flower_bond *nfp_bond = app_fw_flower->nfp_bond; + + pthread_mutex_lock(&nfp_bond->mutex); + LIST_FOREACH(entry, &nfp_bond->group_list, next) { + LIST_REMOVE(entry, next); + rte_free(entry); + } + pthread_mutex_unlock(&nfp_bond->mutex); + + pthread_mutex_destroy(&nfp_bond->mutex); + + rte_free(nfp_bond); + app_fw_flower->nfp_bond = NULL; +} diff --git a/drivers/net/nfp/flower/nfp_flower_bond.h b/drivers/net/nfp/flower/nfp_flower_bond.h new file mode 100644 index 0000000000..be79764a23 --- /dev/null +++ b/drivers/net/nfp/flower/nfp_flower_bond.h @@ -0,0 +1,65 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright(c) 2023 Corigine, Inc. + * All rights reserved. + */ + +#ifndef __NFP_FLOWER_BOND_H__ +#define __NFP_FLOWER_BOND_H__ + +#include <pthread.h> +#include <stdbool.h> +#include <sys/queue.h> + +/* The batch version of bond offload packets between firmware and driver */ +#define NFP_FL_BOND_VERSION_MASK 0x007fffff /* [0, 22] */ + +#define NFP_FL_ENABLE_BOND RTE_BIT32(1) + +/* ID 0 reserved and IDs 1 to 31 are valid */ +#define NFP_FL_BOND_GROUP_MIN 1 +#define NFP_FL_BOND_GROUP_MAX 32 + +/* List entry for each bond group */ +struct nfp_bond_group { + /** List entry */ + LIST_ENTRY(nfp_bond_group) next; + /** Marked if the group needs synced to HW */ + bool dirty; + /** Marked if the group is currently offloaded to NIC */ + bool offloaded; + /** Marked if the group should be removed from NIC */ + bool to_remove; + /** Marked if the group should be removed from driver */ + bool to_destroy; + /** Assigned group ID for host/kernel sync */ + uint32_t group_id; + /** Number of members in group */ + uint32_t member_cnt; + /** Group instance in case of ID reuse */ + uint32_t group_inst; + /** Group main Netdev */ + struct rte_eth_dev *main_dev; +}; + +/* Flower APP priv data for bond offload */ +struct nfp_flower_bond { + /** Marker to reset firmware bond config */ + bool rst_cfg; + /** List of all main/member groups offloaded */ + LIST_HEAD(, nfp_bond_group) group_list; + /** Lock to protect bond group_list */ + pthread_mutex_t mutex; + /** Incremented for each batch of config packets */ + uint32_t batch_ver; + /** Instance allocator for groups */ + uint32_t global_inst; + /** Incremented for each config packet sent */ + uint32_t pkt_num; + /** Pointer to the flower app */ + struct nfp_app_fw_flower *app_fw_flower; +}; + +int nfp_flower_bond_init(struct nfp_app_fw_flower *app_fw_flower); +void nfp_flower_bond_cleanup(struct nfp_app_fw_flower *app_fw_flower); + +#endif /* __NFP_FLOWER_BOND_H__ */ diff --git a/drivers/net/nfp/meson.build b/drivers/net/nfp/meson.build index 46be6f60cd..f0d4bbcecb 100644 --- a/drivers/net/nfp/meson.build +++ b/drivers/net/nfp/meson.build @@ -13,6 +13,7 @@ sources = files( 'flower/nfp_flower_ctrl.c', 'flower/nfp_flower_flow.c', 'flower/nfp_flower_representor.c', + 'flower/nfp_flower_bond.c', 'nfd3/nfp_nfd3_dp.c', 'nfdk/nfp_nfdk_dp.c', 'nfpcore/nfp_cppcore.c', -- 2.39.1 ^ permalink raw reply [flat|nested] 80+ messages in thread
* [PATCH v6 09/14] net/nfp: reset bond configuration of firmware 2023-12-26 7:28 ` [PATCH v6 00/14] Enhance the bond framework to support offload Chaoyong He ` (7 preceding siblings ...) 2023-12-26 7:28 ` [PATCH v6 08/14] net/nfp: add bond firmware creation initialization Chaoyong He @ 2023-12-26 7:28 ` Chaoyong He 2023-12-26 7:28 ` [PATCH v6 10/14] net/nfp: handle link event of bond firmware creation Chaoyong He ` (5 subsequent siblings) 14 siblings, 0 replies; 80+ messages in thread From: Chaoyong He @ 2023-12-26 7:28 UTC (permalink / raw) To: dev; +Cc: oss-drivers, Long Wu, Peng Zhang, Chaoyong He From: Long Wu <long.wu@corigine.com> Driver sends control message to reset the bond firmware configuration in flower NIC initialization. Firmware should reset bond configuration to avoid strange problems caused by residues. Signed-off-by: Long Wu <long.wu@corigine.com> Reviewed-by: Peng Zhang <peng.zhang@corigine.com> Reviewed-by: Chaoyong He <chaoyong.he@corigine.com> --- drivers/net/nfp/flower/nfp_flower.c | 23 ++++++ drivers/net/nfp/flower/nfp_flower_bond.c | 90 ++++++++++++++++++++++++ drivers/net/nfp/flower/nfp_flower_bond.h | 54 ++++++++++++++ drivers/net/nfp/flower/nfp_flower_cmsg.c | 35 +++++++++ drivers/net/nfp/flower/nfp_flower_cmsg.h | 3 + 5 files changed, 205 insertions(+) diff --git a/drivers/net/nfp/flower/nfp_flower.c b/drivers/net/nfp/flower/nfp_flower.c index 195960e00d..fc3ea84828 100644 --- a/drivers/net/nfp/flower/nfp_flower.c +++ b/drivers/net/nfp/flower/nfp_flower.c @@ -589,6 +589,22 @@ nfp_flower_sync_feature_cleanup(struct nfp_app_fw_flower *app_fw_flower) nfp_flower_bond_feature_cleanup(app_fw_flower); } +static int +nfp_flower_start_features(struct nfp_app_fw_flower *app_flower) +{ + int ret; + + if (nfp_flower_support_bond_offload(app_flower)) { + ret = nfp_flower_bond_reset(app_flower->nfp_bond); + if (ret != 0) { + PMD_INIT_LOG(ERR, "Reset bond feature failed"); + return ret; + } + } + + return 0; +} + static int nfp_flower_start_ctrl_vnic(struct nfp_net_hw *net_hw) { @@ -826,6 +842,13 @@ nfp_init_app_fw_flower(struct nfp_pf_dev *pf_dev, goto ctrl_vnic_cleanup; } + /* Start up some features */ + ret = nfp_flower_start_features(app_fw_flower); + if (ret != 0) { + PMD_INIT_LOG(ERR, "Failed to start features"); + goto sync_feature_cleanup; + } + /* Start up flower services */ ret = nfp_flower_enable_services(app_fw_flower); if (ret != 0) { diff --git a/drivers/net/nfp/flower/nfp_flower_bond.c b/drivers/net/nfp/flower/nfp_flower_bond.c index bbd2818e68..4ac27f117c 100644 --- a/drivers/net/nfp/flower/nfp_flower_bond.c +++ b/drivers/net/nfp/flower/nfp_flower_bond.c @@ -6,8 +6,31 @@ #include <rte_malloc.h> +#include "nfp_flower_cmsg.h" #include "nfp_flower_representor.h" +static void +nfp_fl_bond_cmsg_args_init(struct nfp_flower_bond_cmsg_args *cmsg_args, + struct nfp_bond_group *group, + struct rte_eth_dev **active_members, + uint32_t member_cnt, + enum nfp_flower_bond_batch batch) +{ + cmsg_args->group = group; + cmsg_args->active_members = active_members; + cmsg_args->member_cnt = member_cnt; + cmsg_args->batch = batch; +} + +static uint32_t +nfp_fl_get_next_pkt_number(struct nfp_flower_bond *nfp_bond) +{ + nfp_bond->pkt_num++; + nfp_bond->pkt_num &= NFP_FL_BOND_PKT_NUMBER_MASK; + + return nfp_bond->pkt_num; +} + static void nfp_flower_bond_increment_version(struct nfp_flower_bond *nfp_bond) { @@ -57,3 +80,70 @@ nfp_flower_bond_cleanup(struct nfp_app_fw_flower *app_fw_flower) rte_free(nfp_bond); app_fw_flower->nfp_bond = NULL; } + +int +nfp_flower_bond_reset(struct nfp_flower_bond *nfp_bond) +{ + struct nfp_app_fw_flower *app_flower; + enum nfp_flower_bond_batch batch = NFP_FLOWER_BOND_BATCH_FIRST; + struct nfp_flower_bond_cmsg_args init_args; + + app_flower = nfp_bond->app_fw_flower; + app_flower->nfp_bond->rst_cfg = true; + + nfp_fl_bond_cmsg_args_init(&init_args, NULL, NULL, 0, batch); + + return nfp_flower_cmsg_bond_config_group(app_flower, &init_args, &batch); +} + +enum nfp_flower_bond_batch +nfp_flower_bond_cmsg_payload(struct nfp_flower_bond *nfp_bond, + struct nfp_flower_cmsg_bond_config *msg, + struct nfp_flower_bond_cmsg_args *init_args) +{ + uint32_t i; + uint8_t flags = 0; + struct nfp_flower_representor *repr; + enum nfp_flower_bond_batch batch = init_args->batch; + + /* Increment batch version for each new batch of config messages. */ + if (batch == NFP_FLOWER_BOND_BATCH_FIRST) { + flags |= NFP_FL_BOND_FIRST; + nfp_flower_bond_increment_version(nfp_bond); + batch = NFP_FLOWER_BOND_BATCH_MEMBER; + } + + /* If it is a reset msg then it is also the end of the batch. */ + if (nfp_bond->rst_cfg) { + flags |= NFP_FL_BOND_RESET; + batch = NFP_FLOWER_BOND_BATCH_FINISHED; + } + + /* + * To signal the end of a batch, both the switch and last flags are set + * and the reserved SYNC group ID is used. + */ + if (batch == NFP_FLOWER_BOND_BATCH_FINISHED) { + flags |= NFP_FL_BOND_SWITCH | NFP_FL_BOND_LAST; + nfp_bond->rst_cfg = false; + msg->group_id = rte_cpu_to_be_32(NFP_FL_BOND_SYNC_ID); + msg->group_inst = 0; + } else { + msg->group_id = rte_cpu_to_be_32(init_args->group->group_id); + msg->group_inst = rte_cpu_to_be_32(init_args->group->group_inst); + } + + msg->reserved[0] = 0; + msg->reserved[1] = 0; + msg->ttl = NFP_FL_BOND_HOST_TTL; + msg->ctrl_flags = flags; + msg->batch_ver = rte_cpu_to_be_32(nfp_bond->batch_ver); + msg->pkt_number = rte_cpu_to_be_32(nfp_fl_get_next_pkt_number(nfp_bond)); + + for (i = 0; i < init_args->member_cnt; i++) { + repr = init_args->active_members[i]->data->dev_private; + msg->members[i] = rte_cpu_to_be_32(repr->port_id); + } + + return batch; +} diff --git a/drivers/net/nfp/flower/nfp_flower_bond.h b/drivers/net/nfp/flower/nfp_flower_bond.h index be79764a23..e15d9e09d5 100644 --- a/drivers/net/nfp/flower/nfp_flower_bond.h +++ b/drivers/net/nfp/flower/nfp_flower_bond.h @@ -7,18 +7,61 @@ #define __NFP_FLOWER_BOND_H__ #include <pthread.h> +#include <rte_byteorder.h> #include <stdbool.h> #include <sys/queue.h> /* The batch version of bond offload packets between firmware and driver */ #define NFP_FL_BOND_VERSION_MASK 0x007fffff /* [0, 22] */ +#define NFP_FL_BOND_PKT_NUMBER_MASK 0x7fffffff /* [0, 30] */ + #define NFP_FL_ENABLE_BOND RTE_BIT32(1) /* ID 0 reserved and IDs 1 to 31 are valid */ #define NFP_FL_BOND_GROUP_MIN 1 #define NFP_FL_BOND_GROUP_MAX 32 +#define NFP_FL_BOND_HOST_TTL 0xff + +/* Use this ID with zero members to ack a batch config */ +#define NFP_FL_BOND_SYNC_ID 0 + +/* BOND group config flags */ +#define NFP_FL_BOND_LAST RTE_BIT32(1) +#define NFP_FL_BOND_FIRST RTE_BIT32(2) +#define NFP_FL_BOND_DATA RTE_BIT32(3) +#define NFP_FL_BOND_XON RTE_BIT32(4) +#define NFP_FL_BOND_SYNC RTE_BIT32(5) +#define NFP_FL_BOND_SWITCH RTE_BIT32(6) +#define NFP_FL_BOND_RESET RTE_BIT32(7) + +enum nfp_flower_bond_batch { + NFP_FLOWER_BOND_BATCH_FIRST, + NFP_FLOWER_BOND_BATCH_MEMBER, + NFP_FLOWER_BOND_BATCH_FINISHED +}; + +/* Control message payload for bond config */ +struct nfp_flower_cmsg_bond_config { + /** Configuration flags */ + uint8_t ctrl_flags; + /** Reserved for future use */ + uint8_t reserved[2]; + /** Time to live of packet - host always sets to 0xff */ + uint8_t ttl; + /** Config message packet number - increment for each message */ + rte_be32_t pkt_number; + /** Batch version of messages - increment for each batch of messages */ + rte_be32_t batch_ver; + /** Group ID applicable */ + rte_be32_t group_id; + /** Group instance number - increment when group is reused */ + rte_be32_t group_inst; + /** Array of 32-bit words listing all active group members */ + rte_be32_t members[]; +}; + /* List entry for each bond group */ struct nfp_bond_group { /** List entry */ @@ -59,7 +102,18 @@ struct nfp_flower_bond { struct nfp_app_fw_flower *app_fw_flower; }; +struct nfp_flower_bond_cmsg_args { + struct nfp_bond_group *group; + struct rte_eth_dev **active_members; + uint32_t member_cnt; + enum nfp_flower_bond_batch batch; +}; + int nfp_flower_bond_init(struct nfp_app_fw_flower *app_fw_flower); void nfp_flower_bond_cleanup(struct nfp_app_fw_flower *app_fw_flower); +int nfp_flower_bond_reset(struct nfp_flower_bond *nfp_bond); +enum nfp_flower_bond_batch nfp_flower_bond_cmsg_payload(struct nfp_flower_bond *nfp_bond, + struct nfp_flower_cmsg_bond_config *msg, + struct nfp_flower_bond_cmsg_args *init_args); #endif /* __NFP_FLOWER_BOND_H__ */ diff --git a/drivers/net/nfp/flower/nfp_flower_cmsg.c b/drivers/net/nfp/flower/nfp_flower_cmsg.c index 8effe9474d..0cf1bf2281 100644 --- a/drivers/net/nfp/flower/nfp_flower_cmsg.c +++ b/drivers/net/nfp/flower/nfp_flower_cmsg.c @@ -567,3 +567,38 @@ nfp_flower_cmsg_qos_stats(struct nfp_app_fw_flower *app_fw_flower, return 0; } + +int +nfp_flower_cmsg_bond_config_group(struct nfp_app_fw_flower *app_flower, + struct nfp_flower_bond_cmsg_args *init_args, + enum nfp_flower_bond_batch *batch_out) +{ + uint16_t cnt; + uint32_t size; + struct rte_mbuf *mbuf; + struct nfp_flower_bond *nfp_bond; + struct nfp_flower_cmsg_bond_config *msg; + + mbuf = rte_pktmbuf_alloc(app_flower->ctrl_pktmbuf_pool); + if (mbuf == NULL) { + PMD_DRV_LOG(DEBUG, "Alloc mbuf for bond config failed"); + return -ENOMEM; + } + + size = sizeof(*msg) + sizeof(rte_be32_t) * init_args->member_cnt; + msg = nfp_flower_cmsg_init(app_flower, mbuf, + NFP_FLOWER_CMSG_TYPE_LAG_CONFIG, size); + + nfp_bond = app_flower->nfp_bond; + + *batch_out = nfp_flower_bond_cmsg_payload(nfp_bond, msg, init_args); + + cnt = nfp_flower_ctrl_vnic_xmit(app_flower, mbuf); + if (cnt == 0) { + PMD_DRV_LOG(ERR, "Send cmsg through ctrl vnic failed."); + rte_pktmbuf_free(mbuf); + return -EIO; + } + + return 0; +} diff --git a/drivers/net/nfp/flower/nfp_flower_cmsg.h b/drivers/net/nfp/flower/nfp_flower_cmsg.h index 45543816ae..60ab58a3b1 100644 --- a/drivers/net/nfp/flower/nfp_flower_cmsg.h +++ b/drivers/net/nfp/flower/nfp_flower_cmsg.h @@ -988,5 +988,8 @@ int nfp_flower_cmsg_qos_delete(struct nfp_app_fw_flower *app_fw_flower, struct nfp_profile_conf *conf); int nfp_flower_cmsg_qos_stats(struct nfp_app_fw_flower *app_fw_flower, struct nfp_cfg_head *head); +int nfp_flower_cmsg_bond_config_group(struct nfp_app_fw_flower *app_flower, + struct nfp_flower_bond_cmsg_args *init_args, + enum nfp_flower_bond_batch *batch_out); #endif /* __NFP_CMSG_H__ */ -- 2.39.1 ^ permalink raw reply [flat|nested] 80+ messages in thread
* [PATCH v6 10/14] net/nfp: handle link event of bond firmware creation 2023-12-26 7:28 ` [PATCH v6 00/14] Enhance the bond framework to support offload Chaoyong He ` (8 preceding siblings ...) 2023-12-26 7:28 ` [PATCH v6 09/14] net/nfp: reset bond configuration of firmware Chaoyong He @ 2023-12-26 7:28 ` Chaoyong He 2023-12-26 7:28 ` [PATCH v6 11/14] net/nfp: support bond member notification Chaoyong He ` (4 subsequent siblings) 14 siblings, 0 replies; 80+ messages in thread From: Chaoyong He @ 2023-12-26 7:28 UTC (permalink / raw) To: dev; +Cc: oss-drivers, Long Wu, Peng Zhang, Chaoyong He From: Long Wu <long.wu@corigine.com> If NFP physical representor port is a member port of bonding port, its link status changed and firmware has created the bonding port, driver will record the link status and send control message to notify firmware. Signed-off-by: Long Wu <long.wu@corigine.com> Reviewed-by: Peng Zhang <peng.zhang@corigine.com> Reviewed-by: Chaoyong He <chaoyong.he@corigine.com> --- drivers/net/nfp/flower/nfp_flower_bond.c | 259 ++++++++++++++++++ drivers/net/nfp/flower/nfp_flower_bond.h | 22 ++ .../net/nfp/flower/nfp_flower_representor.c | 13 + .../net/nfp/flower/nfp_flower_representor.h | 2 + drivers/net/nfp/meson.build | 2 +- drivers/net/nfp/nfp_net_common.c | 25 ++ drivers/net/nfp/nfp_net_common.h | 1 - 7 files changed, 322 insertions(+), 2 deletions(-) diff --git a/drivers/net/nfp/flower/nfp_flower_bond.c b/drivers/net/nfp/flower/nfp_flower_bond.c index 4ac27f117c..c814c0d4ea 100644 --- a/drivers/net/nfp/flower/nfp_flower_bond.c +++ b/drivers/net/nfp/flower/nfp_flower_bond.c @@ -4,8 +4,11 @@ */ #include "nfp_flower_bond.h" +#include <eth_bond_private.h> +#include <rte_eth_bond.h> #include <rte_malloc.h> +#include "../nfp_logs.h" #include "nfp_flower_cmsg.h" #include "nfp_flower_representor.h" @@ -43,6 +46,72 @@ nfp_flower_bond_increment_version(struct nfp_flower_bond *nfp_bond) nfp_bond->batch_ver += 2; } +static enum nfp_flower_bond_batch +nfp_flower_bond_remove_node(struct nfp_flower_bond *nfp_bond, + struct nfp_bond_group *entry, + enum nfp_flower_bond_batch batch) +{ + enum nfp_flower_bond_batch batch_out; + struct nfp_flower_bond_cmsg_args cmsg_args; + struct nfp_app_fw_flower *app_flower = nfp_bond->app_fw_flower; + + nfp_fl_bond_cmsg_args_init(&cmsg_args, entry, NULL, 0, batch); + if (nfp_flower_cmsg_bond_config_group(app_flower, &cmsg_args, &batch_out) != 0) { + PMD_DRV_LOG(ERR, "group remove failed."); + return batch_out; + } + + entry->to_remove = false; + entry->offloaded = false; + + if (entry->to_destroy) { + LIST_REMOVE(entry, next); + rte_free(entry); + } + + return batch_out; +} + +static void +nfp_flower_bond_member_work_status(struct nfp_bond_group *entry, + struct rte_eth_dev **active_eth_devs, + uint32_t *active) +{ + uint32_t i; + uint32_t *flags; + uint32_t active_count; + uint16_t member_port_id; + struct rte_eth_dev *eth_dev; + struct bond_dev_private *internals; + struct nfp_flower_representor *repr; + + active_count = 0; + + internals = entry->main_dev->data->dev_private; + for (i = 0; i < internals->member_count; i++) { + member_port_id = internals->members[i].port_id; + if (internals->current_primary_port == member_port_id || + internals->mode != BONDING_MODE_ACTIVE_BACKUP) { + eth_dev = &rte_eth_devices[member_port_id]; + repr = eth_dev->data->dev_private; + flags = &repr->bond_port_flags; + + if ((*flags & NFP_FL_BOND_PORT_CHANGED) != 0) { + *flags &= ~NFP_FL_BOND_PORT_CHANGED; + entry->dirty = true; + } + + if ((*flags & NFP_FL_BOND_PORT_TX_ENABLED) != 0 && + (*flags & NFP_FL_BOND_PORT_LINK_UP) != 0) { + active_eth_devs[active_count] = eth_dev; + active_count++; + } + } + } + + *active = active_count; +} + int nfp_flower_bond_init(struct nfp_app_fw_flower *app_fw_flower) { @@ -96,6 +165,132 @@ nfp_flower_bond_reset(struct nfp_flower_bond *nfp_bond) return nfp_flower_cmsg_bond_config_group(app_flower, &init_args, &batch); } +struct nfp_bond_group * +nfp_flower_bond_find_group(struct nfp_flower_bond *nfp_bond, + const struct rte_eth_dev *bond_dev) +{ + struct nfp_bond_group *group; + + LIST_FOREACH(group, &nfp_bond->group_list, next) { + if (group->main_dev == bond_dev) + return group; + } + + return NULL; +} + +void +nfp_flower_bond_do_work(struct nfp_flower_bond *nfp_bond) +{ + int ret; + uint32_t active_count; + struct nfp_bond_group *entry; + struct nfp_app_fw_flower *app_flower; + struct nfp_flower_bond_cmsg_args cmsg_args; + struct rte_eth_dev *active_eth_devs[RTE_MAX_ETHPORTS]; + enum nfp_flower_bond_batch batch = NFP_FLOWER_BOND_BATCH_FIRST; + + app_flower = nfp_bond->app_fw_flower; + + pthread_mutex_lock(&nfp_bond->mutex); + + LIST_FOREACH(entry, &nfp_bond->group_list, next) { + if (entry->to_remove) { + batch = nfp_flower_bond_remove_node(nfp_bond, entry, batch); + continue; + } + + nfp_flower_bond_member_work_status(entry, active_eth_devs, + &active_count); + + if (!entry->dirty) + continue; + + nfp_fl_bond_cmsg_args_init(&cmsg_args, entry, active_eth_devs, + active_count, batch); + + ret = nfp_flower_cmsg_bond_config_group(app_flower, &cmsg_args, &batch); + if (ret == 0) { + entry->offloaded = true; + entry->dirty = false; + } else { + PMD_DRV_LOG(DEBUG, "Group config failed."); + } + } + + /* End the config batch if at least one packet has been batched. */ + if (batch == NFP_FLOWER_BOND_BATCH_MEMBER) { + batch = NFP_FLOWER_BOND_BATCH_FINISHED; + nfp_fl_bond_cmsg_args_init(&cmsg_args, NULL, NULL, 0, batch); + ret = nfp_flower_cmsg_bond_config_group(app_flower, &cmsg_args, &batch); + if (ret != 0) + PMD_DRV_LOG(DEBUG, "Group batch end cmsg failed"); + } + + pthread_mutex_unlock(&nfp_bond->mutex); + + /* + * A batch of packets rx in firmware has been set for at least two timing + * units, so trigger copying at next opportunity to limit latency from + * receiving data to being available for use. + */ + rte_delay_ms(1); +} + +static void +nfp_flower_bond_change_linkstatus_event(struct nfp_flower_bond *nfp_bond, + struct rte_eth_dev *bond_dev, + struct rte_eth_dev *nfp_dev) +{ + uint32_t *bond_port_flags; + struct nfp_bond_group *group; + struct nfp_flower_representor *repr; + + if (!nfp_flower_bond_is_member_port(bond_dev, nfp_dev)) + return; + + pthread_mutex_lock(&nfp_bond->mutex); + + group = nfp_flower_bond_find_group(nfp_bond, bond_dev); + if (group == NULL) { + pthread_mutex_unlock(&nfp_bond->mutex); + return; + } + + pthread_mutex_unlock(&nfp_bond->mutex); + + repr = nfp_dev->data->dev_private; + bond_port_flags = &repr->bond_port_flags; + + if (repr->link.link_status == RTE_ETH_LINK_UP) + *bond_port_flags |= NFP_FL_BOND_PORT_LINK_UP; + else + *bond_port_flags &= ~NFP_FL_BOND_PORT_LINK_UP; + + *bond_port_flags |= NFP_FL_BOND_PORT_TX_ENABLED; + *bond_port_flags |= NFP_FL_BOND_PORT_CHANGED; +} + +int +nfp_flower_bond_event_handle(struct nfp_flower_bond *nfp_bond, + struct rte_eth_dev *bond_dev, + struct rte_eth_dev *nfp_dev, + enum nfp_flower_bond_event event) +{ + switch (event) { + case NFP_FLOWER_CHANGELINKSTATUS: + nfp_flower_bond_change_linkstatus_event(nfp_bond, bond_dev, nfp_dev); + break; + default: + PMD_DRV_LOG(ERR, "Invalid bond offload event."); + return -1; + } + + nfp_flower_bond_do_work(nfp_bond); + + return 0; +} + enum nfp_flower_bond_batch nfp_flower_bond_cmsg_payload(struct nfp_flower_bond *nfp_bond, struct nfp_flower_cmsg_bond_config *msg, @@ -147,3 +342,67 @@ nfp_flower_bond_cmsg_payload(struct nfp_flower_bond *nfp_bond, return batch; } + +bool +nfp_flower_bond_is_member_port(struct rte_eth_dev *bond_dev, + struct rte_eth_dev *nfp_dev) +{ + uint32_t i; + struct bond_dev_private *internals; + + if (!nfp_flower_is_phy_repr(nfp_dev)) + return false; + + internals = bond_dev->data->dev_private; + + for (i = 0; i < internals->member_count; i++) { + if (nfp_dev == &rte_eth_devices[internals->members[i].port_id]) + return true; + } + + return false; +} + +struct rte_eth_dev * +nfp_flower_bond_find_bond_device(struct rte_eth_dev *nfp_dev) +{ + struct nfp_bond_group *group; + struct nfp_flower_representor *repr; + struct rte_eth_dev *bond_dev = NULL; + + if (!nfp_flower_is_phy_repr(nfp_dev)) + return NULL; + + repr = nfp_dev->data->dev_private; + + pthread_mutex_lock(&repr->app_fw_flower->nfp_bond->mutex); + LIST_FOREACH(group, &repr->app_fw_flower->nfp_bond->group_list, next) { + if (nfp_flower_bond_is_member_port(group->main_dev, nfp_dev)) { + bond_dev = group->main_dev; + break; + } + } + pthread_mutex_unlock(&repr->app_fw_flower->nfp_bond->mutex); + + return bond_dev; +} + +bool +nfp_flower_bond_all_member_are_phyrepr(struct rte_eth_dev *bond_dev) +{ + uint32_t i; + struct rte_eth_dev *eth_dev; + struct bond_dev_private *internals; + + if (bond_dev == NULL) + return false; + + internals = bond_dev->data->dev_private; + for (i = 0; i < internals->member_count; i++) { + eth_dev = &rte_eth_devices[internals->members[i].port_id]; + if (!nfp_flower_is_phy_repr(eth_dev)) + return false; + } + + return true; +} diff --git a/drivers/net/nfp/flower/nfp_flower_bond.h b/drivers/net/nfp/flower/nfp_flower_bond.h index e15d9e09d5..e4a09b3427 100644 --- a/drivers/net/nfp/flower/nfp_flower_bond.h +++ b/drivers/net/nfp/flower/nfp_flower_bond.h @@ -36,12 +36,23 @@ #define NFP_FL_BOND_SWITCH RTE_BIT32(6) #define NFP_FL_BOND_RESET RTE_BIT32(7) +/* BOND port state flags. */ +#define NFP_FL_BOND_PORT_LINK_UP RTE_BIT32(0) +#define NFP_FL_BOND_PORT_TX_ENABLED RTE_BIT32(1) +#define NFP_FL_BOND_PORT_CHANGED RTE_BIT32(2) + enum nfp_flower_bond_batch { NFP_FLOWER_BOND_BATCH_FIRST, NFP_FLOWER_BOND_BATCH_MEMBER, NFP_FLOWER_BOND_BATCH_FINISHED }; +enum nfp_flower_bond_event { + NFP_FLOWER_CHANGEUPPER, + NFP_FLOWER_CHANGELINKSTATUS, + NFP_FLOWER_UNREGISTER +}; + /* Control message payload for bond config */ struct nfp_flower_cmsg_bond_config { /** Configuration flags */ @@ -115,5 +126,16 @@ int nfp_flower_bond_reset(struct nfp_flower_bond *nfp_bond); enum nfp_flower_bond_batch nfp_flower_bond_cmsg_payload(struct nfp_flower_bond *nfp_bond, struct nfp_flower_cmsg_bond_config *msg, struct nfp_flower_bond_cmsg_args *init_args); +struct rte_eth_dev *nfp_flower_bond_find_bond_device(struct rte_eth_dev *nfp_dev); +bool nfp_flower_bond_is_member_port(struct rte_eth_dev *bond_dev, + struct rte_eth_dev *nfp_dev); +struct nfp_bond_group *nfp_flower_bond_find_group(struct nfp_flower_bond *nfp_bond, + const struct rte_eth_dev *bond_dev); +int nfp_flower_bond_event_handle(struct nfp_flower_bond *nfp_bond, + struct rte_eth_dev *bond_dev, + struct rte_eth_dev *nfp_dev, + enum nfp_flower_bond_event event); +void nfp_flower_bond_do_work(struct nfp_flower_bond *nfp_bond); +bool nfp_flower_bond_all_member_are_phyrepr(struct rte_eth_dev *bond_dev); #endif /* __NFP_FLOWER_BOND_H__ */ diff --git a/drivers/net/nfp/flower/nfp_flower_representor.c b/drivers/net/nfp/flower/nfp_flower_representor.c index 7d8c055b80..2810a7a271 100644 --- a/drivers/net/nfp/flower/nfp_flower_representor.c +++ b/drivers/net/nfp/flower/nfp_flower_representor.c @@ -827,3 +827,16 @@ nfp_flower_repr_create(struct nfp_app_fw_flower *app_fw_flower) return ret; } + +bool +nfp_flower_is_phy_repr(struct rte_eth_dev *eth_dev) +{ + struct nfp_flower_representor *repr; + + if ((eth_dev->data->dev_flags & RTE_ETH_DEV_REPRESENTOR) != 0) { + repr = eth_dev->data->dev_private; + return repr->repr_type == NFP_REPR_TYPE_PHYS_PORT; + } + + return false; +} diff --git a/drivers/net/nfp/flower/nfp_flower_representor.h b/drivers/net/nfp/flower/nfp_flower_representor.h index 8053617562..164e2f1322 100644 --- a/drivers/net/nfp/flower/nfp_flower_representor.h +++ b/drivers/net/nfp/flower/nfp_flower_representor.h @@ -14,6 +14,7 @@ struct nfp_flower_representor { uint32_t repr_type; uint32_t port_id; uint32_t nfp_idx; /**< Only valid for the repr of physical port */ + uint32_t bond_port_flags; char name[RTE_ETH_NAME_MAX_LEN]; struct rte_ether_addr mac_addr; struct nfp_app_fw_flower *app_fw_flower; @@ -24,5 +25,6 @@ struct nfp_flower_representor { }; int nfp_flower_repr_create(struct nfp_app_fw_flower *app_fw_flower); +bool nfp_flower_is_phy_repr(struct rte_eth_dev *eth_dev); #endif /* __NFP_FLOWER_REPRESENTOR_H__ */ diff --git a/drivers/net/nfp/meson.build b/drivers/net/nfp/meson.build index f0d4bbcecb..c309505c83 100644 --- a/drivers/net/nfp/meson.build +++ b/drivers/net/nfp/meson.build @@ -42,4 +42,4 @@ sources = files( 'nfp_rxtx.c', ) -deps += ['hash', 'security', 'common_nfp'] +deps += ['hash', 'security', 'common_nfp', 'net_bond'] diff --git a/drivers/net/nfp/nfp_net_common.c b/drivers/net/nfp/nfp_net_common.c index a438eb5871..810903c033 100644 --- a/drivers/net/nfp/nfp_net_common.c +++ b/drivers/net/nfp/nfp_net_common.c @@ -9,6 +9,7 @@ #include <rte_alarm.h> +#include "flower/nfp_flower.h" #include "flower/nfp_flower_representor.h" #include "nfd3/nfp_nfd3.h" #include "nfdk/nfp_nfdk.h" @@ -719,6 +720,23 @@ nfp_net_speed_aneg_update(struct rte_eth_dev *dev, return 0; } +static void +nfp_net_bond_link_event_notify(struct rte_eth_dev *dev) +{ + struct rte_eth_dev *bond_dev; + struct nfp_flower_representor *repr = dev->data->dev_private; + + if (!nfp_flower_support_bond_offload(repr->app_fw_flower)) + return; + + bond_dev = nfp_flower_bond_find_bond_device(dev); + if (!nfp_flower_bond_all_member_are_phyrepr(bond_dev)) + return; + + nfp_flower_bond_event_handle(repr->app_fw_flower->nfp_bond, bond_dev, + dev, NFP_FLOWER_CHANGELINKSTATUS); +} + int nfp_net_link_update_common(struct rte_eth_dev *dev, struct nfp_net_hw *hw, @@ -753,6 +771,13 @@ nfp_net_link_update_common(struct rte_eth_dev *dev, PMD_DRV_LOG(INFO, "NIC Link is Up"); else PMD_DRV_LOG(INFO, "NIC Link is Down"); + + /* + * Link status changed and if repr is a member port in bond device, + * we need to call func to do something special. + */ + if ((dev->data->dev_flags & RTE_ETH_DEV_REPRESENTOR) != 0) + nfp_net_bond_link_event_notify(dev); } return ret; diff --git a/drivers/net/nfp/nfp_net_common.h b/drivers/net/nfp/nfp_net_common.h index 66c900e3b8..9d6cf4b016 100644 --- a/drivers/net/nfp/nfp_net_common.h +++ b/drivers/net/nfp/nfp_net_common.h @@ -295,7 +295,6 @@ int nfp_net_fec_get(struct rte_eth_dev *dev, uint32_t *fec_capa); int nfp_net_fec_set(struct rte_eth_dev *dev, uint32_t fec_capa); - #define NFP_PRIV_TO_APP_FW_NIC(app_fw_priv)\ ((struct nfp_app_fw_nic *)app_fw_priv) -- 2.39.1 ^ permalink raw reply [flat|nested] 80+ messages in thread
* [PATCH v6 11/14] net/nfp: support bond member notification 2023-12-26 7:28 ` [PATCH v6 00/14] Enhance the bond framework to support offload Chaoyong He ` (9 preceding siblings ...) 2023-12-26 7:28 ` [PATCH v6 10/14] net/nfp: handle link event of bond firmware creation Chaoyong He @ 2023-12-26 7:28 ` Chaoyong He 2023-12-26 7:28 ` [PATCH v6 12/14] net/nfp: handle bond packets from firmware Chaoyong He ` (3 subsequent siblings) 14 siblings, 0 replies; 80+ messages in thread From: Chaoyong He @ 2023-12-26 7:28 UTC (permalink / raw) To: dev; +Cc: oss-drivers, Long Wu, Peng Zhang, Chaoyong He From: Long Wu <long.wu@corigine.com> A NFP representor port can receive a bond notification. Then driver will parse this notification into one of these two events: 1. Bonding port configuration may have changed. This includes creation of a bonding port, removal/addition of a member port, changing the bond mode, etc. 2. Bonding port is deleted. Signed-off-by: Long Wu <long.wu@corigine.com> Reviewed-by: Peng Zhang <peng.zhang@corigine.com> Reviewed-by: Chaoyong He <chaoyong.he@corigine.com> --- drivers/net/nfp/flower/nfp_flower_bond.c | 183 +++++++++++++++++- drivers/net/nfp/flower/nfp_flower_bond.h | 2 + .../net/nfp/flower/nfp_flower_representor.c | 21 ++ 3 files changed, 205 insertions(+), 1 deletion(-) diff --git a/drivers/net/nfp/flower/nfp_flower_bond.c b/drivers/net/nfp/flower/nfp_flower_bond.c index c814c0d4ea..523e0025ad 100644 --- a/drivers/net/nfp/flower/nfp_flower_bond.c +++ b/drivers/net/nfp/flower/nfp_flower_bond.c @@ -46,6 +46,39 @@ nfp_flower_bond_increment_version(struct nfp_flower_bond *nfp_bond) nfp_bond->batch_ver += 2; } +static void +nfp_flower_bond_group_id_clear(struct nfp_flower_bond *nfp_bond, + uint32_t id_clear) +{ + bool *group_id_map = nfp_bond->group_id_map; + + if (id_clear >= NFP_FL_BOND_GROUP_MAX || id_clear < NFP_FL_BOND_GROUP_MIN) { + PMD_DRV_LOG(ERR, "Try to clear invalid group id %u.", id_clear); + return; + } + + if (group_id_map[id_clear]) + group_id_map[id_clear] = false; +} + +static int +nfp_flower_bond_group_id_get(struct nfp_flower_bond *nfp_bond, + uint32_t *id_ret) +{ + uint32_t id; + bool *group_id_map = nfp_bond->group_id_map; + + for (id = NFP_FL_BOND_GROUP_MIN; id < NFP_FL_BOND_GROUP_MAX; id++) { + if (!group_id_map[id]) { + group_id_map[id] = true; + *id_ret = id; + return 0; + } + } + + return -ENOSPC; +} + static enum nfp_flower_bond_batch nfp_flower_bond_remove_node(struct nfp_flower_bond *nfp_bond, struct nfp_bond_group *entry, @@ -65,6 +98,7 @@ nfp_flower_bond_remove_node(struct nfp_flower_bond *nfp_bond, entry->offloaded = false; if (entry->to_destroy) { + nfp_flower_bond_group_id_clear(nfp_bond, entry->group_id); LIST_REMOVE(entry, next); rte_free(entry); } @@ -139,6 +173,7 @@ nfp_flower_bond_cleanup(struct nfp_app_fw_flower *app_fw_flower) pthread_mutex_lock(&nfp_bond->mutex); LIST_FOREACH(entry, &nfp_bond->group_list, next) { + nfp_flower_bond_group_id_clear(nfp_bond, entry->group_id); LIST_REMOVE(entry, next); rte_free(entry); } @@ -271,16 +306,162 @@ nfp_flower_bond_change_linkstatus_event(struct nfp_flower_bond *nfp_bond, *bond_port_flags |= NFP_FL_BOND_PORT_CHANGED; } +static struct nfp_bond_group * +nfp_flower_bond_group_create(struct nfp_flower_bond *nfp_bond, + struct rte_eth_dev *bond_dev) +{ + uint32_t id; + unsigned int numa_node; + struct nfp_bond_group *group = NULL; + + if (nfp_flower_bond_group_id_get(nfp_bond, &id) < 0) + return NULL; + + numa_node = rte_socket_id(); + + group = rte_zmalloc_socket(NULL, sizeof(struct nfp_bond_group), + RTE_CACHE_LINE_SIZE, numa_node); + if (group == NULL) { + PMD_DRV_LOG(ERR, "Unable malloc memory for nfp bond group"); + nfp_flower_bond_group_id_clear(nfp_bond, id); + return NULL; + } + + group->group_id = id; + group->main_dev = bond_dev; + group->dirty = true; + group->offloaded = false; + group->to_remove = false; + group->to_destroy = false; + group->member_cnt = 0; + group->group_inst = ++nfp_bond->global_inst; + LIST_INSERT_HEAD(&nfp_bond->group_list, group, next); + + return group; +} + +static int +nfp_flower_bond_changeupper_event(struct nfp_flower_bond *nfp_bond, + struct rte_eth_dev *bond_dev) +{ + uint32_t i; + uint16_t port_id; + bool can_offload = true; + uint16_t nfp_member_count; + struct rte_eth_dev *eth_dev; + struct nfp_bond_group *group; + struct bond_dev_private *internals; + struct nfp_flower_representor *repr; + struct nfp_app_fw_flower *app_flower; + + internals = bond_dev->data->dev_private; + app_flower = nfp_bond->app_fw_flower; + nfp_member_count = 0; + + for (i = 0; i < internals->member_count; i++) { + port_id = internals->members[i].port_id; + eth_dev = &rte_eth_devices[port_id]; + if (!nfp_flower_is_phy_repr(eth_dev)) { + can_offload = false; + break; + } + + repr = eth_dev->data->dev_private; + + if (repr->app_fw_flower != app_flower) { + can_offload = false; + break; + } + + if (internals->current_primary_port == port_id || + internals->mode != BONDING_MODE_ACTIVE_BACKUP) + nfp_member_count++; + } + + if (internals != NULL && + internals->mode != BONDING_MODE_ACTIVE_BACKUP && + ((internals->mode != BONDING_MODE_BALANCE && + internals->mode != BONDING_MODE_8023AD) || + internals->balance_xmit_policy != BALANCE_XMIT_POLICY_LAYER34)) { + can_offload = false; + PMD_DRV_LOG(WARNING, "Unable to offload mode %u hash %u.", + internals->mode, + internals->balance_xmit_policy); + } + + pthread_mutex_lock(&nfp_bond->mutex); + + group = nfp_flower_bond_find_group(nfp_bond, bond_dev); + if (nfp_member_count == 0 || !can_offload) { + if (group != NULL && group->offloaded) + /* Set remove flag */ + group->to_remove = true; + + pthread_mutex_unlock(&nfp_bond->mutex); + return 0; + } + + if (group == NULL) { + group = nfp_flower_bond_group_create(nfp_bond, bond_dev); + if (group == NULL) { + pthread_mutex_unlock(&nfp_bond->mutex); + return -1; + } + } + + group->dirty = true; + group->member_cnt = nfp_member_count; + group->to_remove = false; + + pthread_mutex_unlock(&nfp_bond->mutex); + + return 0; +} + +static void +nfp_flower_bond_group_delete(struct nfp_flower_bond *nfp_bond, + struct rte_eth_dev *bond_dev) +{ + struct nfp_bond_group *group; + + pthread_mutex_lock(&nfp_bond->mutex); + + group = nfp_flower_bond_find_group(nfp_bond, bond_dev); + if (group == NULL) { + pthread_mutex_unlock(&nfp_bond->mutex); + PMD_DRV_LOG(WARNING, "Untracked bond got unregistered %s", + bond_dev->device->name); + return; + } + + group->to_remove = true; + group->to_destroy = true; + + pthread_mutex_unlock(&nfp_bond->mutex); +} + int nfp_flower_bond_event_handle(struct nfp_flower_bond *nfp_bond, struct rte_eth_dev *bond_dev, struct rte_eth_dev *nfp_dev, enum nfp_flower_bond_event event) { + int ret = 0; + switch (event) { + case NFP_FLOWER_CHANGEUPPER: + ret = nfp_flower_bond_changeupper_event(nfp_bond, bond_dev); + if (ret != 0) { + PMD_DRV_LOG(ERR, "Change upper event can not work."); + return ret; + } + break; case NFP_FLOWER_CHANGELINKSTATUS: nfp_flower_bond_change_linkstatus_event(nfp_bond, bond_dev, nfp_dev); break; + case NFP_FLOWER_UNREGISTER: + nfp_flower_bond_group_delete(nfp_bond, bond_dev); + break; default: PMD_DRV_LOG(ERR, "Invalid bond offload event."); return -1; @@ -288,7 +469,7 @@ nfp_flower_bond_event_handle(struct nfp_flower_bond *nfp_bond, nfp_flower_bond_do_work(nfp_bond); - return 0; + return ret; } enum nfp_flower_bond_batch diff --git a/drivers/net/nfp/flower/nfp_flower_bond.h b/drivers/net/nfp/flower/nfp_flower_bond.h index e4a09b3427..12e61ff1ce 100644 --- a/drivers/net/nfp/flower/nfp_flower_bond.h +++ b/drivers/net/nfp/flower/nfp_flower_bond.h @@ -109,6 +109,8 @@ struct nfp_flower_bond { uint32_t global_inst; /** Incremented for each config packet sent */ uint32_t pkt_num; + /** ID of bond group in driver, true is used */ + bool group_id_map[NFP_FL_BOND_GROUP_MAX]; /** Pointer to the flower app */ struct nfp_app_fw_flower *app_fw_flower; }; diff --git a/drivers/net/nfp/flower/nfp_flower_representor.c b/drivers/net/nfp/flower/nfp_flower_representor.c index 2810a7a271..43106da90e 100644 --- a/drivers/net/nfp/flower/nfp_flower_representor.c +++ b/drivers/net/nfp/flower/nfp_flower_representor.c @@ -390,6 +390,25 @@ nfp_flower_repr_dev_close(struct rte_eth_dev *dev) return 0; } +static int +nfp_flower_repr_bond_notify_member(struct rte_eth_dev *dev, + struct rte_eth_dev *bond_dev) +{ + struct nfp_flower_representor *repr; + enum nfp_flower_bond_event event = NFP_FLOWER_CHANGEUPPER; + + if (!nfp_flower_bond_all_member_are_phyrepr(bond_dev)) + event = NFP_FLOWER_UNREGISTER; + + repr = dev->data->dev_private; + + if (nfp_flower_support_bond_offload(repr->app_fw_flower)) + return nfp_flower_bond_event_handle(repr->app_fw_flower->nfp_bond, + bond_dev, dev, event); + + return 0; +} + static const struct eth_dev_ops nfp_flower_pf_repr_dev_ops = { .dev_infos_get = nfp_flower_repr_dev_infos_get, @@ -437,6 +456,8 @@ static const struct eth_dev_ops nfp_flower_repr_dev_ops = { .flow_ops_get = nfp_flow_ops_get, .mtr_ops_get = nfp_net_mtr_ops_get, + + .bond_notify_member = nfp_flower_repr_bond_notify_member, }; static uint32_t -- 2.39.1 ^ permalink raw reply [flat|nested] 80+ messages in thread
* [PATCH v6 12/14] net/nfp: handle bond packets from firmware 2023-12-26 7:28 ` [PATCH v6 00/14] Enhance the bond framework to support offload Chaoyong He ` (10 preceding siblings ...) 2023-12-26 7:28 ` [PATCH v6 11/14] net/nfp: support bond member notification Chaoyong He @ 2023-12-26 7:28 ` Chaoyong He 2023-12-26 7:28 ` [PATCH v6 13/14] net/nfp: support getting bond firmware creation Chaoyong He ` (2 subsequent siblings) 14 siblings, 0 replies; 80+ messages in thread From: Chaoyong He @ 2023-12-26 7:28 UTC (permalink / raw) To: dev; +Cc: oss-drivers, Long Wu, Peng Zhang, Chaoyong He From: Long Wu <long.wu@corigine.com> Firmware sends bond firmware creation packets to driver, driver needs to handle it in ctrl VNIC service. Signed-off-by: Long Wu <long.wu@corigine.com> Reviewed-by: Peng Zhang <peng.zhang@corigine.com> Reviewed-by: Chaoyong He <chaoyong.he@corigine.com> --- drivers/net/nfp/flower/nfp_flower_bond.c | 53 ++++++++++++ drivers/net/nfp/flower/nfp_flower_bond.h | 14 +++ drivers/net/nfp/flower/nfp_flower_ctrl.c | 106 +++++++++++++++++++++-- 3 files changed, 168 insertions(+), 5 deletions(-) diff --git a/drivers/net/nfp/flower/nfp_flower_bond.c b/drivers/net/nfp/flower/nfp_flower_bond.c index 523e0025ad..73b616360a 100644 --- a/drivers/net/nfp/flower/nfp_flower_bond.c +++ b/drivers/net/nfp/flower/nfp_flower_bond.c @@ -161,6 +161,8 @@ nfp_flower_bond_init(struct nfp_app_fw_flower *app_fw_flower) LIST_INIT(&nfp_bond->group_list); nfp_flower_bond_increment_version(nfp_bond); nfp_bond->app_fw_flower = app_fw_flower; + nfp_bond->retrans.head = 0; + nfp_bond->retrans.rear = 0; return 0; } @@ -168,6 +170,7 @@ nfp_flower_bond_init(struct nfp_app_fw_flower *app_fw_flower) void nfp_flower_bond_cleanup(struct nfp_app_fw_flower *app_fw_flower) { + struct rte_mbuf *mbuf_tmp; struct nfp_bond_group *entry; struct nfp_flower_bond *nfp_bond = app_fw_flower->nfp_bond; @@ -177,6 +180,12 @@ nfp_flower_bond_cleanup(struct nfp_app_fw_flower *app_fw_flower) LIST_REMOVE(entry, next); rte_free(entry); } + + mbuf_tmp = nfp_fl_bond_get_unprocessed(nfp_bond); + for (; mbuf_tmp != NULL; + mbuf_tmp = nfp_fl_bond_get_unprocessed(nfp_bond)) + rte_pktmbuf_free(mbuf_tmp); + pthread_mutex_unlock(&nfp_bond->mutex); pthread_mutex_destroy(&nfp_bond->mutex); @@ -587,3 +596,47 @@ nfp_flower_bond_all_member_are_phyrepr(struct rte_eth_dev *bond_dev) return true; } + +int +nfp_fl_bond_put_unprocessed(struct nfp_flower_bond *nfp_bond, + struct rte_mbuf *mbuf) +{ + uint16_t rear_real; + struct nfp_flower_bond_retrans *retrans; + struct nfp_flower_cmsg_bond_config *cmsg_payload; + + cmsg_payload = rte_pktmbuf_mtod_offset(mbuf, + struct nfp_flower_cmsg_bond_config *, + NFP_FLOWER_CMSG_HLEN); + + if (rte_be_to_cpu_32(cmsg_payload->group_id) >= NFP_FL_BOND_GROUP_MAX) + return -EINVAL; + + retrans = &nfp_bond->retrans; + rear_real = (retrans->rear + 1) % NFP_FL_BOND_RETRANS_LIMIT; + if (rear_real == retrans->head) + return -ENOSPC; + + retrans->mbufs[retrans->rear] = mbuf; + + retrans->rear = rear_real; + + return 0; +} + +struct rte_mbuf * +nfp_fl_bond_get_unprocessed(struct nfp_flower_bond *nfp_bond) +{ + struct rte_mbuf *mbuf; + struct nfp_flower_bond_retrans *retrans; + + retrans = &nfp_bond->retrans; + if (retrans->rear == retrans->head) + return NULL; + + mbuf = retrans->mbufs[retrans->head]; + + retrans->head = (retrans->head + 1) % NFP_FL_BOND_RETRANS_LIMIT; + + return mbuf; +} diff --git a/drivers/net/nfp/flower/nfp_flower_bond.h b/drivers/net/nfp/flower/nfp_flower_bond.h index 12e61ff1ce..e88d72899f 100644 --- a/drivers/net/nfp/flower/nfp_flower_bond.h +++ b/drivers/net/nfp/flower/nfp_flower_bond.h @@ -41,6 +41,9 @@ #define NFP_FL_BOND_PORT_TX_ENABLED RTE_BIT32(1) #define NFP_FL_BOND_PORT_CHANGED RTE_BIT32(2) +/** We store 100(101-1) mbufs but queue needs 101 */ +#define NFP_FL_BOND_RETRANS_LIMIT 101 + enum nfp_flower_bond_batch { NFP_FLOWER_BOND_BATCH_FIRST, NFP_FLOWER_BOND_BATCH_MEMBER, @@ -53,6 +56,12 @@ enum nfp_flower_bond_event { NFP_FLOWER_UNREGISTER }; +struct nfp_flower_bond_retrans { + struct rte_mbuf *mbufs[NFP_FL_BOND_RETRANS_LIMIT]; + uint16_t head; + uint16_t rear; +}; + /* Control message payload for bond config */ struct nfp_flower_cmsg_bond_config { /** Configuration flags */ @@ -113,6 +122,8 @@ struct nfp_flower_bond { bool group_id_map[NFP_FL_BOND_GROUP_MAX]; /** Pointer to the flower app */ struct nfp_app_fw_flower *app_fw_flower; + /** Store bond offload packets from firmware */ + struct nfp_flower_bond_retrans retrans; }; struct nfp_flower_bond_cmsg_args { @@ -139,5 +150,8 @@ int nfp_flower_bond_event_handle(struct nfp_flower_bond *nfp_bond, enum nfp_flower_bond_event event); void nfp_flower_bond_do_work(struct nfp_flower_bond *nfp_bond); bool nfp_flower_bond_all_member_are_phyrepr(struct rte_eth_dev *bond_dev); +struct rte_mbuf *nfp_fl_bond_get_unprocessed(struct nfp_flower_bond *nfp_bond); +int nfp_fl_bond_put_unprocessed(struct nfp_flower_bond *nfp_bond, + struct rte_mbuf *mbuf); #endif /* __NFP_FLOWER_BOND_H__ */ diff --git a/drivers/net/nfp/flower/nfp_flower_ctrl.c b/drivers/net/nfp/flower/nfp_flower_ctrl.c index c25487c277..75c8c0b20a 100644 --- a/drivers/net/nfp/flower/nfp_flower_ctrl.c +++ b/drivers/net/nfp/flower/nfp_flower_ctrl.c @@ -15,6 +15,36 @@ #define MAX_PKT_BURST 32 +static void +nfp_flower_bond_recover_status(struct nfp_flower_bond *nfp_bond) +{ + int err; + struct rte_mbuf *mbuf_tmp; + struct nfp_bond_group *entry; + + pthread_mutex_lock(&nfp_bond->mutex); + + /* Clear all unprocessed messages */ + mbuf_tmp = nfp_fl_bond_get_unprocessed(nfp_bond); + for (; mbuf_tmp != NULL; + mbuf_tmp = nfp_fl_bond_get_unprocessed(nfp_bond)) + rte_pktmbuf_free(mbuf_tmp); + + /* Mark all groups dirty */ + LIST_FOREACH(entry, &nfp_bond->group_list, next) + entry->dirty = true; + + /* Reset NFP group config */ + err = nfp_flower_bond_reset(nfp_bond); + if (err != 0) + PMD_DRV_LOG(ERR, "Mem err in group reset msg"); + + pthread_mutex_unlock(&nfp_bond->mutex); + + /* Schedule a LAG/BOND config update */ + nfp_flower_bond_do_work(nfp_bond); +} + static uint16_t nfp_flower_ctrl_vnic_recv(void *rx_queue, struct rte_mbuf **rx_pkts, @@ -450,6 +480,55 @@ nfp_flower_cmsg_port_mod_rx(struct nfp_app_fw_flower *app_fw_flower, return 0; } +static bool +nfp_flower_bond_unprocessed_msg(void *app_flower, + struct rte_mbuf *mbuf) +{ + uint16_t cnt; + uint8_t flags; + bool store_tag = false; + struct rte_mbuf *mbuf_tmp; + struct nfp_flower_bond *nfp_bond; + struct nfp_flower_cmsg_bond_config *cmsg_payload; + struct nfp_app_fw_flower *app_fw_flower = app_flower; + + cmsg_payload = rte_pktmbuf_mtod_offset(mbuf, + struct nfp_flower_cmsg_bond_config *, + NFP_FLOWER_CMSG_HLEN); + + flags = cmsg_payload->ctrl_flags; + + /* + * Note the intentional fall through below. If DATA and XON are both + * set, the message will stored and sent again with the rest of the + * unprocessed messages list. + */ + nfp_bond = app_fw_flower->nfp_bond; + + /* Store */ + if ((flags & NFP_FL_BOND_DATA) != 0) { + if (nfp_fl_bond_put_unprocessed(nfp_bond, mbuf) == 0) + store_tag = true; + } + + /* Send stored */ + if ((flags & NFP_FL_BOND_XON) != 0) { + mbuf_tmp = nfp_fl_bond_get_unprocessed(nfp_bond); + for (; mbuf_tmp != NULL; + mbuf_tmp = nfp_fl_bond_get_unprocessed(nfp_bond)) { + cnt = nfp_flower_ctrl_vnic_xmit(app_fw_flower, mbuf_tmp); + if (cnt == 0) + rte_pktmbuf_free(mbuf_tmp); + } + } + + /* Resend all */ + if ((flags & NFP_FL_BOND_SYNC) != 0) + nfp_flower_bond_recover_status(nfp_bond); + + return store_tag; +} + static void nfp_flower_cmsg_rx(struct nfp_app_fw_flower *app_fw_flower, struct rte_mbuf **pkts_burst, @@ -457,6 +536,7 @@ nfp_flower_cmsg_rx(struct nfp_app_fw_flower *app_fw_flower, { uint16_t i; char *meta; + bool mbuf_stored; uint32_t meta_type; uint32_t meta_info; struct nfp_mtr_priv *mtr_priv; @@ -486,18 +566,34 @@ nfp_flower_cmsg_rx(struct nfp_app_fw_flower *app_fw_flower, continue; } - if (cmsg_hdr->type == NFP_FLOWER_CMSG_TYPE_FLOW_STATS) { + mbuf_stored = false; + + switch (cmsg_hdr->type) { + case NFP_FLOWER_CMSG_TYPE_FLOW_STATS: /* We need to deal with stats updates from HW asap */ nfp_flower_cmsg_rx_stats(flow_priv, pkts_burst[i]); - } else if (cmsg_hdr->type == NFP_FLOWER_CMSG_TYPE_QOS_STATS) { + break; + case NFP_FLOWER_CMSG_TYPE_QOS_STATS: /* Handle meter stats */ nfp_flower_cmsg_rx_qos_stats(mtr_priv, pkts_burst[i]); - } else if (cmsg_hdr->type == NFP_FLOWER_CMSG_TYPE_PORT_MOD) { - /* Handle changes to port configuration/status */ + break; + case NFP_FLOWER_CMSG_TYPE_PORT_MOD: + /* Handle meter stats */ nfp_flower_cmsg_port_mod_rx(app_fw_flower, pkts_burst[i]); + break; + case NFP_FLOWER_CMSG_TYPE_LAG_CONFIG: + /* Handle LAG/BOND related packets */ + if (nfp_flower_support_bond_offload(app_fw_flower)) + mbuf_stored = nfp_flower_bond_unprocessed_msg(app_fw_flower, + pkts_burst[i]); + break; + default: + PMD_DRV_LOG(INFO, "Unmatched repr message type: %u", + cmsg_hdr->type); } - rte_pktmbuf_free(pkts_burst[i]); + if (!mbuf_stored) + rte_pktmbuf_free(pkts_burst[i]); } } -- 2.39.1 ^ permalink raw reply [flat|nested] 80+ messages in thread
* [PATCH v6 13/14] net/nfp: support getting bond firmware creation 2023-12-26 7:28 ` [PATCH v6 00/14] Enhance the bond framework to support offload Chaoyong He ` (11 preceding siblings ...) 2023-12-26 7:28 ` [PATCH v6 12/14] net/nfp: handle bond packets from firmware Chaoyong He @ 2023-12-26 7:28 ` Chaoyong He 2023-12-26 7:28 ` [PATCH v6 14/14] net/nfp: support offloading bond-flow Chaoyong He 2024-09-22 22:30 ` [PATCH v6 00/14] Enhance the bond framework to support offload Ferruh Yigit 14 siblings, 0 replies; 80+ messages in thread From: Chaoyong He @ 2023-12-26 7:28 UTC (permalink / raw) To: dev; +Cc: oss-drivers, Long Wu, Peng Zhang, Chaoyong He From: Long Wu <long.wu@corigine.com> Support getting the status that whether the bonding port is created by the nfp firmware. Signed-off-by: Long Wu <long.wu@corigine.com> Reviewed-by: Peng Zhang <peng.zhang@corigine.com> Reviewed-by: Chaoyong He <chaoyong.he@corigine.com> --- drivers/net/nfp/flower/nfp_flower_bond.c | 27 +++++++++++++++++++ drivers/net/nfp/flower/nfp_flower_bond.h | 2 ++ .../net/nfp/flower/nfp_flower_representor.c | 1 + 3 files changed, 30 insertions(+) diff --git a/drivers/net/nfp/flower/nfp_flower_bond.c b/drivers/net/nfp/flower/nfp_flower_bond.c index 73b616360a..c809465ffc 100644 --- a/drivers/net/nfp/flower/nfp_flower_bond.c +++ b/drivers/net/nfp/flower/nfp_flower_bond.c @@ -640,3 +640,30 @@ nfp_fl_bond_get_unprocessed(struct nfp_flower_bond *nfp_bond) return mbuf; } + +int +nfp_flower_bond_fw_create_get(struct rte_eth_dev *bond_dev, + struct rte_eth_dev *nfp_dev) +{ + int ret = -EINVAL; + struct nfp_bond_group *group; + struct nfp_flower_bond *nfp_bond; + struct nfp_flower_representor *repr; + + if (!nfp_flower_bond_is_member_port(bond_dev, nfp_dev)) + return ret; + + repr = nfp_dev->data->dev_private; + nfp_bond = repr->app_fw_flower->nfp_bond; + + pthread_mutex_lock(&nfp_bond->mutex); + LIST_FOREACH(group, &nfp_bond->group_list, next) { + if (group->main_dev == bond_dev && group->offloaded == true) { + ret = 0; + break; + } + } + pthread_mutex_unlock(&nfp_bond->mutex); + + return ret; +} diff --git a/drivers/net/nfp/flower/nfp_flower_bond.h b/drivers/net/nfp/flower/nfp_flower_bond.h index e88d72899f..41cd64d9eb 100644 --- a/drivers/net/nfp/flower/nfp_flower_bond.h +++ b/drivers/net/nfp/flower/nfp_flower_bond.h @@ -153,5 +153,7 @@ bool nfp_flower_bond_all_member_are_phyrepr(struct rte_eth_dev *bond_dev); struct rte_mbuf *nfp_fl_bond_get_unprocessed(struct nfp_flower_bond *nfp_bond); int nfp_fl_bond_put_unprocessed(struct nfp_flower_bond *nfp_bond, struct rte_mbuf *mbuf); +int nfp_flower_bond_fw_create_get(struct rte_eth_dev *bond_dev, + struct rte_eth_dev *nfp_dev); #endif /* __NFP_FLOWER_BOND_H__ */ diff --git a/drivers/net/nfp/flower/nfp_flower_representor.c b/drivers/net/nfp/flower/nfp_flower_representor.c index 43106da90e..8a1b2eff66 100644 --- a/drivers/net/nfp/flower/nfp_flower_representor.c +++ b/drivers/net/nfp/flower/nfp_flower_representor.c @@ -458,6 +458,7 @@ static const struct eth_dev_ops nfp_flower_repr_dev_ops = { .mtr_ops_get = nfp_net_mtr_ops_get, .bond_notify_member = nfp_flower_repr_bond_notify_member, + .bond_fw_create_get = nfp_flower_bond_fw_create_get, }; static uint32_t -- 2.39.1 ^ permalink raw reply [flat|nested] 80+ messages in thread
* [PATCH v6 14/14] net/nfp: support offloading bond-flow 2023-12-26 7:28 ` [PATCH v6 00/14] Enhance the bond framework to support offload Chaoyong He ` (12 preceding siblings ...) 2023-12-26 7:28 ` [PATCH v6 13/14] net/nfp: support getting bond firmware creation Chaoyong He @ 2023-12-26 7:28 ` Chaoyong He 2024-09-22 22:30 ` [PATCH v6 00/14] Enhance the bond framework to support offload Ferruh Yigit 14 siblings, 0 replies; 80+ messages in thread From: Chaoyong He @ 2023-12-26 7:28 UTC (permalink / raw) To: dev; +Cc: oss-drivers, Long Wu, Peng Zhang, Chaoyong He From: Long Wu <long.wu@corigine.com> After bonding port notification and firmware created the bonding port, NFP rte_flow rule adds support for destination port to bonding port. Signed-off-by: Long Wu <long.wu@corigine.com> Reviewed-by: Peng Zhang <peng.zhang@corigine.com> Reviewed-by: Chaoyong He <chaoyong.he@corigine.com> --- drivers/net/nfp/flower/nfp_flower_bond.c | 22 +++ drivers/net/nfp/flower/nfp_flower_bond.h | 6 + drivers/net/nfp/flower/nfp_flower_cmsg.h | 9 + drivers/net/nfp/flower/nfp_flower_flow.c | 232 +++++++++++++++++++++-- 4 files changed, 252 insertions(+), 17 deletions(-) diff --git a/drivers/net/nfp/flower/nfp_flower_bond.c b/drivers/net/nfp/flower/nfp_flower_bond.c index c809465ffc..6e4d60659a 100644 --- a/drivers/net/nfp/flower/nfp_flower_bond.c +++ b/drivers/net/nfp/flower/nfp_flower_bond.c @@ -667,3 +667,25 @@ nfp_flower_bond_fw_create_get(struct rte_eth_dev *bond_dev, return ret; } + +int +nfp_flower_bond_get_output_id(struct nfp_flower_bond *nfp_bond, + const struct rte_eth_dev *bond_dev, + uint32_t *group_id) +{ + struct nfp_bond_group *group; + + pthread_mutex_lock(&nfp_bond->mutex); + + group = nfp_flower_bond_find_group(nfp_bond, bond_dev); + if (group == NULL) { + pthread_mutex_unlock(&nfp_bond->mutex); + return -ENOENT; + } + + *group_id = group->group_id; + + pthread_mutex_unlock(&nfp_bond->mutex); + + return 0; +} diff --git a/drivers/net/nfp/flower/nfp_flower_bond.h b/drivers/net/nfp/flower/nfp_flower_bond.h index 41cd64d9eb..7e451c58f1 100644 --- a/drivers/net/nfp/flower/nfp_flower_bond.h +++ b/drivers/net/nfp/flower/nfp_flower_bond.h @@ -44,6 +44,9 @@ /** We store 100(101-1) mbufs but queue needs 101 */ #define NFP_FL_BOND_RETRANS_LIMIT 101 +/* Flow output action of bonding port */ +#define NFP_FL_BOND_OUT 0xC0DE0000 + enum nfp_flower_bond_batch { NFP_FLOWER_BOND_BATCH_FIRST, NFP_FLOWER_BOND_BATCH_MEMBER, @@ -155,5 +158,8 @@ int nfp_fl_bond_put_unprocessed(struct nfp_flower_bond *nfp_bond, struct rte_mbuf *mbuf); int nfp_flower_bond_fw_create_get(struct rte_eth_dev *bond_dev, struct rte_eth_dev *nfp_dev); +int nfp_flower_bond_get_output_id(struct nfp_flower_bond *nfp_bond, + const struct rte_eth_dev *bond_dev, + uint32_t *group_id); #endif /* __NFP_FLOWER_BOND_H__ */ diff --git a/drivers/net/nfp/flower/nfp_flower_cmsg.h b/drivers/net/nfp/flower/nfp_flower_cmsg.h index 60ab58a3b1..17ef943855 100644 --- a/drivers/net/nfp/flower/nfp_flower_cmsg.h +++ b/drivers/net/nfp/flower/nfp_flower_cmsg.h @@ -712,6 +712,15 @@ struct nfp_fl_act_head { uint8_t len_lw; }; +#define NFP_FL_PRE_BOND_VER_OFF 8 + +struct nfp_fl_pre_bond { + struct nfp_fl_act_head head; + rte_be16_t group_id; + uint8_t lag_version[3]; + uint8_t instance; +}; + struct nfp_fl_act_output { struct nfp_fl_act_head head; rte_be16_t flags; diff --git a/drivers/net/nfp/flower/nfp_flower_flow.c b/drivers/net/nfp/flower/nfp_flower_flow.c index e26be30d18..6e21ad8c26 100644 --- a/drivers/net/nfp/flower/nfp_flower_flow.c +++ b/drivers/net/nfp/flower/nfp_flower_flow.c @@ -5,12 +5,14 @@ #include "nfp_flower_flow.h" +#include <rte_eth_bond.h> #include <rte_flow_driver.h> #include <rte_hash.h> #include <rte_jhash.h> #include <rte_malloc.h> #include "flower/nfp_conntrack.h" +#include "flower/nfp_flower_bond.h" #include "flower/nfp_flower_representor.h" #include "nfpcore/nfp_rtsym.h" #include "nfp_logs.h" @@ -969,9 +971,23 @@ nfp_flow_key_layers_calculate_items(const struct rte_flow_item items[], return 0; } +static bool +nfp_flow_check_bond_action(struct nfp_flower_representor *representor, + uint32_t port_id) +{ + struct rte_eth_dev *bond_dev = &rte_eth_devices[port_id]; + + if (nfp_flower_support_bond_offload(representor->app_fw_flower) || + !nfp_flower_bond_all_member_are_phyrepr(bond_dev)) + return false; + + return true; +} + static int nfp_flow_key_layers_calculate_actions(const struct rte_flow_action actions[], - struct nfp_fl_key_ls *key_ls) + struct nfp_fl_key_ls *key_ls, + struct nfp_flower_representor *representor) { int ret = 0; bool meter_flag = false; @@ -981,6 +997,8 @@ nfp_flow_key_layers_calculate_actions(const struct rte_flow_action actions[], bool mac_set_flag = false; bool ttl_tos_flag = false; const struct rte_flow_action *action; + const struct rte_flow_action_port_id *port_id; + const struct rte_flow_action_ethdev *action_ethdev; for (action = actions; action->type != RTE_FLOW_ACTION_TYPE_END; ++action) { /* Make sure actions length no longer than NFP_FL_MAX_A_SIZ */ @@ -1006,10 +1024,36 @@ nfp_flow_key_layers_calculate_actions(const struct rte_flow_action actions[], case RTE_FLOW_ACTION_TYPE_PORT_ID: PMD_DRV_LOG(DEBUG, "RTE_FLOW_ACTION_TYPE_PORT_ID detected"); key_ls->act_size += sizeof(struct nfp_fl_act_output); + + port_id = action->conf; + if (port_id == NULL) + return -ERANGE; + + if (!rte_eth_bond_is_valid_port(port_id->id)) + break; + + if (!nfp_flow_check_bond_action(representor, port_id->id)) + return -ENOTSUP; + + /* We only add when all checks have passed */ + key_ls->act_size += sizeof(struct nfp_fl_pre_bond); break; case RTE_FLOW_ACTION_TYPE_REPRESENTED_PORT: PMD_DRV_LOG(DEBUG, "RTE_FLOW_ACTION_TYPE_REPRESENTED_PORT detected"); key_ls->act_size += sizeof(struct nfp_fl_act_output); + + action_ethdev = action->conf; + if (action_ethdev == NULL) + return -ERANGE; + + if (!rte_eth_bond_is_valid_port(action_ethdev->port_id)) + break; + + if (!nfp_flow_check_bond_action(representor, action_ethdev->port_id)) + return -ENOTSUP; + + /* We only add when all checks have passed */ + key_ls->act_size += sizeof(struct nfp_fl_pre_bond); break; case RTE_FLOW_ACTION_TYPE_SET_MAC_SRC: PMD_DRV_LOG(DEBUG, "RTE_FLOW_ACTION_TYPE_SET_MAC_SRC detected"); @@ -1150,7 +1194,8 @@ nfp_flow_key_layers_calculate_actions(const struct rte_flow_action actions[], static int nfp_flow_key_layers_calculate(const struct rte_flow_item items[], const struct rte_flow_action actions[], - struct nfp_fl_key_ls *key_ls) + struct nfp_fl_key_ls *key_ls, + struct nfp_flower_representor *representor) { int ret = 0; @@ -1164,7 +1209,7 @@ nfp_flow_key_layers_calculate(const struct rte_flow_item items[], key_ls->tun_type = NFP_FL_TUN_NONE; ret |= nfp_flow_key_layers_calculate_items(items, key_ls); - ret |= nfp_flow_key_layers_calculate_actions(actions, key_ls); + ret |= nfp_flow_key_layers_calculate_actions(actions, key_ls, representor); return ret; } @@ -2160,12 +2205,45 @@ nfp_flow_compile_items(struct nfp_flower_representor *representor, return 0; } +static int +nfp_flower_bond_populate_pre_action(struct nfp_flower_representor *repr, + struct rte_eth_dev *bond_dev, + struct nfp_fl_pre_bond *pre_bond) +{ + rte_be32_t version; + struct nfp_bond_group *group; + struct nfp_flower_bond *nfp_bond; + + nfp_bond = repr->app_fw_flower->nfp_bond; + + pthread_mutex_lock(&nfp_bond->mutex); + + group = nfp_flower_bond_find_group(nfp_bond, bond_dev); + if (group == NULL) { + pthread_mutex_unlock(&nfp_bond->mutex); + PMD_DRV_LOG(ERR, "Invalid entry: group does not exist for bond action"); + return -ENOENT; + } + + pre_bond->group_id = rte_cpu_to_be_16(group->group_id); + version = rte_cpu_to_be_32(nfp_bond->batch_ver << NFP_FL_PRE_BOND_VER_OFF); + memcpy(pre_bond->lag_version, &version, 3); + pre_bond->instance = group->group_inst; + + pthread_mutex_unlock(&nfp_bond->mutex); + + return 0; +} + static int nfp_flow_action_output(char *act_data, const struct rte_flow_action *action, struct nfp_fl_rule_metadata *nfp_flow_meta, - uint32_t output_cnt) + uint32_t output_cnt, + struct nfp_flower_bond *nfp_bond, + bool *bonding_port) { + uint32_t gid; size_t act_size; struct rte_eth_dev *ethdev; struct nfp_fl_act_output *output; @@ -2177,18 +2255,31 @@ nfp_flow_action_output(char *act_data, return -ERANGE; ethdev = &rte_eth_devices[port_id->id]; - representor = ethdev->data->dev_private; + output = (struct nfp_fl_act_output *)act_data; + if (rte_eth_bond_is_valid_port(port_id->id)) { + /* Output port is bonding port */ + if (nfp_flower_bond_get_output_id(nfp_bond, ethdev, &gid) != 0) { + PMD_DRV_LOG(ERR, "Cannot find group id for bond action"); + return -EINVAL; + } + + output->port = rte_cpu_to_be_32(NFP_FL_BOND_OUT | gid); + nfp_flow_meta->shortcut = rte_cpu_to_be_32(NFP_FL_SC_ACT_NULL); + *bonding_port = true; + } else { + representor = ethdev->data->dev_private; + + output->port = rte_cpu_to_be_32(representor->port_id); + nfp_flow_meta->shortcut = rte_cpu_to_be_32(representor->port_id); + } + act_size = sizeof(struct nfp_fl_act_output); - output = (struct nfp_fl_act_output *)act_data; output->head.jump_id = NFP_FL_ACTION_OPCODE_OUTPUT; output->head.len_lw = act_size >> NFP_FL_LW_SIZ; - output->port = rte_cpu_to_be_32(representor->port_id); if (output_cnt == 0) output->flags = rte_cpu_to_be_16(NFP_FL_OUT_FLAGS_LAST); - nfp_flow_meta->shortcut = rte_cpu_to_be_32(representor->port_id); - return 0; } @@ -2196,8 +2287,11 @@ static int nfp_flow_action_output_stage(char *act_data, const struct rte_flow_action *action, struct nfp_fl_rule_metadata *nfp_flow_meta, - uint32_t output_cnt) + uint32_t output_cnt, + struct nfp_flower_bond *nfp_bond, + bool *bonding_port) { + uint32_t gid; size_t act_size; struct rte_eth_dev *ethdev; struct nfp_fl_act_output *output; @@ -2209,17 +2303,70 @@ nfp_flow_action_output_stage(char *act_data, return -ERANGE; ethdev = &rte_eth_devices[action_ethdev->port_id]; - representor = ethdev->data->dev_private; + output = (struct nfp_fl_act_output *)act_data; + if (rte_eth_bond_is_valid_port(action_ethdev->port_id)) { + /* Output port is bonding port */ + if (nfp_flower_bond_get_output_id(nfp_bond, ethdev, &gid) != 0) { + PMD_DRV_LOG(ERR, "Cannot find group id for bond action"); + return -EINVAL; + } + + output->port = rte_cpu_to_be_32(NFP_FL_BOND_OUT | gid); + nfp_flow_meta->shortcut = rte_cpu_to_be_32(NFP_FL_SC_ACT_NULL); + *bonding_port = true; + } else { + representor = ethdev->data->dev_private; + + output->port = rte_cpu_to_be_32(representor->port_id); + nfp_flow_meta->shortcut = rte_cpu_to_be_32(representor->port_id); + } + act_size = sizeof(struct nfp_fl_act_output); - output = (struct nfp_fl_act_output *)act_data; output->head.jump_id = NFP_FL_ACTION_OPCODE_OUTPUT; output->head.len_lw = act_size >> NFP_FL_LW_SIZ; - output->port = rte_cpu_to_be_32(representor->port_id); if (output_cnt == 0) output->flags = rte_cpu_to_be_16(NFP_FL_OUT_FLAGS_LAST); - nfp_flow_meta->shortcut = rte_cpu_to_be_32(representor->port_id); + return 0; +} + +static int +nfp_flow_pre_bond(struct nfp_flower_representor *representor, + char *act_data, + char *actions, + const struct rte_flow_action *action) +{ + int err; + int act_len; + uint32_t port_id; + struct rte_eth_dev *ethdev; + struct nfp_fl_pre_bond *pre_bond; + uint32_t pre_bond_size = sizeof(struct nfp_fl_pre_bond); + + act_len = act_data - actions; + + if (act_len + pre_bond_size > NFP_FL_MAX_A_SIZ) { + PMD_DRV_LOG(ERR, "Maximum allowed action list size exceeded at bond action"); + return -EOPNOTSUPP; + } + + if (act_len != 0) + memmove(actions + pre_bond_size, actions, act_len); + + if (action->type == RTE_FLOW_ACTION_TYPE_REPRESENTED_PORT) + port_id = ((const struct rte_flow_action_ethdev *)action->conf)->port_id; + else + port_id = ((const struct rte_flow_action_port_id *)action->conf)->id; + + ethdev = &rte_eth_devices[port_id]; + pre_bond = (struct nfp_fl_pre_bond *)actions; + err = nfp_flower_bond_populate_pre_action(representor, ethdev, pre_bond); + if (err != 0) + return err; + + pre_bond->head.jump_id = NFP_FL_ACTION_OPCODE_PRE_LAG; + pre_bond->head.len_lw = pre_bond_size >> NFP_FL_LW_SIZ; return 0; } @@ -3502,6 +3649,13 @@ nfp_flow_count_output(const struct rte_flow_action actions[]) return count; } +static bool +nfp_flow_check_action_first(const struct rte_flow_action actions[], + uint32_t count) +{ + return ((nfp_flow_count_output(actions) - count) == 1); +} + static int nfp_flow_compile_action(struct nfp_flower_representor *representor, const struct rte_flow_action actions[], @@ -3517,7 +3671,9 @@ nfp_flow_compile_action(struct nfp_flower_representor *representor, bool tp_set_flag = false; bool mac_set_flag = false; bool ttl_tos_flag = false; + bool bonding_port = false; uint32_t total_actions = 0; + struct nfp_flower_bond *nfp_bond; const struct rte_flow_action *action; struct nfp_flower_meta_tci *meta_tci; struct nfp_fl_rule_metadata *nfp_flow_meta; @@ -3546,7 +3702,10 @@ nfp_flow_compile_action(struct nfp_flower_representor *representor, case RTE_FLOW_ACTION_TYPE_REPRESENTED_PORT: PMD_DRV_LOG(DEBUG, "Process RTE_FLOW_ACTION_TYPE_REPRESENTED_PORT"); count--; - ret = nfp_flow_action_output_stage(position, action, nfp_flow_meta, count); + + nfp_bond = representor->app_fw_flower->nfp_bond; + ret = nfp_flow_action_output_stage(position, action, + nfp_flow_meta, count, nfp_bond, &bonding_port); if (ret != 0) { PMD_DRV_LOG(ERR, "Failed when process" " RTE_FLOW_ACTION_TYPE_REPRESENTED_PORT"); @@ -3554,11 +3713,32 @@ nfp_flow_compile_action(struct nfp_flower_representor *representor, } position += sizeof(struct nfp_fl_act_output); + + /* Output port is not a bonding port */ + if (!bonding_port) + break; + + if (!nfp_flow_check_action_first(actions, count)) { + PMD_DRV_LOG(ERR, "Bond action should be first"); + return -EOPNOTSUPP; + } + + ret = nfp_flow_pre_bond(representor, position, + action_data, action); + if (ret != 0) { + PMD_DRV_LOG(ERR, "Bond action special move failed"); + return ret; + } + + position += sizeof(struct nfp_fl_pre_bond); break; case RTE_FLOW_ACTION_TYPE_PORT_ID: PMD_DRV_LOG(DEBUG, "Process RTE_FLOW_ACTION_TYPE_PORT_ID"); count--; - ret = nfp_flow_action_output(position, action, nfp_flow_meta, count); + + nfp_bond = representor->app_fw_flower->nfp_bond; + ret = nfp_flow_action_output(position, action, + nfp_flow_meta, count, nfp_bond, &bonding_port); if (ret != 0) { PMD_DRV_LOG(ERR, "Failed when process" " RTE_FLOW_ACTION_TYPE_PORT_ID"); @@ -3566,6 +3746,24 @@ nfp_flow_compile_action(struct nfp_flower_representor *representor, } position += sizeof(struct nfp_fl_act_output); + + /* Output port is not a bonding port */ + if (!bonding_port) + break; + + if (!nfp_flow_check_action_first(actions, count)) { + PMD_DRV_LOG(ERR, "Bond action should be first"); + return -EOPNOTSUPP; + } + + ret = nfp_flow_pre_bond(representor, position, + action_data, action); + if (ret != 0) { + PMD_DRV_LOG(ERR, "Bond action special move failed"); + return ret; + } + + position += sizeof(struct nfp_fl_pre_bond); break; case RTE_FLOW_ACTION_TYPE_SET_MAC_SRC: PMD_DRV_LOG(DEBUG, "Process RTE_FLOW_ACTION_TYPE_SET_MAC_SRC"); @@ -3770,7 +3968,7 @@ nfp_flow_process(struct nfp_flower_representor *representor, struct nfp_fl_key_ls key_layer; struct nfp_fl_rule_metadata *nfp_flow_meta; - ret = nfp_flow_key_layers_calculate(items, actions, &key_layer); + ret = nfp_flow_key_layers_calculate(items, actions, &key_layer, representor); if (ret != 0) { PMD_DRV_LOG(ERR, "Key layers calculate failed."); return NULL; -- 2.39.1 ^ permalink raw reply [flat|nested] 80+ messages in thread
* Re: [PATCH v6 00/14] Enhance the bond framework to support offload 2023-12-26 7:28 ` [PATCH v6 00/14] Enhance the bond framework to support offload Chaoyong He ` (13 preceding siblings ...) 2023-12-26 7:28 ` [PATCH v6 14/14] net/nfp: support offloading bond-flow Chaoyong He @ 2024-09-22 22:30 ` Ferruh Yigit 14 siblings, 0 replies; 80+ messages in thread From: Ferruh Yigit @ 2024-09-22 22:30 UTC (permalink / raw) To: Chaoyong He, dev; +Cc: oss-drivers On 12/26/2023 7:28 AM, Chaoyong He wrote: > This patch series try to enhance the bond framework to support the > offload feature better: > * Add new API to make the member port can access some information of the > bond port which belongs. > * Add new API to get the result of whether bond port is created by the > member port. > * Add two command line argument to control if enable member port > notification and dedicated queue features. > * Add logic to support add ports which share the same PCI address into > bond port. > * Also modify the testpmd application to test the new APIs and logics > added by this patch series. > > --- > v2: > * Fix compile error on github-robot by removing the redundancy function > declaration in the header file. > v3: > * Use the hole in the structure for the new added flag data field. > v4: > * Drop two commits not necessary for this series. > * Modify some logic as the review comments from reviewers. > v5: > * Add a new 'rte_eth_bond_flow.h' header file. > * Add the patches of NFP PMD as the example of support bond flow > offload. > v6: > * Try to solve the CI build error. > Hi Chaoyong, This patch is active in the patchwork for a while, it is waiting mainly because of specific driver (bonding driver) implementation in ethdev layer. Are you still pursing for this implementation? As this is almost a year old patch series, I will update it as change requested. If you want to continue with the patch, please send a new version and later request a technical board meeting to discuss how to progress with it. Thanks, ferruh ^ permalink raw reply [flat|nested] 80+ messages in thread
end of thread, other threads:[~2024-09-22 22:30 UTC | newest] Thread overview: 80+ messages (download: mbox.gz / follow: Atom feed) -- links below jump to the message on this page -- 2023-10-05 2:40 [PATCH 0/8] Enhance the bond framework to support offload Chaoyong He 2023-10-05 2:40 ` [PATCH 1/8] ethdev: add member notification for bonding port Chaoyong He 2023-10-05 2:40 ` [PATCH 2/8] ethdev: add API to get hardware creation of " Chaoyong He 2023-10-05 2:40 ` [PATCH 3/8] net/bonding: modify interface comment format Chaoyong He 2023-10-05 2:40 ` [PATCH 4/8] net/bonding: add bonding port arguments Chaoyong He 2023-10-05 2:40 ` [PATCH 5/8] net/bonding: support add port by data name Chaoyong He 2023-10-05 2:40 ` [PATCH 6/8] net/bonding: create new rte flow header file Chaoyong He 2023-10-05 2:40 ` [PATCH 7/8] net/bonding: support checking valid bonding port ID Chaoyong He 2023-10-05 2:40 ` [PATCH 8/8] net/bonding: add commands for bonding port notification Chaoyong He 2023-10-07 1:34 ` [PATCH v2 0/8] Enhance the bond framework to support offload Chaoyong He 2023-10-07 1:34 ` [PATCH v2 1/8] ethdev: add member notification for bonding port Chaoyong He 2023-10-07 15:49 ` Stephen Hemminger 2023-10-08 1:37 ` Chaoyong He 2023-10-07 1:34 ` [PATCH v2 2/8] ethdev: add API to get hardware creation of " Chaoyong He 2023-10-07 1:34 ` [PATCH v2 3/8] net/bonding: modify interface comment format Chaoyong He 2023-10-07 1:34 ` [PATCH v2 4/8] net/bonding: add bonding port arguments Chaoyong He 2023-10-07 1:34 ` [PATCH v2 5/8] net/bonding: support add port by data name Chaoyong He 2023-10-07 1:34 ` [PATCH v2 6/8] net/bonding: create new rte flow header file Chaoyong He 2023-10-07 1:34 ` [PATCH v2 7/8] net/bonding: support checking valid bonding port ID Chaoyong He 2023-10-07 1:34 ` [PATCH v2 8/8] net/bonding: add commands for bonding port notification Chaoyong He 2023-10-08 1:50 ` [PATCH v3 0/8] Enhance the bond framework to support offload Chaoyong He 2023-10-08 1:50 ` [PATCH v3 1/8] ethdev: add member notification for bonding port Chaoyong He 2023-10-08 2:49 ` lihuisong (C) 2023-10-09 3:11 ` 回复: " Long Wu 2023-10-17 8:27 ` lihuisong (C) 2023-10-18 2:16 ` Chaoyong He 2023-10-08 1:50 ` [PATCH v3 2/8] ethdev: add API to get hardware creation of " Chaoyong He 2023-10-08 1:50 ` [PATCH v3 3/8] net/bonding: modify interface comment format Chaoyong He 2023-10-08 1:50 ` [PATCH v3 4/8] net/bonding: add bonding port arguments Chaoyong He 2023-10-08 1:50 ` [PATCH v3 5/8] net/bonding: support add port by data name Chaoyong He 2023-10-08 1:50 ` [PATCH v3 6/8] net/bonding: create new rte flow header file Chaoyong He 2023-10-08 1:50 ` [PATCH v3 7/8] net/bonding: support checking valid bonding port ID Chaoyong He 2023-10-17 8:33 ` lihuisong (C) 2023-10-17 9:25 ` Chaoyong He 2023-10-17 11:34 ` lihuisong (C) 2023-10-18 1:53 ` Chaoyong He 2023-10-17 15:56 ` Stephen Hemminger 2023-10-08 1:50 ` [PATCH v3 8/8] net/bonding: add commands for bonding port notification Chaoyong He 2023-10-13 2:22 ` [PATCH v3 0/8] Enhance the bond framework to support offload Chaoyong He 2023-10-13 12:53 ` Ferruh Yigit 2023-10-18 7:48 ` [PATCH v4 0/6] " Chaoyong He 2023-10-18 7:48 ` [PATCH v4 1/6] ethdev: add member notification for bonding port Chaoyong He 2023-10-18 7:48 ` [PATCH v4 2/6] ethdev: add API to get hardware creation of " Chaoyong He 2023-10-18 7:48 ` [PATCH v4 3/6] net/bonding: add bonding port arguments Chaoyong He 2023-10-18 7:48 ` [PATCH v4 4/6] net/bonding: support add port by data name Chaoyong He 2023-10-18 7:48 ` [PATCH v4 5/6] net/bonding: support checking valid bonding port ID Chaoyong He 2023-10-18 7:48 ` [PATCH v4 6/6] net/bonding: add commands for bonding port notification Chaoyong He 2023-11-15 16:01 ` [PATCH v4 0/6] Enhance the bond framework to support offload Ferruh Yigit 2023-11-16 1:45 ` Chaoyong He 2023-12-26 2:37 ` [PATCH v5 00/14] " Chaoyong He 2023-12-26 2:37 ` [PATCH v5 01/14] ethdev: add member notification for bonding port Chaoyong He 2023-12-26 2:37 ` [PATCH v5 02/14] ethdev: add API to get firmware creation of " Chaoyong He 2023-12-26 2:37 ` [PATCH v5 03/14] net/bonding: add bonding port arguments Chaoyong He 2023-12-26 2:37 ` [PATCH v5 04/14] net/bonding: support add port by data name Chaoyong He 2023-12-26 2:37 ` [PATCH v5 05/14] net/bonding: support checking valid bonding port ID Chaoyong He 2023-12-26 2:37 ` [PATCH v5 06/14] net/bonding: add commands for bonding port notification Chaoyong He 2023-12-26 2:37 ` [PATCH v5 07/14] net/bonding: create new rte flow header file Chaoyong He 2023-12-26 2:37 ` [PATCH v5 08/14] net/nfp: add bond firmware creation initialization Chaoyong He 2023-12-26 2:37 ` [PATCH v5 09/14] net/nfp: reset bond configuration of firmware Chaoyong He 2023-12-26 2:37 ` [PATCH v5 10/14] net/nfp: handle link event of bond firmware creation Chaoyong He 2023-12-26 2:37 ` [PATCH v5 11/14] net/nfp: support bond member notification Chaoyong He 2023-12-26 2:37 ` [PATCH v5 12/14] net/nfp: handle bond packets from firmware Chaoyong He 2023-12-26 2:37 ` [PATCH v5 13/14] net/nfp: support getting bond firmware creation Chaoyong He 2023-12-26 2:37 ` [PATCH v5 14/14] net/nfp: support offloading bond-flow Chaoyong He 2023-12-26 7:28 ` [PATCH v6 00/14] Enhance the bond framework to support offload Chaoyong He 2023-12-26 7:28 ` [PATCH v6 01/14] ethdev: add member notification for bonding port Chaoyong He 2023-12-26 7:28 ` [PATCH v6 02/14] ethdev: add API to get firmware creation of " Chaoyong He 2023-12-26 7:28 ` [PATCH v6 03/14] net/bonding: add bonding port arguments Chaoyong He 2023-12-26 7:28 ` [PATCH v6 04/14] net/bonding: support add port by data name Chaoyong He 2023-12-26 7:28 ` [PATCH v6 05/14] net/bonding: support checking valid bonding port ID Chaoyong He 2023-12-26 7:28 ` [PATCH v6 06/14] net/bonding: add commands for bonding port notification Chaoyong He 2023-12-26 7:28 ` [PATCH v6 07/14] net/bonding: create new rte flow header file Chaoyong He 2023-12-26 7:28 ` [PATCH v6 08/14] net/nfp: add bond firmware creation initialization Chaoyong He 2023-12-26 7:28 ` [PATCH v6 09/14] net/nfp: reset bond configuration of firmware Chaoyong He 2023-12-26 7:28 ` [PATCH v6 10/14] net/nfp: handle link event of bond firmware creation Chaoyong He 2023-12-26 7:28 ` [PATCH v6 11/14] net/nfp: support bond member notification Chaoyong He 2023-12-26 7:28 ` [PATCH v6 12/14] net/nfp: handle bond packets from firmware Chaoyong He 2023-12-26 7:28 ` [PATCH v6 13/14] net/nfp: support getting bond firmware creation Chaoyong He 2023-12-26 7:28 ` [PATCH v6 14/14] net/nfp: support offloading bond-flow Chaoyong He 2024-09-22 22:30 ` [PATCH v6 00/14] Enhance the bond framework to support offload Ferruh Yigit
This is a public inbox, see mirroring instructions for how to clone and mirror all data and code used for this inbox; as well as URLs for NNTP newsgroup(s).