From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 1A10E426E8; Sun, 8 Oct 2023 04:49:29 +0200 (CEST) Received: from mails.dpdk.org (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id C4A15402A8; Sun, 8 Oct 2023 04:49:28 +0200 (CEST) Received: from szxga02-in.huawei.com (szxga02-in.huawei.com [45.249.212.188]) by mails.dpdk.org (Postfix) with ESMTP id 1C2F640263 for ; Sun, 8 Oct 2023 04:49:26 +0200 (CEST) Received: from kwepemm000004.china.huawei.com (unknown [172.30.72.54]) by szxga02-in.huawei.com (SkyGuard) with ESMTP id 4S363q37YmzTmPw; Sun, 8 Oct 2023 10:45:59 +0800 (CST) Received: from [10.67.121.59] (10.67.121.59) by kwepemm000004.china.huawei.com (7.193.23.18) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.31; Sun, 8 Oct 2023 10:49:23 +0800 Message-ID: Date: Sun, 8 Oct 2023 10:49:19 +0800 MIME-Version: 1.0 User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:91.0) Gecko/20100101 Thunderbird/91.2.0 Subject: Re: [PATCH v3 1/8] ethdev: add member notification for bonding port To: Chaoyong He , CC: , Long Wu , James Hershaw References: <20231007013440.1309422-1-chaoyong.he@corigine.com> <20231008015041.1551165-1-chaoyong.he@corigine.com> <20231008015041.1551165-2-chaoyong.he@corigine.com> From: "lihuisong (C)" In-Reply-To: <20231008015041.1551165-2-chaoyong.he@corigine.com> Content-Type: text/plain; charset="UTF-8"; format=flowed Content-Transfer-Encoding: 8bit X-Originating-IP: [10.67.121.59] X-ClientProxiedBy: dggems705-chm.china.huawei.com (10.3.19.182) To kwepemm000004.china.huawei.com (7.193.23.18) X-CFilter-Loop: Reflected X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Hi Chaoyong, some comments as below. 在 2023/10/8 9:50, Chaoyong He 写道: > From: Long Wu > > Bonding PMD does not let member ports know the bonding port's > information, like how many member ports the bonding port has, > what mode the bonding port is in and so on. > > Add the notification interface for bonding port to let member > port know it is added to a bonding port and what the bonding > port's configuration is. If so the member ports have chance to > achieve its bond-flow-offlod or other private bonding functions. "its bond-flow-offlod or other private bonding functions" I wonder that what PMDs can do with this. Can you give an example in PMD to help others review? > > Signed-off-by: Long Wu > Reviewed-by: James Hershaw > Reviewed-by: Chaoyong He > --- > drivers/net/bonding/eth_bond_private.h | 1 + > drivers/net/bonding/rte_eth_bond.h | 46 ++++++++++++++++ > drivers/net/bonding/rte_eth_bond_api.c | 73 ++++++++++++++++++++++++++ > drivers/net/bonding/rte_eth_bond_pmd.c | 27 ++++++++-- > drivers/net/bonding/version.map | 3 ++ > lib/ethdev/ethdev_driver.h | 18 +++++++ > 6 files changed, 165 insertions(+), 3 deletions(-) > > diff --git a/drivers/net/bonding/eth_bond_private.h b/drivers/net/bonding/eth_bond_private.h > index e688894210..f69e85c199 100644 > --- a/drivers/net/bonding/eth_bond_private.h > +++ b/drivers/net/bonding/eth_bond_private.h > @@ -180,6 +180,7 @@ struct bond_dev_private { > uint8_t member_update_idx; > > bool kvargs_processing_is_done; > + bool notify_member; /**< Enable member notification of bonding port. */ > > uint32_t candidate_max_rx_pktlen; > uint32_t max_rx_pktlen; > diff --git a/drivers/net/bonding/rte_eth_bond.h b/drivers/net/bonding/rte_eth_bond.h > index f10165f2c6..737beca446 100644 > --- a/drivers/net/bonding/rte_eth_bond.h > +++ b/drivers/net/bonding/rte_eth_bond.h > @@ -351,6 +351,52 @@ rte_eth_bond_link_up_prop_delay_set(uint16_t bonding_port_id, > int > rte_eth_bond_link_up_prop_delay_get(uint16_t bonding_port_id); > > +/** > + * Set the flag of whether bonding port notifies member ports. > + * > + * @param bonding_port_id > + * Port ID of bonding device. > + * @param notify_member > + * Flag of whether bonding port notifies member ports. > + * > + * @return > + * 0 on success, negative value otherwise. > + */ > +__rte_experimental > +int > +rte_eth_bond_notify_member_flag_set(uint16_t bonding_port_id, bool notify_member); s/notify_membe/notify in input? because function name reveals the meaning already. > + > +/** > + * Get the flag of whether bonding port notifies member ports. > + * > + * @param bonding_port_id > + * Port ID of bonding device. > + * @param notify_member > + * Flag of whether bonding port notifies member ports. > + * > + * @return > + * 0 on success, negative value otherwise. > + */ > +__rte_experimental > +int > +rte_eth_bond_notify_member_flag_get(uint16_t bonding_port_id, bool *notify_member); > + > +/** > + * Notify the member ports of bonding port's information. > + * > + * This interface is called in the following functions: > + * - bond_ethdev_lsc_event_callback() > + * - bond_ethdev_configure() Is this interface used just  in these cases? If so I don't think it should be a API in eth_dev_op. > + * > + * @param bonding_port_id > + * Port ID of bonding device. > + * > + * @return > + * 0 on success, negative value otherwise. > + */ > +__rte_experimental > +int > +rte_eth_bond_notify_members(uint16_t bonding_port_id); > > #ifdef __cplusplus > } > diff --git a/drivers/net/bonding/rte_eth_bond_api.c b/drivers/net/bonding/rte_eth_bond_api.c > index 99e496556a..48c1d050fd 100644 > --- a/drivers/net/bonding/rte_eth_bond_api.c > +++ b/drivers/net/bonding/rte_eth_bond_api.c > @@ -627,6 +627,17 @@ __eth_bond_member_add_lock_free(uint16_t bonding_port_id, uint16_t member_port_i > > member_vlan_filter_set(bonding_port_id, member_port_id); > > + if (internals->notify_member && > + *member_eth_dev->dev_ops->bond_notify_member != NULL) { > + ret = member_eth_dev->dev_ops->bond_notify_member(member_eth_dev, > + bonding_eth_dev); > + if (ret < 0) { > + RTE_BOND_LOG(ERR, "Add member (port %u) notify failed!", > + member_port_id); > + return -1; > + } > + } > + > return 0; > > } > @@ -733,6 +744,10 @@ __eth_bond_member_remove_lock_free(uint16_t bonding_port_id, > member_eth_dev = &rte_eth_devices[member_port_id]; > member_remove(internals, member_eth_dev); > member_eth_dev->data->dev_flags &= (~RTE_ETH_DEV_BONDING_MEMBER); > + if (internals->notify_member && > + *member_eth_dev->dev_ops->bond_notify_member != NULL) > + member_eth_dev->dev_ops->bond_notify_member(member_eth_dev, > + bonding_eth_dev); > > /* first member in the active list will be the primary by default, > * otherwise use first device in list */ > @@ -1098,3 +1113,61 @@ rte_eth_bond_link_up_prop_delay_get(uint16_t bonding_port_id) > > return internals->link_up_delay_ms; > } > + > +int > +rte_eth_bond_notify_member_flag_set(uint16_t bonding_port_id, bool notify_member) > +{ > + struct bond_dev_private *internals; > + > + if (valid_bonding_port_id(bonding_port_id) != 0) > + return -EINVAL; > + > + internals = rte_eth_devices[bonding_port_id].data->dev_private; > + > + internals->notify_member = notify_member; > + > + return 0; > +} > + > +int > +rte_eth_bond_notify_member_flag_get(uint16_t bonding_port_id, bool *notify_member) > +{ > + struct bond_dev_private *internals; > + > + if (valid_bonding_port_id(bonding_port_id) != 0) > + return -EINVAL; > + > + internals = rte_eth_devices[bonding_port_id].data->dev_private; > + > + *notify_member = internals->notify_member; > + > + return 0; > +} > + > +int > +rte_eth_bond_notify_members(uint16_t bonding_port_id) > +{ > + uint32_t i; > + uint16_t member_port_id; > + struct rte_eth_dev *bond_dev; > + struct bond_dev_private *internals; > + struct rte_eth_dev *member_dev[RTE_MAX_ETHPORTS]; > + > + if (valid_bonding_port_id(bonding_port_id) != 0) > + return -EINVAL; > + > + bond_dev = &rte_eth_devices[bonding_port_id]; > + internals = bond_dev->data->dev_private; > + > + for (i = 0; i < internals->member_count; i++) { > + member_port_id = internals->members[i].port_id; > + member_dev[i] = &rte_eth_devices[member_port_id]; > + if (*member_dev[i]->dev_ops->bond_notify_member == NULL) > + return -ENOTSUP; > + } Do we need all member ports support bond_notify_member api? If allow user to select one member port to notify? This might be more general. In addition, this action here is inconsistent with above handle(do not notify member if doesn't supportbond_notify_member API). > + > + for (i = 0; i < internals->member_count; i++) > + member_dev[i]->dev_ops->bond_notify_member(member_dev[i], bond_dev); > + > + return 0; > +} > diff --git a/drivers/net/bonding/rte_eth_bond_pmd.c b/drivers/net/bonding/rte_eth_bond_pmd.c > index 122b1187fd..b99b8b8938 100644 > --- a/drivers/net/bonding/rte_eth_bond_pmd.c > +++ b/drivers/net/bonding/rte_eth_bond_pmd.c > @@ -2968,11 +2968,13 @@ bond_ethdev_lsc_event_callback(uint16_t port_id, enum rte_eth_event_type type, > int valid_member = 0; > uint16_t active_pos, member_idx; > uint16_t i; > + uint16_t bonding_port_id; > > if (type != RTE_ETH_EVENT_INTR_LSC || param == NULL) > return rc; > > - bonding_eth_dev = &rte_eth_devices[*(uint16_t *)param]; > + bonding_port_id = *(uint16_t *)param; > + bonding_eth_dev = &rte_eth_devices[bonding_port_id]; > > if (check_for_bonding_ethdev(bonding_eth_dev)) > return rc; > @@ -3044,8 +3046,10 @@ bond_ethdev_lsc_event_callback(uint16_t port_id, enum rte_eth_event_type type, > * using it. > */ > if (internals->user_defined_primary_port && > - internals->primary_port == port_id) > + internals->primary_port == port_id) { > bond_ethdev_primary_set(internals, port_id); > + rte_eth_bond_notify_members(bonding_port_id); > + } > } else { > if (active_pos == internals->active_member_count) > goto link_update; > @@ -3064,6 +3068,7 @@ bond_ethdev_lsc_event_callback(uint16_t port_id, enum rte_eth_event_type type, > internals->active_members[0]); > else > internals->current_primary_port = internals->primary_port; > + rte_eth_bond_notify_members(bonding_port_id); > mac_address_members_update(bonding_eth_dev); > bond_ethdev_promiscuous_update(bonding_eth_dev); > bond_ethdev_allmulticast_update(bonding_eth_dev); > @@ -3362,6 +3367,7 @@ dump_basic(const struct rte_eth_dev *dev, FILE *f) > struct bond_dev_private instant_priv; > const struct bond_dev_private *internals = &instant_priv; > int mode, i; > + bool notify_member; > > /* Obtain a instance of dev_private to prevent data from being modified. */ > memcpy(&instant_priv, dev->data->dev_private, sizeof(struct bond_dev_private)); > @@ -3431,6 +3437,13 @@ dump_basic(const struct rte_eth_dev *dev, FILE *f) > fprintf(f, "\tUser Defined Primary: [%u]\n", internals->primary_port); > if (internals->member_count > 0) > fprintf(f, "\tCurrent Primary: [%u]\n", internals->current_primary_port); > + > + if (rte_eth_bond_notify_member_flag_get(internals->port_id, ¬ify_member) == 0) > + fprintf(f, "\tNotify Member Ports Flag: %s\n", > + notify_member ? "enable" : "disable"); > + else > + fprintf(f, "\tFailed to get notify member ports flag for bonding port %d\n", > + internals->port_id); > } > > static void > @@ -3983,8 +3996,12 @@ bond_ethdev_configure(struct rte_eth_dev *dev) > * if no kvlist, it means that this bonding device has been created > * through the bonding api. > */ > - if (!kvlist || internals->kvargs_processing_is_done) > + if (!kvlist || internals->kvargs_processing_is_done) { > + if (internals->notify_member && rte_eth_bond_notify_members(port_id) != 0) > + RTE_BOND_LOG(ERR, "Notify member ports failed"); > + > return 0; > + } > > internals->kvargs_processing_is_done = true; > > @@ -4222,6 +4239,10 @@ bond_ethdev_configure(struct rte_eth_dev *dev) > return -1; > } > } > + > + if (internals->notify_member && rte_eth_bond_notify_members(port_id) != 0) > + RTE_BOND_LOG(ERR, "Notify member ports failed"); > + > return 0; > } > > diff --git a/drivers/net/bonding/version.map b/drivers/net/bonding/version.map > index 09ee21c55f..3bd5e8ad11 100644 > --- a/drivers/net/bonding/version.map > +++ b/drivers/net/bonding/version.map > @@ -35,4 +35,7 @@ EXPERIMENTAL { > rte_eth_bond_member_add; > rte_eth_bond_member_remove; > rte_eth_bond_members_get; > + rte_eth_bond_notify_member_flag_get; > + rte_eth_bond_notify_member_flag_set; > + rte_eth_bond_notify_members; > }; > diff --git a/lib/ethdev/ethdev_driver.h b/lib/ethdev/ethdev_driver.h > index deb23ada18..f626f971e5 100644 > --- a/lib/ethdev/ethdev_driver.h > +++ b/lib/ethdev/ethdev_driver.h > @@ -1216,6 +1216,21 @@ typedef int (*eth_count_aggr_ports_t)(struct rte_eth_dev *dev); > typedef int (*eth_map_aggr_tx_affinity_t)(struct rte_eth_dev *dev, uint16_t tx_queue_id, > uint8_t affinity); > > +/** > + * @internal > + * Bonding port notifies the member ports. > + * > + * @param dev > + * Member port (ethdev) handle. > + * @param bonding_dev > + * Bonding port (ethdev) handle. > + * > + * @return > + * Negative on error, 0 on success. > + */ > +typedef int (*eth_bond_notify_member)(struct rte_eth_dev *dev, > + struct rte_eth_dev *bonding_dev); > + > /** > * @internal A structure containing the functions exported by an Ethernet driver. > */ > @@ -1455,6 +1470,9 @@ struct eth_dev_ops { > eth_count_aggr_ports_t count_aggr_ports; > /** Map a Tx queue with an aggregated port of the DPDK port */ > eth_map_aggr_tx_affinity_t map_aggr_tx_affinity; > + > + /** Notify the member port of bonding port information */ > + eth_bond_notify_member bond_notify_member; > }; > > /**