RE: [PATCH v3 1/8] ethdev: add member notification for bonding port

[Chaoyong He <chaoyong.he@corigine.com>]

> 在 2023/10/9 11:11, Long Wu 写道:
> >> Hi Chaoyong,
> >>
> >> some comments as below.
> >>
> >>
> >> 在 2023/10/8 9:50, Chaoyong He 写道:
> >>> From: Long Wu <long.wu@corigine.com>
> >>>
> >>> Bonding PMD does not let member ports know the bonding port's
> >>> information, like how many member ports the bonding port has, what
> >>> mode the bonding port is in and so on.
> >>>
> >>> Add the notification interface for bonding port to let member port
> >>> know it is added to a bonding port and what the bonding port's
> >>> configuration is. If so the member ports have chance to achieve its
> >>> bond-flow-offlod or other private bonding functions.
> >> "its bond-flow-offlod or other private bonding functions"
> >> I wonder that what PMDs can do with this.
> >> Can you give an example in PMD to help others review?
> > After this patch series, I will send out nfp PMD code about "its bond-flow-
> offlod or other private bonding functions ".
> > I can explain here:
> > "bond-flow" means the flow rule's destination port is a bonding port.
> > Now DPDK can use bonding port as the source port in a flow rule and
> member ports can offload this flow.
> > But member ports cannot offload a flow that its destination port is a
> bonding port.
> > Because the member ports don't know the bonding port. (Of course, some
> PMDs has its self-function to let member ports know the bonding port but it
> doesn't a general "DPDK" way).
> > After this "notify patch", DPDK can do "bond-flow-offload", also "other
> private bonding functions"(like hardware balance policy, primary port
> changing and so on) can work.
> I think what you said is more like "bonding offload", right?
> It seems that you cannot do it just based on current these API in this series.
> You probably have other works to upstream for like "bond-flow" and "other
> private bonding functions".
> 

All the efforts are to make the NIC hardware aware that port has been added to the bonding port and then create a bonding port on hardware.
Based on this(Bonding port notification and member port hardware create bonding port), we can do "bond-flow-offload" or "other private bonding functions".

Taking "bond-flow-offload" as an example:
Step1: bonding port configure process --> bonding port notify member ports --> member ports know bonding port configuration --> member ports hardware creates bonding port.

Step2: app adds a flow rule about "bond-flow"(See the explanation above) --> PMD analyzes "bond-flow" then the hardware offloads this flow. (In the next patch series, and not affair the bonding PMD)

> >
> >>> Signed-off-by: Long Wu <long.wu@corigine.com>
> >>> Reviewed-by: James Hershaw <james.hershaw@corigine.com>
> >>> Reviewed-by: Chaoyong He <chaoyong.he@corigine.com>
> >>> ---
> >>>    drivers/net/bonding/eth_bond_private.h |  1 +
> >>>    drivers/net/bonding/rte_eth_bond.h     | 46 ++++++++++++++++
> >>>    drivers/net/bonding/rte_eth_bond_api.c | 73
> >> ++++++++++++++++++++++++++
> >>>    drivers/net/bonding/rte_eth_bond_pmd.c | 27 ++++++++--
> >>>    drivers/net/bonding/version.map        |  3 ++
> >>>    lib/ethdev/ethdev_driver.h             | 18 +++++++
> >>>    6 files changed, 165 insertions(+), 3 deletions(-)
> >>>
> >>> diff --git a/drivers/net/bonding/eth_bond_private.h
> >>> b/drivers/net/bonding/eth_bond_private.h
> >>> index e688894210..f69e85c199 100644
> >>> --- a/drivers/net/bonding/eth_bond_private.h
> >>> +++ b/drivers/net/bonding/eth_bond_private.h
> >>> @@ -180,6 +180,7 @@ struct bond_dev_private {
> >>>     uint8_t member_update_idx;
> >>>
> >>>     bool kvargs_processing_is_done;
> >>> +   bool notify_member; /**< Enable member notification of bonding port.
> >>> +*/
> >>>
> >>>     uint32_t candidate_max_rx_pktlen;
> >>>     uint32_t max_rx_pktlen;
> >>> diff --git a/drivers/net/bonding/rte_eth_bond.h
> >>> b/drivers/net/bonding/rte_eth_bond.h
> >>> index f10165f2c6..737beca446 100644
> >>> --- a/drivers/net/bonding/rte_eth_bond.h
> >>> +++ b/drivers/net/bonding/rte_eth_bond.h
> >>> @@ -351,6 +351,52 @@
> rte_eth_bond_link_up_prop_delay_set(uint16_t
> >> bonding_port_id,
> >>>    int
> >>>    rte_eth_bond_link_up_prop_delay_get(uint16_t bonding_port_id);
> >>>
> >>> +/**
> >>> + * Set the flag of whether bonding port notifies member ports.
> >>> + *
> >>> + * @param bonding_port_id
> >>> + *   Port ID of bonding device.
> >>> + * @param notify_member
> >>> + *   Flag of whether bonding port notifies member ports.
> >>> + *
> >>> + * @return
> >>> + *   0 on success, negative value otherwise.
> >>> + */
> >>> +__rte_experimental
> >>> +int
> >>> +rte_eth_bond_notify_member_flag_set(uint16_t bonding_port_id, bool
> >>> +notify_member);
> >> s/notify_membe/notify in input?
> >> because function name reveals the meaning already.
> > Thank you, you are right.
> >
> >>> +
> >>> +/**
> >>> + * Get the flag of whether bonding port notifies member ports.
> >>> + *
> >>> + * @param bonding_port_id
> >>> + *   Port ID of bonding device.
> >>> + * @param notify_member
> >>> + *   Flag of whether bonding port notifies member ports.
> >>> + *
> >>> + * @return
> >>> + *   0 on success, negative value otherwise.
> >>> + */
> >>> +__rte_experimental
> >>> +int
> >>> +rte_eth_bond_notify_member_flag_get(uint16_t bonding_port_id, bool
> >>> +*notify_member);
> >>> +
> >>> +/**
> >>> + * Notify the member ports of bonding port's information.
> >>> + *
> >>> + * This interface is called in the following functions:
> >>> + * - bond_ethdev_lsc_event_callback()
> >>> + * - bond_ethdev_configure()
> >> Is this interface used just  in these cases?
> >> If so I don't think it should be a API in eth_dev_op.
> > Sorry I'm a bit confused.
> > Do you mean "rte_eth_bond_notify_members" this interface? This interface
> is called in 3 functions.
> User can also call this API, right?

Sorry, I think the user should not call API 'rte_eth_bond_notify_members()', it's the bonding PMD call it.
There exist a so-called `notify_member_flag` in the bonding PMD, which control if the 'rte_eth_bond_notify_members()' will be invoked.
And this `notify_member_flag` is controlled by ` rte_eth_bond_notify_member_flag_set()` and ` rte_eth_bond_notify_member_flag_get()`.

> How should the user use it? Is it when update bonding port information?
> I feel like to know what the time of the notice is.
> >

My commit "net/bonding: add commands for bonding port notification " in this patch series is for app to set the "notify flag".
For example:
testpmd> port stop all
testpmd> create bonded device 4 1
testpmd> set bonding notify_member 2 enable.
testpmd> set bonding mode 4 2
testpmd> set bonding balance_xmit_policy 2 l34.
testpmd> add bonding member 0 2
testpmd> add bonding member 1 2
testpmd> port start 2
("set bonding notify_member 2 enable " is a new command to set  the "notify_member_flag".)

I also add this setting in "vdev", like:
--vdev 'net_bonding0,member=<port 1>,member=<port 2>,socket_id=1,xmit_policy=l34,notify_member=enable'

> >>> ...
> >>> +   struct rte_eth_dev *member_dev[RTE_MAX_ETHPORTS];
> >>> +
> >>> +   if (valid_bonding_port_id(bonding_port_id) != 0)
> >>> +           return -EINVAL;
> >>> +
> >>> +   bond_dev = &rte_eth_devices[bonding_port_id];
> >>> +   internals = bond_dev->data->dev_private;
> >>> +
> >>> +   for (i = 0; i < internals->member_count; i++) {
> >>> +           member_port_id = internals->members[i].port_id;
> >>> +           member_dev[i] = &rte_eth_devices[member_port_id];
> >>> +           if (*member_dev[i]->dev_ops->bond_notify_member == NULL)
> >>> +                   return -ENOTSUP;
> >>> +   }
> >> Do we need all member ports support bond_notify_member api?
> >> If allow user to select one member port to notify? This might be more
> general.
> >> In addition, this action here is inconsistent with above handle(do
> >> not notify member if doesn't supportbond_notify_member API).
> > Yes, we do not need all member ports support this API.
> > I just want to have a stricter restriction on this feature before, but I think
> your suggestion is better.
> anyway, we need to have a clearly comments about this.

Agree, I will add comments in next version.
Thanks

> >
> >>> ...