RE: [PATCH v5 1/2] ethdev: introduce the Tx map API for aggregated ports

["Jiawei(Jonny) Wang" <jiaweiw@nvidia.com>]

> -----Original Message-----
> From: Andrew Rybchenko <andrew.rybchenko@oktetlabs.ru>
> Sent: Friday, February 17, 2023 4:24 PM
> To: Ferruh Yigit <ferruh.yigit@amd.com>; Jiawei(Jonny) Wang
> <jiaweiw@nvidia.com>; Slava Ovsiienko <viacheslavo@nvidia.com>; Ori Kam
> <orika@nvidia.com>; NBU-Contact-Thomas Monjalon (EXTERNAL)
> <thomas@monjalon.net>; Aman Singh <aman.deep.singh@intel.com>; Yuying
> Zhang <yuying.zhang@intel.com>
> Cc: dev@dpdk.org; Raslan Darawsheh <rasland@nvidia.com>
> Subject: Re: [PATCH v5 1/2] ethdev: introduce the Tx map API for aggregated
> ports
> 
> On 2/16/23 20:58, Ferruh Yigit wrote:
> > On 2/14/2023 3:48 PM, Jiawei Wang wrote:
> >> When multiple ports are aggregated into a single DPDK port,
> >> (example: Linux bonding, DPDK bonding, failsafe, etc.), we want to
> >> know which port use for Tx via a queue.
> >>
> >> This patch introduces the new ethdev API
> >> rte_eth_dev_map_aggr_tx_affinity(), it's used to map a Tx queue with
> >> an aggregated port of the DPDK port (specified with port_id), The
> >> affinity is the number of the aggregated port.
> >> Value 0 means no affinity and traffic could be routed to any
> >> aggregated port, this is the default current behavior.
> >>
> >> The maximum number of affinity is given by
> rte_eth_dev_count_aggr_ports().
> >>
> >> Add the trace point for ethdev rte_eth_dev_count_aggr_ports() and
> >> rte_eth_dev_map_aggr_tx_affinity() functions.
> >>
> >> Add the testpmd command line:
> >> testpmd> port config (port_id) txq (queue_id) affinity (value)
> >>
> >> For example, there're two physical ports connected to a single DPDK
> >> port (port id 0), and affinity 1 stood for the first physical port
> >> and affinity 2 stood for the second physical port.
> >> Use the below commands to config tx phy affinity for per Tx Queue:
> >>          port config 0 txq 0 affinity 1
> >>          port config 0 txq 1 affinity 1
> >>          port config 0 txq 2 affinity 2
> >>          port config 0 txq 3 affinity 2
> >>
> >> These commands config the Tx Queue index 0 and Tx Queue index 1 with
> >> phy affinity 1, uses Tx Queue 0 or Tx Queue 1 send packets, these
> >> packets will be sent from the first physical port, and similar with
> >> the second physical port if sending packets with Tx Queue 2 or Tx
> >> Queue 3.
> >>
> >> Signed-off-by: Jiawei Wang <jiaweiw@nvidia.com>
> >
> > <...>
> >
> >> diff --git a/lib/ethdev/rte_ethdev.c b/lib/ethdev/rte_ethdev.c index
> >> dc0a4eb12c..1d5b3a16b2 100644
> >> --- a/lib/ethdev/rte_ethdev.c
> >> +++ b/lib/ethdev/rte_ethdev.c
> >> @@ -6915,6 +6915,55 @@
> rte_eth_buffer_split_get_supported_hdr_ptypes(uint16_t port_id, uint32_t
> *ptypes
> >>   	return j;
> >>   }
> >>
> >> +int rte_eth_dev_count_aggr_ports(uint16_t port_id) {
> >> +	struct rte_eth_dev *dev;
> >> +	int ret;
> >> +
> >> +	RTE_ETH_VALID_PORTID_OR_ERR_RET(port_id, -ENODEV);
> >> +	dev = &rte_eth_devices[port_id];
> >> +
> >> +	if (*dev->dev_ops->count_aggr_ports == NULL)
> >> +		return -ENOTSUP;
> >
> > What do you think to return a default value when dev_ops is not
> > defined, assuming device is not a bounded device.
> > Not sure which one is better for application, return a default value
> > or error.
> 
> I think 0 is better here. It simply means that
> rte_eth_dev_map_aggr_tx_affinity() cannot be used as well as corresponding
> flow API item.
> It will be true even for bonding as long as corresponding API is not supported.
> 

Will send the new patch later with this change.

> >> +	ret = eth_err(port_id, (*dev->dev_ops->count_aggr_ports)(port_id));
> >> +
> >> +	rte_eth_trace_count_aggr_ports(port_id, ret);
> >> +
> >> +	return ret;
> >> +}
> >> +
> >> +int rte_eth_dev_map_aggr_tx_affinity(uint16_t port_id, uint16_t
> tx_queue_id,
> >> +				     uint8_t affinity)
> >> +{
> >> +	struct rte_eth_dev *dev;
> >> +	int ret;
> >> +
> >> +	RTE_ETH_VALID_PORTID_OR_ERR_RET(port_id, -ENODEV);
> >> +	dev = &rte_eth_devices[port_id];
> >> +
> >> +	if (tx_queue_id >= dev->data->nb_tx_queues) {
> >> +		RTE_ETHDEV_LOG(ERR, "Invalid Tx queue_id=%u\n",
> tx_queue_id);
> >> +		return -EINVAL;
> >> +	}
> >> +
> >
> > Although documentation says this API should be called before
> > configure,
> 
> Documentation says "after". Anyway, it is better to check vs dev_configured.
> 

Yes, after device configure, I add the checking and will send the new patch.

> > if user misses it I guess above can crash, is there a way to add
> > runtime check, like checking 'dev->data->dev_configured'?
> >
> >
> >> +	if (*dev->dev_ops->map_aggr_tx_affinity == NULL)
> >> +		return -ENOTSUP;
> >> +
> >> +	if (dev->data->dev_started) {
> >> +		RTE_ETHDEV_LOG(ERR,
> >> +			"Port %u must be stopped to allow configuration\n",
> >> +			port_id);
> >> +		return -EBUSY;
> >> +	}
> >> +
> >> +	ret = eth_err(port_id, (*dev->dev_ops->map_aggr_tx_affinity)(port_id,
> >> +				tx_queue_id, affinity));
> >> +
> >
> > Should API check if port_id is a bonding port before it continue with
> > mapping?
> 
> Since it is a control path I think it is a good idea to call
> rte_eth_dev_count_aggr_ports() and chck affinity value.

OK, add the API call before map and check the affinity value as well.

Will send the v6 patch include all comments/suggestions.

Thanks.