Re: [dpdk-dev] [PATCH] app/testpmd: fix DCB config issue on ixgbe

["Lu, Wenzhuo" <wenzhuo.lu@intel.com>]

Hi Bernard,

> -----Original Message-----
> From: Iremonger, Bernard
> Sent: Wednesday, August 24, 2016 11:22 PM
> To: Iremonger, Bernard; Lu, Wenzhuo; dev@dpdk.org
> Cc: De Lara Guarch, Pablo; Lu, Wenzhuo
> Subject: RE: [dpdk-dev] [PATCH] app/testpmd: fix DCB config issue on ixgbe
> 
> Hi Wenzhuo,
> 
> <snip>
> 
> > > Subject: [dpdk-dev] [PATCH] app/testpmd: fix DCB config issue on
> > > ixgbe
> > >
> > > An issue is found that DCB cannot be configured on ixgbe NICs. It's
> > > said the TX queue number is not right.
> > > On ixgbe the max TX queue number is not fixed, it depends on the
> > > multi- queue mode.
> > >
> > > This patch adds the device configuration before getting info in the
> > > DCB configuration process. So the right info can be got depending on
> > > the configuration.
> > >
> > > Fixes: 96c0450dff86 (ixgbe: fix dropping packets from unsupported Tx
> > > queues)
> 
> As the fix in this patch is to testpmd, I don't think the fixes line is correct.
The bug is introduced by this patch. Before this patch, APP need not care about the multi-queue mode when it want to get the max queue number.

> 
> > > Signed-off-by: Wenzhuo Lu <wenzhuo.lu@intel.com>
> > > ---
> > >  app/test-pmd/testpmd.c | 39 +++++++++++++++++++++------------------
> > >  1 file changed, 21 insertions(+), 18 deletions(-)
> > >
> > > diff --git a/app/test-pmd/testpmd.c b/app/test-pmd/testpmd.c index
> > > 1428974..ba41bea 100644
> > > --- a/app/test-pmd/testpmd.c
> > > +++ b/app/test-pmd/testpmd.c
> > > @@ -1962,17 +1962,30 @@ init_port_dcb_config(portid_t pid,
> > >  		     uint8_t pfc_en)
> > >  {
> > >  	struct rte_eth_conf port_conf;
> > > -	struct rte_eth_dev_info dev_info;
> > >  	struct rte_port *rte_port;
> > >  	int retval;
> > >  	uint16_t i;
> > >
> > > -	rte_eth_dev_info_get(pid, &dev_info);
> > > +	rte_port = &ports[pid];
> > > +
> > > +	memset(&port_conf, 0, sizeof(struct rte_eth_conf));
> > > +	/* Enter DCB configuration status */
> > > +	dcb_config = 1;
> > > +
> > > +	/*set configuration of DCB in vt mode and DCB in non-vt mode*/
> > > +	retval = get_eth_dcb_conf(&port_conf, dcb_mode, num_tcs,
> > > pfc_en);
> > > +	if (retval < 0)
> > > +		return retval;
> > > +	port_conf.rxmode.hw_vlan_filter = 1;
> > > +
> > > +	(void)rte_eth_dev_configure(pid, 0, 0, &port_conf);
> >
> > The return value of rte_eth_dev_configure() should be checked.
> > Calling rte_eth_dev_configure() with  parameters nb_rx_q and nb_tx_q
> > equal to 0 returns -EINVAL, and does nothing.
> > Should the values of nb_rx_q and nb_tx_q be non zero?
The 0 is used on purpose. Because I don't want to configure the queues. The only purpose is to  make all the configuration to write into the device. And that's why the return value is ignored by (void).

> 
> The call to rte_eth_dev_configure() may not be necessary, as it is also called
> when the port is started.
In rte_ethdev.h, we can see it's said that rte_eth_dev_configure should be called before any other function of ethernet API. As the DCB configuration is changed here. We have to call rte_eth_dev_configure to make the configuration is right before we can use rte_eth_dev_info_get.

> 
> > > +	rte_eth_dev_info_get(pid, &rte_port->dev_info);
> > >
> > >  	/* If dev_info.vmdq_pool_base is greater than 0,
> > >  	 * the queue id of vmdq pools is started after pf queues.
> > >  	 */
> > > -	if (dcb_mode == DCB_VT_ENABLED && dev_info.vmdq_pool_base >
> > > 0) {
> > > +	if (dcb_mode == DCB_VT_ENABLED &&
> > > +	    rte_port->dev_info.vmdq_pool_base > 0) {
> > >  		printf("VMDQ_DCB multi-queue mode is nonsensical"
> > >  			" for port %d.", pid);
> > >  		return -1;
> > > @@ -1982,13 +1995,13 @@ init_port_dcb_config(portid_t pid,
> > >  	 * and has the same number of rxq and txq in dcb mode
> > >  	 */
> > >  	if (dcb_mode == DCB_VT_ENABLED) {
> > > -		nb_rxq = dev_info.max_rx_queues;
> > > -		nb_txq = dev_info.max_tx_queues;
> > > +		nb_rxq = rte_port->dev_info.max_rx_queues;
> > > +		nb_txq = rte_port->dev_info.max_tx_queues;
> 
> If nb_rxq and nb_txq are set to max_rx_queues and max_tx_queues respectively,
> there is a failure when the port is started in ixgbe_check_mq_mode() at line
> 1990 in ixgbe_ethdev.c.
> SRIOV is active, nb_rx_q=128 nb_tx_q=128 queue number must be less than or
> equal to 1.
I haven't hit this issue. Would you like to give more details about how to hit it? I'll check if I miss something.

> 
> nb_rxq and nb_txq are equal to 1 at this point, if they are not changed, port start
> completes successfully.
> 
> 
> > >  	} else {
> > >  		/*if vt is disabled, use all pf queues */
> > > -		if (dev_info.vmdq_pool_base == 0) {
> > > -			nb_rxq = dev_info.max_rx_queues;
> > > -			nb_txq = dev_info.max_tx_queues;
> > > +		if (rte_port->dev_info.vmdq_pool_base == 0) {
> > > +			nb_rxq = rte_port->dev_info.max_rx_queues;
> > > +			nb_txq = rte_port->dev_info.max_tx_queues;
> > >  		} else {
> > >  			nb_rxq = (queueid_t)num_tcs;
> > >  			nb_txq = (queueid_t)num_tcs;
> > > @@ -1997,16 +2010,6 @@ init_port_dcb_config(portid_t pid,
> > >  	}
> > >  	rx_free_thresh = 64;
> > >
> > > -	memset(&port_conf, 0, sizeof(struct rte_eth_conf));
> > > -	/* Enter DCB configuration status */
> > > -	dcb_config = 1;
> > > -
> > > -	/*set configuration of DCB in vt mode and DCB in non-vt mode*/
> > > -	retval = get_eth_dcb_conf(&port_conf, dcb_mode, num_tcs,
> > > pfc_en);
> > > -	if (retval < 0)
> > > -		return retval;
> > > -
> > > -	rte_port = &ports[pid];
> > >  	memcpy(&rte_port->dev_conf, &port_conf, sizeof(struct
> > > rte_eth_conf));
> > >
> > >  	rxtx_port_config(rte_port);
> > > --
> > > 1.9.3
> Regards,
> 
> Bernard