Re: [PATCH] net/tap: allow more that 4 queues

[Stephen Hemminger <stephen@networkplumber.org>]

On Wed, 6 Mar 2024 16:14:51 +0000
Ferruh Yigit <ferruh.yigit@amd.com> wrote:

> On 2/29/2024 5:56 PM, Stephen Hemminger wrote:
> > The tap device needs to exchange file descriptors for tx and rx.
> > But the EAL MP layer has limit of 8 file descriptors per message.
> > The ideal resolution would be to increase the number of file
> > descriptors allowed for rte_mp_sendmsg(), but this would break
> > the ABI. Workaround the constraint by breaking into multiple messages.
> > 
> > Do not hide errors about MP message failures.
> > 
> > Signed-off-by: Stephen Hemminger <stephen@networkplumber.org>
> > ---
> >  drivers/net/tap/rte_eth_tap.c | 40 +++++++++++++++++++++++++++++------
> >  1 file changed, 33 insertions(+), 7 deletions(-)
> > 
> > diff --git a/drivers/net/tap/rte_eth_tap.c b/drivers/net/tap/rte_eth_tap.c
> > index 69d9da695bed..df18c328f498 100644
> > --- a/drivers/net/tap/rte_eth_tap.c
> > +++ b/drivers/net/tap/rte_eth_tap.c
> > @@ -863,21 +863,44 @@ tap_mp_req_on_rxtx(struct rte_eth_dev *dev)
> >  		msg.fds[fd_iterator++] = process_private->txq_fds[i];
> >  		msg.num_fds++;
> >  		request_param->txq_count++;
> > +
> > +		/* Need to break request into chunks */
> > +		if (fd_iterator >= RTE_MP_MAX_FD_NUM) {
> > +			err = rte_mp_sendmsg(&msg);
> > +			if (err < 0)
> > +				goto fail;
> > +
> > +			fd_iterator = 0;
> > +			msg.num_fds = 0;
> > +			request_param->txq_count = 0;
> > +		}
> >  	}
> >  	for (i = 0; i < dev->data->nb_rx_queues; i++) {
> >  		msg.fds[fd_iterator++] = process_private->rxq_fds[i];
> >  		msg.num_fds++;
> >  		request_param->rxq_count++;
> > +
> > +		if (fd_iterator >= RTE_MP_MAX_FD_NUM) {
> > +			err = rte_mp_sendmsg(&msg);
> > +			if (err < 0)
> > +				goto fail;
> > +
> > +			fd_iterator = 0;
> > +			msg.num_fds = 0;
> > +			request_param->rxq_count = 0;
> > +		}
> >  	}  
> 
> Hi Stephen,
> 
> Did you able to verify with more than 4 queues?
> 
> As far as I can see, in the secondary counterpart of the
> 'rte_mp_sendmsg()', each time secondary index starts from 0, and
> subsequent calls overwrites the fds in secondary.
> So practically still only 4 queues works.

I got 4 queues setup, but looks like they are trash in secondary.
Probably best to revert this and fix it by bumping RTE_MP_MAX_FD_NUM.
This is better, but does take some ABI issue handling.