From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id E406443C5A; Wed, 6 Mar 2024 21:21:07 +0100 (CET) Received: from mails.dpdk.org (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id D2F5442E71; Wed, 6 Mar 2024 21:21:07 +0100 (CET) Received: from mail-pl1-f171.google.com (mail-pl1-f171.google.com [209.85.214.171]) by mails.dpdk.org (Postfix) with ESMTP id 07EA042E6F for ; Wed, 6 Mar 2024 21:21:07 +0100 (CET) Received: by mail-pl1-f171.google.com with SMTP id d9443c01a7336-1d944e8f367so1250335ad.0 for ; Wed, 06 Mar 2024 12:21:06 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=networkplumber-org.20230601.gappssmtp.com; s=20230601; t=1709756466; x=1710361266; darn=dpdk.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:subject:cc:to:from:date:from:to:cc:subject:date :message-id:reply-to; bh=tqTmdqnEqr5VpSA6XQ/fjcumfesbYWYQ7vb/tilL8Rk=; b=lQY11OBVDWiQNDRXGWv9qu/oZxfsPrU3dlxInmrEgK9rvxxwsfGdly8kO7YwrzaDN0 6BKSPzMy+1RV9wkQYoZIeApY6JQ6uajVWfJIVIAra66cv98j1V5R1HI165n/ZzWny/7K CJ7FdacRrMELHLu4MwAmmMYPbPQ75SR8Kb7Pcg9dlQ03I7EfcCR54i+Fn4cc/YJAlM0i T+CKd9aZHQWGyYn00iGeehprZ+SKvT09jVP9CzkFTbQ8aEXwI/2TynL5UufFq3feYA39 +MyC7P/2Xl7qR6Zkd8gU5payMzYiiVaLxyiBob1OMjym19+TpZBsy63MU22kEu305Ihq cB2w== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1709756466; x=1710361266; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:subject:cc:to:from:date:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=tqTmdqnEqr5VpSA6XQ/fjcumfesbYWYQ7vb/tilL8Rk=; b=G+PMU+jpQWGoqL8nBjxEg28LRUvWM4BCZMfwzuyp3BaCaNYeHdq+MTIccAuCVDx0ku sjHRaN/25nJXvUvld5smdpuSrhGnItcjqIj+fIDSOGvbqAj0kezoe/DAvxYkkosSX5Wd pUIGmnzkAe0mZiSoTT4t1f1Ilvt8+fYvNJjawhBlyPc/5KuJx4ESKY/gqcU11zPMpUv9 AkB1X+/GeV/Tpp7mXpQhQnOuzQfcQKgoylwHSMtB50yY/6yxAFwWh+SqZ+XTyPnGFkui NNbhxW0SX9AejrICLahFDw5w/0abGaIurIz0RxYKcJ9jbYFdQNzCTmXWK5he8tcNK8BT p1xQ== X-Gm-Message-State: AOJu0YwCYUUnVzBplWiHgtj4J5cDRFVf2FV7Fjxc4NX991jjXtaMFsq5 ioIANqq2KTyqInHeBiM947JUOTOC1ZyyCM8Lm8XXHnQzdqiA4Fl/HWuDFm4kpes= X-Google-Smtp-Source: AGHT+IHK09m0gA2JVyrPYXQtPE/Y2tZ5J9vgYGsFt6qT3hiTN8/Q/r3Dht5jXS+TQi3QWtjIXwJZMQ== X-Received: by 2002:a17:902:7845:b0:1dc:b382:da8d with SMTP id e5-20020a170902784500b001dcb382da8dmr5354796pln.38.1709756466131; Wed, 06 Mar 2024 12:21:06 -0800 (PST) Received: from hermes.local (204-195-123-141.wavecable.com. [204.195.123.141]) by smtp.gmail.com with ESMTPSA id u10-20020a170903124a00b001d54b763995sm12955165plh.129.2024.03.06.12.21.05 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 06 Mar 2024 12:21:05 -0800 (PST) Date: Wed, 6 Mar 2024 12:21:04 -0800 From: Stephen Hemminger To: Ferruh Yigit Cc: dev@dpdk.org Subject: Re: [PATCH] net/tap: allow more that 4 queues Message-ID: <20240306122104.143a7424@hermes.local> In-Reply-To: References: <20240229175620.122949-1-stephen@networkplumber.org> MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org On Wed, 6 Mar 2024 16:14:51 +0000 Ferruh Yigit wrote: > On 2/29/2024 5:56 PM, Stephen Hemminger wrote: > > The tap device needs to exchange file descriptors for tx and rx. > > But the EAL MP layer has limit of 8 file descriptors per message. > > The ideal resolution would be to increase the number of file > > descriptors allowed for rte_mp_sendmsg(), but this would break > > the ABI. Workaround the constraint by breaking into multiple messages. > > > > Do not hide errors about MP message failures. > > > > Signed-off-by: Stephen Hemminger > > --- > > drivers/net/tap/rte_eth_tap.c | 40 +++++++++++++++++++++++++++++------ > > 1 file changed, 33 insertions(+), 7 deletions(-) > > > > diff --git a/drivers/net/tap/rte_eth_tap.c b/drivers/net/tap/rte_eth_tap.c > > index 69d9da695bed..df18c328f498 100644 > > --- a/drivers/net/tap/rte_eth_tap.c > > +++ b/drivers/net/tap/rte_eth_tap.c > > @@ -863,21 +863,44 @@ tap_mp_req_on_rxtx(struct rte_eth_dev *dev) > > msg.fds[fd_iterator++] = process_private->txq_fds[i]; > > msg.num_fds++; > > request_param->txq_count++; > > + > > + /* Need to break request into chunks */ > > + if (fd_iterator >= RTE_MP_MAX_FD_NUM) { > > + err = rte_mp_sendmsg(&msg); > > + if (err < 0) > > + goto fail; > > + > > + fd_iterator = 0; > > + msg.num_fds = 0; > > + request_param->txq_count = 0; > > + } > > } > > for (i = 0; i < dev->data->nb_rx_queues; i++) { > > msg.fds[fd_iterator++] = process_private->rxq_fds[i]; > > msg.num_fds++; > > request_param->rxq_count++; > > + > > + if (fd_iterator >= RTE_MP_MAX_FD_NUM) { > > + err = rte_mp_sendmsg(&msg); > > + if (err < 0) > > + goto fail; > > + > > + fd_iterator = 0; > > + msg.num_fds = 0; > > + request_param->rxq_count = 0; > > + } > > } > > Hi Stephen, > > Did you able to verify with more than 4 queues? > > As far as I can see, in the secondary counterpart of the > 'rte_mp_sendmsg()', each time secondary index starts from 0, and > subsequent calls overwrites the fds in secondary. > So practically still only 4 queues works. I got 4 queues setup, but looks like they are trash in secondary. Probably best to revert this and fix it by bumping RTE_MP_MAX_FD_NUM. This is better, but does take some ABI issue handling.