From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 66A38A0353; Thu, 24 Feb 2022 10:28:57 +0100 (CET) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 327D041155; Thu, 24 Feb 2022 10:28:57 +0100 (CET) Received: from out4-smtp.messagingengine.com (unknown [66.111.4.28]) by mails.dpdk.org (Postfix) with ESMTP id A13824114D for ; Thu, 24 Feb 2022 10:28:56 +0100 (CET) Received: from compute3.internal (compute3.nyi.internal [10.202.2.43]) by mailout.nyi.internal (Postfix) with ESMTP id 44BDD5C0A46; Thu, 24 Feb 2022 04:28:50 -0500 (EST) Received: from mailfrontend2 ([10.202.2.163]) by compute3.internal (MEProxy); Thu, 24 Feb 2022 04:28:50 -0500 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=monjalon.net; h= cc:cc:content-transfer-encoding:content-type:date:date:from:from :in-reply-to:in-reply-to:message-id:mime-version:references :reply-to:sender:subject:subject:to:to; s=fm3; bh=NyJwYW/8IaayzR BcK8V/RE85NumTTz/Gqd+sf5JcivI=; b=P1/7foWT07siShC1zVBpK4Usrq+3dl iPkCM24rCfNn+KKtog12aa+fu0BoWZ3rnG6UulrS7XkjIAbK/BM0ZwGpAnS3SbGc tJ9iCjojsNwc7WzgPapY4oIK9xyLStZTi6dJddV85Ci4eY7hnUMLIn0ojXGWQMT6 +hiiL7mR2zHArS/8fzlEOzX0UVl0SNWcHrtY7Ew7nwxwxhy7mY5Rc8/9ukdkmsix 3q07VS7X9zyDjDBw6d5QskgVknLdYPR1q3c40+goFctH7VvD/pRxnMx8anq4vUjG uayqGZA49ydsFv571bL8XAVzy1LlItU+8gdmEDpPR8VDEgpybdy6Tg2Q== DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d= messagingengine.com; h=cc:cc:content-transfer-encoding :content-type:date:date:from:from:in-reply-to:in-reply-to :message-id:mime-version:references:reply-to:sender:subject :subject:to:to:x-me-proxy:x-me-proxy:x-me-sender:x-me-sender :x-sasl-enc; s=fm2; bh=NyJwYW/8IaayzRBcK8V/RE85NumTTz/Gqd+sf5Jci vI=; b=VWxhkuwKvztiSrXM9YiBCLjyMAKax29aIsLUJHkxsRVVPE7iC4Cd/1e2X SA8nFUOPrJKZgmpiJwPfNXNoN5JC/Bc4fgmoMAhzUTXvJVNjZzO2R4qN3ESRHm0K ITgmSTxDUeb7LlpfpXycIJ6FSPr26WZhYlcd+zHGa+DDBBlU8ZCgaOWfYHzWpA5w Emq8eUArNaI9KWTdfNwtWV1RSln317dImvNa/N8YPNl7TvWG4Z6LgRWnW33gtNIb xaTslP6aY1q7YfLwmZ9wmVeoNLatuAb6XnAe0kor69FFJ+QMs1OIYzGqKe6/asjs FmA6ztE5CFN0qiyiWBh055V4BonyA== X-ME-Sender: X-ME-Received: X-ME-Proxy-Cause: gggruggvucftvghtrhhoucdtuddrgedvvddrledvgddtgecutefuodetggdotefrodftvf curfhrohhfihhlvgemucfhrghsthforghilhdpqfgfvfdpuffrtefokffrpgfnqfghnecu uegrihhlohhuthemuceftddtnecusecvtfgvtghiphhivghnthhsucdlqddutddtmdenuc fjughrpefhvffufffkjghfggfgtgesthfuredttddtvdenucfhrhhomhepvfhhohhmrghs ucfoohhnjhgrlhhonhcuoehthhhomhgrshesmhhonhhjrghlohhnrdhnvghtqeenucggtf frrghtthgvrhhnpedugefgvdefudfftdefgeelgffhueekgfffhfeujedtteeutdejueei iedvffegheenucevlhhushhtvghrufhiiigvpedtnecurfgrrhgrmhepmhgrihhlfhhroh hmpehthhhomhgrshesmhhonhhjrghlohhnrdhnvght X-ME-Proxy: Received: by mail.messagingengine.com (Postfix) with ESMTPA; Thu, 24 Feb 2022 04:28:48 -0500 (EST) From: Thomas Monjalon To: Andrew Rybchenko , Ferruh Yigit , Dmitry Kozlyuk Cc: dev@dpdk.org, Raslan Darawsheh , Slava Ovsiienko , "Li, Xiaoyun" , "Zhang, Yuying" , "Singh, Aman Deep" , "dev@dpdk.org" Subject: Re: [PATCH] app/testpmd: skip stopped queues when forwarding Date: Thu, 24 Feb 2022 10:28:46 +0100 Message-ID: <109715067.nniJfEyVGO@thomas> In-Reply-To: References: <20220113092103.282538-1-dkozlyuk@nvidia.com> MIME-Version: 1.0 Content-Transfer-Encoding: 7Bit Content-Type: text/plain; charset="us-ascii" X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org 21/02/2022 09:58, Dmitry Kozlyuk: > Andrew, Ferruh, Thomas, which behavior does ethdev assume (see below)? For the whole device stop, this is the documentation: " The transmit and receive functions should not be invoked when the device is stopped. " There is also this comment on rte_eth_dev_reset: " Note: To avoid unexpected behavior, the application should stop calling Tx and Rx functions before calling rte_eth_dev_reset( ). For thread safety, all these controlling functions should be called from the same thread. " For queue stop, there is no documented expectation. There is this comment for queue callback removal: " The memory for the callback can be subsequently freed back by the application by calling rte_free(): - Immediately - if the port is stopped, or the user knows that no callbacks are in flight e.g. if called from the thread doing Rx/Tx on that queue. - After a short delay - where the delay is sufficient to allow any in-flight callbacks to complete. Alternately, the RCU mechanism can be used to detect when data plane threads have ceased referencing the callback memory. " > > This patch was created with the assumption > > that stopped queues may not be used for Rx/Tx. > > No-op behavior of rte_eth_rx/tx_burst() > > for a stopped queue is not documented. Indeed, it is not documented. I suggest working on this documentation first. testpmd could be adjusted later if needed. > > Yes, at least some PMDs implement it this way. > > But is this behavior intended? > > > > It is the application that stops the queue or starts it deferred. > > Stopping is non-atomic, so polling the queue is not allowed during it. > > Hence, if the application intends to stop queues, it must either: > > > > a) Know the queue is not polled before stopping it. > > Use case: a secondary that was polling the queue has crashed, > > the primary is doing a recovery to free all mbufs. > > There is no issue since there is no poller to touch the queue. > > > > b) Tell the poller to skip the queue for the time of stopping it. > > Use case: deferred queue start or queue reconfiguration. > > But if the poller is cooperating anyway, > > what prevents it from not touching the queue for longer? > > > > The same considerations apply to starting a queue. > > > > No-op behavior is convenient from the application point of view. > > But it also means that pollers of stopped queues > > will go all the way down to PMD Rx/Tx routines, wasting cycles, > > and some PMDs will do a check for the queue state, > > even though it may never be needed for a particular application. Yes that's the question: Do we want 1/ to allow apps to poll stopped queues, adding checks in PMDs, or do we prefer 2/ saving check cycles and expect apps to not poll?