Re: [PATCH] app/testpmd: skip stopped queues when forwarding

[Thomas Monjalon <thomas@monjalon.net>]

21/02/2022 09:58, Dmitry Kozlyuk:
> Andrew, Ferruh, Thomas, which behavior does ethdev assume (see below)?

For the whole device stop, this is the documentation:
"
  The transmit and receive functions should not be invoked
  when the device is stopped.
"

There is also this comment on rte_eth_dev_reset:
"
  Note: To avoid unexpected behavior, the application should stop calling
  Tx and Rx functions before calling rte_eth_dev_reset( ). For thread
  safety, all these controlling functions should be called from the same
  thread.
"

For queue stop, there is no documented expectation.

There is this comment for queue callback removal:
"
  The memory for the callback can be
  subsequently freed back by the application by calling rte_free():
 
  - Immediately - if the port is stopped, or the user knows that no
    callbacks are in flight e.g. if called from the thread doing Rx/Tx
    on that queue.
 
  - After a short delay - where the delay is sufficient to allow any
    in-flight callbacks to complete. Alternately, the RCU mechanism can be
    used to detect when data plane threads have ceased referencing the
    callback memory.
"

> > This patch was created with the assumption
> > that stopped queues may not be used for Rx/Tx.
> > No-op behavior of rte_eth_rx/tx_burst()
> > for a stopped queue is not documented.

Indeed, it is not documented.
I suggest working on this documentation first.
testpmd could be adjusted later if needed.

> > Yes, at least some PMDs implement it this way.
> > But is this behavior intended?
> > 
> > It is the application that stops the queue or starts it deferred.
> > Stopping is non-atomic, so polling the queue is not allowed during it.
> > Hence, if the application intends to stop queues, it must either:
> > 
> > a) Know the queue is not polled before stopping it.
> >    Use case: a secondary that was polling the queue has crashed,
> >    the primary is doing a recovery to free all mbufs.
> >    There is no issue since there is no poller to touch the queue.
> > 
> > b) Tell the poller to skip the queue for the time of stopping it.
> >    Use case: deferred queue start or queue reconfiguration.
> >    But if the poller is cooperating anyway,
> >    what prevents it from not touching the queue for longer?
> > 
> > The same considerations apply to starting a queue.
> > 
> > No-op behavior is convenient from the application point of view.
> > But it also means that pollers of stopped queues
> > will go all the way down to PMD Rx/Tx routines, wasting cycles,
> > and some PMDs will do a check for the queue state,
> > even though it may never be needed for a particular application.

Yes that's the question: Do we want
1/ to allow apps to poll stopped queues, adding checks in PMDs,
or do we prefer
2/ saving check cycles and expect apps to not poll?