Re: [dpdk-dev] [PATCH 2/5] ethdev: add capability to keep shared objects on restart

[Dmitry Kozlyuk <dkozlyuk@oss.nvidia.com>]

> -----Original Message-----
> From: Ferruh Yigit <ferruh.yigit@intel.com>
> Sent: 15 октября 2021 г. 19:27
> To: Dmitry Kozlyuk <dkozlyuk@nvidia.com>; dev@dpdk.org; Andrew Rybchenko
> <andrew.rybchenko@oktetlabs.ru>; Ori Kam <orika@nvidia.com>; Raslan
> Darawsheh <rasland@nvidia.com>
> Cc: NBU-Contact-Thomas Monjalon <thomas@monjalon.net>; Qi Zhang
> <qi.z.zhang@intel.com>; jerinj@marvell.com; Maxime Coquelin
> <maxime.coquelin@redhat.com>
> Subject: Re: [PATCH 2/5] ethdev: add capability to keep shared objects on
> restart
> 
> External email: Use caution opening links or attachments
> 
> 
> On 10/15/2021 1:35 PM, Dmitry Kozlyuk wrote:
> >> -----Original Message-----
> >> From: Ferruh Yigit <ferruh.yigit@intel.com>
> >> [...]
> >>> Introducing UNKNOWN state seems wrong to me.
> >>> What should an application do when it is reported?
> >>> Now there's just no way to learn how the PMD behaves,
> >>> but if it provides a response, it can't be "I don't know what I do".
> >>>
> >>
> >> I agree 'unknown' state is not ideal, but my intentions is prevent
> >> drivers that not implemented this new feature report wrong capability.
> >>
> >> Without capability, application already doesn't know how underlying
> >> PMD behaves, so this is by default 'unknown' state.
> >> I suggest keeping that state until driver explicitly updates its state
> >> to the correct value.
> >
> > My concern is that when all the drivers are changed to report a proper
> > capability, UNKNOWN remains in the API meaning "there's a bug in DPDK".
> >
> 
> When all drivers are changed, of course we can remove the 'unknown' flag.
> 
> > Instead of UNKNOWN response we can declare that rte_flow_flush()
> > must be called unless the application wants to keep the rules
> > and has made sure it's possible, or the behavior is undefined.
> > (Can be viewed as "UNKNOWN by default", but is simpler.)
> > This way neither UNKNOWN state is needed,
> > nor the bit saying the flow rules are flushed.
> > Here is why, let's consider KEEP and FLUSH combinations:
> >
> > (1) FLUSH=0, KEEP=0 is equivalent to UNKNOWN, i.e. the application
> >                      must explicitly flush the rules itself
> >                      in order to get deterministic behavior.
> > (2) FLUSH=1, KEEP=0 means PMD flushes all rules on the device stop.
> > (3) FLUSH=0, KEEP=1 means PMD can keep at least some rules,
> >                      exact support must be checked with
> rte_flow_create()
> >                      when the device is stopped.
> > (4) FLUSH=1, KEEP=1 is forbidden.
> >
> 
> What is 'FLUSH' here? Are you proposing a new capability?
> 
> > If the application doesn't need the PMD to keep flow rules,
> > it can as well flush them always before the device stop
> > regardless of whether the driver does it automatically or not.
> > It's even simpler and probably as efficient. Testpmd does this.
> > If the application wants to take advantage of rule-keeping ability,
> > it just tests the KEEP bit. If it is unset that's the previous case,
> > application should call rte_flow_flush() before the device stop to be
> sure.
> > Otherwise, the application can test capability to keep flow rule kinds
> > it is interested in (see my reply to Andrew).
> >
> 
> Overall this is an optimization, application can workaround without this
> capability.
> 
> If driver doesn't set KEEP capability, it is not clear what does it
> mean, driver doesn't keep rules or driver is not updated yet.
> I suggest to update comment to clarify the meaning of the missing KEEP
> flag.
> 
> And unless we have two explicit status flags application can never be
> sure that driver doesn't keep rules after stop. I am don't know if
> application wants to know this.
> 
> Other concern is how PMD maintainers will know that there is something
> to update here, I am sure many driver maintainers won't even be aware of
> this, your patch doesn't even cc them. Your approach feels like you are
> thinking only single PMD and ignore rest.
> 
> My intention was to have a way to follow drivers that is not updated,
> by marking them with UNKNOWN flag. But this also doesn't work with new
> drivers, they may forget setting capability.
> 
> 
> What about following:
> 1) Clarify KEEP flag meaning:
> having KEEP: flow rules are kept after stop
> missing KEEP: unknown behavior
> 
> 2) Mark all PMDs with useless flag:
> dev_capa &= ~KEEP
> Maintainer can remove or update this later, and we can easily track it.

Item 1) is almost what I did in v2. The difference (or clarification) is that
if the bit is set, it doesn't mean that all rules are kept.
It allows the PMD to not support keeping some kinds of rules.
Please see the doc update about how the kind is defined
and how the application can test what is unsupported.

This complication is needed so that if a PMD cannot keep some exotic kind of rules,
it is not forced to remove the capability completely,
blocking optimizations even if the application doesn't use problematic rule kinds.
It makes the capability future-proof.

The second flag (FLUSH) would not be of much help.
Consider it is not set, but the PMD can keep some kinds of rules.
The application still needs to test all the kinds it needs.
But it needs to do the same if the KEEP bit is set.
Only if it is set the application can skip the tests and rte_flow_flush(),
but these optimizations are small compared to keeping the rules itself.

Item 2) needs not to be done, because the absence of the bit is *the* useless value:
it means the unspecified same behavior as it is presently.
It is worth noting that currently any application that relies on the PMD
to keep or flush the rules is non-portable, because PMD is allowed to do anything.
To get a reliable behavior application must explicitly clear the rules.

Regarding you concern about maintainers forgetting to update PMDs,
I think there are better task-tracking tools then constants in the code
(the authors of golang's context.TODO may disagree :)