DPDK patches and discussions
 help / color / mirror / Atom feed
* [dpdk-dev] [PATCH] doc: flow rule removal on port stop
@ 2020-09-16 11:18 Gregory Etelson
  2020-11-17 19:18 ` Gregory Etelson
  2020-11-18 16:15 ` [dpdk-dev] [PATCH v2] " Gregory Etelson
  0 siblings, 2 replies; 11+ messages in thread
From: Gregory Etelson @ 2020-09-16 11:18 UTC (permalink / raw)
  To: dev
  Cc: getelson, matan, rasland, Ori Kam, Ori Kam, John McNamara,
	Marko Kovacevic

There is a discrepancy between RTE ETHDEV API and flow rules guide
regarding flow rules maintenance after port stop.  RTE ETHDEV API in
librte_ethdev.h declares that flow rules will not be stored in PMD
after port stop:
 >>>>> Quite start
 Please note that some configuration is not stored between calls to
 rte_eth_dev_stop()/rte_eth_dev_start(). The following configuration
 will be retained:

 - MTU
 - flow control settings
 - receive mode configuration (promiscuous mode, all-multicast mode,
   hardware checksum mode, RSS/VMDQ settings etc.)
 - VLAN filtering configuration
 - default MAC address
 - MAC addresses supplied to MAC address array
 - flow director filtering mode (but not filtering rules)
 - NIC queue statistics mappings
 <<<< Quote end

PMD cannot always correctly restore flow rules after port stop / port
start because application may alter port configuration after port stop
without PMD knowledge about undergoing changes.  Consider the
following scenario:
application configures 2 queues 0 and 1 and creates a flow rule with
'queue index 1' action. After that application stops the port and
removes queue 1.
Although PMD can implement flow rule shadow copy to be used for
restore after port start, attempt to restore flow rule from shadow
will fail in example above and PMD could not notify application about
that failure.  As the result, flow rules map in HW will differ from
what application expects.  In addition, flow rules shadow copy used
for port start restore consumes considerable amount of system memory,
especially in systems with millions of flow rules.

Signed-off-by: Gregory Etelson <getelson@nvidia.com>
Acked-by: Ori Kam <orika@nvidia.com>
---
 doc/guides/prog_guide/rte_flow.rst | 5 ++---
 1 file changed, 2 insertions(+), 3 deletions(-)

diff --git a/doc/guides/prog_guide/rte_flow.rst b/doc/guides/prog_guide/rte_flow.rst
index 944e8242d6..dfe5a40f8e 100644
--- a/doc/guides/prog_guide/rte_flow.rst
+++ b/doc/guides/prog_guide/rte_flow.rst
@@ -3055,10 +3055,9 @@ Caveats
   temporarily replacing the burst function pointers), an appropriate error
   code must be returned (``EBUSY``).
 
-- PMDs, not applications, are responsible for maintaining flow rules
+- Applications, not PMDs, are responsible for maintaining flow rules
   configuration when stopping and restarting a port or performing other
-  actions which may affect them. They can only be destroyed explicitly by
-  applications.
+  actions which may affect them.
 
 For devices exposing multiple ports sharing global settings affected by flow
 rules:
-- 
2.25.1


^ permalink raw reply	[flat|nested] 11+ messages in thread

* [dpdk-dev] [PATCH] doc: flow rule removal on port stop
  2020-09-16 11:18 [dpdk-dev] [PATCH] doc: flow rule removal on port stop Gregory Etelson
@ 2020-11-17 19:18 ` Gregory Etelson
  2020-11-17 19:56   ` Andrew Rybchenko
  2020-11-18 16:15 ` [dpdk-dev] [PATCH v2] " Gregory Etelson
  1 sibling, 1 reply; 11+ messages in thread
From: Gregory Etelson @ 2020-11-17 19:18 UTC (permalink / raw)
  To: getelson; +Cc: dev, john.mcnamara, marko.kovacevic, matan, orika, thomas

There is a discrepancy between RTE ETHDEV API and flow rules guide
regarding flow rules maintenance after port stop.  RTE ETHDEV API in
librte_ethdev.h declares that flow rules will not be stored in PMD
after port stop:
 >>>>> Quite start
 Please note that some configuration is not stored between calls to
 rte_eth_dev_stop()/rte_eth_dev_start(). The following configuration
 will be retained:

 - MTU
 - flow control settings
 - receive mode configuration (promiscuous mode, all-multicast mode,
   hardware checksum mode, RSS/VMDQ settings etc.)
 - VLAN filtering configuration
 - default MAC address
 - MAC addresses supplied to MAC address array
 - flow director filtering mode (but not filtering rules)
 - NIC queue statistics mappings
 <<<< Quote end

PMD cannot always correctly restore flow rules after port stop / port
start because application may alter port configuration after port stop
without PMD knowledge about undergoing changes.  Consider the
following scenario:
application configures 2 queues 0 and 1 and creates a flow rule with
'queue index 1' action. After that application stops the port and
removes queue 1.
Although PMD can implement flow rule shadow copy to be used for
restore after port start, attempt to restore flow rule from shadow
will fail in example above and PMD could not notify application about
that failure.  As the result, flow rules map in HW will differ from
what application expects.  In addition, flow rules shadow copy used
for port start restore consumes considerable amount of system memory,
especially in systems with millions of flow rules.

Signed-off-by: Gregory Etelson <getelson@nvidia.com>
Acked-by: Ori Kam <orika@nvidia.com>
---
 doc/guides/prog_guide/rte_flow.rst | 5 ++---
 1 file changed, 2 insertions(+), 3 deletions(-)

diff --git a/doc/guides/prog_guide/rte_flow.rst b/doc/guides/prog_guide/rte_flow.rst
index 944e8242d6..dfe5a40f8e 100644
--- a/doc/guides/prog_guide/rte_flow.rst
+++ b/doc/guides/prog_guide/rte_flow.rst
@@ -3055,10 +3055,9 @@ Caveats
   temporarily replacing the burst function pointers), an appropriate error
   code must be returned (``EBUSY``).
 
-- PMDs, not applications, are responsible for maintaining flow rules
+- Applications, not PMDs, are responsible for maintaining flow rules
   configuration when stopping and restarting a port or performing other
-  actions which may affect them. They can only be destroyed explicitly by
-  applications.
+  actions which may affect them.
 
 For devices exposing multiple ports sharing global settings affected by flow
 rules:
-- 
2.25.1


^ permalink raw reply	[flat|nested] 11+ messages in thread

* Re: [dpdk-dev] [PATCH] doc: flow rule removal on port stop
  2020-11-17 19:18 ` Gregory Etelson
@ 2020-11-17 19:56   ` Andrew Rybchenko
  2020-11-18  8:59     ` Gregory Etelson
  0 siblings, 1 reply; 11+ messages in thread
From: Andrew Rybchenko @ 2020-11-17 19:56 UTC (permalink / raw)
  To: Gregory Etelson; +Cc: dev, john.mcnamara, marko.kovacevic, matan, orika, thomas

On 11/17/20 10:18 PM, Gregory Etelson wrote:
> There is a discrepancy between RTE ETHDEV API and flow rules guide
> regarding flow rules maintenance after port stop.  RTE ETHDEV API in
> librte_ethdev.h declares that flow rules will not be stored in PMD
> after port stop:
>   >>>>> Quite start
>   Please note that some configuration is not stored between calls to
>   rte_eth_dev_stop()/rte_eth_dev_start(). The following configuration
>   will be retained:
> 
>   - MTU
>   - flow control settings
>   - receive mode configuration (promiscuous mode, all-multicast mode,
>     hardware checksum mode, RSS/VMDQ settings etc.)
>   - VLAN filtering configuration
>   - default MAC address
>   - MAC addresses supplied to MAC address array
>   - flow director filtering mode (but not filtering rules)
>   - NIC queue statistics mappings
>   <<<< Quote end
> 
> PMD cannot always correctly restore flow rules after port stop / port
> start because application may alter port configuration after port stop
> without PMD knowledge about undergoing changes.  Consider the
> following scenario:
> application configures 2 queues 0 and 1 and creates a flow rule with
> 'queue index 1' action. After that application stops the port and
> removes queue 1.
> Although PMD can implement flow rule shadow copy to be used for
> restore after port start, attempt to restore flow rule from shadow
> will fail in example above and PMD could not notify application about
> that failure.  As the result, flow rules map in HW will differ from
> what application expects.  In addition, flow rules shadow copy used
> for port start restore consumes considerable amount of system memory,
> especially in systems with millions of flow rules.
> 
> Signed-off-by: Gregory Etelson <getelson@nvidia.com>
> Acked-by: Ori Kam <orika@nvidia.com>
> ---
>   doc/guides/prog_guide/rte_flow.rst | 5 ++---
>   1 file changed, 2 insertions(+), 3 deletions(-)
> 
> diff --git a/doc/guides/prog_guide/rte_flow.rst b/doc/guides/prog_guide/rte_flow.rst
> index 944e8242d6..dfe5a40f8e 100644
> --- a/doc/guides/prog_guide/rte_flow.rst
> +++ b/doc/guides/prog_guide/rte_flow.rst
> @@ -3055,10 +3055,9 @@ Caveats
>     temporarily replacing the burst function pointers), an appropriate error
>     code must be returned (``EBUSY``).
>   
> -- PMDs, not applications, are responsible for maintaining flow rules
> +- Applications, not PMDs, are responsible for maintaining flow rules
>     configuration when stopping and restarting a port or performing other
> -  actions which may affect them. They can only be destroyed explicitly by
> -  applications.
> +  actions which may affect them.
>   
>   For devices exposing multiple ports sharing global settings affected by flow
>   rules:
> 

Re-reading it, it still looks vague. What happens on:
  - port stop without removal of flow rule before
  - port close without removal of flow rules before
  - port reset (which could be stop/start, e.g. to recover from error 
condition)

^ permalink raw reply	[flat|nested] 11+ messages in thread

* Re: [dpdk-dev] [PATCH] doc: flow rule removal on port stop
  2020-11-17 19:56   ` Andrew Rybchenko
@ 2020-11-18  8:59     ` Gregory Etelson
  2020-11-18  9:04       ` Andrew Rybchenko
  0 siblings, 1 reply; 11+ messages in thread
From: Gregory Etelson @ 2020-11-18  8:59 UTC (permalink / raw)
  To: Andrew Rybchenko
  Cc: dev, john.mcnamara, marko.kovacevic, Matan Azrad, Ori Kam,
	NBU-Contact-Thomas Monjalon

Hello Andrew,

> On 11/17/20 10:18 PM, Gregory Etelson wrote:
> > There is a discrepancy between RTE ETHDEV API and flow rules guide
> > regarding flow rules maintenance after port stop.  RTE ETHDEV API in
> > librte_ethdev.h declares that flow rules will not be stored in PMD
> > after port stop:
> >   >>>>> Quite start
> >   Please note that some configuration is not stored between calls to
> >   rte_eth_dev_stop()/rte_eth_dev_start(). The following configuration
> >   will be retained:
> >
> >   - MTU
> >   - flow control settings
> >   - receive mode configuration (promiscuous mode, all-multicast mode,
> >     hardware checksum mode, RSS/VMDQ settings etc.)
> >   - VLAN filtering configuration
> >   - default MAC address
> >   - MAC addresses supplied to MAC address array
> >   - flow director filtering mode (but not filtering rules)
> >   - NIC queue statistics mappings
> >   <<<< Quote end
> >
> > PMD cannot always correctly restore flow rules after port stop / port
> > start because application may alter port configuration after port stop
> > without PMD knowledge about undergoing changes.  Consider the
> > following scenario:
> > application configures 2 queues 0 and 1 and creates a flow rule with
> > 'queue index 1' action. After that application stops the port and
> > removes queue 1.
> > Although PMD can implement flow rule shadow copy to be used for
> > restore after port start, attempt to restore flow rule from shadow
> > will fail in example above and PMD could not notify application about
> > that failure.  As the result, flow rules map in HW will differ from
> > what application expects.  In addition, flow rules shadow copy used
> > for port start restore consumes considerable amount of system memory,
> > especially in systems with millions of flow rules.
> >
> > Signed-off-by: Gregory Etelson <getelson@nvidia.com>
> > Acked-by: Ori Kam <orika@nvidia.com>
> > ---
> >   doc/guides/prog_guide/rte_flow.rst | 5 ++---
> >   1 file changed, 2 insertions(+), 3 deletions(-)
> >
> > diff --git a/doc/guides/prog_guide/rte_flow.rst
> > b/doc/guides/prog_guide/rte_flow.rst
> > index 944e8242d6..dfe5a40f8e 100644
> > --- a/doc/guides/prog_guide/rte_flow.rst
> > +++ b/doc/guides/prog_guide/rte_flow.rst
> > @@ -3055,10 +3055,9 @@ Caveats
> >     temporarily replacing the burst function pointers), an appropriate
> error
> >     code must be returned (``EBUSY``).
> >
> > -- PMDs, not applications, are responsible for maintaining flow rules
> > +- Applications, not PMDs, are responsible for maintaining flow rules
> >     configuration when stopping and restarting a port or performing
> > other
> > -  actions which may affect them. They can only be destroyed
> > explicitly by
> > -  applications.
> > +  actions which may affect them.
> >
> >   For devices exposing multiple ports sharing global settings affected
> by flow
> >   rules:
> >
> 
> Re-reading it, it still looks vague. What happens on:
>   - port stop without removal of flow rule before
>   - port close without removal of flow rules before
>   - port reset (which could be stop/start, e.g. to recover from error
> condition)

PMD should remove all flows related to hardware resource that was invalidated.

  

^ permalink raw reply	[flat|nested] 11+ messages in thread

* Re: [dpdk-dev] [PATCH] doc: flow rule removal on port stop
  2020-11-18  8:59     ` Gregory Etelson
@ 2020-11-18  9:04       ` Andrew Rybchenko
  2020-11-18  9:06         ` Gregory Etelson
  0 siblings, 1 reply; 11+ messages in thread
From: Andrew Rybchenko @ 2020-11-18  9:04 UTC (permalink / raw)
  To: Gregory Etelson
  Cc: dev, john.mcnamara, marko.kovacevic, Matan Azrad, Ori Kam,
	NBU-Contact-Thomas Monjalon

On 11/18/20 11:59 AM, Gregory Etelson wrote:
> Hello Andrew,
> 
>> On 11/17/20 10:18 PM, Gregory Etelson wrote:
>>> There is a discrepancy between RTE ETHDEV API and flow rules guide
>>> regarding flow rules maintenance after port stop.  RTE ETHDEV API in
>>> librte_ethdev.h declares that flow rules will not be stored in PMD
>>> after port stop:
>>>   >>>>> Quite start
>>>   Please note that some configuration is not stored between calls to
>>>   rte_eth_dev_stop()/rte_eth_dev_start(). The following configuration
>>>   will be retained:
>>>
>>>   - MTU
>>>   - flow control settings
>>>   - receive mode configuration (promiscuous mode, all-multicast mode,
>>>     hardware checksum mode, RSS/VMDQ settings etc.)
>>>   - VLAN filtering configuration
>>>   - default MAC address
>>>   - MAC addresses supplied to MAC address array
>>>   - flow director filtering mode (but not filtering rules)
>>>   - NIC queue statistics mappings
>>>   <<<< Quote end
>>>
>>> PMD cannot always correctly restore flow rules after port stop / port
>>> start because application may alter port configuration after port stop
>>> without PMD knowledge about undergoing changes.  Consider the
>>> following scenario:
>>> application configures 2 queues 0 and 1 and creates a flow rule with
>>> 'queue index 1' action. After that application stops the port and
>>> removes queue 1.
>>> Although PMD can implement flow rule shadow copy to be used for
>>> restore after port start, attempt to restore flow rule from shadow
>>> will fail in example above and PMD could not notify application about
>>> that failure.  As the result, flow rules map in HW will differ from
>>> what application expects.  In addition, flow rules shadow copy used
>>> for port start restore consumes considerable amount of system memory,
>>> especially in systems with millions of flow rules.
>>>
>>> Signed-off-by: Gregory Etelson <getelson@nvidia.com>
>>> Acked-by: Ori Kam <orika@nvidia.com>
>>> ---
>>>   doc/guides/prog_guide/rte_flow.rst | 5 ++---
>>>   1 file changed, 2 insertions(+), 3 deletions(-)
>>>
>>> diff --git a/doc/guides/prog_guide/rte_flow.rst
>>> b/doc/guides/prog_guide/rte_flow.rst
>>> index 944e8242d6..dfe5a40f8e 100644
>>> --- a/doc/guides/prog_guide/rte_flow.rst
>>> +++ b/doc/guides/prog_guide/rte_flow.rst
>>> @@ -3055,10 +3055,9 @@ Caveats
>>>     temporarily replacing the burst function pointers), an appropriate
>> error
>>>     code must be returned (``EBUSY``).
>>>
>>> -- PMDs, not applications, are responsible for maintaining flow rules
>>> +- Applications, not PMDs, are responsible for maintaining flow rules
>>>     configuration when stopping and restarting a port or performing
>>> other
>>> -  actions which may affect them. They can only be destroyed
>>> explicitly by
>>> -  applications.
>>> +  actions which may affect them.
>>>
>>>   For devices exposing multiple ports sharing global settings affected
>> by flow
>>>   rules:
>>>
>>
>> Re-reading it, it still looks vague. What happens on:
>>   - port stop without removal of flow rule before
>>   - port close without removal of flow rules before
>>   - port reset (which could be stop/start, e.g. to recover from error
>> condition)
> 
> PMD should remove all flows related to hardware resource that was invalidated.

Stop? Close? I agree and documentation should say so in a bit
clear way.

^ permalink raw reply	[flat|nested] 11+ messages in thread

* Re: [dpdk-dev] [PATCH] doc: flow rule removal on port stop
  2020-11-18  9:04       ` Andrew Rybchenko
@ 2020-11-18  9:06         ` Gregory Etelson
  0 siblings, 0 replies; 11+ messages in thread
From: Gregory Etelson @ 2020-11-18  9:06 UTC (permalink / raw)
  To: Andrew Rybchenko
  Cc: dev, john.mcnamara, marko.kovacevic, Matan Azrad, Ori Kam,
	NBU-Contact-Thomas Monjalon

> >> On 11/17/20 10:18 PM, Gregory Etelson wrote:
> >>> There is a discrepancy between RTE ETHDEV API and flow rules guide
> >>> regarding flow rules maintenance after port stop.  RTE ETHDEV API in
> >>> librte_ethdev.h declares that flow rules will not be stored in PMD
> >>> after port stop:
> >>>   >>>>> Quite start
> >>>   Please note that some configuration is not stored between calls to
> >>>   rte_eth_dev_stop()/rte_eth_dev_start(). The following configuration
> >>>   will be retained:
> >>>
> >>>   - MTU
> >>>   - flow control settings
> >>>   - receive mode configuration (promiscuous mode, all-multicast mode,
> >>>     hardware checksum mode, RSS/VMDQ settings etc.)
> >>>   - VLAN filtering configuration
> >>>   - default MAC address
> >>>   - MAC addresses supplied to MAC address array
> >>>   - flow director filtering mode (but not filtering rules)
> >>>   - NIC queue statistics mappings
> >>>   <<<< Quote end
> >>>
> >>> PMD cannot always correctly restore flow rules after port stop /
> >>> port start because application may alter port configuration after
> >>> port stop without PMD knowledge about undergoing changes.  Consider
> >>> the following scenario:
> >>> application configures 2 queues 0 and 1 and creates a flow rule with
> >>> 'queue index 1' action. After that application stops the port and
> >>> removes queue 1.
> >>> Although PMD can implement flow rule shadow copy to be used for
> >>> restore after port start, attempt to restore flow rule from shadow
> >>> will fail in example above and PMD could not notify application
> >>> about that failure.  As the result, flow rules map in HW will differ
> >>> from what application expects.  In addition, flow rules shadow copy
> >>> used for port start restore consumes considerable amount of system
> >>> memory, especially in systems with millions of flow rules.
> >>>
> >>> Signed-off-by: Gregory Etelson <getelson@nvidia.com>
> >>> Acked-by: Ori Kam <orika@nvidia.com>
> >>> ---
> >>>   doc/guides/prog_guide/rte_flow.rst | 5 ++---
> >>>   1 file changed, 2 insertions(+), 3 deletions(-)
> >>>
> >>> diff --git a/doc/guides/prog_guide/rte_flow.rst
> >>> b/doc/guides/prog_guide/rte_flow.rst
> >>> index 944e8242d6..dfe5a40f8e 100644
> >>> --- a/doc/guides/prog_guide/rte_flow.rst
> >>> +++ b/doc/guides/prog_guide/rte_flow.rst
> >>> @@ -3055,10 +3055,9 @@ Caveats
> >>>     temporarily replacing the burst function pointers), an
> >>> appropriate
> >> error
> >>>     code must be returned (``EBUSY``).
> >>>
> >>> -- PMDs, not applications, are responsible for maintaining flow
> >>> rules
> >>> +- Applications, not PMDs, are responsible for maintaining flow
> >>> +rules
> >>>     configuration when stopping and restarting a port or performing
> >>> other
> >>> -  actions which may affect them. They can only be destroyed
> >>> explicitly by
> >>> -  applications.
> >>> +  actions which may affect them.
> >>>
> >>>   For devices exposing multiple ports sharing global settings
> >>> affected
> >> by flow
> >>>   rules:
> >>>
> >>
> >> Re-reading it, it still looks vague. What happens on:
> >>   - port stop without removal of flow rule before
> >>   - port close without removal of flow rules before
> >>   - port reset (which could be stop/start, e.g. to recover from error
> >> condition)
> >
> > PMD should remove all flows related to hardware resource that was
> invalidated.
> 
> Stop? Close? I agree and documentation should say so in a bit clear way.

I'll post updated document patch.

^ permalink raw reply	[flat|nested] 11+ messages in thread

* [dpdk-dev] [PATCH v2] doc: flow rule removal on port stop
  2020-09-16 11:18 [dpdk-dev] [PATCH] doc: flow rule removal on port stop Gregory Etelson
  2020-11-17 19:18 ` Gregory Etelson
@ 2020-11-18 16:15 ` Gregory Etelson
  2020-11-22 16:55   ` Thomas Monjalon
  2020-11-24 14:41   ` Ajit Khaparde
  1 sibling, 2 replies; 11+ messages in thread
From: Gregory Etelson @ 2020-11-18 16:15 UTC (permalink / raw)
  To: getelson; +Cc: dev, matan, thomas, andrew.rybchenko, Ori Kam

There is a discrepancy between RTE ETHDEV API and flow rules guide
regarding flow rules maintenance after port stop.  RTE ETHDEV API in
librte_ethdev.h declares that flow rules will not be stored in PMD
after port stop:
 >>>>> Quite start
 Please note that some configuration is not stored between calls to
 rte_eth_dev_stop()/rte_eth_dev_start(). The following configuration
 will be retained:

 - MTU
 - flow control settings
 - receive mode configuration (promiscuous mode, all-multicast mode,
   hardware checksum mode, RSS/VMDQ settings etc.)
 - VLAN filtering configuration
 - default MAC address
 - MAC addresses supplied to MAC address array
 - flow director filtering mode (but not filtering rules)
 - NIC queue statistics mappings
 <<<< Quote end

PMD cannot always correctly restore flow rules after port stop / port
start because application may alter port configuration after port stop
without PMD knowledge about undergoing changes.  Consider the
following scenario:
application configures 2 queues 0 and 1 and creates a flow rule with
'queue index 1' action. After that application stops the port and
removes queue 1.
Although PMD can implement flow rule shadow copy to be used for
restore after port start, attempt to restore flow rule from shadow
will fail in example above and PMD could not notify application about
that failure.  As the result, flow rules map in HW will differ from
what application expects.  In addition, flow rules shadow copy used
for port start restore consumes considerable amount of system memory,
especially in systems with millions of flow rules.

Signed-off-by: Gregory Etelson <getelson@nvidia.com>
Acked-by: Ori Kam <orika@nvidia.com>
---
 doc/guides/prog_guide/rte_flow.rst | 10 ++++++----
 1 file changed, 6 insertions(+), 4 deletions(-)

diff --git a/doc/guides/prog_guide/rte_flow.rst b/doc/guides/prog_guide/rte_flow.rst
index ea203e0ca4..4cff9332fa 100644
--- a/doc/guides/prog_guide/rte_flow.rst
+++ b/doc/guides/prog_guide/rte_flow.rst
@@ -3229,10 +3229,12 @@ Caveats
   temporarily replacing the burst function pointers), an appropriate error
   code must be returned (``EBUSY``).
 
-- PMDs, not applications, are responsible for maintaining flow rules
-  configuration when stopping and restarting a port or performing other
-  actions which may affect them. They can only be destroyed explicitly by
-  applications.
+- Applications, not PMDs, are responsible for maintaining flow rules
+  configuration when closing, stopping or restarting a port or performing other
+  actions which may affect them.
+  Applications must assume that after port close, stop or restart all flows
+  related to that port are not valid, hardware rules are destroyed and relevant
+  PMD resources are released.
 
 For devices exposing multiple ports sharing global settings affected by flow
 rules:
-- 
2.29.2


^ permalink raw reply	[flat|nested] 11+ messages in thread

* Re: [dpdk-dev] [PATCH v2] doc: flow rule removal on port stop
  2020-11-18 16:15 ` [dpdk-dev] [PATCH v2] " Gregory Etelson
@ 2020-11-22 16:55   ` Thomas Monjalon
  2020-11-24 11:04     ` Thomas Monjalon
  2020-11-24 14:41   ` Ajit Khaparde
  1 sibling, 1 reply; 11+ messages in thread
From: Thomas Monjalon @ 2020-11-22 16:55 UTC (permalink / raw)
  To: andrew.rybchenko; +Cc: getelson, dev, matan, Ori Kam, ferruh.yigit

Andrew, any comment on this v2?
(disclosure: I did not read it)


18/11/2020 17:15, Gregory Etelson:
> There is a discrepancy between RTE ETHDEV API and flow rules guide
> regarding flow rules maintenance after port stop.  RTE ETHDEV API in
> librte_ethdev.h declares that flow rules will not be stored in PMD
> after port stop:
>  >>>>> Quite start
>  Please note that some configuration is not stored between calls to
>  rte_eth_dev_stop()/rte_eth_dev_start(). The following configuration
>  will be retained:
> 
>  - MTU
>  - flow control settings
>  - receive mode configuration (promiscuous mode, all-multicast mode,
>    hardware checksum mode, RSS/VMDQ settings etc.)
>  - VLAN filtering configuration
>  - default MAC address
>  - MAC addresses supplied to MAC address array
>  - flow director filtering mode (but not filtering rules)
>  - NIC queue statistics mappings
>  <<<< Quote end
> 
> PMD cannot always correctly restore flow rules after port stop / port
> start because application may alter port configuration after port stop
> without PMD knowledge about undergoing changes.  Consider the
> following scenario:
> application configures 2 queues 0 and 1 and creates a flow rule with
> 'queue index 1' action. After that application stops the port and
> removes queue 1.
> Although PMD can implement flow rule shadow copy to be used for
> restore after port start, attempt to restore flow rule from shadow
> will fail in example above and PMD could not notify application about
> that failure.  As the result, flow rules map in HW will differ from
> what application expects.  In addition, flow rules shadow copy used
> for port start restore consumes considerable amount of system memory,
> especially in systems with millions of flow rules.
> 
> Signed-off-by: Gregory Etelson <getelson@nvidia.com>
> Acked-by: Ori Kam <orika@nvidia.com>
> ---
>  doc/guides/prog_guide/rte_flow.rst | 10 ++++++----
>  1 file changed, 6 insertions(+), 4 deletions(-)
> 
> diff --git a/doc/guides/prog_guide/rte_flow.rst b/doc/guides/prog_guide/rte_flow.rst
> index ea203e0ca4..4cff9332fa 100644
> --- a/doc/guides/prog_guide/rte_flow.rst
> +++ b/doc/guides/prog_guide/rte_flow.rst
> @@ -3229,10 +3229,12 @@ Caveats
>    temporarily replacing the burst function pointers), an appropriate error
>    code must be returned (``EBUSY``).
>  
> -- PMDs, not applications, are responsible for maintaining flow rules
> -  configuration when stopping and restarting a port or performing other
> -  actions which may affect them. They can only be destroyed explicitly by
> -  applications.
> +- Applications, not PMDs, are responsible for maintaining flow rules
> +  configuration when closing, stopping or restarting a port or performing other
> +  actions which may affect them.
> +  Applications must assume that after port close, stop or restart all flows
> +  related to that port are not valid, hardware rules are destroyed and relevant
> +  PMD resources are released.
>  
>  For devices exposing multiple ports sharing global settings affected by flow
>  rules:
> 






^ permalink raw reply	[flat|nested] 11+ messages in thread

* Re: [dpdk-dev] [PATCH v2] doc: flow rule removal on port stop
  2020-11-22 16:55   ` Thomas Monjalon
@ 2020-11-24 11:04     ` Thomas Monjalon
  0 siblings, 0 replies; 11+ messages in thread
From: Thomas Monjalon @ 2020-11-24 11:04 UTC (permalink / raw)
  To: andrew.rybchenko, ferruh.yigit; +Cc: dev, getelson, dev, matan, Ori Kam

There is also a testpmd patch pending about this behaviour:
	https://patches.dpdk.org/patch/75353/

22/11/2020 17:55, Thomas Monjalon:
> Andrew, any comment on this v2?
> (disclosure: I did not read it)
> 
> 
> 18/11/2020 17:15, Gregory Etelson:
> > There is a discrepancy between RTE ETHDEV API and flow rules guide
> > regarding flow rules maintenance after port stop.  RTE ETHDEV API in
> > librte_ethdev.h declares that flow rules will not be stored in PMD
> > after port stop:
> >  >>>>> Quite start
> >  Please note that some configuration is not stored between calls to
> >  rte_eth_dev_stop()/rte_eth_dev_start(). The following configuration
> >  will be retained:
> > 
> >  - MTU
> >  - flow control settings
> >  - receive mode configuration (promiscuous mode, all-multicast mode,
> >    hardware checksum mode, RSS/VMDQ settings etc.)
> >  - VLAN filtering configuration
> >  - default MAC address
> >  - MAC addresses supplied to MAC address array
> >  - flow director filtering mode (but not filtering rules)
> >  - NIC queue statistics mappings
> >  <<<< Quote end
> > 
> > PMD cannot always correctly restore flow rules after port stop / port
> > start because application may alter port configuration after port stop
> > without PMD knowledge about undergoing changes.  Consider the
> > following scenario:
> > application configures 2 queues 0 and 1 and creates a flow rule with
> > 'queue index 1' action. After that application stops the port and
> > removes queue 1.
> > Although PMD can implement flow rule shadow copy to be used for
> > restore after port start, attempt to restore flow rule from shadow
> > will fail in example above and PMD could not notify application about
> > that failure.  As the result, flow rules map in HW will differ from
> > what application expects.  In addition, flow rules shadow copy used
> > for port start restore consumes considerable amount of system memory,
> > especially in systems with millions of flow rules.
> > 
> > Signed-off-by: Gregory Etelson <getelson@nvidia.com>
> > Acked-by: Ori Kam <orika@nvidia.com>
> > ---
> >  doc/guides/prog_guide/rte_flow.rst | 10 ++++++----
> >  1 file changed, 6 insertions(+), 4 deletions(-)
> > 
> > diff --git a/doc/guides/prog_guide/rte_flow.rst b/doc/guides/prog_guide/rte_flow.rst
> > index ea203e0ca4..4cff9332fa 100644
> > --- a/doc/guides/prog_guide/rte_flow.rst
> > +++ b/doc/guides/prog_guide/rte_flow.rst
> > @@ -3229,10 +3229,12 @@ Caveats
> >    temporarily replacing the burst function pointers), an appropriate error
> >    code must be returned (``EBUSY``).
> >  
> > -- PMDs, not applications, are responsible for maintaining flow rules
> > -  configuration when stopping and restarting a port or performing other
> > -  actions which may affect them. They can only be destroyed explicitly by
> > -  applications.
> > +- Applications, not PMDs, are responsible for maintaining flow rules
> > +  configuration when closing, stopping or restarting a port or performing other
> > +  actions which may affect them.
> > +  Applications must assume that after port close, stop or restart all flows
> > +  related to that port are not valid, hardware rules are destroyed and relevant
> > +  PMD resources are released.
> >  
> >  For devices exposing multiple ports sharing global settings affected by flow
> >  rules:





^ permalink raw reply	[flat|nested] 11+ messages in thread

* Re: [dpdk-dev] [PATCH v2] doc: flow rule removal on port stop
  2020-11-18 16:15 ` [dpdk-dev] [PATCH v2] " Gregory Etelson
  2020-11-22 16:55   ` Thomas Monjalon
@ 2020-11-24 14:41   ` Ajit Khaparde
  2020-11-25 23:33     ` Thomas Monjalon
  1 sibling, 1 reply; 11+ messages in thread
From: Ajit Khaparde @ 2020-11-24 14:41 UTC (permalink / raw)
  To: Gregory Etelson
  Cc: dpdk-dev, Matan Azrad, Thomas Monjalon, Andrew Rybchenko, Ori Kam

On Wed, Nov 18, 2020 at 8:15 AM Gregory Etelson <getelson@nvidia.com> wrote:
>
> There is a discrepancy between RTE ETHDEV API and flow rules guide
> regarding flow rules maintenance after port stop.  RTE ETHDEV API in
> librte_ethdev.h declares that flow rules will not be stored in PMD
> after port stop:
>  >>>>> Quite start
>  Please note that some configuration is not stored between calls to
>  rte_eth_dev_stop()/rte_eth_dev_start(). The following configuration
>  will be retained:
>
>  - MTU
>  - flow control settings
>  - receive mode configuration (promiscuous mode, all-multicast mode,
>    hardware checksum mode, RSS/VMDQ settings etc.)
>  - VLAN filtering configuration
>  - default MAC address
>  - MAC addresses supplied to MAC address array
>  - flow director filtering mode (but not filtering rules)
>  - NIC queue statistics mappings
>  <<<< Quote end
>
> PMD cannot always correctly restore flow rules after port stop / port
> start because application may alter port configuration after port stop
> without PMD knowledge about undergoing changes.  Consider the
> following scenario:
> application configures 2 queues 0 and 1 and creates a flow rule with
> 'queue index 1' action. After that application stops the port and
> removes queue 1.
> Although PMD can implement flow rule shadow copy to be used for
> restore after port start, attempt to restore flow rule from shadow
> will fail in example above and PMD could not notify application about
> that failure.  As the result, flow rules map in HW will differ from
> what application expects.  In addition, flow rules shadow copy used
> for port start restore consumes considerable amount of system memory,
> especially in systems with millions of flow rules.
>
> Signed-off-by: Gregory Etelson <getelson@nvidia.com>
> Acked-by: Ori Kam <orika@nvidia.com>
Acked-by: Ajit Khaparde <ajit.khaparde@broadcom.com>

> ---
>  doc/guides/prog_guide/rte_flow.rst | 10 ++++++----
>  1 file changed, 6 insertions(+), 4 deletions(-)
>
> diff --git a/doc/guides/prog_guide/rte_flow.rst b/doc/guides/prog_guide/rte_flow.rst
> index ea203e0ca4..4cff9332fa 100644
> --- a/doc/guides/prog_guide/rte_flow.rst
> +++ b/doc/guides/prog_guide/rte_flow.rst
> @@ -3229,10 +3229,12 @@ Caveats
>    temporarily replacing the burst function pointers), an appropriate error
>    code must be returned (``EBUSY``).
>
> -- PMDs, not applications, are responsible for maintaining flow rules
> -  configuration when stopping and restarting a port or performing other
> -  actions which may affect them. They can only be destroyed explicitly by
> -  applications.
> +- Applications, not PMDs, are responsible for maintaining flow rules
> +  configuration when closing, stopping or restarting a port or performing other
> +  actions which may affect them.
> +  Applications must assume that after port close, stop or restart all flows
> +  related to that port are not valid, hardware rules are destroyed and relevant
> +  PMD resources are released.
>
>  For devices exposing multiple ports sharing global settings affected by flow
>  rules:
> --
> 2.29.2
>

^ permalink raw reply	[flat|nested] 11+ messages in thread

* Re: [dpdk-dev] [PATCH v2] doc: flow rule removal on port stop
  2020-11-24 14:41   ` Ajit Khaparde
@ 2020-11-25 23:33     ` Thomas Monjalon
  0 siblings, 0 replies; 11+ messages in thread
From: Thomas Monjalon @ 2020-11-25 23:33 UTC (permalink / raw)
  To: Gregory Etelson
  Cc: dev, Matan Azrad, Andrew Rybchenko, Ori Kam, Ajit Khaparde,
	ferruh.yigit, kalesh-anakkur.purayil, asafp

24/11/2020 15:41, Ajit Khaparde:
> On Wed, Nov 18, 2020 at 8:15 AM Gregory Etelson <getelson@nvidia.com> wrote:
> >
> > There is a discrepancy between RTE ETHDEV API and flow rules guide
> > regarding flow rules maintenance after port stop.  RTE ETHDEV API in
> > librte_ethdev.h declares that flow rules will not be stored in PMD
> > after port stop:
> >  >>>>> Quite start
> >  Please note that some configuration is not stored between calls to
> >  rte_eth_dev_stop()/rte_eth_dev_start(). The following configuration
> >  will be retained:
> >
> >  - MTU
> >  - flow control settings
> >  - receive mode configuration (promiscuous mode, all-multicast mode,
> >    hardware checksum mode, RSS/VMDQ settings etc.)
> >  - VLAN filtering configuration
> >  - default MAC address
> >  - MAC addresses supplied to MAC address array
> >  - flow director filtering mode (but not filtering rules)
> >  - NIC queue statistics mappings
> >  <<<< Quote end
> >
> > PMD cannot always correctly restore flow rules after port stop / port
> > start because application may alter port configuration after port stop
> > without PMD knowledge about undergoing changes.  Consider the
> > following scenario:
> > application configures 2 queues 0 and 1 and creates a flow rule with
> > 'queue index 1' action. After that application stops the port and
> > removes queue 1.
> > Although PMD can implement flow rule shadow copy to be used for
> > restore after port start, attempt to restore flow rule from shadow
> > will fail in example above and PMD could not notify application about
> > that failure.  As the result, flow rules map in HW will differ from
> > what application expects.  In addition, flow rules shadow copy used
> > for port start restore consumes considerable amount of system memory,
> > especially in systems with millions of flow rules.
> >
> > Signed-off-by: Gregory Etelson <getelson@nvidia.com>
> > Acked-by: Ori Kam <orika@nvidia.com>
> Acked-by: Ajit Khaparde <ajit.khaparde@broadcom.com>
> 
> > ---
> > -- PMDs, not applications, are responsible for maintaining flow rules
> > -  configuration when stopping and restarting a port or performing other
> > -  actions which may affect them. They can only be destroyed explicitly by
> > -  applications.
> > +- Applications, not PMDs, are responsible for maintaining flow rules
> > +  configuration when closing, stopping or restarting a port or performing other
> > +  actions which may affect them.
> > +  Applications must assume that after port close, stop or restart all flows
> > +  related to that port are not valid, hardware rules are destroyed and relevant
> > +  PMD resources are released.

In short summary, this rte_flow doc change has 3 reasons:
	- consistency with ethdev API doc
	- avoid unsolvable automatic flow update after re-configuration
	- reduce memory consumption for flow rules

Andrew was asking, in previous version, how to manage reset for error recovery.
As it has been discussed in other threads, an error recovery should be notified
to the application.
We already have RTE_ETH_EVENT_INTR_RESET for VF in case of PF reset,
and a more general recovery notification mechanism is being discussed:
	https://patches.dpdk.org/patch/80094/
Then it will be possible to notify the application that the flow rules
must be restored (among other recovery measures).

For the case of port stop/close, doc update applied, thanks.



^ permalink raw reply	[flat|nested] 11+ messages in thread

end of thread, other threads:[~2020-11-25 23:34 UTC | newest]

Thread overview: 11+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2020-09-16 11:18 [dpdk-dev] [PATCH] doc: flow rule removal on port stop Gregory Etelson
2020-11-17 19:18 ` Gregory Etelson
2020-11-17 19:56   ` Andrew Rybchenko
2020-11-18  8:59     ` Gregory Etelson
2020-11-18  9:04       ` Andrew Rybchenko
2020-11-18  9:06         ` Gregory Etelson
2020-11-18 16:15 ` [dpdk-dev] [PATCH v2] " Gregory Etelson
2020-11-22 16:55   ` Thomas Monjalon
2020-11-24 11:04     ` Thomas Monjalon
2020-11-24 14:41   ` Ajit Khaparde
2020-11-25 23:33     ` Thomas Monjalon

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).