[dpdk-dev] [PATCH 1/3] ethdev: remove deprecation notice for packet type set

[dpdk-dev] [PATCH 1/3] ethdev: remove deprecation notice for packet type set

[Andrew Rybchenko]

The functionality is added, but the deprecation notice is not
removed.

Fixes: 5d4813acda2c ("ethdev: add packet type range function")

Signed-off-by: Andrew Rybchenko <arybchenko@solarflare.com>
---
 doc/guides/rel_notes/deprecation.rst | 15 ---------------
 1 file changed, 15 deletions(-)

diff --git a/doc/guides/rel_notes/deprecation.rst b/doc/guides/rel_notes/deprecation.rst
index f545801f6d..65076acb13 100644
--- a/doc/guides/rel_notes/deprecation.rst
+++ b/doc/guides/rel_notes/deprecation.rst
@@ -68,21 +68,6 @@ Deprecation Notices
   In 19.11 PMDs will still update the fields even when the offloads are not
   enabled.
 
-* ethdev: New function ``rte_eth_dev_set_supported_ptypes`` will be added in
-  19.11.
-  This will allow application to request PMD to set specific ptypes defined
-  through ``rte_eth_dev_set_supported_ptypes`` in ``rte_mbuf::packet_type``.
-  If application doesn't want any ptype information it can call
-  ``rte_eth_dev_set_supported_ptypes(ethdev_id, RTE_PTYPE_UNKNOWN)`` and PMD
-  will set ``rte_mbuf::packet_type`` to ``0``.
-  If application doesn't call ``rte_eth_dev_set_supported_ptypes`` PMD can
-  return ``rte_mbuf::packet_type`` with ``rte_eth_dev_get_supported_ptypes``.
-  If application is interested only in L2/L3 layer, it can inform the PMD
-  to update ``rte_mbuf::packet_type`` with L2/L3 ptype by calling
-  ``rte_eth_dev_set_supported_ptypes(ethdev_id, RTE_PTYPE_L2_MASK | RTE_PTYPE_L3_MASK)``.
-  This scheme will allow PMDs to avoid lookup to internal ptype table on Rx and
-  thereby improve Rx performance if application wishes do so.
-
 * cryptodev: support for using IV with all sizes is added, J0 still can
   be used but only when IV length in following structs ``rte_crypto_auth_xform``,
   ``rte_crypto_aead_xform`` is set to zero. When IV length is greater or equal
-- 
2.17.1

[dpdk-dev] [PATCH 2/3] ethdev: remove deprecation notice about RSS hash offload

[Andrew Rybchenko]

The offload was added, but the deprecation notice was not removed.

Fixes: 5d308972954c ("ethdev: add mbuf RSS update as an offload")

Signed-off-by: Andrew Rybchenko <arybchenko@solarflare.com>
---
 doc/guides/rel_notes/deprecation.rst | 7 +++----
 1 file changed, 3 insertions(+), 4 deletions(-)

diff --git a/doc/guides/rel_notes/deprecation.rst b/doc/guides/rel_notes/deprecation.rst
index 65076acb13..2f7e659700 100644
--- a/doc/guides/rel_notes/deprecation.rst
+++ b/doc/guides/rel_notes/deprecation.rst
@@ -59,13 +59,12 @@ Deprecation Notices
   - ``rte_eth_dev_stop``
   - ``rte_eth_dev_close``
 
-* ethdev: New offload flags ``DEV_RX_OFFLOAD_RSS_HASH`` and
-  ``DEV_RX_OFFLOAD_FLOW_MARK`` will be added in 19.11.
+* ethdev: New offload flags ``DEV_RX_OFFLOAD_FLOW_MARK`` will be added in 19.11.
   This will allow application to enable or disable PMDs from updating
-  ``rte_mbuf::hash::rss`` and ``rte_mbuf::hash::fdir`` respectively.
+  ``rte_mbuf::hash::fdir``.
   This scheme will allow PMDs to avoid writes to ``rte_mbuf`` fields on Rx and
   thereby improve Rx performance if application wishes do so.
-  In 19.11 PMDs will still update the fields even when the offloads are not
+  In 19.11 PMDs will still update the field even when the offloads are not
   enabled.
 
 * cryptodev: support for using IV with all sizes is added, J0 still can
-- 
2.17.1

[dpdk-dev] [PATCH 3/3] ethdev: improve flow mark Rx offload deprecation notice

[Andrew Rybchenko]

The deprecation notice is required since it adds more requirements
when RTE flow mark and flag actions may be used and require
changes in applications.

Signed-off-by: Andrew Rybchenko <arybchenko@solarflare.com>
---
 doc/guides/rel_notes/deprecation.rst | 14 +++++++-------
 1 file changed, 7 insertions(+), 7 deletions(-)

diff --git a/doc/guides/rel_notes/deprecation.rst b/doc/guides/rel_notes/deprecation.rst
index 2f7e659700..7060a6a0d0 100644
--- a/doc/guides/rel_notes/deprecation.rst
+++ b/doc/guides/rel_notes/deprecation.rst
@@ -59,13 +59,13 @@ Deprecation Notices
   - ``rte_eth_dev_stop``
   - ``rte_eth_dev_close``
 
-* ethdev: New offload flags ``DEV_RX_OFFLOAD_FLOW_MARK`` will be added in 19.11.
-  This will allow application to enable or disable PMDs from updating
-  ``rte_mbuf::hash::fdir``.
-  This scheme will allow PMDs to avoid writes to ``rte_mbuf`` fields on Rx and
-  thereby improve Rx performance if application wishes do so.
-  In 19.11 PMDs will still update the field even when the offloads are not
-  enabled.
+* ethdev: New offload flag ``DEV_RX_OFFLOAD_FLOW_MARK`` will be added in 20.02.
+  This will provide application an information if ``RTE_FLOW_ACTION_TYPE_MARK``
+  or ``RTE_FLOW_ACTION_TYPE_FLAG`` is supported and, what is more important,
+  allow an application to let PMD know that it would like to use these
+  features.
+  PMD may use the information to choose optimal datapath implementation and
+  configure HW appropriately to optimize performance and/or resources usage.
 
 * cryptodev: support for using IV with all sizes is added, J0 still can
   be used but only when IV length in following structs ``rte_crypto_auth_xform``,
-- 
2.17.1

[dpdk-dev] [PATCH v2 1/3] ethdev: remove deprecation notice for packet type set

[Andrew Rybchenko]

The functionality is added, but the deprecation notice is not
removed.

Fixes: 5d4813acda2c ("ethdev: add packet type range function")

Signed-off-by: Andrew Rybchenko <arybchenko@solarflare.com>
---
v2:
 - none

 doc/guides/rel_notes/deprecation.rst | 15 ---------------
 1 file changed, 15 deletions(-)

diff --git a/doc/guides/rel_notes/deprecation.rst b/doc/guides/rel_notes/deprecation.rst
index f545801f6d..65076acb13 100644
--- a/doc/guides/rel_notes/deprecation.rst
+++ b/doc/guides/rel_notes/deprecation.rst
@@ -68,21 +68,6 @@ Deprecation Notices
   In 19.11 PMDs will still update the fields even when the offloads are not
   enabled.
 
-* ethdev: New function ``rte_eth_dev_set_supported_ptypes`` will be added in
-  19.11.
-  This will allow application to request PMD to set specific ptypes defined
-  through ``rte_eth_dev_set_supported_ptypes`` in ``rte_mbuf::packet_type``.
-  If application doesn't want any ptype information it can call
-  ``rte_eth_dev_set_supported_ptypes(ethdev_id, RTE_PTYPE_UNKNOWN)`` and PMD
-  will set ``rte_mbuf::packet_type`` to ``0``.
-  If application doesn't call ``rte_eth_dev_set_supported_ptypes`` PMD can
-  return ``rte_mbuf::packet_type`` with ``rte_eth_dev_get_supported_ptypes``.
-  If application is interested only in L2/L3 layer, it can inform the PMD
-  to update ``rte_mbuf::packet_type`` with L2/L3 ptype by calling
-  ``rte_eth_dev_set_supported_ptypes(ethdev_id, RTE_PTYPE_L2_MASK | RTE_PTYPE_L3_MASK)``.
-  This scheme will allow PMDs to avoid lookup to internal ptype table on Rx and
-  thereby improve Rx performance if application wishes do so.
-
 * cryptodev: support for using IV with all sizes is added, J0 still can
   be used but only when IV length in following structs ``rte_crypto_auth_xform``,
   ``rte_crypto_aead_xform`` is set to zero. When IV length is greater or equal
-- 
2.17.1

[dpdk-dev] [PATCH v2 2/3] ethdev: remove deprecation notice about RSS hash offload

[Andrew Rybchenko]

The offload was added, but the deprecation notice was not removed.

Fixes: 5d308972954c ("ethdev: add mbuf RSS update as an offload")

Signed-off-by: Andrew Rybchenko <arybchenko@solarflare.com>
---
v2:
 - 'offloads are' -> 'offload is' in the last sentence

 doc/guides/rel_notes/deprecation.rst | 7 +++----
 1 file changed, 3 insertions(+), 4 deletions(-)

diff --git a/doc/guides/rel_notes/deprecation.rst b/doc/guides/rel_notes/deprecation.rst
index 65076acb13..afa94b43e2 100644
--- a/doc/guides/rel_notes/deprecation.rst
+++ b/doc/guides/rel_notes/deprecation.rst
@@ -59,13 +59,12 @@ Deprecation Notices
   - ``rte_eth_dev_stop``
   - ``rte_eth_dev_close``
 
-* ethdev: New offload flags ``DEV_RX_OFFLOAD_RSS_HASH`` and
-  ``DEV_RX_OFFLOAD_FLOW_MARK`` will be added in 19.11.
+* ethdev: New offload flags ``DEV_RX_OFFLOAD_FLOW_MARK`` will be added in 19.11.
   This will allow application to enable or disable PMDs from updating
-  ``rte_mbuf::hash::rss`` and ``rte_mbuf::hash::fdir`` respectively.
+  ``rte_mbuf::hash::fdir``.
   This scheme will allow PMDs to avoid writes to ``rte_mbuf`` fields on Rx and
   thereby improve Rx performance if application wishes do so.
-  In 19.11 PMDs will still update the fields even when the offloads are not
+  In 19.11 PMDs will still update the field even when the offload is not
   enabled.
 
 * cryptodev: support for using IV with all sizes is added, J0 still can
-- 
2.17.1

[dpdk-dev] [PATCH v2 3/3] ethdev: improve flow mark Rx offload deprecation notice

[Andrew Rybchenko]

The deprecation notice is required since it adds more requirements
when RTE flow mark and flag actions may be used and require
changes in applications.

Signed-off-by: Andrew Rybchenko <arybchenko@solarflare.com>
---
v2:
 - rebase after changes in the previous patch

 doc/guides/rel_notes/deprecation.rst | 14 +++++++-------
 1 file changed, 7 insertions(+), 7 deletions(-)

diff --git a/doc/guides/rel_notes/deprecation.rst b/doc/guides/rel_notes/deprecation.rst
index afa94b43e2..7060a6a0d0 100644
--- a/doc/guides/rel_notes/deprecation.rst
+++ b/doc/guides/rel_notes/deprecation.rst
@@ -59,13 +59,13 @@ Deprecation Notices
   - ``rte_eth_dev_stop``
   - ``rte_eth_dev_close``
 
-* ethdev: New offload flags ``DEV_RX_OFFLOAD_FLOW_MARK`` will be added in 19.11.
-  This will allow application to enable or disable PMDs from updating
-  ``rte_mbuf::hash::fdir``.
-  This scheme will allow PMDs to avoid writes to ``rte_mbuf`` fields on Rx and
-  thereby improve Rx performance if application wishes do so.
-  In 19.11 PMDs will still update the field even when the offload is not
-  enabled.
+* ethdev: New offload flag ``DEV_RX_OFFLOAD_FLOW_MARK`` will be added in 20.02.
+  This will provide application an information if ``RTE_FLOW_ACTION_TYPE_MARK``
+  or ``RTE_FLOW_ACTION_TYPE_FLAG`` is supported and, what is more important,
+  allow an application to let PMD know that it would like to use these
+  features.
+  PMD may use the information to choose optimal datapath implementation and
+  configure HW appropriately to optimize performance and/or resources usage.
 
 * cryptodev: support for using IV with all sizes is added, J0 still can
   be used but only when IV length in following structs ``rte_crypto_auth_xform``,
-- 
2.17.1

Re: [dpdk-dev] [PATCH v2 1/3] ethdev: remove deprecation notice for packet type set

[Ferruh Yigit]

On 11/19/2019 12:12 PM, Andrew Rybchenko wrote:
> The functionality is added, but the deprecation notice is not
> removed.
> 
> Fixes: 5d4813acda2c ("ethdev: add packet type range function")
> 
> Signed-off-by: Andrew Rybchenko <arybchenko@solarflare.com>

Acked-by: Ferruh Yigit <ferruh.yigit@intel.com>

Re: [dpdk-dev] [PATCH v2 2/3] ethdev: remove deprecation notice about RSS hash offload

[Ferruh Yigit]

On 11/19/2019 12:12 PM, Andrew Rybchenko wrote:
> The offload was added, but the deprecation notice was not removed.
> 
> Fixes: 5d308972954c ("ethdev: add mbuf RSS update as an offload")
> 
> Signed-off-by: Andrew Rybchenko <arybchenko@solarflare.com>

Acked-by: Ferruh Yigit <ferruh.yigit@intel.com>

Re: [dpdk-dev] [PATCH v2 3/3] ethdev: improve flow mark Rx offload deprecation notice

[Thomas Monjalon]

19/11/2019 13:12, Andrew Rybchenko:
> The deprecation notice is required since it adds more requirements
> when RTE flow mark and flag actions may be used and require
> changes in applications.

I am still not sure what is the best solution here.
I continued to think about it in this thread:
	http://mails.dpdk.org/archives/dev/2019-November/151960.html

I think we cannot require any application change until 20.11
in order to keep API (and behaviour) compatibility.
If something would be implemented in 20.02,
it must be a new and optional API.
That's why I think no deprecation notice is required.

[...]
> +* ethdev: New offload flag ``DEV_RX_OFFLOAD_FLOW_MARK`` will be added in 20.02.
> +  This will provide application an information if ``RTE_FLOW_ACTION_TYPE_MARK``
> +  or ``RTE_FLOW_ACTION_TYPE_FLAG`` is supported and, what is more important,
> +  allow an application to let PMD know that it would like to use these
> +  features.
> +  PMD may use the information to choose optimal datapath implementation and
> +  configure HW appropriately to optimize performance and/or resources usage.

Re: [dpdk-dev] [PATCH v2 3/3] ethdev: improve flow mark Rx offload deprecation notice

[Andrew Rybchenko]

On 11/22/19 1:01 AM, Thomas Monjalon wrote:
> 19/11/2019 13:12, Andrew Rybchenko:
>> The deprecation notice is required since it adds more requirements
>> when RTE flow mark and flag actions may be used and require
>> changes in applications.
> I am still not sure what is the best solution here.
> I continued to think about it in this thread:
> 	http://mails.dpdk.org/archives/dev/2019-November/151960.html
>
> I think we cannot require any application change until 20.11
> in order to keep API (and behaviour) compatibility.

Expected, but still very disappointing.

The feature is implemented by Pavan (@ Marvell), supported by me,
used by Qi (@ Intel), looks better than alternatives from application
developer point of view [1] and finally postponed for 1 year without really
strong motivation. I disagree that it is tightly related to moving
mark/flag to
dynamic field/flag and absolutely blocked by it. Yes, I know that the are
concerns from the very beginning, but the problem is explained [2] and clear
and no full-featured alternative solution is suggested. Solution suggested
by Ori has many significant drawbacks as explained in [2] and highlighted
in further discussion.

[1] http://inbox.dpdk.org/dev/1573203631946.15959@kth.se/
[2]
http://inbox.dpdk.org/dev/f170105b-9c60-1b04-cb18-52e0951ddcdb@solarflare.com/

> If something would be implemented in 20.02,
> it must be a new and optional API.

Flow mark and flag may work without the offload with some drivers,
but some drivers require the offload to make it work. Flow API error
should contain message which says that the offload is disabled and
must be enabled.

> That's why I think no deprecation notice is required.
>
> [...]
>> +* ethdev: New offload flag ``DEV_RX_OFFLOAD_FLOW_MARK`` will be added in 20.02.
>> +  This will provide application an information if ``RTE_FLOW_ACTION_TYPE_MARK``
>> +  or ``RTE_FLOW_ACTION_TYPE_FLAG`` is supported and, what is more important,
>> +  allow an application to let PMD know that it would like to use these
>> +  features.
>> +  PMD may use the information to choose optimal datapath implementation and
>> +  configure HW appropriately to optimize performance and/or resources usage.

Re: [dpdk-dev] [PATCH v2 3/3] ethdev: improve flow mark Rx offload deprecation notice

[Thomas Monjalon]

22/11/2019 11:12, Andrew Rybchenko:
> On 11/22/19 1:01 AM, Thomas Monjalon wrote:
> > 19/11/2019 13:12, Andrew Rybchenko:
> >> The deprecation notice is required since it adds more requirements
> >> when RTE flow mark and flag actions may be used and require
> >> changes in applications.
> > I am still not sure what is the best solution here.
> > I continued to think about it in this thread:
> > 	http://mails.dpdk.org/archives/dev/2019-November/151960.html
> >
> > I think we cannot require any application change until 20.11
> > in order to keep API (and behaviour) compatibility.
> 
> Expected, but still very disappointing.
> 
> The feature is implemented by Pavan (@ Marvell), supported by me,
> used by Qi (@ Intel), looks better than alternatives from application
> developer point of view [1] and finally postponed for 1 year without really
> strong motivation.

I see different valuable point of views. This is enough motivation.

And no, it is not postponed by one year.
Next release can implement a new API.

> I disagree that it is tightly related to moving
> mark/flag to
> dynamic field/flag and absolutely blocked by it. Yes, I know that the are
> concerns from the very beginning, but the problem is explained [2] and clear
> and no full-featured alternative solution is suggested. Solution suggested
> by Ori has many significant drawbacks as explained in [2] and highlighted
> in further discussion.

I disagree with working only on mark action while there are a lot
of other configs which have to be implemented in drivers.

The reality is that some drivers decided to have some "optimizations"
disabling some features, and you want the application to opt-in
in order to allow your optimized paths.
Note that opt-in is different of really enabling an offload.
For some basic port-level features like RSS hash,
it is enabled with an offload flag before starting the port,
acting as an opt-in.
Some features have some dedicated API, which may be enabled after
starting the port, and no way to opt-in (or opt-out) before start.
A lot of features are using rte_flow API which is in this situation.
If we take the opt-in path, let's not do it only for the mark action,
but let's create a real API for it:
	rte_eth_dev_optin()
	rte_eth_dev_optinall()
	rte_eth_dev_optoutl()

I think the motivation is strong enough.

> [1] http://inbox.dpdk.org/dev/1573203631946.15959@kth.se/
> [2]
> http://inbox.dpdk.org/dev/f170105b-9c60-1b04-cb18-52e0951ddcdb@solarflare.com/
> 
> > If something would be implemented in 20.02,
> > it must be a new and optional API.
> 
> Flow mark and flag may work without the offload with some drivers,
> but some drivers require the offload to make it work. Flow API error
> should contain message which says that the offload is disabled and
> must be enabled.

Yes, the PMD should return an explicit error about a feature being disabled.
How does it impact ethdev API?

> > That's why I think no deprecation notice is required.
> >
> > [...]
> >> +* ethdev: New offload flag ``DEV_RX_OFFLOAD_FLOW_MARK`` will be added in 20.02.
> >> +  This will provide application an information if ``RTE_FLOW_ACTION_TYPE_MARK``
> >> +  or ``RTE_FLOW_ACTION_TYPE_FLAG`` is supported and, what is more important,
> >> +  allow an application to let PMD know that it would like to use these
> >> +  features.
> >> +  PMD may use the information to choose optimal datapath implementation and
> >> +  configure HW appropriately to optimize performance and/or resources usage.

Re: [dpdk-dev] [PATCH v2 3/3] ethdev: improve flow mark Rx offload deprecation notice

[Andrew Rybchenko]

On 11/22/19 2:15 PM, Thomas Monjalon wrote:
> 22/11/2019 11:12, Andrew Rybchenko:
>> On 11/22/19 1:01 AM, Thomas Monjalon wrote:
>>> 19/11/2019 13:12, Andrew Rybchenko:
>>>> The deprecation notice is required since it adds more requirements
>>>> when RTE flow mark and flag actions may be used and require
>>>> changes in applications.
>>> I am still not sure what is the best solution here.
>>> I continued to think about it in this thread:
>>> 	http://mails.dpdk.org/archives/dev/2019-November/151960.html
>>>
>>> I think we cannot require any application change until 20.11
>>> in order to keep API (and behaviour) compatibility.
>> Expected, but still very disappointing.
>>
>> The feature is implemented by Pavan (@ Marvell), supported by me,
>> used by Qi (@ Intel), looks better than alternatives from application
>> developer point of view [1] and finally postponed for 1 year without really
>> strong motivation.
> I see different valuable point of views. This is enough motivation.

It looks like I miss it in previous discussion, I would be thankful if
you give me links to read or hints how to find.

> And no, it is not postponed by one year.
> Next release can implement a new API.
>
>> I disagree that it is tightly related to moving
>> mark/flag to
>> dynamic field/flag and absolutely blocked by it. Yes, I know that the are
>> concerns from the very beginning, but the problem is explained [2] and clear
>> and no full-featured alternative solution is suggested. Solution suggested
>> by Ori has many significant drawbacks as explained in [2] and highlighted
>> in further discussion.
> I disagree with working only on mark action while there are a lot
> of other configs which have to be implemented in drivers.
>
> The reality is that some drivers decided to have some "optimizations"
> disabling some features, and you want the application to opt-in
> in order to allow your optimized paths.

Strictly speaking it is not about driver optimized paths only, but HW
configuration as well which can be done on start-up only (not dynamic) and
could be per-queue in fact.

> Note that opt-in is different of really enabling an offload.
> For some basic port-level features like RSS hash,
> it is enabled with an offload flag before starting the port,
> acting as an opt-in.

Could you highlight the difference between opt-in and offload.
What is the key difference which makes one solution better
than another? Why different mechanism is required?

> Some features have some dedicated API, which may be enabled after
> starting the port, and no way to opt-in (or opt-out) before start.

It sounds like you have examples in your mind. Please, share.

> A lot of features are using rte_flow API which is in this situation.
> If we take the opt-in path, let's not do it only for the mark action,
> but let's create a real API for it:
> 	rte_eth_dev_optin()
> 	rte_eth_dev_optinall()
> 	rte_eth_dev_optoutl()

Introducing new types of controls would make configuration more and
more complex. I think that many different types of control would
over-complicate it. May be it is unavoidable, but it should be clear
why the problem cannot be solved using existing types of controls
(e.g. offloads).

> I think the motivation is strong enough.
>
>> [1] http://inbox.dpdk.org/dev/1573203631946.15959@kth.se/
>> [2]
>> http://inbox.dpdk.org/dev/f170105b-9c60-1b04-cb18-52e0951ddcdb@solarflare.com/
>>
>>> If something would be implemented in 20.02,
>>> it must be a new and optional API.
>> Flow mark and flag may work without the offload with some drivers,
>> but some drivers require the offload to make it work. Flow API error
>> should contain message which says that the offload is disabled and
>> must be enabled.
> Yes, the PMD should return an explicit error about a feature being disabled.
> How does it impact ethdev API?

It is still the offload discussed in the deprecation notice.
The solution is far from ideal, since allows the difference in PMDs
behaviour and an application debugged on one PMD may not
work using another PMD (unfortunately it is true in any case, but
such definition makes it 100% legal).

>>> That's why I think no deprecation notice is required.
>>>
>>> [...]
>>>> +* ethdev: New offload flag ``DEV_RX_OFFLOAD_FLOW_MARK`` will be added in 20.02.
>>>> +  This will provide application an information if ``RTE_FLOW_ACTION_TYPE_MARK``
>>>> +  or ``RTE_FLOW_ACTION_TYPE_FLAG`` is supported and, what is more important,
>>>> +  allow an application to let PMD know that it would like to use these
>>>> +  features.
>>>> +  PMD may use the information to choose optimal datapath implementation and
>>>> +  configure HW appropriately to optimize performance and/or resources usage.

Re: [dpdk-dev] [PATCH v2 3/3] ethdev: improve flow mark Rx offload deprecation notice

[Jerin Jacob]

On Fri, Nov 22, 2019 at 8:54 PM Andrew Rybchenko
<arybchenko@solarflare.com> wrote:
>
> On 11/22/19 2:15 PM, Thomas Monjalon wrote:
> > 22/11/2019 11:12, Andrew Rybchenko:
> >> On 11/22/19 1:01 AM, Thomas Monjalon wrote:
> >>> 19/11/2019 13:12, Andrew Rybchenko:
> >>>> The deprecation notice is required since it adds more requirements
> >>>> when RTE flow mark and flag actions may be used and require
> >>>> changes in applications.
> >>> I am still not sure what is the best solution here.
> >>> I continued to think about it in this thread:
> >>>     http://mails.dpdk.org/archives/dev/2019-November/151960.html
> >>>
> >>> I think we cannot require any application change until 20.11
> >>> in order to keep API (and behaviour) compatibility.
> >> Expected, but still very disappointing.
> >>
> >> The feature is implemented by Pavan (@ Marvell), supported by me,
> >> used by Qi (@ Intel), looks better than alternatives from application
> >> developer point of view [1] and finally postponed for 1 year without really
> >> strong motivation.
> > I see different valuable point of views. This is enough motivation.
>
> It looks like I miss it in previous discussion, I would be thankful if
> you give me links to read or hints how to find.
>
> > And no, it is not postponed by one year.
> > Next release can implement a new API.
> >
> >> I disagree that it is tightly related to moving
> >> mark/flag to
> >> dynamic field/flag and absolutely blocked by it. Yes, I know that the are
> >> concerns from the very beginning, but the problem is explained [2] and clear
> >> and no full-featured alternative solution is suggested. Solution suggested
> >> by Ori has many significant drawbacks as explained in [2] and highlighted
> >> in further discussion.
> > I disagree with working only on mark action while there are a lot
> > of other configs which have to be implemented in drivers.
> >
> > The reality is that some drivers decided to have some "optimizations"
> > disabling some features, and you want the application to opt-in
> > in order to allow your optimized paths.
>
> Strictly speaking it is not about driver optimized paths only, but HW
> configuration as well which can be done on start-up only (not dynamic) and
> could be per-queue in fact.
>
> > Note that opt-in is different of really enabling an offload.
> > For some basic port-level features like RSS hash,
> > it is enabled with an offload flag before starting the port,
> > acting as an opt-in.
>
> Could you highlight the difference between opt-in and offload.
> What is the key difference which makes one solution better
> than another? Why different mechanism is required?
>
> > Some features have some dedicated API, which may be enabled after
> > starting the port, and no way to opt-in (or opt-out) before start.
>
> It sounds like you have examples in your mind. Please, share.
>
> > A lot of features are using rte_flow API which is in this situation.
> > If we take the opt-in path, let's not do it only for the mark action,
> > but let's create a real API for it:
> >       rte_eth_dev_optin()
> >       rte_eth_dev_optinall()
> >       rte_eth_dev_optoutl()
>
> Introducing new types of controls would make configuration more and
> more complex. I think that many different types of control would
> over-complicate it. May be it is unavoidable, but it should be clear

I agree with Andrew here.
Another thing to consider is the behavior of pre rte_eth_dev_opt*()
API after reconfigure.
Does application needs to call these API again after the reconfigure to bring
back the old state prior to reconfiguring?


> why the problem cannot be solved using existing types of controls
> (e.g. offloads).

Re: [dpdk-dev] [PATCH v2 3/3] ethdev: improve flow mark Rx offload deprecation notice

[Thomas Monjalon]

22/11/2019 12:53, Andrew Rybchenko:
> On 11/22/19 2:15 PM, Thomas Monjalon wrote:
> > 22/11/2019 11:12, Andrew Rybchenko:
> >> On 11/22/19 1:01 AM, Thomas Monjalon wrote:
> >>> 19/11/2019 13:12, Andrew Rybchenko:
> >>>> The deprecation notice is required since it adds more requirements
> >>>> when RTE flow mark and flag actions may be used and require
> >>>> changes in applications.
> >>> I am still not sure what is the best solution here.
> >>> I continued to think about it in this thread:
> >>> 	http://mails.dpdk.org/archives/dev/2019-November/151960.html
> >>>
> >>> I think we cannot require any application change until 20.11
> >>> in order to keep API (and behaviour) compatibility.
> >> Expected, but still very disappointing.
> >>
> >> The feature is implemented by Pavan (@ Marvell), supported by me,
> >> used by Qi (@ Intel), looks better than alternatives from application
> >> developer point of view [1] and finally postponed for 1 year without really
> >> strong motivation.
> > 
> > I see different valuable point of views. This is enough motivation.
> 
> It looks like I miss it in previous discussion, I would be thankful if
> you give me links to read or hints how to find.

http://mails.dpdk.org/archives/dev/2019-November/150793.html

> > And no, it is not postponed by one year.
> > Next release can implement a new API.
> >
> >> I disagree that it is tightly related to moving
> >> mark/flag to
> >> dynamic field/flag and absolutely blocked by it. Yes, I know that the are
> >> concerns from the very beginning, but the problem is explained [2] and clear
> >> and no full-featured alternative solution is suggested. Solution suggested
> >> by Ori has many significant drawbacks as explained in [2] and highlighted
> >> in further discussion.
> > 
> > I disagree with working only on mark action while there are a lot
> > of other configs which have to be implemented in drivers.
> >
> > The reality is that some drivers decided to have some "optimizations"
> > disabling some features, and you want the application to opt-in
> > in order to allow your optimized paths.
> 
> Strictly speaking it is not about driver optimized paths only, but HW
> configuration as well which can be done on start-up only (not dynamic) and
> could be per-queue in fact.

OK good point, we can optimize both driver and hardware configuration
before enabling a queue.
Note all these threads are long but one of the benefits
is to get the definition of the need, which was lacking.

> > Note that opt-in is different of really enabling an offload.
> > For some basic port-level features like RSS hash,
> > it is enabled with an offload flag before starting the port,
> > acting as an opt-in.
> 
> Could you highlight the difference between opt-in and offload.
> What is the key difference which makes one solution better
> than another? Why different mechanism is required?

Configuring a feature means providing all infos to make
the processing effective.
Opt-in a feature means asking for a processing to be available
when it will be configured later.
Configuration implies opt-in of course.

For now, we have only configuration APIs, no opt-in.

The need you want to address is to opt-in for a feature
before enabling a queue, and configure it later.

> > Some features have some dedicated API, which may be enabled after
> > starting the port, and no way to opt-in (or opt-out) before start.
> 
> It sounds like you have examples in your mind. Please, share.

All rte_flow examples are some examples of configuration API
which can be done after start, without a way to opt-in in advance.
Other examples of APIs not clearly forbidden to use after start:
	- rte_eth_dev_set_mtu()
	- rte_eth_dev_vlan_filter()
	- rte_eth_dev_rss_reta_update()
	- rte_eth_mirror_rule_set()
	- rte_eth_dev_udp_tunnel_port_add()
	- rte_eth_dev_l2_tunnel_offload_set
	- rte_eth_timesync_enable()

> > A lot of features are using rte_flow API which is in this situation.
> > If we take the opt-in path, let's not do it only for the mark action,
> > but let's create a real API for it:
> > 	rte_eth_dev_optin()
> > 	rte_eth_dev_optinall()
> > 	rte_eth_dev_optoutl()
> 
> Introducing new types of controls would make configuration more and
> more complex. I think that many different types of control would
> over-complicate it. May be it is unavoidable, but it should be clear
> why the problem cannot be solved using existing types of controls
> (e.g. offloads).

The offload control is used as an effective configuration for now.
The features which are configured with DEV_RX_OFFLOAD_*
do not need any other API to be used.
Extending DEV_RX_OFFLOAD_* bits for enabling features which
must be configured via other API anyway, is possible.
The real problem is that features in DEV_RX_OFFLOAD_* are supposed
to be disabled by default. If we add some opt-in features here,
we cannot enable them by default for API compatibility and do the
right thing by default.

Choosing DEV_RX_OFFLOAD_* bits or rte_eth_dev_opt* functions is a detail.
The real decision is to change the API for using all these features.
Can we keep all features available by default (opt-out)?

And more importantly, again, it should be done for all features at once,
not only for the rte_flow mark.

> > I think the motivation is strong enough.
> >
> >> [1] http://inbox.dpdk.org/dev/1573203631946.15959@kth.se/
> >> [2]
> >> http://inbox.dpdk.org/dev/f170105b-9c60-1b04-cb18-52e0951ddcdb@solarflare.com/
> >>
> >>> If something would be implemented in 20.02,
> >>> it must be a new and optional API.
> >> 
> >> Flow mark and flag may work without the offload with some drivers,
> >> but some drivers require the offload to make it work. Flow API error
> >> should contain message which says that the offload is disabled and
> >> must be enabled.
> > 
> > Yes, the PMD should return an explicit error about a feature being disabled.
> > How does it impact ethdev API?
> 
> It is still the offload discussed in the deprecation notice.
> The solution is far from ideal, since allows the difference in PMDs
> behaviour and an application debugged on one PMD may not
> work using another PMD (unfortunately it is true in any case, but
> such definition makes it 100% legal).

Do you mean PMDs have different capabilities and optimizations?
I think I don't get your point.

> >>> That's why I think no deprecation notice is required.
> >>>
> >>> [...]
> >>>> +* ethdev: New offload flag ``DEV_RX_OFFLOAD_FLOW_MARK`` will be added in 20.02.
> >>>> +  This will provide application an information if ``RTE_FLOW_ACTION_TYPE_MARK``
> >>>> +  or ``RTE_FLOW_ACTION_TYPE_FLAG`` is supported and, what is more important,
> >>>> +  allow an application to let PMD know that it would like to use these
> >>>> +  features.
> >>>> +  PMD may use the information to choose optimal datapath implementation and
> >>>> +  configure HW appropriately to optimize performance and/or resources usage.

Re: [dpdk-dev] [PATCH v2 3/3] ethdev: improve flow mark Rx offload deprecation notice

[Jerin Jacob]

On Sat, Nov 23, 2019 at 3:58 AM Thomas Monjalon <thomas@monjalon.net> wrote:
>
> 22/11/2019 12:53, Andrew Rybchenko:
> > On 11/22/19 2:15 PM, Thomas Monjalon wrote:
> > > 22/11/2019 11:12, Andrew Rybchenko:
> > >> On 11/22/19 1:01 AM, Thomas Monjalon wrote:
> > >>> 19/11/2019 13:12, Andrew Rybchenko:
> > >>>> The deprecation notice is required since it adds more requirements
> > >>>> when RTE flow mark and flag actions may be used and require
> > >>>> changes in applications.
> > >>> I am still not sure what is the best solution here.
> > >>> I continued to think about it in this thread:
> > >>>   http://mails.dpdk.org/archives/dev/2019-November/151960.html
> > >>>
> > >>> I think we cannot require any application change until 20.11
> > >>> in order to keep API (and behaviour) compatibility.
> > >> Expected, but still very disappointing.
> > >>
> > >> The feature is implemented by Pavan (@ Marvell), supported by me,
> > >> used by Qi (@ Intel), looks better than alternatives from application
> > >> developer point of view [1] and finally postponed for 1 year without really
> > >> strong motivation.
> > >
> > > I see different valuable point of views. This is enough motivation.
> >
> > It looks like I miss it in previous discussion, I would be thankful if
> > you give me links to read or hints how to find.
>
> http://mails.dpdk.org/archives/dev/2019-November/150793.html
>
> > Introducing new types of controls would make configuration more and
> > more complex. I think that many different types of control would
> > over-complicate it. May be it is unavoidable, but it should be clear
> > why the problem cannot be solved using existing types of controls
> > (e.g. offloads).
>
> The offload control is used as an effective configuration for now.
> The features which are configured with DEV_RX_OFFLOAD_*
> do not need any other API to be used.
> Extending DEV_RX_OFFLOAD_* bits for enabling features which
> must be configured via other API anyway, is possible.
> The real problem is that features in DEV_RX_OFFLOAD_* are supposed
> to be disabled by default. If we add some opt-in features here,
> we cannot enable them by default for API compatibility and do the
> right thing by default.
>
> Choosing DEV_RX_OFFLOAD_* bits or rte_eth_dev_opt* functions is a detail.
> The real decision is to change the API for using all these features.
> Can we keep all features available by default (opt-out)?

IMO, *rte_eth_dev_opt* has following problems

1) It is not multi-process friendly. If we are changing the Rx/Tx
function pointer, based on
the selected offload, then, using *rte_eth_dev_opt* scheme won't
really work(if the new API
called after the secondary process launch)

2) If we are taking rte_eth_dev_opt path then by default feature has
to be enabled to
not break the functional ABI. That scheme won't scale if as when we
keep adding the new features.
It is always easy for the application to define "what it wants" vs
"what it does not want"

3) Defining the device state after the reconfigure operation.

IMO, if any operation is connected to fastpath it is better to use
DEV_RX_OFFLOAD_ like
this feature where enable or disable PMDs from updating
``rte_mbuf::hash::fdir`` so that if possible
we can use different Rx function pointer if possible(Hence it can work
with the multi-process case case)

Re: [dpdk-dev] [PATCH v2 3/3] ethdev: improve flow mark Rx offload deprecation notice

[Thomas Monjalon]

23/11/2019 10:42, Jerin Jacob:
> On Sat, Nov 23, 2019 at 3:58 AM Thomas Monjalon <thomas@monjalon.net> wrote:
> > 22/11/2019 12:53, Andrew Rybchenko:
> > > On 11/22/19 2:15 PM, Thomas Monjalon wrote:
> > > > 22/11/2019 11:12, Andrew Rybchenko:
> > > >> On 11/22/19 1:01 AM, Thomas Monjalon wrote:
> > > >>> 19/11/2019 13:12, Andrew Rybchenko:
> > > >>>> The deprecation notice is required since it adds more requirements
> > > >>>> when RTE flow mark and flag actions may be used and require
> > > >>>> changes in applications.
> > > >>> I am still not sure what is the best solution here.
> > > >>> I continued to think about it in this thread:
> > > >>>   http://mails.dpdk.org/archives/dev/2019-November/151960.html
> > > >>>
> > > >>> I think we cannot require any application change until 20.11
> > > >>> in order to keep API (and behaviour) compatibility.
> > > >> Expected, but still very disappointing.
> > > >>
> > > >> The feature is implemented by Pavan (@ Marvell), supported by me,
> > > >> used by Qi (@ Intel), looks better than alternatives from application
> > > >> developer point of view [1] and finally postponed for 1 year without really
> > > >> strong motivation.
> > > >
> > > > I see different valuable point of views. This is enough motivation.
> > >
> > > It looks like I miss it in previous discussion, I would be thankful if
> > > you give me links to read or hints how to find.
> >
> > http://mails.dpdk.org/archives/dev/2019-November/150793.html
> >
> > > Introducing new types of controls would make configuration more and
> > > more complex. I think that many different types of control would
> > > over-complicate it. May be it is unavoidable, but it should be clear
> > > why the problem cannot be solved using existing types of controls
> > > (e.g. offloads).
> >
> > The offload control is used as an effective configuration for now.
> > The features which are configured with DEV_RX_OFFLOAD_*
> > do not need any other API to be used.
> > Extending DEV_RX_OFFLOAD_* bits for enabling features which
> > must be configured via other API anyway, is possible.
> > The real problem is that features in DEV_RX_OFFLOAD_* are supposed
> > to be disabled by default. If we add some opt-in features here,
> > we cannot enable them by default for API compatibility and do the
> > right thing by default.
> >
> > Choosing DEV_RX_OFFLOAD_* bits or rte_eth_dev_opt* functions is a detail.
> > The real decision is to change the API for using all these features.
> > Can we keep all features available by default (opt-out)?
> 
> IMO, *rte_eth_dev_opt* has following problems
> 
> 1) It is not multi-process friendly. If we are changing the Rx/Tx
> function pointer, based on
> the selected offload, then, using *rte_eth_dev_opt* scheme won't
> really work(if the new API
> called after the secondary process launch)

Yes it must be used before launching the secondary process.
It is the same as DEV_RX_OFFLOAD_* config.

> 2) If we are taking rte_eth_dev_opt path then by default feature has
> to be enabled to
> not break the functional ABI. That scheme won't scale if as when we
> keep adding the new features.
> It is always easy for the application to define "what it wants" vs
> "what it does not want"

Yes, opt-in may look more natural than opt-out.
But opt-in makes the default more complex, and changes the API.

> 3) Defining the device state after the reconfigure operation.
> 
> IMO, if any operation is connected to fastpath it is better to use
> DEV_RX_OFFLOAD_ like
> this feature where enable or disable PMDs from updating
> ``rte_mbuf::hash::fdir`` so that if possible
> we can use different Rx function pointer if possible(Hence it can work
> with the multi-process case case)

I reply to 2 and 3 together.

We decided that offloads must be disabled by default.
This is what we have in 19.11:
	- Some offloads are enabled before start with DEV_?X_OFFLOAD_*
	- Some offloads are enabled with functions at any time

For the second type of offloads, you want to know, before start,
whether it will be used or not.
If adding the second type of offloads (like rte_flow ones)
to DEV_?X_OFFLOAD_*, it means it must be enabled 2 times:
	- before start with offload bits
	- later with more precise functions

I would like to avoid changing the default behaviour,
which is to enable an offload only one time.
That's why I think this second category of offloads should
offer opt-out (global disabling), so it will continue
to work by default if they are configured.

I hope you understand the difference between the two categories.
For now, it looks I failed to explain it clearly enough.

Re: [dpdk-dev] [PATCH v2 3/3] ethdev: improve flow mark Rx offload deprecation notice

[Jerin Jacob]

On Sun, Nov 24, 2019 at 3:12 AM Thomas Monjalon <thomas@monjalon.net> wrote:
>
> 23/11/2019 10:42, Jerin Jacob:
> > On Sat, Nov 23, 2019 at 3:58 AM Thomas Monjalon <thomas@monjalon.net> wrote:
> > > 22/11/2019 12:53, Andrew Rybchenko:
> > > > On 11/22/19 2:15 PM, Thomas Monjalon wrote:
> > > > > 22/11/2019 11:12, Andrew Rybchenko:
> > > > >> On 11/22/19 1:01 AM, Thomas Monjalon wrote:
> > > > >>> 19/11/2019 13:12, Andrew Rybchenko:
> > > > >>>> The deprecation notice is required since it adds more requirements
> > > > >>>> when RTE flow mark and flag actions may be used and require
> > > > >>>> changes in applications.
> > > > >>> I am still not sure what is the best solution here.
> > > > >>> I continued to think about it in this thread:
> > > > >>>   http://mails.dpdk.org/archives/dev/2019-November/151960.html
> > > > >>>
> > > > >>> I think we cannot require any application change until 20.11
> > > > >>> in order to keep API (and behaviour) compatibility.
> > > > >> Expected, but still very disappointing.
> > > > >>
> > > > >> The feature is implemented by Pavan (@ Marvell), supported by me,
> > > > >> used by Qi (@ Intel), looks better than alternatives from application
> > > > >> developer point of view [1] and finally postponed for 1 year without really
> > > > >> strong motivation.
> > > > >
> > > > > I see different valuable point of views. This is enough motivation.
> > > >
> > > > It looks like I miss it in previous discussion, I would be thankful if
> > > > you give me links to read or hints how to find.
> > >
> > > http://mails.dpdk.org/archives/dev/2019-November/150793.html
> > >
> > > > Introducing new types of controls would make configuration more and
> > > > more complex. I think that many different types of control would
> > > > over-complicate it. May be it is unavoidable, but it should be clear
> > > > why the problem cannot be solved using existing types of controls
> > > > (e.g. offloads).
> > >
> > > The offload control is used as an effective configuration for now.
> > > The features which are configured with DEV_RX_OFFLOAD_*
> > > do not need any other API to be used.
> > > Extending DEV_RX_OFFLOAD_* bits for enabling features which
> > > must be configured via other API anyway, is possible.
> > > The real problem is that features in DEV_RX_OFFLOAD_* are supposed
> > > to be disabled by default. If we add some opt-in features here,
> > > we cannot enable them by default for API compatibility and do the
> > > right thing by default.
> > >
> > > Choosing DEV_RX_OFFLOAD_* bits or rte_eth_dev_opt* functions is a detail.
> > > The real decision is to change the API for using all these features.
> > > Can we keep all features available by default (opt-out)?
> >
> > IMO, *rte_eth_dev_opt* has following problems
> >
> > 1) It is not multi-process friendly. If we are changing the Rx/Tx
> > function pointer, based on
> > the selected offload, then, using *rte_eth_dev_opt* scheme won't
> > really work(if the new API
> > called after the secondary process launch)
>
> Yes it must be used before launching the secondary process.
> It is the same as DEV_RX_OFFLOAD_* config.

Yes. rte_eth_dev_opt_* has another dimension to enable and disable as API.
So, we need to document, opt-in -> start() -> opt-out case won't work
in multi process
case.

>
> > 2) If we are taking rte_eth_dev_opt path then by default feature has
> > to be enabled to
> > not break the functional ABI. That scheme won't scale if as when we
> > keep adding the new features.
> > It is always easy for the application to define "what it wants" vs
> > "what it does not want"
>
> Yes, opt-in may look more natural than opt-out.
> But opt-in makes the default more complex, and changes the API.
>
> > 3) Defining the device state after the reconfigure operation.
> >
> > IMO, if any operation is connected to fastpath it is better to use
> > DEV_RX_OFFLOAD_ like
> > this feature where enable or disable PMDs from updating
> > ``rte_mbuf::hash::fdir`` so that if possible
> > we can use different Rx function pointer if possible(Hence it can work
> > with the multi-process case case)
>
> I reply to 2 and 3 together.
>
> We decided that offloads must be disabled by default.
> This is what we have in 19.11:
>         - Some offloads are enabled before start with DEV_?X_OFFLOAD_*
>         - Some offloads are enabled with functions at any time
>
> For the second type of offloads, you want to know, before start,
> whether it will be used or not.
> If adding the second type of offloads (like rte_flow ones)
> to DEV_?X_OFFLOAD_*, it means it must be enabled 2 times:
>         - before start with offload bits
>         - later with more precise functions
>
> I would like to avoid changing the default behaviour,
> which is to enable an offload only one time.
> That's why I think this second category of offloads should
> offer opt-out (global disabling), so it will continue
> to work by default if they are configured.
>
> I hope you understand the difference between the two categories.

I understand the difference. The only point of "difference in opinion" is
the default behavior of the feature/offload. If it is in RX_OFFLOAD scheme then
by default it is disabled. opt_* scheme makes this new feature/offload
enabled default to avoid changing the default behavior.

It is good to avoid functional ABI change. But bad as,
1) New API starts bloating the ethdev API.
2) It is diffcult for application guys to figure out what are features need to
be disabled to performance as he/she does not know, for the given release,
the enabled features.

Item (1) is a trade-off between elegance vs ABI compatibility. No
strong opinion on this.

To fix the item (2), Can we get have an API in ethdev to get enabled
features so that
the application can probe and disable if required?

For example, rte_eth_dev_set_ptypes() comes in same category, By default,
ptype parsing is enabled. I think, we can have a general interface to
"probe" the by default enabled features
and disable it if required. Not scattered API for each feature.
The above scheme fixe my concerns.

Thoughts?

> For now, it looks I failed to explain it clearly enough.

No, the explanation is very clear.


>
>

Re: [dpdk-dev] [PATCH v2 3/3] ethdev: improve flow mark Rx offload deprecation notice

[Thomas Monjalon]

25/11/2019 11:44, Jerin Jacob:
> On Sun, Nov 24, 2019 at 3:12 AM Thomas Monjalon <thomas@monjalon.net> wrote:
> >
> > 23/11/2019 10:42, Jerin Jacob:
> > > On Sat, Nov 23, 2019 at 3:58 AM Thomas Monjalon <thomas@monjalon.net> wrote:
> > > > 22/11/2019 12:53, Andrew Rybchenko:
> > > > > On 11/22/19 2:15 PM, Thomas Monjalon wrote:
> > > > > > 22/11/2019 11:12, Andrew Rybchenko:
> > > > > >> On 11/22/19 1:01 AM, Thomas Monjalon wrote:
> > > > > >>> 19/11/2019 13:12, Andrew Rybchenko:
> > > > > >>>> The deprecation notice is required since it adds more requirements
> > > > > >>>> when RTE flow mark and flag actions may be used and require
> > > > > >>>> changes in applications.
> > > > > >>> I am still not sure what is the best solution here.
> > > > > >>> I continued to think about it in this thread:
> > > > > >>>   http://mails.dpdk.org/archives/dev/2019-November/151960.html
> > > > > >>>
> > > > > >>> I think we cannot require any application change until 20.11
> > > > > >>> in order to keep API (and behaviour) compatibility.
> > > > > >> Expected, but still very disappointing.
> > > > > >>
> > > > > >> The feature is implemented by Pavan (@ Marvell), supported by me,
> > > > > >> used by Qi (@ Intel), looks better than alternatives from application
> > > > > >> developer point of view [1] and finally postponed for 1 year without really
> > > > > >> strong motivation.
> > > > > >
> > > > > > I see different valuable point of views. This is enough motivation.
> > > > >
> > > > > It looks like I miss it in previous discussion, I would be thankful if
> > > > > you give me links to read or hints how to find.
> > > >
> > > > http://mails.dpdk.org/archives/dev/2019-November/150793.html
> > > >
> > > > > Introducing new types of controls would make configuration more and
> > > > > more complex. I think that many different types of control would
> > > > > over-complicate it. May be it is unavoidable, but it should be clear
> > > > > why the problem cannot be solved using existing types of controls
> > > > > (e.g. offloads).
> > > >
> > > > The offload control is used as an effective configuration for now.
> > > > The features which are configured with DEV_RX_OFFLOAD_*
> > > > do not need any other API to be used.
> > > > Extending DEV_RX_OFFLOAD_* bits for enabling features which
> > > > must be configured via other API anyway, is possible.
> > > > The real problem is that features in DEV_RX_OFFLOAD_* are supposed
> > > > to be disabled by default. If we add some opt-in features here,
> > > > we cannot enable them by default for API compatibility and do the
> > > > right thing by default.
> > > >
> > > > Choosing DEV_RX_OFFLOAD_* bits or rte_eth_dev_opt* functions is a detail.
> > > > The real decision is to change the API for using all these features.
> > > > Can we keep all features available by default (opt-out)?
> > >
> > > IMO, *rte_eth_dev_opt* has following problems
> > >
> > > 1) It is not multi-process friendly. If we are changing the Rx/Tx
> > > function pointer, based on
> > > the selected offload, then, using *rte_eth_dev_opt* scheme won't
> > > really work(if the new API
> > > called after the secondary process launch)
> >
> > Yes it must be used before launching the secondary process.
> > It is the same as DEV_RX_OFFLOAD_* config.
> 
> Yes. rte_eth_dev_opt_* has another dimension to enable and disable as API.
> So, we need to document, opt-in -> start() -> opt-out case won't work
> in multi process
> case.
> 
> >
> > > 2) If we are taking rte_eth_dev_opt path then by default feature has
> > > to be enabled to
> > > not break the functional ABI. That scheme won't scale if as when we
> > > keep adding the new features.
> > > It is always easy for the application to define "what it wants" vs
> > > "what it does not want"
> >
> > Yes, opt-in may look more natural than opt-out.
> > But opt-in makes the default more complex, and changes the API.
> >
> > > 3) Defining the device state after the reconfigure operation.
> > >
> > > IMO, if any operation is connected to fastpath it is better to use
> > > DEV_RX_OFFLOAD_ like
> > > this feature where enable or disable PMDs from updating
> > > ``rte_mbuf::hash::fdir`` so that if possible
> > > we can use different Rx function pointer if possible(Hence it can work
> > > with the multi-process case case)
> >
> > I reply to 2 and 3 together.
> >
> > We decided that offloads must be disabled by default.
> > This is what we have in 19.11:
> >         - Some offloads are enabled before start with DEV_?X_OFFLOAD_*
> >         - Some offloads are enabled with functions at any time
> >
> > For the second type of offloads, you want to know, before start,
> > whether it will be used or not.
> > If adding the second type of offloads (like rte_flow ones)
> > to DEV_?X_OFFLOAD_*, it means it must be enabled 2 times:
> >         - before start with offload bits
> >         - later with more precise functions
> >
> > I would like to avoid changing the default behaviour,
> > which is to enable an offload only one time.
> > That's why I think this second category of offloads should
> > offer opt-out (global disabling), so it will continue
> > to work by default if they are configured.
> >
> > I hope you understand the difference between the two categories.
> 
> I understand the difference. The only point of "difference in opinion" is
> the default behavior of the feature/offload. If it is in RX_OFFLOAD scheme then
> by default it is disabled. opt_* scheme makes this new feature/offload
> enabled default to avoid changing the default behavior.

OK, this is where we disagree.
I am for keeping what we agreed this year: all offloads are disabled by default.
But I am against the need for double enablement.
The offloads which are enabled with a specific function should not need
to be also enabled (opt-in) before start.

> It is good to avoid functional ABI change. But bad as,
> 1) New API starts bloating the ethdev API.

In general, I want to clean-up the ethdev API during next year.

> 2) It is diffcult for application guys to figure out what are features need to
> be disabled to performance as he/she does not know, for the given release,
> the enabled features.

Yes this is a good point.

> Item (1) is a trade-off between elegance vs ABI compatibility. No
> strong opinion on this.
> 
> To fix the item (2), Can we get have an API in ethdev to get enabled
> features so that
> the application can probe and disable if required?

We can think about something like that.
Note that there is also a need to better advertise all capabilities.

> For example, rte_eth_dev_set_ptypes() comes in same category, By default,
> ptype parsing is enabled. I think, we can have a general interface to
> "probe" the by default enabled features
> and disable it if required. Not scattered API for each feature.

This is an issue. The packet type parsing should be disabled by default.

> The above scheme fixe my concerns.
> 
> Thoughts?

Re: [dpdk-dev] [PATCH v2 1/3] ethdev: remove deprecation notice for packet type set

[Ferruh Yigit]

On 11/19/2019 3:04 PM, Ferruh Yigit wrote:
> On 11/19/2019 12:12 PM, Andrew Rybchenko wrote:
>> The functionality is added, but the deprecation notice is not
>> removed.
>>
>> Fixes: 5d4813acda2c ("ethdev: add packet type range function")
>>
>> Signed-off-by: Andrew Rybchenko <arybchenko@solarflare.com>
> 
> Acked-by: Ferruh Yigit <ferruh.yigit@intel.com>
> 

Applied to dpdk-next-net/master, thanks.

Re: [dpdk-dev] [PATCH v2 2/3] ethdev: remove deprecation notice about RSS hash offload

[Ferruh Yigit]

On 11/19/2019 3:04 PM, Ferruh Yigit wrote:
> On 11/19/2019 12:12 PM, Andrew Rybchenko wrote:
>> The offload was added, but the deprecation notice was not removed.
>>
>> Fixes: 5d308972954c ("ethdev: add mbuf RSS update as an offload")
>>
>> Signed-off-by: Andrew Rybchenko <arybchenko@solarflare.com>
> 
> Acked-by: Ferruh Yigit <ferruh.yigit@intel.com>
> 

Applied to dpdk-next-net/master, thanks.

Re: [dpdk-dev] [PATCH v2 3/3] ethdev: improve flow mark Rx offload deprecation notice

[Jerin Jacob]

On Mon, Nov 25, 2019 at 8:39 PM Thomas Monjalon <thomas@monjalon.net> wrote:
>
> 25/11/2019 11:44, Jerin Jacob:
> > On Sun, Nov 24, 2019 at 3:12 AM Thomas Monjalon <thomas@monjalon.net> wrote:
> > >
> > > 23/11/2019 10:42, Jerin Jacob:
> > > > On Sat, Nov 23, 2019 at 3:58 AM Thomas Monjalon <thomas@monjalon.net> wrote:
> > > > > 22/11/2019 12:53, Andrew Rybchenko:
> > > > > > On 11/22/19 2:15 PM, Thomas Monjalon wrote:
> > > > > > > 22/11/2019 11:12, Andrew Rybchenko:
> > > > > > >> On 11/22/19 1:01 AM, Thomas Monjalon wrote:
> > > > > > >>> 19/11/2019 13:12, Andrew Rybchenko:
> > > > > > >>>> The deprecation notice is required since it adds more requirements
> > > > > > >>>> when RTE flow mark and flag actions may be used and require
> > > > > > >>>> changes in applications.
> > > > > > >>> I am still not sure what is the best solution here.
> > > > > > >>> I continued to think about it in this thread:
> > > > > > >>>   http://mails.dpdk.org/archives/dev/2019-November/151960.html
> > > > > > >>>
> > > > > > >>> I think we cannot require any application change until 20.11
> > > > > > >>> in order to keep API (and behaviour) compatibility.
> > > > > > >> Expected, but still very disappointing.
> > > > > > >>
> > > > > > >> The feature is implemented by Pavan (@ Marvell), supported by me,
> > > > > > >> used by Qi (@ Intel), looks better than alternatives from application
> > > > > > >> developer point of view [1] and finally postponed for 1 year without really
> > > > > > >> strong motivation.
> > > > > > >
> > > > > > > I see different valuable point of views. This is enough motivation.
> > > > > >
> > > > > > It looks like I miss it in previous discussion, I would be thankful if
> > > > > > you give me links to read or hints how to find.
> > > > >
> > > > > http://mails.dpdk.org/archives/dev/2019-November/150793.html
> > > > >
> > > > > > Introducing new types of controls would make configuration more and
> > > > > > more complex. I think that many different types of control would
> > > > > > over-complicate it. May be it is unavoidable, but it should be clear
> > > > > > why the problem cannot be solved using existing types of controls
> > > > > > (e.g. offloads).
> > > > >
> > > > > The offload control is used as an effective configuration for now.
> > > > > The features which are configured with DEV_RX_OFFLOAD_*
> > > > > do not need any other API to be used.
> > > > > Extending DEV_RX_OFFLOAD_* bits for enabling features which
> > > > > must be configured via other API anyway, is possible.
> > > > > The real problem is that features in DEV_RX_OFFLOAD_* are supposed
> > > > > to be disabled by default. If we add some opt-in features here,
> > > > > we cannot enable them by default for API compatibility and do the
> > > > > right thing by default.
> > > > >
> > > > > Choosing DEV_RX_OFFLOAD_* bits or rte_eth_dev_opt* functions is a detail.
> > > > > The real decision is to change the API for using all these features.
> > > > > Can we keep all features available by default (opt-out)?
> > > >
> > > > IMO, *rte_eth_dev_opt* has following problems
> > > >
> > > > 1) It is not multi-process friendly. If we are changing the Rx/Tx
> > > > function pointer, based on
> > > > the selected offload, then, using *rte_eth_dev_opt* scheme won't
> > > > really work(if the new API
> > > > called after the secondary process launch)
> > >
> > > Yes it must be used before launching the secondary process.
> > > It is the same as DEV_RX_OFFLOAD_* config.
> >
> > Yes. rte_eth_dev_opt_* has another dimension to enable and disable as API.
> > So, we need to document, opt-in -> start() -> opt-out case won't work
> > in multi process
> > case.
> >
> > >
> > > > 2) If we are taking rte_eth_dev_opt path then by default feature has
> > > > to be enabled to
> > > > not break the functional ABI. That scheme won't scale if as when we
> > > > keep adding the new features.
> > > > It is always easy for the application to define "what it wants" vs
> > > > "what it does not want"
> > >
> > > Yes, opt-in may look more natural than opt-out.
> > > But opt-in makes the default more complex, and changes the API.
> > >
> > > > 3) Defining the device state after the reconfigure operation.
> > > >
> > > > IMO, if any operation is connected to fastpath it is better to use
> > > > DEV_RX_OFFLOAD_ like
> > > > this feature where enable or disable PMDs from updating
> > > > ``rte_mbuf::hash::fdir`` so that if possible
> > > > we can use different Rx function pointer if possible(Hence it can work
> > > > with the multi-process case case)
> > >
> > > I reply to 2 and 3 together.
> > >
> > > We decided that offloads must be disabled by default.
> > > This is what we have in 19.11:
> > >         - Some offloads are enabled before start with DEV_?X_OFFLOAD_*
> > >         - Some offloads are enabled with functions at any time
> > >
> > > For the second type of offloads, you want to know, before start,
> > > whether it will be used or not.
> > > If adding the second type of offloads (like rte_flow ones)
> > > to DEV_?X_OFFLOAD_*, it means it must be enabled 2 times:
> > >         - before start with offload bits
> > >         - later with more precise functions
> > >
> > > I would like to avoid changing the default behaviour,
> > > which is to enable an offload only one time.
> > > That's why I think this second category of offloads should
> > > offer opt-out (global disabling), so it will continue
> > > to work by default if they are configured.
> > >
> > > I hope you understand the difference between the two categories.
> >
> > I understand the difference. The only point of "difference in opinion" is
> > the default behavior of the feature/offload. If it is in RX_OFFLOAD scheme then
> > by default it is disabled. opt_* scheme makes this new feature/offload
> > enabled default to avoid changing the default behavior.
>
> OK, this is where we disagree.
> I am for keeping what we agreed this year: all offloads are disabled by default.
> But I am against the need for double enablement.
> The offloads which are enabled with a specific function should not need
> to be also enabled (opt-in) before start.

OK.

>
> > It is good to avoid functional ABI change. But bad as,
> > 1) New API starts bloating the ethdev API.
>
> In general, I want to clean-up the ethdev API during next year.
>
> > 2) It is diffcult for application guys to figure out what are features need to
> > be disabled to performance as he/she does not know, for the given release,
> > the enabled features.
>
> Yes this is a good point.
>
> > Item (1) is a trade-off between elegance vs ABI compatibility. No
> > strong opinion on this.
> >
> > To fix the item (2), Can we get have an API in ethdev to get enabled
> > features so that
> > the application can probe and disable if required?
>
> We can think about something like that.
> Note that there is also a need to better advertise all capabilities.
>
> > For example, rte_eth_dev_set_ptypes() comes in same category, By default,
> > ptype parsing is enabled. I think, we can have a general interface to
> > "probe" the by default enabled features
> > and disable it if required. Not scattered API for each feature.
>
> This is an issue. The packet type parsing should be disabled by default.

IMO, It makes sense to disable by default.

Isn't conflicting? One thread, we are saying for in order to make,
existing application work without breaking ABI, Default should be
enabled.

Thoughts?

And what would be DEFAULT behavior for the mbuf MARK updation feature?
(That's where this thread started).


>
> > The above scheme fixe my concerns.
> >
> > Thoughts?
>
>
>

Re: [dpdk-dev] [PATCH v2 3/3] ethdev: improve flow mark Rx offload deprecation notice

[Thomas Monjalon]

02/12/2019 05:21, Jerin Jacob:
> On Mon, Nov 25, 2019 at 8:39 PM Thomas Monjalon <thomas@monjalon.net> wrote:
> >
> > 25/11/2019 11:44, Jerin Jacob:
> > > On Sun, Nov 24, 2019 at 3:12 AM Thomas Monjalon <thomas@monjalon.net> wrote:
> > > >
> > > > 23/11/2019 10:42, Jerin Jacob:
> > > > > On Sat, Nov 23, 2019 at 3:58 AM Thomas Monjalon <thomas@monjalon.net> wrote:
> > > > > > 22/11/2019 12:53, Andrew Rybchenko:
> > > > > > > On 11/22/19 2:15 PM, Thomas Monjalon wrote:
> > > > > > > > 22/11/2019 11:12, Andrew Rybchenko:
> > > > > > > >> On 11/22/19 1:01 AM, Thomas Monjalon wrote:
> > > > > > > >>> 19/11/2019 13:12, Andrew Rybchenko:
> > > > > > > >>>> The deprecation notice is required since it adds more requirements
> > > > > > > >>>> when RTE flow mark and flag actions may be used and require
> > > > > > > >>>> changes in applications.
> > > > > > > >>> I am still not sure what is the best solution here.
> > > > > > > >>> I continued to think about it in this thread:
> > > > > > > >>>   http://mails.dpdk.org/archives/dev/2019-November/151960.html
> > > > > > > >>>
> > > > > > > >>> I think we cannot require any application change until 20.11
> > > > > > > >>> in order to keep API (and behaviour) compatibility.
> > > > > > > >> Expected, but still very disappointing.
> > > > > > > >>
> > > > > > > >> The feature is implemented by Pavan (@ Marvell), supported by me,
> > > > > > > >> used by Qi (@ Intel), looks better than alternatives from application
> > > > > > > >> developer point of view [1] and finally postponed for 1 year without really
> > > > > > > >> strong motivation.
> > > > > > > >
> > > > > > > > I see different valuable point of views. This is enough motivation.
> > > > > > >
> > > > > > > It looks like I miss it in previous discussion, I would be thankful if
> > > > > > > you give me links to read or hints how to find.
> > > > > >
> > > > > > http://mails.dpdk.org/archives/dev/2019-November/150793.html
> > > > > >
> > > > > > > Introducing new types of controls would make configuration more and
> > > > > > > more complex. I think that many different types of control would
> > > > > > > over-complicate it. May be it is unavoidable, but it should be clear
> > > > > > > why the problem cannot be solved using existing types of controls
> > > > > > > (e.g. offloads).
> > > > > >
> > > > > > The offload control is used as an effective configuration for now.
> > > > > > The features which are configured with DEV_RX_OFFLOAD_*
> > > > > > do not need any other API to be used.
> > > > > > Extending DEV_RX_OFFLOAD_* bits for enabling features which
> > > > > > must be configured via other API anyway, is possible.
> > > > > > The real problem is that features in DEV_RX_OFFLOAD_* are supposed
> > > > > > to be disabled by default. If we add some opt-in features here,
> > > > > > we cannot enable them by default for API compatibility and do the
> > > > > > right thing by default.
> > > > > >
> > > > > > Choosing DEV_RX_OFFLOAD_* bits or rte_eth_dev_opt* functions is a detail.
> > > > > > The real decision is to change the API for using all these features.
> > > > > > Can we keep all features available by default (opt-out)?
> > > > >
> > > > > IMO, *rte_eth_dev_opt* has following problems
> > > > >
> > > > > 1) It is not multi-process friendly. If we are changing the Rx/Tx
> > > > > function pointer, based on
> > > > > the selected offload, then, using *rte_eth_dev_opt* scheme won't
> > > > > really work(if the new API
> > > > > called after the secondary process launch)
> > > >
> > > > Yes it must be used before launching the secondary process.
> > > > It is the same as DEV_RX_OFFLOAD_* config.
> > >
> > > Yes. rte_eth_dev_opt_* has another dimension to enable and disable as API.
> > > So, we need to document, opt-in -> start() -> opt-out case won't work
> > > in multi process
> > > case.
> > >
> > > >
> > > > > 2) If we are taking rte_eth_dev_opt path then by default feature has
> > > > > to be enabled to
> > > > > not break the functional ABI. That scheme won't scale if as when we
> > > > > keep adding the new features.
> > > > > It is always easy for the application to define "what it wants" vs
> > > > > "what it does not want"
> > > >
> > > > Yes, opt-in may look more natural than opt-out.
> > > > But opt-in makes the default more complex, and changes the API.
> > > >
> > > > > 3) Defining the device state after the reconfigure operation.
> > > > >
> > > > > IMO, if any operation is connected to fastpath it is better to use
> > > > > DEV_RX_OFFLOAD_ like
> > > > > this feature where enable or disable PMDs from updating
> > > > > ``rte_mbuf::hash::fdir`` so that if possible
> > > > > we can use different Rx function pointer if possible(Hence it can work
> > > > > with the multi-process case case)
> > > >
> > > > I reply to 2 and 3 together.
> > > >
> > > > We decided that offloads must be disabled by default.
> > > > This is what we have in 19.11:
> > > >         - Some offloads are enabled before start with DEV_?X_OFFLOAD_*
> > > >         - Some offloads are enabled with functions at any time
> > > >
> > > > For the second type of offloads, you want to know, before start,
> > > > whether it will be used or not.
> > > > If adding the second type of offloads (like rte_flow ones)
> > > > to DEV_?X_OFFLOAD_*, it means it must be enabled 2 times:
> > > >         - before start with offload bits
> > > >         - later with more precise functions
> > > >
> > > > I would like to avoid changing the default behaviour,
> > > > which is to enable an offload only one time.
> > > > That's why I think this second category of offloads should
> > > > offer opt-out (global disabling), so it will continue
> > > > to work by default if they are configured.
> > > >
> > > > I hope you understand the difference between the two categories.
> > >
> > > I understand the difference. The only point of "difference in opinion" is
> > > the default behavior of the feature/offload. If it is in RX_OFFLOAD scheme then
> > > by default it is disabled. opt_* scheme makes this new feature/offload
> > > enabled default to avoid changing the default behavior.
> >
> > OK, this is where we disagree.
> > I am for keeping what we agreed this year: all offloads are disabled by default.
> > But I am against the need for double enablement.
> > The offloads which are enabled with a specific function should not need
> > to be also enabled (opt-in) before start.
> 
> OK.
> 
> >
> > > It is good to avoid functional ABI change. But bad as,
> > > 1) New API starts bloating the ethdev API.
> >
> > In general, I want to clean-up the ethdev API during next year.
> >
> > > 2) It is diffcult for application guys to figure out what are features need to
> > > be disabled to performance as he/she does not know, for the given release,
> > > the enabled features.
> >
> > Yes this is a good point.
> >
> > > Item (1) is a trade-off between elegance vs ABI compatibility. No
> > > strong opinion on this.
> > >
> > > To fix the item (2), Can we get have an API in ethdev to get enabled
> > > features so that
> > > the application can probe and disable if required?
> >
> > We can think about something like that.
> > Note that there is also a need to better advertise all capabilities.
> >
> > > For example, rte_eth_dev_set_ptypes() comes in same category, By default,
> > > ptype parsing is enabled. I think, we can have a general interface to
> > > "probe" the by default enabled features
> > > and disable it if required. Not scattered API for each feature.
> >
> > This is an issue. The packet type parsing should be disabled by default.
> 
> IMO, It makes sense to disable by default.
> 
> Isn't conflicting? One thread, we are saying for in order to make,
> existing application work without breaking ABI, Default should be
> enabled.
> 
> Thoughts?

Every offloads should be disabled by default.
This is a good reason to break the behaviour in 20.11.

> And what would be DEFAULT behavior for the mbuf MARK updation feature?
> (That's where this thread started).

As all other features, mark is disabled by default.
Using a rte_flow rule, it can be enabled.
No need to pre-enable it.

Re: [dpdk-dev] [PATCH v2 3/3] ethdev: improve flow mark Rx offload deprecation notice

[Jerin Jacob]

On Mon, Dec 2, 2019 at 6:16 PM Thomas Monjalon <thomas@monjalon.net> wrote:
>
> 02/12/2019 05:21, Jerin Jacob:
> > On Mon, Nov 25, 2019 at 8:39 PM Thomas Monjalon <thomas@monjalon.net> wrote:
> > >
> > > 25/11/2019 11:44, Jerin Jacob:
> > > > On Sun, Nov 24, 2019 at 3:12 AM Thomas Monjalon <thomas@monjalon.net> wrote:
> > > > >
> > > > > 23/11/2019 10:42, Jerin Jacob:
> > > > > > On Sat, Nov 23, 2019 at 3:58 AM Thomas Monjalon <thomas@monjalon.net> wrote:
> > > > > > > 22/11/2019 12:53, Andrew Rybchenko:
> > > > > > > > On 11/22/19 2:15 PM, Thomas Monjalon wrote:
> > > > > > > > > 22/11/2019 11:12, Andrew Rybchenko:
> > > > > > > > >> On 11/22/19 1:01 AM, Thomas Monjalon wrote:
> > > > > > > > >>> 19/11/2019 13:12, Andrew Rybchenko:
> > > > > > > > >>>> The deprecation notice is required since it adds more requirements
> > > > > > > > >>>> when RTE flow mark and flag actions may be used and require
> > > > > > > > >>>> changes in applications.
> > > > > > > > >>> I am still not sure what is the best solution here.
> > > > > > > > >>> I continued to think about it in this thread:
> > > > > > > > >>>   http://mails.dpdk.org/archives/dev/2019-November/151960.html
> > > > > > > > >>>
> > > > > > > > >>> I think we cannot require any application change until 20.11
> > > > > > > > >>> in order to keep API (and behaviour) compatibility.
> > > > > > > > >> Expected, but still very disappointing.
> > > > > > > > >>
> > > > > > > > >> The feature is implemented by Pavan (@ Marvell), supported by me,
> > > > > > > > >> used by Qi (@ Intel), looks better than alternatives from application
> > > > > > > > >> developer point of view [1] and finally postponed for 1 year without really
> > > > > > > > >> strong motivation.
> > > > > > > > >
> > > > > > > > > I see different valuable point of views. This is enough motivation.
> > > > > > > >
> > > > > > > > It looks like I miss it in previous discussion, I would be thankful if
> > > > > > > > you give me links to read or hints how to find.
> > > > > > >
> > > > > > > http://mails.dpdk.org/archives/dev/2019-November/150793.html
> > > > > > >
> > > > > > > > Introducing new types of controls would make configuration more and
> > > > > > > > more complex. I think that many different types of control would
> > > > > > > > over-complicate it. May be it is unavoidable, but it should be clear
> > > > > > > > why the problem cannot be solved using existing types of controls
> > > > > > > > (e.g. offloads).
> > > > > > >
> > > > > > > The offload control is used as an effective configuration for now.
> > > > > > > The features which are configured with DEV_RX_OFFLOAD_*
> > > > > > > do not need any other API to be used.
> > > > > > > Extending DEV_RX_OFFLOAD_* bits for enabling features which
> > > > > > > must be configured via other API anyway, is possible.
> > > > > > > The real problem is that features in DEV_RX_OFFLOAD_* are supposed
> > > > > > > to be disabled by default. If we add some opt-in features here,
> > > > > > > we cannot enable them by default for API compatibility and do the
> > > > > > > right thing by default.
> > > > > > >
> > > > > > > Choosing DEV_RX_OFFLOAD_* bits or rte_eth_dev_opt* functions is a detail.
> > > > > > > The real decision is to change the API for using all these features.
> > > > > > > Can we keep all features available by default (opt-out)?
> > > > > >
> > > > > > IMO, *rte_eth_dev_opt* has following problems
> > > > > >
> > > > > > 1) It is not multi-process friendly. If we are changing the Rx/Tx
> > > > > > function pointer, based on
> > > > > > the selected offload, then, using *rte_eth_dev_opt* scheme won't
> > > > > > really work(if the new API
> > > > > > called after the secondary process launch)
> > > > >
> > > > > Yes it must be used before launching the secondary process.
> > > > > It is the same as DEV_RX_OFFLOAD_* config.
> > > >
> > > > Yes. rte_eth_dev_opt_* has another dimension to enable and disable as API.
> > > > So, we need to document, opt-in -> start() -> opt-out case won't work
> > > > in multi process
> > > > case.
> > > >
> > > > >
> > > > > > 2) If we are taking rte_eth_dev_opt path then by default feature has
> > > > > > to be enabled to
> > > > > > not break the functional ABI. That scheme won't scale if as when we
> > > > > > keep adding the new features.
> > > > > > It is always easy for the application to define "what it wants" vs
> > > > > > "what it does not want"
> > > > >
> > > > > Yes, opt-in may look more natural than opt-out.
> > > > > But opt-in makes the default more complex, and changes the API.
> > > > >
> > > > > > 3) Defining the device state after the reconfigure operation.
> > > > > >
> > > > > > IMO, if any operation is connected to fastpath it is better to use
> > > > > > DEV_RX_OFFLOAD_ like
> > > > > > this feature where enable or disable PMDs from updating
> > > > > > ``rte_mbuf::hash::fdir`` so that if possible
> > > > > > we can use different Rx function pointer if possible(Hence it can work
> > > > > > with the multi-process case case)
> > > > >
> > > > > I reply to 2 and 3 together.
> > > > >
> > > > > We decided that offloads must be disabled by default.
> > > > > This is what we have in 19.11:
> > > > >         - Some offloads are enabled before start with DEV_?X_OFFLOAD_*
> > > > >         - Some offloads are enabled with functions at any time
> > > > >
> > > > > For the second type of offloads, you want to know, before start,
> > > > > whether it will be used or not.
> > > > > If adding the second type of offloads (like rte_flow ones)
> > > > > to DEV_?X_OFFLOAD_*, it means it must be enabled 2 times:
> > > > >         - before start with offload bits
> > > > >         - later with more precise functions
> > > > >
> > > > > I would like to avoid changing the default behaviour,
> > > > > which is to enable an offload only one time.
> > > > > That's why I think this second category of offloads should
> > > > > offer opt-out (global disabling), so it will continue
> > > > > to work by default if they are configured.
> > > > >
> > > > > I hope you understand the difference between the two categories.
> > > >
> > > > I understand the difference. The only point of "difference in opinion" is
> > > > the default behavior of the feature/offload. If it is in RX_OFFLOAD scheme then
> > > > by default it is disabled. opt_* scheme makes this new feature/offload
> > > > enabled default to avoid changing the default behavior.
> > >
> > > OK, this is where we disagree.
> > > I am for keeping what we agreed this year: all offloads are disabled by default.
> > > But I am against the need for double enablement.
> > > The offloads which are enabled with a specific function should not need
> > > to be also enabled (opt-in) before start.
> >
> > OK.
> >
> > >
> > > > It is good to avoid functional ABI change. But bad as,
> > > > 1) New API starts bloating the ethdev API.
> > >
> > > In general, I want to clean-up the ethdev API during next year.
> > >
> > > > 2) It is diffcult for application guys to figure out what are features need to
> > > > be disabled to performance as he/she does not know, for the given release,
> > > > the enabled features.
> > >
> > > Yes this is a good point.
> > >
> > > > Item (1) is a trade-off between elegance vs ABI compatibility. No
> > > > strong opinion on this.
> > > >
> > > > To fix the item (2), Can we get have an API in ethdev to get enabled
> > > > features so that
> > > > the application can probe and disable if required?
> > >
> > > We can think about something like that.
> > > Note that there is also a need to better advertise all capabilities.
> > >
> > > > For example, rte_eth_dev_set_ptypes() comes in same category, By default,
> > > > ptype parsing is enabled. I think, we can have a general interface to
> > > > "probe" the by default enabled features
> > > > and disable it if required. Not scattered API for each feature.
> > >
> > > This is an issue. The packet type parsing should be disabled by default.
> >
> > IMO, It makes sense to disable by default.
> >
> > Isn't conflicting? One thread, we are saying for in order to make,
> > existing application work without breaking ABI, Default should be
> > enabled.
> >
> > Thoughts?
>
> Every offloads should be disabled by default.
> This is a good reason to break the behaviour in 20.11.

Ack.

>
> > And what would be DEFAULT behavior for the mbuf MARK updation feature?
> > (That's where this thread started).
>
> As all other features, mark is disabled by default.
> Using a rte_flow rule, it can be enabled.
> No need to pre-enable it.

Ok.


>
>

Re: [dpdk-dev] [PATCH v2 3/3] ethdev: improve flow mark Rx offload deprecation notice

[Andrew Rybchenko]

On 12/2/19 2:09 PM, Jerin Jacob wrote:
> On Mon, Dec 2, 2019 at 6:16 PM Thomas Monjalon <thomas@monjalon.net> wrote:
>>
>> 02/12/2019 05:21, Jerin Jacob:
>>> On Mon, Nov 25, 2019 at 8:39 PM Thomas Monjalon <thomas@monjalon.net> wrote:
>>>>
>>>> 25/11/2019 11:44, Jerin Jacob:
>>>>> On Sun, Nov 24, 2019 at 3:12 AM Thomas Monjalon <thomas@monjalon.net> wrote:
>>>>>>
>>>>>> 23/11/2019 10:42, Jerin Jacob:
>>>>>>> On Sat, Nov 23, 2019 at 3:58 AM Thomas Monjalon <thomas@monjalon.net> wrote:
>>>>>>>> 22/11/2019 12:53, Andrew Rybchenko:
>>>>>>>>> On 11/22/19 2:15 PM, Thomas Monjalon wrote:
>>>>>>>>>> 22/11/2019 11:12, Andrew Rybchenko:
>>>>>>>>>>> On 11/22/19 1:01 AM, Thomas Monjalon wrote:
>>>>>>>>>>>> 19/11/2019 13:12, Andrew Rybchenko:
>>>>>>>>>>>>> The deprecation notice is required since it adds more requirements
>>>>>>>>>>>>> when RTE flow mark and flag actions may be used and require
>>>>>>>>>>>>> changes in applications.
>>>>>>>>>>>> I am still not sure what is the best solution here.
>>>>>>>>>>>> I continued to think about it in this thread:
>>>>>>>>>>>>   http://mails.dpdk.org/archives/dev/2019-November/151960.html
>>>>>>>>>>>>
>>>>>>>>>>>> I think we cannot require any application change until 20.11
>>>>>>>>>>>> in order to keep API (and behaviour) compatibility.
>>>>>>>>>>> Expected, but still very disappointing.
>>>>>>>>>>>
>>>>>>>>>>> The feature is implemented by Pavan (@ Marvell), supported by me,
>>>>>>>>>>> used by Qi (@ Intel), looks better than alternatives from application
>>>>>>>>>>> developer point of view [1] and finally postponed for 1 year without really
>>>>>>>>>>> strong motivation.
>>>>>>>>>>
>>>>>>>>>> I see different valuable point of views. This is enough motivation.
>>>>>>>>>
>>>>>>>>> It looks like I miss it in previous discussion, I would be thankful if
>>>>>>>>> you give me links to read or hints how to find.
>>>>>>>>
>>>>>>>> http://mails.dpdk.org/archives/dev/2019-November/150793.html
>>>>>>>>
>>>>>>>>> Introducing new types of controls would make configuration more and
>>>>>>>>> more complex. I think that many different types of control would
>>>>>>>>> over-complicate it. May be it is unavoidable, but it should be clear
>>>>>>>>> why the problem cannot be solved using existing types of controls
>>>>>>>>> (e.g. offloads).
>>>>>>>>
>>>>>>>> The offload control is used as an effective configuration for now.
>>>>>>>> The features which are configured with DEV_RX_OFFLOAD_*
>>>>>>>> do not need any other API to be used.
>>>>>>>> Extending DEV_RX_OFFLOAD_* bits for enabling features which
>>>>>>>> must be configured via other API anyway, is possible.
>>>>>>>> The real problem is that features in DEV_RX_OFFLOAD_* are supposed
>>>>>>>> to be disabled by default. If we add some opt-in features here,
>>>>>>>> we cannot enable them by default for API compatibility and do the
>>>>>>>> right thing by default.
>>>>>>>>
>>>>>>>> Choosing DEV_RX_OFFLOAD_* bits or rte_eth_dev_opt* functions is a detail.
>>>>>>>> The real decision is to change the API for using all these features.
>>>>>>>> Can we keep all features available by default (opt-out)?
>>>>>>>
>>>>>>> IMO, *rte_eth_dev_opt* has following problems
>>>>>>>
>>>>>>> 1) It is not multi-process friendly. If we are changing the Rx/Tx
>>>>>>> function pointer, based on
>>>>>>> the selected offload, then, using *rte_eth_dev_opt* scheme won't
>>>>>>> really work(if the new API
>>>>>>> called after the secondary process launch)
>>>>>>
>>>>>> Yes it must be used before launching the secondary process.
>>>>>> It is the same as DEV_RX_OFFLOAD_* config.
>>>>>
>>>>> Yes. rte_eth_dev_opt_* has another dimension to enable and disable as API.
>>>>> So, we need to document, opt-in -> start() -> opt-out case won't work
>>>>> in multi process
>>>>> case.
>>>>>
>>>>>>
>>>>>>> 2) If we are taking rte_eth_dev_opt path then by default feature has
>>>>>>> to be enabled to
>>>>>>> not break the functional ABI. That scheme won't scale if as when we
>>>>>>> keep adding the new features.
>>>>>>> It is always easy for the application to define "what it wants" vs
>>>>>>> "what it does not want"
>>>>>>
>>>>>> Yes, opt-in may look more natural than opt-out.
>>>>>> But opt-in makes the default more complex, and changes the API.
>>>>>>
>>>>>>> 3) Defining the device state after the reconfigure operation.
>>>>>>>
>>>>>>> IMO, if any operation is connected to fastpath it is better to use
>>>>>>> DEV_RX_OFFLOAD_ like
>>>>>>> this feature where enable or disable PMDs from updating
>>>>>>> ``rte_mbuf::hash::fdir`` so that if possible
>>>>>>> we can use different Rx function pointer if possible(Hence it can work
>>>>>>> with the multi-process case case)
>>>>>>
>>>>>> I reply to 2 and 3 together.
>>>>>>
>>>>>> We decided that offloads must be disabled by default.
>>>>>> This is what we have in 19.11:
>>>>>>         - Some offloads are enabled before start with DEV_?X_OFFLOAD_*
>>>>>>         - Some offloads are enabled with functions at any time
>>>>>>
>>>>>> For the second type of offloads, you want to know, before start,
>>>>>> whether it will be used or not.
>>>>>> If adding the second type of offloads (like rte_flow ones)
>>>>>> to DEV_?X_OFFLOAD_*, it means it must be enabled 2 times:
>>>>>>         - before start with offload bits
>>>>>>         - later with more precise functions
>>>>>>
>>>>>> I would like to avoid changing the default behaviour,
>>>>>> which is to enable an offload only one time.
>>>>>> That's why I think this second category of offloads should
>>>>>> offer opt-out (global disabling), so it will continue
>>>>>> to work by default if they are configured.
>>>>>>
>>>>>> I hope you understand the difference between the two categories.
>>>>>
>>>>> I understand the difference. The only point of "difference in opinion" is
>>>>> the default behavior of the feature/offload. If it is in RX_OFFLOAD scheme then
>>>>> by default it is disabled. opt_* scheme makes this new feature/offload
>>>>> enabled default to avoid changing the default behavior.
>>>>
>>>> OK, this is where we disagree.
>>>> I am for keeping what we agreed this year: all offloads are disabled by default.
>>>> But I am against the need for double enablement.
>>>> The offloads which are enabled with a specific function should not need
>>>> to be also enabled (opt-in) before start.
>>>
>>> OK.
>>>
>>>>
>>>>> It is good to avoid functional ABI change. But bad as,
>>>>> 1) New API starts bloating the ethdev API.
>>>>
>>>> In general, I want to clean-up the ethdev API during next year.
>>>>
>>>>> 2) It is diffcult for application guys to figure out what are features need to
>>>>> be disabled to performance as he/she does not know, for the given release,
>>>>> the enabled features.
>>>>
>>>> Yes this is a good point.
>>>>
>>>>> Item (1) is a trade-off between elegance vs ABI compatibility. No
>>>>> strong opinion on this.
>>>>>
>>>>> To fix the item (2), Can we get have an API in ethdev to get enabled
>>>>> features so that
>>>>> the application can probe and disable if required?
>>>>
>>>> We can think about something like that.
>>>> Note that there is also a need to better advertise all capabilities.
>>>>
>>>>> For example, rte_eth_dev_set_ptypes() comes in same category, By default,
>>>>> ptype parsing is enabled. I think, we can have a general interface to
>>>>> "probe" the by default enabled features
>>>>> and disable it if required. Not scattered API for each feature.
>>>>
>>>> This is an issue. The packet type parsing should be disabled by default.
>>>
>>> IMO, It makes sense to disable by default.
>>>
>>> Isn't conflicting? One thread, we are saying for in order to make,
>>> existing application work without breaking ABI, Default should be
>>> enabled.
>>>
>>> Thoughts?
>>
>> Every offloads should be disabled by default.
>> This is a good reason to break the behaviour in 20.11.
> 
> Ack.

Yes, I agree as well, but in general we already have an
exception MBUF_FAST_FREE which is just a nice wrap for
enabled by default support for mbufs from different
mempools and support for mbuf reference counters.
I don't suggest to change it. Just want to highlight
that we already have exceptions (nicely wrapped).
That's why I would not touch packet type parsing
"offload". Packet type parsing is not just on/off and
adding on/off in addition to existing API looks overkill.
Yes, it is one more exception, but nicely wrapped as well.

>>> And what would be DEFAULT behavior for the mbuf MARK updation feature?
>>> (That's where this thread started).
>>
>> As all other features, mark is disabled by default.
>> Using a rte_flow rule, it can be enabled.
>> No need to pre-enable it.
> 
> Ok.

But it returns us to the point where we started [1]:

The problem:
~~~~~~~~~~~~
PMD wants to know before port start if application wants to
to use flow MARK/FLAG in the future. It is required because:

1. HW may be configured in a different way to reserve resources
   for MARK/FLAG delivery.

2. Datapath implementation choice may depend on it (e.g. vPMD
   is faster, but does not support MARK).

opt-in/opt-out solution has drawbacks mentioned above.
Also I'm not sure if opt-in/opt-out is per-queue or per-port.
(Offloads may be naturally per-queue and it is a big advantage).

IMHO feature which should be opt-out is almost equivalent to
offload enabled by default. It has the same negative properties
as enabled by default offloads.

Am I missing something again?

From my point of view I see no problem in necessity to say
in advance (before device start) that application would like
to use some features at run time.

Yes, all features which may be controlled at run-time are
headache for optimizations (VLAN offloads).

[1]
http://inbox.dpdk.org/dev/f170105b-9c60-1b04-cb18-52e0951ddcdb@solarflare.com/

Re: [dpdk-dev] [PATCH v2 3/3] ethdev: improve flow mark Rx offload deprecation notice

[Jerin Jacob]

On Mon, Dec 2, 2019 at 5:27 PM Andrew Rybchenko
<arybchenko@solarflare.com> wrote:
>
> On 12/2/19 2:09 PM, Jerin Jacob wrote:
> > On Mon, Dec 2, 2019 at 6:16 PM Thomas Monjalon <thomas@monjalon.net> wrote:
> >>
> >> 02/12/2019 05:21, Jerin Jacob:
> >>> On Mon, Nov 25, 2019 at 8:39 PM Thomas Monjalon <thomas@monjalon.net> wrote:
> >>>>
> >>>> 25/11/2019 11:44, Jerin Jacob:
> >>>>> On Sun, Nov 24, 2019 at 3:12 AM Thomas Monjalon <thomas@monjalon.net> wrote:
> >>>>>>
> >>>>>> 23/11/2019 10:42, Jerin Jacob:
> >>>>>>> On Sat, Nov 23, 2019 at 3:58 AM Thomas Monjalon <thomas@monjalon.net> wrote:
> >>>>>>>> 22/11/2019 12:53, Andrew Rybchenko:
> >>>>>>>>> On 11/22/19 2:15 PM, Thomas Monjalon wrote:
> >>>>>>>>>> 22/11/2019 11:12, Andrew Rybchenko:
> >>>>>>>>>>> On 11/22/19 1:01 AM, Thomas Monjalon wrote:
> >>>>>>>>>>>> 19/11/2019 13:12, Andrew Rybchenko:
> >>>>>>>>>>>>> The deprecation notice is required since it adds more requirements
> >>>>>>>>>>>>> when RTE flow mark and flag actions may be used and require
> >>>>>>>>>>>>> changes in applications.
> >>>>>>>>>>>> I am still not sure what is the best solution here.
> >>>>>>>>>>>> I continued to think about it in this thread:
> >>>>>>>>>>>>   http://mails.dpdk.org/archives/dev/2019-November/151960.html
> >>>>>>>>>>>>
> >>>>>>>>>>>> I think we cannot require any application change until 20.11
> >>>>>>>>>>>> in order to keep API (and behaviour) compatibility.
> >>>>>>>>>>> Expected, but still very disappointing.
> >>>>>>>>>>>
> >>>>>>>>>>> The feature is implemented by Pavan (@ Marvell), supported by me,
> >>>>>>>>>>> used by Qi (@ Intel), looks better than alternatives from application
> >>>>>>>>>>> developer point of view [1] and finally postponed for 1 year without really
> >>>>>>>>>>> strong motivation.
> >>>>>>>>>>
> >>>>>>>>>> I see different valuable point of views. This is enough motivation.
> >>>>>>>>>
> >>>>>>>>> It looks like I miss it in previous discussion, I would be thankful if
> >>>>>>>>> you give me links to read or hints how to find.
> >>>>>>>>
> >>>>>>>> http://mails.dpdk.org/archives/dev/2019-November/150793.html
> >>>>>>>>
> >>>>>>>>> Introducing new types of controls would make configuration more and
> >>>>>>>>> more complex. I think that many different types of control would
> >>>>>>>>> over-complicate it. May be it is unavoidable, but it should be clear
> >>>>>>>>> why the problem cannot be solved using existing types of controls
> >>>>>>>>> (e.g. offloads).
> >>>>>>>>
> >>>>>>>> The offload control is used as an effective configuration for now.
> >>>>>>>> The features which are configured with DEV_RX_OFFLOAD_*
> >>>>>>>> do not need any other API to be used.
> >>>>>>>> Extending DEV_RX_OFFLOAD_* bits for enabling features which
> >>>>>>>> must be configured via other API anyway, is possible.
> >>>>>>>> The real problem is that features in DEV_RX_OFFLOAD_* are supposed
> >>>>>>>> to be disabled by default. If we add some opt-in features here,
> >>>>>>>> we cannot enable them by default for API compatibility and do the
> >>>>>>>> right thing by default.
> >>>>>>>>
> >>>>>>>> Choosing DEV_RX_OFFLOAD_* bits or rte_eth_dev_opt* functions is a detail.
> >>>>>>>> The real decision is to change the API for using all these features.
> >>>>>>>> Can we keep all features available by default (opt-out)?
> >>>>>>>
> >>>>>>> IMO, *rte_eth_dev_opt* has following problems
> >>>>>>>
> >>>>>>> 1) It is not multi-process friendly. If we are changing the Rx/Tx
> >>>>>>> function pointer, based on
> >>>>>>> the selected offload, then, using *rte_eth_dev_opt* scheme won't
> >>>>>>> really work(if the new API
> >>>>>>> called after the secondary process launch)
> >>>>>>
> >>>>>> Yes it must be used before launching the secondary process.
> >>>>>> It is the same as DEV_RX_OFFLOAD_* config.
> >>>>>
> >>>>> Yes. rte_eth_dev_opt_* has another dimension to enable and disable as API.
> >>>>> So, we need to document, opt-in -> start() -> opt-out case won't work
> >>>>> in multi process
> >>>>> case.
> >>>>>
> >>>>>>
> >>>>>>> 2) If we are taking rte_eth_dev_opt path then by default feature has
> >>>>>>> to be enabled to
> >>>>>>> not break the functional ABI. That scheme won't scale if as when we
> >>>>>>> keep adding the new features.
> >>>>>>> It is always easy for the application to define "what it wants" vs
> >>>>>>> "what it does not want"
> >>>>>>
> >>>>>> Yes, opt-in may look more natural than opt-out.
> >>>>>> But opt-in makes the default more complex, and changes the API.
> >>>>>>
> >>>>>>> 3) Defining the device state after the reconfigure operation.
> >>>>>>>
> >>>>>>> IMO, if any operation is connected to fastpath it is better to use
> >>>>>>> DEV_RX_OFFLOAD_ like
> >>>>>>> this feature where enable or disable PMDs from updating
> >>>>>>> ``rte_mbuf::hash::fdir`` so that if possible
> >>>>>>> we can use different Rx function pointer if possible(Hence it can work
> >>>>>>> with the multi-process case case)
> >>>>>>
> >>>>>> I reply to 2 and 3 together.
> >>>>>>
> >>>>>> We decided that offloads must be disabled by default.
> >>>>>> This is what we have in 19.11:
> >>>>>>         - Some offloads are enabled before start with DEV_?X_OFFLOAD_*
> >>>>>>         - Some offloads are enabled with functions at any time
> >>>>>>
> >>>>>> For the second type of offloads, you want to know, before start,
> >>>>>> whether it will be used or not.
> >>>>>> If adding the second type of offloads (like rte_flow ones)
> >>>>>> to DEV_?X_OFFLOAD_*, it means it must be enabled 2 times:
> >>>>>>         - before start with offload bits
> >>>>>>         - later with more precise functions
> >>>>>>
> >>>>>> I would like to avoid changing the default behaviour,
> >>>>>> which is to enable an offload only one time.
> >>>>>> That's why I think this second category of offloads should
> >>>>>> offer opt-out (global disabling), so it will continue
> >>>>>> to work by default if they are configured.
> >>>>>>
> >>>>>> I hope you understand the difference between the two categories.
> >>>>>
> >>>>> I understand the difference. The only point of "difference in opinion" is
> >>>>> the default behavior of the feature/offload. If it is in RX_OFFLOAD scheme then
> >>>>> by default it is disabled. opt_* scheme makes this new feature/offload
> >>>>> enabled default to avoid changing the default behavior.
> >>>>
> >>>> OK, this is where we disagree.
> >>>> I am for keeping what we agreed this year: all offloads are disabled by default.
> >>>> But I am against the need for double enablement.
> >>>> The offloads which are enabled with a specific function should not need
> >>>> to be also enabled (opt-in) before start.
> >>>
> >>> OK.
> >>>
> >>>>
> >>>>> It is good to avoid functional ABI change. But bad as,
> >>>>> 1) New API starts bloating the ethdev API.
> >>>>
> >>>> In general, I want to clean-up the ethdev API during next year.
> >>>>
> >>>>> 2) It is diffcult for application guys to figure out what are features need to
> >>>>> be disabled to performance as he/she does not know, for the given release,
> >>>>> the enabled features.
> >>>>
> >>>> Yes this is a good point.
> >>>>
> >>>>> Item (1) is a trade-off between elegance vs ABI compatibility. No
> >>>>> strong opinion on this.
> >>>>>
> >>>>> To fix the item (2), Can we get have an API in ethdev to get enabled
> >>>>> features so that
> >>>>> the application can probe and disable if required?
> >>>>
> >>>> We can think about something like that.
> >>>> Note that there is also a need to better advertise all capabilities.
> >>>>
> >>>>> For example, rte_eth_dev_set_ptypes() comes in same category, By default,
> >>>>> ptype parsing is enabled. I think, we can have a general interface to
> >>>>> "probe" the by default enabled features
> >>>>> and disable it if required. Not scattered API for each feature.
> >>>>
> >>>> This is an issue. The packet type parsing should be disabled by default.
> >>>
> >>> IMO, It makes sense to disable by default.
> >>>
> >>> Isn't conflicting? One thread, we are saying for in order to make,
> >>> existing application work without breaking ABI, Default should be
> >>> enabled.
> >>>
> >>> Thoughts?
> >>
> >> Every offloads should be disabled by default.
> >> This is a good reason to break the behaviour in 20.11.
> >
> > Ack.
>
> Yes, I agree as well, but in general we already have an
> exception MBUF_FAST_FREE which is just a nice wrap for
> enabled by default support for mbufs from different
> mempools and support for mbuf reference counters.
> I don't suggest to change it. Just want to highlight
> that we already have exceptions (nicely wrapped).
> That's why I would not touch packet type parsing
> "offload". Packet type parsing is not just on/off and
> adding on/off in addition to existing API looks overkill.
> Yes, it is one more exception, but nicely wrapped as well.

I am all for making offloads disabled by default.

>
> >>> And what would be DEFAULT behavior for the mbuf MARK updation feature?
> >>> (That's where this thread started).
> >>
> >> As all other features, mark is disabled by default.
> >> Using a rte_flow rule, it can be enabled.
> >> No need to pre-enable it.
> >
> > Ok.
>
> But it returns us to the point where we started [1]:
>
> The problem:
> ~~~~~~~~~~~~
> PMD wants to know before port start if application wants to
> to use flow MARK/FLAG in the future. It is required because:
>
> 1. HW may be configured in a different way to reserve resources
>    for MARK/FLAG delivery.
>
> 2. Datapath implementation choice may depend on it (e.g. vPMD
>    is faster, but does not support MARK).
>
> opt-in/opt-out solution has drawbacks mentioned above.
> Also I'm not sure if opt-in/opt-out is per-queue or per-port.
> (Offloads may be naturally per-queue and it is a big advantage).
>
> IMHO feature which should be opt-out is almost equivalent to
> offload enabled by default. It has the same negative properties
> as enabled by default offloads.
>
> Am I missing something again?
>
> From my point of view I see no problem in necessity to say
> in advance (before device start) that application would like
> to use some features at run time.

I agree with your problem definition and solution as offload.

I think, our constraint is, we can not change functional ABI behavior
for the next year. i.e The existing application should work for the
next year without
changing the code.

I think, it all boiling down to adhere to that constraint or not for
this specific case.

Once that is decided, we can wrap it in offload flags vs opt scheme
(by default enabled scheme).

>
> Yes, all features which may be controlled at run-time are
> headache for optimizations (VLAN offloads).
>
> [1]
> http://inbox.dpdk.org/dev/f170105b-9c60-1b04-cb18-52e0951ddcdb@solarflare.com/

Re: [dpdk-dev] [PATCH v2 3/3] ethdev: improve flow mark Rx offload deprecation notice

[Andrew Rybchenko]

On 12/5/19 11:12 AM, Jerin Jacob wrote:
> On Mon, Dec 2, 2019 at 5:27 PM Andrew Rybchenko
> <arybchenko@solarflare.com> wrote:
>>
>> On 12/2/19 2:09 PM, Jerin Jacob wrote:
>>> On Mon, Dec 2, 2019 at 6:16 PM Thomas Monjalon <thomas@monjalon.net> wrote:
>>>>
>>>> 02/12/2019 05:21, Jerin Jacob:
>>>>> On Mon, Nov 25, 2019 at 8:39 PM Thomas Monjalon <thomas@monjalon.net> wrote:
>>>>>>
>>>>>> 25/11/2019 11:44, Jerin Jacob:
>>>>>>> On Sun, Nov 24, 2019 at 3:12 AM Thomas Monjalon <thomas@monjalon.net> wrote:
>>>>>>>>
>>>>>>>> 23/11/2019 10:42, Jerin Jacob:
>>>>>>>>> On Sat, Nov 23, 2019 at 3:58 AM Thomas Monjalon <thomas@monjalon.net> wrote:
>>>>>>>>>> 22/11/2019 12:53, Andrew Rybchenko:
>>>>>>>>>>> On 11/22/19 2:15 PM, Thomas Monjalon wrote:
>>>>>>>>>>>> 22/11/2019 11:12, Andrew Rybchenko:
>>>>>>>>>>>>> On 11/22/19 1:01 AM, Thomas Monjalon wrote:
>>>>>>>>>>>>>> 19/11/2019 13:12, Andrew Rybchenko:
>>>>>>>>>>>>>>> The deprecation notice is required since it adds more requirements
>>>>>>>>>>>>>>> when RTE flow mark and flag actions may be used and require
>>>>>>>>>>>>>>> changes in applications.
>>>>>>>>>>>>>> I am still not sure what is the best solution here.
>>>>>>>>>>>>>> I continued to think about it in this thread:
>>>>>>>>>>>>>>   http://mails.dpdk.org/archives/dev/2019-November/151960.html
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> I think we cannot require any application change until 20.11
>>>>>>>>>>>>>> in order to keep API (and behaviour) compatibility.
>>>>>>>>>>>>> Expected, but still very disappointing.
>>>>>>>>>>>>>
>>>>>>>>>>>>> The feature is implemented by Pavan (@ Marvell), supported by me,
>>>>>>>>>>>>> used by Qi (@ Intel), looks better than alternatives from application
>>>>>>>>>>>>> developer point of view [1] and finally postponed for 1 year without really
>>>>>>>>>>>>> strong motivation.
>>>>>>>>>>>>
>>>>>>>>>>>> I see different valuable point of views. This is enough motivation.
>>>>>>>>>>>
>>>>>>>>>>> It looks like I miss it in previous discussion, I would be thankful if
>>>>>>>>>>> you give me links to read or hints how to find.
>>>>>>>>>>
>>>>>>>>>> http://mails.dpdk.org/archives/dev/2019-November/150793.html
>>>>>>>>>>
>>>>>>>>>>> Introducing new types of controls would make configuration more and
>>>>>>>>>>> more complex. I think that many different types of control would
>>>>>>>>>>> over-complicate it. May be it is unavoidable, but it should be clear
>>>>>>>>>>> why the problem cannot be solved using existing types of controls
>>>>>>>>>>> (e.g. offloads).
>>>>>>>>>>
>>>>>>>>>> The offload control is used as an effective configuration for now.
>>>>>>>>>> The features which are configured with DEV_RX_OFFLOAD_*
>>>>>>>>>> do not need any other API to be used.
>>>>>>>>>> Extending DEV_RX_OFFLOAD_* bits for enabling features which
>>>>>>>>>> must be configured via other API anyway, is possible.
>>>>>>>>>> The real problem is that features in DEV_RX_OFFLOAD_* are supposed
>>>>>>>>>> to be disabled by default. If we add some opt-in features here,
>>>>>>>>>> we cannot enable them by default for API compatibility and do the
>>>>>>>>>> right thing by default.
>>>>>>>>>>
>>>>>>>>>> Choosing DEV_RX_OFFLOAD_* bits or rte_eth_dev_opt* functions is a detail.
>>>>>>>>>> The real decision is to change the API for using all these features.
>>>>>>>>>> Can we keep all features available by default (opt-out)?
>>>>>>>>>
>>>>>>>>> IMO, *rte_eth_dev_opt* has following problems
>>>>>>>>>
>>>>>>>>> 1) It is not multi-process friendly. If we are changing the Rx/Tx
>>>>>>>>> function pointer, based on
>>>>>>>>> the selected offload, then, using *rte_eth_dev_opt* scheme won't
>>>>>>>>> really work(if the new API
>>>>>>>>> called after the secondary process launch)
>>>>>>>>
>>>>>>>> Yes it must be used before launching the secondary process.
>>>>>>>> It is the same as DEV_RX_OFFLOAD_* config.
>>>>>>>
>>>>>>> Yes. rte_eth_dev_opt_* has another dimension to enable and disable as API.
>>>>>>> So, we need to document, opt-in -> start() -> opt-out case won't work
>>>>>>> in multi process
>>>>>>> case.
>>>>>>>
>>>>>>>>
>>>>>>>>> 2) If we are taking rte_eth_dev_opt path then by default feature has
>>>>>>>>> to be enabled to
>>>>>>>>> not break the functional ABI. That scheme won't scale if as when we
>>>>>>>>> keep adding the new features.
>>>>>>>>> It is always easy for the application to define "what it wants" vs
>>>>>>>>> "what it does not want"
>>>>>>>>
>>>>>>>> Yes, opt-in may look more natural than opt-out.
>>>>>>>> But opt-in makes the default more complex, and changes the API.
>>>>>>>>
>>>>>>>>> 3) Defining the device state after the reconfigure operation.
>>>>>>>>>
>>>>>>>>> IMO, if any operation is connected to fastpath it is better to use
>>>>>>>>> DEV_RX_OFFLOAD_ like
>>>>>>>>> this feature where enable or disable PMDs from updating
>>>>>>>>> ``rte_mbuf::hash::fdir`` so that if possible
>>>>>>>>> we can use different Rx function pointer if possible(Hence it can work
>>>>>>>>> with the multi-process case case)
>>>>>>>>
>>>>>>>> I reply to 2 and 3 together.
>>>>>>>>
>>>>>>>> We decided that offloads must be disabled by default.
>>>>>>>> This is what we have in 19.11:
>>>>>>>>         - Some offloads are enabled before start with DEV_?X_OFFLOAD_*
>>>>>>>>         - Some offloads are enabled with functions at any time
>>>>>>>>
>>>>>>>> For the second type of offloads, you want to know, before start,
>>>>>>>> whether it will be used or not.
>>>>>>>> If adding the second type of offloads (like rte_flow ones)
>>>>>>>> to DEV_?X_OFFLOAD_*, it means it must be enabled 2 times:
>>>>>>>>         - before start with offload bits
>>>>>>>>         - later with more precise functions
>>>>>>>>
>>>>>>>> I would like to avoid changing the default behaviour,
>>>>>>>> which is to enable an offload only one time.
>>>>>>>> That's why I think this second category of offloads should
>>>>>>>> offer opt-out (global disabling), so it will continue
>>>>>>>> to work by default if they are configured.
>>>>>>>>
>>>>>>>> I hope you understand the difference between the two categories.
>>>>>>>
>>>>>>> I understand the difference. The only point of "difference in opinion" is
>>>>>>> the default behavior of the feature/offload. If it is in RX_OFFLOAD scheme then
>>>>>>> by default it is disabled. opt_* scheme makes this new feature/offload
>>>>>>> enabled default to avoid changing the default behavior.
>>>>>>
>>>>>> OK, this is where we disagree.
>>>>>> I am for keeping what we agreed this year: all offloads are disabled by default.
>>>>>> But I am against the need for double enablement.
>>>>>> The offloads which are enabled with a specific function should not need
>>>>>> to be also enabled (opt-in) before start.
>>>>>
>>>>> OK.
>>>>>
>>>>>>
>>>>>>> It is good to avoid functional ABI change. But bad as,
>>>>>>> 1) New API starts bloating the ethdev API.
>>>>>>
>>>>>> In general, I want to clean-up the ethdev API during next year.
>>>>>>
>>>>>>> 2) It is diffcult for application guys to figure out what are features need to
>>>>>>> be disabled to performance as he/she does not know, for the given release,
>>>>>>> the enabled features.
>>>>>>
>>>>>> Yes this is a good point.
>>>>>>
>>>>>>> Item (1) is a trade-off between elegance vs ABI compatibility. No
>>>>>>> strong opinion on this.
>>>>>>>
>>>>>>> To fix the item (2), Can we get have an API in ethdev to get enabled
>>>>>>> features so that
>>>>>>> the application can probe and disable if required?
>>>>>>
>>>>>> We can think about something like that.
>>>>>> Note that there is also a need to better advertise all capabilities.
>>>>>>
>>>>>>> For example, rte_eth_dev_set_ptypes() comes in same category, By default,
>>>>>>> ptype parsing is enabled. I think, we can have a general interface to
>>>>>>> "probe" the by default enabled features
>>>>>>> and disable it if required. Not scattered API for each feature.
>>>>>>
>>>>>> This is an issue. The packet type parsing should be disabled by default.
>>>>>
>>>>> IMO, It makes sense to disable by default.
>>>>>
>>>>> Isn't conflicting? One thread, we are saying for in order to make,
>>>>> existing application work without breaking ABI, Default should be
>>>>> enabled.
>>>>>
>>>>> Thoughts?
>>>>
>>>> Every offloads should be disabled by default.
>>>> This is a good reason to break the behaviour in 20.11.
>>>
>>> Ack.
>>
>> Yes, I agree as well, but in general we already have an
>> exception MBUF_FAST_FREE which is just a nice wrap for
>> enabled by default support for mbufs from different
>> mempools and support for mbuf reference counters.
>> I don't suggest to change it. Just want to highlight
>> that we already have exceptions (nicely wrapped).
>> That's why I would not touch packet type parsing
>> "offload". Packet type parsing is not just on/off and
>> adding on/off in addition to existing API looks overkill.
>> Yes, it is one more exception, but nicely wrapped as well.
> 
> I am all for making offloads disabled by default.
> 
>>
>>>>> And what would be DEFAULT behavior for the mbuf MARK updation feature?
>>>>> (That's where this thread started).
>>>>
>>>> As all other features, mark is disabled by default.
>>>> Using a rte_flow rule, it can be enabled.
>>>> No need to pre-enable it.
>>>
>>> Ok.
>>
>> But it returns us to the point where we started [1]:
>>
>> The problem:
>> ~~~~~~~~~~~~
>> PMD wants to know before port start if application wants to
>> to use flow MARK/FLAG in the future. It is required because:
>>
>> 1. HW may be configured in a different way to reserve resources
>>    for MARK/FLAG delivery.
>>
>> 2. Datapath implementation choice may depend on it (e.g. vPMD
>>    is faster, but does not support MARK).
>>
>> opt-in/opt-out solution has drawbacks mentioned above.
>> Also I'm not sure if opt-in/opt-out is per-queue or per-port.
>> (Offloads may be naturally per-queue and it is a big advantage).
>>
>> IMHO feature which should be opt-out is almost equivalent to
>> offload enabled by default. It has the same negative properties
>> as enabled by default offloads.
>>
>> Am I missing something again?
>>
>> From my point of view I see no problem in necessity to say
>> in advance (before device start) that application would like
>> to use some features at run time.
> 
> I agree with your problem definition and solution as offload.
> 
> I think, our constraint is, we can not change functional ABI behavior
> for the next year. i.e The existing application should work for the
> next year without
> changing the code.
> 
> I think, it all boiling down to adhere to that constraint or not for
> this specific case.

May be the escape is to avoid consistency checks in generic
code (not sure that such checks are doable/required in this
case, but anyway) and make the behaviour change vendor/driver-
specific. I understand that it is far from ideal solution.

May be offload should be combined with opt-out as a way to
disable. I.e. offload is positive (not negative), but enabled
by default (i.e. automatically added to offloads as we do
for RSS_HASH) with an experimental opt-out to disable it.

As the result:
1. There is no changes in behaviour from application point of
   view.
2. Application which care about performance and ready to use
   experimental opt-out to optimize performance can do it.
   (i.e. use opt-out to avoid the offload enabled by default).
3. Later when window to normalize behaviour opens, opt-out
   becomes NOP (i.e. it still could be preserved for some
   time to simplify transition).
4. The offload is enabled by default during transition
   period only since it represents a feature which had
   no offload flag before and was always enabled before.
5. As an offload the feature may be controlled per-device
   and per-queue natively.

It still does not sort out "necessity to enable twice"
concern which for specified above "the problem", IMO,
contradicts to "disabled by default offloads" (I read
it as "the best performance" by default).

> Once that is decided, we can wrap it in offload flags vs opt scheme
> (by default enabled scheme).

Yes. May be I don't understand all the details of the opt
scheme right now, but I don't like what I can imagine as
described above.

>>
>> Yes, all features which may be controlled at run-time are
>> headache for optimizations (VLAN offloads).
>>
>> [1]
>> http://inbox.dpdk.org/dev/f170105b-9c60-1b04-cb18-52e0951ddcdb@solarflare.com/

Re: [dpdk-dev] [PATCH v2 3/3] ethdev: improve flow mark Rx offload deprecation notice

[Jerin Jacob]

On Mon, Dec 9, 2019 at 2:47 PM Andrew Rybchenko
<arybchenko@solarflare.com> wrote:
>
> On 12/5/19 11:12 AM, Jerin Jacob wrote:
> > On Mon, Dec 2, 2019 at 5:27 PM Andrew Rybchenko
> > <arybchenko@solarflare.com> wrote:
> >>


>> >>>
> >>> Ack.
> >>
> >> Yes, I agree as well, but in general we already have an
> >> exception MBUF_FAST_FREE which is just a nice wrap for
> >> enabled by default support for mbufs from different
> >> mempools and support for mbuf reference counters.
> >> I don't suggest to change it. Just want to highlight
> >> that we already have exceptions (nicely wrapped).
> >> That's why I would not touch packet type parsing
> >> "offload". Packet type parsing is not just on/off and
> >> adding on/off in addition to existing API looks overkill.
> >> Yes, it is one more exception, but nicely wrapped as well.
> >
> > I am all for making offloads disabled by default.
> >
> >>
> >>>>> And what would be DEFAULT behavior for the mbuf MARK updation feature?
> >>>>> (That's where this thread started).
> >>>>
> >>>> As all other features, mark is disabled by default.
> >>>> Using a rte_flow rule, it can be enabled.
> >>>> No need to pre-enable it.
> >>>
> >>> Ok.
> >>
> >> But it returns us to the point where we started [1]:
> >>
> >> The problem:
> >> ~~~~~~~~~~~~
> >> PMD wants to know before port start if application wants to
> >> to use flow MARK/FLAG in the future. It is required because:
> >>
> >> 1. HW may be configured in a different way to reserve resources
> >>    for MARK/FLAG delivery.
> >>
> >> 2. Datapath implementation choice may depend on it (e.g. vPMD
> >>    is faster, but does not support MARK).
> >>
> >> opt-in/opt-out solution has drawbacks mentioned above.
> >> Also I'm not sure if opt-in/opt-out is per-queue or per-port.
> >> (Offloads may be naturally per-queue and it is a big advantage).
> >>
> >> IMHO feature which should be opt-out is almost equivalent to
> >> offload enabled by default. It has the same negative properties
> >> as enabled by default offloads.
> >>
> >> Am I missing something again?
> >>
> >> From my point of view I see no problem in necessity to say
> >> in advance (before device start) that application would like
> >> to use some features at run time.
> >
> > I agree with your problem definition and solution as offload.
> >
> > I think, our constraint is, we can not change functional ABI behavior
> > for the next year. i.e The existing application should work for the
> > next year without
> > changing the code.
> >
> > I think, it all boiling down to adhere to that constraint or not for
> > this specific case.
>
> May be the escape is to avoid consistency checks in generic
> code (not sure that such checks are doable/required in this
> case, but anyway) and make the behaviour change vendor/driver-
> specific. I understand that it is far from ideal solution.
>
> May be offload should be combined with opt-out as a way to
> disable. I.e. offload is positive (not negative), but enabled
> by default (i.e. automatically added to offloads as we do
> for RSS_HASH) with an experimental opt-out to disable it.
>
> As the result:
> 1. There is no changes in behaviour from application point of
>    view.
> 2. Application which care about performance and ready to use
>    experimental opt-out to optimize performance can do it.
>    (i.e. use opt-out to avoid the offload enabled by default).
> 3. Later when window to normalize behaviour opens, opt-out
>    becomes NOP (i.e. it still could be preserved for some
>    time to simplify transition).
> 4. The offload is enabled by default during transition
>    period only since it represents a feature which had
>    no offload flag before and was always enabled before.
> 5. As an offload the feature may be controlled per-device
>    and per-queue natively.

Looks good to me.
It makes sense to have a generic opt API to have for year ABI,
which works on

- per queue/per port
- Enable by default to keep backward compatible.
- Have a generic signature to allow probe() all the enabled opt-in features
and then disable if required by the application.
- I think, rte_eth_dev_set_ptypes()  needs to change to generic API as
it comes under opt-in/out scheme.


>
> It still does not sort out "necessity to enable twice"
> concern which for specified above "the problem", IMO,
> contradicts to "disabled by default offloads" (I read
> it as "the best performance" by default).
>
> > Once that is decided, we can wrap it in offload flags vs opt scheme
> > (by default enabled scheme).
>
> Yes. May be I don't understand all the details of the opt
> scheme right now, but I don't like what I can imagine as
> described above.
>
> >>
> >> Yes, all features which may be controlled at run-time are
> >> headache for optimizations (VLAN offloads).
> >>
> >> [1]
> >> http://inbox.dpdk.org/dev/f170105b-9c60-1b04-cb18-52e0951ddcdb@solarflare.com/
>

Re: [dpdk-dev] [PATCH v2 3/3] ethdev: improve flow mark Rx offload deprecation notice

[Andrew Rybchenko]

On 12/16/19 10:38 AM, Jerin Jacob wrote:
> On Mon, Dec 9, 2019 at 2:47 PM Andrew Rybchenko
> <arybchenko@solarflare.com> wrote:
>>
>> On 12/5/19 11:12 AM, Jerin Jacob wrote:
>>> On Mon, Dec 2, 2019 at 5:27 PM Andrew Rybchenko
>>> <arybchenko@solarflare.com> wrote:
>>>> 
>>>>>>
>>>>> Ack.
>>>>
>>>> Yes, I agree as well, but in general we already have an
>>>> exception MBUF_FAST_FREE which is just a nice wrap for
>>>> enabled by default support for mbufs from different
>>>> mempools and support for mbuf reference counters.
>>>> I don't suggest to change it. Just want to highlight
>>>> that we already have exceptions (nicely wrapped).
>>>> That's why I would not touch packet type parsing
>>>> "offload". Packet type parsing is not just on/off and
>>>> adding on/off in addition to existing API looks overkill.
>>>> Yes, it is one more exception, but nicely wrapped as well.
>>>
>>> I am all for making offloads disabled by default.
>>>
>>>>
>>>>>>> And what would be DEFAULT behavior for the mbuf MARK updation feature?
>>>>>>> (That's where this thread started).
>>>>>>
>>>>>> As all other features, mark is disabled by default.
>>>>>> Using a rte_flow rule, it can be enabled.
>>>>>> No need to pre-enable it.
>>>>>
>>>>> Ok.
>>>>
>>>> But it returns us to the point where we started [1]:
>>>>
>>>> The problem:
>>>> ~~~~~~~~~~~~
>>>> PMD wants to know before port start if application wants to
>>>> to use flow MARK/FLAG in the future. It is required because:
>>>>
>>>> 1. HW may be configured in a different way to reserve resources
>>>>    for MARK/FLAG delivery.
>>>>
>>>> 2. Datapath implementation choice may depend on it (e.g. vPMD
>>>>    is faster, but does not support MARK).
>>>>
>>>> opt-in/opt-out solution has drawbacks mentioned above.
>>>> Also I'm not sure if opt-in/opt-out is per-queue or per-port.
>>>> (Offloads may be naturally per-queue and it is a big advantage).
>>>>
>>>> IMHO feature which should be opt-out is almost equivalent to
>>>> offload enabled by default. It has the same negative properties
>>>> as enabled by default offloads.
>>>>
>>>> Am I missing something again?
>>>>
>>>> From my point of view I see no problem in necessity to say
>>>> in advance (before device start) that application would like
>>>> to use some features at run time.
>>>
>>> I agree with your problem definition and solution as offload.
>>>
>>> I think, our constraint is, we can not change functional ABI behavior
>>> for the next year. i.e The existing application should work for the
>>> next year without
>>> changing the code.
>>>
>>> I think, it all boiling down to adhere to that constraint or not for
>>> this specific case.
>>
>> May be the escape is to avoid consistency checks in generic
>> code (not sure that such checks are doable/required in this
>> case, but anyway) and make the behaviour change vendor/driver-
>> specific. I understand that it is far from ideal solution.
>>
>> May be offload should be combined with opt-out as a way to
>> disable. I.e. offload is positive (not negative), but enabled
>> by default (i.e. automatically added to offloads as we do
>> for RSS_HASH) with an experimental opt-out to disable it.
>>
>> As the result:
>> 1. There is no changes in behaviour from application point of
>>    view.
>> 2. Application which care about performance and ready to use
>>    experimental opt-out to optimize performance can do it.
>>    (i.e. use opt-out to avoid the offload enabled by default).
>> 3. Later when window to normalize behaviour opens, opt-out
>>    becomes NOP (i.e. it still could be preserved for some
>>    time to simplify transition).
>> 4. The offload is enabled by default during transition
>>    period only since it represents a feature which had
>>    no offload flag before and was always enabled before.
>> 5. As an offload the feature may be controlled per-device
>>    and per-queue natively.
> 
> Looks good to me.
> It makes sense to have a generic opt API to have for year ABI,
> which works on
> 
> - per queue/per port
> - Enable by default to keep backward compatible.
> - Have a generic signature to allow probe() all the enabled opt-in features
> and then disable if required by the application.

I'd like to clarify to be sure that we're on the same page:
1. Add DEV_RX_OFFLOAD_FLOW_MARK offload:
   - enabled by default till 20.11 to preserve behaviour
   - applications may migrate and explicitly enable
   - disabled by default since 20.11 to switch to generic
     policy which require offloads to be disabled by default
2. Add experimental opt-out which allow to disable the
   offload to optimize performance for applications which
   would like to care about it early
   - opt-out remains but becomes NOP in 20.11

> - I think, rte_eth_dev_set_ptypes()  needs to change to generic API as
> it comes under opt-in/out scheme.

I'm not sure that I understand how it should look like for
ptypes.

>>
>> It still does not sort out "necessity to enable twice"
>> concern which for specified above "the problem", IMO,
>> contradicts to "disabled by default offloads" (I read
>> it as "the best performance" by default).
>>
>>> Once that is decided, we can wrap it in offload flags vs opt scheme
>>> (by default enabled scheme).
>>
>> Yes. May be I don't understand all the details of the opt
>> scheme right now, but I don't like what I can imagine as
>> described above.
>>
>>>>
>>>> Yes, all features which may be controlled at run-time are
>>>> headache for optimizations (VLAN offloads).
>>>>
>>>> [1]
>>>> http://inbox.dpdk.org/dev/f170105b-9c60-1b04-cb18-52e0951ddcdb@solarflare.com/
>>