[dpdk-dev] [PATCH] doc: announce API change in mbuf

[dpdk-dev] [PATCH] doc: announce API change in mbuf

[Thomas Monjalon]

In order to prepare for adding more features requiring more space in mbuf,
some static fields must become dynamic.
Some small layout changes may have performance benefits as well.

The deprecation notice for atomic refcount is moved and reworded
to fit below the layout deprecation.

Signed-off-by: Thomas Monjalon <thomas@monjalon.net>
---
 doc/guides/rel_notes/deprecation.rst | 26 ++++++++++++++++++++------
 1 file changed, 20 insertions(+), 6 deletions(-)

diff --git a/doc/guides/rel_notes/deprecation.rst b/doc/guides/rel_notes/deprecation.rst
index ea4cfa7a48..015c386c76 100644
--- a/doc/guides/rel_notes/deprecation.rst
+++ b/doc/guides/rel_notes/deprecation.rst
@@ -67,6 +67,26 @@ Deprecation Notices
   us extending existing enum/define.
   One solution can be using a fixed size array instead of ``.*MAX.*`` value.
 
+* mbuf: Some fields will be converted to dynamic API in DPDK 20.11
+  in order to reserve more space for the dynamic fields.
+  The following static fields will be moved as dynamic:
+
+  - ``timestamp``
+  - ``userdata`` / ``udata64``
+  - ``seqn``
+
+  As a consequence, the layout of the ``struct rte_mbuf`` will be re-arranged,
+  avoiding impact on vectorized implementation of the driver datapaths,
+  while evaluating performance gains of a better use of the first cache line.
+
+  The deprecated unioned fields ``buf_physaddr`` and ``refcnt_atomic``
+  (as explained below) will be removed in DPDK 20.11.
+
+* mbuf: ``refcnt_atomic`` member in structures ``rte_mbuf`` and
+  ``rte_mbuf_ext_shared_info`` is of type ``rte_atomic16_t``.
+  Due to adoption of C11 atomic builtins, the field ``refcnt_atomic``
+  will be replaced with ``refcnt`` of type ``uint16_t`` in DPDK 20.11.
+
 * ethdev: Split the ``struct eth_dev_ops`` struct to hide it as much as possible
   will be done in 20.11.
   Currently the ``struct eth_dev_ops`` struct is accessible by the application
@@ -132,12 +152,6 @@ Deprecation Notices
   in "rte_sched.h". These changes are aligned to improvements suggested in the
   RFC https://mails.dpdk.org/archives/dev/2018-November/120035.html.
 
-* mbuf: ``refcnt_atomic`` member in structures ``rte_mbuf`` and
-  ``rte_mbuf_ext_shared_info`` is of type ``rte_atomic16_t``. Due to adoption
-  of C11 atomic builtins it will be of type ``uint16_t``. ``refcnt_atomic``
-  will be removed in 20.11. It will be replaced with ``refcnt`` of type
-  ``uint16_t``.
-
 * metrics: The function ``rte_metrics_init`` will have a non-void return
   in order to notify errors instead of calling ``rte_exit``.
 
-- 
2.27.0

Re: [dpdk-dev] [PATCH] doc: announce API change in mbuf

[Olivier Matz]

On Fri, Jul 31, 2020 at 06:03:49PM +0200, Thomas Monjalon wrote:
> In order to prepare for adding more features requiring more space in mbuf,
> some static fields must become dynamic.
> Some small layout changes may have performance benefits as well.
> 
> The deprecation notice for atomic refcount is moved and reworded
> to fit below the layout deprecation.
> 
> Signed-off-by: Thomas Monjalon <thomas@monjalon.net>

Acked-by: Olivier Matz <olivier.matz@6wind.com>

Re: [dpdk-dev] [PATCH] doc: announce API change in mbuf

[Andrew Rybchenko]

On 8/2/20 10:28 PM, Olivier Matz wrote:
> On Fri, Jul 31, 2020 at 06:03:49PM +0200, Thomas Monjalon wrote:
>> In order to prepare for adding more features requiring more space in mbuf,
>> some static fields must become dynamic.
>> Some small layout changes may have performance benefits as well.
>>
>> The deprecation notice for atomic refcount is moved and reworded
>> to fit below the layout deprecation.
>>
>> Signed-off-by: Thomas Monjalon <thomas@monjalon.net>
> 
> Acked-by: Olivier Matz <olivier.matz@6wind.com>
> 

Acked-by: Andrew Rybchenko <arybchenko@solarflare.com>

Re: [dpdk-dev] [PATCH] doc: announce API change in mbuf

[David Marchand]

On Fri, Jul 31, 2020 at 6:04 PM Thomas Monjalon <thomas@monjalon.net> wrote:
>
> In order to prepare for adding more features requiring more space in mbuf,
> some static fields must become dynamic.
> Some small layout changes may have performance benefits as well.
>
> The deprecation notice for atomic refcount is moved and reworded
> to fit below the layout deprecation.
>
> Signed-off-by: Thomas Monjalon <thomas@monjalon.net>

Acked-by: David Marchand <david.marchand@redhat.com>


-- 
David Marchand

Re: [dpdk-dev] [PATCH] doc: announce API change in mbuf

[Kinsella, Ray]

On 31/07/2020 17:03, Thomas Monjalon wrote:
> In order to prepare for adding more features requiring more space in mbuf,
> some static fields must become dynamic.
> Some small layout changes may have performance benefits as well.
> 
> The deprecation notice for atomic refcount is moved and reworded
> to fit below the layout deprecation.
> 
> Signed-off-by: Thomas Monjalon <thomas@monjalon.net>
> ---
>  doc/guides/rel_notes/deprecation.rst | 26 ++++++++++++++++++++------
>  1 file changed, 20 insertions(+), 6 deletions(-)
> 
> diff --git a/doc/guides/rel_notes/deprecation.rst b/doc/guides/rel_notes/deprecation.rst
> index ea4cfa7a48..015c386c76 100644
> --- a/doc/guides/rel_notes/deprecation.rst
> +++ b/doc/guides/rel_notes/deprecation.rst
> @@ -67,6 +67,26 @@ Deprecation Notices
>    us extending existing enum/define.
>    One solution can be using a fixed size array instead of ``.*MAX.*`` value.
>  
> +* mbuf: Some fields will be converted to dynamic API in DPDK 20.11
> +  in order to reserve more space for the dynamic fields.
> +  The following static fields will be moved as dynamic:
> +
> +  - ``timestamp``
> +  - ``userdata`` / ``udata64``
> +  - ``seqn``
> +
> +  As a consequence, the layout of the ``struct rte_mbuf`` will be re-arranged,
> +  avoiding impact on vectorized implementation of the driver datapaths,
> +  while evaluating performance gains of a better use of the first cache line.
> +
> +  The deprecated unioned fields ``buf_physaddr`` and ``refcnt_atomic``
> +  (as explained below) will be removed in DPDK 20.11.

So what does anyone using these fields do - except panic perhaps.
Can we include a reference to the DPDK Userspace presentation, 
in advance of the documentation supporting the dynamic mbuf.
To prepare people for what is to come ... 

https://www.youtube.com/watch?v=Ttl6MlhmzWY

> +
> +* mbuf: ``refcnt_atomic`` member in structures ``rte_mbuf`` and
> +  ``rte_mbuf_ext_shared_info`` is of type ``rte_atomic16_t``.
> +  Due to adoption of C11 atomic builtins, the field ``refcnt_atomic``
> +  will be replaced with ``refcnt`` of type ``uint16_t`` in DPDK 20.11.
> +
>  * ethdev: Split the ``struct eth_dev_ops`` struct to hide it as much as possible
>    will be done in 20.11.
>    Currently the ``struct eth_dev_ops`` struct is accessible by the application
> @@ -132,12 +152,6 @@ Deprecation Notices
>    in "rte_sched.h". These changes are aligned to improvements suggested in the
>    RFC https://mails.dpdk.org/archives/dev/2018-November/120035.html.
>  
> -* mbuf: ``refcnt_atomic`` member in structures ``rte_mbuf`` and
> -  ``rte_mbuf_ext_shared_info`` is of type ``rte_atomic16_t``. Due to adoption
> -  of C11 atomic builtins it will be of type ``uint16_t``. ``refcnt_atomic``
> -  will be removed in 20.11. It will be replaced with ``refcnt`` of type
> -  ``uint16_t``.
> -
>  * metrics: The function ``rte_metrics_init`` will have a non-void return
>    in order to notify errors instead of calling ``rte_exit``.
>  
> 

Re: [dpdk-dev] [PATCH] doc: announce API change in mbuf

[Thomas Monjalon]

05/08/2020 11:28, Kinsella, Ray:
> 
> On 31/07/2020 17:03, Thomas Monjalon wrote:
> > In order to prepare for adding more features requiring more space in mbuf,
> > some static fields must become dynamic.
> > Some small layout changes may have performance benefits as well.
> > 
> > The deprecation notice for atomic refcount is moved and reworded
> > to fit below the layout deprecation.
> > 
> > Signed-off-by: Thomas Monjalon <thomas@monjalon.net>
> > ---
> 
> So what does anyone using these fields do - except panic perhaps.
> Can we include a reference to the DPDK Userspace presentation, 
> in advance of the documentation supporting the dynamic mbuf.
> To prepare people for what is to come ... 
> 
> https://www.youtube.com/watch?v=Ttl6MlhmzWY

Good idea, link added.

Applied

[dpdk-dev] [PATCH] doc: announce API change in mbuf

[Thomas Monjalon]

In order to prepare for a long term stable API, the mbuf library
has to change: allowing more features as dynamic fields,
and fixing the lack of rte_ prefix in the namespace.
The namespace fix should not break the compatibility by keeping
some aliases during few releases to give time for migration.

Signed-off-by: Thomas Monjalon <thomas@monjalon.net>
---
 doc/guides/rel_notes/deprecation.rst | 5 +++++
 1 file changed, 5 insertions(+)

diff --git a/doc/guides/rel_notes/deprecation.rst b/doc/guides/rel_notes/deprecation.rst
index 37b8592b6..e4939f1fe 100644
--- a/doc/guides/rel_notes/deprecation.rst
+++ b/doc/guides/rel_notes/deprecation.rst
@@ -51,6 +51,11 @@ Deprecation Notices
   structure would be made internal (or removed if all dependencies are cleared)
   in future releases.
 
+* mbuf: Some fields will be moved or converted to dynamic API in DPDK 19.11
+  in order to offer more space for the dynamic fields in future.
+  The namespace will be fixed at the same time while keeping a backward
+  compatibility for some time.
+
 * ethdev: the legacy filter API, including
   ``rte_eth_dev_filter_supported()``, ``rte_eth_dev_filter_ctrl()`` as well
   as filter types MACVLAN, ETHERTYPE, FLEXIBLE, SYN, NTUPLE, TUNNEL, FDIR,
-- 
2.21.0

Re: [dpdk-dev] [PATCH] doc: announce API change in mbuf

[Olivier Matz]

On Thu, Aug 01, 2019 at 02:41:23PM +0200, Thomas Monjalon wrote:
> In order to prepare for a long term stable API, the mbuf library
> has to change: allowing more features as dynamic fields,
> and fixing the lack of rte_ prefix in the namespace.
> The namespace fix should not break the compatibility by keeping
> some aliases during few releases to give time for migration.
> 
> Signed-off-by: Thomas Monjalon <thomas@monjalon.net>

Thanks Thomas for sending the deprecation notice.

Acked-by: Olivier Matz <olivier.matz@6wind.com>

Re: [dpdk-dev] [PATCH] doc: announce API change in mbuf

[Andrew Rybchenko]

On 8/1/19 3:41 PM, Thomas Monjalon wrote:
> In order to prepare for a long term stable API, the mbuf library
> has to change: allowing more features as dynamic fields,
> and fixing the lack of rte_ prefix in the namespace.
> The namespace fix should not break the compatibility by keeping
> some aliases during few releases to give time for migration.
>
> Signed-off-by: Thomas Monjalon <thomas@monjalon.net>
> ---
>   doc/guides/rel_notes/deprecation.rst | 5 +++++
>   1 file changed, 5 insertions(+)
>
> diff --git a/doc/guides/rel_notes/deprecation.rst b/doc/guides/rel_notes/deprecation.rst
> index 37b8592b6..e4939f1fe 100644
> --- a/doc/guides/rel_notes/deprecation.rst
> +++ b/doc/guides/rel_notes/deprecation.rst
> @@ -51,6 +51,11 @@ Deprecation Notices
>     structure would be made internal (or removed if all dependencies are cleared)
>     in future releases.
>   
> +* mbuf: Some fields will be moved or converted to dynamic API in DPDK 19.11
> +  in order to offer more space for the dynamic fields in future.
> +  The namespace will be fixed at the same time while keeping a backward
> +  compatibility for some time.
> +
>   * ethdev: the legacy filter API, including
>     ``rte_eth_dev_filter_supported()``, ``rte_eth_dev_filter_ctrl()`` as well
>     as filter types MACVLAN, ETHERTYPE, FLEXIBLE, SYN, NTUPLE, TUNNEL, FDIR,

The deprecation notice is better than nothing, but "some fields" is
hardly useful since it doe not provide enough information if an
application is affected or not.

Also we have no yet information on performance impact of
switching to dynamic fields.

May be it is not a problem if switching does not happen.

I have no strong opinion, just trying to raise my concerns.
May be you already have answers on it.

Re: [dpdk-dev] [PATCH] doc: announce API change in mbuf

[Thomas Monjalon]

05/08/2019 10:05, Andrew Rybchenko:
> On 8/1/19 3:41 PM, Thomas Monjalon wrote:
> > In order to prepare for a long term stable API, the mbuf library
> > has to change: allowing more features as dynamic fields,
> > and fixing the lack of rte_ prefix in the namespace.
> > The namespace fix should not break the compatibility by keeping
> > some aliases during few releases to give time for migration.
> >
> > Signed-off-by: Thomas Monjalon <thomas@monjalon.net>
[..]
> > +* mbuf: Some fields will be moved or converted to dynamic API in DPDK 19.11
> > +  in order to offer more space for the dynamic fields in future.
> > +  The namespace will be fixed at the same time while keeping a backward
> > +  compatibility for some time.
> 
> The deprecation notice is better than nothing, but "some fields" is
> hardly useful since it doe not provide enough information if an
> application is affected or not.
> 
> Also we have no yet information on performance impact of
> switching to dynamic fields.
> 
> May be it is not a problem if switching does not happen.
> 
> I have no strong opinion, just trying to raise my concerns.
> May be you already have answers on it.

Unfortunately I don't have answers because I did not start this work.
We will have to evaluate the impact and discuss what to move or not.