Re: [dpdk-dev] [PATCHv2 2/2] ABI: Add some documentation

["Gonzalez Monroy, Sergio" <sergio.gonzalez.monroy@intel.com>]

On 25/06/2015 08:42, Gonzalez Monroy, Sergio wrote:
> On 25/06/2015 08:19, Zhang, Helin wrote:
>>
>>> -----Original Message-----
>>> From: dev [mailto:dev-bounces@dpdk.org] On Behalf Of Neil Horman
>>> Sent: Thursday, June 25, 2015 2:35 AM
>>> To: dev@dpdk.org
>>> Subject: [dpdk-dev] [PATCHv2 2/2] ABI: Add some documentation
>>>
>>> People have been asking for ways to use the ABI macros, heres some 
>>> docs to
>>> clarify their use.  Included is:
>>>
>>> * An overview of what ABI is
>>> * Details of the ABI deprecation process
>>> * Details of the versioning macros
>>> * Examples of their use
>>> * Details of how to use the ABI validator
>>>
>>> Thanks to John Mcnamara, who duplicated much of this effort at Intel 
>>> while I was
>>> working on it.  Much of the introductory material was gathered and 
>>> cleaned up
>>> by him
>>>
>>> Signed-off-by: Neil Horman <nhorman@tuxdriver.com>
>>> CC: john.mcnamara@intel.com
>>> CC: thomas.monjalon@6wind.com
>>>
>>> Change notes:
>>>
>>> v2)
>>>       * Fixed RST indentations and spelling errors
>>>       * Rebased to upstream to fix index.rst conflict
>>> ---
>>>   doc/guides/guidelines/index.rst      |   1 +
>>>   doc/guides/guidelines/versioning.rst | 456
>>> +++++++++++++++++++++++++++++++++++
>>>   2 files changed, 457 insertions(+)
>>>   create mode 100644 doc/guides/guidelines/versioning.rst
>>>
>>> diff --git a/doc/guides/guidelines/index.rst 
>>> b/doc/guides/guidelines/index.rst
>>> index 0ee9ab3..bfb9fa3 100644
>>> --- a/doc/guides/guidelines/index.rst
>>> +++ b/doc/guides/guidelines/index.rst
>>> @@ -7,3 +7,4 @@ Guidelines
>>>
>>>       coding_style
>>>       design
>>> +    versioning
>>> diff --git a/doc/guides/guidelines/versioning.rst
>>> b/doc/guides/guidelines/versioning.rst
>>> new file mode 100644
>>> index 0000000..2aef526
>>> --- /dev/null
>>> +++ b/doc/guides/guidelines/versioning.rst
>>> @@ -0,0 +1,456 @@
>>> +Managing ABI updates
>>> +====================
>>> +
>>> +Description
>>> +-----------
>>> +
>>> +This document details some methods for handling ABI management in the
>>> DPDK.
>>> +Note this document is not exhaustive, in that C library versioning is
>>> +flexible allowing multiple methods to achieve various goals, but it
>>> +will provide the user with some introductory methods
>>> +
>>> +General Guidelines
>>> +------------------
>>> +
>>> +#. Whenever possible, ABI should be preserved #. The addition of
>>> +symbols is generally not problematic #. The modification of symbols 
>>> can
>>> +generally be managed with versioning #. The removal of symbols
>>> +generally is an ABI break and requires bumping of the
>>> +   LIBABIVER macro
>>> +
>>> +What is an ABI
>>> +--------------
>>> +
>>> +An ABI (Application Binary Interface) is the set of runtime interfaces
>>> +exposed by a library. It is similar to an API (Application Programming
>>> +Interface) but is the result of compilation.  It is also effectively
>>> +cloned when applications link to dynamic libraries.  That is to say
>>> +when an application is compiled to link against dynamic libraries, it
>>> +is assumed that the ABI remains constant between the time the 
>>> application is
>>> compiled/linked, and the time that it runs.
>>> +Therefore, in the case of dynamic linking, it is critical that an ABI
>>> +is preserved, or (when modified), done in such a way that the
>>> +application is unable to behave improperly or in an unexpected 
>>> fashion.
>>> +
>>> +The DPDK ABI policy
>>> +-------------------
>>> +
>>> +ABI versions are set at the time of major release labeling, and the 
>>> ABI
>>> +may change multiple times, without warning, between the last release
>>> +label and the HEAD label of the git tree.
>>> +
>>> +ABI versions, once released, are available until such time as their
>>> +deprecation has been noted in the Release Notes for at least one major
>>> +release cycle. For example consider the case where the ABI for DPDK 
>>> 2.0
>>> +has been shipped and then a decision is made to modify it during the
>>> +development of DPDK 2.1. The decision will be recorded in the Release
>>> +Notes for the DPDK 2.1 release and the modification will be made 
>>> available in
>>> the DPDK 2.2 release.
>>> +
>>> +ABI versions may be deprecated in whole or in part as needed by a 
>>> given
>>> +update.
>>> +
>>> +Some ABI changes may be too significant to reasonably maintain 
>>> multiple
>>> +versions. In those cases ABI's may be updated without backward
>>> +compatibility being provided. The requirements for doing so are:
>>> +
>>> +#. At least 3 acknowledgments of the need to do so must be made on the
>>> +   dpdk.org mailing list.
>>> +
>>> +#. A full deprecation cycle, as explained above, must be made to offer
>>> +   downstream consumers sufficient warning of the change.
>>> +
>>> +#. The ``LIBABIVER`` variable in the makefile(s) where the ABI 
>>> changes are
>>> +   incorporated must be incremented in parallel with the ABI changes
>>> +   themselves.
>>> +
>>> +Note that the above process for ABI deprecation should not be
>>> +undertaken lightly. ABI stability is extremely important for 
>>> downstream
>>> +consumers of the DPDK, especially when distributed in shared object
>>> +form. Every effort should be made to preserve the ABI whenever
>>> +possible. The ABI should only be changed for significant reasons, such
>>> +as performance enhancements. ABI breakage due to changes such as
>>> +reorganizing public structure fields for aesthetic or readability 
>>> purposes should
>>> be avoided.
>>> +
>>> +Examples of Deprecation Notices
>>> +-------------------------------
>>> +
>>> +The following are some examples of ABI deprecation notices which would
>>> +be added to the Release Notes:
>>> +
>>> +* The Macro ``#RTE_FOO`` is deprecated and will be removed with 
>>> version
>>> +2.0,
>>> +  to be replaced with the inline function ``rte_foo()``.
>>> +
>>> +* The function ``rte_mbuf_grok()`` has been updated to include a new
>>> +parameter
>>> +  in version 2.0. Backwards compatibility will be maintained for this
>>> +function
>>> +  until the release of version 2.1
>>> +
>>> +* The members of ``struct rte_foo`` have been reorganized in release
>>> +2.0 for
>>> +  performance reasons. Existing binary applications will have 
>>> backwards
>>> +  compatibility in release 2.0, while newly built binaries will 
>>> need to
>>> +  reference the new structure variant ``struct rte_foo2``.
>>> +Compatibility will
>>> +  be removed in release 2.2, and all applications will require 
>>> updating
>>> +and
>>> +  rebuilding to the new structure at that time, which will be renamed
>>> +to the
>>> +  original ``struct rte_foo``.
>>> +
>>> +* Significant ABI changes are planned for the ``librte_dostuff``
>>> +library. The
>>> +  upcoming release 2.0 will not contain these changes, but release 2.1
>>> +will,
>>> +  and no backwards compatibility is planned due to the extensive 
>>> nature
>>> +of
>>> +  these changes. Binaries using this library built prior to version 
>>> 2.1
>>> +will
>>> +  require updating and recompilation.
>>> +
>>> +Versioning Macros
>>> +-----------------
>>> +
>>> +When a symbol is exported from a library to provide an API, it also
>>> +provides a calling convention (ABI) that is embodied in its name,
>>> +return type and arguments. Occasionally that function may need to
>>> +change to accommodate new functionality or behavior. When that occurs,
>>> +it is desirable to allow for backward compatibility for a time with
>>> +older binaries that are dynamically linked to the DPDK.
>>> +
>>> +To support backward compatibility the
>>> +``lib/librte_compat/rte_compat.h``
>>> +header file provides macros to use when updating exported functions.
>>> +These macros are used in conjunction with the
>>> +``rte_<library>_version.map`` file for a given library to allow
>>> +multiple versions of a symbol to exist in a shared library so that 
>>> older binaries
>>> need not be immediately recompiled.
>>> +
>>> +The macros exported are:
>>> +
>>> +* ``VERSION_SYMBOL(b, e, n)``: Creates a symbol version table entry
>>> +binding
>>> +  unversioned symbol ``b`` to the internal function ``b_e``.
>>> +
>>> +
>>> +* ``BASE_SYMBOL(b, e)``: Creates a symbol version table entry binding
>>> +  unversioned symbol ``b`` to the internal function ``b_e``.
>>> +
>>> +* ``BIND_DEFAULT_SYMBOL(b, e, n)``: Creates a symbol version entry
>>> +instructing
>>> +  the linker to bind references to symbol ``b`` to the internal symbol
>>> +  ``b_e``.
>>> +
>>> +
>>> +Examples of ABI Macro use
>>> +-------------------------
>>> +
>>> +Updating a public API
>>> +~~~~~~~~~~~~~~~~~~~~~
>>> +
>>> +Assume we have a function as follows
>>> +
>>> +.. code-block:: c
>>> +
>>> + /*
>>> +  * Create an acl context object for apps to
>>> +  * manipulate
>>> +  */
>>> + struct rte_acl_ctx *
>>> + rte_acl_create(const struct rte_acl_param *param) {
>>> +        ...
>>> + }
>>> +
>>> +
>>> +Assume that struct rte_acl_ctx is a private structure, and that a
>>> +developer wishes to enhance the acl api so that a debugging flag 
>>> can be
>>> +enabled on a per-context basis.  This requires an addition to the
>>> +structure (which, being private, is safe), but it also requires
>>> +modifying the code as follows
>>> +
>>> +.. code-block:: c
>>> +
>>> + /*
>>> +  * Create an acl context object for apps to
>>> +  * manipulate
>>> +  */
>>> + struct rte_acl_ctx *
>>> + rte_acl_create(const struct rte_acl_param *param, int debug) {
>>> +        ...
>>> + }
>>> +
>>> +
>>> +Note also that, being a public function, the header file prototype 
>>> must
>>> +also be changed, as must all the call sites, to reflect the new ABI
>>> +footprint.  We will maintain previous ABI versions that are accessible
>>> +only to previously compiled binaries
>>> +
>>> +The addition of a parameter to the function is ABI breaking as the
>>> +function is public, and existing application may use it in its current
>>> +form.  However, the compatibility macros in DPDK allow a developer to
>>> +use symbol versioning so that multiple functions can be mapped to the
>>> +same public symbol based on when an application was linked to it.  To
>>> +see how this is done, we start with the requisite libraries version 
>>> map
>>> +file.  Initially the version map file for the acl library looks like
>>> +this
>>> +
>>> +.. code-block:: none
>>> +
>>> +   DPDK_2.0 {
>>> +        global:
>>> +
>>> +        rte_acl_add_rules;
>>> +        rte_acl_build;
>>> +        rte_acl_classify;
>>> +        rte_acl_classify_alg;
>>> +        rte_acl_classify_scalar;
>>> +        rte_acl_create;
>>> +        rte_acl_dump;
>>> +        rte_acl_find_existing;
>>> +        rte_acl_free;
>>> +        rte_acl_ipv4vlan_add_rules;
>>> +        rte_acl_ipv4vlan_build;
>>> +        rte_acl_list_dump;
>>> +        rte_acl_reset;
>>> +        rte_acl_reset_rules;
>>> +        rte_acl_set_ctx_classify;
>>> +
>>> +        local: *;
>>> +   };
>>> +
>>> +This file needs to be modified as follows
>>> +
>>> +.. code-block:: none
>>> +
>>> +   DPDK_2.0 {
>>> +        global:
>>> +
>>> +        rte_acl_add_rules;
>>> +        rte_acl_build;
>>> +        rte_acl_classify;
>>> +        rte_acl_classify_alg;
>>> +        rte_acl_classify_scalar;
>>> +        rte_acl_create;
>>> +        rte_acl_dump;
>>> +        rte_acl_find_existing;
>>> +        rte_acl_free;
>>> +        rte_acl_ipv4vlan_add_rules;
>>> +        rte_acl_ipv4vlan_build;
>>> +        rte_acl_list_dump;
>>> +        rte_acl_reset;
>>> +        rte_acl_reset_rules;
>>> +        rte_acl_set_ctx_classify;
>>> +
>>> +        local: *;
>>> +   };
>>> +
>>> +   DPDK_2.1 {
>>> +        global:
>>> +        rte_acl_create;
>> One question, does it need a line of "local: *;", like it did in
>> librte_ether/rte_ether_version.map?
> No, it does not. You only need to specify 'local' in the default/base 
> node, which
> in this case/example is DPDK_2.0.
>
> Sergio
Just to be clear, as I think I may have misused the term 'default' here, 
it is recommended
to just specify 'local: *;' in just one node (it could confuse the 
linker) and it doesn't really
matter which one.

Quoting http://www.akkadia.org/drepper/symbol-versioning:
"It makes no sense at all to associate versions with symbols which are 
not exported. Therefore the `local:' sections of all but the base 
version are empty and the `local:' section of the base version simply 
contains `*'. This will match all symbols which are not explicitly 
mentioned in any `global:' list."

Sergio