From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mga11.intel.com (mga11.intel.com [192.55.52.93]) by dpdk.org (Postfix) with ESMTP id 49A7CC560 for ; Thu, 25 Jun 2015 10:00:27 +0200 (CEST) Received: from fmsmga002.fm.intel.com ([10.253.24.26]) by fmsmga102.fm.intel.com with ESMTP; 25 Jun 2015 01:00:26 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.13,675,1427785200"; d="scan'208";a="750128482" Received: from smonroyx-mobl.ger.corp.intel.com (HELO [10.237.221.15]) ([10.237.221.15]) by fmsmga002.fm.intel.com with ESMTP; 25 Jun 2015 01:00:24 -0700 Message-ID: <558BB518.20104@intel.com> Date: Thu, 25 Jun 2015 09:00:24 +0100 From: "Gonzalez Monroy, Sergio" User-Agent: Mozilla/5.0 (Windows NT 6.3; WOW64; rv:31.0) Gecko/20100101 Thunderbird/31.7.0 MIME-Version: 1.0 To: "Zhang, Helin" , Neil Horman , "dev@dpdk.org" References: <1435088014-18973-1-git-send-email-nhorman@tuxdriver.com> <1435170885-17643-1-git-send-email-nhorman@tuxdriver.com> <1435170885-17643-2-git-send-email-nhorman@tuxdriver.com> <558BB0FD.9040803@intel.com> In-Reply-To: <558BB0FD.9040803@intel.com> Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 7bit Subject: Re: [dpdk-dev] [PATCHv2 2/2] ABI: Add some documentation X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: patches and discussions about DPDK List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 25 Jun 2015 08:00:28 -0000 On 25/06/2015 08:42, Gonzalez Monroy, Sergio wrote: > On 25/06/2015 08:19, Zhang, Helin wrote: >> >>> -----Original Message----- >>> From: dev [mailto:dev-bounces@dpdk.org] On Behalf Of Neil Horman >>> Sent: Thursday, June 25, 2015 2:35 AM >>> To: dev@dpdk.org >>> Subject: [dpdk-dev] [PATCHv2 2/2] ABI: Add some documentation >>> >>> People have been asking for ways to use the ABI macros, heres some >>> docs to >>> clarify their use. Included is: >>> >>> * An overview of what ABI is >>> * Details of the ABI deprecation process >>> * Details of the versioning macros >>> * Examples of their use >>> * Details of how to use the ABI validator >>> >>> Thanks to John Mcnamara, who duplicated much of this effort at Intel >>> while I was >>> working on it. Much of the introductory material was gathered and >>> cleaned up >>> by him >>> >>> Signed-off-by: Neil Horman >>> CC: john.mcnamara@intel.com >>> CC: thomas.monjalon@6wind.com >>> >>> Change notes: >>> >>> v2) >>> * Fixed RST indentations and spelling errors >>> * Rebased to upstream to fix index.rst conflict >>> --- >>> doc/guides/guidelines/index.rst | 1 + >>> doc/guides/guidelines/versioning.rst | 456 >>> +++++++++++++++++++++++++++++++++++ >>> 2 files changed, 457 insertions(+) >>> create mode 100644 doc/guides/guidelines/versioning.rst >>> >>> diff --git a/doc/guides/guidelines/index.rst >>> b/doc/guides/guidelines/index.rst >>> index 0ee9ab3..bfb9fa3 100644 >>> --- a/doc/guides/guidelines/index.rst >>> +++ b/doc/guides/guidelines/index.rst >>> @@ -7,3 +7,4 @@ Guidelines >>> >>> coding_style >>> design >>> + versioning >>> diff --git a/doc/guides/guidelines/versioning.rst >>> b/doc/guides/guidelines/versioning.rst >>> new file mode 100644 >>> index 0000000..2aef526 >>> --- /dev/null >>> +++ b/doc/guides/guidelines/versioning.rst >>> @@ -0,0 +1,456 @@ >>> +Managing ABI updates >>> +==================== >>> + >>> +Description >>> +----------- >>> + >>> +This document details some methods for handling ABI management in the >>> DPDK. >>> +Note this document is not exhaustive, in that C library versioning is >>> +flexible allowing multiple methods to achieve various goals, but it >>> +will provide the user with some introductory methods >>> + >>> +General Guidelines >>> +------------------ >>> + >>> +#. Whenever possible, ABI should be preserved #. The addition of >>> +symbols is generally not problematic #. The modification of symbols >>> can >>> +generally be managed with versioning #. The removal of symbols >>> +generally is an ABI break and requires bumping of the >>> + LIBABIVER macro >>> + >>> +What is an ABI >>> +-------------- >>> + >>> +An ABI (Application Binary Interface) is the set of runtime interfaces >>> +exposed by a library. It is similar to an API (Application Programming >>> +Interface) but is the result of compilation. It is also effectively >>> +cloned when applications link to dynamic libraries. That is to say >>> +when an application is compiled to link against dynamic libraries, it >>> +is assumed that the ABI remains constant between the time the >>> application is >>> compiled/linked, and the time that it runs. >>> +Therefore, in the case of dynamic linking, it is critical that an ABI >>> +is preserved, or (when modified), done in such a way that the >>> +application is unable to behave improperly or in an unexpected >>> fashion. >>> + >>> +The DPDK ABI policy >>> +------------------- >>> + >>> +ABI versions are set at the time of major release labeling, and the >>> ABI >>> +may change multiple times, without warning, between the last release >>> +label and the HEAD label of the git tree. >>> + >>> +ABI versions, once released, are available until such time as their >>> +deprecation has been noted in the Release Notes for at least one major >>> +release cycle. For example consider the case where the ABI for DPDK >>> 2.0 >>> +has been shipped and then a decision is made to modify it during the >>> +development of DPDK 2.1. The decision will be recorded in the Release >>> +Notes for the DPDK 2.1 release and the modification will be made >>> available in >>> the DPDK 2.2 release. >>> + >>> +ABI versions may be deprecated in whole or in part as needed by a >>> given >>> +update. >>> + >>> +Some ABI changes may be too significant to reasonably maintain >>> multiple >>> +versions. In those cases ABI's may be updated without backward >>> +compatibility being provided. The requirements for doing so are: >>> + >>> +#. At least 3 acknowledgments of the need to do so must be made on the >>> + dpdk.org mailing list. >>> + >>> +#. A full deprecation cycle, as explained above, must be made to offer >>> + downstream consumers sufficient warning of the change. >>> + >>> +#. The ``LIBABIVER`` variable in the makefile(s) where the ABI >>> changes are >>> + incorporated must be incremented in parallel with the ABI changes >>> + themselves. >>> + >>> +Note that the above process for ABI deprecation should not be >>> +undertaken lightly. ABI stability is extremely important for >>> downstream >>> +consumers of the DPDK, especially when distributed in shared object >>> +form. Every effort should be made to preserve the ABI whenever >>> +possible. The ABI should only be changed for significant reasons, such >>> +as performance enhancements. ABI breakage due to changes such as >>> +reorganizing public structure fields for aesthetic or readability >>> purposes should >>> be avoided. >>> + >>> +Examples of Deprecation Notices >>> +------------------------------- >>> + >>> +The following are some examples of ABI deprecation notices which would >>> +be added to the Release Notes: >>> + >>> +* The Macro ``#RTE_FOO`` is deprecated and will be removed with >>> version >>> +2.0, >>> + to be replaced with the inline function ``rte_foo()``. >>> + >>> +* The function ``rte_mbuf_grok()`` has been updated to include a new >>> +parameter >>> + in version 2.0. Backwards compatibility will be maintained for this >>> +function >>> + until the release of version 2.1 >>> + >>> +* The members of ``struct rte_foo`` have been reorganized in release >>> +2.0 for >>> + performance reasons. Existing binary applications will have >>> backwards >>> + compatibility in release 2.0, while newly built binaries will >>> need to >>> + reference the new structure variant ``struct rte_foo2``. >>> +Compatibility will >>> + be removed in release 2.2, and all applications will require >>> updating >>> +and >>> + rebuilding to the new structure at that time, which will be renamed >>> +to the >>> + original ``struct rte_foo``. >>> + >>> +* Significant ABI changes are planned for the ``librte_dostuff`` >>> +library. The >>> + upcoming release 2.0 will not contain these changes, but release 2.1 >>> +will, >>> + and no backwards compatibility is planned due to the extensive >>> nature >>> +of >>> + these changes. Binaries using this library built prior to version >>> 2.1 >>> +will >>> + require updating and recompilation. >>> + >>> +Versioning Macros >>> +----------------- >>> + >>> +When a symbol is exported from a library to provide an API, it also >>> +provides a calling convention (ABI) that is embodied in its name, >>> +return type and arguments. Occasionally that function may need to >>> +change to accommodate new functionality or behavior. When that occurs, >>> +it is desirable to allow for backward compatibility for a time with >>> +older binaries that are dynamically linked to the DPDK. >>> + >>> +To support backward compatibility the >>> +``lib/librte_compat/rte_compat.h`` >>> +header file provides macros to use when updating exported functions. >>> +These macros are used in conjunction with the >>> +``rte__version.map`` file for a given library to allow >>> +multiple versions of a symbol to exist in a shared library so that >>> older binaries >>> need not be immediately recompiled. >>> + >>> +The macros exported are: >>> + >>> +* ``VERSION_SYMBOL(b, e, n)``: Creates a symbol version table entry >>> +binding >>> + unversioned symbol ``b`` to the internal function ``b_e``. >>> + >>> + >>> +* ``BASE_SYMBOL(b, e)``: Creates a symbol version table entry binding >>> + unversioned symbol ``b`` to the internal function ``b_e``. >>> + >>> +* ``BIND_DEFAULT_SYMBOL(b, e, n)``: Creates a symbol version entry >>> +instructing >>> + the linker to bind references to symbol ``b`` to the internal symbol >>> + ``b_e``. >>> + >>> + >>> +Examples of ABI Macro use >>> +------------------------- >>> + >>> +Updating a public API >>> +~~~~~~~~~~~~~~~~~~~~~ >>> + >>> +Assume we have a function as follows >>> + >>> +.. code-block:: c >>> + >>> + /* >>> + * Create an acl context object for apps to >>> + * manipulate >>> + */ >>> + struct rte_acl_ctx * >>> + rte_acl_create(const struct rte_acl_param *param) { >>> + ... >>> + } >>> + >>> + >>> +Assume that struct rte_acl_ctx is a private structure, and that a >>> +developer wishes to enhance the acl api so that a debugging flag >>> can be >>> +enabled on a per-context basis. This requires an addition to the >>> +structure (which, being private, is safe), but it also requires >>> +modifying the code as follows >>> + >>> +.. code-block:: c >>> + >>> + /* >>> + * Create an acl context object for apps to >>> + * manipulate >>> + */ >>> + struct rte_acl_ctx * >>> + rte_acl_create(const struct rte_acl_param *param, int debug) { >>> + ... >>> + } >>> + >>> + >>> +Note also that, being a public function, the header file prototype >>> must >>> +also be changed, as must all the call sites, to reflect the new ABI >>> +footprint. We will maintain previous ABI versions that are accessible >>> +only to previously compiled binaries >>> + >>> +The addition of a parameter to the function is ABI breaking as the >>> +function is public, and existing application may use it in its current >>> +form. However, the compatibility macros in DPDK allow a developer to >>> +use symbol versioning so that multiple functions can be mapped to the >>> +same public symbol based on when an application was linked to it. To >>> +see how this is done, we start with the requisite libraries version >>> map >>> +file. Initially the version map file for the acl library looks like >>> +this >>> + >>> +.. code-block:: none >>> + >>> + DPDK_2.0 { >>> + global: >>> + >>> + rte_acl_add_rules; >>> + rte_acl_build; >>> + rte_acl_classify; >>> + rte_acl_classify_alg; >>> + rte_acl_classify_scalar; >>> + rte_acl_create; >>> + rte_acl_dump; >>> + rte_acl_find_existing; >>> + rte_acl_free; >>> + rte_acl_ipv4vlan_add_rules; >>> + rte_acl_ipv4vlan_build; >>> + rte_acl_list_dump; >>> + rte_acl_reset; >>> + rte_acl_reset_rules; >>> + rte_acl_set_ctx_classify; >>> + >>> + local: *; >>> + }; >>> + >>> +This file needs to be modified as follows >>> + >>> +.. code-block:: none >>> + >>> + DPDK_2.0 { >>> + global: >>> + >>> + rte_acl_add_rules; >>> + rte_acl_build; >>> + rte_acl_classify; >>> + rte_acl_classify_alg; >>> + rte_acl_classify_scalar; >>> + rte_acl_create; >>> + rte_acl_dump; >>> + rte_acl_find_existing; >>> + rte_acl_free; >>> + rte_acl_ipv4vlan_add_rules; >>> + rte_acl_ipv4vlan_build; >>> + rte_acl_list_dump; >>> + rte_acl_reset; >>> + rte_acl_reset_rules; >>> + rte_acl_set_ctx_classify; >>> + >>> + local: *; >>> + }; >>> + >>> + DPDK_2.1 { >>> + global: >>> + rte_acl_create; >> One question, does it need a line of "local: *;", like it did in >> librte_ether/rte_ether_version.map? > No, it does not. You only need to specify 'local' in the default/base > node, which > in this case/example is DPDK_2.0. > > Sergio Just to be clear, as I think I may have misused the term 'default' here, it is recommended to just specify 'local: *;' in just one node (it could confuse the linker) and it doesn't really matter which one. Quoting http://www.akkadia.org/drepper/symbol-versioning: "It makes no sense at all to associate versions with symbols which are not exported. Therefore the `local:' sections of all but the base version are empty and the `local:' section of the base version simply contains `*'. This will match all symbols which are not explicitly mentioned in any `global:' list." Sergio