From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from smtp.tuxdriver.com (charlotte.tuxdriver.com [70.61.120.58]) by dpdk.org (Postfix) with ESMTP id A0102C584 for ; Thu, 25 Jun 2015 14:25:48 +0200 (CEST) Received: from hmsreliant.think-freely.org ([2001:470:8:a08:7aac:c0ff:fec2:933b] helo=localhost) by smtp.tuxdriver.com with esmtpsa (TLSv1:AES128-SHA:128) (Exim 4.63) (envelope-from ) id 1Z86Du-0000WR-LT; Thu, 25 Jun 2015 08:25:47 -0400 Date: Thu, 25 Jun 2015 08:25:35 -0400 From: Neil Horman To: "Zhang, Helin" Message-ID: <20150625122535.GB17026@hmsreliant.think-freely.org> References: <1435088014-18973-1-git-send-email-nhorman@tuxdriver.com> <1435170885-17643-1-git-send-email-nhorman@tuxdriver.com> <1435170885-17643-2-git-send-email-nhorman@tuxdriver.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: User-Agent: Mutt/1.5.23 (2014-03-12) X-Spam-Score: -2.9 (--) X-Spam-Status: No Cc: "dev@dpdk.org" Subject: Re: [dpdk-dev] [PATCHv2 2/2] ABI: Add some documentation X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: patches and discussions about DPDK List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 25 Jun 2015 12:25:49 -0000 On Thu, Jun 25, 2015 at 07:19:49AM +0000, Zhang, Helin wrote: > > > > -----Original Message----- > > From: dev [mailto:dev-bounces@dpdk.org] On Behalf Of Neil Horman > > Sent: Thursday, June 25, 2015 2:35 AM > > To: dev@dpdk.org > > Subject: [dpdk-dev] [PATCHv2 2/2] ABI: Add some documentation > > > > People have been asking for ways to use the ABI macros, heres some docs to > > clarify their use. Included is: > > > > * An overview of what ABI is > > * Details of the ABI deprecation process > > * Details of the versioning macros > > * Examples of their use > > * Details of how to use the ABI validator > > > > Thanks to John Mcnamara, who duplicated much of this effort at Intel while I was > > working on it. Much of the introductory material was gathered and cleaned up > > by him > > > > Signed-off-by: Neil Horman > > CC: john.mcnamara@intel.com > > CC: thomas.monjalon@6wind.com > > > > Change notes: > > > > v2) > > * Fixed RST indentations and spelling errors > > * Rebased to upstream to fix index.rst conflict > > --- > > doc/guides/guidelines/index.rst | 1 + > > doc/guides/guidelines/versioning.rst | 456 > > +++++++++++++++++++++++++++++++++++ > > 2 files changed, 457 insertions(+) > > create mode 100644 doc/guides/guidelines/versioning.rst > > > > diff --git a/doc/guides/guidelines/index.rst b/doc/guides/guidelines/index.rst > > index 0ee9ab3..bfb9fa3 100644 > > --- a/doc/guides/guidelines/index.rst > > +++ b/doc/guides/guidelines/index.rst > > @@ -7,3 +7,4 @@ Guidelines > > > > coding_style > > design > > + versioning > > diff --git a/doc/guides/guidelines/versioning.rst > > b/doc/guides/guidelines/versioning.rst > > new file mode 100644 > > index 0000000..2aef526 > > --- /dev/null > > +++ b/doc/guides/guidelines/versioning.rst > > @@ -0,0 +1,456 @@ > > +Managing ABI updates > > +==================== > > + > > +Description > > +----------- > > + > > +This document details some methods for handling ABI management in the > > DPDK. > > +Note this document is not exhaustive, in that C library versioning is > > +flexible allowing multiple methods to achieve various goals, but it > > +will provide the user with some introductory methods > > + > > +General Guidelines > > +------------------ > > + > > +#. Whenever possible, ABI should be preserved #. The addition of > > +symbols is generally not problematic #. The modification of symbols can > > +generally be managed with versioning #. The removal of symbols > > +generally is an ABI break and requires bumping of the > > + LIBABIVER macro > > + > > +What is an ABI > > +-------------- > > + > > +An ABI (Application Binary Interface) is the set of runtime interfaces > > +exposed by a library. It is similar to an API (Application Programming > > +Interface) but is the result of compilation. It is also effectively > > +cloned when applications link to dynamic libraries. That is to say > > +when an application is compiled to link against dynamic libraries, it > > +is assumed that the ABI remains constant between the time the application is > > compiled/linked, and the time that it runs. > > +Therefore, in the case of dynamic linking, it is critical that an ABI > > +is preserved, or (when modified), done in such a way that the > > +application is unable to behave improperly or in an unexpected fashion. > > + > > +The DPDK ABI policy > > +------------------- > > + > > +ABI versions are set at the time of major release labeling, and the ABI > > +may change multiple times, without warning, between the last release > > +label and the HEAD label of the git tree. > > + > > +ABI versions, once released, are available until such time as their > > +deprecation has been noted in the Release Notes for at least one major > > +release cycle. For example consider the case where the ABI for DPDK 2.0 > > +has been shipped and then a decision is made to modify it during the > > +development of DPDK 2.1. The decision will be recorded in the Release > > +Notes for the DPDK 2.1 release and the modification will be made available in > > the DPDK 2.2 release. > > + > > +ABI versions may be deprecated in whole or in part as needed by a given > > +update. > > + > > +Some ABI changes may be too significant to reasonably maintain multiple > > +versions. In those cases ABI's may be updated without backward > > +compatibility being provided. The requirements for doing so are: > > + > > +#. At least 3 acknowledgments of the need to do so must be made on the > > + dpdk.org mailing list. > > + > > +#. A full deprecation cycle, as explained above, must be made to offer > > + downstream consumers sufficient warning of the change. > > + > > +#. The ``LIBABIVER`` variable in the makefile(s) where the ABI changes are > > + incorporated must be incremented in parallel with the ABI changes > > + themselves. > > + > > +Note that the above process for ABI deprecation should not be > > +undertaken lightly. ABI stability is extremely important for downstream > > +consumers of the DPDK, especially when distributed in shared object > > +form. Every effort should be made to preserve the ABI whenever > > +possible. The ABI should only be changed for significant reasons, such > > +as performance enhancements. ABI breakage due to changes such as > > +reorganizing public structure fields for aesthetic or readability purposes should > > be avoided. > > + > > +Examples of Deprecation Notices > > +------------------------------- > > + > > +The following are some examples of ABI deprecation notices which would > > +be added to the Release Notes: > > + > > +* The Macro ``#RTE_FOO`` is deprecated and will be removed with version > > +2.0, > > + to be replaced with the inline function ``rte_foo()``. > > + > > +* The function ``rte_mbuf_grok()`` has been updated to include a new > > +parameter > > + in version 2.0. Backwards compatibility will be maintained for this > > +function > > + until the release of version 2.1 > > + > > +* The members of ``struct rte_foo`` have been reorganized in release > > +2.0 for > > + performance reasons. Existing binary applications will have backwards > > + compatibility in release 2.0, while newly built binaries will need to > > + reference the new structure variant ``struct rte_foo2``. > > +Compatibility will > > + be removed in release 2.2, and all applications will require updating > > +and > > + rebuilding to the new structure at that time, which will be renamed > > +to the > > + original ``struct rte_foo``. > > + > > +* Significant ABI changes are planned for the ``librte_dostuff`` > > +library. The > > + upcoming release 2.0 will not contain these changes, but release 2.1 > > +will, > > + and no backwards compatibility is planned due to the extensive nature > > +of > > + these changes. Binaries using this library built prior to version 2.1 > > +will > > + require updating and recompilation. > > + > > +Versioning Macros > > +----------------- > > + > > +When a symbol is exported from a library to provide an API, it also > > +provides a calling convention (ABI) that is embodied in its name, > > +return type and arguments. Occasionally that function may need to > > +change to accommodate new functionality or behavior. When that occurs, > > +it is desirable to allow for backward compatibility for a time with > > +older binaries that are dynamically linked to the DPDK. > > + > > +To support backward compatibility the > > +``lib/librte_compat/rte_compat.h`` > > +header file provides macros to use when updating exported functions. > > +These macros are used in conjunction with the > > +``rte__version.map`` file for a given library to allow > > +multiple versions of a symbol to exist in a shared library so that older binaries > > need not be immediately recompiled. > > + > > +The macros exported are: > > + > > +* ``VERSION_SYMBOL(b, e, n)``: Creates a symbol version table entry > > +binding > > + unversioned symbol ``b`` to the internal function ``b_e``. > > + > > + > > +* ``BASE_SYMBOL(b, e)``: Creates a symbol version table entry binding > > + unversioned symbol ``b`` to the internal function ``b_e``. > > + > > +* ``BIND_DEFAULT_SYMBOL(b, e, n)``: Creates a symbol version entry > > +instructing > > + the linker to bind references to symbol ``b`` to the internal symbol > > + ``b_e``. > > + > > + > > +Examples of ABI Macro use > > +------------------------- > > + > > +Updating a public API > > +~~~~~~~~~~~~~~~~~~~~~ > > + > > +Assume we have a function as follows > > + > > +.. code-block:: c > > + > > + /* > > + * Create an acl context object for apps to > > + * manipulate > > + */ > > + struct rte_acl_ctx * > > + rte_acl_create(const struct rte_acl_param *param) { > > + ... > > + } > > + > > + > > +Assume that struct rte_acl_ctx is a private structure, and that a > > +developer wishes to enhance the acl api so that a debugging flag can be > > +enabled on a per-context basis. This requires an addition to the > > +structure (which, being private, is safe), but it also requires > > +modifying the code as follows > > + > > +.. code-block:: c > > + > > + /* > > + * Create an acl context object for apps to > > + * manipulate > > + */ > > + struct rte_acl_ctx * > > + rte_acl_create(const struct rte_acl_param *param, int debug) { > > + ... > > + } > > + > > + > > +Note also that, being a public function, the header file prototype must > > +also be changed, as must all the call sites, to reflect the new ABI > > +footprint. We will maintain previous ABI versions that are accessible > > +only to previously compiled binaries > > + > > +The addition of a parameter to the function is ABI breaking as the > > +function is public, and existing application may use it in its current > > +form. However, the compatibility macros in DPDK allow a developer to > > +use symbol versioning so that multiple functions can be mapped to the > > +same public symbol based on when an application was linked to it. To > > +see how this is done, we start with the requisite libraries version map > > +file. Initially the version map file for the acl library looks like > > +this > > + > > +.. code-block:: none > > + > > + DPDK_2.0 { > > + global: > > + > > + rte_acl_add_rules; > > + rte_acl_build; > > + rte_acl_classify; > > + rte_acl_classify_alg; > > + rte_acl_classify_scalar; > > + rte_acl_create; > > + rte_acl_dump; > > + rte_acl_find_existing; > > + rte_acl_free; > > + rte_acl_ipv4vlan_add_rules; > > + rte_acl_ipv4vlan_build; > > + rte_acl_list_dump; > > + rte_acl_reset; > > + rte_acl_reset_rules; > > + rte_acl_set_ctx_classify; > > + > > + local: *; > > + }; > > + > > +This file needs to be modified as follows > > + > > +.. code-block:: none > > + > > + DPDK_2.0 { > > + global: > > + > > + rte_acl_add_rules; > > + rte_acl_build; > > + rte_acl_classify; > > + rte_acl_classify_alg; > > + rte_acl_classify_scalar; > > + rte_acl_create; > > + rte_acl_dump; > > + rte_acl_find_existing; > > + rte_acl_free; > > + rte_acl_ipv4vlan_add_rules; > > + rte_acl_ipv4vlan_build; > > + rte_acl_list_dump; > > + rte_acl_reset; > > + rte_acl_reset_rules; > > + rte_acl_set_ctx_classify; > > + > > + local: *; > > + }; > > + > > + DPDK_2.1 { > > + global: > > + rte_acl_create; > One question, does it need a line of "local: *;", like it did in > librte_ether/rte_ether_version.map? > It shouldn't. local just specifies that any symbol not already declared global be unpublished in the ELF file (i.e. not global). You can declare it again in the next version node, but since the 2.1 node inherits from the 2.0 node, its implied.