Re: [dpdk-dev] [PATCH 0/2] Add ABI Version Testing to app/test

[Bruce Richardson <bruce.richardson@intel.com>]

On Tue, May 28, 2019 at 12:51:56PM +0100, Ray Kinsella wrote:
> This patchset adds ABI Version Testing functionality to the app/test unit
> test framework.
> 
> The patchset is intended to address two issues previously raised during ML
> conversations on ABI Stability;
> 1. How do we unit test still supported previous ABI Versions.
> 2. How to we unit test inline functions from still supported previous ABI
> Versions.
> 
> The more obvious way to achieve both of the above is to simply archive
> pre-built binaries compiled against previous versions of DPDK for use unit
> testing previous ABI Versions, and while this should still be done as an
> additional check, this approach does not scale well, must every DPDK
> developer have a local copy of these binaries to test with, before
> upstreaming changes?
> 
> Instead starting with rte_lpm, I did the following:-
> 
> * I reproduced mostly unmodified unit tests from previous ABI Versions,
>   in this case v2.0 and v16.04
> * I reproduced the rte_lpm interface header from these previous ABI
>   Versions,including the inline functions and remapping symbols to
>   appropriate versions.
> * I added support for multiple abi versions to the app/test unit test
>   framework to allow users to switch between abi versions (set_abi_version),
>   without further polluting the already long list of unit tests available in
>   app/test.
> 
> The intention here is that, in future as developers need to depreciate
> APIs, their associated unit tests may move into the ABI Version testing
> mechanism of the app/test instead of being replaced by the latest set of
> unit tests as would be the case today.
> 
> ToDo:
> * Refactor the v2.0 and v16.04 unit tests to separate functional and
>   performance test cases.
> * Add support for trigger ABI Version unit tests from the app/test command
>   line.
> 
While I admire the goal, given the amount of code that seems to be involved
here, I'm not sure if the "test" binary is the place to put this. I think
it might be better as a separate ABI compatiblity test app.

A separate question is whether this is really necessary to ensure ABI
compatibility? Do other projects do this? Is the info from the abi
compatiblity checker script already in DPDK, or from other
already-available tools not sufficient?

/Bruce