DPDK patches and discussions
 help / color / Atom feed
From: Neil Horman <nhorman@tuxdriver.com>
To: Ferruh Yigit <ferruh.yigit@intel.com>
Cc: "Ananyev, Konstantin" <konstantin.ananyev@intel.com>,
	Thomas Monjalon <thomas@monjalon.net>,
	Akhil Goyal <akhil.goyal@nxp.com>,
	"Trahe, Fiona" <fiona.trahe@intel.com>,
	David Marchand <david.marchand@redhat.com>,
	Anoob Joseph <anoobj@marvell.com>,
	"Kusztal, ArkadiuszX" <arkadiuszx.kusztal@intel.com>,
	"dev@dpdk.org" <dev@dpdk.org>,
	"Richardson, Bruce" <bruce.richardson@intel.com>,
	"Mcnamara, John" <john.mcnamara@intel.com>,
	"dodji@seketeli.net" <dodji@seketeli.net>,
	Andrew Rybchenko <arybchenko@solarflare.com>,
	"aconole@redhat.com" <aconole@redhat.com>
Subject: Re: [dpdk-dev] [PATCH v2 4/4] add ABI checks
Date: Mon, 3 Feb 2020 06:50:34 -0500
Message-ID: <20200203115034.GA25978@hmswarspite.think-freely.org> (raw)
In-Reply-To: <666f2cc7-0906-7a07-a582-87800f321a00@intel.com>

On Mon, Feb 03, 2020 at 09:30:06AM +0000, Ferruh Yigit wrote:
> On 2/2/2020 2:41 PM, Ananyev, Konstantin wrote:
> > 
> >> 31/01/2020 15:16, Trahe, Fiona:
> >>> On 1/30/2020 8:18 PM, Thomas Monjalon wrote:
> >>>> 30/01/2020 17:09, Ferruh Yigit:
> >>>>> On 1/29/2020 8:13 PM, Akhil Goyal wrote:
> >>>>>>
> >>>>>> I believe these enums will be used only in case of ASYM case which is experimental.
> >>>>>
> >>>>> Independent from being experiment and not, this shouldn't be a problem, I think
> >>>>> this is a false positive.
> >>>>>
> >>>>> The ABI break can happen when a struct has been shared between the application
> >>>>> and the library (DPDK) and the layout of that memory know differently by
> >>>>> application and the library.
> >>>>>
> >>>>> Here in all cases, there is no layout/size change.
> >>>>>
> >>>>> As to the value changes of the enums, since application compiled with old DPDK,
> >>>>> it will know only up to '6', 7 and more means invalid to the application. So it
> >>>>> won't send these values also it should ignore these values from library. Only
> >>>>> consequence is old application won't able to use new features those new enums
> >>>>> provide but that is expected/normal.
> >>>>
> >>>> If library give higher value than expected by the application,
> >>>> if the application uses this value as array index,
> >>>> there can be an access out of bounds.
> >>>
> >>> [Fiona] All asymmetric APIs are experimental so above shouldn't be a problem.
> >>> But for the same issue with sym crypto below, I believe Ferruh's explanation makes
> >>> sense and I don't see how there can be an API breakage.
> >>> So if an application hasn't compiled against the new lib it will be still using the old value
> >>> which will be within bounds. If it's picking up the higher new value from the lib it must
> >>> have been compiled against the lib so shouldn't have problems.
> >>
> >> You say there is no ABI issue because the application will be re-compiled
> >> for the updated library. Indeed, compilation fixes compatibility issues.
> >> But this is not relevant for ABI compatibility.
> >> ABI compatibility means we can upgrade the library without recompiling
> >> the application and it must work.
> >> You think it is a false positive because you assume the application
> >> "picks" the new value. I think you miss the case where the new value
> >> is returned by a function in the upgraded library.
> >>
> >>> There are also no structs on the API which contain arrays using this
> >>> for sizing, so I don't see an opportunity for an appl to have a
> >>> mismatch in memory addresses.
> >>
> >> Let me demonstrate where the API may "use" the new value
> >> RTE_CRYPTO_AEAD_CHACHA20_POLY1305 and how it impacts the application.
> >>
> >> Once upon a time a DPDK application counting the number of devices
> >> supporting each AEAD algo (in order to find the best supported algo).
> >> It is done in an array indexed by algo id:
> >> int aead_dev_count[RTE_CRYPTO_AEAD_LIST_END];
> >> The application is compiled with DPDK 19.11,
> >> where RTE_CRYPTO_AEAD_LIST_END = 3.
> >> So the size of the application array aead_dev_count is 3.
> >> This binary is run with DPDK 20.02,
> >> where RTE_CRYPTO_AEAD_CHACHA20_POLY1305 = 3.
> >> When calling rte_cryptodev_info_get() on a device QAT_GEN3,
> >> rte_cryptodev_info.capabilities.sym.aead.algo is set to
> >> RTE_CRYPTO_AEAD_CHACHA20_POLY1305 (= 3).
> >> The application uses this value:
> >> ++ aead_dev_count[info.capabilities.sym.aead.algo];
> >> The application is crashing because of out of bound access.
> > 
> > I'd say this is an example of bad written app.
> > It probably should check that returned by library value doesn't
> > exceed its internal array size.
> 
> +1
> 
> Application should ignore values >= MAX.
> 
The example is still somewhat valid in it general principle though.  While
extending an ennumeration may be flagged by libabigail as an ABI breakage, its
not necessecarily a false positive.  By extending the ennumeration, all the
previous entries in an array defined by said ennumeration remain constant in
their offsets, so you can 'get away with such a change' in terms of preserving
backwards compatibility in the above example, but you cannot, for example,
shuffle the values in the ennumeration, as doing so would cause a functional
breakage (i.e. requesting an instance of RTE_CRYPTO_AEAD_CHACHA20_POLY1305 might
instead give you an instance of RTE_CRYPTO_AEAD_AES_GCM.  

These sorts of changes are the type that we could collectively waive in terms of
ABI checking, as they should be ok, but the errors from libabigail should be
taken as an indicator that this API could be rewritten (for example by removing
the abi entirely, and adding an API call that returns an array of instance name
and ids), so that changes of the above sort arent required.


> Do you suggest we don't extend any enum or define between ABI breakage releases
> to be sure bad written applications not affected?
> 
As noted above, we could waive such corner cases, and probably be fine, but the
error from the ABI check still serves a valid purpose in that its an indicator
that your library API is ABI sensitive to code changes that re-architecture may
address

Neil


  reply index

Thread overview: 104+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2019-12-20 15:20 [dpdk-dev] [PATCH] " David Marchand
2019-12-20 15:32 ` Richardson, Bruce
2019-12-20 16:20   ` Kinsella, Ray
2019-12-20 21:00     ` Thomas Monjalon
2020-01-06 13:17       ` Aaron Conole
2020-01-15 13:07         ` Burakov, Anatoly
2020-01-14 23:19     ` Thomas Monjalon
2020-01-15 11:33       ` Neil Horman
2020-01-15 12:38         ` Thomas Monjalon
2020-01-16 11:52           ` Neil Horman
2020-01-16 14:20             ` Thomas Monjalon
2020-01-16 18:49               ` Neil Horman
2020-01-16 20:01                 ` Thomas Monjalon
2020-01-17 19:01                   ` Neil Horman
2020-01-17 21:26                     ` David Marchand
2019-12-20 20:25 ` Neil Horman
2020-01-29 17:26 ` [dpdk-dev] [PATCH v2 0/4] " David Marchand
2020-01-29 17:26   ` [dpdk-dev] [PATCH v2 1/4] hash: fix meson headers packaging David Marchand
2020-01-30 10:12     ` Luca Boccassi
2020-01-30 10:54       ` David Marchand
2020-01-30 10:56         ` Luca Boccassi
2020-01-29 17:26   ` [dpdk-dev] [PATCH v2 2/4] build: split build helper David Marchand
2020-01-29 17:26   ` [dpdk-dev] [PATCH v2 3/4] build: test meson installation David Marchand
2020-01-29 17:26   ` [dpdk-dev] [PATCH v2 4/4] add ABI checks David Marchand
2020-01-29 17:42     ` Thomas Monjalon
2020-01-29 18:10       ` Anoob Joseph
2020-01-29 20:03         ` David Marchand
2020-01-29 20:13           ` Akhil Goyal
2020-01-30 16:09             ` Ferruh Yigit
2020-01-30 20:18               ` Thomas Monjalon
2020-01-31  9:03                 ` Ferruh Yigit
2020-01-31 10:26                   ` Ananyev, Konstantin
2020-01-31 14:16                 ` Trahe, Fiona
2020-02-02 13:05                   ` Thomas Monjalon
2020-02-02 14:41                     ` Ananyev, Konstantin
2020-02-03  9:30                       ` Ferruh Yigit
2020-02-03 11:50                         ` Neil Horman [this message]
2020-02-03 13:09                           ` Ferruh Yigit
2020-02-03 14:00                             ` Dodji Seketeli
2020-02-03 14:46                               ` Ferruh Yigit
2020-02-03 15:08                             ` Trahe, Fiona
2020-02-03 17:09                         ` Thomas Monjalon
2020-02-03 17:34                           ` Thomas Monjalon
2020-02-03 18:55                             ` Ray Kinsella
2020-02-03 21:07                               ` Thomas Monjalon
2020-02-04  9:46                                 ` Ferruh Yigit
2020-02-04 10:24                                 ` Thomas Monjalon
2020-02-04 12:44                                   ` Trahe, Fiona
2020-02-04 15:52                                     ` Trahe, Fiona
2020-02-04 15:59                                       ` Thomas Monjalon
2020-02-04 17:46                                         ` Trahe, Fiona
2020-02-13 14:51                                           ` Kusztal, ArkadiuszX
2020-03-16 12:57                                             ` Trahe, Fiona
2020-03-16 13:09                                               ` Thomas Monjalon
2020-03-17 13:27                                                 ` Kusztal, ArkadiuszX
2020-03-17 15:10                                                   ` Thomas Monjalon
2020-03-17 19:10                                                     ` Kusztal, ArkadiuszX
2020-02-04 12:57                                   ` Kevin Traynor
2020-02-04 14:44                                   ` Aaron Conole
2020-02-04 19:49                                     ` Neil Horman
2020-02-04  9:51                               ` David Marchand
2020-02-04 10:10                                 ` Trahe, Fiona
2020-02-04 10:38                                   ` Thomas Monjalon
2020-02-05 11:10                                 ` Ray Kinsella
2020-02-03 17:40                           ` Ferruh Yigit
2020-02-03 18:40                             ` Thomas Monjalon
2020-02-04  9:19                               ` Ferruh Yigit
2020-02-04  9:45                                 ` Thomas Monjalon
2020-02-04  9:56                                   ` Ferruh Yigit
2020-02-04 10:08                                     ` Bruce Richardson
2020-02-04 10:17                                     ` Kevin Traynor
2020-02-04 10:16                             ` Akhil Goyal
2020-02-04 10:28                               ` Thomas Monjalon
2020-02-04 10:32                                 ` Akhil Goyal
2020-02-04 11:35                                   ` Bruce Richardson
2020-02-04 22:10                                   ` Neil Horman
2020-02-05  6:16                                     ` [dpdk-dev] [EXT] " Anoob Joseph
2020-02-05 14:33                                       ` Trahe, Fiona
2020-02-04 21:59                               ` [dpdk-dev] " Neil Horman
2020-01-30 13:06         ` Trahe, Fiona
2020-01-30 15:59           ` Thomas Monjalon
2020-01-30 16:42             ` Ferruh Yigit
2020-01-30 23:49             ` Ananyev, Konstantin
2020-01-31  9:07               ` Ferruh Yigit
2020-01-31  9:37                 ` Ananyev, Konstantin
2020-01-30 10:57   ` [dpdk-dev] [PATCH v2 0/4] " Luca Boccassi
2020-01-30 16:00 ` [dpdk-dev] [PATCH v3 " David Marchand
2020-01-30 16:00   ` [dpdk-dev] [PATCH v3 1/4] hash: fix meson headers packaging David Marchand
2020-01-30 18:01     ` Wang, Yipeng1
2020-01-30 18:40       ` Honnappa Nagarahalli
2020-02-05 19:51         ` Wang, Yipeng1
2020-01-30 16:00   ` [dpdk-dev] [PATCH v3 2/4] build: split build helper David Marchand
2020-01-30 16:00   ` [dpdk-dev] [PATCH v3 3/4] build: test meson installation David Marchand
2020-01-30 22:17     ` Thomas Monjalon
2020-01-30 16:00   ` [dpdk-dev] [PATCH v3 4/4] add ABI checks David Marchand
2020-01-30 22:32     ` Thomas Monjalon
2020-02-01 15:29       ` David Marchand
2020-01-30 22:44     ` Thomas Monjalon
2020-02-02 21:08 ` [dpdk-dev] [PATCH v4 0/3] " David Marchand
2020-02-02 21:08   ` [dpdk-dev] [PATCH v4 1/3] hash: fix meson headers packaging David Marchand
2020-02-05 19:53     ` Wang, Yipeng1
2020-02-02 21:08   ` [dpdk-dev] [PATCH v4 2/3] build: split build helper David Marchand
2020-02-02 21:08   ` [dpdk-dev] [PATCH v4 3/3] add ABI checks David Marchand
2020-02-05 14:13   ` [dpdk-dev] [PATCH v4 0/3] " Thomas Monjalon

Reply instructions:

You may reply publically to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=20200203115034.GA25978@hmswarspite.think-freely.org \
    --to=nhorman@tuxdriver.com \
    --cc=aconole@redhat.com \
    --cc=akhil.goyal@nxp.com \
    --cc=anoobj@marvell.com \
    --cc=arkadiuszx.kusztal@intel.com \
    --cc=arybchenko@solarflare.com \
    --cc=bruce.richardson@intel.com \
    --cc=david.marchand@redhat.com \
    --cc=dev@dpdk.org \
    --cc=dodji@seketeli.net \
    --cc=ferruh.yigit@intel.com \
    --cc=fiona.trahe@intel.com \
    --cc=john.mcnamara@intel.com \
    --cc=konstantin.ananyev@intel.com \
    --cc=thomas@monjalon.net \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link

DPDK patches and discussions

Archives are clonable:
	git clone --mirror http://inbox.dpdk.org/dev/0 dev/git/0.git

	# If you have public-inbox 1.1+ installed, you may
	# initialize and index your mirror using the following commands:
	public-inbox-init -V2 dev dev/ http://inbox.dpdk.org/dev \
		dev@dpdk.org
	public-inbox-index dev


Newsgroup available over NNTP:
	nntp://inbox.dpdk.org/inbox.dpdk.dev


AGPL code for this site: git clone https://public-inbox.org/ public-inbox