From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from dpdk.org (dpdk.org [92.243.14.124]) by inbox.dpdk.org (Postfix) with ESMTP id 98615A0530; Mon, 3 Feb 2020 18:40:29 +0100 (CET) Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id 0991E1BFB5; Mon, 3 Feb 2020 18:40:29 +0100 (CET) Received: from mga05.intel.com (mga05.intel.com [192.55.52.43]) by dpdk.org (Postfix) with ESMTP id 7C2B51BFB3 for ; Mon, 3 Feb 2020 18:40:26 +0100 (CET) X-Amp-Result: SKIPPED(no attachment in message) X-Amp-File-Uploaded: False Received: from orsmga002.jf.intel.com ([10.7.209.21]) by fmsmga105.fm.intel.com with ESMTP/TLS/DHE-RSA-AES256-GCM-SHA384; 03 Feb 2020 09:40:25 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.70,398,1574150400"; d="scan'208";a="249070247" Received: from fyigit-mobl.ger.corp.intel.com (HELO [10.237.221.61]) ([10.237.221.61]) by orsmga002.jf.intel.com with ESMTP; 03 Feb 2020 09:40:21 -0800 To: Thomas Monjalon , "Ananyev, Konstantin" , Akhil Goyal , "Trahe, Fiona" Cc: David Marchand , Anoob Joseph , "Kusztal, ArkadiuszX" , dev@dpdk.org, "Richardson, Bruce" , nhorman@tuxdriver.com, "Mcnamara, John" , dodji@seketeli.net, Andrew Rybchenko , aconole@redhat.com, bluca@debian.org, ktraynor@redhat.com References: <20191220152058.10739-1-david.marchand@redhat.com> <666f2cc7-0906-7a07-a582-87800f321a00@intel.com> <7566080.EvYhyI6sBW@xps> From: Ferruh Yigit Autocrypt: addr=ferruh.yigit@intel.com; prefer-encrypt=mutual; keydata= mQINBFXZCFABEADCujshBOAaqPZpwShdkzkyGpJ15lmxiSr3jVMqOtQS/sB3FYLT0/d3+bvy qbL9YnlbPyRvZfnP3pXiKwkRoR1RJwEo2BOf6hxdzTmLRtGtwWzI9MwrUPj6n/ldiD58VAGQ +iR1I/z9UBUN/ZMksElA2D7Jgg7vZ78iKwNnd+vLBD6I61kVrZ45Vjo3r+pPOByUBXOUlxp9 GWEKKIrJ4eogqkVNSixN16VYK7xR+5OUkBYUO+sE6etSxCr7BahMPKxH+XPlZZjKrxciaWQb +dElz3Ab4Opl+ZT/bK2huX+W+NJBEBVzjTkhjSTjcyRdxvS1gwWRuXqAml/sh+KQjPV1PPHF YK5LcqLkle+OKTCa82OvUb7cr+ALxATIZXQkgmn+zFT8UzSS3aiBBohg3BtbTIWy51jNlYdy ezUZ4UxKSsFuUTPt+JjHQBvF7WKbmNGS3fCid5Iag4tWOfZoqiCNzxApkVugltxoc6rG2TyX CmI2rP0mQ0GOsGXA3+3c1MCdQFzdIn/5tLBZyKy4F54UFo35eOX8/g7OaE+xrgY/4bZjpxC1 1pd66AAtKb3aNXpHvIfkVV6NYloo52H+FUE5ZDPNCGD0/btFGPWmWRmkPybzColTy7fmPaGz cBcEEqHK4T0aY4UJmE7Ylvg255Kz7s6wGZe6IR3N0cKNv++O7QARAQABtCVGZXJydWggWWln aXQgPGZlcnJ1aC55aWdpdEBpbnRlbC5jb20+iQJUBBMBCgA+AhsDAh4BAheABQsJCAcDBRUK CQgLBRYCAwEAFiEE0jZTh0IuwoTjmYHH+TPrQ98TYR8FAl1meboFCQlupOoACgkQ+TPrQ98T YR9ACBAAv2tomhyxY0Tp9Up7mNGLfEdBu/7joB/vIdqMRv63ojkwr9orQq5V16V/25+JEAD0 60cKodBDM6HdUvqLHatS8fooWRueSXHKYwJ3vxyB2tWDyZrLzLI1jxEvunGodoIzUOtum0Ce gPynnfQCelXBja0BwLXJMplM6TY1wXX22ap0ZViC0m714U5U4LQpzjabtFtjT8qOUR6L7hfy YQ72PBuktGb00UR/N5UrR6GqB0x4W41aZBHXfUQnvWIMmmCrRUJX36hOTYBzh+x86ULgg7H2 1499tA4o6rvE13FiGccplBNWCAIroAe/G11rdoN5NBgYVXu++38gTa/MBmIt6zRi6ch15oLA Ln2vHOdqhrgDuxjhMpG2bpNE36DG/V9WWyWdIRlz3NYPCDM/S3anbHlhjStXHOz1uHOnerXM 1jEjcsvmj1vSyYoQMyRcRJmBZLrekvgZeh7nJzbPHxtth8M7AoqiZ/o/BpYU+0xZ+J5/szWZ aYxxmIRu5ejFf+Wn9s5eXNHmyqxBidpCWvcbKYDBnkw2+Y9E5YTpL0mS0dCCOlrO7gca27ux ybtbj84aaW1g0CfIlUnOtHgMCmz6zPXThb+A8H8j3O6qmPoVqT3qnq3Uhy6GOoH8Fdu2Vchh TWiF5yo+pvUagQP6LpslffufSnu+RKAagkj7/RSuZV25Ag0EV9ZMvgEQAKc0Db17xNqtSwEv mfp4tkddwW9XA0tWWKtY4KUdd/jijYqc3fDD54ESYpV8QWj0xK4YM0dLxnDU2IYxjEshSB1T qAatVWz9WtBYvzalsyTqMKP3w34FciuL7orXP4AibPtrHuIXWQOBECcVZTTOdZYGAzaYzxiA ONzF9eTiwIqe9/oaOjTwTLnOarHt16QApTYQSnxDUQljeNvKYt1lZE/gAUUxNLWsYyTT+22/ vU0GDUahsJxs1+f1yEr+OGrFiEAmqrzpF0lCS3f/3HVTU6rS9cK3glVUeaTF4+1SK5ZNO35p iVQCwphmxa+dwTG/DvvHYCtgOZorTJ+OHfvCnSVjsM4kcXGjJPy3JZmUtyL9UxEbYlrffGPQ I3gLXIGD5AN5XdAXFCjjaID/KR1c9RHd7Oaw0Pdcq9UtMLgM1vdX8RlDuMGPrj5sQrRVbgYH fVU/TQCk1C9KhzOwg4Ap2T3tE1umY/DqrXQgsgH71PXFucVjOyHMYXXugLT8YQ0gcBPHy9mZ qw5mgOI5lCl6d4uCcUT0l/OEtPG/rA1lxz8ctdFBVOQOxCvwRG2QCgcJ/UTn5vlivul+cThi 6ERPvjqjblLncQtRg8izj2qgmwQkvfj+h7Ex88bI8iWtu5+I3K3LmNz/UxHBSWEmUnkg4fJl Rr7oItHsZ0ia6wWQ8lQnABEBAAGJAjwEGAEKACYCGwwWIQTSNlOHQi7ChOOZgcf5M+tD3xNh HwUCXWZ5wAUJB3FgggAKCRD5M+tD3xNhH2O+D/9OEz62YuJQLuIuOfL67eFTIB5/1+0j8Tsu o2psca1PUQ61SZJZOMl6VwNxpdvEaolVdrpnSxUF31kPEvR0Igy8HysQ11pj8AcgH0a9FrvU /8k2Roccd2ZIdpNLkirGFZR7LtRw41Kt1Jg+lafI0efkiHKMT/6D/P1EUp1RxOBNtWGV2hrd 0Yg9ds+VMphHHU69fDH02SwgpvXwG8Qm14Zi5WQ66R4CtTkHuYtA63sS17vMl8fDuTCtvfPF HzvdJLIhDYN3Mm1oMjKLlq4PUdYh68Fiwm+boJoBUFGuregJFlO3hM7uHBDhSEnXQr5mqpPM 6R/7Q5BjAxrwVBisH0yQGjsWlnysRWNfExAE2sRePSl0or9q19ddkRYltl6X4FDUXy2DTXa9 a+Fw4e1EvmcF3PjmTYs9IE3Vc64CRQXkhujcN4ZZh5lvOpU8WgyDxFq7bavFnSS6kx7Tk29/ wNJBp+cf9qsQxLbqhW5kfORuZGecus0TLcmpZEFKKjTJBK9gELRBB/zoN3j41hlEl7uTUXTI JQFLhpsFlEdKLujyvT/aCwP3XWT+B2uZDKrMAElF6ltpTxI53JYi22WO7NH7MR16Fhi4R6vh FHNBOkiAhUpoXRZXaCR6+X4qwA8CwHGqHRBfYFSU/Ulq1ZLR+S3hNj2mbnSx0lBs1eEqe2vh cA== Message-ID: <78e8ecf2-2239-897e-e34c-aee7227f3d42@intel.com> Date: Mon, 3 Feb 2020 17:40:20 +0000 MIME-Version: 1.0 In-Reply-To: <7566080.EvYhyI6sBW@xps> Content-Type: text/plain; charset=utf-8 Content-Language: en-US Content-Transfer-Encoding: 8bit Subject: Re: [dpdk-dev] [PATCH v2 4/4] add ABI checks X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" Archived-At: List-Archive: On 2/3/2020 5:09 PM, Thomas Monjalon wrote: > 03/02/2020 10:30, Ferruh Yigit: >> On 2/2/2020 2:41 PM, Ananyev, Konstantin wrote: >>> 02/02/2020 14:05, Thomas Monjalon: >>>> 31/01/2020 15:16, Trahe, Fiona: >>>>> On 1/30/2020 8:18 PM, Thomas Monjalon wrote: >>>>>> 30/01/2020 17:09, Ferruh Yigit: >>>>>>> On 1/29/2020 8:13 PM, Akhil Goyal wrote: >>>>>>>> >>>>>>>> I believe these enums will be used only in case of ASYM case which is experimental. >>>>>>> >>>>>>> Independent from being experiment and not, this shouldn't be a problem, I think >>>>>>> this is a false positive. >>>>>>> >>>>>>> The ABI break can happen when a struct has been shared between the application >>>>>>> and the library (DPDK) and the layout of that memory know differently by >>>>>>> application and the library. >>>>>>> >>>>>>> Here in all cases, there is no layout/size change. >>>>>>> >>>>>>> As to the value changes of the enums, since application compiled with old DPDK, >>>>>>> it will know only up to '6', 7 and more means invalid to the application. So it >>>>>>> won't send these values also it should ignore these values from library. Only >>>>>>> consequence is old application won't able to use new features those new enums >>>>>>> provide but that is expected/normal. >>>>>> >>>>>> If library give higher value than expected by the application, >>>>>> if the application uses this value as array index, >>>>>> there can be an access out of bounds. >>>>> >>>>> [Fiona] All asymmetric APIs are experimental so above shouldn't be a problem. >>>>> But for the same issue with sym crypto below, I believe Ferruh's explanation makes >>>>> sense and I don't see how there can be an API breakage. >>>>> So if an application hasn't compiled against the new lib it will be still using the old value >>>>> which will be within bounds. If it's picking up the higher new value from the lib it must >>>>> have been compiled against the lib so shouldn't have problems. >>>> >>>> You say there is no ABI issue because the application will be re-compiled >>>> for the updated library. Indeed, compilation fixes compatibility issues. >>>> But this is not relevant for ABI compatibility. >>>> ABI compatibility means we can upgrade the library without recompiling >>>> the application and it must work. >>>> You think it is a false positive because you assume the application >>>> "picks" the new value. I think you miss the case where the new value >>>> is returned by a function in the upgraded library. >>>> >>>>> There are also no structs on the API which contain arrays using this >>>>> for sizing, so I don't see an opportunity for an appl to have a >>>>> mismatch in memory addresses. >>>> >>>> Let me demonstrate where the API may "use" the new value >>>> RTE_CRYPTO_AEAD_CHACHA20_POLY1305 and how it impacts the application. >>>> >>>> Once upon a time a DPDK application counting the number of devices >>>> supporting each AEAD algo (in order to find the best supported algo). >>>> It is done in an array indexed by algo id: >>>> int aead_dev_count[RTE_CRYPTO_AEAD_LIST_END]; >>>> The application is compiled with DPDK 19.11, >>>> where RTE_CRYPTO_AEAD_LIST_END = 3. >>>> So the size of the application array aead_dev_count is 3. >>>> This binary is run with DPDK 20.02, >>>> where RTE_CRYPTO_AEAD_CHACHA20_POLY1305 = 3. >>>> When calling rte_cryptodev_info_get() on a device QAT_GEN3, >>>> rte_cryptodev_info.capabilities.sym.aead.algo is set to >>>> RTE_CRYPTO_AEAD_CHACHA20_POLY1305 (= 3). >>>> The application uses this value: >>>> ++ aead_dev_count[info.capabilities.sym.aead.algo]; >>>> The application is crashing because of out of bound access. >>> >>> I'd say this is an example of bad written app. >>> It probably should check that returned by library value doesn't >>> exceed its internal array size. >> >> +1 >> >> Application should ignore values >= MAX. > > Of course, blaming the API user is a lot easier than looking at the API. > Here the API has RTE_CRYPTO_AEAD_LIST_END which can be understood > as the max value for the application. > Value ranges are part of the ABI compatibility contract. > It seems you expect the application developer to be aware that > DPDK could return a higher value, so the application should > check every enum values after calling an API. CRAZY. > > When we decide to announce an ABI compatibility and do some marketing, > everyone is OK. But when we need to really make our ABI compatible, > I see little or no effort. DISAPPOINTING. This is not to blame the user or to do less work, this is more sane approach that library provides the _END/_MAX value and application uses it as valid range check. > >> Do you suggest we don't extend any enum or define between ABI breakage releases >> to be sure bad written applications not affected? > > I suggest we must consider not breaking any assumption made on the API. > Here we are breaking the enum range because nothing mentions _LIST_END > is not really the absolute end of the enum. > The solution is to make the change below in 20.02 + backport in 19.11.1: > > - _LIST_END > + _LIST_END, /* an ABI-compatible version may increase this value */ > + _LIST_MAX = _LIST_END + 42 /* room for ABI-compatible additions */ > }; > What is the point of "_LIST_MAX" here? Application should know the "_LIST_END" of when it has been compiled for the valid range check. Next time it is compiled "_LIST_END" may be different value but same logic applies. When "_LIST_END" is missing, application can't protect itself, in that case library should send only the values application knows when it is compiled, this means either we can't extend our enum/defines until next ABI breakage, or we need to do ABI versioning to the functions that returns an enum each time enum value extended. I believe it is saner to provide _END/_MAX values to the application to use. And if required comment them to clarify the expected usage. But in above suggestion application can't use or rely on "_LIST_MAX", it doesn't mean anything to application. > Then *_LIST_END values could be ignored by libabigail with such a change. > > If such a patch is not done by tomorrow, I will have to revert > Chacha-Poly commits before 20.02-rc2, because > > 1/ LIST_END, without any comment, means "size of range" > 2/ we do not blame users for undocumented ABI changes > 3/ we respect the ABI compatibility contract > >