From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from dpdk.org (dpdk.org [92.243.14.124]) by inbox.dpdk.org (Postfix) with ESMTP id 44054A0530; Mon, 3 Feb 2020 19:56:17 +0100 (CET) Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id B0A881BFA7; Mon, 3 Feb 2020 19:56:16 +0100 (CET) Received: from qrelay119.mxroute.com (qrelay119.mxroute.com [172.82.139.119]) by dpdk.org (Postfix) with ESMTP id 9E37B1BFA0 for ; Mon, 3 Feb 2020 19:56:15 +0100 (CET) Received: from filter003.mxroute.com [168.235.111.26] (Authenticated sender: mN4UYu2MZsgR) by qrelay119.mxroute.com (ZoneMTA) with ESMTPA id 1700c68f94a0004255.002 for ; Mon, 03 Feb 2020 18:56:11 +0000 X-Zone-Loop: a978d63f9e189c67709ed15a8cb25e37dfccac41ed56 X-Originating-IP: [168.235.111.26] Received: from galaxy.mxroute.com (unknown [23.92.70.113]) by filter003.mxroute.com (Postfix) with ESMTPS id A3C336000A; Mon, 3 Feb 2020 18:56:05 +0000 (UTC) Received: from [192.198.151.44] by galaxy.mxroute.com with esmtpsa (TLSv1.2:ECDHE-RSA-AES128-GCM-SHA256:128) (Exim 4.91) (envelope-from ) id 1iygZs-0008EJ-Fv; Mon, 03 Feb 2020 13:36:08 -0500 To: Thomas Monjalon , David Marchand , nhorman@tuxdriver.com, bluca@debian.org, ktraynor@redhat.com Cc: "Ananyev, Konstantin" , Akhil Goyal , "Trahe, Fiona" , Ferruh Yigit , dev@dpdk.org, Anoob Joseph , "Kusztal, ArkadiuszX" , "Richardson, Bruce" , "Mcnamara, John" , dodji@seketeli.net, Andrew Rybchenko , aconole@redhat.com References: <20191220152058.10739-1-david.marchand@redhat.com> <666f2cc7-0906-7a07-a582-87800f321a00@intel.com> <7566080.EvYhyI6sBW@xps> <2336620.usQuhbGJ8B@xps> From: Ray Kinsella Autocrypt: addr=mdr@ashroe.eu; keydata= mQINBFv8B3wBEAC+5ImcgbIvadt3axrTnt7Sxch3FsmWTTomXfB8YiuHT8KL8L/bFRQSL1f6 ASCHu3M89EjYazlY+vJUWLr0BhK5t/YI7bQzrOuYrl9K94vlLwzD19s/zB/g5YGGR5plJr0s JtJsFGEvF9LL3e+FKMRXveQxBB8A51nAHfwG0WSyx53d61DYz7lp4/Y4RagxaJoHp9lakn8j HV2N6rrnF+qt5ukj5SbbKWSzGg5HQF2t0QQ5tzWhCAKTfcPlnP0GymTBfNMGOReWivi3Qqzr S51Xo7hoGujUgNAM41sxpxmhx8xSwcQ5WzmxgAhJ/StNV9cb3HWIoE5StCwQ4uXOLplZNGnS uxNdegvKB95NHZjRVRChg/uMTGpg9PqYbTIFoPXjuk27sxZLRJRrueg4tLbb3HM39CJwSB++ YICcqf2N+GVD48STfcIlpp12/HI+EcDSThzfWFhaHDC0hyirHxJyHXjnZ8bUexI/5zATn/ux TpMbc/vicJxeN+qfaVqPkCbkS71cHKuPluM3jE8aNCIBNQY1/j87k5ELzg3qaesLo2n1krBH bKvFfAmQuUuJT84/IqfdVtrSCTabvDuNBDpYBV0dGbTwaRfE7i+LiJJclUr8lOvHUpJ4Y6a5 0cxEPxm498G12Z3NoY/mP5soItPIPtLR0rA0fage44zSPwp6cQARAQABtBxSYXkgS2luc2Vs bGEgPG1kckBhc2hyb2UuZXU+iQJUBBMBCAA+FiEEcDUDlKDJaDuJlfZfdJdaH/sCCpsFAlv8 B3wCGyMFCQlmAYAFCwkIBwIGFQoJCAsCBBYCAwECHgECF4AACgkQdJdaH/sCCptdtRAAl0oE msa+djBVYLIsax+0f8acidtWg2l9f7kc2hEjp9h9aZCpPchQvhhemtew/nKavik3RSnLTAyn B3C/0GNlmvI1l5PFROOgPZwz4xhJKGN7jOsRrbkJa23a8ly5UXwF3Vqnlny7D3z+7cu1qq/f VRK8qFyWkAb+xgqeZ/hTcbJUWtW+l5Zb+68WGEp8hB7TuJLEWb4+VKgHTpQ4vElYj8H3Z94a 04s2PJMbLIZSgmKDASnyrKY0CzTpPXx5rSJ1q+B1FCsfepHLqt3vKSALa3ld6bJ8fSJtDUJ7 JLiU8dFZrywgDIVme01jPbjJtUScW6jONLvhI8Z2sheR71UoKqGomMHNQpZ03ViVWBEALzEt TcjWgJFn8yAmxqM4nBnZ+hE3LbMo34KCHJD4eg18ojDt3s9VrDLa+V9fNxUHPSib9FD9UX/1 +nGfU/ZABmiTuUDM7WZdXri7HaMpzDRJUKI6b+/uunF8xH/h/MHW16VuMzgI5dkOKKv1LejD dT5mA4R+2zBS+GsM0oa2hUeX9E5WwjaDzXtVDg6kYq8YvEd+m0z3M4e6diFeLS77/sAOgaYL 92UcoKD+Beym/fVuC6/55a0e12ksTmgk5/ZoEdoNQLlVgd2INtvnO+0k5BJcn66ZjKn3GbEC VqFbrnv1GnA58nEInRCTzR1k26h9nmS5Ag0EW/wHfAEQAMth1vHr3fOZkVOPfod3M6DkQir5 xJvUW5EHgYUjYCPIa2qzgIVVuLDqZgSCCinyooG5dUJONVHj3nCbITCpJp4eB3PI84RPfDcC hf/V34N/Gx5mTeoymSZDBmXT8YtvV/uJvn+LvHLO4ZJdvq5ZxmDyxfXFmkm3/lLw0+rrNdK5 pt6OnVlCqEU9tcDBezjUwDtOahyV20XqxtUttN4kQWbDRkhT+HrA9WN9l2HX91yEYC+zmF1S OhBqRoTPLrR6g4sCWgFywqztpvZWhyIicJipnjac7qL/wRS+wrWfsYy6qWLIV80beN7yoa6v ccnuy4pu2uiuhk9/edtlmFE4dNdoRf7843CV9k1yRASTlmPkU59n0TJbw+okTa9fbbQgbIb1 pWsAuicRHyLUIUz4f6kPgdgty2FgTKuPuIzJd1s8s6p2aC1qo+Obm2gnBTduB+/n1Jw+vKpt 07d+CKEKu4CWwvZZ8ktJJLeofi4hMupTYiq+oMzqH+V1k6QgNm0Da489gXllU+3EFC6W1qKj tkvQzg2rYoWeYD1Qn8iXcO4Fpk6wzylclvatBMddVlQ6qrYeTmSbCsk+m2KVrz5vIyja0o5Y yfeN29s9emXnikmNfv/dA5fpi8XCANNnz3zOfA93DOB9DBf0TQ2/OrSPGjB3op7RCfoPBZ7u AjJ9dM7VABEBAAGJAjwEGAEIACYWIQRwNQOUoMloO4mV9l90l1of+wIKmwUCW/wHfAIbDAUJ CWYBgAAKCRB0l1of+wIKm3KlD/9w/LOG5rtgtCUWPl4B3pZvGpNym6XdK8cop9saOnE85zWf u+sKWCrxNgYkYP7aZrYMPwqDvilxhbTsIJl5HhPgpTO1b0i+c0n1Tij3EElj5UCg3q8mEc17 c+5jRrY3oz77g7E3oPftAjaq1ybbXjY4K32o3JHFR6I8wX3m9wJZJe1+Y+UVrrjY65gZFxcA thNVnWKErarVQGjeNgHV4N1uF3pIx3kT1N4GSnxhoz4Bki91kvkbBhUgYfNflGURfZT3wIKK +d50jd7kqRouXUCzTdzmDh7jnYrcEFM4nvyaYu0JjSS5R672d9SK5LVIfWmoUGzqD4AVmUW8 pcv461+PXchuS8+zpltR9zajl72Q3ymlT4BTAQOlCWkD0snBoKNUB5d2EXPNV13nA0qlm4U2 GpROfJMQXjV6fyYRvttKYfM5xYKgRgtP0z5lTAbsjg9WFKq0Fndh7kUlmHjuAIwKIV4Tzo75 QO2zC0/NTaTjmrtiXhP+vkC4pcrOGNsbHuaqvsc/ZZ0siXyYsqbctj/sCd8ka2r94u+c7o4l BGaAm+FtwAfEAkXHu4y5Phuv2IRR+x1wTey1U1RaEPgN8xq0LQ1OitX4t2mQwjdPihZQBCnZ wzOrkbzlJMNrMKJpEgulmxAHmYJKgvZHXZXtLJSejFjR0GdHJcL5rwVOMWB8cg== Message-ID: <4ed777ce-8320-4636-2c9c-62bb96b66392@ashroe.eu> Date: Mon, 3 Feb 2020 18:55:59 +0000 User-Agent: Mozilla/5.0 (Windows NT 10.0; WOW64; rv:68.0) Gecko/20100101 Thunderbird/68.4.2 MIME-Version: 1.0 In-Reply-To: <2336620.usQuhbGJ8B@xps> Content-Type: text/plain; charset=utf-8 Content-Language: en-US Content-Transfer-Encoding: 8bit X-AuthUser: mdr@ashroe.eu Subject: Re: [dpdk-dev] [PATCH v2 4/4] add ABI checks X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" On 03/02/2020 17:34, Thomas Monjalon wrote: > 03/02/2020 18:09, Thomas Monjalon: >> 03/02/2020 10:30, Ferruh Yigit: >>> On 2/2/2020 2:41 PM, Ananyev, Konstantin wrote: >>>> 02/02/2020 14:05, Thomas Monjalon: >>>>> 31/01/2020 15:16, Trahe, Fiona: >>>>>> On 1/30/2020 8:18 PM, Thomas Monjalon wrote: >>>>>>> If library give higher value than expected by the application, >>>>>>> if the application uses this value as array index, >>>>>>> there can be an access out of bounds. >>>>>> >>>>>> [Fiona] All asymmetric APIs are experimental so above shouldn't be a problem. >>>>>> But for the same issue with sym crypto below, I believe Ferruh's explanation makes >>>>>> sense and I don't see how there can be an API breakage. >>>>>> So if an application hasn't compiled against the new lib it will be still using the old value >>>>>> which will be within bounds. If it's picking up the higher new value from the lib it must >>>>>> have been compiled against the lib so shouldn't have problems. >>>>> >>>>> You say there is no ABI issue because the application will be re-compiled >>>>> for the updated library. Indeed, compilation fixes compatibility issues. >>>>> But this is not relevant for ABI compatibility. >>>>> ABI compatibility means we can upgrade the library without recompiling >>>>> the application and it must work. >>>>> You think it is a false positive because you assume the application >>>>> "picks" the new value. I think you miss the case where the new value >>>>> is returned by a function in the upgraded library. >>>>> >>>>>> There are also no structs on the API which contain arrays using this >>>>>> for sizing, so I don't see an opportunity for an appl to have a >>>>>> mismatch in memory addresses. >>>>> >>>>> Let me demonstrate where the API may "use" the new value >>>>> RTE_CRYPTO_AEAD_CHACHA20_POLY1305 and how it impacts the application. >>>>> >>>>> Once upon a time a DPDK application counting the number of devices >>>>> supporting each AEAD algo (in order to find the best supported algo). >>>>> It is done in an array indexed by algo id: >>>>> int aead_dev_count[RTE_CRYPTO_AEAD_LIST_END]; >>>>> The application is compiled with DPDK 19.11, >>>>> where RTE_CRYPTO_AEAD_LIST_END = 3. >>>>> So the size of the application array aead_dev_count is 3. >>>>> This binary is run with DPDK 20.02, >>>>> where RTE_CRYPTO_AEAD_CHACHA20_POLY1305 = 3. >>>>> When calling rte_cryptodev_info_get() on a device QAT_GEN3, >>>>> rte_cryptodev_info.capabilities.sym.aead.algo is set to >>>>> RTE_CRYPTO_AEAD_CHACHA20_POLY1305 (= 3). >>>>> The application uses this value: >>>>> ++ aead_dev_count[info.capabilities.sym.aead.algo]; >>>>> The application is crashing because of out of bound access. >>>> >>>> I'd say this is an example of bad written app. >>>> It probably should check that returned by library value doesn't >>>> exceed its internal array size. >>> >>> +1 >>> >>> Application should ignore values >= MAX. >> >> Of course, blaming the API user is a lot easier than looking at the API. >> Here the API has RTE_CRYPTO_AEAD_LIST_END which can be understood >> as the max value for the application. >> Value ranges are part of the ABI compatibility contract. >> It seems you expect the application developer to be aware that >> DPDK could return a higher value, so the application should >> check every enum values after calling an API. CRAZY. >> >> When we decide to announce an ABI compatibility and do some marketing, >> everyone is OK. But when we need to really make our ABI compatible, >> I see little or no effort. DISAPPOINTING. >> >>> Do you suggest we don't extend any enum or define between ABI breakage releases >>> to be sure bad written applications not affected? >> >> I suggest we must consider not breaking any assumption made on the API. >> Here we are breaking the enum range because nothing mentions _LIST_END >> is not really the absolute end of the enum. >> The solution is to make the change below in 20.02 + backport in 19.11.1: > > Thinking twice, merging such change before 20.11 is breaking the > ABI assumption based on the API 19.11.0. > I ask the release maintainers (Luca, Kevin, David and me) and > the ABI maintainers (Neil and Ray) to vote for a or b solution: > a) add comment and LIST_MAX as below in 20.02 + 19.11.1 That would still be an ABI breakage though right. > b) wait 20.11 and revert Chacha-Poly from 20.02 Thanks for analysis above Fiona, Ferruh and all. That is a nasty one alright - there is no "good" answer here. I agree with Ferruh's sentiments overall, we should rethink this API for 20.11. Could do without an enumeration? There a c) though right. We could work around the issue by api versioning rte_cryptodev_info_get() and friends. So they only support/acknowledge the existence of Chacha-Poly for applications build against > 20.02. It would be painful I know. It would also mean that Chacha-Poly would only be available to those building against >= 20.02. > > >> - _LIST_END >> + _LIST_END, /* an ABI-compatible version may increase this value */ >> + _LIST_MAX = _LIST_END + 42 /* room for ABI-compatible additions */ >> }; >> >> Then *_LIST_END values could be ignored by libabigail with such a change. >> >> If such a patch is not done by tomorrow, I will have to revert >> Chacha-Poly commits before 20.02-rc2, because >> >> 1/ LIST_END, without any comment, means "size of range" >> 2/ we do not blame users for undocumented ABI changes >> 3/ we respect the ABI compatibility contract