From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from dpdk.org (dpdk.org [92.243.14.124]) by inbox.dpdk.org (Postfix) with ESMTP id 805C3A053A; Wed, 5 Feb 2020 12:10:21 +0100 (CET) Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id 97A3F1C1A0; Wed, 5 Feb 2020 12:10:20 +0100 (CET) Received: from qrelay241.mxroute.com (qrelay241.mxroute.com [172.82.139.241]) by dpdk.org (Postfix) with ESMTP id 6AF881C136 for ; Wed, 5 Feb 2020 12:10:18 +0100 (CET) Received: from filter003.mxroute.com ([168.235.111.26] 168-235-111-26.cloud.ramnode.com) (Authenticated sender: mN4UYu2MZsgR) by qrelay241.mxroute.com (ZoneMTA) with ESMTPA id 170150b24ae0004255.002 for ; Wed, 05 Feb 2020 11:10:16 +0000 X-Zone-Loop: a5a062569240509e71284d00f7451b35781aa9ee11f8 X-Originating-IP: [168.235.111.26] Received: from galaxy.mxroute.com (unknown [23.92.70.113]) by filter003.mxroute.com (Postfix) with ESMTPS id 4610D60032; Wed, 5 Feb 2020 11:10:11 +0000 (UTC) Received: from irdmzpr02-ext.ir.intel.com ([192.198.151.37]) by galaxy.mxroute.com with esmtpsa (TLSv1.2:ECDHE-RSA-AES128-GCM-SHA256:128) (Exim 4.91) (envelope-from ) id 1izIFx-0000Lf-Rj; Wed, 05 Feb 2020 05:50:06 -0500 To: David Marchand Cc: Thomas Monjalon , Neil Horman , Luca Boccassi , Kevin Traynor , "Ananyev, Konstantin" , Akhil Goyal , "Trahe, Fiona" , Ferruh Yigit , dev , Anoob Joseph , "Kusztal, ArkadiuszX" , "Richardson, Bruce" , "Mcnamara, John" , dodji@seketeli.net, Andrew Rybchenko , Aaron Conole References: <20191220152058.10739-1-david.marchand@redhat.com> <666f2cc7-0906-7a07-a582-87800f321a00@intel.com> <7566080.EvYhyI6sBW@xps> <2336620.usQuhbGJ8B@xps> <4ed777ce-8320-4636-2c9c-62bb96b66392@ashroe.eu> From: Ray Kinsella Autocrypt: addr=mdr@ashroe.eu; keydata= mQINBFv8B3wBEAC+5ImcgbIvadt3axrTnt7Sxch3FsmWTTomXfB8YiuHT8KL8L/bFRQSL1f6 ASCHu3M89EjYazlY+vJUWLr0BhK5t/YI7bQzrOuYrl9K94vlLwzD19s/zB/g5YGGR5plJr0s JtJsFGEvF9LL3e+FKMRXveQxBB8A51nAHfwG0WSyx53d61DYz7lp4/Y4RagxaJoHp9lakn8j HV2N6rrnF+qt5ukj5SbbKWSzGg5HQF2t0QQ5tzWhCAKTfcPlnP0GymTBfNMGOReWivi3Qqzr S51Xo7hoGujUgNAM41sxpxmhx8xSwcQ5WzmxgAhJ/StNV9cb3HWIoE5StCwQ4uXOLplZNGnS uxNdegvKB95NHZjRVRChg/uMTGpg9PqYbTIFoPXjuk27sxZLRJRrueg4tLbb3HM39CJwSB++ YICcqf2N+GVD48STfcIlpp12/HI+EcDSThzfWFhaHDC0hyirHxJyHXjnZ8bUexI/5zATn/ux TpMbc/vicJxeN+qfaVqPkCbkS71cHKuPluM3jE8aNCIBNQY1/j87k5ELzg3qaesLo2n1krBH bKvFfAmQuUuJT84/IqfdVtrSCTabvDuNBDpYBV0dGbTwaRfE7i+LiJJclUr8lOvHUpJ4Y6a5 0cxEPxm498G12Z3NoY/mP5soItPIPtLR0rA0fage44zSPwp6cQARAQABtBxSYXkgS2luc2Vs bGEgPG1kckBhc2hyb2UuZXU+iQJUBBMBCAA+FiEEcDUDlKDJaDuJlfZfdJdaH/sCCpsFAlv8 B3wCGyMFCQlmAYAFCwkIBwIGFQoJCAsCBBYCAwECHgECF4AACgkQdJdaH/sCCptdtRAAl0oE msa+djBVYLIsax+0f8acidtWg2l9f7kc2hEjp9h9aZCpPchQvhhemtew/nKavik3RSnLTAyn B3C/0GNlmvI1l5PFROOgPZwz4xhJKGN7jOsRrbkJa23a8ly5UXwF3Vqnlny7D3z+7cu1qq/f VRK8qFyWkAb+xgqeZ/hTcbJUWtW+l5Zb+68WGEp8hB7TuJLEWb4+VKgHTpQ4vElYj8H3Z94a 04s2PJMbLIZSgmKDASnyrKY0CzTpPXx5rSJ1q+B1FCsfepHLqt3vKSALa3ld6bJ8fSJtDUJ7 JLiU8dFZrywgDIVme01jPbjJtUScW6jONLvhI8Z2sheR71UoKqGomMHNQpZ03ViVWBEALzEt TcjWgJFn8yAmxqM4nBnZ+hE3LbMo34KCHJD4eg18ojDt3s9VrDLa+V9fNxUHPSib9FD9UX/1 +nGfU/ZABmiTuUDM7WZdXri7HaMpzDRJUKI6b+/uunF8xH/h/MHW16VuMzgI5dkOKKv1LejD dT5mA4R+2zBS+GsM0oa2hUeX9E5WwjaDzXtVDg6kYq8YvEd+m0z3M4e6diFeLS77/sAOgaYL 92UcoKD+Beym/fVuC6/55a0e12ksTmgk5/ZoEdoNQLlVgd2INtvnO+0k5BJcn66ZjKn3GbEC VqFbrnv1GnA58nEInRCTzR1k26h9nmS5Ag0EW/wHfAEQAMth1vHr3fOZkVOPfod3M6DkQir5 xJvUW5EHgYUjYCPIa2qzgIVVuLDqZgSCCinyooG5dUJONVHj3nCbITCpJp4eB3PI84RPfDcC hf/V34N/Gx5mTeoymSZDBmXT8YtvV/uJvn+LvHLO4ZJdvq5ZxmDyxfXFmkm3/lLw0+rrNdK5 pt6OnVlCqEU9tcDBezjUwDtOahyV20XqxtUttN4kQWbDRkhT+HrA9WN9l2HX91yEYC+zmF1S OhBqRoTPLrR6g4sCWgFywqztpvZWhyIicJipnjac7qL/wRS+wrWfsYy6qWLIV80beN7yoa6v ccnuy4pu2uiuhk9/edtlmFE4dNdoRf7843CV9k1yRASTlmPkU59n0TJbw+okTa9fbbQgbIb1 pWsAuicRHyLUIUz4f6kPgdgty2FgTKuPuIzJd1s8s6p2aC1qo+Obm2gnBTduB+/n1Jw+vKpt 07d+CKEKu4CWwvZZ8ktJJLeofi4hMupTYiq+oMzqH+V1k6QgNm0Da489gXllU+3EFC6W1qKj tkvQzg2rYoWeYD1Qn8iXcO4Fpk6wzylclvatBMddVlQ6qrYeTmSbCsk+m2KVrz5vIyja0o5Y yfeN29s9emXnikmNfv/dA5fpi8XCANNnz3zOfA93DOB9DBf0TQ2/OrSPGjB3op7RCfoPBZ7u AjJ9dM7VABEBAAGJAjwEGAEIACYWIQRwNQOUoMloO4mV9l90l1of+wIKmwUCW/wHfAIbDAUJ CWYBgAAKCRB0l1of+wIKm3KlD/9w/LOG5rtgtCUWPl4B3pZvGpNym6XdK8cop9saOnE85zWf u+sKWCrxNgYkYP7aZrYMPwqDvilxhbTsIJl5HhPgpTO1b0i+c0n1Tij3EElj5UCg3q8mEc17 c+5jRrY3oz77g7E3oPftAjaq1ybbXjY4K32o3JHFR6I8wX3m9wJZJe1+Y+UVrrjY65gZFxcA thNVnWKErarVQGjeNgHV4N1uF3pIx3kT1N4GSnxhoz4Bki91kvkbBhUgYfNflGURfZT3wIKK +d50jd7kqRouXUCzTdzmDh7jnYrcEFM4nvyaYu0JjSS5R672d9SK5LVIfWmoUGzqD4AVmUW8 pcv461+PXchuS8+zpltR9zajl72Q3ymlT4BTAQOlCWkD0snBoKNUB5d2EXPNV13nA0qlm4U2 GpROfJMQXjV6fyYRvttKYfM5xYKgRgtP0z5lTAbsjg9WFKq0Fndh7kUlmHjuAIwKIV4Tzo75 QO2zC0/NTaTjmrtiXhP+vkC4pcrOGNsbHuaqvsc/ZZ0siXyYsqbctj/sCd8ka2r94u+c7o4l BGaAm+FtwAfEAkXHu4y5Phuv2IRR+x1wTey1U1RaEPgN8xq0LQ1OitX4t2mQwjdPihZQBCnZ wzOrkbzlJMNrMKJpEgulmxAHmYJKgvZHXZXtLJSejFjR0GdHJcL5rwVOMWB8cg== Message-ID: <38175d6a-09a3-60a9-d920-785d9ef17c41@ashroe.eu> Date: Wed, 5 Feb 2020 11:10:02 +0000 User-Agent: Mozilla/5.0 (Windows NT 10.0; WOW64; rv:68.0) Gecko/20100101 Thunderbird/68.4.2 MIME-Version: 1.0 In-Reply-To: Content-Type: text/plain; charset=utf-8 Content-Language: en-US Content-Transfer-Encoding: 7bit X-AuthUser: mdr@ashroe.eu Subject: Re: [dpdk-dev] [PATCH v2 4/4] add ABI checks X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" On 04/02/2020 09:51, David Marchand wrote: > On Mon, Feb 3, 2020 at 7:56 PM Ray Kinsella wrote: >> On 03/02/2020 17:34, Thomas Monjalon wrote: >>> 03/02/2020 18:09, Thomas Monjalon: >>>> 03/02/2020 10:30, Ferruh Yigit: >>>>> On 2/2/2020 2:41 PM, Ananyev, Konstantin wrote: >>>>>> 02/02/2020 14:05, Thomas Monjalon: >>>>>>> 31/01/2020 15:16, Trahe, Fiona: >>>>>>>> On 1/30/2020 8:18 PM, Thomas Monjalon wrote: >>>>>>>>> If library give higher value than expected by the application, >>>>>>>>> if the application uses this value as array index, >>>>>>>>> there can be an access out of bounds. >>>>>>>> >>>>>>>> [Fiona] All asymmetric APIs are experimental so above shouldn't be a problem. >>>>>>>> But for the same issue with sym crypto below, I believe Ferruh's explanation makes >>>>>>>> sense and I don't see how there can be an API breakage. >>>>>>>> So if an application hasn't compiled against the new lib it will be still using the old value >>>>>>>> which will be within bounds. If it's picking up the higher new value from the lib it must >>>>>>>> have been compiled against the lib so shouldn't have problems. >>>>>>> >>>>>>> You say there is no ABI issue because the application will be re-compiled >>>>>>> for the updated library. Indeed, compilation fixes compatibility issues. >>>>>>> But this is not relevant for ABI compatibility. >>>>>>> ABI compatibility means we can upgrade the library without recompiling >>>>>>> the application and it must work. >>>>>>> You think it is a false positive because you assume the application >>>>>>> "picks" the new value. I think you miss the case where the new value >>>>>>> is returned by a function in the upgraded library. >>>>>>> >>>>>>>> There are also no structs on the API which contain arrays using this >>>>>>>> for sizing, so I don't see an opportunity for an appl to have a >>>>>>>> mismatch in memory addresses. >>>>>>> >>>>>>> Let me demonstrate where the API may "use" the new value >>>>>>> RTE_CRYPTO_AEAD_CHACHA20_POLY1305 and how it impacts the application. >>>>>>> >>>>>>> Once upon a time a DPDK application counting the number of devices >>>>>>> supporting each AEAD algo (in order to find the best supported algo). >>>>>>> It is done in an array indexed by algo id: >>>>>>> int aead_dev_count[RTE_CRYPTO_AEAD_LIST_END]; >>>>>>> The application is compiled with DPDK 19.11, >>>>>>> where RTE_CRYPTO_AEAD_LIST_END = 3. >>>>>>> So the size of the application array aead_dev_count is 3. >>>>>>> This binary is run with DPDK 20.02, >>>>>>> where RTE_CRYPTO_AEAD_CHACHA20_POLY1305 = 3. >>>>>>> When calling rte_cryptodev_info_get() on a device QAT_GEN3, >>>>>>> rte_cryptodev_info.capabilities.sym.aead.algo is set to >>>>>>> RTE_CRYPTO_AEAD_CHACHA20_POLY1305 (= 3). >>>>>>> The application uses this value: >>>>>>> ++ aead_dev_count[info.capabilities.sym.aead.algo]; >>>>>>> The application is crashing because of out of bound access. >>>>>> >>>>>> I'd say this is an example of bad written app. >>>>>> It probably should check that returned by library value doesn't >>>>>> exceed its internal array size. >>>>> >>>>> +1 >>>>> >>>>> Application should ignore values >= MAX. >>>> >>>> Of course, blaming the API user is a lot easier than looking at the API. >>>> Here the API has RTE_CRYPTO_AEAD_LIST_END which can be understood >>>> as the max value for the application. >>>> Value ranges are part of the ABI compatibility contract. >>>> It seems you expect the application developer to be aware that >>>> DPDK could return a higher value, so the application should >>>> check every enum values after calling an API. CRAZY. >>>> >>>> When we decide to announce an ABI compatibility and do some marketing, >>>> everyone is OK. But when we need to really make our ABI compatible, >>>> I see little or no effort. DISAPPOINTING. >>>> >>>>> Do you suggest we don't extend any enum or define between ABI breakage releases >>>>> to be sure bad written applications not affected? >>>> >>>> I suggest we must consider not breaking any assumption made on the API. >>>> Here we are breaking the enum range because nothing mentions _LIST_END >>>> is not really the absolute end of the enum. >>>> The solution is to make the change below in 20.02 + backport in 19.11.1: >>> >>> Thinking twice, merging such change before 20.11 is breaking the >>> ABI assumption based on the API 19.11.0. >>> I ask the release maintainers (Luca, Kevin, David and me) and >>> the ABI maintainers (Neil and Ray) to vote for a or b solution: >>> a) add comment and LIST_MAX as below in 20.02 + 19.11.1 >> >> That would still be an ABI breakage though right. > > Yes. > > >> >>> b) wait 20.11 and revert Chacha-Poly from 20.02 >> >> Thanks for analysis above Fiona, Ferruh and all. >> >> That is a nasty one alright - there is no "good" answer here. >> I agree with Ferruh's sentiments overall, we should rethink this API for 20.11. >> Could do without an enumeration? >> >> There a c) though right. >> We could work around the issue by api versioning rte_cryptodev_info_get() and friends. > > It has a lot of friends, but it sounds like the right approach. +1 > Is someone looking into this? Looks to be in hand now. > > >> So they only support/acknowledge the existence of Chacha-Poly for applications build against > 20.02. >> >> It would be painful I know. >> It would also mean that Chacha-Poly would only be available to those building against >= 20.02. > > Yes. > > > -- > David Marchand >