From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 9616145AAC; Fri, 4 Oct 2024 11:38:34 +0200 (CEST) Received: from mails.dpdk.org (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 63EC94027F; Fri, 4 Oct 2024 11:38:34 +0200 (CEST) Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) by mails.dpdk.org (Postfix) with ESMTP id 35E0440268 for ; Fri, 4 Oct 2024 11:38:32 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1728034711; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: in-reply-to:in-reply-to:references:references; bh=xsFUeMJdvfzF+sBOgP7tMZyuMfbImLWkPVVjHZDN0Bc=; b=ifoLOncg5Hsv1q79zUDUwIxpzY7EJlh/Z4tDfMKT/5N7b35ZH1GooSr+PQlpc5rQ9QIreJ zVXvn0W0l2YkT17RqIBmPMYfhlNbb4mKLDbFnpFcbxVZV33VYzgzfYi0hWjSdbvDN9gMhz znNZ0sU0kqTJWUSdYGBzOZYxKxRaEYs= Received: from mail-qv1-f71.google.com (mail-qv1-f71.google.com [209.85.219.71]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-332-Jg-9_Lk7O62hQCuiw7uDog-1; Fri, 04 Oct 2024 05:38:30 -0400 X-MC-Unique: Jg-9_Lk7O62hQCuiw7uDog-1 Received: by mail-qv1-f71.google.com with SMTP id 6a1803df08f44-6cb36194756so27183406d6.0 for ; Fri, 04 Oct 2024 02:38:30 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1728034710; x=1728639510; h=mime-version:user-agent:message-id:in-reply-to:date:references :organization:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=xsFUeMJdvfzF+sBOgP7tMZyuMfbImLWkPVVjHZDN0Bc=; b=MF1euKIWhbIYXa6fVrMS6+IouOwijEbABlOswMcZm9641A0rmpaQMIdwHt5KaxGHU6 6yLlw0zlp0JJzMO8/HJ/CKXwJmwln0HSsZa3DbIlex8z0ngG67lRcTseBE7SjKS1p8eV T5IdoRyGWkdO8YbY5lJMl+NiAiv9iP3QZaxTv+aerZ+Mztpxqkr1q53hM1BbY68EpCsK pKIvE80M9WSH1PR0rtcNPHVv540ZsXTxWXFLBEVDb0VP2lVb35ysrNJznYqmf/rReUZR yPiZj6TVhPU8OoPVQSosSCyiF6ncBoM5dn4vxjZiDR8I76T82LliKr9s+JrsApyPa3Qe 8OcQ== X-Forwarded-Encrypted: i=1; AJvYcCVfAwCx95zIj06ED3+vamMbInxuo2zgn1tVxPkUQafJovesHYOI3E4/9qVWJOQMN09kUzQ=@dpdk.org X-Gm-Message-State: AOJu0YxrmoXTrvfACwOhs/T5o0EbebtvDYfQVhGRHPZ90HUr9G53sJvW mW4XtzZqiUMcdN6MeRh95LJicC/z8ZLBy7VzBVV/Qt1AeKMs+ES5tmp9qwgGoGK1i5DyUbTqI10 gtGhstDaSeO+HQvWim7gcdUVcCdal7ofawtQ7xiLv X-Received: by 2002:a05:6214:4413:b0:6cb:79ce:bdef with SMTP id 6a1803df08f44-6cb9a2fc065mr26956586d6.14.1728034709727; Fri, 04 Oct 2024 02:38:29 -0700 (PDT) X-Google-Smtp-Source: AGHT+IFsnXHSm4hCbbQi7F6p9oSEDWa0Kev8K/V5Orzft0FL0Rrr7z1LhumeGjyY/RBtgtZKZcDOeg== X-Received: by 2002:a05:6214:4413:b0:6cb:79ce:bdef with SMTP id 6a1803df08f44-6cb9a2fc065mr26956386d6.14.1728034709302; Fri, 04 Oct 2024 02:38:29 -0700 (PDT) Received: from localhost (88-120-130-27.subs.proxad.net. [88.120.130.27]) by smtp.gmail.com with ESMTPSA id 6a1803df08f44-6cb937f78a0sm14117676d6.127.2024.10.04.02.38.28 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 04 Oct 2024 02:38:28 -0700 (PDT) Received: by localhost (Postfix, from userid 1000) id 798CEC1B73C5; Fri, 4 Oct 2024 11:38:26 +0200 (CEST) From: Dodji Seketeli To: Ferruh Yigit Cc: Akhil Goyal , dev@dpdk.org, Dodji Seketeli , thomas@monjalon.net, david.marchand@redhat.com, hemant.agrawal@nxp.com, anoobj@marvell.com, pablo.de.lara.guarch@intel.com, fiona.trahe@intel.com, declan.doherty@intel.com, matan@nvidia.com, g.singh@nxp.com, fanzhang.oss@gmail.com, jianjay.zhou@huawei.com, asomalap@amd.com, ruifeng.wang@arm.com, konstantin.v.ananyev@yandex.ru, radu.nicolau@intel.com, ajit.khaparde@broadcom.com, rnagadheeraj@marvell.com, mdr@ashroe.eu Subject: Re: [PATCH] [RFC] cryptodev: replace LIST_END enumerators with APIs Organization: Red Hat / France References: <20240905101438.3888274-1-gakhil@marvell.com> <10591cfb-d78e-4d50-9bb5-f6d1246662b3@amd.com> X-Operating-System: AlmaLinux 9.4 X-URL: http://www.redhat.com Date: Fri, 04 Oct 2024 11:38:26 +0200 In-Reply-To: <10591cfb-d78e-4d50-9bb5-f6d1246662b3@amd.com> (Ferruh Yigit's message of "Fri, 4 Oct 2024 04:54:38 +0100") Message-ID: <87msjkdx7x.fsf@redhat.com> User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/27.2 (gnu/linux) MIME-Version: 1.0 X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-Type: text/plain X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Hello, Ferruh Yigit writes: > On 9/5/2024 11:14 AM, Akhil Goyal wrote: >> Replace *_LIST_END enumerators from asymmetric crypto >> lib to avoid ABI breakage for every new addition in >> enums with inline APIs. >> >> Signed-off-by: Akhil Goyal >> --- >> This patch was discussed in ML long time back. >> https://patches.dpdk.org/project/dpdk/patch/20211008204516.3497060-1-gakhil@marvell.com/ >> Now added inline APIs for getting the list end which need to be updated >> for each new entry to the enum. This shall help in avoiding ABI break >> for adding new algo. >> > > Hi Akhil, > > *I think* this hides the problem instead of fixing it, and this may be > partially because of the tooling (libabigail) limitation. So, I am not sure to understand how this would be a libabigail limitation in general. Let me explain. The abidiff tool that compares two binaries and emits reports about their ABI changes can be given a suppression specification file which instructs the tool about what type of change to avoid emitting reports about. It turns out there is a construct specifically designed to handle the case where an enumerator is added right before the last enumerator of a enum. Such a change causes the value of the last enumerator to increase. For instance, consider this enum: enum foo_enum { FOO_FIRST, FOO_SECOND, FOO_END }; If I add a new enumerator right before the last enumerator, I would get this: enum foo_enum { FOO_FIRST, FOO_SECOND, FOO_THIRD, FOO_END }; This change cause the value of the the FOOD_END enumerator to increase. And that increase might be problematic. At the moment, for it being problematic or not has to be the result of a careful review. So, by default, abidiff will complain by saying that the value of FOO_END was changed. But you, as a community of practice, can decide that this kind of change to the value of the last enumerator is not a problematic change, after careful review of your code and practice. You thus can specify that the tool using a suppression specification which has the following syntax: [suppress_type] type_kind = enum changed_enumerators = FOO_END, ANOTHER_ENUM_END, AND_ANOTHER_ENUM_END or, alternatively, you can specify the enumerators you want to suppress the changes for as a list of regular expressions: [suppress_type] type_kind = enum changed_enumerators_regexp = .*_END$, .*_LAST$, .*_LIST_END$ Wouldn't that be enough to address your use case here (honest question)? > This patch prevents the libabigail warning, true, but it doesn't improve > anything from the application's perspective. > Before or after this patch, application knows a fixed value as END value. > > Not all changes in the END (MAX) enum values cause ABI break, but tool > warns on all, that is why I think this may be tooling limitation [1]. > (Just to stress, I am NOT talking about regular enum values change, I am > talking about only END (MAX) value changes caused by appending new enum > items.) > > As far as I can see (please Dodji, David correct me if I am wrong) ABI > break only happens if application and library exchange enum values in > the API (directly or within a struct). Sorry, I am not sure to understand what you mean by "exchange enum values". I would say is that if a change to an enumerator value causes a change to the layout of a type which is part of the ABI, then that enumerator value change might change the ABI. That ABI change might be problematic (i.e an ABI break) or not. > > Exchanging enum values via API cause ABI issues because: > 1. enum size is not fixed, it uses min storage size that can hold all > values, adding new enums may cause its size change and of course this > breaks ABI. > 2. Library can send enum values that application doesn't know, this may > cause application behave undefined. > 3. Application sending MAX value (or more) to API expects error, but > that may be a valid value in the new library and applications gets > unexpected result. > > > Let's assume above 1. happens, with this patch warning is prevented, but > ABI break still can occur. > > One option can be not exchanging enums in APIs at all, this way changes > in the END (MAX) enum values are harmless. But I can see cryptodev does > this a lot. > > When enum is exchanged in APIs, and if we want to be strict about the > ABI, safest option can be not appending to enums at all, and keeping END > (MAX) items can be a way to enable warnings for this. > > > More relaxed option is protect against only 1. since that is the real > ABI issue, 2 & 3 are more defensive programming, so as long as enum size > is not changed we may ignore the END (MAX) value change warnings. > > > > [1] It would be better if tool gives END (MAX) enum value warnings only > if it is exchanged in an API, but not sure if this can be possible to > detect. I believe that if you want to know if an enumerator value is *USED* by a type (which I believe is at the root of what you are alluding to), then you would need a static analysis tool that works at the source level. Or, you need a human review of the code once the binary analysis tool told you that that value of the enumerator changed. Why ? please let me give you an example: enum foo_enum { FOO_FIRST, FOO_SECOND, FOO_END }; int array[FOO_END]; Once this is compiled into binary, what libabigail is going to see by analyzing the binary is that 'array' is an array of 2 integers. The information about the size of the array being initially an enumerator value is lost. To detect that, you need source level analysis. But then, by reviewing the code, this is a construct that you can spot and allow or forbid, depending on your goals as a project. [...] Cheers, -- Dodji