[PATCH v2] eal/unix: support ZSTD compression for firmware

[PATCH v2] eal/unix: support ZSTD compression for firmware

[David Marchand]

Ubuntu 24.04 started to compress firmware files with ZSTD compression.

Bugzilla ID: 1437
Cc: stable@dpdk.org

Signed-off-by: David Marchand <david.marchand@redhat.com>
---
Chances since v1:
- fixed link issue when libarchive is not available,
- Cc'd stable@dpdk.org,

---
 lib/eal/unix/eal_firmware.c | 24 ++++++++++++++++++------
 1 file changed, 18 insertions(+), 6 deletions(-)

diff --git a/lib/eal/unix/eal_firmware.c b/lib/eal/unix/eal_firmware.c
index 1d47e879c8..31de027598 100644
--- a/lib/eal/unix/eal_firmware.c
+++ b/lib/eal/unix/eal_firmware.c
@@ -16,6 +16,8 @@
 #include "eal_firmware.h"
 #include "eal_private.h"
 
+static const char * const compression_algorithms[] = { "xz", "zst" };
+
 #ifdef RTE_HAS_LIBARCHIVE
 
 struct firmware_read_ctx {
@@ -37,7 +39,11 @@ firmware_open(struct firmware_read_ctx *ctx, const char *name, size_t blocksize)
 
 	err = archive_read_support_filter_xz(ctx->a);
 	if (err != ARCHIVE_OK && err != ARCHIVE_WARN)
-		goto error;
+		EAL_LOG(DEBUG, "could not initialise libarchive for xz compression");
+
+	err = archive_read_support_filter_zstd(ctx->a);
+	if (err != ARCHIVE_OK && err != ARCHIVE_WARN)
+		EAL_LOG(DEBUG, "could not initialise libarchive for zstd compression");
 
 	if (archive_read_open_filename(ctx->a, name, blocksize) != ARCHIVE_OK)
 		goto error;
@@ -148,16 +154,22 @@ rte_firmware_read(const char *name, void **buf, size_t *bufsz)
 
 	ret = firmware_read(name, buf, bufsz);
 	if (ret < 0) {
-		snprintf(path, sizeof(path), "%s.xz", name);
-		path[PATH_MAX - 1] = '\0';
+		unsigned int i;
+
+		for (i = 0; i < RTE_DIM(compression_algorithms); i++) {
+			snprintf(path, sizeof(path), "%s.%s", name,
+				compression_algorithms[i]);
+			path[PATH_MAX - 1] = '\0';
+			if (access(path, F_OK) != 0)
+				continue;
 #ifndef RTE_HAS_LIBARCHIVE
-		if (access(path, F_OK) == 0) {
 			EAL_LOG(WARNING, "libarchive not linked, %s cannot be decompressed",
 				path);
-		}
 #else
-		ret = firmware_read(path, buf, bufsz);
+			ret = firmware_read(path, buf, bufsz);
 #endif
+			break;
+		}
 	}
 	return ret;
 }
-- 
2.44.0

Re: [PATCH v2] eal/unix: support ZSTD compression for firmware

[Bruce Richardson]

On Mon, May 13, 2024 at 01:12:04PM +0200, David Marchand wrote:
> Ubuntu 24.04 started to compress firmware files with ZSTD compression.
> 
> Bugzilla ID: 1437
> Cc: stable@dpdk.org
> 
> Signed-off-by: David Marchand <david.marchand@redhat.com>
> ---
> Chances since v1:
> - fixed link issue when libarchive is not available,
> - Cc'd stable@dpdk.org,
> 
> ---
>  lib/eal/unix/eal_firmware.c | 24 ++++++++++++++++++------
>  1 file changed, 18 insertions(+), 6 deletions(-)
> 

Acked-by: Bruce Richardson <bruce.richardson@intel.com>

Verified that port initializes ok with zstd compressed firmware, and we get
suitable error if libarchive is not installed.

Tested-by: Bruce Richardson <bruce.richardson@intel.com>

One minor suggestion below.

> diff --git a/lib/eal/unix/eal_firmware.c b/lib/eal/unix/eal_firmware.c
> index 1d47e879c8..31de027598 100644
> --- a/lib/eal/unix/eal_firmware.c
> +++ b/lib/eal/unix/eal_firmware.c
> @@ -16,6 +16,8 @@
>  #include "eal_firmware.h"
>  #include "eal_private.h"
>  
> +static const char * const compression_algorithms[] = { "xz", "zst" };
Very minor nit: these are the suffixes used to find the files, more than
they are the compression algorithms. There I'd suggest renaming to
"compression_suffixes".
[Perhaps just fix on apply if taking this suggestion]

> +
>  #ifdef RTE_HAS_LIBARCHIVE
>  
>  struct firmware_read_ctx {
<snip>

Re: [PATCH v2] eal/unix: support ZSTD compression for firmware

[David Marchand]

Hello Bruce,

On Mon, May 13, 2024 at 1:37 PM Bruce Richardson
<bruce.richardson@intel.com> wrote:
> > @@ -16,6 +16,8 @@
> >  #include "eal_firmware.h"
> >  #include "eal_private.h"
> >
> > +static const char * const compression_algorithms[] = { "xz", "zst" };
> Very minor nit: these are the suffixes used to find the files, more than
> they are the compression algorithms. There I'd suggest renaming to
> "compression_suffixes".
> [Perhaps just fix on apply if taking this suggestion]

I'll fix when applying.
Thanks for the review and tests.


-- 
David Marchand

Re: [PATCH v2] eal/unix: support ZSTD compression for firmware

[David Marchand]

On Mon, May 13, 2024 at 1:37 PM Bruce Richardson
<bruce.richardson@intel.com> wrote:
> On Mon, May 13, 2024 at 01:12:04PM +0200, David Marchand wrote:
> > Ubuntu 24.04 started to compress firmware files with ZSTD compression.
> >
> > Bugzilla ID: 1437
> > Cc: stable@dpdk.org
> >
> > Signed-off-by: David Marchand <david.marchand@redhat.com>
> Acked-by: Bruce Richardson <bruce.richardson@intel.com>

Applied with additional nit from Bruce on variable name.
Thanks.


-- 
David Marchand