[PATCH] compressdev: fix end of comp PMD list macro conflict

[PATCH] compressdev: fix end of comp PMD list macro conflict

[Michael Baum]

The "rte_compressdev_info_get()" function retrieves the contextual
information of a device.
The output structure "dev_info" contains a list of devices supported
capabilities for each supported algorithm.

In this function description, it says the element after the last valid
element has op field set to "RTE_COMP_ALGO_LIST_END".
On the other hand, when this function used by
"rte_compressdev_capability_get()" function, it uses
"RTE_COMP_ALGO_UNSPECIFIED" as end of list as same as the
"RTE_COMP_END_OF_CAPABILITIES_LIST()".

The mlx5 and qat PMDs use "RTE_COMP_ALGO_LIST_END" as the end of
capabilities list. When "rte_compressdev_capability_get()" function is
called with unsupported algorithm, it might read memory out of bound.

This patch change the "rte_compressdev_info_get()" function description
to say using "RTE_COMP_ALGO_UNSPECIFIED" as the end of capabilities
list.
In addition, it moves both mlx5 and qat PMDs to use
"RTE_COMP_ALGO_UNSPECIFIED" through
"RTE_COMP_END_OF_CAPABILITIES_LIST()" macro.

Fixes: 5d432f364078 ("compressdev: add device capabilities")
Fixes: 2d148597ce76 ("compress/qat: add gen-specific implementation")
Fixes: 384bac8d6555 ("compress/mlx5: add supported capabilities")
Cc: fiona.trahe@intel.com
Cc: roy.fan.zhang@intel.com
Cc: matan@nvidia.com
Cc: stable@dpdk.org

Signed-off-by: Michael Baum <michaelba@nvidia.com>

---

After this change, I'm not sure about the purpose of
"RTE_COMP_ALGO_LIST_END".
There is no any other use of it in DPDK code, and it isn't represent the
number of algorithms supported by the API since the
"RTE_COMP_ALGO_UNSPECIFIED" is part of the enum.

Due to the compress API is experimental I think the
"RTE_COMP_ALGO_LIST_END" can be removed.



 drivers/compress/mlx5/mlx5_compress.c        | 4 +---
 drivers/compress/qat/dev/qat_comp_pmd_gen1.c | 2 +-
 drivers/compress/qat/dev/qat_comp_pmd_gen4.c | 2 +-
 lib/compressdev/rte_compressdev.h            | 2 +-
 4 files changed, 4 insertions(+), 6 deletions(-)

diff --git a/drivers/compress/mlx5/mlx5_compress.c b/drivers/compress/mlx5/mlx5_compress.c
index fb2bda9745..459e4b5e8a 100644
--- a/drivers/compress/mlx5/mlx5_compress.c
+++ b/drivers/compress/mlx5/mlx5_compress.c
@@ -96,9 +96,7 @@ static const struct rte_compressdev_capabilities mlx5_caps[] = {
 				      RTE_COMP_FF_HUFFMAN_DYNAMIC,
 		.window_size = {.min = 10, .max = 15, .increment = 1},
 	},
-	{
-		.algo = RTE_COMP_ALGO_LIST_END,
-	}
+	RTE_COMP_END_OF_CAPABILITIES_LIST()
 };
 
 static void
diff --git a/drivers/compress/qat/dev/qat_comp_pmd_gen1.c b/drivers/compress/qat/dev/qat_comp_pmd_gen1.c
index 12d9d89072..3a8484eef1 100644
--- a/drivers/compress/qat/dev/qat_comp_pmd_gen1.c
+++ b/drivers/compress/qat/dev/qat_comp_pmd_gen1.c
@@ -26,7 +26,7 @@ const struct rte_compressdev_capabilities qat_gen1_comp_capabilities[] = {
 				RTE_COMP_FF_OOP_LB_IN_SGL_OUT |
 				RTE_COMP_FF_STATEFUL_DECOMPRESSION,
 	 .window_size = {.min = 15, .max = 15, .increment = 0} },
-	{RTE_COMP_ALGO_LIST_END, 0, {0, 0, 0} } };
+	 RTE_COMP_END_OF_CAPABILITIES_LIST() };
 
 static int
 qat_comp_dev_config_gen1(struct rte_compressdev *dev,
diff --git a/drivers/compress/qat/dev/qat_comp_pmd_gen4.c b/drivers/compress/qat/dev/qat_comp_pmd_gen4.c
index 79b2ceb414..05906f13e0 100644
--- a/drivers/compress/qat/dev/qat_comp_pmd_gen4.c
+++ b/drivers/compress/qat/dev/qat_comp_pmd_gen4.c
@@ -25,7 +25,7 @@ qat_gen4_comp_capabilities[] = {
 				RTE_COMP_FF_OOP_SGL_IN_LB_OUT |
 				RTE_COMP_FF_OOP_LB_IN_SGL_OUT,
 	 .window_size = {.min = 15, .max = 15, .increment = 0} },
-	{RTE_COMP_ALGO_LIST_END, 0, {0, 0, 0} } };
+	 RTE_COMP_END_OF_CAPABILITIES_LIST() };
 
 static int
 qat_comp_dev_config_gen4(struct rte_compressdev *dev,
diff --git a/lib/compressdev/rte_compressdev.h b/lib/compressdev/rte_compressdev.h
index 42bda9fc79..7eb5c58798 100644
--- a/lib/compressdev/rte_compressdev.h
+++ b/lib/compressdev/rte_compressdev.h
@@ -353,7 +353,7 @@ rte_compressdev_stats_reset(uint8_t dev_id);
  * @note The capabilities field of dev_info is set to point to the first
  * element of an array of struct rte_compressdev_capabilities.
  * The element after the last valid element has it's op field set to
- * RTE_COMP_ALGO_LIST_END.
+ * RTE_COMP_ALGO_UNSPECIFIED.
  */
 __rte_experimental
 void
-- 
2.25.1

RE: [PATCH] compressdev: fix end of comp PMD list macro conflict

[Michael Baum]

Hi All, any comments?

From: Michael Baum <michaelba@nvidia.com> 

> The "rte_compressdev_info_get()" function retrieves the contextual information
> of a device.
> The output structure "dev_info" contains a list of devices supported capabilities
> for each supported algorithm.
> 
> In this function description, it says the element after the last valid element has
> op field set to "RTE_COMP_ALGO_LIST_END".
> On the other hand, when this function used by
> "rte_compressdev_capability_get()" function, it uses
> "RTE_COMP_ALGO_UNSPECIFIED" as end of list as same as the
> "RTE_COMP_END_OF_CAPABILITIES_LIST()".
> 
> The mlx5 and qat PMDs use "RTE_COMP_ALGO_LIST_END" as the end of
> capabilities list. When "rte_compressdev_capability_get()" function is called with
> unsupported algorithm, it might read memory out of bound.
> 
> This patch change the "rte_compressdev_info_get()" function description to say
> using "RTE_COMP_ALGO_UNSPECIFIED" as the end of capabilities list.
> In addition, it moves both mlx5 and qat PMDs to use
> "RTE_COMP_ALGO_UNSPECIFIED" through
> "RTE_COMP_END_OF_CAPABILITIES_LIST()" macro.
> 
> Fixes: 5d432f364078 ("compressdev: add device capabilities")
> Fixes: 2d148597ce76 ("compress/qat: add gen-specific implementation")
> Fixes: 384bac8d6555 ("compress/mlx5: add supported capabilities")
> Cc: fiona.trahe@intel.com
> Cc: roy.fan.zhang@intel.com
> Cc: matan@nvidia.com
> Cc: stable@dpdk.org
> 
> Signed-off-by: Michael Baum <michaelba@nvidia.com>
> 
> ---
> 
> After this change, I'm not sure about the purpose of
> "RTE_COMP_ALGO_LIST_END".
> There is no any other use of it in DPDK code, and it isn't represent the number of
> algorithms supported by the API since the "RTE_COMP_ALGO_UNSPECIFIED" is
> part of the enum.
> 
> Due to the compress API is experimental I think the
> "RTE_COMP_ALGO_LIST_END" can be removed.
> 
> 
> 
>  drivers/compress/mlx5/mlx5_compress.c        | 4 +---
>  drivers/compress/qat/dev/qat_comp_pmd_gen1.c | 2 +-
> drivers/compress/qat/dev/qat_comp_pmd_gen4.c | 2 +-
>  lib/compressdev/rte_compressdev.h            | 2 +-
>  4 files changed, 4 insertions(+), 6 deletions(-)
> 
> diff --git a/drivers/compress/mlx5/mlx5_compress.c
> b/drivers/compress/mlx5/mlx5_compress.c
> index fb2bda9745..459e4b5e8a 100644
> --- a/drivers/compress/mlx5/mlx5_compress.c
> +++ b/drivers/compress/mlx5/mlx5_compress.c
> @@ -96,9 +96,7 @@ static const struct rte_compressdev_capabilities
> mlx5_caps[] = {
>                                       RTE_COMP_FF_HUFFMAN_DYNAMIC,
>                 .window_size = {.min = 10, .max = 15, .increment = 1},
>         },
> -       {
> -               .algo = RTE_COMP_ALGO_LIST_END,
> -       }
> +       RTE_COMP_END_OF_CAPABILITIES_LIST()
>  };
> 
>  static void
> diff --git a/drivers/compress/qat/dev/qat_comp_pmd_gen1.c
> b/drivers/compress/qat/dev/qat_comp_pmd_gen1.c
> index 12d9d89072..3a8484eef1 100644
> --- a/drivers/compress/qat/dev/qat_comp_pmd_gen1.c
> +++ b/drivers/compress/qat/dev/qat_comp_pmd_gen1.c
> @@ -26,7 +26,7 @@ const struct rte_compressdev_capabilities
> qat_gen1_comp_capabilities[] = {
>                                 RTE_COMP_FF_OOP_LB_IN_SGL_OUT |
>                                 RTE_COMP_FF_STATEFUL_DECOMPRESSION,
>          .window_size = {.min = 15, .max = 15, .increment = 0} },
> -       {RTE_COMP_ALGO_LIST_END, 0, {0, 0, 0} } };
> +        RTE_COMP_END_OF_CAPABILITIES_LIST() };
> 
>  static int
>  qat_comp_dev_config_gen1(struct rte_compressdev *dev, diff --git
> a/drivers/compress/qat/dev/qat_comp_pmd_gen4.c
> b/drivers/compress/qat/dev/qat_comp_pmd_gen4.c
> index 79b2ceb414..05906f13e0 100644
> --- a/drivers/compress/qat/dev/qat_comp_pmd_gen4.c
> +++ b/drivers/compress/qat/dev/qat_comp_pmd_gen4.c
> @@ -25,7 +25,7 @@ qat_gen4_comp_capabilities[] = {
>                                 RTE_COMP_FF_OOP_SGL_IN_LB_OUT |
>                                 RTE_COMP_FF_OOP_LB_IN_SGL_OUT,
>          .window_size = {.min = 15, .max = 15, .increment = 0} },
> -       {RTE_COMP_ALGO_LIST_END, 0, {0, 0, 0} } };
> +        RTE_COMP_END_OF_CAPABILITIES_LIST() };
> 
>  static int
>  qat_comp_dev_config_gen4(struct rte_compressdev *dev, diff --git
> a/lib/compressdev/rte_compressdev.h b/lib/compressdev/rte_compressdev.h
> index 42bda9fc79..7eb5c58798 100644
> --- a/lib/compressdev/rte_compressdev.h
> +++ b/lib/compressdev/rte_compressdev.h
> @@ -353,7 +353,7 @@ rte_compressdev_stats_reset(uint8_t dev_id);
>   * @note The capabilities field of dev_info is set to point to the first
>   * element of an array of struct rte_compressdev_capabilities.
>   * The element after the last valid element has it's op field set to
> - * RTE_COMP_ALGO_LIST_END.
> + * RTE_COMP_ALGO_UNSPECIFIED.
>   */
>  __rte_experimental
>  void
> --
> 2.25.1

RE: [EXT] [PATCH] compressdev: fix end of comp PMD list macro conflict

[Akhil Goyal]

> The "rte_compressdev_info_get()" function retrieves the contextual
> information of a device.
> The output structure "dev_info" contains a list of devices supported
> capabilities for each supported algorithm.
> 
> In this function description, it says the element after the last valid
> element has op field set to "RTE_COMP_ALGO_LIST_END".
> On the other hand, when this function used by
> "rte_compressdev_capability_get()" function, it uses
> "RTE_COMP_ALGO_UNSPECIFIED" as end of list as same as the
> "RTE_COMP_END_OF_CAPABILITIES_LIST()".
> 
> The mlx5 and qat PMDs use "RTE_COMP_ALGO_LIST_END" as the end of
> capabilities list. When "rte_compressdev_capability_get()" function is
> called with unsupported algorithm, it might read memory out of bound.
> 
> This patch change the "rte_compressdev_info_get()" function description
> to say using "RTE_COMP_ALGO_UNSPECIFIED" as the end of capabilities
> list.
> In addition, it moves both mlx5 and qat PMDs to use
> "RTE_COMP_ALGO_UNSPECIFIED" through
> "RTE_COMP_END_OF_CAPABILITIES_LIST()" macro.
> 
> Fixes: 5d432f364078 ("compressdev: add device capabilities")
> Fixes: 2d148597ce76 ("compress/qat: add gen-specific implementation")
> Fixes: 384bac8d6555 ("compress/mlx5: add supported capabilities")
> Cc: fiona.trahe@intel.com
> Cc: roy.fan.zhang@intel.com
> Cc: matan@nvidia.com
> Cc: stable@dpdk.org
> 
> Signed-off-by: Michael Baum <michaelba@nvidia.com>
> 
> ---
> 
> After this change, I'm not sure about the purpose of
> "RTE_COMP_ALGO_LIST_END".
> There is no any other use of it in DPDK code, and it isn't represent the
> number of algorithms supported by the API since the
> "RTE_COMP_ALGO_UNSPECIFIED" is part of the enum.
> 
> Due to the compress API is experimental I think the
> "RTE_COMP_ALGO_LIST_END" can be removed.
> 
+1 to remove the list end enums. This will also help in avoiding ABI breakage
When we make this lib as stable.

RE: [EXT] [PATCH] compressdev: fix end of comp PMD list macro conflict

[Akhil Goyal]

> Subject: RE: [EXT] [PATCH] compressdev: fix end of comp PMD list macro
> conflict
> 
> > The "rte_compressdev_info_get()" function retrieves the contextual
> > information of a device.
> > The output structure "dev_info" contains a list of devices supported
> > capabilities for each supported algorithm.
> >
> > In this function description, it says the element after the last valid
> > element has op field set to "RTE_COMP_ALGO_LIST_END".
> > On the other hand, when this function used by
> > "rte_compressdev_capability_get()" function, it uses
> > "RTE_COMP_ALGO_UNSPECIFIED" as end of list as same as the
> > "RTE_COMP_END_OF_CAPABILITIES_LIST()".
> >
> > The mlx5 and qat PMDs use "RTE_COMP_ALGO_LIST_END" as the end of
> > capabilities list. When "rte_compressdev_capability_get()" function is
> > called with unsupported algorithm, it might read memory out of bound.
> >
> > This patch change the "rte_compressdev_info_get()" function description
> > to say using "RTE_COMP_ALGO_UNSPECIFIED" as the end of capabilities
> > list.
> > In addition, it moves both mlx5 and qat PMDs to use
> > "RTE_COMP_ALGO_UNSPECIFIED" through
> > "RTE_COMP_END_OF_CAPABILITIES_LIST()" macro.
> >
> > Fixes: 5d432f364078 ("compressdev: add device capabilities")
> > Fixes: 2d148597ce76 ("compress/qat: add gen-specific implementation")
> > Fixes: 384bac8d6555 ("compress/mlx5: add supported capabilities")
> > Cc: fiona.trahe@intel.com
> > Cc: roy.fan.zhang@intel.com
> > Cc: matan@nvidia.com
> > Cc: stable@dpdk.org
> >
> > Signed-off-by: Michael Baum <michaelba@nvidia.com>
> >
> > ---
> >
> > After this change, I'm not sure about the purpose of
> > "RTE_COMP_ALGO_LIST_END".
> > There is no any other use of it in DPDK code, and it isn't represent the
> > number of algorithms supported by the API since the
> > "RTE_COMP_ALGO_UNSPECIFIED" is part of the enum.
> >
> > Due to the compress API is experimental I think the
> > "RTE_COMP_ALGO_LIST_END" can be removed.
> >
> +1 to remove the list end enums. This will also help in avoiding ABI breakage
> When we make this lib as stable.

Even RTE_COMP_HASH_ALGO_LIST_END can also be removed.
It is not used anywhere.

RE: [EXT] [PATCH] compressdev: fix end of comp PMD list macro conflict

[Akhil Goyal]

Hi,
> > >
> > > After this change, I'm not sure about the purpose of
> > > "RTE_COMP_ALGO_LIST_END".
> > > There is no any other use of it in DPDK code, and it isn't represent the
> > > number of algorithms supported by the API since the
> > > "RTE_COMP_ALGO_UNSPECIFIED" is part of the enum.
> > >
> > > Due to the compress API is experimental I think the
> > > "RTE_COMP_ALGO_LIST_END" can be removed.
> > >
> > +1 to remove the list end enums. This will also help in avoiding ABI breakage
> > When we make this lib as stable.
> 
> Even RTE_COMP_HASH_ALGO_LIST_END can also be removed.
> It is not used anywhere.

Can you send a patch to remove these list end enums along with this patch?

-Akhil

RE: [EXT] [PATCH] compressdev: fix end of comp PMD list macro conflict

[Michael Baum]

Hi,
> 
> 
> Hi,
> > > >
> > > > After this change, I'm not sure about the purpose of
> > > > "RTE_COMP_ALGO_LIST_END".
> > > > There is no any other use of it in DPDK code, and it isn't
> > > > represent the number of algorithms supported by the API since the
> > > > "RTE_COMP_ALGO_UNSPECIFIED" is part of the enum.
> > > >
> > > > Due to the compress API is experimental I think the
> > > > "RTE_COMP_ALGO_LIST_END" can be removed.
> > > >
> > > +1 to remove the list end enums. This will also help in avoiding ABI
> > > +breakage
> > > When we make this lib as stable.
> >
> > Even RTE_COMP_HASH_ALGO_LIST_END can also be removed.
> > It is not used anywhere.
> 
> Can you send a patch to remove these list end enums along with this patch?

In the same patch? Or add one?
> 
> -Akhil

RE: [EXT] [PATCH] compressdev: fix end of comp PMD list macro conflict

[Akhil Goyal]

> > Hi,
> > > > >
> > > > > After this change, I'm not sure about the purpose of
> > > > > "RTE_COMP_ALGO_LIST_END".
> > > > > There is no any other use of it in DPDK code, and it isn't
> > > > > represent the number of algorithms supported by the API since the
> > > > > "RTE_COMP_ALGO_UNSPECIFIED" is part of the enum.
> > > > >
> > > > > Due to the compress API is experimental I think the
> > > > > "RTE_COMP_ALGO_LIST_END" can be removed.
> > > > >
> > > > +1 to remove the list end enums. This will also help in avoiding ABI
> > > > +breakage
> > > > When we make this lib as stable.
> > >
> > > Even RTE_COMP_HASH_ALGO_LIST_END can also be removed.
> > > It is not used anywhere.
> >
> > Can you send a patch to remove these list end enums along with this patch?
> 
> In the same patch? Or add one?
Separate patch would be better as the current patch is talking about a conflict.
Removing the enums need not be backported, but this patch is required to be backported.

[PATCH v2 1/2] compressdev: fix end of comp PMD list macro conflict

[Michael Baum]

The "rte_compressdev_info_get()" function retrieves the contextual
information of a device.
The output structure "dev_info" contains a list of devices supported
capabilities for each supported algorithm.

In this function description, it says the element after the last valid
element has op field set to "RTE_COMP_ALGO_LIST_END".
On the other hand, when this function used by
"rte_compressdev_capability_get()" function, it uses
"RTE_COMP_ALGO_UNSPECIFIED" as end of list as same as the
"RTE_COMP_END_OF_CAPABILITIES_LIST()".

The mlx5 and qat PMDs use "RTE_COMP_ALGO_LIST_END" as the end of
capabilities list. When "rte_compressdev_capability_get()" function is
called with unsupported algorithm, it might read memory out of bound.

This patch change the "rte_compressdev_info_get()" function description
to say using "RTE_COMP_ALGO_UNSPECIFIED" as the end of capabilities
list.
In addition, it moves both mlx5 and qat PMDs to use
"RTE_COMP_ALGO_UNSPECIFIED" through
"RTE_COMP_END_OF_CAPABILITIES_LIST()" macro.

Fixes: 5d432f364078 ("compressdev: add device capabilities")
Fixes: 2d148597ce76 ("compress/qat: add gen-specific implementation")
Fixes: 384bac8d6555 ("compress/mlx5: add supported capabilities")
Cc: fiona.trahe@intel.com
Cc: roy.fan.zhang@intel.com
Cc: matan@nvidia.com
Cc: stable@dpdk.org

Signed-off-by: Michael Baum <michaelba@nvidia.com>
---
 drivers/compress/mlx5/mlx5_compress.c        | 4 +---
 drivers/compress/qat/dev/qat_comp_pmd_gen1.c | 2 +-
 drivers/compress/qat/dev/qat_comp_pmd_gen4.c | 2 +-
 lib/compressdev/rte_compressdev.h            | 2 +-
 4 files changed, 4 insertions(+), 6 deletions(-)

diff --git a/drivers/compress/mlx5/mlx5_compress.c b/drivers/compress/mlx5/mlx5_compress.c
index fb2bda9745..459e4b5e8a 100644
--- a/drivers/compress/mlx5/mlx5_compress.c
+++ b/drivers/compress/mlx5/mlx5_compress.c
@@ -96,9 +96,7 @@ static const struct rte_compressdev_capabilities mlx5_caps[] = {
 				      RTE_COMP_FF_HUFFMAN_DYNAMIC,
 		.window_size = {.min = 10, .max = 15, .increment = 1},
 	},
-	{
-		.algo = RTE_COMP_ALGO_LIST_END,
-	}
+	RTE_COMP_END_OF_CAPABILITIES_LIST()
 };
 
 static void
diff --git a/drivers/compress/qat/dev/qat_comp_pmd_gen1.c b/drivers/compress/qat/dev/qat_comp_pmd_gen1.c
index 12d9d89072..3a8484eef1 100644
--- a/drivers/compress/qat/dev/qat_comp_pmd_gen1.c
+++ b/drivers/compress/qat/dev/qat_comp_pmd_gen1.c
@@ -26,7 +26,7 @@ const struct rte_compressdev_capabilities qat_gen1_comp_capabilities[] = {
 				RTE_COMP_FF_OOP_LB_IN_SGL_OUT |
 				RTE_COMP_FF_STATEFUL_DECOMPRESSION,
 	 .window_size = {.min = 15, .max = 15, .increment = 0} },
-	{RTE_COMP_ALGO_LIST_END, 0, {0, 0, 0} } };
+	 RTE_COMP_END_OF_CAPABILITIES_LIST() };
 
 static int
 qat_comp_dev_config_gen1(struct rte_compressdev *dev,
diff --git a/drivers/compress/qat/dev/qat_comp_pmd_gen4.c b/drivers/compress/qat/dev/qat_comp_pmd_gen4.c
index 79b2ceb414..05906f13e0 100644
--- a/drivers/compress/qat/dev/qat_comp_pmd_gen4.c
+++ b/drivers/compress/qat/dev/qat_comp_pmd_gen4.c
@@ -25,7 +25,7 @@ qat_gen4_comp_capabilities[] = {
 				RTE_COMP_FF_OOP_SGL_IN_LB_OUT |
 				RTE_COMP_FF_OOP_LB_IN_SGL_OUT,
 	 .window_size = {.min = 15, .max = 15, .increment = 0} },
-	{RTE_COMP_ALGO_LIST_END, 0, {0, 0, 0} } };
+	 RTE_COMP_END_OF_CAPABILITIES_LIST() };
 
 static int
 qat_comp_dev_config_gen4(struct rte_compressdev *dev,
diff --git a/lib/compressdev/rte_compressdev.h b/lib/compressdev/rte_compressdev.h
index 42bda9fc79..7eb5c58798 100644
--- a/lib/compressdev/rte_compressdev.h
+++ b/lib/compressdev/rte_compressdev.h
@@ -353,7 +353,7 @@ rte_compressdev_stats_reset(uint8_t dev_id);
  * @note The capabilities field of dev_info is set to point to the first
  * element of an array of struct rte_compressdev_capabilities.
  * The element after the last valid element has it's op field set to
- * RTE_COMP_ALGO_LIST_END.
+ * RTE_COMP_ALGO_UNSPECIFIED.
  */
 __rte_experimental
 void
-- 
2.25.1