[PATCH] examples/ipsec-secgw: fix cryptodev to SA mapping

[PATCH] examples/ipsec-secgw: fix cryptodev to SA mapping

[Radu Nicolau]

There are use cases where a SA should be able to use different cryptodevs on
different lcores, for example there can be cryptodevs with just 1 qp per VF.
For this purpose this patch relaxes the check in create lookaside session function.
Also add a check to verify that a CQP is available for the current lcore.

Fixes: a8ade12123c3 ("examples/ipsec-secgw: create lookaside sessions at init")
Cc: stable@dpdk.org
Cc: vfialko@marvell.com

Signed-off-by: Radu Nicolau <radu.nicolau@intel.com>
---
 examples/ipsec-secgw/ipsec.c | 13 ++++++++-----
 1 file changed, 8 insertions(+), 5 deletions(-)

diff --git a/examples/ipsec-secgw/ipsec.c b/examples/ipsec-secgw/ipsec.c
index f5cec4a928..593eab4e73 100644
--- a/examples/ipsec-secgw/ipsec.c
+++ b/examples/ipsec-secgw/ipsec.c
@@ -288,10 +288,9 @@ create_lookaside_session(struct ipsec_ctx *ipsec_ctx_lcore[],
 		if (cdev_id == RTE_CRYPTO_MAX_DEVS)
 			cdev_id = ipsec_ctx->tbl[cdev_id_qp].id;
 		else if (cdev_id != ipsec_ctx->tbl[cdev_id_qp].id) {
-			RTE_LOG(ERR, IPSEC,
-					"SA mapping to multiple cryptodevs is "
-					"not supported!");
-			return -EINVAL;
+			RTE_LOG(WARNING, IPSEC,
+				"SA mapped to multiple cryptodevs for SPI %d\n",
+				sa->spi);
 		}
 
 		/* Store per core queue pair information */
@@ -908,7 +907,11 @@ ipsec_enqueue(ipsec_xform_fn xform_func, struct ipsec_ctx *ipsec_ctx,
 			continue;
 		}
 
-		enqueue_cop(sa->cqp[ipsec_ctx->lcore_id], &priv->cop);
+		if (sa->cqp[ipsec_ctx->lcore_id])
+			enqueue_cop(sa->cqp[ipsec_ctx->lcore_id], &priv->cop);
+		else
+			RTE_LOG(ERR, IPSEC, "No CQP available for lcore %d\n",
+					ipsec_ctx->lcore_id);
 	}
 }
 
-- 
2.25.1

RE: [PATCH] examples/ipsec-secgw: fix cryptodev to SA mapping

[Ku, Ting-Kai]

Fixes: a8ade12123c3 ("examples/ipsec-secgw: create lookaside sessions at init")
Cc: stable@dpdk.org
Cc: vfialko@marvell.com

Signed-off-by: Radu Nicolau <radu.nicolau@intel.com>
Tested-by: Ting-Kai Ku <ting-kai.ku@intel.com>
---
 examples/ipsec-secgw/ipsec.c | 13 ++++++++-----
 1 file changed, 8 insertions(+), 5 deletions(-)

diff --git a/examples/ipsec-secgw/ipsec.c b/examples/ipsec-secgw/ipsec.c index f5cec4a928..593eab4e73 100644
--- a/examples/ipsec-secgw/ipsec.c
+++ b/examples/ipsec-secgw/ipsec.c
@@ -288,10 +288,9 @@ create_lookaside_session(struct ipsec_ctx *ipsec_ctx_lcore[],
 		if (cdev_id == RTE_CRYPTO_MAX_DEVS)
 			cdev_id = ipsec_ctx->tbl[cdev_id_qp].id;
 		else if (cdev_id != ipsec_ctx->tbl[cdev_id_qp].id) {
-			RTE_LOG(ERR, IPSEC,
-					"SA mapping to multiple cryptodevs is "
-					"not supported!");
-			return -EINVAL;
+			RTE_LOG(WARNING, IPSEC,
+				"SA mapped to multiple cryptodevs for SPI %d\n",
+				sa->spi);
 		}
 
 		/* Store per core queue pair information */ @@ -908,7 +907,11 @@ ipsec_enqueue(ipsec_xform_fn xform_func, struct ipsec_ctx *ipsec_ctx,
 			continue;
 		}
 
-		enqueue_cop(sa->cqp[ipsec_ctx->lcore_id], &priv->cop);
+		if (sa->cqp[ipsec_ctx->lcore_id])
+			enqueue_cop(sa->cqp[ipsec_ctx->lcore_id], &priv->cop);
+		else
+			RTE_LOG(ERR, IPSEC, "No CQP available for lcore %d\n",
+					ipsec_ctx->lcore_id);
 	}
 }
 
--
2.25.1

-----Original Message-----
From: Nicolau, Radu <radu.nicolau@intel.com> 
Sent: Monday, December 11, 2023 5:54 PM
To: Nicolau, Radu <radu.nicolau@intel.com>; Akhil Goyal <gakhil@marvell.com>
Cc: dev@dpdk.org; Power, Ciara <ciara.power@intel.com>; Ku, Ting-Kai <ting-kai.ku@intel.com>; stable@dpdk.org; vfialko@marvell.com
Subject: [PATCH] examples/ipsec-secgw: fix cryptodev to SA mapping

There are use cases where a SA should be able to use different cryptodevs on different lcores, for example there can be cryptodevs with just 1 qp per VF.
For this purpose this patch relaxes the check in create lookaside session function.
Also add a check to verify that a CQP is available for the current lcore.

Fixes: a8ade12123c3 ("examples/ipsec-secgw: create lookaside sessions at init")
Cc: stable@dpdk.org
Cc: vfialko@marvell.com

Signed-off-by: Radu Nicolau <radu.nicolau@intel.com>
---
 examples/ipsec-secgw/ipsec.c | 13 ++++++++-----
 1 file changed, 8 insertions(+), 5 deletions(-)

diff --git a/examples/ipsec-secgw/ipsec.c b/examples/ipsec-secgw/ipsec.c index f5cec4a928..593eab4e73 100644
--- a/examples/ipsec-secgw/ipsec.c
+++ b/examples/ipsec-secgw/ipsec.c
@@ -288,10 +288,9 @@ create_lookaside_session(struct ipsec_ctx *ipsec_ctx_lcore[],
 		if (cdev_id == RTE_CRYPTO_MAX_DEVS)
 			cdev_id = ipsec_ctx->tbl[cdev_id_qp].id;
 		else if (cdev_id != ipsec_ctx->tbl[cdev_id_qp].id) {
-			RTE_LOG(ERR, IPSEC,
-					"SA mapping to multiple cryptodevs is "
-					"not supported!");
-			return -EINVAL;
+			RTE_LOG(WARNING, IPSEC,
+				"SA mapped to multiple cryptodevs for SPI %d\n",
+				sa->spi);
 		}
 
 		/* Store per core queue pair information */ @@ -908,7 +907,11 @@ ipsec_enqueue(ipsec_xform_fn xform_func, struct ipsec_ctx *ipsec_ctx,
 			continue;
 		}
 
-		enqueue_cop(sa->cqp[ipsec_ctx->lcore_id], &priv->cop);
+		if (sa->cqp[ipsec_ctx->lcore_id])
+			enqueue_cop(sa->cqp[ipsec_ctx->lcore_id], &priv->cop);
+		else
+			RTE_LOG(ERR, IPSEC, "No CQP available for lcore %d\n",
+					ipsec_ctx->lcore_id);
 	}
 }
 
--
2.25.1

RE: [PATCH] examples/ipsec-secgw: fix cryptodev to SA mapping

[Power, Ciara]

Hi Radu,

> -----Original Message-----
> From: Nicolau, Radu <radu.nicolau@intel.com>
> Sent: Monday, December 11, 2023 9:54 AM
> To: Nicolau, Radu <radu.nicolau@intel.com>; Akhil Goyal
> <gakhil@marvell.com>
> Cc: dev@dpdk.org; Power, Ciara <ciara.power@intel.com>; Ku, Ting-Kai <ting-
> kai.ku@intel.com>; stable@dpdk.org; vfialko@marvell.com
> Subject: [PATCH] examples/ipsec-secgw: fix cryptodev to SA mapping
> 
> There are use cases where a SA should be able to use different cryptodevs on
> different lcores, for example there can be cryptodevs with just 1 qp per VF.
> For this purpose this patch relaxes the check in create lookaside session
> function.
> Also add a check to verify that a CQP is available for the current lcore.
> 
> Fixes: a8ade12123c3 ("examples/ipsec-secgw: create lookaside sessions at
> init")
> Cc: stable@dpdk.org
> Cc: vfialko@marvell.com
> 
> Signed-off-by: Radu Nicolau <radu.nicolau@intel.com>
> ---
>  examples/ipsec-secgw/ipsec.c | 13 ++++++++-----
>  1 file changed, 8 insertions(+), 5 deletions(-)

Acked-by: Ciara Power <ciara.power@intel.com>

Re: [PATCH] examples/ipsec-secgw: fix cryptodev to SA mapping

[Ji, Kai]

[-- Attachment #1: Type: text/plain, Size: 1007 bytes --]

Acked-by: Kai Ji <kai.ji@intel.com>


________________________________
From: Radu Nicolau <radu.nicolau@intel.com>
Sent: 11 December 2023 09:53
To: Nicolau, Radu <radu.nicolau@intel.com>; Akhil Goyal <gakhil@marvell.com>
Cc: dev@dpdk.org <dev@dpdk.org>; Power, Ciara <ciara.power@intel.com>; Ku, Ting-Kai <ting-kai.ku@intel.com>; stable@dpdk.org <stable@dpdk.org>; vfialko@marvell.com <vfialko@marvell.com>
Subject: [PATCH] examples/ipsec-secgw: fix cryptodev to SA mapping

There are use cases where a SA should be able to use different cryptodevs on
different lcores, for example there can be cryptodevs with just 1 qp per VF.
For this purpose this patch relaxes the check in create lookaside session function.
Also add a check to verify that a CQP is available for the current lcore.

Fixes: a8ade12123c3 ("examples/ipsec-secgw: create lookaside sessions at init")
Cc: stable@dpdk.org
Cc: vfialko@marvell.com

Signed-off-by: Radu Nicolau <radu.nicolau@intel.com>
---

--
2.25.1


[-- Attachment #2: Type: text/html, Size: 2025 bytes --]

RE: [EXT] [PATCH] examples/ipsec-secgw: fix cryptodev to SA mapping

[Akhil Goyal]

> @@ -908,7 +907,11 @@ ipsec_enqueue(ipsec_xform_fn xform_func, struct
> ipsec_ctx *ipsec_ctx,
>  			continue;
>  		}
> 
> -		enqueue_cop(sa->cqp[ipsec_ctx->lcore_id], &priv->cop);
> +		if (sa->cqp[ipsec_ctx->lcore_id])
> +			enqueue_cop(sa->cqp[ipsec_ctx->lcore_id], &priv->cop);
> +		else
> +			RTE_LOG(ERR, IPSEC, "No CQP available for lcore %d\n",
> +					ipsec_ctx->lcore_id);

Can we add likely here?

[PATCH v2] examples/ipsec-secgw: fix cryptodev to SA mapping

[Radu Nicolau]

There are use cases where a SA should be able to use different cryptodevs on
different lcores, for example there can be cryptodevs with just 1 qp per VF.
For this purpose this patch relaxes the check in create lookaside session function.
Also add a check to verify that a CQP is available for the current lcore.

Fixes: a8ade12123c3 ("examples/ipsec-secgw: create lookaside sessions at init")
Cc: stable@dpdk.org
Cc: vfialko@marvell.com

Signed-off-by: Radu Nicolau <radu.nicolau@intel.com>
Signed-off-by: Radu Nicolau <radu.nicolau@intel.com>
Tested-by: Ting-Kai Ku <ting-kai.ku@intel.com>
Signed-off-by: Radu Nicolau <radu.nicolau@intel.com>
Acked-by: Ciara Power <ciara.power@intel.com>
Acked-by: Kai Ji <kai.ji@intel.com>
---
v2: add likely to CQP available branch

 examples/ipsec-secgw/ipsec.c | 13 ++++++++-----
 1 file changed, 8 insertions(+), 5 deletions(-)

diff --git a/examples/ipsec-secgw/ipsec.c b/examples/ipsec-secgw/ipsec.c
index f5cec4a928..7bb9646736 100644
--- a/examples/ipsec-secgw/ipsec.c
+++ b/examples/ipsec-secgw/ipsec.c
@@ -288,10 +288,9 @@ create_lookaside_session(struct ipsec_ctx *ipsec_ctx_lcore[],
 		if (cdev_id == RTE_CRYPTO_MAX_DEVS)
 			cdev_id = ipsec_ctx->tbl[cdev_id_qp].id;
 		else if (cdev_id != ipsec_ctx->tbl[cdev_id_qp].id) {
-			RTE_LOG(ERR, IPSEC,
-					"SA mapping to multiple cryptodevs is "
-					"not supported!");
-			return -EINVAL;
+			RTE_LOG(WARNING, IPSEC,
+				"SA mapped to multiple cryptodevs for SPI %d\n",
+				sa->spi);
 		}
 
 		/* Store per core queue pair information */
@@ -908,7 +907,11 @@ ipsec_enqueue(ipsec_xform_fn xform_func, struct ipsec_ctx *ipsec_ctx,
 			continue;
 		}
 
-		enqueue_cop(sa->cqp[ipsec_ctx->lcore_id], &priv->cop);
+		if (likely(sa->cqp[ipsec_ctx->lcore_id]))
+			enqueue_cop(sa->cqp[ipsec_ctx->lcore_id], &priv->cop);
+		else
+			RTE_LOG(ERR, IPSEC, "No CQP available for lcore %d\n",
+					ipsec_ctx->lcore_id);
 	}
 }
 
-- 
2.34.1

RE: [EXT] [PATCH v2] examples/ipsec-secgw: fix cryptodev to SA mapping

[Anoob Joseph]

Hi Radu,

Using same session across multiple devices would only work if the device instances are same. For example, if you use QAT device 0 & QAT device 1, then the sharing is okay. But if we have QAT and some SW crypto PMD, then the same session will not work. Is it possible to throw a warning when the crypto devices are not of same type?

Thanks,
Anoob

> -----Original Message-----
> From: Radu Nicolau <radu.nicolau@intel.com>
> Sent: Friday, February 23, 2024 4:31 PM
> To: dev@dpdk.org
> Cc: Radu Nicolau <radu.nicolau@intel.com>; stable@dpdk.org; Volodymyr Fialko
> <vfialko@marvell.com>; Ting-Kai Ku <ting-kai.ku@intel.com>; Ciara Power
> <ciara.power@intel.com>; Kai Ji <kai.ji@intel.com>; Akhil Goyal
> <gakhil@marvell.com>
> Subject: [EXT] [PATCH v2] examples/ipsec-secgw: fix cryptodev to SA mapping
> 
> External Email
> 
> ----------------------------------------------------------------------
> There are use cases where a SA should be able to use different cryptodevs on
> different lcores, for example there can be cryptodevs with just 1 qp per VF.
> For this purpose this patch relaxes the check in create lookaside session function.
> Also add a check to verify that a CQP is available for the current lcore.
> 
> Fixes: a8ade12123c3 ("examples/ipsec-secgw: create lookaside sessions at init")
> Cc: stable@dpdk.org
> Cc: vfialko@marvell.com
> 
> Signed-off-by: Radu Nicolau <radu.nicolau@intel.com>
> Signed-off-by: Radu Nicolau <radu.nicolau@intel.com>
> Tested-by: Ting-Kai Ku <ting-kai.ku@intel.com>
> Signed-off-by: Radu Nicolau <radu.nicolau@intel.com>
> Acked-by: Ciara Power <ciara.power@intel.com>
> Acked-by: Kai Ji <kai.ji@intel.com>
> ---
> v2: add likely to CQP available branch
> 
>  examples/ipsec-secgw/ipsec.c | 13 ++++++++-----
>  1 file changed, 8 insertions(+), 5 deletions(-)
> 
> diff --git a/examples/ipsec-secgw/ipsec.c b/examples/ipsec-secgw/ipsec.c index
> f5cec4a928..7bb9646736 100644
> --- a/examples/ipsec-secgw/ipsec.c
> +++ b/examples/ipsec-secgw/ipsec.c
> @@ -288,10 +288,9 @@ create_lookaside_session(struct ipsec_ctx
> *ipsec_ctx_lcore[],
>  		if (cdev_id == RTE_CRYPTO_MAX_DEVS)
>  			cdev_id = ipsec_ctx->tbl[cdev_id_qp].id;
>  		else if (cdev_id != ipsec_ctx->tbl[cdev_id_qp].id) {
> -			RTE_LOG(ERR, IPSEC,
> -					"SA mapping to multiple cryptodevs is "
> -					"not supported!");
> -			return -EINVAL;
> +			RTE_LOG(WARNING, IPSEC,
> +				"SA mapped to multiple cryptodevs for SPI
> %d\n",
> +				sa->spi);
>  		}
> 
>  		/* Store per core queue pair information */ @@ -908,7 +907,11
> @@ ipsec_enqueue(ipsec_xform_fn xform_func, struct ipsec_ctx *ipsec_ctx,
>  			continue;
>  		}
> 
> -		enqueue_cop(sa->cqp[ipsec_ctx->lcore_id], &priv->cop);
> +		if (likely(sa->cqp[ipsec_ctx->lcore_id]))
> +			enqueue_cop(sa->cqp[ipsec_ctx->lcore_id], &priv->cop);
> +		else
> +			RTE_LOG(ERR, IPSEC, "No CQP available for lcore %d\n",
> +					ipsec_ctx->lcore_id);
>  	}
>  }
> 
> --
> 2.34.1

[PATCH v3] examples/ipsec-secgw: fix cryptodev to SA mapping

[Radu Nicolau]

There are use cases where a SA should be able to use different cryptodevs on
different lcores, for example there can be cryptodevs with just 1 qp per VF.
For this purpose this patch relaxes the check in create lookaside session function.
Also add a check to verify that a CQP is available for the current lcore.

Fixes: a8ade12123c3 ("examples/ipsec-secgw: create lookaside sessions at init")
Cc: stable@dpdk.org
Cc: vfialko@marvell.com

Signed-off-by: Radu Nicolau <radu.nicolau@intel.com>
Tested-by: Ting-Kai Ku <ting-kai.ku@intel.com>
Acked-by: Ciara Power <ciara.power@intel.com>
Acked-by: Kai Ji <kai.ji@intel.com>
---
v3: check if the cryptodev are not of the same type

 examples/ipsec-secgw/ipsec.c | 25 ++++++++++++++++++++-----
 1 file changed, 20 insertions(+), 5 deletions(-)

diff --git a/examples/ipsec-secgw/ipsec.c b/examples/ipsec-secgw/ipsec.c
index f5cec4a928..b59576c049 100644
--- a/examples/ipsec-secgw/ipsec.c
+++ b/examples/ipsec-secgw/ipsec.c
@@ -288,10 +288,21 @@ create_lookaside_session(struct ipsec_ctx *ipsec_ctx_lcore[],
 		if (cdev_id == RTE_CRYPTO_MAX_DEVS)
 			cdev_id = ipsec_ctx->tbl[cdev_id_qp].id;
 		else if (cdev_id != ipsec_ctx->tbl[cdev_id_qp].id) {
-			RTE_LOG(ERR, IPSEC,
-					"SA mapping to multiple cryptodevs is "
-					"not supported!");
-			return -EINVAL;
+			struct rte_cryptodev_info dev_info_1, dev_info_2;
+			rte_cryptodev_info_get(cdev_id, &dev_info_1);
+			rte_cryptodev_info_get(ipsec_ctx->tbl[cdev_id_qp].id,
+					&dev_info_2);
+			if (dev_info_1.driver_id == dev_info_2.driver_id) {
+				RTE_LOG(WARNING, IPSEC,
+					"SA mapped to multiple cryptodevs for SPI %d\n",
+					sa->spi);
+
+			} else {
+				RTE_LOG(WARNING, IPSEC,
+					"SA mapped to multiple cryptodevs of different types for SPI %d\n",
+					sa->spi);
+
+			}
 		}
 
 		/* Store per core queue pair information */
@@ -908,7 +919,11 @@ ipsec_enqueue(ipsec_xform_fn xform_func, struct ipsec_ctx *ipsec_ctx,
 			continue;
 		}
 
-		enqueue_cop(sa->cqp[ipsec_ctx->lcore_id], &priv->cop);
+		if (likely(sa->cqp[ipsec_ctx->lcore_id]))
+			enqueue_cop(sa->cqp[ipsec_ctx->lcore_id], &priv->cop);
+		else
+			RTE_LOG(ERR, IPSEC, "No CQP available for lcore %d\n",
+					ipsec_ctx->lcore_id);
 	}
 }
 
-- 
2.34.1

Re: [PATCH v3] examples/ipsec-secgw: fix cryptodev to SA mapping

[Patrick Robb]

[-- Attachment #1: Type: text/plain, Size: 3320 bytes --]

Recheck-request: iol-broadcom-Performance

This patch should not fail a performance test in CI - checking with a rerun
now.

On Mon, Feb 26, 2024 at 5:25 AM Radu Nicolau <radu.nicolau@intel.com> wrote:

> There are use cases where a SA should be able to use different cryptodevs
> on
> different lcores, for example there can be cryptodevs with just 1 qp per
> VF.
> For this purpose this patch relaxes the check in create lookaside session
> function.
> Also add a check to verify that a CQP is available for the current lcore.
>
> Fixes: a8ade12123c3 ("examples/ipsec-secgw: create lookaside sessions at
> init")
> Cc: stable@dpdk.org
> Cc: vfialko@marvell.com
>
> Signed-off-by: Radu Nicolau <radu.nicolau@intel.com>
> Tested-by: Ting-Kai Ku <ting-kai.ku@intel.com>
> Acked-by: Ciara Power <ciara.power@intel.com>
> Acked-by: Kai Ji <kai.ji@intel.com>
> ---
> v3: check if the cryptodev are not of the same type
>
>  examples/ipsec-secgw/ipsec.c | 25 ++++++++++++++++++++-----
>  1 file changed, 20 insertions(+), 5 deletions(-)
>
> diff --git a/examples/ipsec-secgw/ipsec.c b/examples/ipsec-secgw/ipsec.c
> index f5cec4a928..b59576c049 100644
> --- a/examples/ipsec-secgw/ipsec.c
> +++ b/examples/ipsec-secgw/ipsec.c
> @@ -288,10 +288,21 @@ create_lookaside_session(struct ipsec_ctx
> *ipsec_ctx_lcore[],
>                 if (cdev_id == RTE_CRYPTO_MAX_DEVS)
>                         cdev_id = ipsec_ctx->tbl[cdev_id_qp].id;
>                 else if (cdev_id != ipsec_ctx->tbl[cdev_id_qp].id) {
> -                       RTE_LOG(ERR, IPSEC,
> -                                       "SA mapping to multiple cryptodevs
> is "
> -                                       "not supported!");
> -                       return -EINVAL;
> +                       struct rte_cryptodev_info dev_info_1, dev_info_2;
> +                       rte_cryptodev_info_get(cdev_id, &dev_info_1);
> +
>  rte_cryptodev_info_get(ipsec_ctx->tbl[cdev_id_qp].id,
> +                                       &dev_info_2);
> +                       if (dev_info_1.driver_id == dev_info_2.driver_id) {
> +                               RTE_LOG(WARNING, IPSEC,
> +                                       "SA mapped to multiple cryptodevs
> for SPI %d\n",
> +                                       sa->spi);
> +
> +                       } else {
> +                               RTE_LOG(WARNING, IPSEC,
> +                                       "SA mapped to multiple cryptodevs
> of different types for SPI %d\n",
> +                                       sa->spi);
> +
> +                       }
>                 }
>
>                 /* Store per core queue pair information */
> @@ -908,7 +919,11 @@ ipsec_enqueue(ipsec_xform_fn xform_func, struct
> ipsec_ctx *ipsec_ctx,
>                         continue;
>                 }
>
> -               enqueue_cop(sa->cqp[ipsec_ctx->lcore_id], &priv->cop);
> +               if (likely(sa->cqp[ipsec_ctx->lcore_id]))
> +                       enqueue_cop(sa->cqp[ipsec_ctx->lcore_id],
> &priv->cop);
> +               else
> +                       RTE_LOG(ERR, IPSEC, "No CQP available for lcore
> %d\n",
> +                                       ipsec_ctx->lcore_id);
>         }
>  }
>
> --
> 2.34.1
>
>

[-- Attachment #2: Type: text/html, Size: 4685 bytes --]

RE: [EXT] [PATCH v3] examples/ipsec-secgw: fix cryptodev to SA mapping

[Anoob Joseph]

Hi Radu,

Thanks for making the changes. I've one more question. Please see inline.

Thanks,
Anoob

> -----Original Message-----
> From: Radu Nicolau <radu.nicolau@intel.com>
> Sent: Monday, February 26, 2024 3:56 PM
> To: dev@dpdk.org
> Cc: Anoob Joseph <anoobj@marvell.com>; Radu Nicolau
> <radu.nicolau@intel.com>; stable@dpdk.org; Volodymyr Fialko
> <vfialko@marvell.com>; Ting-Kai Ku <ting-kai.ku@intel.com>; Ciara Power
> <ciara.power@intel.com>; Kai Ji <kai.ji@intel.com>; Akhil Goyal
> <gakhil@marvell.com>
> Subject: [EXT] [PATCH v3] examples/ipsec-secgw: fix cryptodev to SA mapping
> 
> External Email
> 
> ----------------------------------------------------------------------
> There are use cases where a SA should be able to use different cryptodevs on
> different lcores, for example there can be cryptodevs with just 1 qp per VF.
> For this purpose this patch relaxes the check in create lookaside session function.
> Also add a check to verify that a CQP is available for the current lcore.
> 
> Fixes: a8ade12123c3 ("examples/ipsec-secgw: create lookaside sessions at init")
> Cc: stable@dpdk.org
> Cc: vfialko@marvell.com
> 
> Signed-off-by: Radu Nicolau <radu.nicolau@intel.com>
> Tested-by: Ting-Kai Ku <ting-kai.ku@intel.com>
> Acked-by: Ciara Power <ciara.power@intel.com>
> Acked-by: Kai Ji <kai.ji@intel.com>
> ---
> v3: check if the cryptodev are not of the same type
> 
>  examples/ipsec-secgw/ipsec.c | 25 ++++++++++++++++++++-----
>  1 file changed, 20 insertions(+), 5 deletions(-)
> 
> diff --git a/examples/ipsec-secgw/ipsec.c b/examples/ipsec-secgw/ipsec.c index
> f5cec4a928..b59576c049 100644
> --- a/examples/ipsec-secgw/ipsec.c
> +++ b/examples/ipsec-secgw/ipsec.c
> @@ -288,10 +288,21 @@ create_lookaside_session(struct ipsec_ctx
> *ipsec_ctx_lcore[],
>  		if (cdev_id == RTE_CRYPTO_MAX_DEVS)
>  			cdev_id = ipsec_ctx->tbl[cdev_id_qp].id;
>  		else if (cdev_id != ipsec_ctx->tbl[cdev_id_qp].id) {
> -			RTE_LOG(ERR, IPSEC,
> -					"SA mapping to multiple cryptodevs is "
> -					"not supported!");
> -			return -EINVAL;
> +			struct rte_cryptodev_info dev_info_1, dev_info_2;
> +			rte_cryptodev_info_get(cdev_id, &dev_info_1);
> +			rte_cryptodev_info_get(ipsec_ctx->tbl[cdev_id_qp].id,
> +					&dev_info_2);
> +			if (dev_info_1.driver_id == dev_info_2.driver_id) {
> +				RTE_LOG(WARNING, IPSEC,
> +					"SA mapped to multiple cryptodevs for
> SPI %d\n",
> +					sa->spi);
> +
> +			} else {
> +				RTE_LOG(WARNING, IPSEC,
> +					"SA mapped to multiple cryptodevs of
> different types for SPI %d\n",
> +					sa->spi);
> +
> +			}
>  		}
> 
>  		/* Store per core queue pair information */ @@ -908,7 +919,11
> @@ ipsec_enqueue(ipsec_xform_fn xform_func, struct ipsec_ctx *ipsec_ctx,
>  			continue;
>  		}
> 
> -		enqueue_cop(sa->cqp[ipsec_ctx->lcore_id], &priv->cop);
> +		if (likely(sa->cqp[ipsec_ctx->lcore_id]))
> +			enqueue_cop(sa->cqp[ipsec_ctx->lcore_id], &priv->cop);
> +		else
> +			RTE_LOG(ERR, IPSEC, "No CQP available for lcore %d\n",
> +					ipsec_ctx->lcore_id);

[Anoob] Throwing an error won't be good enough, right? Won't this lead to packet leaks? Since it is datapath, can't we assume that the configuration would be done correctly in control path?

I would suggest drop this specific change and we can enable multiple cryptodevs with lookaside SAs with the changes proposed.

>  	}
>  }
> 
> --
> 2.34.1

Re: [EXT] [PATCH v3] examples/ipsec-secgw: fix cryptodev to SA mapping

[Radu Nicolau]

Hi Anoob, reply inline.

Regards,

Radu

On 27-Feb-24 5:19 AM, Anoob Joseph wrote:
> Hi Radu,
>
> Thanks for making the changes. I've one more question. Please see inline.
>
> Thanks,
> Anoob
>
>> -----Original Message-----
>> From: Radu Nicolau <radu.nicolau@intel.com>
>> Sent: Monday, February 26, 2024 3:56 PM
>> To: dev@dpdk.org
>> Cc: Anoob Joseph <anoobj@marvell.com>; Radu Nicolau
>> <radu.nicolau@intel.com>; stable@dpdk.org; Volodymyr Fialko
>> <vfialko@marvell.com>; Ting-Kai Ku <ting-kai.ku@intel.com>; Ciara Power
>> <ciara.power@intel.com>; Kai Ji <kai.ji@intel.com>; Akhil Goyal
>> <gakhil@marvell.com>
>> Subject: [EXT] [PATCH v3] examples/ipsec-secgw: fix cryptodev to SA mapping
>>
>> External Email
>>
>> ----------------------------------------------------------------------
>> There are use cases where a SA should be able to use different cryptodevs on
>> different lcores, for example there can be cryptodevs with just 1 qp per VF.
>> For this purpose this patch relaxes the check in create lookaside session function.
>> Also add a check to verify that a CQP is available for the current lcore.
>>
>> Fixes: a8ade12123c3 ("examples/ipsec-secgw: create lookaside sessions at init")
>> Cc: stable@dpdk.org
>> Cc: vfialko@marvell.com
>>
>> Signed-off-by: Radu Nicolau <radu.nicolau@intel.com>
>> Tested-by: Ting-Kai Ku <ting-kai.ku@intel.com>
>> Acked-by: Ciara Power <ciara.power@intel.com>
>> Acked-by: Kai Ji <kai.ji@intel.com>
>> ---
>> v3: check if the cryptodev are not of the same type
>>
>>   examples/ipsec-secgw/ipsec.c | 25 ++++++++++++++++++++-----
>>   1 file changed, 20 insertions(+), 5 deletions(-)
>>
>> diff --git a/examples/ipsec-secgw/ipsec.c b/examples/ipsec-secgw/ipsec.c index
>> f5cec4a928..b59576c049 100644
>> --- a/examples/ipsec-secgw/ipsec.c
>> +++ b/examples/ipsec-secgw/ipsec.c
>> @@ -288,10 +288,21 @@ create_lookaside_session(struct ipsec_ctx
>> *ipsec_ctx_lcore[],
>>   		if (cdev_id == RTE_CRYPTO_MAX_DEVS)
>>   			cdev_id = ipsec_ctx->tbl[cdev_id_qp].id;
>>   		else if (cdev_id != ipsec_ctx->tbl[cdev_id_qp].id) {
>> -			RTE_LOG(ERR, IPSEC,
>> -					"SA mapping to multiple cryptodevs is "
>> -					"not supported!");
>> -			return -EINVAL;
>> +			struct rte_cryptodev_info dev_info_1, dev_info_2;
>> +			rte_cryptodev_info_get(cdev_id, &dev_info_1);
>> +			rte_cryptodev_info_get(ipsec_ctx->tbl[cdev_id_qp].id,
>> +					&dev_info_2);
>> +			if (dev_info_1.driver_id == dev_info_2.driver_id) {
>> +				RTE_LOG(WARNING, IPSEC,
>> +					"SA mapped to multiple cryptodevs for
>> SPI %d\n",
>> +					sa->spi);
>> +
>> +			} else {
>> +				RTE_LOG(WARNING, IPSEC,
>> +					"SA mapped to multiple cryptodevs of
>> different types for SPI %d\n",
>> +					sa->spi);
>> +
>> +			}
>>   		}
>>
>>   		/* Store per core queue pair information */ @@ -908,7 +919,11
>> @@ ipsec_enqueue(ipsec_xform_fn xform_func, struct ipsec_ctx *ipsec_ctx,
>>   			continue;
>>   		}
>>
>> -		enqueue_cop(sa->cqp[ipsec_ctx->lcore_id], &priv->cop);
>> +		if (likely(sa->cqp[ipsec_ctx->lcore_id]))
>> +			enqueue_cop(sa->cqp[ipsec_ctx->lcore_id], &priv->cop);
>> +		else
>> +			RTE_LOG(ERR, IPSEC, "No CQP available for lcore %d\n",
>> +					ipsec_ctx->lcore_id);
> [Anoob] Throwing an error won't be good enough, right? Won't this lead to packet leaks? Since it is datapath, can't we assume that the configuration would be done correctly in control path?
>
> I would suggest drop this specific change and we can enable multiple cryptodevs with lookaside SAs with the changes proposed.
With the change that removed the lazy initialization of SAs 
("examples/ipsec-secgw: create lookaside sessions at init") we can't 
assume anymore that a worker core has the proper CQP assigned, that is 
the reason I added the check here, the control path had no errors but 
there was no CQP assigned to a lcore. Indeed there will be dropped 
packets but at least there will be no segfault and the user will have an 
indication on what needs to be done - assign more cryptodevs.
>>   	}
>>   }
>>
>> --
>> 2.34.1

RE: [EXT] [PATCH v3] examples/ipsec-secgw: fix cryptodev to SA mapping

[Anoob Joseph]

Hi Radu,

Please see inline.

Thanks,
Anoob

> -----Original Message-----
> From: Radu Nicolau <radu.nicolau@intel.com>
> Sent: Tuesday, February 27, 2024 3:41 PM
> To: Anoob Joseph <anoobj@marvell.com>
> Cc: stable@dpdk.org; Volodymyr Fialko <vfialko@marvell.com>; Ting-Kai Ku
> <ting-kai.ku@intel.com>; Ciara Power <ciara.power@intel.com>; Kai Ji
> <kai.ji@intel.com>; Akhil Goyal <gakhil@marvell.com>; dev@dpdk.org
> Subject: Re: [EXT] [PATCH v3] examples/ipsec-secgw: fix cryptodev to SA
> mapping
> 
> Hi Anoob, reply inline.
> 
> Regards,
> 
> Radu
> 
> On 27-Feb-24 5:19 AM, Anoob Joseph wrote:
> > Hi Radu,
> >
> > Thanks for making the changes. I've one more question. Please see inline.
> >
> > Thanks,
> > Anoob
> >
> >> -----Original Message-----
> >> From: Radu Nicolau <radu.nicolau@intel.com>
> >> Sent: Monday, February 26, 2024 3:56 PM
> >> To: dev@dpdk.org
> >> Cc: Anoob Joseph <anoobj@marvell.com>; Radu Nicolau
> >> <radu.nicolau@intel.com>; stable@dpdk.org; Volodymyr Fialko
> >> <vfialko@marvell.com>; Ting-Kai Ku <ting-kai.ku@intel.com>; Ciara
> >> Power <ciara.power@intel.com>; Kai Ji <kai.ji@intel.com>; Akhil Goyal
> >> <gakhil@marvell.com>
> >> Subject: [EXT] [PATCH v3] examples/ipsec-secgw: fix cryptodev to SA
> >> mapping
> >>
> >> External Email
> >>
> >> ---------------------------------------------------------------------
> >> - There are use cases where a SA should be able to use different
> >> cryptodevs on different lcores, for example there can be cryptodevs
> >> with just 1 qp per VF.
> >> For this purpose this patch relaxes the check in create lookaside session
> function.
> >> Also add a check to verify that a CQP is available for the current lcore.
> >>
> >> Fixes: a8ade12123c3 ("examples/ipsec-secgw: create lookaside sessions
> >> at init")
> >> Cc: stable@dpdk.org
> >> Cc: vfialko@marvell.com
> >>
> >> Signed-off-by: Radu Nicolau <radu.nicolau@intel.com>
> >> Tested-by: Ting-Kai Ku <ting-kai.ku@intel.com>
> >> Acked-by: Ciara Power <ciara.power@intel.com>
> >> Acked-by: Kai Ji <kai.ji@intel.com>
> >> ---
> >> v3: check if the cryptodev are not of the same type
> >>
> >>   examples/ipsec-secgw/ipsec.c | 25 ++++++++++++++++++++-----
> >>   1 file changed, 20 insertions(+), 5 deletions(-)
> >>
> >> diff --git a/examples/ipsec-secgw/ipsec.c
> >> b/examples/ipsec-secgw/ipsec.c index
> >> f5cec4a928..b59576c049 100644
> >> --- a/examples/ipsec-secgw/ipsec.c
> >> +++ b/examples/ipsec-secgw/ipsec.c
> >> @@ -288,10 +288,21 @@ create_lookaside_session(struct ipsec_ctx
> >> *ipsec_ctx_lcore[],
> >>   		if (cdev_id == RTE_CRYPTO_MAX_DEVS)
> >>   			cdev_id = ipsec_ctx->tbl[cdev_id_qp].id;
> >>   		else if (cdev_id != ipsec_ctx->tbl[cdev_id_qp].id) {
> >> -			RTE_LOG(ERR, IPSEC,
> >> -					"SA mapping to multiple cryptodevs is
> "
> >> -					"not supported!");
> >> -			return -EINVAL;
> >> +			struct rte_cryptodev_info dev_info_1, dev_info_2;
> >> +			rte_cryptodev_info_get(cdev_id, &dev_info_1);
> >> +			rte_cryptodev_info_get(ipsec_ctx->tbl[cdev_id_qp].id,
> >> +					&dev_info_2);
> >> +			if (dev_info_1.driver_id == dev_info_2.driver_id) {
> >> +				RTE_LOG(WARNING, IPSEC,
> >> +					"SA mapped to multiple cryptodevs
> for
> >> SPI %d\n",
> >> +					sa->spi);
> >> +
> >> +			} else {
> >> +				RTE_LOG(WARNING, IPSEC,
> >> +					"SA mapped to multiple cryptodevs of
> >> different types for SPI %d\n",
> >> +					sa->spi);
> >> +
> >> +			}
> >>   		}
> >>
> >>   		/* Store per core queue pair information */ @@ -908,7
> +919,11 @@
> >> ipsec_enqueue(ipsec_xform_fn xform_func, struct ipsec_ctx *ipsec_ctx,
> >>   			continue;
> >>   		}
> >>
> >> -		enqueue_cop(sa->cqp[ipsec_ctx->lcore_id], &priv->cop);
> >> +		if (likely(sa->cqp[ipsec_ctx->lcore_id]))
> >> +			enqueue_cop(sa->cqp[ipsec_ctx->lcore_id], &priv-
> >cop);
> >> +		else
> >> +			RTE_LOG(ERR, IPSEC, "No CQP available for lcore
> %d\n",
> >> +					ipsec_ctx->lcore_id);
> > [Anoob] Throwing an error won't be good enough, right? Won't this lead to
> packet leaks? Since it is datapath, can't we assume that the configuration
> would be done correctly in control path?
> >
> > I would suggest drop this specific change and we can enable multiple
> cryptodevs with lookaside SAs with the changes proposed.
> With the change that removed the lazy initialization of SAs
> ("examples/ipsec-secgw: create lookaside sessions at init") we can't assume
> anymore that a worker core has the proper CQP assigned, that is the reason I
> added the check here, the control path had no errors but there was no CQP
> assigned to a lcore. Indeed there will be dropped packets but at least there will
> be no segfault and the user will have an indication on what needs to be done -
> assign more cryptodevs.

[Anoob] I understand your concern. I was just worried about an extra check coming in data path which can impact performance benchmarks with valid configuration. Can we keep the check under a debug flag? Is that okay? 

> >>   	}
> >>   }
> >>
> >> --
> >> 2.34.1

[PATCH v4] examples/ipsec-secgw: fix cryptodev to SA mapping

[Radu Nicolau]

There are use cases where a SA should be able to use different cryptodevs on
different lcores, for example there can be cryptodevs with just 1 qp per VF.
For this purpose this patch relaxes the check in create lookaside session function.
Also add a check to verify that a CQP is available for the current lcore.

Fixes: a8ade12123c3 ("examples/ipsec-secgw: create lookaside sessions at init")
Cc: stable@dpdk.org
Cc: vfialko@marvell.com

Signed-off-by: Radu Nicolau <radu.nicolau@intel.com>
Tested-by: Ting-Kai Ku <ting-kai.ku@intel.com>
Acked-by: Ciara Power <ciara.power@intel.com>
Acked-by: Kai Ji <kai.ji@intel.com>
---
v4: replaced plain if with RTE_ASSERT

 examples/ipsec-secgw/ipsec.c | 20 ++++++++++++++++----
 1 file changed, 16 insertions(+), 4 deletions(-)

diff --git a/examples/ipsec-secgw/ipsec.c b/examples/ipsec-secgw/ipsec.c
index f5cec4a928..c321108119 100644
--- a/examples/ipsec-secgw/ipsec.c
+++ b/examples/ipsec-secgw/ipsec.c
@@ -288,10 +288,21 @@ create_lookaside_session(struct ipsec_ctx *ipsec_ctx_lcore[],
 		if (cdev_id == RTE_CRYPTO_MAX_DEVS)
 			cdev_id = ipsec_ctx->tbl[cdev_id_qp].id;
 		else if (cdev_id != ipsec_ctx->tbl[cdev_id_qp].id) {
-			RTE_LOG(ERR, IPSEC,
-					"SA mapping to multiple cryptodevs is "
-					"not supported!");
-			return -EINVAL;
+			struct rte_cryptodev_info dev_info_1, dev_info_2;
+			rte_cryptodev_info_get(cdev_id, &dev_info_1);
+			rte_cryptodev_info_get(ipsec_ctx->tbl[cdev_id_qp].id,
+					&dev_info_2);
+			if (dev_info_1.driver_id == dev_info_2.driver_id) {
+				RTE_LOG(WARNING, IPSEC,
+					"SA mapped to multiple cryptodevs for SPI %d\n",
+					sa->spi);
+
+			} else {
+				RTE_LOG(WARNING, IPSEC,
+					"SA mapped to multiple cryptodevs of different types for SPI %d\n",
+					sa->spi);
+
+			}
 		}
 
 		/* Store per core queue pair information */
@@ -908,6 +919,7 @@ ipsec_enqueue(ipsec_xform_fn xform_func, struct ipsec_ctx *ipsec_ctx,
 			continue;
 		}
 
+		RTE_ASSERT(sa->cqp[ipsec_ctx->lcore_id] != NULL);
 		enqueue_cop(sa->cqp[ipsec_ctx->lcore_id], &priv->cop);
 	}
 }
-- 
2.34.1

RE: [EXT] [PATCH v4] examples/ipsec-secgw: fix cryptodev to SA mapping

[Anoob Joseph]

> 
> ----------------------------------------------------------------------
> There are use cases where a SA should be able to use different cryptodevs on
> different lcores, for example there can be cryptodevs with just 1 qp per VF.
> For this purpose this patch relaxes the check in create lookaside session
> function.
> Also add a check to verify that a CQP is available for the current lcore.
> 
> Fixes: a8ade12123c3 ("examples/ipsec-secgw: create lookaside sessions at
> init")
> Cc: stable@dpdk.org
> Cc: vfialko@marvell.com
> 
> Signed-off-by: Radu Nicolau <radu.nicolau@intel.com>
> Tested-by: Ting-Kai Ku <ting-kai.ku@intel.com>
> Acked-by: Ciara Power <ciara.power@intel.com>
> Acked-by: Kai Ji <kai.ji@intel.com>

Acked-by: Anoob Joseph <anoobj@marvell.com>

RE: [EXT] [PATCH v4] examples/ipsec-secgw: fix cryptodev to SA mapping

[Akhil Goyal]

> > There are use cases where a SA should be able to use different cryptodevs on
> > different lcores, for example there can be cryptodevs with just 1 qp per VF.
> > For this purpose this patch relaxes the check in create lookaside session
> > function.
> > Also add a check to verify that a CQP is available for the current lcore.
> >
> > Fixes: a8ade12123c3 ("examples/ipsec-secgw: create lookaside sessions at
> > init")
> > Cc: stable@dpdk.org
> > Cc: vfialko@marvell.com
> >
> > Signed-off-by: Radu Nicolau <radu.nicolau@intel.com>
> > Tested-by: Ting-Kai Ku <ting-kai.ku@intel.com>
> > Acked-by: Ciara Power <ciara.power@intel.com>
> > Acked-by: Kai Ji <kai.ji@intel.com>
> 
> Acked-by: Anoob Joseph <anoobj@marvell.com>
> 
Applied to dpdk-next-crypto
Thanks.