[dpdk-dev] [PATCH] examples/ipsec-secgw: fix usage of incorrect port

[dpdk-dev] [PATCH] examples/ipsec-secgw: fix usage of incorrect port

[Anoob Joseph]

When security offload is enabled, the packet should be forwarded on the
port configured in the SA. Security session will be configured on that
port only, and sending the packet on other ports could result in
unencrypted packets being sent out.

This would have performance improvements too, as the per packet LPM
lookup would be avoided for IPsec packets, in inline mode.

Fixes: ec17993a145a ("examples/ipsec-secgw: support security offload")

Signed-off-by: Anoob Joseph <anoob.joseph@cavium.com>
---
 examples/ipsec-secgw/ipsec-secgw.c | 92 ++++++++++++++++++++++++++++++++------
 1 file changed, 78 insertions(+), 14 deletions(-)

diff --git a/examples/ipsec-secgw/ipsec-secgw.c b/examples/ipsec-secgw/ipsec-secgw.c
index c98454a..80cdff3 100644
--- a/examples/ipsec-secgw/ipsec-secgw.c
+++ b/examples/ipsec-secgw/ipsec-secgw.c
@@ -585,31 +585,72 @@ process_pkts_outbound_nosp(struct ipsec_ctx *ipsec_ctx,
 		traffic->ip6.num = nb_pkts_out;
 }
 
+static inline int32_t
+get_hop_for_offload_pkt(struct rte_mbuf *pkt)
+{
+	struct ipsec_mbuf_metadata *priv;
+	struct ipsec_sa *sa;
+
+	priv = get_priv(pkt);
+
+	sa = priv->sa;
+	if (unlikely(sa == NULL)) {
+		RTE_LOG(ERR, IPSEC, "SA not saved in private data\n");
+		return -1;
+	}
+
+	return sa->portid;
+}
+
 static inline void
 route4_pkts(struct rt_ctx *rt_ctx, struct rte_mbuf *pkts[], uint8_t nb_pkts)
 {
 	uint32_t hop[MAX_PKT_BURST * 2];
 	uint32_t dst_ip[MAX_PKT_BURST * 2];
+	int32_t pkt_hop = 0;
 	uint16_t i, offset;
+	uint16_t lpm_pkts = 0;
 
 	if (nb_pkts == 0)
 		return;
 
+	/* Need to do an LPM lookup for non-offload packets. Offload packets
+	 * will have port ID in the SA
+	 */
+
 	for (i = 0; i < nb_pkts; i++) {
-		offset = offsetof(struct ip, ip_dst);
-		dst_ip[i] = *rte_pktmbuf_mtod_offset(pkts[i],
-				uint32_t *, offset);
-		dst_ip[i] = rte_be_to_cpu_32(dst_ip[i]);
+		if (!(pkts[i]->ol_flags & PKT_TX_SEC_OFFLOAD)) {
+			/* Security offload not enabled. So an LPM lookup is
+			 * required to get the hop
+			 */
+			offset = offsetof(struct ip, ip_dst);
+			dst_ip[lpm_pkts] = *rte_pktmbuf_mtod_offset(pkts[i],
+					uint32_t *, offset);
+			dst_ip[lpm_pkts] = rte_be_to_cpu_32(dst_ip[lpm_pkts]);
+			lpm_pkts++;
+		}
 	}
 
-	rte_lpm_lookup_bulk((struct rte_lpm *)rt_ctx, dst_ip, hop, nb_pkts);
+	rte_lpm_lookup_bulk((struct rte_lpm *)rt_ctx, dst_ip, hop, lpm_pkts);
+
+	lpm_pkts = 0;
 
 	for (i = 0; i < nb_pkts; i++) {
-		if ((hop[i] & RTE_LPM_LOOKUP_SUCCESS) == 0) {
+		if ((pkts[i]->ol_flags & PKT_TX_SEC_OFFLOAD)) {
+			/* Read hop from the SA */
+			pkt_hop = get_hop_for_offload_pkt(pkts[i]);
+		} else {
+			/* Need to use hop returned by lookup */
+			pkt_hop = hop[lpm_pkts++];
+			if ((pkt_hop & RTE_LPM_LOOKUP_SUCCESS) == 0)
+				pkt_hop = -1;
+		}
+
+		if (pkt_hop == -1) {
 			rte_pktmbuf_free(pkts[i]);
 			continue;
 		}
-		send_single_packet(pkts[i], hop[i] & 0xff);
+		send_single_packet(pkts[i], pkt_hop & 0xff);
 	}
 }
 
@@ -619,26 +660,49 @@ route6_pkts(struct rt_ctx *rt_ctx, struct rte_mbuf *pkts[], uint8_t nb_pkts)
 	int32_t hop[MAX_PKT_BURST * 2];
 	uint8_t dst_ip[MAX_PKT_BURST * 2][16];
 	uint8_t *ip6_dst;
+	int32_t pkt_hop = 0;
 	uint16_t i, offset;
+	uint16_t lpm_pkts = 0;
 
 	if (nb_pkts == 0)
 		return;
 
+	/* Need to do an LPM lookup for non-offload packets. Offload packets
+	 * will have port ID in the SA
+	 */
+
 	for (i = 0; i < nb_pkts; i++) {
-		offset = offsetof(struct ip6_hdr, ip6_dst);
-		ip6_dst = rte_pktmbuf_mtod_offset(pkts[i], uint8_t *, offset);
-		memcpy(&dst_ip[i][0], ip6_dst, 16);
+		if (!(pkts[i]->ol_flags & PKT_TX_SEC_OFFLOAD)) {
+			/* Security offload not enabled. So an LPM lookup is
+			 * required to get the hop
+			 */
+			offset = offsetof(struct ip6_hdr, ip6_dst);
+			ip6_dst = rte_pktmbuf_mtod_offset(pkts[i], uint8_t *,
+					offset);
+			memcpy(&dst_ip[lpm_pkts][0], ip6_dst, 16);
+			lpm_pkts++;
+		}
 	}
 
-	rte_lpm6_lookup_bulk_func((struct rte_lpm6 *)rt_ctx, dst_ip,
-			hop, nb_pkts);
+	rte_lpm6_lookup_bulk_func((struct rte_lpm6 *)rt_ctx, dst_ip, hop,
+			lpm_pkts);
+
+	lpm_pkts = 0;
 
 	for (i = 0; i < nb_pkts; i++) {
-		if (hop[i] == -1) {
+		if ((pkts[i]->ol_flags & PKT_TX_SEC_OFFLOAD) == 0) {
+			/* Read hop from the SA */
+			pkt_hop = get_hop_for_offload_pkt(pkts[i]);
+		} else {
+			/* Need to use hop returned by lookup */
+			pkt_hop = hop[lpm_pkts++];
+		}
+
+		if (pkt_hop == -1) {
 			rte_pktmbuf_free(pkts[i]);
 			continue;
 		}
-		send_single_packet(pkts[i], hop[i] & 0xff);
+		send_single_packet(pkts[i], pkt_hop & 0xff);
 	}
 }
 
-- 
2.7.4

Re: [dpdk-dev] [PATCH] examples/ipsec-secgw: fix usage of incorrect port

[Radu Nicolau]

Hi,

Comments below

On 11/13/2017 4:13 PM, Anoob Joseph wrote:
> When security offload is enabled, the packet should be forwarded on the
> port configured in the SA. Security session will be configured on that
> port only, and sending the packet on other ports could result in
> unencrypted packets being sent out.
With a properly configured SP, SA and routing rule this will not happen, 
so we don't need to do this fix to make up for a wrongly written 
configuration file.
I'm almost sure that the app will behave in the same way (i.e. forward 
unencrypted) for lookaside crypto if the configuration is incorrect.
>
> This would have performance improvements too, as the per packet LPM
> lookup would be avoided for IPsec packets, in inline mode.
Yes, there will be some performance gain, but not sure how much 
considering that LPM lookup is reasonably fast.

So I'm not sure if ack or nack, maybe Sergio can give a second opinion.
But if ack, you will have to update the patch to include in the doc this 
behavior, the port configured in the SA takes precedence over the one in 
the routing rule.

Regards,
Radu

Re: [dpdk-dev] [PATCH] examples/ipsec-secgw: fix usage of incorrect port

[Anoob Joseph]

Hi,

Comments below


On 13-11-2017 22:53, Radu Nicolau wrote:
> Hi,
>
> Comments below
>
> On 11/13/2017 4:13 PM, Anoob Joseph wrote:
>> When security offload is enabled, the packet should be forwarded on the
>> port configured in the SA. Security session will be configured on that
>> port only, and sending the packet on other ports could result in
>> unencrypted packets being sent out.
> With a properly configured SP, SA and routing rule this will not 
> happen, so we don't need to do this fix to make up for a wrongly 
> written configuration file.
> I'm almost sure that the app will behave in the same way (i.e. forward 
> unencrypted) for lookaside crypto if the configuration is incorrect.
The lookaside crypto will ensure encryption, even if the LPM port is 
different.
>>
>> This would have performance improvements too, as the per packet LPM
>> lookup would be avoided for IPsec packets, in inline mode.
> Yes, there will be some performance gain, but not sure how much 
> considering that LPM lookup is reasonably fast.
The 2nd lookup is significant for inline protocol for which I plan to 
submit some patches. In case of inline protocol, the packet need not 
have final headers by the time it is submitted to the ethernet driver. 
For example, in case of ESP in tunnel mode, tunnel IPs from the SA need 
to be used for LPM lookup. So all such cases(tunnel/transport, ipv4 
tunnel in ipv6 and vice versa etc) need to be valuated and the final 
addresses need to be determined before an LPM lookup can be done, which 
adds significant overhead per packet.
>
> So I'm not sure if ack or nack, maybe Sergio can give a second opinion.
> But if ack, you will have to update the patch to include in the doc 
> this behavior, the port configured in the SA takes precedence over the 
> one in the routing rule.
>
> Regards,
> Radu

Thanks,
Anoob

Re: [dpdk-dev] [PATCH] examples/ipsec-secgw: fix usage of incorrect port

[Nicolau, Radu]

Hi,

Please send a v2 with the doc update that describes the new behavior and I will ack it.

Regards,
Radu

> -----Original Message-----
> From: Anoob Joseph [mailto:ajoseph@caviumnetworks.com]
> Sent: Monday, November 13, 2017 7:25 PM
> To: Nicolau, Radu <radu.nicolau@intel.com>; Anoob Joseph
> <anoob.joseph@cavium.com>; Akhil Goyal <akhil.goyal@nxp.com>;
> Doherty, Declan <declan.doherty@intel.com>; Gonzalez Monroy, Sergio
> <sergio.gonzalez.monroy@intel.com>
> Cc: narayanaprasad.athreya@cavium.com;
> jerin.jacobkollanukkaran@cavium.com; dev@dpdk.org
> Subject: Re: [PATCH] examples/ipsec-secgw: fix usage of incorrect port
> 
> Hi,
> 
> Comments below
> 
> 
> On 13-11-2017 22:53, Radu Nicolau wrote:
> > Hi,
> >
> > Comments below
> >
> > On 11/13/2017 4:13 PM, Anoob Joseph wrote:
> >> When security offload is enabled, the packet should be forwarded on
> >> the port configured in the SA. Security session will be configured on
> >> that port only, and sending the packet on other ports could result in
> >> unencrypted packets being sent out.
> > With a properly configured SP, SA and routing rule this will not
> > happen, so we don't need to do this fix to make up for a wrongly
> > written configuration file.
> > I'm almost sure that the app will behave in the same way (i.e. forward
> > unencrypted) for lookaside crypto if the configuration is incorrect.
> The lookaside crypto will ensure encryption, even if the LPM port is different.
> >>
> >> This would have performance improvements too, as the per packet LPM
> >> lookup would be avoided for IPsec packets, in inline mode.
> > Yes, there will be some performance gain, but not sure how much
> > considering that LPM lookup is reasonably fast.
> The 2nd lookup is significant for inline protocol for which I plan to submit
> some patches. In case of inline protocol, the packet need not have final
> headers by the time it is submitted to the ethernet driver.
> For example, in case of ESP in tunnel mode, tunnel IPs from the SA need to
> be used for LPM lookup. So all such cases(tunnel/transport, ipv4 tunnel in
> ipv6 and vice versa etc) need to be valuated and the final addresses need to
> be determined before an LPM lookup can be done, which adds significant
> overhead per packet.
> >
> > So I'm not sure if ack or nack, maybe Sergio can give a second opinion.
> > But if ack, you will have to update the patch to include in the doc
> > this behavior, the port configured in the SA takes precedence over the
> > one in the routing rule.
> >
> > Regards,
> > Radu
> 
> Thanks,
> Anoob

[dpdk-dev] [PATCH v2] examples/ipsec-secgw: fix usage of incorrect port

[Anoob Joseph]

When security offload is enabled, the packet should be forwarded on the
port configured in the SA. Security session will be configured on that
port only, and sending the packet on other ports could result in
unencrypted packets being sent out.

This would have performance improvements too, as the per packet LPM
lookup would be avoided for IPsec packets, in inline mode.

Fixes: ec17993a145a ("examples/ipsec-secgw: support security offload")

Signed-off-by: Anoob Joseph <anoob.joseph@caviumnetworks.com>
---
v2:
* Updated documentation with the change in behavior for outbound inline
  offloaded packets.

 doc/guides/sample_app_ug/ipsec_secgw.rst | 10 +++-
 examples/ipsec-secgw/ipsec-secgw.c       | 92 +++++++++++++++++++++++++++-----
 2 files changed, 87 insertions(+), 15 deletions(-)

diff --git a/doc/guides/sample_app_ug/ipsec_secgw.rst b/doc/guides/sample_app_ug/ipsec_secgw.rst
index d6cfdbf..d04e153 100644
--- a/doc/guides/sample_app_ug/ipsec_secgw.rst
+++ b/doc/guides/sample_app_ug/ipsec_secgw.rst
@@ -61,6 +61,12 @@ In case of complete protocol offload, the processing of headers(ESP and outer
 IP header) is done by the hardware and the application does not need to
 add/remove them during outbound/inbound processing.
 
+For inline offloaded outbound traffic, the application need not do the LPM
+lookup for routing, as the port on which the packet has to be forwarded, will
+be part of the SA. Security parameters will be configured on that port only,
+and sending the packet on other ports could result in unencrypted packets being
+sent out.
+
 The Path for IPsec Inbound traffic is:
 
 *  Read packets from the port.
@@ -543,7 +549,9 @@ where each options means:
  ``<port_id>``
 
  * Port/device ID of the ethernet/crypto accelerator for which the SA is
-   configured. This option is used when *type* is NOT *no-offload*
+   configured. For *inline-crypto-offload* and *inline-protocol-offload*, this
+   port will be used for routing. The routing table will not be referred in
+   this case.
 
  * Optional: No, if *type* is not *no-offload*
 
diff --git a/examples/ipsec-secgw/ipsec-secgw.c b/examples/ipsec-secgw/ipsec-secgw.c
index c98454a..80cdff3 100644
--- a/examples/ipsec-secgw/ipsec-secgw.c
+++ b/examples/ipsec-secgw/ipsec-secgw.c
@@ -585,31 +585,72 @@ process_pkts_outbound_nosp(struct ipsec_ctx *ipsec_ctx,
 		traffic->ip6.num = nb_pkts_out;
 }
 
+static inline int32_t
+get_hop_for_offload_pkt(struct rte_mbuf *pkt)
+{
+	struct ipsec_mbuf_metadata *priv;
+	struct ipsec_sa *sa;
+
+	priv = get_priv(pkt);
+
+	sa = priv->sa;
+	if (unlikely(sa == NULL)) {
+		RTE_LOG(ERR, IPSEC, "SA not saved in private data\n");
+		return -1;
+	}
+
+	return sa->portid;
+}
+
 static inline void
 route4_pkts(struct rt_ctx *rt_ctx, struct rte_mbuf *pkts[], uint8_t nb_pkts)
 {
 	uint32_t hop[MAX_PKT_BURST * 2];
 	uint32_t dst_ip[MAX_PKT_BURST * 2];
+	int32_t pkt_hop = 0;
 	uint16_t i, offset;
+	uint16_t lpm_pkts = 0;
 
 	if (nb_pkts == 0)
 		return;
 
+	/* Need to do an LPM lookup for non-offload packets. Offload packets
+	 * will have port ID in the SA
+	 */
+
 	for (i = 0; i < nb_pkts; i++) {
-		offset = offsetof(struct ip, ip_dst);
-		dst_ip[i] = *rte_pktmbuf_mtod_offset(pkts[i],
-				uint32_t *, offset);
-		dst_ip[i] = rte_be_to_cpu_32(dst_ip[i]);
+		if (!(pkts[i]->ol_flags & PKT_TX_SEC_OFFLOAD)) {
+			/* Security offload not enabled. So an LPM lookup is
+			 * required to get the hop
+			 */
+			offset = offsetof(struct ip, ip_dst);
+			dst_ip[lpm_pkts] = *rte_pktmbuf_mtod_offset(pkts[i],
+					uint32_t *, offset);
+			dst_ip[lpm_pkts] = rte_be_to_cpu_32(dst_ip[lpm_pkts]);
+			lpm_pkts++;
+		}
 	}
 
-	rte_lpm_lookup_bulk((struct rte_lpm *)rt_ctx, dst_ip, hop, nb_pkts);
+	rte_lpm_lookup_bulk((struct rte_lpm *)rt_ctx, dst_ip, hop, lpm_pkts);
+
+	lpm_pkts = 0;
 
 	for (i = 0; i < nb_pkts; i++) {
-		if ((hop[i] & RTE_LPM_LOOKUP_SUCCESS) == 0) {
+		if ((pkts[i]->ol_flags & PKT_TX_SEC_OFFLOAD)) {
+			/* Read hop from the SA */
+			pkt_hop = get_hop_for_offload_pkt(pkts[i]);
+		} else {
+			/* Need to use hop returned by lookup */
+			pkt_hop = hop[lpm_pkts++];
+			if ((pkt_hop & RTE_LPM_LOOKUP_SUCCESS) == 0)
+				pkt_hop = -1;
+		}
+
+		if (pkt_hop == -1) {
 			rte_pktmbuf_free(pkts[i]);
 			continue;
 		}
-		send_single_packet(pkts[i], hop[i] & 0xff);
+		send_single_packet(pkts[i], pkt_hop & 0xff);
 	}
 }
 
@@ -619,26 +660,49 @@ route6_pkts(struct rt_ctx *rt_ctx, struct rte_mbuf *pkts[], uint8_t nb_pkts)
 	int32_t hop[MAX_PKT_BURST * 2];
 	uint8_t dst_ip[MAX_PKT_BURST * 2][16];
 	uint8_t *ip6_dst;
+	int32_t pkt_hop = 0;
 	uint16_t i, offset;
+	uint16_t lpm_pkts = 0;
 
 	if (nb_pkts == 0)
 		return;
 
+	/* Need to do an LPM lookup for non-offload packets. Offload packets
+	 * will have port ID in the SA
+	 */
+
 	for (i = 0; i < nb_pkts; i++) {
-		offset = offsetof(struct ip6_hdr, ip6_dst);
-		ip6_dst = rte_pktmbuf_mtod_offset(pkts[i], uint8_t *, offset);
-		memcpy(&dst_ip[i][0], ip6_dst, 16);
+		if (!(pkts[i]->ol_flags & PKT_TX_SEC_OFFLOAD)) {
+			/* Security offload not enabled. So an LPM lookup is
+			 * required to get the hop
+			 */
+			offset = offsetof(struct ip6_hdr, ip6_dst);
+			ip6_dst = rte_pktmbuf_mtod_offset(pkts[i], uint8_t *,
+					offset);
+			memcpy(&dst_ip[lpm_pkts][0], ip6_dst, 16);
+			lpm_pkts++;
+		}
 	}
 
-	rte_lpm6_lookup_bulk_func((struct rte_lpm6 *)rt_ctx, dst_ip,
-			hop, nb_pkts);
+	rte_lpm6_lookup_bulk_func((struct rte_lpm6 *)rt_ctx, dst_ip, hop,
+			lpm_pkts);
+
+	lpm_pkts = 0;
 
 	for (i = 0; i < nb_pkts; i++) {
-		if (hop[i] == -1) {
+		if ((pkts[i]->ol_flags & PKT_TX_SEC_OFFLOAD) == 0) {
+			/* Read hop from the SA */
+			pkt_hop = get_hop_for_offload_pkt(pkts[i]);
+		} else {
+			/* Need to use hop returned by lookup */
+			pkt_hop = hop[lpm_pkts++];
+		}
+
+		if (pkt_hop == -1) {
 			rte_pktmbuf_free(pkts[i]);
 			continue;
 		}
-		send_single_packet(pkts[i], hop[i] & 0xff);
+		send_single_packet(pkts[i], pkt_hop & 0xff);
 	}
 }
 
-- 
2.7.4

Re: [dpdk-dev] [PATCH v2] examples/ipsec-secgw: fix usage of incorrect port

[Radu Nicolau]

On 11/14/2017 3:37 PM, Anoob Joseph wrote:
> When security offload is enabled, the packet should be forwarded on the
> port configured in the SA. Security session will be configured on that
> port only, and sending the packet on other ports could result in
> unencrypted packets being sent out.
>
> This would have performance improvements too, as the per packet LPM
> lookup would be avoided for IPsec packets, in inline mode.
>
> Fixes: ec17993a145a ("examples/ipsec-secgw: support security offload")
>
> Signed-off-by: Anoob Joseph <anoob.joseph@caviumnetworks.com>
> ---
> v2:
> * Updated documentation with the change in behavior for outbound inline
>    offloaded packets.
>
>   doc/guides/sample_app_ug/ipsec_secgw.rst | 10 +++-
>   examples/ipsec-secgw/ipsec-secgw.c       | 92 +++++++++++++++++++++++++++-----
>   2 files changed, 87 insertions(+), 15 deletions(-)
>
> diff --git a/doc/guides/sample_app_ug/ipsec_secgw.rst b/doc/guides/sample_app_ug/ipsec_secgw.rst
> index d6cfdbf..d04e153 100644
> --- a/doc/guides/sample_app_ug/ipsec_secgw.rst
> +++ b/doc/guides/sample_app_ug/ipsec_secgw.rst
> @@ -61,6 +61,12 @@ In case of complete protocol offload, the processing of headers(ESP and outer
>   IP header) is done by the hardware and the application does not need to
>   add/remove them during outbound/inbound processing.
>   
> +For inline offloaded outbound traffic, the application need not do the LPM
> +lookup for routing, as the port on which the packet has to be forwarded, will
extra 
comma......................................................................................................................^here
And maybe change need not to will not, to reflect the actual behavior.
> <snip>
>   
> @@ -619,26 +660,49 @@ route6_pkts(struct rt_ctx *rt_ctx, struct rte_mbuf *pkts[], uint8_t nb_pkts)
>   	int32_t hop[MAX_PKT_BURST * 2];
>   	uint8_t dst_ip[MAX_PKT_BURST * 2][16];
>   	uint8_t *ip6_dst;
> +	int32_t pkt_hop = 0;
>   	uint16_t i, offset;
> +	uint16_t lpm_pkts = 0;
>   
>   	if (nb_pkts == 0)
>   		return;
>   
> +	/* Need to do an LPM lookup for non-offload packets. Offload packets
> +	 * will have port ID in the SA
> +	 */
> +
>   	for (i = 0; i < nb_pkts; i++) {
> -		offset = offsetof(struct ip6_hdr, ip6_dst);
> -		ip6_dst = rte_pktmbuf_mtod_offset(pkts[i], uint8_t *, offset);
> -		memcpy(&dst_ip[i][0], ip6_dst, 16);
> +		if (!(pkts[i]->ol_flags & PKT_TX_SEC_OFFLOAD)) {
> +			/* Security offload not enabled. So an LPM lookup is
> +			 * required to get the hop
> +			 */
> +			offset = offsetof(struct ip6_hdr, ip6_dst);
> +			ip6_dst = rte_pktmbuf_mtod_offset(pkts[i], uint8_t *,
> +					offset);
> +			memcpy(&dst_ip[lpm_pkts][0], ip6_dst, 16);
> +			lpm_pkts++;
> +		}
>   	}
>   
> -	rte_lpm6_lookup_bulk_func((struct rte_lpm6 *)rt_ctx, dst_ip,
> -			hop, nb_pkts);
> +	rte_lpm6_lookup_bulk_func((struct rte_lpm6 *)rt_ctx, dst_ip, hop,
> +			lpm_pkts);
> +
> +	lpm_pkts = 0;
>   
>   	for (i = 0; i < nb_pkts; i++) {
> -		if (hop[i] == -1) {
> +		if ((pkts[i]->ol_flags & PKT_TX_SEC_OFFLOAD) == 0) {
The if condition is wrong here.
> +			/* Read hop from the SA */
> +			pkt_hop = get_hop_for_offload_pkt(pkts[i]);
> +		} else {
> +			/* Need to use hop returned by lookup */
> +			pkt_hop = hop[lpm_pkts++];
> +		}
> +
> +		if (pkt_hop == -1) {
>   			rte_pktmbuf_free(pkts[i]);
>   			continue;
>   		}
> -		send_single_packet(pkts[i], hop[i] & 0xff);
> +		send_single_packet(pkts[i], pkt_hop & 0xff);
>   	}
>   }
>   

[dpdk-dev] [PATCH v3] examples/ipsec-secgw: fix usage of incorrect port

[Anoob Joseph]

When security offload is enabled, the packet should be forwarded on the
port configured in the SA. Security session will be configured on that
port only, and sending the packet on other ports could result in
unencrypted packets being sent out.

This would have performance improvements too, as the per packet LPM
lookup would be avoided for IPsec packets, in inline mode.

Fixes: ec17993a145a ("examples/ipsec-secgw: support security offload")

Signed-off-by: Anoob Joseph <anoob.joseph@caviumnetworks.com>
---
v3
* Bug fix (fixed a wrong if condition)
* Minor changes in documentation

v2:
* Updated documentation with the change in behavior for outbound inline
  offloaded packets.

 doc/guides/sample_app_ug/ipsec_secgw.rst | 10 +++-
 examples/ipsec-secgw/ipsec-secgw.c       | 92 +++++++++++++++++++++++++++-----
 2 files changed, 87 insertions(+), 15 deletions(-)

diff --git a/doc/guides/sample_app_ug/ipsec_secgw.rst b/doc/guides/sample_app_ug/ipsec_secgw.rst
index d6cfdbf..ae18acd 100644
--- a/doc/guides/sample_app_ug/ipsec_secgw.rst
+++ b/doc/guides/sample_app_ug/ipsec_secgw.rst
@@ -61,6 +61,12 @@ In case of complete protocol offload, the processing of headers(ESP and outer
 IP header) is done by the hardware and the application does not need to
 add/remove them during outbound/inbound processing.
 
+For inline offloaded outbound traffic, the application will not do the LPM
+lookup for routing, as the port on which the packet has to be forwarded will be
+part of the SA. Security parameters will be configured on that port only, and
+sending the packet on other ports could result in unencrypted packets being
+sent out.
+
 The Path for IPsec Inbound traffic is:
 
 *  Read packets from the port.
@@ -543,7 +549,9 @@ where each options means:
  ``<port_id>``
 
  * Port/device ID of the ethernet/crypto accelerator for which the SA is
-   configured. This option is used when *type* is NOT *no-offload*
+   configured. For *inline-crypto-offload* and *inline-protocol-offload*, this
+   port will be used for routing. The routing table will not be referred in
+   this case.
 
  * Optional: No, if *type* is not *no-offload*
 
diff --git a/examples/ipsec-secgw/ipsec-secgw.c b/examples/ipsec-secgw/ipsec-secgw.c
index c98454a..cfcb9d5 100644
--- a/examples/ipsec-secgw/ipsec-secgw.c
+++ b/examples/ipsec-secgw/ipsec-secgw.c
@@ -585,31 +585,72 @@ process_pkts_outbound_nosp(struct ipsec_ctx *ipsec_ctx,
 		traffic->ip6.num = nb_pkts_out;
 }
 
+static inline int32_t
+get_hop_for_offload_pkt(struct rte_mbuf *pkt)
+{
+	struct ipsec_mbuf_metadata *priv;
+	struct ipsec_sa *sa;
+
+	priv = get_priv(pkt);
+
+	sa = priv->sa;
+	if (unlikely(sa == NULL)) {
+		RTE_LOG(ERR, IPSEC, "SA not saved in private data\n");
+		return -1;
+	}
+
+	return sa->portid;
+}
+
 static inline void
 route4_pkts(struct rt_ctx *rt_ctx, struct rte_mbuf *pkts[], uint8_t nb_pkts)
 {
 	uint32_t hop[MAX_PKT_BURST * 2];
 	uint32_t dst_ip[MAX_PKT_BURST * 2];
+	int32_t pkt_hop = 0;
 	uint16_t i, offset;
+	uint16_t lpm_pkts = 0;
 
 	if (nb_pkts == 0)
 		return;
 
+	/* Need to do an LPM lookup for non-offload packets. Offload packets
+	 * will have port ID in the SA
+	 */
+
 	for (i = 0; i < nb_pkts; i++) {
-		offset = offsetof(struct ip, ip_dst);
-		dst_ip[i] = *rte_pktmbuf_mtod_offset(pkts[i],
-				uint32_t *, offset);
-		dst_ip[i] = rte_be_to_cpu_32(dst_ip[i]);
+		if (!(pkts[i]->ol_flags & PKT_TX_SEC_OFFLOAD)) {
+			/* Security offload not enabled. So an LPM lookup is
+			 * required to get the hop
+			 */
+			offset = offsetof(struct ip, ip_dst);
+			dst_ip[lpm_pkts] = *rte_pktmbuf_mtod_offset(pkts[i],
+					uint32_t *, offset);
+			dst_ip[lpm_pkts] = rte_be_to_cpu_32(dst_ip[lpm_pkts]);
+			lpm_pkts++;
+		}
 	}
 
-	rte_lpm_lookup_bulk((struct rte_lpm *)rt_ctx, dst_ip, hop, nb_pkts);
+	rte_lpm_lookup_bulk((struct rte_lpm *)rt_ctx, dst_ip, hop, lpm_pkts);
+
+	lpm_pkts = 0;
 
 	for (i = 0; i < nb_pkts; i++) {
-		if ((hop[i] & RTE_LPM_LOOKUP_SUCCESS) == 0) {
+		if (pkts[i]->ol_flags & PKT_TX_SEC_OFFLOAD) {
+			/* Read hop from the SA */
+			pkt_hop = get_hop_for_offload_pkt(pkts[i]);
+		} else {
+			/* Need to use hop returned by lookup */
+			pkt_hop = hop[lpm_pkts++];
+			if ((pkt_hop & RTE_LPM_LOOKUP_SUCCESS) == 0)
+				pkt_hop = -1;
+		}
+
+		if (pkt_hop == -1) {
 			rte_pktmbuf_free(pkts[i]);
 			continue;
 		}
-		send_single_packet(pkts[i], hop[i] & 0xff);
+		send_single_packet(pkts[i], pkt_hop & 0xff);
 	}
 }
 
@@ -619,26 +660,49 @@ route6_pkts(struct rt_ctx *rt_ctx, struct rte_mbuf *pkts[], uint8_t nb_pkts)
 	int32_t hop[MAX_PKT_BURST * 2];
 	uint8_t dst_ip[MAX_PKT_BURST * 2][16];
 	uint8_t *ip6_dst;
+	int32_t pkt_hop = 0;
 	uint16_t i, offset;
+	uint16_t lpm_pkts = 0;
 
 	if (nb_pkts == 0)
 		return;
 
+	/* Need to do an LPM lookup for non-offload packets. Offload packets
+	 * will have port ID in the SA
+	 */
+
 	for (i = 0; i < nb_pkts; i++) {
-		offset = offsetof(struct ip6_hdr, ip6_dst);
-		ip6_dst = rte_pktmbuf_mtod_offset(pkts[i], uint8_t *, offset);
-		memcpy(&dst_ip[i][0], ip6_dst, 16);
+		if (!(pkts[i]->ol_flags & PKT_TX_SEC_OFFLOAD)) {
+			/* Security offload not enabled. So an LPM lookup is
+			 * required to get the hop
+			 */
+			offset = offsetof(struct ip6_hdr, ip6_dst);
+			ip6_dst = rte_pktmbuf_mtod_offset(pkts[i], uint8_t *,
+					offset);
+			memcpy(&dst_ip[lpm_pkts][0], ip6_dst, 16);
+			lpm_pkts++;
+		}
 	}
 
-	rte_lpm6_lookup_bulk_func((struct rte_lpm6 *)rt_ctx, dst_ip,
-			hop, nb_pkts);
+	rte_lpm6_lookup_bulk_func((struct rte_lpm6 *)rt_ctx, dst_ip, hop,
+			lpm_pkts);
+
+	lpm_pkts = 0;
 
 	for (i = 0; i < nb_pkts; i++) {
-		if (hop[i] == -1) {
+		if (pkts[i]->ol_flags & PKT_TX_SEC_OFFLOAD) {
+			/* Read hop from the SA */
+			pkt_hop = get_hop_for_offload_pkt(pkts[i]);
+		} else {
+			/* Need to use hop returned by lookup */
+			pkt_hop = hop[lpm_pkts++];
+		}
+
+		if (pkt_hop == -1) {
 			rte_pktmbuf_free(pkts[i]);
 			continue;
 		}
-		send_single_packet(pkts[i], hop[i] & 0xff);
+		send_single_packet(pkts[i], pkt_hop & 0xff);
 	}
 }
 
-- 
2.7.4

Re: [dpdk-dev] [PATCH v3] examples/ipsec-secgw: fix usage of incorrect port

[Akhil Goyal]

Hi Anoob,

On 11/15/2017 3:11 PM, Anoob Joseph wrote:
> When security offload is enabled, the packet should be forwarded on the
> port configured in the SA. Security session will be configured on that
> port only, and sending the packet on other ports could result in
> unencrypted packets being sent out.
> 
> This would have performance improvements too, as the per packet LPM
> lookup would be avoided for IPsec packets, in inline mode.
> 
> Fixes: ec17993a145a ("examples/ipsec-secgw: support security offload")
> 
> Signed-off-by: Anoob Joseph <anoob.joseph@caviumnetworks.com>
> ---
> v3
> * Bug fix (fixed a wrong if condition)
> * Minor changes in documentation
> 
> v2:
> * Updated documentation with the change in behavior for outbound inline
>    offloaded packets.
> 
>   doc/guides/sample_app_ug/ipsec_secgw.rst | 10 +++-
>   examples/ipsec-secgw/ipsec-secgw.c       | 92 +++++++++++++++++++++++++++-----
>   2 files changed, 87 insertions(+), 15 deletions(-)
> 
> diff --git a/doc/guides/sample_app_ug/ipsec_secgw.rst b/doc/guides/sample_app_ug/ipsec_secgw.rst
> index d6cfdbf..ae18acd 100644
> --- a/doc/guides/sample_app_ug/ipsec_secgw.rst
> +++ b/doc/guides/sample_app_ug/ipsec_secgw.rst
> @@ -61,6 +61,12 @@ In case of complete protocol offload, the processing of headers(ESP and outer
>   IP header) is done by the hardware and the application does not need to
>   add/remove them during outbound/inbound processing.
>   
> +For inline offloaded outbound traffic, the application will not do the LPM
> +lookup for routing, as the port on which the packet has to be forwarded will be
> +part of the SA. Security parameters will be configured on that port only, and
> +sending the packet on other ports could result in unencrypted packets being
> +sent out.
> +
>   The Path for IPsec Inbound traffic is:
>   
>   *  Read packets from the port.
> @@ -543,7 +549,9 @@ where each options means:
>    ``<port_id>``
>   
>    * Port/device ID of the ethernet/crypto accelerator for which the SA is
> -   configured. This option is used when *type* is NOT *no-offload*
> +   configured. For *inline-crypto-offload* and *inline-protocol-offload*, this
> +   port will be used for routing. The routing table will not be referred in
> +   this case.
>   
>    * Optional: No, if *type* is not *no-offload*
>   
> diff --git a/examples/ipsec-secgw/ipsec-secgw.c b/examples/ipsec-secgw/ipsec-secgw.c
> index c98454a..cfcb9d5 100644
> --- a/examples/ipsec-secgw/ipsec-secgw.c
> +++ b/examples/ipsec-secgw/ipsec-secgw.c
> @@ -585,31 +585,72 @@ process_pkts_outbound_nosp(struct ipsec_ctx *ipsec_ctx,
>   		traffic->ip6.num = nb_pkts_out;
>   }
>   
> +static inline int32_t
> +get_hop_for_offload_pkt(struct rte_mbuf *pkt)
> +{
> +	struct ipsec_mbuf_metadata *priv;
> +	struct ipsec_sa *sa;
> +
> +	priv = get_priv(pkt);
> +
> +	sa = priv->sa;
> +	if (unlikely(sa == NULL)) {
> +		RTE_LOG(ERR, IPSEC, "SA not saved in private data\n");
> +		return -1;
> +	}
> +
> +	return sa->portid;
> +}
> +
>   static inline void
>   route4_pkts(struct rt_ctx *rt_ctx, struct rte_mbuf *pkts[], uint8_t nb_pkts)
>   {
>   	uint32_t hop[MAX_PKT_BURST * 2];
>   	uint32_t dst_ip[MAX_PKT_BURST * 2];
> +	int32_t pkt_hop = 0;
>   	uint16_t i, offset;
> +	uint16_t lpm_pkts = 0;
>   
>   	if (nb_pkts == 0)
>   		return;
>   
> +	/* Need to do an LPM lookup for non-offload packets. Offload packets
> +	 * will have port ID in the SA
> +	 */
> +
>   	for (i = 0; i < nb_pkts; i++) {
> -		offset = offsetof(struct ip, ip_dst);
> -		dst_ip[i] = *rte_pktmbuf_mtod_offset(pkts[i],
> -				uint32_t *, offset);
> -		dst_ip[i] = rte_be_to_cpu_32(dst_ip[i]);
> +		if (!(pkts[i]->ol_flags & PKT_TX_SEC_OFFLOAD)) {
> +			/* Security offload not enabled. So an LPM lookup is
> +			 * required to get the hop
> +			 */
> +			offset = offsetof(struct ip, ip_dst);
> +			dst_ip[lpm_pkts] = *rte_pktmbuf_mtod_offset(pkts[i],
> +					uint32_t *, offset);
> +			dst_ip[lpm_pkts] = rte_be_to_cpu_32(dst_ip[lpm_pkts]);
> +			lpm_pkts++;
> +		}
>   	}
>   
> -	rte_lpm_lookup_bulk((struct rte_lpm *)rt_ctx, dst_ip, hop, nb_pkts);
> +	rte_lpm_lookup_bulk((struct rte_lpm *)rt_ctx, dst_ip, hop, lpm_pkts);
> +
> +	lpm_pkts = 0;
>   
>   	for (i = 0; i < nb_pkts; i++) {
> -		if ((hop[i] & RTE_LPM_LOOKUP_SUCCESS) == 0) {
> +		if (pkts[i]->ol_flags & PKT_TX_SEC_OFFLOAD) {
> +			/* Read hop from the SA */
> +			pkt_hop = get_hop_for_offload_pkt(pkts[i]);
> +		} else {
> +			/* Need to use hop returned by lookup */
> +			pkt_hop = hop[lpm_pkts++];
> +			if ((pkt_hop & RTE_LPM_LOOKUP_SUCCESS) == 0)
> +				pkt_hop = -1;
> +		}
> +
I believe the following check is redundant for non inline case. I 
believe get_hop_for_offload_pkt can also set the RTE_LPM_LOOKUP_SUCCESS 
if route is success and take the (pkt_hop & RTE_LPM_LOOKUP_SUCCESS) == 0 
check outside the if else block and free the packet if it is unsuccessful.

Same comment for route6_pkts. Checking with -1 may not be a good idea if 
we have a flag available for the same.
Others can comment.

> +		if (pkt_hop == -1) {
>   			rte_pktmbuf_free(pkts[i]);
>   			continue;
>   		}
> -		send_single_packet(pkts[i], hop[i] & 0xff);
> +		send_single_packet(pkts[i], pkt_hop & 0xff);
>   	}
>   }
>   
> @@ -619,26 +660,49 @@ route6_pkts(struct rt_ctx *rt_ctx, struct rte_mbuf *pkts[], uint8_t nb_pkts)
>   	int32_t hop[MAX_PKT_BURST * 2];
>   	uint8_t dst_ip[MAX_PKT_BURST * 2][16];
>   	uint8_t *ip6_dst;
> +	int32_t pkt_hop = 0;
>   	uint16_t i, offset;
> +	uint16_t lpm_pkts = 0;
>   
>   	if (nb_pkts == 0)
>   		return;
>   
> +	/* Need to do an LPM lookup for non-offload packets. Offload packets
> +	 * will have port ID in the SA
> +	 */
> +
>   	for (i = 0; i < nb_pkts; i++) {
> -		offset = offsetof(struct ip6_hdr, ip6_dst);
> -		ip6_dst = rte_pktmbuf_mtod_offset(pkts[i], uint8_t *, offset);
> -		memcpy(&dst_ip[i][0], ip6_dst, 16);
> +		if (!(pkts[i]->ol_flags & PKT_TX_SEC_OFFLOAD)) {
> +			/* Security offload not enabled. So an LPM lookup is
> +			 * required to get the hop
> +			 */
> +			offset = offsetof(struct ip6_hdr, ip6_dst);
> +			ip6_dst = rte_pktmbuf_mtod_offset(pkts[i], uint8_t *,
> +					offset);
> +			memcpy(&dst_ip[lpm_pkts][0], ip6_dst, 16);
> +			lpm_pkts++;
> +		}
>   	}
>   
> -	rte_lpm6_lookup_bulk_func((struct rte_lpm6 *)rt_ctx, dst_ip,
> -			hop, nb_pkts);
> +	rte_lpm6_lookup_bulk_func((struct rte_lpm6 *)rt_ctx, dst_ip, hop,
> +			lpm_pkts);
> +
> +	lpm_pkts = 0;
>   
>   	for (i = 0; i < nb_pkts; i++) {
> -		if (hop[i] == -1) {
> +		if (pkts[i]->ol_flags & PKT_TX_SEC_OFFLOAD) {
> +			/* Read hop from the SA */
> +			pkt_hop = get_hop_for_offload_pkt(pkts[i]);
> +		} else {
> +			/* Need to use hop returned by lookup */
> +			pkt_hop = hop[lpm_pkts++];
> +		}
> +
> +		if (pkt_hop == -1) {
>   			rte_pktmbuf_free(pkts[i]);
>   			continue;
>   		}
> -		send_single_packet(pkts[i], hop[i] & 0xff);
> +		send_single_packet(pkts[i], pkt_hop & 0xff);
>   	}
>   }
>   
> 

Re: [dpdk-dev] [PATCH v3] examples/ipsec-secgw: fix usage of incorrect port

[Anoob]

Hi Akhil,

Please see inline.

Thanks,

Anoob


On 11/24/2017 02:58 PM, Akhil Goyal wrote:
> Hi Anoob,
>
> On 11/15/2017 3:11 PM, Anoob Joseph wrote:
>> When security offload is enabled, the packet should be forwarded on the
>> port configured in the SA. Security session will be configured on that
>> port only, and sending the packet on other ports could result in
>> unencrypted packets being sent out.
>>
>> This would have performance improvements too, as the per packet LPM
>> lookup would be avoided for IPsec packets, in inline mode.
>>
>> Fixes: ec17993a145a ("examples/ipsec-secgw: support security offload")
>>
>> Signed-off-by: Anoob Joseph <anoob.joseph@caviumnetworks.com>
>> ---
>> v3
>> * Bug fix (fixed a wrong if condition)
>> * Minor changes in documentation
>>
>> v2:
>> * Updated documentation with the change in behavior for outbound inline
>>    offloaded packets.
>>
>>   doc/guides/sample_app_ug/ipsec_secgw.rst | 10 +++-
>>   examples/ipsec-secgw/ipsec-secgw.c       | 92 
>> +++++++++++++++++++++++++++-----
>>   2 files changed, 87 insertions(+), 15 deletions(-)
>>
>> diff --git a/doc/guides/sample_app_ug/ipsec_secgw.rst 
>> b/doc/guides/sample_app_ug/ipsec_secgw.rst
>> index d6cfdbf..ae18acd 100644
>> --- a/doc/guides/sample_app_ug/ipsec_secgw.rst
>> +++ b/doc/guides/sample_app_ug/ipsec_secgw.rst
>> @@ -61,6 +61,12 @@ In case of complete protocol offload, the 
>> processing of headers(ESP and outer
>>   IP header) is done by the hardware and the application does not 
>> need to
>>   add/remove them during outbound/inbound processing.
>>   +For inline offloaded outbound traffic, the application will not do 
>> the LPM
>> +lookup for routing, as the port on which the packet has to be 
>> forwarded will be
>> +part of the SA. Security parameters will be configured on that port 
>> only, and
>> +sending the packet on other ports could result in unencrypted 
>> packets being
>> +sent out.
>> +
>>   The Path for IPsec Inbound traffic is:
>>     *  Read packets from the port.
>> @@ -543,7 +549,9 @@ where each options means:
>>    ``<port_id>``
>>      * Port/device ID of the ethernet/crypto accelerator for which 
>> the SA is
>> -   configured. This option is used when *type* is NOT *no-offload*
>> +   configured. For *inline-crypto-offload* and 
>> *inline-protocol-offload*, this
>> +   port will be used for routing. The routing table will not be 
>> referred in
>> +   this case.
>>      * Optional: No, if *type* is not *no-offload*
>>   diff --git a/examples/ipsec-secgw/ipsec-secgw.c 
>> b/examples/ipsec-secgw/ipsec-secgw.c
>> index c98454a..cfcb9d5 100644
>> --- a/examples/ipsec-secgw/ipsec-secgw.c
>> +++ b/examples/ipsec-secgw/ipsec-secgw.c
>> @@ -585,31 +585,72 @@ process_pkts_outbound_nosp(struct ipsec_ctx 
>> *ipsec_ctx,
>>           traffic->ip6.num = nb_pkts_out;
>>   }
>>   +static inline int32_t
>> +get_hop_for_offload_pkt(struct rte_mbuf *pkt)
>> +{
>> +    struct ipsec_mbuf_metadata *priv;
>> +    struct ipsec_sa *sa;
>> +
>> +    priv = get_priv(pkt);
>> +
>> +    sa = priv->sa;
>> +    if (unlikely(sa == NULL)) {
>> +        RTE_LOG(ERR, IPSEC, "SA not saved in private data\n");
>> +        return -1;
>> +    }
>> +
>> +    return sa->portid;
>> +}
>> +
>>   static inline void
>>   route4_pkts(struct rt_ctx *rt_ctx, struct rte_mbuf *pkts[], uint8_t 
>> nb_pkts)
>>   {
>>       uint32_t hop[MAX_PKT_BURST * 2];
>>       uint32_t dst_ip[MAX_PKT_BURST * 2];
>> +    int32_t pkt_hop = 0;
>>       uint16_t i, offset;
>> +    uint16_t lpm_pkts = 0;
>>         if (nb_pkts == 0)
>>           return;
>>   +    /* Need to do an LPM lookup for non-offload packets. Offload 
>> packets
>> +     * will have port ID in the SA
>> +     */
>> +
>>       for (i = 0; i < nb_pkts; i++) {
>> -        offset = offsetof(struct ip, ip_dst);
>> -        dst_ip[i] = *rte_pktmbuf_mtod_offset(pkts[i],
>> -                uint32_t *, offset);
>> -        dst_ip[i] = rte_be_to_cpu_32(dst_ip[i]);
>> +        if (!(pkts[i]->ol_flags & PKT_TX_SEC_OFFLOAD)) {
>> +            /* Security offload not enabled. So an LPM lookup is
>> +             * required to get the hop
>> +             */
>> +            offset = offsetof(struct ip, ip_dst);
>> +            dst_ip[lpm_pkts] = *rte_pktmbuf_mtod_offset(pkts[i],
>> +                    uint32_t *, offset);
>> +            dst_ip[lpm_pkts] = rte_be_to_cpu_32(dst_ip[lpm_pkts]);
>> +            lpm_pkts++;
>> +        }
>>       }
>>   -    rte_lpm_lookup_bulk((struct rte_lpm *)rt_ctx, dst_ip, hop, 
>> nb_pkts);
>> +    rte_lpm_lookup_bulk((struct rte_lpm *)rt_ctx, dst_ip, hop, 
>> lpm_pkts);
>> +
>> +    lpm_pkts = 0;
>>         for (i = 0; i < nb_pkts; i++) {
>> -        if ((hop[i] & RTE_LPM_LOOKUP_SUCCESS) == 0) {
>> +        if (pkts[i]->ol_flags & PKT_TX_SEC_OFFLOAD) {
>> +            /* Read hop from the SA */
>> +            pkt_hop = get_hop_for_offload_pkt(pkts[i]);
>> +        } else {
>> +            /* Need to use hop returned by lookup */
>> +            pkt_hop = hop[lpm_pkts++];
>> +            if ((pkt_hop & RTE_LPM_LOOKUP_SUCCESS) == 0)
>> +                pkt_hop = -1;
>> +        }
>> +
> I believe the following check is redundant for non inline case. I 
> believe get_hop_for_offload_pkt can also set the 
> RTE_LPM_LOOKUP_SUCCESS if route is success and take the (pkt_hop & 
> RTE_LPM_LOOKUP_SUCCESS) == 0 check outside the if else block and free 
> the packet if it is unsuccessful.
>
> Same comment for route6_pkts. Checking with -1 may not be a good idea 
> if we have a flag available for the same.
> Others can comment.
The problem is ipv4 & ipv6 LPM lookups return different error values, 
but we are using a single routine to get the hop for offload packets. 
The flag(RTE_LPM_LOOKUP_SUCCESS) is only for ipv4 lookups. For ipv6, 
error is -1. If we need a cleaner solution, we can have ipv4 & ipv6 
variants of "get_hop_for_offload_pkt". But that would be repetition of 
some code.
>
>> +        if (pkt_hop == -1) {
>>               rte_pktmbuf_free(pkts[i]);
>>               continue;
>>           }
>> -        send_single_packet(pkts[i], hop[i] & 0xff);
>> +        send_single_packet(pkts[i], pkt_hop & 0xff);
>>       }
>>   }
>>   @@ -619,26 +660,49 @@ route6_pkts(struct rt_ctx *rt_ctx, struct 
>> rte_mbuf *pkts[], uint8_t nb_pkts)
>>       int32_t hop[MAX_PKT_BURST * 2];
>>       uint8_t dst_ip[MAX_PKT_BURST * 2][16];
>>       uint8_t *ip6_dst;
>> +    int32_t pkt_hop = 0;
>>       uint16_t i, offset;
>> +    uint16_t lpm_pkts = 0;
>>         if (nb_pkts == 0)
>>           return;
>>   +    /* Need to do an LPM lookup for non-offload packets. Offload 
>> packets
>> +     * will have port ID in the SA
>> +     */
>> +
>>       for (i = 0; i < nb_pkts; i++) {
>> -        offset = offsetof(struct ip6_hdr, ip6_dst);
>> -        ip6_dst = rte_pktmbuf_mtod_offset(pkts[i], uint8_t *, offset);
>> -        memcpy(&dst_ip[i][0], ip6_dst, 16);
>> +        if (!(pkts[i]->ol_flags & PKT_TX_SEC_OFFLOAD)) {
>> +            /* Security offload not enabled. So an LPM lookup is
>> +             * required to get the hop
>> +             */
>> +            offset = offsetof(struct ip6_hdr, ip6_dst);
>> +            ip6_dst = rte_pktmbuf_mtod_offset(pkts[i], uint8_t *,
>> +                    offset);
>> +            memcpy(&dst_ip[lpm_pkts][0], ip6_dst, 16);
>> +            lpm_pkts++;
>> +        }
>>       }
>>   -    rte_lpm6_lookup_bulk_func((struct rte_lpm6 *)rt_ctx, dst_ip,
>> -            hop, nb_pkts);
>> +    rte_lpm6_lookup_bulk_func((struct rte_lpm6 *)rt_ctx, dst_ip, hop,
>> +            lpm_pkts);
>> +
>> +    lpm_pkts = 0;
>>         for (i = 0; i < nb_pkts; i++) {
>> -        if (hop[i] == -1) {
>> +        if (pkts[i]->ol_flags & PKT_TX_SEC_OFFLOAD) {
>> +            /* Read hop from the SA */
>> +            pkt_hop = get_hop_for_offload_pkt(pkts[i]);
>> +        } else {
>> +            /* Need to use hop returned by lookup */
>> +            pkt_hop = hop[lpm_pkts++];
>> +        }
>> +
>> +        if (pkt_hop == -1) {
>>               rte_pktmbuf_free(pkts[i]);
>>               continue;
>>           }
>> -        send_single_packet(pkts[i], hop[i] & 0xff);
>> +        send_single_packet(pkts[i], pkt_hop & 0xff);
>>       }
>>   }
>>
>

Re: [dpdk-dev] [PATCH v3] examples/ipsec-secgw: fix usage of incorrect port

[Akhil Goyal]

Hi Anoob,

On 11/24/2017 3:28 PM, Anoob wrote:
>>>   static inline void
>>>   route4_pkts(struct rt_ctx *rt_ctx, struct rte_mbuf *pkts[], uint8_t 
>>> nb_pkts)
>>>   {
>>>       uint32_t hop[MAX_PKT_BURST * 2];
>>>       uint32_t dst_ip[MAX_PKT_BURST * 2];
>>> +    int32_t pkt_hop = 0;
>>>       uint16_t i, offset;
>>> +    uint16_t lpm_pkts = 0;
>>>         if (nb_pkts == 0)
>>>           return;
>>>   +    /* Need to do an LPM lookup for non-offload packets. Offload 
>>> packets
>>> +     * will have port ID in the SA
>>> +     */
>>> +
>>>       for (i = 0; i < nb_pkts; i++) {
>>> -        offset = offsetof(struct ip, ip_dst);
>>> -        dst_ip[i] = *rte_pktmbuf_mtod_offset(pkts[i],
>>> -                uint32_t *, offset);
>>> -        dst_ip[i] = rte_be_to_cpu_32(dst_ip[i]);
>>> +        if (!(pkts[i]->ol_flags & PKT_TX_SEC_OFFLOAD)) {
>>> +            /* Security offload not enabled. So an LPM lookup is
>>> +             * required to get the hop
>>> +             */
>>> +            offset = offsetof(struct ip, ip_dst);
>>> +            dst_ip[lpm_pkts] = *rte_pktmbuf_mtod_offset(pkts[i],
>>> +                    uint32_t *, offset);
>>> +            dst_ip[lpm_pkts] = rte_be_to_cpu_32(dst_ip[lpm_pkts]);
>>> +            lpm_pkts++;
>>> +        }
>>>       }
>>>   -    rte_lpm_lookup_bulk((struct rte_lpm *)rt_ctx, dst_ip, hop, 
>>> nb_pkts);
>>> +    rte_lpm_lookup_bulk((struct rte_lpm *)rt_ctx, dst_ip, hop, 
>>> lpm_pkts);
>>> +
>>> +    lpm_pkts = 0;
>>>         for (i = 0; i < nb_pkts; i++) {
>>> -        if ((hop[i] & RTE_LPM_LOOKUP_SUCCESS) == 0) {
>>> +        if (pkts[i]->ol_flags & PKT_TX_SEC_OFFLOAD) {
>>> +            /* Read hop from the SA */
>>> +            pkt_hop = get_hop_for_offload_pkt(pkts[i]);
>>> +        } else {
>>> +            /* Need to use hop returned by lookup */
>>> +            pkt_hop = hop[lpm_pkts++];
>>> +            if ((pkt_hop & RTE_LPM_LOOKUP_SUCCESS) == 0)
>>> +                pkt_hop = -1;
>>> +        }
>>> +
>> I believe the following check is redundant for non inline case. I 
>> believe get_hop_for_offload_pkt can also set the 
>> RTE_LPM_LOOKUP_SUCCESS if route is success and take the (pkt_hop & 
>> RTE_LPM_LOOKUP_SUCCESS) == 0 check outside the if else block and free 
>> the packet if it is unsuccessful.
>>
>> Same comment for route6_pkts. Checking with -1 may not be a good idea 
>> if we have a flag available for the same.
>> Others can comment.
> The problem is ipv4 & ipv6 LPM lookups return different error values, 
> but we are using a single routine to get the hop for offload packets. 
> The flag(RTE_LPM_LOOKUP_SUCCESS) is only for ipv4 lookups. For ipv6, 
> error is -1. If we need a cleaner solution, we can have ipv4 & ipv6 
> variants of "get_hop_for_offload_pkt". But that would be repetition of 
> some code.

my concern over this patch is that there is an addition of an extra 
check in the non inline case and we can get rid of that with some 
changes in the code(lib/app). Regarding route6_pkts, the code looks 
cleaner than route4_pkts

-Akhil

Re: [dpdk-dev] [PATCH v3] examples/ipsec-secgw: fix usage of incorrect port

[Anoob Joseph]

Hi Akhil,


On 24-11-2017 16:19, Akhil Goyal wrote:
> Hi Anoob,
>
> On 11/24/2017 3:28 PM, Anoob wrote:
>>>>   static inline void
>>>>   route4_pkts(struct rt_ctx *rt_ctx, struct rte_mbuf *pkts[], 
>>>> uint8_t nb_pkts)
>>>>   {
>>>>       uint32_t hop[MAX_PKT_BURST * 2];
>>>>       uint32_t dst_ip[MAX_PKT_BURST * 2];
>>>> +    int32_t pkt_hop = 0;
>>>>       uint16_t i, offset;
>>>> +    uint16_t lpm_pkts = 0;
>>>>         if (nb_pkts == 0)
>>>>           return;
>>>>   +    /* Need to do an LPM lookup for non-offload packets. Offload 
>>>> packets
>>>> +     * will have port ID in the SA
>>>> +     */
>>>> +
>>>>       for (i = 0; i < nb_pkts; i++) {
>>>> -        offset = offsetof(struct ip, ip_dst);
>>>> -        dst_ip[i] = *rte_pktmbuf_mtod_offset(pkts[i],
>>>> -                uint32_t *, offset);
>>>> -        dst_ip[i] = rte_be_to_cpu_32(dst_ip[i]);
>>>> +        if (!(pkts[i]->ol_flags & PKT_TX_SEC_OFFLOAD)) {
>>>> +            /* Security offload not enabled. So an LPM lookup is
>>>> +             * required to get the hop
>>>> +             */
>>>> +            offset = offsetof(struct ip, ip_dst);
>>>> +            dst_ip[lpm_pkts] = *rte_pktmbuf_mtod_offset(pkts[i],
>>>> +                    uint32_t *, offset);
>>>> +            dst_ip[lpm_pkts] = rte_be_to_cpu_32(dst_ip[lpm_pkts]);
>>>> +            lpm_pkts++;
>>>> +        }
>>>>       }
>>>>   -    rte_lpm_lookup_bulk((struct rte_lpm *)rt_ctx, dst_ip, hop, 
>>>> nb_pkts);
>>>> +    rte_lpm_lookup_bulk((struct rte_lpm *)rt_ctx, dst_ip, hop, 
>>>> lpm_pkts);
>>>> +
>>>> +    lpm_pkts = 0;
>>>>         for (i = 0; i < nb_pkts; i++) {
>>>> -        if ((hop[i] & RTE_LPM_LOOKUP_SUCCESS) == 0) {
>>>> +        if (pkts[i]->ol_flags & PKT_TX_SEC_OFFLOAD) {
>>>> +            /* Read hop from the SA */
>>>> +            pkt_hop = get_hop_for_offload_pkt(pkts[i]);
>>>> +        } else {
>>>> +            /* Need to use hop returned by lookup */
>>>> +            pkt_hop = hop[lpm_pkts++];
>>>> +            if ((pkt_hop & RTE_LPM_LOOKUP_SUCCESS) == 0)
>>>> +                pkt_hop = -1;
>>>> +        }
>>>> +
>>> I believe the following check is redundant for non inline case. I 
>>> believe get_hop_for_offload_pkt can also set the 
>>> RTE_LPM_LOOKUP_SUCCESS if route is success and take the (pkt_hop & 
>>> RTE_LPM_LOOKUP_SUCCESS) == 0 check outside the if else block and 
>>> free the packet if it is unsuccessful.
>>>
>>> Same comment for route6_pkts. Checking with -1 may not be a good 
>>> idea if we have a flag available for the same.
>>> Others can comment.
>> The problem is ipv4 & ipv6 LPM lookups return different error values, 
>> but we are using a single routine to get the hop for offload packets. 
>> The flag(RTE_LPM_LOOKUP_SUCCESS) is only for ipv4 lookups. For ipv6, 
>> error is -1. If we need a cleaner solution, we can have ipv4 & ipv6 
>> variants of "get_hop_for_offload_pkt". But that would be repetition 
>> of some code.
>
> my concern over this patch is that there is an addition of an extra 
> check in the non inline case and we can get rid of that with some 
> changes in the code(lib/app). Regarding route6_pkts, the code looks 
> cleaner than route4_pkts
If we have ipv4 and ipv6 variants of the "get_hop_for_offload_packet" 
function, the code would look much cleaner. Shall I update the patch 
with such a change and send v4?
>
>
> -Akhil

Thanks,
Anoob

Re: [dpdk-dev] [PATCH v3] examples/ipsec-secgw: fix usage of incorrect port

[Akhil Goyal]

Hi Anoob,
On 11/29/2017 9:51 AM, Anoob Joseph wrote:
> Hi Akhil,
> 
> 
> On 24-11-2017 16:19, Akhil Goyal wrote:
>> Hi Anoob,
>>
>> On 11/24/2017 3:28 PM, Anoob wrote:
>>>>>   static inline void
>>>>>   route4_pkts(struct rt_ctx *rt_ctx, struct rte_mbuf *pkts[], 
>>>>> uint8_t nb_pkts)
>>>>>   {
>>>>>       uint32_t hop[MAX_PKT_BURST * 2];
>>>>>       uint32_t dst_ip[MAX_PKT_BURST * 2];
>>>>> +    int32_t pkt_hop = 0;
>>>>>       uint16_t i, offset;
>>>>> +    uint16_t lpm_pkts = 0;
>>>>>         if (nb_pkts == 0)
>>>>>           return;
>>>>>   +    /* Need to do an LPM lookup for non-offload packets. Offload 
>>>>> packets
>>>>> +     * will have port ID in the SA
>>>>> +     */
>>>>> +
>>>>>       for (i = 0; i < nb_pkts; i++) {
>>>>> -        offset = offsetof(struct ip, ip_dst);
>>>>> -        dst_ip[i] = *rte_pktmbuf_mtod_offset(pkts[i],
>>>>> -                uint32_t *, offset);
>>>>> -        dst_ip[i] = rte_be_to_cpu_32(dst_ip[i]);
>>>>> +        if (!(pkts[i]->ol_flags & PKT_TX_SEC_OFFLOAD)) {
>>>>> +            /* Security offload not enabled. So an LPM lookup is
>>>>> +             * required to get the hop
>>>>> +             */
>>>>> +            offset = offsetof(struct ip, ip_dst);
>>>>> +            dst_ip[lpm_pkts] = *rte_pktmbuf_mtod_offset(pkts[i],
>>>>> +                    uint32_t *, offset);
>>>>> +            dst_ip[lpm_pkts] = rte_be_to_cpu_32(dst_ip[lpm_pkts]);
>>>>> +            lpm_pkts++;
>>>>> +        }
>>>>>       }
>>>>>   -    rte_lpm_lookup_bulk((struct rte_lpm *)rt_ctx, dst_ip, hop, 
>>>>> nb_pkts);
>>>>> +    rte_lpm_lookup_bulk((struct rte_lpm *)rt_ctx, dst_ip, hop, 
>>>>> lpm_pkts);
>>>>> +
>>>>> +    lpm_pkts = 0;
>>>>>         for (i = 0; i < nb_pkts; i++) {
>>>>> -        if ((hop[i] & RTE_LPM_LOOKUP_SUCCESS) == 0) {
>>>>> +        if (pkts[i]->ol_flags & PKT_TX_SEC_OFFLOAD) {
>>>>> +            /* Read hop from the SA */
>>>>> +            pkt_hop = get_hop_for_offload_pkt(pkts[i]);
>>>>> +        } else {
>>>>> +            /* Need to use hop returned by lookup */
>>>>> +            pkt_hop = hop[lpm_pkts++];
>>>>> +            if ((pkt_hop & RTE_LPM_LOOKUP_SUCCESS) == 0)
>>>>> +                pkt_hop = -1;
>>>>> +        }
>>>>> +
>>>> I believe the following check is redundant for non inline case. I 
>>>> believe get_hop_for_offload_pkt can also set the 
>>>> RTE_LPM_LOOKUP_SUCCESS if route is success and take the (pkt_hop & 
>>>> RTE_LPM_LOOKUP_SUCCESS) == 0 check outside the if else block and 
>>>> free the packet if it is unsuccessful.
>>>>
>>>> Same comment for route6_pkts. Checking with -1 may not be a good 
>>>> idea if we have a flag available for the same.
>>>> Others can comment.
>>> The problem is ipv4 & ipv6 LPM lookups return different error values, 
>>> but we are using a single routine to get the hop for offload packets. 
>>> The flag(RTE_LPM_LOOKUP_SUCCESS) is only for ipv4 lookups. For ipv6, 
>>> error is -1. If we need a cleaner solution, we can have ipv4 & ipv6 
>>> variants of "get_hop_for_offload_pkt". But that would be repetition 
>>> of some code.
>>
>> my concern over this patch is that there is an addition of an extra 
>> check in the non inline case and we can get rid of that with some 
>> changes in the code(lib/app). Regarding route6_pkts, the code looks 
>> cleaner than route4_pkts
> If we have ipv4 and ipv6 variants of the "get_hop_for_offload_packet" 
> function, the code would look much cleaner. Shall I update the patch 
> with such a change and send v4?

I believe we shall get rid of "RTE_LPM_LOOKUP_SUCCESS" from the 
rte_lpm_lookup_bulk(), we shall have similar error flags for v4 and v6 
APIs. Either we can have RTE_LPM_LOOKUP_SUCCESS or -1 as check for errors.
Sergio can comment on this.

Duplicating code for get_hop_for_offload_packet may not be a good idea.

-Akhil

Re: [dpdk-dev] [PATCH v3] examples/ipsec-secgw: fix usage of incorrect port

[Anoob]

Hi Akhil,

On 12/04/2017 01:19 PM, Akhil Goyal wrote:
> Hi Anoob,
> On 11/29/2017 9:51 AM, Anoob Joseph wrote:
>> Hi Akhil,
>>
>>
>> On 24-11-2017 16:19, Akhil Goyal wrote:
>>> Hi Anoob,
>>>
>>> On 11/24/2017 3:28 PM, Anoob wrote:
>>>>>>   static inline void
>>>>>>   route4_pkts(struct rt_ctx *rt_ctx, struct rte_mbuf *pkts[], 
>>>>>> uint8_t nb_pkts)
>>>>>>   {
>>>>>>       uint32_t hop[MAX_PKT_BURST * 2];
>>>>>>       uint32_t dst_ip[MAX_PKT_BURST * 2];
>>>>>> +    int32_t pkt_hop = 0;
>>>>>>       uint16_t i, offset;
>>>>>> +    uint16_t lpm_pkts = 0;
>>>>>>         if (nb_pkts == 0)
>>>>>>           return;
>>>>>>   +    /* Need to do an LPM lookup for non-offload packets. 
>>>>>> Offload packets
>>>>>> +     * will have port ID in the SA
>>>>>> +     */
>>>>>> +
>>>>>>       for (i = 0; i < nb_pkts; i++) {
>>>>>> -        offset = offsetof(struct ip, ip_dst);
>>>>>> -        dst_ip[i] = *rte_pktmbuf_mtod_offset(pkts[i],
>>>>>> -                uint32_t *, offset);
>>>>>> -        dst_ip[i] = rte_be_to_cpu_32(dst_ip[i]);
>>>>>> +        if (!(pkts[i]->ol_flags & PKT_TX_SEC_OFFLOAD)) {
>>>>>> +            /* Security offload not enabled. So an LPM lookup is
>>>>>> +             * required to get the hop
>>>>>> +             */
>>>>>> +            offset = offsetof(struct ip, ip_dst);
>>>>>> +            dst_ip[lpm_pkts] = *rte_pktmbuf_mtod_offset(pkts[i],
>>>>>> +                    uint32_t *, offset);
>>>>>> +            dst_ip[lpm_pkts] = rte_be_to_cpu_32(dst_ip[lpm_pkts]);
>>>>>> +            lpm_pkts++;
>>>>>> +        }
>>>>>>       }
>>>>>>   -    rte_lpm_lookup_bulk((struct rte_lpm *)rt_ctx, dst_ip, hop, 
>>>>>> nb_pkts);
>>>>>> +    rte_lpm_lookup_bulk((struct rte_lpm *)rt_ctx, dst_ip, hop, 
>>>>>> lpm_pkts);
>>>>>> +
>>>>>> +    lpm_pkts = 0;
>>>>>>         for (i = 0; i < nb_pkts; i++) {
>>>>>> -        if ((hop[i] & RTE_LPM_LOOKUP_SUCCESS) == 0) {
>>>>>> +        if (pkts[i]->ol_flags & PKT_TX_SEC_OFFLOAD) {
>>>>>> +            /* Read hop from the SA */
>>>>>> +            pkt_hop = get_hop_for_offload_pkt(pkts[i]);
>>>>>> +        } else {
>>>>>> +            /* Need to use hop returned by lookup */
>>>>>> +            pkt_hop = hop[lpm_pkts++];
>>>>>> +            if ((pkt_hop & RTE_LPM_LOOKUP_SUCCESS) == 0)
>>>>>> +                pkt_hop = -1;
>>>>>> +        }
>>>>>> +
>>>>> I believe the following check is redundant for non inline case. I 
>>>>> believe get_hop_for_offload_pkt can also set the 
>>>>> RTE_LPM_LOOKUP_SUCCESS if route is success and take the (pkt_hop & 
>>>>> RTE_LPM_LOOKUP_SUCCESS) == 0 check outside the if else block and 
>>>>> free the packet if it is unsuccessful.
>>>>>
>>>>> Same comment for route6_pkts. Checking with -1 may not be a good 
>>>>> idea if we have a flag available for the same.
>>>>> Others can comment.
>>>> The problem is ipv4 & ipv6 LPM lookups return different error 
>>>> values, but we are using a single routine to get the hop for 
>>>> offload packets. The flag(RTE_LPM_LOOKUP_SUCCESS) is only for ipv4 
>>>> lookups. For ipv6, error is -1. If we need a cleaner solution, we 
>>>> can have ipv4 & ipv6 variants of "get_hop_for_offload_pkt". But 
>>>> that would be repetition of some code.
>>>
>>> my concern over this patch is that there is an addition of an extra 
>>> check in the non inline case and we can get rid of that with some 
>>> changes in the code(lib/app). Regarding route6_pkts, the code looks 
>>> cleaner than route4_pkts
>> If we have ipv4 and ipv6 variants of the "get_hop_for_offload_packet" 
>> function, the code would look much cleaner. Shall I update the patch 
>> with such a change and send v4?
>
> I believe we shall get rid of "RTE_LPM_LOOKUP_SUCCESS" from the 
> rte_lpm_lookup_bulk(), we shall have similar error flags for v4 and v6 
> APIs. Either we can have RTE_LPM_LOOKUP_SUCCESS or -1 as check for 
> errors.
This will call for an ABI change. And LPM library has multiple variants 
for v4 & v6 lookups. We will need to modify all such instances. I've 
CCed Bruce for his opinion on this matter. If maintainers can decide on 
how to address this properly, I can plan my next steps accordingly.
> Sergio can comment on this.
>
> Duplicating code for get_hop_for_offload_packet may not be a good idea.
>
> -Akhil
>

Re: [dpdk-dev] [PATCH v3] examples/ipsec-secgw: fix usage of incorrect port

[Radu Nicolau]

Hi,


On 12/6/2017 11:08 AM, Anoob wrote:
> Hi Akhil,
>
> On 12/04/2017 01:19 PM, Akhil Goyal wrote:
>> Hi Anoob,
>> On 11/29/2017 9:51 AM, Anoob Joseph wrote:
>>> Hi Akhil,
>>>
>>>
>>> On 24-11-2017 16:19, Akhil Goyal wrote:
>>>> Hi Anoob,
>>>>
>>>> On 11/24/2017 3:28 PM, Anoob wrote:
>>>>>>>   static inline void
>>>>>>>   route4_pkts(struct rt_ctx *rt_ctx, struct rte_mbuf *pkts[], 
>>>>>>> uint8_t nb_pkts)
>>>>>>>   {
>>>>>>>       uint32_t hop[MAX_PKT_BURST * 2];
>>>>>>>       uint32_t dst_ip[MAX_PKT_BURST * 2];
>>>>>>> +    int32_t pkt_hop = 0;
>>>>>>>       uint16_t i, offset;
>>>>>>> +    uint16_t lpm_pkts = 0;
>>>>>>>         if (nb_pkts == 0)
>>>>>>>           return;
>>>>>>>   +    /* Need to do an LPM lookup for non-offload packets. 
>>>>>>> Offload packets
>>>>>>> +     * will have port ID in the SA
>>>>>>> +     */
>>>>>>> +
>>>>>>>       for (i = 0; i < nb_pkts; i++) {
>>>>>>> -        offset = offsetof(struct ip, ip_dst);
>>>>>>> -        dst_ip[i] = *rte_pktmbuf_mtod_offset(pkts[i],
>>>>>>> -                uint32_t *, offset);
>>>>>>> -        dst_ip[i] = rte_be_to_cpu_32(dst_ip[i]);
>>>>>>> +        if (!(pkts[i]->ol_flags & PKT_TX_SEC_OFFLOAD)) {
>>>>>>> +            /* Security offload not enabled. So an LPM lookup is
>>>>>>> +             * required to get the hop
>>>>>>> +             */
>>>>>>> +            offset = offsetof(struct ip, ip_dst);
>>>>>>> +            dst_ip[lpm_pkts] = *rte_pktmbuf_mtod_offset(pkts[i],
>>>>>>> +                    uint32_t *, offset);
>>>>>>> +            dst_ip[lpm_pkts] = rte_be_to_cpu_32(dst_ip[lpm_pkts]);
>>>>>>> +            lpm_pkts++;
>>>>>>> +        }
>>>>>>>       }
>>>>>>>   -    rte_lpm_lookup_bulk((struct rte_lpm *)rt_ctx, dst_ip, 
>>>>>>> hop, nb_pkts);
>>>>>>> +    rte_lpm_lookup_bulk((struct rte_lpm *)rt_ctx, dst_ip, hop, 
>>>>>>> lpm_pkts);
>>>>>>> +
>>>>>>> +    lpm_pkts = 0;
>>>>>>>         for (i = 0; i < nb_pkts; i++) {
>>>>>>> -        if ((hop[i] & RTE_LPM_LOOKUP_SUCCESS) == 0) {
>>>>>>> +        if (pkts[i]->ol_flags & PKT_TX_SEC_OFFLOAD) {
>>>>>>> +            /* Read hop from the SA */
>>>>>>> +            pkt_hop = get_hop_for_offload_pkt(pkts[i]);
>>>>>>> +        } else {
>>>>>>> +            /* Need to use hop returned by lookup */
>>>>>>> +            pkt_hop = hop[lpm_pkts++];
>>>>>>> +            if ((pkt_hop & RTE_LPM_LOOKUP_SUCCESS) == 0)
>>>>>>> +                pkt_hop = -1;
>>>>>>> +        }
>>>>>>> +
>>>>>> I believe the following check is redundant for non inline case. I 
>>>>>> believe get_hop_for_offload_pkt can also set the 
>>>>>> RTE_LPM_LOOKUP_SUCCESS if route is success and take the (pkt_hop 
>>>>>> & RTE_LPM_LOOKUP_SUCCESS) == 0 check outside the if else block 
>>>>>> and free the packet if it is unsuccessful.
>>>>>>
>>>>>> Same comment for route6_pkts. Checking with -1 may not be a good 
>>>>>> idea if we have a flag available for the same.
>>>>>> Others can comment.
>>>>> The problem is ipv4 & ipv6 LPM lookups return different error 
>>>>> values, but we are using a single routine to get the hop for 
>>>>> offload packets. The flag(RTE_LPM_LOOKUP_SUCCESS) is only for ipv4 
>>>>> lookups. For ipv6, error is -1. If we need a cleaner solution, we 
>>>>> can have ipv4 & ipv6 variants of "get_hop_for_offload_pkt". But 
>>>>> that would be repetition of some code.
>>>>
>>>> my concern over this patch is that there is an addition of an extra 
>>>> check in the non inline case and we can get rid of that with some 
>>>> changes in the code(lib/app). Regarding route6_pkts, the code looks 
>>>> cleaner than route4_pkts
>>> If we have ipv4 and ipv6 variants of the 
>>> "get_hop_for_offload_packet" function, the code would look much 
>>> cleaner. Shall I update the patch with such a change and send v4?
>>
>> I believe we shall get rid of "RTE_LPM_LOOKUP_SUCCESS" from the 
>> rte_lpm_lookup_bulk(), we shall have similar error flags for v4 and 
>> v6 APIs. Either we can have RTE_LPM_LOOKUP_SUCCESS or -1 as check for 
>> errors.
> This will call for an ABI change. And LPM library has multiple 
> variants for v4 & v6 lookups. We will need to modify all such 
> instances. I've CCed Bruce for his opinion on this matter. If 
> maintainers can decide on how to address this properly, I can plan my 
> next steps accordingly.
Maybe this alternative approach will help: change the 
get_hop_for_offload_packet to return -1 for v6 and clear 
RTE_LPM_LOOKUP_SUCCESS flag for v4 errors. This will be on the error 
path so the extra code to check the pkt type will have no performance 
impact, and the route function can be cleaner and we can lose the extra 
if in the v4 one.
>> Sergio can comment on this.
>>
>> Duplicating code for get_hop_for_offload_packet may not be a good idea.
>>
>> -Akhil
>>
>

Re: [dpdk-dev] [PATCH v3] examples/ipsec-secgw: fix usage of incorrect port

[Anoob Joseph]

Hi,


On 12/11/2017 03:56 PM, Radu Nicolau wrote:
> Hi,
>
>
> On 12/6/2017 11:08 AM, Anoob wrote:
>> Hi Akhil,
>>
>> On 12/04/2017 01:19 PM, Akhil Goyal wrote:
>>> Hi Anoob,
>>> On 11/29/2017 9:51 AM, Anoob Joseph wrote:
>>>> Hi Akhil,
>>>>
>>>>
>>>> On 24-11-2017 16:19, Akhil Goyal wrote:
>>>>> Hi Anoob,
>>>>>
>>>>> On 11/24/2017 3:28 PM, Anoob wrote:
>>>>>>>>   static inline void
>>>>>>>>   route4_pkts(struct rt_ctx *rt_ctx, struct rte_mbuf *pkts[], 
>>>>>>>> uint8_t nb_pkts)
>>>>>>>>   {
>>>>>>>>       uint32_t hop[MAX_PKT_BURST * 2];
>>>>>>>>       uint32_t dst_ip[MAX_PKT_BURST * 2];
>>>>>>>> +    int32_t pkt_hop = 0;
>>>>>>>>       uint16_t i, offset;
>>>>>>>> +    uint16_t lpm_pkts = 0;
>>>>>>>>         if (nb_pkts == 0)
>>>>>>>>           return;
>>>>>>>>   +    /* Need to do an LPM lookup for non-offload packets. 
>>>>>>>> Offload packets
>>>>>>>> +     * will have port ID in the SA
>>>>>>>> +     */
>>>>>>>> +
>>>>>>>>       for (i = 0; i < nb_pkts; i++) {
>>>>>>>> -        offset = offsetof(struct ip, ip_dst);
>>>>>>>> -        dst_ip[i] = *rte_pktmbuf_mtod_offset(pkts[i],
>>>>>>>> -                uint32_t *, offset);
>>>>>>>> -        dst_ip[i] = rte_be_to_cpu_32(dst_ip[i]);
>>>>>>>> +        if (!(pkts[i]->ol_flags & PKT_TX_SEC_OFFLOAD)) {
>>>>>>>> +            /* Security offload not enabled. So an LPM lookup is
>>>>>>>> +             * required to get the hop
>>>>>>>> +             */
>>>>>>>> +            offset = offsetof(struct ip, ip_dst);
>>>>>>>> +            dst_ip[lpm_pkts] = *rte_pktmbuf_mtod_offset(pkts[i],
>>>>>>>> +                    uint32_t *, offset);
>>>>>>>> +            dst_ip[lpm_pkts] = 
>>>>>>>> rte_be_to_cpu_32(dst_ip[lpm_pkts]);
>>>>>>>> +            lpm_pkts++;
>>>>>>>> +        }
>>>>>>>>       }
>>>>>>>>   -    rte_lpm_lookup_bulk((struct rte_lpm *)rt_ctx, dst_ip, 
>>>>>>>> hop, nb_pkts);
>>>>>>>> +    rte_lpm_lookup_bulk((struct rte_lpm *)rt_ctx, dst_ip, hop, 
>>>>>>>> lpm_pkts);
>>>>>>>> +
>>>>>>>> +    lpm_pkts = 0;
>>>>>>>>         for (i = 0; i < nb_pkts; i++) {
>>>>>>>> -        if ((hop[i] & RTE_LPM_LOOKUP_SUCCESS) == 0) {
>>>>>>>> +        if (pkts[i]->ol_flags & PKT_TX_SEC_OFFLOAD) {
>>>>>>>> +            /* Read hop from the SA */
>>>>>>>> +            pkt_hop = get_hop_for_offload_pkt(pkts[i]);
>>>>>>>> +        } else {
>>>>>>>> +            /* Need to use hop returned by lookup */
>>>>>>>> +            pkt_hop = hop[lpm_pkts++];
>>>>>>>> +            if ((pkt_hop & RTE_LPM_LOOKUP_SUCCESS) == 0)
>>>>>>>> +                pkt_hop = -1;
>>>>>>>> +        }
>>>>>>>> +
>>>>>>> I believe the following check is redundant for non inline case. 
>>>>>>> I believe get_hop_for_offload_pkt can also set the 
>>>>>>> RTE_LPM_LOOKUP_SUCCESS if route is success and take the (pkt_hop 
>>>>>>> & RTE_LPM_LOOKUP_SUCCESS) == 0 check outside the if else block 
>>>>>>> and free the packet if it is unsuccessful.
>>>>>>>
>>>>>>> Same comment for route6_pkts. Checking with -1 may not be a good 
>>>>>>> idea if we have a flag available for the same.
>>>>>>> Others can comment.
>>>>>> The problem is ipv4 & ipv6 LPM lookups return different error 
>>>>>> values, but we are using a single routine to get the hop for 
>>>>>> offload packets. The flag(RTE_LPM_LOOKUP_SUCCESS) is only for 
>>>>>> ipv4 lookups. For ipv6, error is -1. If we need a cleaner 
>>>>>> solution, we can have ipv4 & ipv6 variants of 
>>>>>> "get_hop_for_offload_pkt". But that would be repetition of some 
>>>>>> code.
>>>>>
>>>>> my concern over this patch is that there is an addition of an 
>>>>> extra check in the non inline case and we can get rid of that with 
>>>>> some changes in the code(lib/app). Regarding route6_pkts, the code 
>>>>> looks cleaner than route4_pkts
>>>> If we have ipv4 and ipv6 variants of the 
>>>> "get_hop_for_offload_packet" function, the code would look much 
>>>> cleaner. Shall I update the patch with such a change and send v4?
>>>
>>> I believe we shall get rid of "RTE_LPM_LOOKUP_SUCCESS" from the 
>>> rte_lpm_lookup_bulk(), we shall have similar error flags for v4 and 
>>> v6 APIs. Either we can have RTE_LPM_LOOKUP_SUCCESS or -1 as check 
>>> for errors.
>> This will call for an ABI change. And LPM library has multiple 
>> variants for v4 & v6 lookups. We will need to modify all such 
>> instances. I've CCed Bruce for his opinion on this matter. If 
>> maintainers can decide on how to address this properly, I can plan my 
>> next steps accordingly.
> Maybe this alternative approach will help: change the 
> get_hop_for_offload_packet to return -1 for v6 and clear 
> RTE_LPM_LOOKUP_SUCCESS flag for v4 errors. This will be on the error 
> path so the extra code to check the pkt type will have no performance 
> impact, and the route function can be cleaner and we can lose the 
> extra if in the v4 one.
That should be fine I guess. So the get_hop_for_offload_packet will have 
one more argument to specify whether it is ipv4 or ipv6, right?

I'll revise the patch with this suggestion.
>>> Sergio can comment on this.
>>>
>>> Duplicating code for get_hop_for_offload_packet may not be a good idea.
>>>
>>> -Akhil
>>>
>>
>

[dpdk-dev] [PATCH v4] examples/ipsec-secgw: fix usage of incorrect port

[Anoob Joseph]

When security offload is enabled, the packet should be forwarded on the
port configured in the SA. Security session will be configured on that
port only, and sending the packet on other ports could result in
unencrypted packets being sent out.

This would have performance improvements too, as the per packet LPM
lookup would be avoided for IPsec packets, in inline mode.

Fixes: ec17993a145a ("examples/ipsec-secgw: support security offload")

Signed-off-by: Anoob Joseph <anoob.joseph@caviumnetworks.com>
---
v4
* Made get_hop_for_offload_pkt() be aware of the packet type (ipv4/ipv6) and
  return success/error values accordingly

v3
* Bug fix (fixed a wrong if condition)
* Minor changes in documentation

v2:
* Updated documentation with the change in behavior for outbound inline
  offloaded packets.

 doc/guides/sample_app_ug/ipsec_secgw.rst |  10 ++-
 examples/ipsec-secgw/ipsec-secgw.c       | 101 ++++++++++++++++++++++++++-----
 2 files changed, 96 insertions(+), 15 deletions(-)

diff --git a/doc/guides/sample_app_ug/ipsec_secgw.rst b/doc/guides/sample_app_ug/ipsec_secgw.rst
index d6cfdbf..ae18acd 100644
--- a/doc/guides/sample_app_ug/ipsec_secgw.rst
+++ b/doc/guides/sample_app_ug/ipsec_secgw.rst
@@ -61,6 +61,12 @@ In case of complete protocol offload, the processing of headers(ESP and outer
 IP header) is done by the hardware and the application does not need to
 add/remove them during outbound/inbound processing.
 
+For inline offloaded outbound traffic, the application will not do the LPM
+lookup for routing, as the port on which the packet has to be forwarded will be
+part of the SA. Security parameters will be configured on that port only, and
+sending the packet on other ports could result in unencrypted packets being
+sent out.
+
 The Path for IPsec Inbound traffic is:
 
 *  Read packets from the port.
@@ -543,7 +549,9 @@ where each options means:
  ``<port_id>``
 
  * Port/device ID of the ethernet/crypto accelerator for which the SA is
-   configured. This option is used when *type* is NOT *no-offload*
+   configured. For *inline-crypto-offload* and *inline-protocol-offload*, this
+   port will be used for routing. The routing table will not be referred in
+   this case.
 
  * Optional: No, if *type* is not *no-offload*
 
diff --git a/examples/ipsec-secgw/ipsec-secgw.c b/examples/ipsec-secgw/ipsec-secgw.c
index c98454a..c75dd0d 100644
--- a/examples/ipsec-secgw/ipsec-secgw.c
+++ b/examples/ipsec-secgw/ipsec-secgw.c
@@ -585,31 +585,81 @@ process_pkts_outbound_nosp(struct ipsec_ctx *ipsec_ctx,
 		traffic->ip6.num = nb_pkts_out;
 }
 
+static inline int32_t
+get_hop_for_offload_pkt(struct rte_mbuf *pkt, int is_ipv6)
+{
+	struct ipsec_mbuf_metadata *priv;
+	struct ipsec_sa *sa;
+
+	priv = get_priv(pkt);
+
+	sa = priv->sa;
+	if (unlikely(sa == NULL)) {
+		RTE_LOG(ERR, IPSEC, "SA not saved in private data\n");
+		goto fail;
+	}
+
+	if (is_ipv6)
+		return sa->portid;
+
+	/* else */
+	return (sa->portid | RTE_LPM_LOOKUP_SUCCESS);
+
+fail:
+	if (is_ipv6)
+		return -1;
+
+	/* else */
+	return 0;
+}
+
 static inline void
 route4_pkts(struct rt_ctx *rt_ctx, struct rte_mbuf *pkts[], uint8_t nb_pkts)
 {
 	uint32_t hop[MAX_PKT_BURST * 2];
 	uint32_t dst_ip[MAX_PKT_BURST * 2];
+	int32_t pkt_hop = 0;
 	uint16_t i, offset;
+	uint16_t lpm_pkts = 0;
 
 	if (nb_pkts == 0)
 		return;
 
+	/* Need to do an LPM lookup for non-offload packets. Offload packets
+	 * will have port ID in the SA
+	 */
+
 	for (i = 0; i < nb_pkts; i++) {
-		offset = offsetof(struct ip, ip_dst);
-		dst_ip[i] = *rte_pktmbuf_mtod_offset(pkts[i],
-				uint32_t *, offset);
-		dst_ip[i] = rte_be_to_cpu_32(dst_ip[i]);
+		if (!(pkts[i]->ol_flags & PKT_TX_SEC_OFFLOAD)) {
+			/* Security offload not enabled. So an LPM lookup is
+			 * required to get the hop
+			 */
+			offset = offsetof(struct ip, ip_dst);
+			dst_ip[lpm_pkts] = *rte_pktmbuf_mtod_offset(pkts[i],
+					uint32_t *, offset);
+			dst_ip[lpm_pkts] = rte_be_to_cpu_32(dst_ip[lpm_pkts]);
+			lpm_pkts++;
+		}
 	}
 
-	rte_lpm_lookup_bulk((struct rte_lpm *)rt_ctx, dst_ip, hop, nb_pkts);
+	rte_lpm_lookup_bulk((struct rte_lpm *)rt_ctx, dst_ip, hop, lpm_pkts);
+
+	lpm_pkts = 0;
 
 	for (i = 0; i < nb_pkts; i++) {
-		if ((hop[i] & RTE_LPM_LOOKUP_SUCCESS) == 0) {
+		if (pkts[i]->ol_flags & PKT_TX_SEC_OFFLOAD) {
+			/* Read hop from the SA */
+			pkt_hop = get_hop_for_offload_pkt(pkts[i], 0);
+		} else {
+			/* Need to use hop returned by lookup */
+			pkt_hop = hop[lpm_pkts++];
+		}
+
+		if ((pkt_hop & RTE_LPM_LOOKUP_SUCCESS) == 0) {
 			rte_pktmbuf_free(pkts[i]);
 			continue;
 		}
-		send_single_packet(pkts[i], hop[i] & 0xff);
+		send_single_packet(pkts[i], pkt_hop & 0xff);
 	}
 }
 
@@ -619,26 +669,49 @@ route6_pkts(struct rt_ctx *rt_ctx, struct rte_mbuf *pkts[], uint8_t nb_pkts)
 	int32_t hop[MAX_PKT_BURST * 2];
 	uint8_t dst_ip[MAX_PKT_BURST * 2][16];
 	uint8_t *ip6_dst;
+	int32_t pkt_hop = 0;
 	uint16_t i, offset;
+	uint16_t lpm_pkts = 0;
 
 	if (nb_pkts == 0)
 		return;
 
+	/* Need to do an LPM lookup for non-offload packets. Offload packets
+	 * will have port ID in the SA
+	 */
+
 	for (i = 0; i < nb_pkts; i++) {
-		offset = offsetof(struct ip6_hdr, ip6_dst);
-		ip6_dst = rte_pktmbuf_mtod_offset(pkts[i], uint8_t *, offset);
-		memcpy(&dst_ip[i][0], ip6_dst, 16);
+		if (!(pkts[i]->ol_flags & PKT_TX_SEC_OFFLOAD)) {
+			/* Security offload not enabled. So an LPM lookup is
+			 * required to get the hop
+			 */
+			offset = offsetof(struct ip6_hdr, ip6_dst);
+			ip6_dst = rte_pktmbuf_mtod_offset(pkts[i], uint8_t *,
+					offset);
+			memcpy(&dst_ip[lpm_pkts][0], ip6_dst, 16);
+			lpm_pkts++;
+		}
 	}
 
-	rte_lpm6_lookup_bulk_func((struct rte_lpm6 *)rt_ctx, dst_ip,
-			hop, nb_pkts);
+	rte_lpm6_lookup_bulk_func((struct rte_lpm6 *)rt_ctx, dst_ip, hop,
+			lpm_pkts);
+
+	lpm_pkts = 0;
 
 	for (i = 0; i < nb_pkts; i++) {
-		if (hop[i] == -1) {
+		if (pkts[i]->ol_flags & PKT_TX_SEC_OFFLOAD) {
+			/* Read hop from the SA */
+			pkt_hop = get_hop_for_offload_pkt(pkts[i], 1);
+		} else {
+			/* Need to use hop returned by lookup */
+			pkt_hop = hop[lpm_pkts++];
+		}
+
+		if (pkt_hop == -1) {
 			rte_pktmbuf_free(pkts[i]);
 			continue;
 		}
-		send_single_packet(pkts[i], hop[i] & 0xff);
+		send_single_packet(pkts[i], pkt_hop & 0xff);
 	}
 }
 
-- 
2.7.4

Re: [dpdk-dev] [PATCH v4] examples/ipsec-secgw: fix usage of incorrect port

[Anoob Joseph]

Hi Akhil, Radu,

I've updated the patch with the suggestions. But I think it will be 
better if we have two separate get_hop_for_offload_pkt() functions(for 
ipv4 & ipv6), since the return type etc is different. It might be 
cleaner that way.

Thanks,

Anoob


On 12/11/2017 09:05 PM, Anoob Joseph wrote:
> When security offload is enabled, the packet should be forwarded on the
> port configured in the SA. Security session will be configured on that
> port only, and sending the packet on other ports could result in
> unencrypted packets being sent out.
>
> This would have performance improvements too, as the per packet LPM
> lookup would be avoided for IPsec packets, in inline mode.
>
> Fixes: ec17993a145a ("examples/ipsec-secgw: support security offload")
>
> Signed-off-by: Anoob Joseph <anoob.joseph@caviumnetworks.com>
> ---
> v4
> * Made get_hop_for_offload_pkt() be aware of the packet type (ipv4/ipv6) and
>    return success/error values accordingly
>
> v3
> * Bug fix (fixed a wrong if condition)
> * Minor changes in documentation
>
> v2:
> * Updated documentation with the change in behavior for outbound inline
>    offloaded packets.
>
>   doc/guides/sample_app_ug/ipsec_secgw.rst |  10 ++-
>   examples/ipsec-secgw/ipsec-secgw.c       | 101 ++++++++++++++++++++++++++-----
>   2 files changed, 96 insertions(+), 15 deletions(-)
>
> diff --git a/doc/guides/sample_app_ug/ipsec_secgw.rst b/doc/guides/sample_app_ug/ipsec_secgw.rst
> index d6cfdbf..ae18acd 100644
> --- a/doc/guides/sample_app_ug/ipsec_secgw.rst
> +++ b/doc/guides/sample_app_ug/ipsec_secgw.rst
> @@ -61,6 +61,12 @@ In case of complete protocol offload, the processing of headers(ESP and outer
>   IP header) is done by the hardware and the application does not need to
>   add/remove them during outbound/inbound processing.
>   
> +For inline offloaded outbound traffic, the application will not do the LPM
> +lookup for routing, as the port on which the packet has to be forwarded will be
> +part of the SA. Security parameters will be configured on that port only, and
> +sending the packet on other ports could result in unencrypted packets being
> +sent out.
> +
>   The Path for IPsec Inbound traffic is:
>   
>   *  Read packets from the port.
> @@ -543,7 +549,9 @@ where each options means:
>    ``<port_id>``
>   
>    * Port/device ID of the ethernet/crypto accelerator for which the SA is
> -   configured. This option is used when *type* is NOT *no-offload*
> +   configured. For *inline-crypto-offload* and *inline-protocol-offload*, this
> +   port will be used for routing. The routing table will not be referred in
> +   this case.
>   
>    * Optional: No, if *type* is not *no-offload*
>   
> diff --git a/examples/ipsec-secgw/ipsec-secgw.c b/examples/ipsec-secgw/ipsec-secgw.c
> index c98454a..c75dd0d 100644
> --- a/examples/ipsec-secgw/ipsec-secgw.c
> +++ b/examples/ipsec-secgw/ipsec-secgw.c
> @@ -585,31 +585,81 @@ process_pkts_outbound_nosp(struct ipsec_ctx *ipsec_ctx,
>   		traffic->ip6.num = nb_pkts_out;
>   }
>   
> +static inline int32_t
> +get_hop_for_offload_pkt(struct rte_mbuf *pkt, int is_ipv6)
> +{
> +	struct ipsec_mbuf_metadata *priv;
> +	struct ipsec_sa *sa;
> +
> +	priv = get_priv(pkt);
> +
> +	sa = priv->sa;
> +	if (unlikely(sa == NULL)) {
> +		RTE_LOG(ERR, IPSEC, "SA not saved in private data\n");
> +		goto fail;
> +	}
> +
> +	if (is_ipv6)
> +		return sa->portid;
> +
> +	/* else */
> +	return (sa->portid | RTE_LPM_LOOKUP_SUCCESS);
> +
> +fail:
> +	if (is_ipv6)
> +		return -1;
> +
> +	/* else */
> +	return 0;
> +}
> +
>   static inline void
>   route4_pkts(struct rt_ctx *rt_ctx, struct rte_mbuf *pkts[], uint8_t nb_pkts)
>   {
>   	uint32_t hop[MAX_PKT_BURST * 2];
>   	uint32_t dst_ip[MAX_PKT_BURST * 2];
> +	int32_t pkt_hop = 0;
>   	uint16_t i, offset;
> +	uint16_t lpm_pkts = 0;
>   
>   	if (nb_pkts == 0)
>   		return;
>   
> +	/* Need to do an LPM lookup for non-offload packets. Offload packets
> +	 * will have port ID in the SA
> +	 */
> +
>   	for (i = 0; i < nb_pkts; i++) {
> -		offset = offsetof(struct ip, ip_dst);
> -		dst_ip[i] = *rte_pktmbuf_mtod_offset(pkts[i],
> -				uint32_t *, offset);
> -		dst_ip[i] = rte_be_to_cpu_32(dst_ip[i]);
> +		if (!(pkts[i]->ol_flags & PKT_TX_SEC_OFFLOAD)) {
> +			/* Security offload not enabled. So an LPM lookup is
> +			 * required to get the hop
> +			 */
> +			offset = offsetof(struct ip, ip_dst);
> +			dst_ip[lpm_pkts] = *rte_pktmbuf_mtod_offset(pkts[i],
> +					uint32_t *, offset);
> +			dst_ip[lpm_pkts] = rte_be_to_cpu_32(dst_ip[lpm_pkts]);
> +			lpm_pkts++;
> +		}
>   	}
>   
> -	rte_lpm_lookup_bulk((struct rte_lpm *)rt_ctx, dst_ip, hop, nb_pkts);
> +	rte_lpm_lookup_bulk((struct rte_lpm *)rt_ctx, dst_ip, hop, lpm_pkts);
> +
> +	lpm_pkts = 0;
>   
>   	for (i = 0; i < nb_pkts; i++) {
> -		if ((hop[i] & RTE_LPM_LOOKUP_SUCCESS) == 0) {
> +		if (pkts[i]->ol_flags & PKT_TX_SEC_OFFLOAD) {
> +			/* Read hop from the SA */
> +			pkt_hop = get_hop_for_offload_pkt(pkts[i], 0);
> +		} else {
> +			/* Need to use hop returned by lookup */
> +			pkt_hop = hop[lpm_pkts++];
> +		}
> +
> +		if ((pkt_hop & RTE_LPM_LOOKUP_SUCCESS) == 0) {
>   			rte_pktmbuf_free(pkts[i]);
>   			continue;
>   		}
> -		send_single_packet(pkts[i], hop[i] & 0xff);
> +		send_single_packet(pkts[i], pkt_hop & 0xff);
>   	}
>   }
>   
> @@ -619,26 +669,49 @@ route6_pkts(struct rt_ctx *rt_ctx, struct rte_mbuf *pkts[], uint8_t nb_pkts)
>   	int32_t hop[MAX_PKT_BURST * 2];
>   	uint8_t dst_ip[MAX_PKT_BURST * 2][16];
>   	uint8_t *ip6_dst;
> +	int32_t pkt_hop = 0;
>   	uint16_t i, offset;
> +	uint16_t lpm_pkts = 0;
>   
>   	if (nb_pkts == 0)
>   		return;
>   
> +	/* Need to do an LPM lookup for non-offload packets. Offload packets
> +	 * will have port ID in the SA
> +	 */
> +
>   	for (i = 0; i < nb_pkts; i++) {
> -		offset = offsetof(struct ip6_hdr, ip6_dst);
> -		ip6_dst = rte_pktmbuf_mtod_offset(pkts[i], uint8_t *, offset);
> -		memcpy(&dst_ip[i][0], ip6_dst, 16);
> +		if (!(pkts[i]->ol_flags & PKT_TX_SEC_OFFLOAD)) {
> +			/* Security offload not enabled. So an LPM lookup is
> +			 * required to get the hop
> +			 */
> +			offset = offsetof(struct ip6_hdr, ip6_dst);
> +			ip6_dst = rte_pktmbuf_mtod_offset(pkts[i], uint8_t *,
> +					offset);
> +			memcpy(&dst_ip[lpm_pkts][0], ip6_dst, 16);
> +			lpm_pkts++;
> +		}
>   	}
>   
> -	rte_lpm6_lookup_bulk_func((struct rte_lpm6 *)rt_ctx, dst_ip,
> -			hop, nb_pkts);
> +	rte_lpm6_lookup_bulk_func((struct rte_lpm6 *)rt_ctx, dst_ip, hop,
> +			lpm_pkts);
> +
> +	lpm_pkts = 0;
>   
>   	for (i = 0; i < nb_pkts; i++) {
> -		if (hop[i] == -1) {
> +		if (pkts[i]->ol_flags & PKT_TX_SEC_OFFLOAD) {
> +			/* Read hop from the SA */
> +			pkt_hop = get_hop_for_offload_pkt(pkts[i], 1);
> +		} else {
> +			/* Need to use hop returned by lookup */
> +			pkt_hop = hop[lpm_pkts++];
> +		}
> +
> +		if (pkt_hop == -1) {
>   			rte_pktmbuf_free(pkts[i]);
>   			continue;
>   		}
> -		send_single_packet(pkts[i], hop[i] & 0xff);
> +		send_single_packet(pkts[i], pkt_hop & 0xff);
>   	}
>   }
>   

Re: [dpdk-dev] [PATCH v4] examples/ipsec-secgw: fix usage of incorrect port

[Akhil Goyal]

Hi Anoob,

One minor comment inline. Please correct it.
Apart from that,
Acked-by: Akhil Goyal <akhil.goyal@nxp.com>

Thanks.

On 12/11/2017 9:05 PM, Anoob Joseph wrote:
> When security offload is enabled, the packet should be forwarded on the
> port configured in the SA. Security session will be configured on that
> port only, and sending the packet on other ports could result in
> unencrypted packets being sent out.
> 
> This would have performance improvements too, as the per packet LPM
> lookup would be avoided for IPsec packets, in inline mode.
> 
> Fixes: ec17993a145a ("examples/ipsec-secgw: support security offload")
> 
> Signed-off-by: Anoob Joseph <anoob.joseph@caviumnetworks.com>
> ---
> v4
> * Made get_hop_for_offload_pkt() be aware of the packet type (ipv4/ipv6) and
>    return success/error values accordingly
> 
> v3
> * Bug fix (fixed a wrong if condition)
> * Minor changes in documentation
> 
> v2:
> * Updated documentation with the change in behavior for outbound inline
>    offloaded packets.
> 
>   doc/guides/sample_app_ug/ipsec_secgw.rst |  10 ++-
>   examples/ipsec-secgw/ipsec-secgw.c       | 101 ++++++++++++++++++++++++++-----
>   2 files changed, 96 insertions(+), 15 deletions(-)
> 
> diff --git a/doc/guides/sample_app_ug/ipsec_secgw.rst b/doc/guides/sample_app_ug/ipsec_secgw.rst
> index d6cfdbf..ae18acd 100644
> --- a/doc/guides/sample_app_ug/ipsec_secgw.rst
> +++ b/doc/guides/sample_app_ug/ipsec_secgw.rst
> @@ -61,6 +61,12 @@ In case of complete protocol offload, the processing of headers(ESP and outer
>   IP header) is done by the hardware and the application does not need to
>   add/remove them during outbound/inbound processing.
>   
> +For inline offloaded outbound traffic, the application will not do the LPM
> +lookup for routing, as the port on which the packet has to be forwarded will be
> +part of the SA. Security parameters will be configured on that port only, and
> +sending the packet on other ports could result in unencrypted packets being
> +sent out.
> +
>   The Path for IPsec Inbound traffic is:
>   
>   *  Read packets from the port.
> @@ -543,7 +549,9 @@ where each options means:
>    ``<port_id>``
>   
>    * Port/device ID of the ethernet/crypto accelerator for which the SA is
> -   configured. This option is used when *type* is NOT *no-offload*
> +   configured. For *inline-crypto-offload* and *inline-protocol-offload*, this
> +   port will be used for routing. The routing table will not be referred in
> +   this case.
>   
>    * Optional: No, if *type* is not *no-offload*
>   
> diff --git a/examples/ipsec-secgw/ipsec-secgw.c b/examples/ipsec-secgw/ipsec-secgw.c
> index c98454a..c75dd0d 100644
> --- a/examples/ipsec-secgw/ipsec-secgw.c
> +++ b/examples/ipsec-secgw/ipsec-secgw.c
> @@ -585,31 +585,81 @@ process_pkts_outbound_nosp(struct ipsec_ctx *ipsec_ctx,
>   		traffic->ip6.num = nb_pkts_out;
>   }
>   
> +static inline int32_t
> +get_hop_for_offload_pkt(struct rte_mbuf *pkt, int is_ipv6)
> +{
> +	struct ipsec_mbuf_metadata *priv;
> +	struct ipsec_sa *sa;
> +
> +	priv = get_priv(pkt);
> +
> +	sa = priv->sa;
> +	if (unlikely(sa == NULL)) {
> +		RTE_LOG(ERR, IPSEC, "SA not saved in private data\n");
> +		goto fail;
> +	}
> +
> +	if (is_ipv6)
> +		return sa->portid;
> +
> +	/* else */
> +	return (sa->portid | RTE_LPM_LOOKUP_SUCCESS);
> +
> +fail:
> +	if (is_ipv6)
> +		return -1;
> +
> +	/* else */
> +	return 0;
> +}
> +
>   static inline void
>   route4_pkts(struct rt_ctx *rt_ctx, struct rte_mbuf *pkts[], uint8_t nb_pkts)
>   {
>   	uint32_t hop[MAX_PKT_BURST * 2];
>   	uint32_t dst_ip[MAX_PKT_BURST * 2];
> +	int32_t pkt_hop = 0;
>   	uint16_t i, offset;
> +	uint16_t lpm_pkts = 0;
>   
>   	if (nb_pkts == 0)
>   		return;
>   
> +	/* Need to do an LPM lookup for non-offload packets. Offload packets
> +	 * will have port ID in the SA
the comment should be "Need to do an LPM lookup for non-inline packets. 
Inline packets will have port ID in the SA".
> +	 */
> +
>   	for (i = 0; i < nb_pkts; i++) {
> -		offset = offsetof(struct ip, ip_dst);
> -		dst_ip[i] = *rte_pktmbuf_mtod_offset(pkts[i],
> -				uint32_t *, offset);
> -		dst_ip[i] = rte_be_to_cpu_32(dst_ip[i]);
> +		if (!(pkts[i]->ol_flags & PKT_TX_SEC_OFFLOAD)) {
> +			/* Security offload not enabled. So an LPM lookup is
> +			 * required to get the hop
> +			 */
> +			offset = offsetof(struct ip, ip_dst);
> +			dst_ip[lpm_pkts] = *rte_pktmbuf_mtod_offset(pkts[i],
> +					uint32_t *, offset);
> +			dst_ip[lpm_pkts] = rte_be_to_cpu_32(dst_ip[lpm_pkts]);
> +			lpm_pkts++;
> +		}
>   	}
>   
> -	rte_lpm_lookup_bulk((struct rte_lpm *)rt_ctx, dst_ip, hop, nb_pkts);
> +	rte_lpm_lookup_bulk((struct rte_lpm *)rt_ctx, dst_ip, hop, lpm_pkts);
> +
> +	lpm_pkts = 0;
>   
>   	for (i = 0; i < nb_pkts; i++) {
> -		if ((hop[i] & RTE_LPM_LOOKUP_SUCCESS) == 0) {
> +		if (pkts[i]->ol_flags & PKT_TX_SEC_OFFLOAD) {
> +			/* Read hop from the SA */
> +			pkt_hop = get_hop_for_offload_pkt(pkts[i], 0);
> +		} else {
> +			/* Need to use hop returned by lookup */
> +			pkt_hop = hop[lpm_pkts++];
> +		}
> +
> +		if ((pkt_hop & RTE_LPM_LOOKUP_SUCCESS) == 0) {
>   			rte_pktmbuf_free(pkts[i]);
>   			continue;
>   		}
> -		send_single_packet(pkts[i], hop[i] & 0xff);
> +		send_single_packet(pkts[i], pkt_hop & 0xff);
>   	}
>   }
>   
> @@ -619,26 +669,49 @@ route6_pkts(struct rt_ctx *rt_ctx, struct rte_mbuf *pkts[], uint8_t nb_pkts)
>   	int32_t hop[MAX_PKT_BURST * 2];
>   	uint8_t dst_ip[MAX_PKT_BURST * 2][16];
>   	uint8_t *ip6_dst;
> +	int32_t pkt_hop = 0;
>   	uint16_t i, offset;
> +	uint16_t lpm_pkts = 0;
>   
>   	if (nb_pkts == 0)
>   		return;
>   
> +	/* Need to do an LPM lookup for non-offload packets. Offload packets
> +	 * will have port ID in the SA
Same here. Comments need to be updated.
> +	 */
> +
>   	for (i = 0; i < nb_pkts; i++) {
> -		offset = offsetof(struct ip6_hdr, ip6_dst);
> -		ip6_dst = rte_pktmbuf_mtod_offset(pkts[i], uint8_t *, offset);
> -		memcpy(&dst_ip[i][0], ip6_dst, 16);
> +		if (!(pkts[i]->ol_flags & PKT_TX_SEC_OFFLOAD)) {
> +			/* Security offload not enabled. So an LPM lookup is
> +			 * required to get the hop
> +			 */
> +			offset = offsetof(struct ip6_hdr, ip6_dst);
> +			ip6_dst = rte_pktmbuf_mtod_offset(pkts[i], uint8_t *,
> +					offset);
> +			memcpy(&dst_ip[lpm_pkts][0], ip6_dst, 16);
> +			lpm_pkts++;
> +		}
>   	}
>   
> -	rte_lpm6_lookup_bulk_func((struct rte_lpm6 *)rt_ctx, dst_ip,
> -			hop, nb_pkts);
> +	rte_lpm6_lookup_bulk_func((struct rte_lpm6 *)rt_ctx, dst_ip, hop,
> +			lpm_pkts);
> +
> +	lpm_pkts = 0;
>   
>   	for (i = 0; i < nb_pkts; i++) {
> -		if (hop[i] == -1) {
> +		if (pkts[i]->ol_flags & PKT_TX_SEC_OFFLOAD) {
> +			/* Read hop from the SA */
> +			pkt_hop = get_hop_for_offload_pkt(pkts[i], 1);
> +		} else {
> +			/* Need to use hop returned by lookup */
> +			pkt_hop = hop[lpm_pkts++];
> +		}
> +
> +		if (pkt_hop == -1) {
>   			rte_pktmbuf_free(pkts[i]);
>   			continue;
>   		}
> -		send_single_packet(pkts[i], hop[i] & 0xff);
> +		send_single_packet(pkts[i], pkt_hop & 0xff);
>   	}
>   }
>   
> 

[dpdk-dev] [PATCH v5] examples/ipsec-secgw: fix usage of incorrect port

[Anoob Joseph]

When security offload is enabled, the packet should be forwarded on the
port configured in the SA. Security session will be configured on that
port only, and sending the packet on other ports could result in
unencrypted packets being sent out.

This would have performance improvements too, as the per packet LPM
lookup would be avoided for IPsec packets, in inline mode.

Fixes: ec17993a145a ("examples/ipsec-secgw: support security offload")

Signed-off-by: Anoob Joseph <anoob.joseph@caviumnetworks.com>
Acked-by: Akhil Goyal <akhil.goyal@nxp.com>
---
v5
* Updated comments as suggested by Akhil

v4
* Made get_hop_for_offload_pkt() be aware of the packet type (ipv4/ipv6) and
  return success/error values accordingly

v3
* Bug fix (fixed a wrong if condition)
* Minor changes in documentation

v2:
* Updated documentation with the change in behavior for outbound inline
  offloaded packets.

 doc/guides/sample_app_ug/ipsec_secgw.rst |  10 ++-
 examples/ipsec-secgw/ipsec-secgw.c       | 101 ++++++++++++++++++++++++++-----
 2 files changed, 96 insertions(+), 15 deletions(-)

diff --git a/doc/guides/sample_app_ug/ipsec_secgw.rst b/doc/guides/sample_app_ug/ipsec_secgw.rst
index d6cfdbf..ae18acd 100644
--- a/doc/guides/sample_app_ug/ipsec_secgw.rst
+++ b/doc/guides/sample_app_ug/ipsec_secgw.rst
@@ -61,6 +61,12 @@ In case of complete protocol offload, the processing of headers(ESP and outer
 IP header) is done by the hardware and the application does not need to
 add/remove them during outbound/inbound processing.
 
+For inline offloaded outbound traffic, the application will not do the LPM
+lookup for routing, as the port on which the packet has to be forwarded will be
+part of the SA. Security parameters will be configured on that port only, and
+sending the packet on other ports could result in unencrypted packets being
+sent out.
+
 The Path for IPsec Inbound traffic is:
 
 *  Read packets from the port.
@@ -543,7 +549,9 @@ where each options means:
  ``<port_id>``
 
  * Port/device ID of the ethernet/crypto accelerator for which the SA is
-   configured. This option is used when *type* is NOT *no-offload*
+   configured. For *inline-crypto-offload* and *inline-protocol-offload*, this
+   port will be used for routing. The routing table will not be referred in
+   this case.
 
  * Optional: No, if *type* is not *no-offload*
 
diff --git a/examples/ipsec-secgw/ipsec-secgw.c b/examples/ipsec-secgw/ipsec-secgw.c
index c98454a..2a406ab 100644
--- a/examples/ipsec-secgw/ipsec-secgw.c
+++ b/examples/ipsec-secgw/ipsec-secgw.c
@@ -585,31 +585,81 @@ process_pkts_outbound_nosp(struct ipsec_ctx *ipsec_ctx,
 		traffic->ip6.num = nb_pkts_out;
 }
 
+static inline int32_t
+get_hop_for_offload_pkt(struct rte_mbuf *pkt, int is_ipv6)
+{
+	struct ipsec_mbuf_metadata *priv;
+	struct ipsec_sa *sa;
+
+	priv = get_priv(pkt);
+
+	sa = priv->sa;
+	if (unlikely(sa == NULL)) {
+		RTE_LOG(ERR, IPSEC, "SA not saved in private data\n");
+		goto fail;
+	}
+
+	if (is_ipv6)
+		return sa->portid;
+
+	/* else */
+	return (sa->portid | RTE_LPM_LOOKUP_SUCCESS);
+
+fail:
+	if (is_ipv6)
+		return -1;
+
+	/* else */
+	return 0;
+}
+
 static inline void
 route4_pkts(struct rt_ctx *rt_ctx, struct rte_mbuf *pkts[], uint8_t nb_pkts)
 {
 	uint32_t hop[MAX_PKT_BURST * 2];
 	uint32_t dst_ip[MAX_PKT_BURST * 2];
+	int32_t pkt_hop = 0;
 	uint16_t i, offset;
+	uint16_t lpm_pkts = 0;
 
 	if (nb_pkts == 0)
 		return;
 
+	/* Need to do an LPM lookup for non-inline packets. Inline packets will
+	 * have port ID in the SA
+	 */
+
 	for (i = 0; i < nb_pkts; i++) {
-		offset = offsetof(struct ip, ip_dst);
-		dst_ip[i] = *rte_pktmbuf_mtod_offset(pkts[i],
-				uint32_t *, offset);
-		dst_ip[i] = rte_be_to_cpu_32(dst_ip[i]);
+		if (!(pkts[i]->ol_flags & PKT_TX_SEC_OFFLOAD)) {
+			/* Security offload not enabled. So an LPM lookup is
+			 * required to get the hop
+			 */
+			offset = offsetof(struct ip, ip_dst);
+			dst_ip[lpm_pkts] = *rte_pktmbuf_mtod_offset(pkts[i],
+					uint32_t *, offset);
+			dst_ip[lpm_pkts] = rte_be_to_cpu_32(dst_ip[lpm_pkts]);
+			lpm_pkts++;
+		}
 	}
 
-	rte_lpm_lookup_bulk((struct rte_lpm *)rt_ctx, dst_ip, hop, nb_pkts);
+	rte_lpm_lookup_bulk((struct rte_lpm *)rt_ctx, dst_ip, hop, lpm_pkts);
+
+	lpm_pkts = 0;
 
 	for (i = 0; i < nb_pkts; i++) {
-		if ((hop[i] & RTE_LPM_LOOKUP_SUCCESS) == 0) {
+		if (pkts[i]->ol_flags & PKT_TX_SEC_OFFLOAD) {
+			/* Read hop from the SA */
+			pkt_hop = get_hop_for_offload_pkt(pkts[i], 0);
+		} else {
+			/* Need to use hop returned by lookup */
+			pkt_hop = hop[lpm_pkts++];
+		}
+
+		if ((pkt_hop & RTE_LPM_LOOKUP_SUCCESS) == 0) {
 			rte_pktmbuf_free(pkts[i]);
 			continue;
 		}
-		send_single_packet(pkts[i], hop[i] & 0xff);
+		send_single_packet(pkts[i], pkt_hop & 0xff);
 	}
 }
 
@@ -619,26 +669,49 @@ route6_pkts(struct rt_ctx *rt_ctx, struct rte_mbuf *pkts[], uint8_t nb_pkts)
 	int32_t hop[MAX_PKT_BURST * 2];
 	uint8_t dst_ip[MAX_PKT_BURST * 2][16];
 	uint8_t *ip6_dst;
+	int32_t pkt_hop = 0;
 	uint16_t i, offset;
+	uint16_t lpm_pkts = 0;
 
 	if (nb_pkts == 0)
 		return;
 
+	/* Need to do an LPM lookup for non-inline packets. Inline packets will
+	 * have port ID in the SA
+	 */
+
 	for (i = 0; i < nb_pkts; i++) {
-		offset = offsetof(struct ip6_hdr, ip6_dst);
-		ip6_dst = rte_pktmbuf_mtod_offset(pkts[i], uint8_t *, offset);
-		memcpy(&dst_ip[i][0], ip6_dst, 16);
+		if (!(pkts[i]->ol_flags & PKT_TX_SEC_OFFLOAD)) {
+			/* Security offload not enabled. So an LPM lookup is
+			 * required to get the hop
+			 */
+			offset = offsetof(struct ip6_hdr, ip6_dst);
+			ip6_dst = rte_pktmbuf_mtod_offset(pkts[i], uint8_t *,
+					offset);
+			memcpy(&dst_ip[lpm_pkts][0], ip6_dst, 16);
+			lpm_pkts++;
+		}
 	}
 
-	rte_lpm6_lookup_bulk_func((struct rte_lpm6 *)rt_ctx, dst_ip,
-			hop, nb_pkts);
+	rte_lpm6_lookup_bulk_func((struct rte_lpm6 *)rt_ctx, dst_ip, hop,
+			lpm_pkts);
+
+	lpm_pkts = 0;
 
 	for (i = 0; i < nb_pkts; i++) {
-		if (hop[i] == -1) {
+		if (pkts[i]->ol_flags & PKT_TX_SEC_OFFLOAD) {
+			/* Read hop from the SA */
+			pkt_hop = get_hop_for_offload_pkt(pkts[i], 1);
+		} else {
+			/* Need to use hop returned by lookup */
+			pkt_hop = hop[lpm_pkts++];
+		}
+
+		if (pkt_hop == -1) {
 			rte_pktmbuf_free(pkts[i]);
 			continue;
 		}
-		send_single_packet(pkts[i], hop[i] & 0xff);
+		send_single_packet(pkts[i], pkt_hop & 0xff);
 	}
 }
 
-- 
2.7.4

Re: [dpdk-dev] [PATCH v5] examples/ipsec-secgw: fix usage of incorrect port

[Radu Nicolau]

On 12/12/2017 8:32 AM, Anoob Joseph wrote:
> When security offload is enabled, the packet should be forwarded on the
> port configured in the SA. Security session will be configured on that
> port only, and sending the packet on other ports could result in
> unencrypted packets being sent out.
>
> This would have performance improvements too, as the per packet LPM
> lookup would be avoided for IPsec packets, in inline mode.
>
> Fixes: ec17993a145a ("examples/ipsec-secgw: support security offload")
>
> Signed-off-by: Anoob Joseph <anoob.joseph@caviumnetworks.com>
> Acked-by: Akhil Goyal <akhil.goyal@nxp.com>
Acked-by: Radu Nicolau <radu.nicolau@intel.com>

Re: [dpdk-dev] [PATCH v5] examples/ipsec-secgw: fix usage of incorrect port

[De Lara Guarch, Pablo]

> -----Original Message-----
> From: dev [mailto:dev-bounces@dpdk.org] On Behalf Of Radu Nicolau
> Sent: Tuesday, December 12, 2017 11:27 AM
> To: Anoob Joseph <anoob.joseph@caviumnetworks.com>; Akhil Goyal
> <akhil.goyal@nxp.com>; Doherty, Declan <declan.doherty@intel.com>;
> Gonzalez Monroy, Sergio <sergio.gonzalez.monroy@intel.com>
> Cc: Narayana Prasad <narayanaprasad.athreya@caviumnetworks.com>;
> Jerin Jacob <jerin.jacob@caviumnetworks.com>; dev@dpdk.org
> Subject: Re: [dpdk-dev] [PATCH v5] examples/ipsec-secgw: fix usage of
> incorrect port
> 
> 
> 
> On 12/12/2017 8:32 AM, Anoob Joseph wrote:
> > When security offload is enabled, the packet should be forwarded on
> > the port configured in the SA. Security session will be configured on
> > that port only, and sending the packet on other ports could result in
> > unencrypted packets being sent out.
> >
> > This would have performance improvements too, as the per packet LPM
> > lookup would be avoided for IPsec packets, in inline mode.
> >
> > Fixes: ec17993a145a ("examples/ipsec-secgw: support security offload")
> >
> > Signed-off-by: Anoob Joseph <anoob.joseph@caviumnetworks.com>
> > Acked-by: Akhil Goyal <akhil.goyal@nxp.com>
> Acked-by: Radu Nicolau <radu.nicolau@intel.com>

Applied to dpdk-next-crypto.
Thanks,

Pablo