From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mail-wm0-f51.google.com (mail-wm0-f51.google.com [74.125.82.51]) by dpdk.org (Postfix) with ESMTP id 9181C1AEE8 for ; Sun, 15 Oct 2017 14:51:41 +0200 (CEST) Received: by mail-wm0-f51.google.com with SMTP id i124so29127661wmf.3 for ; Sun, 15 Oct 2017 05:51:41 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=dev-mellanox-co-il.20150623.gappssmtp.com; s=20150623; h=subject:to:references:from:message-id:date:user-agent:mime-version :in-reply-to:content-transfer-encoding:content-language; bh=cW30XExWgaDg2zkW+3XUX6s1WbjHnMV/KHl32+ofHbA=; b=05somonLHsdAYQNNNYS4MVo3s2i17aj+7nIY/EVxD4xaBGloTwoSnogpEV3IUHKMhi SFh+Qs+kFXlfFJJOma84NfH5EOWcBOqkMp/vb97sjclYGYuCAyiTIORrQ4iXUntffB1Z IpWvMkDgYhkF497cTE81qBX1eFZZOO7Q/JABAtpE7Ybnik5fRAuUfAXv+DJk1AgDqjoV bU+6EBVikprd81Cv1Kl9rH4JWAEuEIwD/+eahKwiWiKpOeiZCicEy8x1Gwdq/JSRRZ8S C/N2yqWw/MPRL+dPeylX9BKWLDsLX1o8rD0nKh5QFCT+zp3vBDfnbTkPl2aASlzgxEyc LO/w== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:subject:to:references:from:message-id:date :user-agent:mime-version:in-reply-to:content-transfer-encoding :content-language; bh=cW30XExWgaDg2zkW+3XUX6s1WbjHnMV/KHl32+ofHbA=; b=PQa5pr+OvjgSPWGmhvNjIXgjpsIzpA2QeUo1y7IdQ/VTJLwYElnIDkoan0eFVZD2LV YiabqzJxCWfonhLwh0JIVXD/eGWaff1a9ERZSt2MUm2MjgjrKNSto1yYhy68oNcEALeE qbOno3PZCjir71ugNQZOppHrUWk7IHGMHoV8+BxGr18wRpHwu5TyDSd53He9Z1C4jWsd ID7xQYWmTUAX67jv4aCfumxMaHXwKQCfQ8WRDEl6LX7bi3WuBiUVs/Iq4ql4tOXoYIIY oO714ON/mTqb7BLN7yN1E0UE87oKq7q/mDisxQq8Tz0N7Azy11gVnrtS30VpM1WNgfBX qG3A== X-Gm-Message-State: AMCzsaUx5211QA9uoJzPtovDZ5rXjTp5kunmvoCTLjzY/yn43QhONVF4 PNLPa9btStGDqlpf+bJSFo5L2Fu5QDM= X-Google-Smtp-Source: ABhQp+S7pk8ZUWlpvNCjRpkKDTKlEF6eNCgLN1YvWQT/nSPdFkFHkg8VE75LyNFHxykso1ci2I98og== X-Received: by 10.28.111.203 with SMTP id c72mr5910082wmi.42.1508071900662; Sun, 15 Oct 2017 05:51:40 -0700 (PDT) Received: from [10.0.38.219] ([193.47.165.251]) by smtp.gmail.com with ESMTPSA id 10sm8749062wmy.35.2017.10.15.05.51.39 for (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Sun, 15 Oct 2017 05:51:39 -0700 (PDT) To: dev@dpdk.org References: <20171006181151.4758-1-akhil.goyal@nxp.com> <20171014221734.15511-1-akhil.goyal@nxp.com> <20171014221734.15511-13-akhil.goyal@nxp.com> From: Aviad Yehezkel Message-ID: Date: Sun, 15 Oct 2017 15:51:38 +0300 User-Agent: Mozilla/5.0 (Windows NT 10.0; WOW64; rv:52.0) Gecko/20100101 Thunderbird/52.3.0 MIME-Version: 1.0 In-Reply-To: <20171014221734.15511-13-akhil.goyal@nxp.com> Content-Type: text/plain; charset=utf-8; format=flowed Content-Transfer-Encoding: 7bit Content-Language: en-US Subject: Re: [dpdk-dev] [PATCH v4 12/12] examples/ipsec-secgw: add support for security offload X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 15 Oct 2017 12:51:41 -0000 On 10/15/2017 1:17 AM, Akhil Goyal wrote: > Ipsec-secgw application is modified so that it can support > following type of actions for crypto operations > 1. full protocol offload using crypto devices. > 2. inline ipsec using ethernet devices to perform crypto operations > 3. full protocol offload using ethernet devices. > 4. non protocol offload > > Signed-off-by: Akhil Goyal > Signed-off-by: Radu Nicolau > Signed-off-by: Boris Pismenny > Signed-off-by: Declan Doherty > Signed-off-by: Aviad Yehezkel > --- > doc/guides/sample_app_ug/ipsec_secgw.rst | 52 +++++- > examples/ipsec-secgw/esp.c | 120 ++++++++---- > examples/ipsec-secgw/esp.h | 10 - > examples/ipsec-secgw/ipsec-secgw.c | 5 + > examples/ipsec-secgw/ipsec.c | 308 ++++++++++++++++++++++++++----- > examples/ipsec-secgw/ipsec.h | 32 +++- > examples/ipsec-secgw/sa.c | 151 +++++++++++---- > 7 files changed, 545 insertions(+), 133 deletions(-) > > diff --git a/doc/guides/sample_app_ug/ipsec_secgw.rst b/doc/guides/sample_app_ug/ipsec_secgw.rst > index b675cba..892977e 100644 > --- a/doc/guides/sample_app_ug/ipsec_secgw.rst > +++ b/doc/guides/sample_app_ug/ipsec_secgw.rst > @@ -52,13 +52,22 @@ The application classifies the ports as *Protected* and *Unprotected*. > Thus, traffic received on an Unprotected or Protected port is consider > Inbound or Outbound respectively. > > +The application also supports complete IPSec protocol offload to hardware > +(Look aside crypto accelarator or using ethernet device). It also support > +inline ipsec processing by the supported ethernet device during transmission. > +These modes can be selected during the SA creation configuration. > + > +In case of complete protocol offload, the processing of headers(ESP and outer > +IP header) is done by the hardware and the application does not need to > +add/remove them during outbound/inbound processing. > + > The Path for IPsec Inbound traffic is: > > * Read packets from the port. > * Classify packets between IPv4 and ESP. > * Perform Inbound SA lookup for ESP packets based on their SPI. > -* Perform Verification/Decryption. > -* Remove ESP and outer IP header > +* Perform Verification/Decryption (Not needed in case of inline ipsec). > +* Remove ESP and outer IP header (Not needed in case of protocol offload). > * Inbound SP check using ACL of decrypted packets and any other IPv4 packets. > * Routing. > * Write packet to port. > @@ -68,8 +77,8 @@ The Path for the IPsec Outbound traffic is: > * Read packets from the port. > * Perform Outbound SP check using ACL of all IPv4 traffic. > * Perform Outbound SA lookup for packets that need IPsec protection. > -* Add ESP and outer IP header. > -* Perform Encryption/Digest. > +* Add ESP and outer IP header (Not needed in case protocol offload). > +* Perform Encryption/Digest (Not needed in case of inline ipsec). > * Routing. > * Write packet to port. > > @@ -385,7 +394,7 @@ The SA rule syntax is shown as follows: > .. code-block:: console > > sa > - > + > > where each options means: > > @@ -526,6 +535,34 @@ where each options means: > * *dst X.X.X.X* for IPv4 > * *dst XXXX:XXXX:XXXX:XXXX:XXXX:XXXX:XXXX:XXXX* for IPv6 > > +```` > + > + * Action type to specify the security action. This option specify > + the SA to be performed with look aside protocol offload to HW > + accelerator or protocol offload on ethernet device or inline > + crypto processing on the ethernet device during transmission. > + > + * Optional: Yes, default type *no-offload* > + > + * Available options: > + > + * *lookaside-protocol-offload*: look aside protocol offload to HW accelerator > + * *inline-protocol-offload*: inline protocol offload on ethernet device > + * *inline-crypto-offload*: inline crypto processing on ethernet device > + * *no-offload*: no offloading to hardware > + > + ```` > + > + * Port/device ID of the ethernet/crypto accelerator for which the SA is > + configured. This option is used when *type* is NOT *no-offload* > + > + * Optional: No, if *type* is not *no-offload* > + > + * Syntax: > + > + * *port_id X* X is a valid device number in decimal > + > + > Example SA rules: > > .. code-block:: console > @@ -545,6 +582,11 @@ Example SA rules: > aead_key de:ad:be:ef:de:ad:be:ef:de:ad:be:ef:de:ad:be:ef:de:ad:be:ef \ > mode ipv4-tunnel src 172.16.2.5 dst 172.16.1.5 > > + sa out 5 cipher_algo aes-128-cbc cipher_key 0:0:0:0:0:0:0:0:0:0:0:0:0:0:0:0 \ > + auth_algo sha1-hmac auth_key 0:0:0:0:0:0:0:0:0:0:0:0:0:0:0:0:0:0:0:0 \ > + mode ipv4-tunnel src 172.16.1.5 dst 172.16.2.5 \ > + type lookaside-protocol-offload port_id 4 > + > Routing rule syntax > ^^^^^^^^^^^^^^^^^^^ > > diff --git a/examples/ipsec-secgw/esp.c b/examples/ipsec-secgw/esp.c > index 12c6f8c..781b162 100644 > --- a/examples/ipsec-secgw/esp.c > +++ b/examples/ipsec-secgw/esp.c > @@ -58,8 +58,11 @@ esp_inbound(struct rte_mbuf *m, struct ipsec_sa *sa, > struct rte_crypto_sym_op *sym_cop; > int32_t payload_len, ip_hdr_len; > > - RTE_ASSERT(m != NULL); > RTE_ASSERT(sa != NULL); > + if (sa->type == RTE_SECURITY_ACTION_TYPE_INLINE_CRYPTO) > + return 0; > + > + RTE_ASSERT(m != NULL); > RTE_ASSERT(cop != NULL); > > ip4 = rte_pktmbuf_mtod(m, struct ip *); > @@ -175,29 +178,44 @@ esp_inbound_post(struct rte_mbuf *m, struct ipsec_sa *sa, > RTE_ASSERT(sa != NULL); > RTE_ASSERT(cop != NULL); > > + if (sa->type == RTE_SECURITY_ACTION_TYPE_INLINE_CRYPTO) { > + if (m->ol_flags & PKT_RX_SEC_OFFLOAD) { > + if (m->ol_flags & PKT_RX_SEC_OFFLOAD_FAILED) > + cop->status = RTE_CRYPTO_OP_STATUS_ERROR; > + else > + cop->status = RTE_CRYPTO_OP_STATUS_SUCCESS; > + } else > + cop->status = RTE_CRYPTO_OP_STATUS_NOT_PROCESSED; > + } > + > if (cop->status != RTE_CRYPTO_OP_STATUS_SUCCESS) { > RTE_LOG(ERR, IPSEC_ESP, "failed crypto op\n"); > return -1; > } > > - nexthdr = rte_pktmbuf_mtod_offset(m, uint8_t*, > - rte_pktmbuf_pkt_len(m) - sa->digest_len - 1); > - pad_len = nexthdr - 1; > + if (sa->type == RTE_SECURITY_ACTION_TYPE_INLINE_CRYPTO && > + sa->ol_flags & RTE_SECURITY_RX_HW_TRAILER_OFFLOAD) { > + nexthdr = &m->inner_esp_next_proto; > + } else { > + nexthdr = rte_pktmbuf_mtod_offset(m, uint8_t*, > + rte_pktmbuf_pkt_len(m) - sa->digest_len - 1); > + pad_len = nexthdr - 1; > + > + padding = pad_len - *pad_len; > + for (i = 0; i < *pad_len; i++) { > + if (padding[i] != i + 1) { > + RTE_LOG(ERR, IPSEC_ESP, "invalid padding\n"); > + return -EINVAL; > + } > + } > > - padding = pad_len - *pad_len; > - for (i = 0; i < *pad_len; i++) { > - if (padding[i] != i + 1) { > - RTE_LOG(ERR, IPSEC_ESP, "invalid padding\n"); > + if (rte_pktmbuf_trim(m, *pad_len + 2 + sa->digest_len)) { > + RTE_LOG(ERR, IPSEC_ESP, > + "failed to remove pad_len + digest\n"); > return -EINVAL; > } > } > > - if (rte_pktmbuf_trim(m, *pad_len + 2 + sa->digest_len)) { > - RTE_LOG(ERR, IPSEC_ESP, > - "failed to remove pad_len + digest\n"); > - return -EINVAL; > - } > - > if (unlikely(sa->flags == TRANSPORT)) { > ip = rte_pktmbuf_mtod(m, struct ip *); > ip4 = (struct ip *)rte_pktmbuf_adj(m, > @@ -226,7 +244,7 @@ esp_outbound(struct rte_mbuf *m, struct ipsec_sa *sa, > struct ip *ip4; > struct ip6_hdr *ip6; > struct esp_hdr *esp = NULL; > - uint8_t *padding, *new_ip, nlp; > + uint8_t *padding = NULL, *new_ip, nlp; > struct rte_crypto_sym_op *sym_cop; > int32_t i; > uint16_t pad_payload_len, pad_len = 0; > @@ -236,7 +254,6 @@ esp_outbound(struct rte_mbuf *m, struct ipsec_sa *sa, > RTE_ASSERT(sa != NULL); > RTE_ASSERT(sa->flags == IP4_TUNNEL || sa->flags == IP6_TUNNEL || > sa->flags == TRANSPORT); > - RTE_ASSERT(cop != NULL); > > ip4 = rte_pktmbuf_mtod(m, struct ip *); > if (likely(ip4->ip_v == IPVERSION)) { > @@ -290,12 +307,19 @@ esp_outbound(struct rte_mbuf *m, struct ipsec_sa *sa, > return -EINVAL; > } > > - padding = (uint8_t *)rte_pktmbuf_append(m, pad_len + sa->digest_len); > - if (unlikely(padding == NULL)) { > - RTE_LOG(ERR, IPSEC_ESP, "not enough mbuf trailing space\n"); > - return -ENOSPC; > + /* Add trailer padding if it is not constructed by HW */ > + if (sa->type != RTE_SECURITY_ACTION_TYPE_INLINE_CRYPTO || > + (sa->type == RTE_SECURITY_ACTION_TYPE_INLINE_CRYPTO && > + !(sa->ol_flags & RTE_SECURITY_TX_HW_TRAILER_OFFLOAD))) { > + padding = (uint8_t *)rte_pktmbuf_append(m, pad_len + > + sa->digest_len); > + if (unlikely(padding == NULL)) { > + RTE_LOG(ERR, IPSEC_ESP, > + "not enough mbuf trailing space\n"); > + return -ENOSPC; > + } > + rte_prefetch0(padding); > } > - rte_prefetch0(padding); > > switch (sa->flags) { > case IP4_TUNNEL: > @@ -328,15 +352,46 @@ esp_outbound(struct rte_mbuf *m, struct ipsec_sa *sa, > esp->spi = rte_cpu_to_be_32(sa->spi); > esp->seq = rte_cpu_to_be_32((uint32_t)sa->seq); > > + /* set iv */ > uint64_t *iv = (uint64_t *)(esp + 1); > + if (sa->aead_algo == RTE_CRYPTO_AEAD_AES_GCM) { > + *iv = rte_cpu_to_be_64(sa->seq); > + } else { > + switch (sa->cipher_algo) { > + case RTE_CRYPTO_CIPHER_NULL: > + case RTE_CRYPTO_CIPHER_AES_CBC: > + memset(iv, 0, sa->iv_len); > + break; > + case RTE_CRYPTO_CIPHER_AES_CTR: > + *iv = rte_cpu_to_be_64(sa->seq); > + break; > + default: > + RTE_LOG(ERR, IPSEC_ESP, > + "unsupported cipher algorithm %u\n", > + sa->cipher_algo); > + return -EINVAL; > + } > + } > + > + if (sa->type == RTE_SECURITY_ACTION_TYPE_INLINE_CRYPTO) { > + if (sa->ol_flags & RTE_SECURITY_TX_HW_TRAILER_OFFLOAD) { > + /* Set the inner esp next protocol for HW trailer */ > + m->inner_esp_next_proto = nlp; > + m->packet_type |= RTE_PTYPE_TUNNEL_ESP; > + } else { > + padding[pad_len - 2] = pad_len - 2; > + padding[pad_len - 1] = nlp; > + } > + goto done; > + } > > + RTE_ASSERT(cop != NULL); > sym_cop = get_sym_cop(cop); > sym_cop->m_src = m; > > if (sa->aead_algo == RTE_CRYPTO_AEAD_AES_GCM) { > uint8_t *aad; > > - *iv = rte_cpu_to_be_64(sa->seq); > sym_cop->aead.data.offset = ip_hdr_len + > sizeof(struct esp_hdr) + sa->iv_len; > sym_cop->aead.data.length = pad_payload_len; > @@ -365,13 +420,11 @@ esp_outbound(struct rte_mbuf *m, struct ipsec_sa *sa, > switch (sa->cipher_algo) { > case RTE_CRYPTO_CIPHER_NULL: > case RTE_CRYPTO_CIPHER_AES_CBC: > - memset(iv, 0, sa->iv_len); > sym_cop->cipher.data.offset = ip_hdr_len + > sizeof(struct esp_hdr); > sym_cop->cipher.data.length = pad_payload_len + sa->iv_len; > break; > case RTE_CRYPTO_CIPHER_AES_CTR: > - *iv = rte_cpu_to_be_64(sa->seq); > sym_cop->cipher.data.offset = ip_hdr_len + > sizeof(struct esp_hdr) + sa->iv_len; > sym_cop->cipher.data.length = pad_payload_len; > @@ -413,21 +466,26 @@ esp_outbound(struct rte_mbuf *m, struct ipsec_sa *sa, > rte_pktmbuf_pkt_len(m) - sa->digest_len); > } > > +done: > return 0; > } > > int > -esp_outbound_post(struct rte_mbuf *m __rte_unused, > - struct ipsec_sa *sa __rte_unused, > - struct rte_crypto_op *cop) > +esp_outbound_post(struct rte_mbuf *m, > + struct ipsec_sa *sa, > + struct rte_crypto_op *cop) > { > RTE_ASSERT(m != NULL); > RTE_ASSERT(sa != NULL); > - RTE_ASSERT(cop != NULL); > > - if (cop->status != RTE_CRYPTO_OP_STATUS_SUCCESS) { > - RTE_LOG(ERR, IPSEC_ESP, "Failed crypto op\n"); > - return -1; > + if (sa->type == RTE_SECURITY_ACTION_TYPE_INLINE_CRYPTO) { > + m->ol_flags |= PKT_TX_SEC_OFFLOAD; > + } else { > + RTE_ASSERT(cop != NULL); > + if (cop->status != RTE_CRYPTO_OP_STATUS_SUCCESS) { > + RTE_LOG(ERR, IPSEC_ESP, "Failed crypto op\n"); > + return -1; > + } > } > > return 0; > diff --git a/examples/ipsec-secgw/esp.h b/examples/ipsec-secgw/esp.h > index fa5cc8a..23601e3 100644 > --- a/examples/ipsec-secgw/esp.h > +++ b/examples/ipsec-secgw/esp.h > @@ -35,16 +35,6 @@ > > struct mbuf; > > -/* RFC4303 */ > -struct esp_hdr { > - uint32_t spi; > - uint32_t seq; > - /* Payload */ > - /* Padding */ > - /* Pad Length */ > - /* Next Header */ > - /* Integrity Check Value - ICV */ > -}; > > int > esp_inbound(struct rte_mbuf *m, struct ipsec_sa *sa, > diff --git a/examples/ipsec-secgw/ipsec-secgw.c b/examples/ipsec-secgw/ipsec-secgw.c > index f931de6..6e18e84 100644 > --- a/examples/ipsec-secgw/ipsec-secgw.c > +++ b/examples/ipsec-secgw/ipsec-secgw.c > @@ -1317,6 +1317,11 @@ port_init(uint16_t portid) > printf("Creating queues: nb_rx_queue=%d nb_tx_queue=%u...\n", > nb_rx_queue, nb_tx_queue); > > + if (dev_info.rx_offload_capa & DEV_RX_OFFLOAD_SECURITY) > + port_conf.rxmode.enable_sec = 1; > + if (dev_info.tx_offload_capa & DEV_TX_OFFLOAD_SECURITY) > + port_conf.txmode.enable_sec = 1; > + > ret = rte_eth_dev_configure(portid, nb_rx_queue, nb_tx_queue, > &port_conf); > if (ret < 0) > diff --git a/examples/ipsec-secgw/ipsec.c b/examples/ipsec-secgw/ipsec.c > index daa1d7b..6423e3e 100644 > --- a/examples/ipsec-secgw/ipsec.c > +++ b/examples/ipsec-secgw/ipsec.c > @@ -37,7 +37,9 @@ > #include > #include > #include > +#include > #include > +#include > #include > #include > > @@ -49,7 +51,7 @@ create_session(struct ipsec_ctx *ipsec_ctx, struct ipsec_sa *sa) > { > struct rte_cryptodev_info cdev_info; > unsigned long cdev_id_qp = 0; > - int32_t ret; > + int32_t ret = 0; > struct cdev_key key = { 0 }; > > key.lcore_id = (uint8_t)rte_lcore_id(); > @@ -58,16 +60,19 @@ create_session(struct ipsec_ctx *ipsec_ctx, struct ipsec_sa *sa) > key.auth_algo = (uint8_t)sa->auth_algo; > key.aead_algo = (uint8_t)sa->aead_algo; > > - ret = rte_hash_lookup_data(ipsec_ctx->cdev_map, &key, > - (void **)&cdev_id_qp); > - if (ret < 0) { > - RTE_LOG(ERR, IPSEC, "No cryptodev: core %u, cipher_algo %u, " > - "auth_algo %u aead_algo %u\n", > - key.lcore_id, > - key.cipher_algo, > - key.auth_algo, > - key.aead_algo); > - return -1; > + if (sa->type == RTE_SECURITY_ACTION_TYPE_NONE) { > + ret = rte_hash_lookup_data(ipsec_ctx->cdev_map, &key, > + (void **)&cdev_id_qp); > + if (ret < 0) { > + RTE_LOG(ERR, IPSEC, > + "No cryptodev: core %u, cipher_algo %u, " > + "auth_algo %u aead_algo %u\n", > + key.lcore_id, > + key.cipher_algo, > + key.auth_algo, > + key.aead_algo); > + return -1; > + } > } > > RTE_LOG_DP(DEBUG, IPSEC, "Create session for SA spi %u on cryptodev " > @@ -75,23 +80,153 @@ create_session(struct ipsec_ctx *ipsec_ctx, struct ipsec_sa *sa) > ipsec_ctx->tbl[cdev_id_qp].id, > ipsec_ctx->tbl[cdev_id_qp].qp); > > - sa->crypto_session = rte_cryptodev_sym_session_create( > - ipsec_ctx->session_pool); > - rte_cryptodev_sym_session_init(ipsec_ctx->tbl[cdev_id_qp].id, > - sa->crypto_session, sa->xforms, > - ipsec_ctx->session_pool); > - > - rte_cryptodev_info_get(ipsec_ctx->tbl[cdev_id_qp].id, &cdev_info); > - if (cdev_info.sym.max_nb_sessions_per_qp > 0) { > - ret = rte_cryptodev_queue_pair_attach_sym_session( > - ipsec_ctx->tbl[cdev_id_qp].id, > - ipsec_ctx->tbl[cdev_id_qp].qp, > - sa->crypto_session); > - if (ret < 0) { > - RTE_LOG(ERR, IPSEC, > - "Session cannot be attached to qp %u ", > - ipsec_ctx->tbl[cdev_id_qp].qp); > - return -1; > + if (sa->type != RTE_SECURITY_ACTION_TYPE_NONE) { > + struct rte_security_session_conf sess_conf = { > + .action_type = sa->type, > + .protocol = RTE_SECURITY_PROTOCOL_IPSEC, > + .ipsec = { > + .spi = sa->spi, > + .salt = sa->salt, > + .options = { 0 }, > + .direction = sa->direction, > + .proto = RTE_SECURITY_IPSEC_SA_PROTO_ESP, > + .mode = (sa->flags == IP4_TUNNEL || > + sa->flags == IP6_TUNNEL) ? > + RTE_SECURITY_IPSEC_SA_MODE_TUNNEL : > + RTE_SECURITY_IPSEC_SA_MODE_TRANSPORT, > + }, > + .crypto_xform = sa->xforms > + > + }; > + > + if (sa->type == RTE_SECURITY_ACTION_TYPE_LOOKASIDE_PROTOCOL) { > + struct rte_security_ctx *ctx = (struct rte_security_ctx *) > + rte_cryptodev_get_sec_ctx( > + ipsec_ctx->tbl[cdev_id_qp].id); > + > + if (sess_conf.ipsec.mode == > + RTE_SECURITY_IPSEC_SA_MODE_TUNNEL) { > + struct rte_security_ipsec_tunnel_param *tunnel = > + &sess_conf.ipsec.tunnel; > + if (sa->flags == IP4_TUNNEL) { > + tunnel->type = > + RTE_SECURITY_IPSEC_TUNNEL_IPV4; > + tunnel->ipv4.ttl = IPDEFTTL; > + > + memcpy((uint8_t *)&tunnel->ipv4.src_ip, > + (uint8_t *)&sa->src.ip.ip4, 4); > + > + memcpy((uint8_t *)&tunnel->ipv4.dst_ip, > + (uint8_t *)&sa->dst.ip.ip4, 4); > + } > + /* TODO support for Transport and IPV6 tunnel */ > + } > + > + sa->sec_session = rte_security_session_create(ctx, > + &sess_conf, ipsec_ctx->session_pool); > + if (sa->sec_session == NULL) { > + RTE_LOG(ERR, IPSEC, > + "SEC Session init failed: err: %d\n", ret); > + return -1; > + } > + } else if (sa->type == RTE_SECURITY_ACTION_TYPE_INLINE_CRYPTO) { > + struct rte_flow_error err; > + struct rte_security_ctx *ctx = (struct rte_security_ctx *) > + rte_eth_dev_get_sec_ctx( > + sa->portid); > + const struct rte_security_capability *sec_cap; > + > + sa->sec_session = rte_security_session_create(ctx, > + &sess_conf, ipsec_ctx->session_pool); > + if (sa->sec_session == NULL) { > + RTE_LOG(ERR, IPSEC, > + "SEC Session init failed: err: %d\n", ret); > + return -1; > + } > + > + sec_cap = rte_security_capabilities_get(ctx); > + > + /* iterate until ESP tunnel*/ > + while (sec_cap->action != > + RTE_SECURITY_ACTION_TYPE_NONE) { > + > + if (sec_cap->action == sa->type && > + sec_cap->protocol == > + RTE_SECURITY_PROTOCOL_IPSEC && > + sec_cap->ipsec.mode == > + RTE_SECURITY_IPSEC_SA_MODE_TUNNEL && > + sec_cap->ipsec.direction == sa->direction) > + break; > + sec_cap++; > + } > + > + if (sec_cap->action == RTE_SECURITY_ACTION_TYPE_NONE) { > + RTE_LOG(ERR, IPSEC, > + "No suitable security capability found\n"); > + return -1; > + } > + > + sa->ol_flags = sec_cap->ol_flags; > + sa->security_ctx = ctx; > + sa->pattern[0].type = RTE_FLOW_ITEM_TYPE_ETH; > + > + sa->pattern[1].type = RTE_FLOW_ITEM_TYPE_IPV4; > + sa->pattern[1].mask = &rte_flow_item_ipv4_mask; > + if (sa->flags & IP6_TUNNEL) { > + sa->pattern[1].spec = &sa->ipv6_spec; > + memcpy(sa->ipv6_spec.hdr.dst_addr, > + sa->dst.ip.ip6.ip6_b, 16); > + memcpy(sa->ipv6_spec.hdr.src_addr, > + sa->src.ip.ip6.ip6_b, 16); > + } else { > + sa->pattern[1].spec = &sa->ipv4_spec; > + sa->ipv4_spec.hdr.dst_addr = sa->dst.ip.ip4; > + sa->ipv4_spec.hdr.src_addr = sa->src.ip.ip4; > + } > + > + sa->pattern[2].type = RTE_FLOW_ITEM_TYPE_ESP; > + sa->pattern[2].spec = &sa->esp_spec; > + sa->pattern[2].mask = &rte_flow_item_esp_mask; > + sa->esp_spec.hdr.spi = sa->spi; > + > + sa->pattern[3].type = RTE_FLOW_ITEM_TYPE_END; > + > + sa->action[0].type = RTE_FLOW_ACTION_TYPE_SECURITY; > + sa->action[0].conf = sa->sec_session; > + > + sa->action[1].type = RTE_FLOW_ACTION_TYPE_END; > + > + sa->attr.egress = (sa->direction == > + RTE_SECURITY_IPSEC_SA_DIR_EGRESS); > + sa->flow = rte_flow_create(sa->portid, > + &sa->attr, sa->pattern, sa->action, &err); > + if (sa->flow == NULL) { > + RTE_LOG(ERR, IPSEC, > + "Failed to create ipsec flow msg: %s\n", > + err.message); > + return -1; > + } > + } > + } else { > + sa->crypto_session = rte_cryptodev_sym_session_create( > + ipsec_ctx->session_pool); > + rte_cryptodev_sym_session_init(ipsec_ctx->tbl[cdev_id_qp].id, > + sa->crypto_session, sa->xforms, > + ipsec_ctx->session_pool); > + > + rte_cryptodev_info_get(ipsec_ctx->tbl[cdev_id_qp].id, > + &cdev_info); > + if (cdev_info.sym.max_nb_sessions_per_qp > 0) { > + ret = rte_cryptodev_queue_pair_attach_sym_session( > + ipsec_ctx->tbl[cdev_id_qp].id, > + ipsec_ctx->tbl[cdev_id_qp].qp, > + sa->crypto_session); > + if (ret < 0) { > + RTE_LOG(ERR, IPSEC, > + "Session cannot be attached to qp %u\n", > + ipsec_ctx->tbl[cdev_id_qp].qp); > + return -1; > + } > } > } > sa->cdev_id_qp = cdev_id_qp; > @@ -129,7 +264,9 @@ ipsec_enqueue(ipsec_xform_fn xform_func, struct ipsec_ctx *ipsec_ctx, > { > int32_t ret = 0, i; > struct ipsec_mbuf_metadata *priv; > + struct rte_crypto_sym_op *sym_cop; > struct ipsec_sa *sa; > + struct cdev_qp *cqp; > > for (i = 0; i < nb_pkts; i++) { > if (unlikely(sas[i] == NULL)) { > @@ -144,23 +281,76 @@ ipsec_enqueue(ipsec_xform_fn xform_func, struct ipsec_ctx *ipsec_ctx, > sa = sas[i]; > priv->sa = sa; > > - priv->cop.type = RTE_CRYPTO_OP_TYPE_SYMMETRIC; > - priv->cop.status = RTE_CRYPTO_OP_STATUS_NOT_PROCESSED; > - > - rte_prefetch0(&priv->sym_cop); > - > - if ((unlikely(sa->crypto_session == NULL)) && > - create_session(ipsec_ctx, sa)) { > - rte_pktmbuf_free(pkts[i]); > - continue; > - } > - > - rte_crypto_op_attach_sym_session(&priv->cop, > - sa->crypto_session); > - > - ret = xform_func(pkts[i], sa, &priv->cop); > - if (unlikely(ret)) { > - rte_pktmbuf_free(pkts[i]); > + switch (sa->type) { > + case RTE_SECURITY_ACTION_TYPE_LOOKASIDE_PROTOCOL: > + priv->cop.type = RTE_CRYPTO_OP_TYPE_SYMMETRIC; > + priv->cop.status = RTE_CRYPTO_OP_STATUS_NOT_PROCESSED; > + > + rte_prefetch0(&priv->sym_cop); > + > + if ((unlikely(sa->sec_session == NULL)) && > + create_session(ipsec_ctx, sa)) { > + rte_pktmbuf_free(pkts[i]); > + continue; > + } > + > + sym_cop = get_sym_cop(&priv->cop); > + sym_cop->m_src = pkts[i]; > + > + rte_security_attach_session(&priv->cop, > + sa->sec_session); > + break; > + case RTE_SECURITY_ACTION_TYPE_NONE: > + > + priv->cop.type = RTE_CRYPTO_OP_TYPE_SYMMETRIC; > + priv->cop.status = RTE_CRYPTO_OP_STATUS_NOT_PROCESSED; > + > + rte_prefetch0(&priv->sym_cop); > + > + if ((unlikely(sa->crypto_session == NULL)) && > + create_session(ipsec_ctx, sa)) { > + rte_pktmbuf_free(pkts[i]); > + continue; > + } > + > + rte_crypto_op_attach_sym_session(&priv->cop, > + sa->crypto_session); > + > + ret = xform_func(pkts[i], sa, &priv->cop); > + if (unlikely(ret)) { > + rte_pktmbuf_free(pkts[i]); > + continue; > + } > + break; > + case RTE_SECURITY_ACTION_TYPE_INLINE_PROTOCOL: > + break; > + case RTE_SECURITY_ACTION_TYPE_INLINE_CRYPTO: > + priv->cop.type = RTE_CRYPTO_OP_TYPE_SYMMETRIC; > + priv->cop.status = RTE_CRYPTO_OP_STATUS_NOT_PROCESSED; > + > + rte_prefetch0(&priv->sym_cop); > + > + if ((unlikely(sa->sec_session == NULL)) && > + create_session(ipsec_ctx, sa)) { > + rte_pktmbuf_free(pkts[i]); > + continue; > + } > + > + rte_security_attach_session(&priv->cop, > + sa->sec_session); > + > + ret = xform_func(pkts[i], sa, &priv->cop); > + if (unlikely(ret)) { > + rte_pktmbuf_free(pkts[i]); > + continue; > + } > + > + cqp = &ipsec_ctx->tbl[sa->cdev_id_qp]; > + cqp->ol_pkts[cqp->ol_pkts_cnt++] = pkts[i]; > + if (sa->ol_flags & RTE_SECURITY_TX_OLOAD_NEED_MDATA) > + rte_security_set_pkt_metadata( > + sa->security_ctx, > + sa->sec_session, pkts[i], NULL); > continue; > } > > @@ -171,7 +361,7 @@ ipsec_enqueue(ipsec_xform_fn xform_func, struct ipsec_ctx *ipsec_ctx, > > static inline int > ipsec_dequeue(ipsec_xform_fn xform_func, struct ipsec_ctx *ipsec_ctx, > - struct rte_mbuf *pkts[], uint16_t max_pkts) > + struct rte_mbuf *pkts[], uint16_t max_pkts) > { > int32_t nb_pkts = 0, ret = 0, i, j, nb_cops; > struct ipsec_mbuf_metadata *priv; > @@ -186,6 +376,19 @@ ipsec_dequeue(ipsec_xform_fn xform_func, struct ipsec_ctx *ipsec_ctx, > if (ipsec_ctx->last_qp == ipsec_ctx->nb_qps) > ipsec_ctx->last_qp %= ipsec_ctx->nb_qps; > > + while (cqp->ol_pkts_cnt > 0 && nb_pkts < max_pkts) { > + pkt = cqp->ol_pkts[--cqp->ol_pkts_cnt]; > + rte_prefetch0(pkt); > + priv = get_priv(pkt); > + sa = priv->sa; > + ret = xform_func(pkt, sa, &priv->cop); > + if (unlikely(ret)) { > + rte_pktmbuf_free(pkt); > + continue; > + } > + pkts[nb_pkts++] = pkt; > + } > + > if (cqp->in_flight == 0) > continue; > > @@ -203,11 +406,14 @@ ipsec_dequeue(ipsec_xform_fn xform_func, struct ipsec_ctx *ipsec_ctx, > > RTE_ASSERT(sa != NULL); > > - ret = xform_func(pkt, sa, cops[j]); > - if (unlikely(ret)) > - rte_pktmbuf_free(pkt); > - else > - pkts[nb_pkts++] = pkt; > + if (sa->type == RTE_SECURITY_ACTION_TYPE_NONE) { > + ret = xform_func(pkt, sa, cops[j]); > + if (unlikely(ret)) { > + rte_pktmbuf_free(pkt); > + continue; > + } > + } > + pkts[nb_pkts++] = pkt; > } > } > > diff --git a/examples/ipsec-secgw/ipsec.h b/examples/ipsec-secgw/ipsec.h > index 9e22b1b..613785f 100644 > --- a/examples/ipsec-secgw/ipsec.h > +++ b/examples/ipsec-secgw/ipsec.h > @@ -38,6 +38,8 @@ > > #include > #include > +#include > +#include > > #define RTE_LOGTYPE_IPSEC RTE_LOGTYPE_USER1 > #define RTE_LOGTYPE_IPSEC_ESP RTE_LOGTYPE_USER2 > @@ -99,7 +101,10 @@ struct ipsec_sa { > uint32_t cdev_id_qp; > uint64_t seq; > uint32_t salt; > - struct rte_cryptodev_sym_session *crypto_session; > + union { > + struct rte_cryptodev_sym_session *crypto_session; > + struct rte_security_session *sec_session; > + }; > enum rte_crypto_cipher_algorithm cipher_algo; > enum rte_crypto_auth_algorithm auth_algo; > enum rte_crypto_aead_algorithm aead_algo; > @@ -117,7 +122,28 @@ struct ipsec_sa { > uint8_t auth_key[MAX_KEY_SIZE]; > uint16_t auth_key_len; > uint16_t aad_len; > - struct rte_crypto_sym_xform *xforms; > + union { > + struct rte_crypto_sym_xform *xforms; > + struct rte_security_ipsec_xform *sec_xform; > + }; > + enum rte_security_session_action_type type; > + enum rte_security_ipsec_sa_direction direction; > + uint16_t portid; > + struct rte_security_ctx *security_ctx; > + uint32_t ol_flags; > + > +#define MAX_RTE_FLOW_PATTERN (4) > +#define MAX_RTE_FLOW_ACTIONS (2) > + struct rte_flow_item pattern[MAX_RTE_FLOW_PATTERN]; > + struct rte_flow_action action[MAX_RTE_FLOW_ACTIONS]; > + struct rte_flow_attr attr; > + union { > + struct rte_flow_item_ipv4 ipv4_spec; > + struct rte_flow_item_ipv6 ipv6_spec; > + }; > + struct rte_flow_item_esp esp_spec; > + struct rte_flow *flow; > + struct rte_security_session_conf sess_conf; > } __rte_cache_aligned; > > struct ipsec_mbuf_metadata { > @@ -133,6 +159,8 @@ struct cdev_qp { > uint16_t in_flight; > uint16_t len; > struct rte_crypto_op *buf[MAX_PKT_BURST] __rte_aligned(sizeof(void *)); > + struct rte_mbuf *ol_pkts[MAX_PKT_BURST] __rte_aligned(sizeof(void *)); > + uint16_t ol_pkts_cnt; > }; > > struct ipsec_ctx { > diff --git a/examples/ipsec-secgw/sa.c b/examples/ipsec-secgw/sa.c > index ef94475..d8ee47b 100644 > --- a/examples/ipsec-secgw/sa.c > +++ b/examples/ipsec-secgw/sa.c > @@ -41,16 +41,20 @@ > > #include > #include > +#include > #include > #include > #include > #include > #include > +#include > > #include "ipsec.h" > #include "esp.h" > #include "parser.h" > > +#define IPDEFTTL 64 > + > struct supported_cipher_algo { > const char *keyword; > enum rte_crypto_cipher_algorithm algo; > @@ -238,6 +242,8 @@ parse_sa_tokens(char **tokens, uint32_t n_tokens, > uint32_t src_p = 0; > uint32_t dst_p = 0; > uint32_t mode_p = 0; > + uint32_t type_p = 0; > + uint32_t portid_p = 0; > > if (strcmp(tokens[0], "in") == 0) { > ri = &nb_sa_in; > @@ -550,6 +556,52 @@ parse_sa_tokens(char **tokens, uint32_t n_tokens, > continue; > } > > + if (strcmp(tokens[ti], "type") == 0) { > + APP_CHECK_PRESENCE(type_p, tokens[ti], status); > + if (status->status < 0) > + return; > + > + INCREMENT_TOKEN_INDEX(ti, n_tokens, status); > + if (status->status < 0) > + return; > + > + if (strcmp(tokens[ti], "inline-crypto-offload") == 0) > + rule->type = > + RTE_SECURITY_ACTION_TYPE_INLINE_CRYPTO; > + else if (strcmp(tokens[ti], > + "inline-protocol-offload") == 0) > + rule->type = > + RTE_SECURITY_ACTION_TYPE_INLINE_PROTOCOL; > + else if (strcmp(tokens[ti], > + "lookaside-protocol-offload") == 0) > + rule->type = > + RTE_SECURITY_ACTION_TYPE_LOOKASIDE_PROTOCOL; > + else if (strcmp(tokens[ti], "no-offload") == 0) > + rule->type = RTE_SECURITY_ACTION_TYPE_NONE; > + else { > + APP_CHECK(0, status, "Invalid input \"%s\"", > + tokens[ti]); > + return; > + } > + > + type_p = 1; > + continue; > + } > + > + if (strcmp(tokens[ti], "port_id") == 0) { > + APP_CHECK_PRESENCE(portid_p, tokens[ti], status); > + if (status->status < 0) > + return; > + INCREMENT_TOKEN_INDEX(ti, n_tokens, status); > + if (status->status < 0) > + return; > + rule->portid = atoi(tokens[ti]); > + if (status->status < 0) > + return; > + portid_p = 1; > + continue; > + } > + > /* unrecognizeable input */ > APP_CHECK(0, status, "unrecognized input \"%s\"", > tokens[ti]); > @@ -580,6 +632,14 @@ parse_sa_tokens(char **tokens, uint32_t n_tokens, > if (status->status < 0) > return; > > + if ((rule->type != RTE_SECURITY_ACTION_TYPE_NONE) && (portid_p == 0)) > + printf("Missing portid option, falling back to non-offload\n"); > + > + if (!type_p || !portid_p) { > + rule->type = RTE_SECURITY_ACTION_TYPE_NONE; > + rule->portid = -1; > + } > + > *ri = *ri + 1; > } > > @@ -647,9 +707,11 @@ print_one_sa_rule(const struct ipsec_sa *sa, int inbound) > > struct sa_ctx { > struct ipsec_sa sa[IPSEC_SA_MAX_ENTRIES]; > - struct { > - struct rte_crypto_sym_xform a; > - struct rte_crypto_sym_xform b; > + union { > + struct { > + struct rte_crypto_sym_xform a; > + struct rte_crypto_sym_xform b; > + }; > } xf[IPSEC_SA_MAX_ENTRIES]; > }; > > @@ -682,6 +744,33 @@ sa_create(const char *name, int32_t socket_id) > } > > static int > +check_eth_dev_caps(uint16_t portid, uint32_t inbound) > +{ > + struct rte_eth_dev_info dev_info; > + > + rte_eth_dev_info_get(portid, &dev_info); > + > + if (inbound) { > + if ((dev_info.rx_offload_capa & > + DEV_RX_OFFLOAD_SECURITY) == 0) { > + RTE_LOG(WARNING, PORT, > + "hardware RX IPSec offload is not supported\n"); > + return -EINVAL; > + } > + > + } else { /* outbound */ > + if ((dev_info.tx_offload_capa & > + DEV_TX_OFFLOAD_SECURITY) == 0) { > + RTE_LOG(WARNING, PORT, > + "hardware TX IPSec offload is not supported\n"); > + return -EINVAL; > + } > + } > + return 0; > +} > + > + > +static int > sa_add_rules(struct sa_ctx *sa_ctx, const struct ipsec_sa entries[], > uint32_t nb_entries, uint32_t inbound) > { > @@ -700,6 +789,16 @@ sa_add_rules(struct sa_ctx *sa_ctx, const struct ipsec_sa entries[], > *sa = entries[i]; > sa->seq = 0; > > + if (sa->type == RTE_SECURITY_ACTION_TYPE_INLINE_PROTOCOL || > + sa->type == RTE_SECURITY_ACTION_TYPE_INLINE_CRYPTO) { > + if (check_eth_dev_caps(sa->portid, inbound)) > + return -EINVAL; > + } > + > + sa->direction = (inbound == 1) ? > + RTE_SECURITY_IPSEC_SA_DIR_INGRESS : > + RTE_SECURITY_IPSEC_SA_DIR_EGRESS; > + > switch (sa->flags) { > case IP4_TUNNEL: > sa->src.ip.ip4 = rte_cpu_to_be_32(sa->src.ip.ip4); > @@ -709,37 +808,21 @@ sa_add_rules(struct sa_ctx *sa_ctx, const struct ipsec_sa entries[], > if (sa->aead_algo == RTE_CRYPTO_AEAD_AES_GCM) { > iv_length = 16; > > - if (inbound) { > - sa_ctx->xf[idx].a.type = RTE_CRYPTO_SYM_XFORM_AEAD; > - sa_ctx->xf[idx].a.aead.algo = sa->aead_algo; > - sa_ctx->xf[idx].a.aead.key.data = sa->cipher_key; > - sa_ctx->xf[idx].a.aead.key.length = > - sa->cipher_key_len; > - sa_ctx->xf[idx].a.aead.op = > - RTE_CRYPTO_AEAD_OP_DECRYPT; > - sa_ctx->xf[idx].a.next = NULL; > - sa_ctx->xf[idx].a.aead.iv.offset = IV_OFFSET; > - sa_ctx->xf[idx].a.aead.iv.length = iv_length; > - sa_ctx->xf[idx].a.aead.aad_length = > - sa->aad_len; > - sa_ctx->xf[idx].a.aead.digest_length = > - sa->digest_len; > - } else { /* outbound */ > - sa_ctx->xf[idx].a.type = RTE_CRYPTO_SYM_XFORM_AEAD; > - sa_ctx->xf[idx].a.aead.algo = sa->aead_algo; > - sa_ctx->xf[idx].a.aead.key.data = sa->cipher_key; > - sa_ctx->xf[idx].a.aead.key.length = > - sa->cipher_key_len; > - sa_ctx->xf[idx].a.aead.op = > - RTE_CRYPTO_AEAD_OP_ENCRYPT; > - sa_ctx->xf[idx].a.next = NULL; > - sa_ctx->xf[idx].a.aead.iv.offset = IV_OFFSET; > - sa_ctx->xf[idx].a.aead.iv.length = iv_length; > - sa_ctx->xf[idx].a.aead.aad_length = > - sa->aad_len; > - sa_ctx->xf[idx].a.aead.digest_length = > - sa->digest_len; > - } > + sa_ctx->xf[idx].a.type = RTE_CRYPTO_SYM_XFORM_AEAD; > + sa_ctx->xf[idx].a.aead.algo = sa->aead_algo; > + sa_ctx->xf[idx].a.aead.key.data = sa->cipher_key; > + sa_ctx->xf[idx].a.aead.key.length = > + sa->cipher_key_len; > + sa_ctx->xf[idx].a.aead.op = (inbound == 1) ? > + RTE_CRYPTO_AEAD_OP_DECRYPT : > + RTE_CRYPTO_AEAD_OP_ENCRYPT; > + sa_ctx->xf[idx].a.next = NULL; > + sa_ctx->xf[idx].a.aead.iv.offset = IV_OFFSET; > + sa_ctx->xf[idx].a.aead.iv.length = iv_length; > + sa_ctx->xf[idx].a.aead.aad_length = > + sa->aad_len; > + sa_ctx->xf[idx].a.aead.digest_length = > + sa->digest_len; > > sa->xforms = &sa_ctx->xf[idx].a; > Tested-by: Aviad Yehezkel