From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mga03.intel.com (mga03.intel.com [134.134.136.65]) by dpdk.org (Postfix) with ESMTP id 080067EE3 for ; Fri, 24 Aug 2018 18:54:04 +0200 (CEST) X-Amp-Result: SKIPPED(no attachment in message) X-Amp-File-Uploaded: False Received: from orsmga006.jf.intel.com ([10.7.209.51]) by orsmga103.jf.intel.com with ESMTP/TLS/DHE-RSA-AES256-GCM-SHA384; 24 Aug 2018 09:54:03 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.53,283,1531810800"; d="scan'208";a="68866918" Received: from sivswdev02.ir.intel.com (HELO localhost.localdomain) ([10.237.217.46]) by orsmga006.jf.intel.com with ESMTP; 24 Aug 2018 09:53:34 -0700 From: Konstantin Ananyev To: dev@dpdk.org Cc: Konstantin Ananyev , Mohammad Abdul Awal , Declan Doherty Date: Fri, 24 Aug 2018 17:53:18 +0100 Message-Id: <1535129598-27301-1-git-send-email-konstantin.ananyev@intel.com> X-Mailer: git-send-email 1.7.0.7 Subject: [dpdk-dev] [RFC] ipsec: new library for IPsec data-path processing X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 24 Aug 2018 16:54:05 -0000 This RFC introduces a new library within DPDK: librte_ipsec. The aim is to provide DPDK native high performance library for IPsec data-path processing. The library is supposed to utilize existing DPDK crypto-dev and security API to provide application with transparent IPsec processing API. The library is concentrated on data-path protocols processing (ESP and AH), IKE protocol(s) implementation is out of scope for that library. Though hook/callback mechanisms will be defined to allow integrate it with existing IKE implementations. Due to quite complex nature of IPsec protocol suite and variety of user requirements and usage scenarios a few API levels will be provided: 1) Security Association (SA-level) API Operates at SA level, provides functions to: - initialize/teardown SA object - process inbound/outbound ESP/AH packets associated with the given SA (decrypt/encrypt, authenticate, check integrity, add/remove ESP/AH related headers and data, etc.). 2) Security Association Database (SAD) API API to create/manage/destroy IPsec SAD. While DPDK IPsec library plans to have its own implementation, the intention is to keep it as independent from the other parts of IPsec library as possible. That is supposed to give users the ability to provide their own implementation of the SAD compatible with the other parts of the IPsec library. 3) IPsec Context (CTX) API This is supposed to be a high-level API, where each IPsec CTX is an abstraction of 'independent copy of the IPsec stack'. CTX owns set of SAs, SADs and assigned to it crypto-dev queues, etc. and provides: - de-multiplexing stream of inbound packets to particular SAs and further IPsec related processing. - IPsec related processing for the outbound packets. - SA add/delete/update functionality Current RFC concentrates on SA-level API only (1), detailed discussion for 2) and 3) will be subjects for separate RFC(s). SA (low) level API ================== API described below operates on SA level. It provides functionality that allows user for given SA to process inbound and outbound IPsec packets. To be more specific: - for inbound ESP/AH packets perform decryption, authentication, integrity checking, remove ESP/AH related headers - for outbound packets perform payload encryption, attach ICV, update/add IP headers, add ESP/AH headers/trailers, setup related mbuf felids (ol_flags, tx_offloads, etc.). - initialize/un-initialize given SA based on user provided parameters. Processed inbound/outbound packets could be grouped by user provided flow id (opaque 64-bit number associated by user with given SA). SA-level API is based on top of crypto-dev/security API and relies on them to perform actual cipher and integrity checking. Due to the nature of crypto-dev API (enqueue/deque model) we use asynchronous API for IPsec packets destined to be processed by crypto-device: rte_ipsec_crypto_prepare()->rte_cryptodev_enqueue_burst()-> rte_cryptodev_dequeue_burst()->rte_ipsec_crypto_process(). Though for packets destined for inline processing no extra overhead is required and simple and synchronous API: rte_ipsec_inline_process() is introduced for that case. The following functionality: - match inbound/outbound packets to particular SA - manage crypto/security devices - provide SAD/SPD related functionality - determine what crypto/security device has to be used for given packet(s) is out of scope for SA-level API. Below is the brief (and simplified) overview of expected SA-level API usage. /* allocate and initialize SA */ size_t sz = rte_ipsec_sa_size(); struct rte_ipsec_sa *sa = rte_malloc(sz); struct rte_ipsec_sa_prm prm; /* fill prm */ rc = rte_ipsec_sa_init(sa, &prm); if (rc != 0) { /*handle error */} ..... /* process inbound/outbound IPsec packets that belongs to given SA */ /* inline IPsec processing was done for these packets */ if (use_inline_ipsec) n = rte_ipsec_inline_process(sa, pkts, nb_pkts); /* use crypto-device to process the packets */ else { struct rte_crypto_op *cop[nb_pkts]; struct rte_ipsec_group grp[nb_pkts]; .... /* prepare crypto ops */ n = rte_ipsec_crypto_prepare(sa, pkts, cops, nb_pkts); /* enqueue crypto ops to related crypto-dev */ n = rte_cryptodev_enqueue_burst(..., cops, n); if (n != nb_pkts) { /*handle failed packets */} /* dequeue finished crypto ops from related crypto-dev */ n = rte_cryptodev_dequeue_burst(..., cops, nb_pkts); /* finish IPsec processing for associated packets */ n = rte_ipsec_crypto_process(cop, pkts, grp, n); /* now we have group of packets grouped by SA flow id */ .... } ... /* uninit given SA */ rte_ipsec_sa_fini(sa); Planned scope for 18.11: ======================== - SA-level API definition - ESP tunnel mode support (both IPv4/IPv6) - Supported algorithms: AES-CBC, AES-GCM, HMAC-SHA1, NULL. - UT Note: Still WIP, so not all planned for 18.11 functionality is in place. Post 18.11: =========== - ESP transport mode support (both IPv4/IPv6) - update examples/ipsec-secgw to use librte_ipsec - SAD and high-level API definition and implementation Signed-off-by: Mohammad Abdul Awal Signed-off-by: Declan Doherty Signed-off-by: Konstantin Ananyev --- config/common_base | 5 + lib/Makefile | 2 + lib/librte_ipsec/Makefile | 24 + lib/librte_ipsec/meson.build | 10 + lib/librte_ipsec/pad.h | 45 ++ lib/librte_ipsec/rte_ipsec.h | 245 +++++++++ lib/librte_ipsec/rte_ipsec_version.map | 13 + lib/librte_ipsec/sa.c | 921 +++++++++++++++++++++++++++++++++ lib/librte_net/rte_esp.h | 10 +- lib/meson.build | 2 + mk/rte.app.mk | 2 + 11 files changed, 1278 insertions(+), 1 deletion(-) create mode 100644 lib/librte_ipsec/Makefile create mode 100644 lib/librte_ipsec/meson.build create mode 100644 lib/librte_ipsec/pad.h create mode 100644 lib/librte_ipsec/rte_ipsec.h create mode 100644 lib/librte_ipsec/rte_ipsec_version.map create mode 100644 lib/librte_ipsec/sa.c diff --git a/config/common_base b/config/common_base index 4bcbaf923..c95602c05 100644 --- a/config/common_base +++ b/config/common_base @@ -879,6 +879,11 @@ CONFIG_RTE_LIBRTE_BPF=y CONFIG_RTE_LIBRTE_BPF_ELF=n # +# Compile librte_ipsec +# +CONFIG_RTE_LIBRTE_IPSEC=y + +# # Compile the test application # CONFIG_RTE_APP_TEST=y diff --git a/lib/Makefile b/lib/Makefile index afa604e20..58998dedd 100644 --- a/lib/Makefile +++ b/lib/Makefile @@ -105,6 +105,8 @@ DEPDIRS-librte_gso := librte_eal librte_mbuf librte_ethdev librte_net DEPDIRS-librte_gso += librte_mempool DIRS-$(CONFIG_RTE_LIBRTE_BPF) += librte_bpf DEPDIRS-librte_bpf := librte_eal librte_mempool librte_mbuf librte_ethdev +DIRS-$(CONFIG_RTE_LIBRTE_IPSEC) += librte_ipsec +DEPDIRS-librte_ipsec := librte_eal librte_mbuf librte_cryptodev librte_security ifeq ($(CONFIG_RTE_EXEC_ENV_LINUXAPP),y) DIRS-$(CONFIG_RTE_LIBRTE_KNI) += librte_kni diff --git a/lib/librte_ipsec/Makefile b/lib/librte_ipsec/Makefile new file mode 100644 index 000000000..15441cf41 --- /dev/null +++ b/lib/librte_ipsec/Makefile @@ -0,0 +1,24 @@ +# SPDX-License-Identifier: BSD-3-Clause +# Copyright(c) 2018 Intel Corporation + +include $(RTE_SDK)/mk/rte.vars.mk + +# library name +LIB = librte_ipsec.a + +CFLAGS += -O3 +CFLAGS += $(WERROR_FLAGS) -I$(SRCDIR) +CFLAGS += -DALLOW_EXPERIMENTAL_API +LDLIBS += -lrte_eal -lrte_mbuf -lrte_cryptodev -lrte_security + +EXPORT_MAP := rte_ipsec_version.map + +LIBABIVER := 1 + +# all source are stored in SRCS-y +SRCS-$(CONFIG_RTE_LIBRTE_IPSEC) += sa.c + +# install header files +SYMLINK-$(CONFIG_RTE_LIBRTE_IPSEC)-include += rte_ipsec.h + +include $(RTE_SDK)/mk/rte.lib.mk diff --git a/lib/librte_ipsec/meson.build b/lib/librte_ipsec/meson.build new file mode 100644 index 000000000..79c55a8be --- /dev/null +++ b/lib/librte_ipsec/meson.build @@ -0,0 +1,10 @@ +# SPDX-License-Identifier: BSD-3-Clause +# Copyright(c) 2018 Intel Corporation + +allow_experimental_apis = true + +sources=files('sa.c') + +install_headers = files('rte_ipsec.h') + +deps += ['mbuf', 'net', 'cryptodev', 'security'] diff --git a/lib/librte_ipsec/pad.h b/lib/librte_ipsec/pad.h new file mode 100644 index 000000000..2f5ccd00e --- /dev/null +++ b/lib/librte_ipsec/pad.h @@ -0,0 +1,45 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright(c) 2018 Intel Corporation + */ + +#ifndef _PAD_H_ +#define _PAD_H_ + +#define IPSEC_MAX_PAD_SIZE UINT8_MAX + +static const uint8_t esp_pad_bytes[IPSEC_MAX_PAD_SIZE] = { + 1, 2, 3, 4, 5, 6, 7, 8, + 9, 10, 11, 12, 13, 14, 15, 16, + 17, 18, 19, 20, 21, 22, 23, 24, + 25, 26, 27, 28, 29, 30, 31, 32, + 33, 34, 35, 36, 37, 38, 39, 40, + 41, 42, 43, 44, 45, 46, 47, 48, + 49, 50, 51, 52, 53, 54, 55, 56, + 57, 58, 59, 60, 61, 62, 63, 64, + 65, 66, 67, 68, 69, 70, 71, 72, + 73, 74, 75, 76, 77, 78, 79, 80, + 81, 82, 83, 84, 85, 86, 87, 88, + 89, 90, 91, 92, 93, 94, 95, 96, + 97, 98, 99, 100, 101, 102, 103, 104, + 105, 106, 107, 108, 109, 110, 111, 112, + 113, 114, 115, 116, 117, 118, 119, 120, + 121, 122, 123, 124, 125, 126, 127, 128, + 129, 130, 131, 132, 133, 134, 135, 136, + 137, 138, 139, 140, 141, 142, 143, 144, + 145, 146, 147, 148, 149, 150, 151, 152, + 153, 154, 155, 156, 157, 158, 159, 160, + 161, 162, 163, 164, 165, 166, 167, 168, + 169, 170, 171, 172, 173, 174, 175, 176, + 177, 178, 179, 180, 181, 182, 183, 184, + 185, 186, 187, 188, 189, 190, 191, 192, + 193, 194, 195, 196, 197, 198, 199, 200, + 201, 202, 203, 204, 205, 206, 207, 208, + 209, 210, 211, 212, 213, 214, 215, 216, + 217, 218, 219, 220, 221, 222, 223, 224, + 225, 226, 227, 228, 229, 230, 231, 232, + 233, 234, 235, 236, 237, 238, 239, 240, + 241, 242, 243, 244, 245, 246, 247, 248, + 249, 250, 251, 252, 253, 254, 255, +}; + +#endif /* _PAD_H_ */ diff --git a/lib/librte_ipsec/rte_ipsec.h b/lib/librte_ipsec/rte_ipsec.h new file mode 100644 index 000000000..d1154eede --- /dev/null +++ b/lib/librte_ipsec/rte_ipsec.h @@ -0,0 +1,245 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright(c) 2018 Intel Corporation + */ + +#ifndef _RTE_IPSEC_H_ +#define _RTE_IPSEC_H_ + +/** + * @file rte_ipsec.h + * @b EXPERIMENTAL: this API may change without prior notice + * + * RTE IPsec support. + * librte_ipsec provides a framework for data-path IPsec protocol + * processing (ESP/AH). + * IKEv2 protocol support right now is out of scope of that draft. + * Though it tries to define related API in such way, that it could be adopted + * by IKEv2 implementation. + */ + +#include +#include +#include +#include + +#ifdef __cplusplus +extern "C" { +#endif + +/** + * An opaque structure to represent Security Association (SA). + */ +struct rte_ipsec_sa; + +/** + * SA initialization parameters. + */ +struct rte_ipsec_sa_prm { + + uint64_t flowid; /**< provided and interpreted by user */ + struct rte_security_ipsec_xform ipsec_xform; /**< SA configuration */ + union { + struct { + uint8_t hdr_len; /**< tunnel header len */ + uint8_t hdr_l3_off; /**< offset for IPv4/IPv6 header */ + uint8_t next_proto; /**< next header protocol */ + const void *hdr; /**< tunnel header template */ + } tun; /**< tunnel mode repated parameters */ + struct { + uint8_t proto; /**< next header protocol */ + } trs; /**< transport mode repated parameters */ + }; + + struct { + enum rte_security_session_action_type type; + struct rte_security_ctx *sctx; + struct rte_security_session *sses; + uint32_t ol_flags; + } sec; /**< rte_security related parameters */ + + struct { + struct rte_crypto_sym_xform *xform; + struct rte_mempool *pool; + /** +#include +#include +#include +#include +#include "pad.h" + +#define IPSEC_MAX_HDR_SIZE 64 +#define IPSEC_MAX_IV_SIZE (2 * sizeof(uint64_t)) + +#define IPSEC_MAX_CRYPTO_DEVS (UINT8_MAX + 1) + +/* ??? these definitions probably has to be in rte_crypto_sym.h */ +union sym_op_ofslen { + uint64_t raw; + struct { + uint32_t offset; + uint32_t length; + }; +}; + +union sym_op_data { + __uint128_t raw; + struct { + uint8_t *va; + rte_iova_t pa; + }; +}; + +struct rte_ipsec_sa { + uint64_t type; /* type of given SA */ + uint64_t flowid; /* user defined */ + uint32_t spi; + uint32_t salt; + uint64_t sqn; + uint64_t *iv_ptr; + uint8_t aad_len; + uint8_t hdr_len; + uint8_t hdr_l3_off; + uint8_t icv_len; + uint8_t iv_len; + uint8_t pad_align; + uint8_t proto; /* next proto */ + /* template for crypto op fields */ + struct { + union sym_op_ofslen cipher; + union sym_op_ofslen auth; + uint8_t type; + uint8_t status; + uint8_t sess_type; + } ctp; + struct { + uint64_t v8; + uint64_t v[IPSEC_MAX_IV_SIZE / sizeof(uint64_t)]; + } iv; + uint8_t hdr[IPSEC_MAX_HDR_SIZE]; + + struct { + struct rte_security_session *sec; + uint32_t ol_flags; + struct rte_security_ctx *sctx; + + /* + * !!! should be removed if we do crypto sym session properly + * bitmap of crypto devs for which that session was initialised. + */ + rte_ymm_t cdev_bmap; + + /* + * !!! as alternative we need a space in cryptodev_sym_session + * to store ptr to SA (uint64_t udata or so). + */ + struct rte_cryptodev_sym_session crypto; + } session __rte_cache_min_aligned; + +} __rte_cache_aligned; + +#define CS_TO_SA(cs) ((cs) - offsetof(struct rte_ipsec_sa, session.crypto)) + +/* some helper structures */ +struct crypto_xform { + struct rte_crypto_auth_xform *auth; + struct rte_crypto_cipher_xform *cipher; + struct rte_crypto_aead_xform *aead; +}; + +static inline struct rte_ipsec_sa * +cses2sa(uintptr_t p) +{ + p -= offsetof(struct rte_ipsec_sa, session.crypto); + return (struct rte_ipsec_sa *)p; +} + +static int +check_crypto_xform(struct crypto_xform *xform) +{ + uintptr_t p; + + p = (uintptr_t)xform->auth | (uintptr_t)xform->cipher; + + /* either aead or both auth and cipher should be not NULLs */ + if (xform->aead) { + if (p) + return -EINVAL; + } else if (p == (uintptr_t)xform->auth) { + return -EINVAL; + } + + return 0; +} + +static int +fill_crypto_xform(struct crypto_xform *xform, + const struct rte_ipsec_sa_prm *prm) +{ + struct rte_crypto_sym_xform *xf; + + memset(xform, 0, sizeof(*xform)); + + for (xf = prm->crypto.xform; xf != NULL; xf = xf->next) { + if (xf->type == RTE_CRYPTO_SYM_XFORM_AUTH) { + if (xform->auth != NULL) + return -EINVAL; + xform->auth = &xf->auth; + } else if (xf->type == RTE_CRYPTO_SYM_XFORM_CIPHER) { + if (xform->cipher != NULL) + return -EINVAL; + xform->cipher = &xf->cipher; + } else if (xf->type == RTE_CRYPTO_SYM_XFORM_AEAD) { + if (xform->aead != NULL) + return -EINVAL; + xform->aead = &xf->aead; + } else + return -EINVAL; + } + + return check_crypto_xform(xform); +} + +/* + * !!! we might not need session fini - if cryptodev layer would have similar + * functionality. + */ +static void +crypto_session_fini(struct rte_ipsec_sa *sa) +{ + uint64_t v; + size_t sz; + uint32_t i, j; + + sz = sizeof(sa->session.cdev_bmap.u64[0]) * CHAR_BIT; + + for (i = 0; i != RTE_DIM(sa->session.cdev_bmap.u64); i++) { + + v = sa->session.cdev_bmap.u64[0]; + for (j = 0; v != 0; v >>= 1, j++) { + if ((v & 1) != 0) + rte_cryptodev_sym_session_clear(i * sz + j, + &sa->session.crypto); + } + sa->session.cdev_bmap.u64[i] = 0; + } +} + +static int +crypto_session_init(struct rte_ipsec_sa *sa, const struct rte_ipsec_sa_prm *prm) +{ + size_t sz; + uint32_t i, k; + int32_t rc; + + rc = 0; + sz = sizeof(sa->session.cdev_bmap.u64[0]) * CHAR_BIT; + + for (i = 0; i != prm->crypto.nb_dev; i++) { + + k = prm->crypto.devid[i]; + rc = rte_cryptodev_sym_session_init(k, &sa->session.crypto, + prm->crypto.xform, prm->crypto.pool); + if (rc != 0) + break; + sa->session.cdev_bmap.u64[k / sz] |= 1 << (k % sz); + } + + return rc; +} + +uint64_t __rte_experimental +rte_ipsec_sa_type(const struct rte_ipsec_sa *sa) +{ + return sa->type; +} + +size_t __rte_experimental +rte_ipsec_sa_size(void) +{ + size_t sz; + + sz = sizeof(struct rte_ipsec_sa) + + rte_cryptodev_sym_get_header_session_size(); + sz = RTE_ALIGN_CEIL(sz, RTE_CACHE_LINE_SIZE); + return sz; +} + +void __rte_experimental +rte_ipsec_sa_fini(struct rte_ipsec_sa *sa) +{ + size_t sz; + + sz = rte_ipsec_sa_size(); + crypto_session_fini(sa); + memset(sa, 0, sz); +} + +static uint64_t +fill_sa_type(const struct rte_ipsec_sa_prm *prm) +{ + uint64_t tp; + + tp = 0; + + if (prm->ipsec_xform.proto == RTE_SECURITY_IPSEC_SA_PROTO_AH) + tp |= RTE_IPSEC_SATP_PROTO_AH; + else if (prm->ipsec_xform.proto == RTE_SECURITY_IPSEC_SA_PROTO_ESP) + tp |= RTE_IPSEC_SATP_PROTO_ESP; + + if (prm->ipsec_xform.direction == RTE_SECURITY_IPSEC_SA_DIR_EGRESS) + tp |= RTE_IPSEC_SATP_DIR_OB; + else + tp |= RTE_IPSEC_SATP_DIR_IB; + + if (prm->ipsec_xform.mode == RTE_SECURITY_IPSEC_SA_MODE_TUNNEL) { + if (prm->ipsec_xform.tunnel.type == + RTE_SECURITY_IPSEC_TUNNEL_IPV4) + tp |= RTE_IPSEC_SATP_MODE_TUNLV4; + else + tp |= RTE_IPSEC_SATP_MODE_TUNLV6; + + if (prm->tun.next_proto == IPPROTO_IPIP) + tp |= RTE_IPSEC_SATP_IPV4; + else if (prm->tun.next_proto == IPPROTO_IPV6) + tp |= RTE_IPSEC_SATP_IPV4; + } else { + tp |= RTE_IPSEC_SATP_MODE_TRANS; + if (prm->trs.proto == IPPROTO_IPIP) + tp |= RTE_IPSEC_SATP_IPV4; + else if (prm->trs.proto == IPPROTO_IPV6) + tp |= RTE_IPSEC_SATP_IPV4; + } + + /* !!! some inline ipsec support right now */ + if (prm->sec.type == RTE_SECURITY_ACTION_TYPE_INLINE_CRYPTO) + tp |= RTE_IPSEC_SATP_USE_INLN; + else + tp |= RTE_IPSEC_SATP_USE_LKSD; + + return tp; +} + +static void +esp_inb_tun_init(struct rte_ipsec_sa *sa) +{ + /* these params may differ with new algorithms support */ + sa->ctp.auth.offset = 0; + sa->ctp.auth.length = sa->icv_len; + sa->ctp.cipher.offset = sizeof(struct esp_hdr) + sa->iv_len; + sa->ctp.cipher.length = sa->ctp.auth.length + sa->ctp.cipher.offset; +} + +static void +esp_outb_tun_init(struct rte_ipsec_sa *sa, const struct rte_ipsec_sa_prm *prm) +{ + sa->hdr_len = prm->tun.hdr_len; + sa->hdr_l3_off = prm->tun.hdr_l3_off; + memcpy(sa->hdr, prm->tun.hdr, sa->hdr_len); + + /* these params may differ with new algorithms support */ + sa->ctp.auth.offset = sa->hdr_len; + sa->ctp.auth.length = sizeof(struct esp_hdr) + sa->iv_len; + sa->ctp.cipher.offset = sa->hdr_len + sizeof(struct esp_hdr); + sa->ctp.cipher.length = sa->iv_len; +} + +static int +esp_tun_init(struct rte_ipsec_sa *sa, const struct rte_ipsec_sa_prm *prm, + const struct crypto_xform *cxf) +{ + int32_t rc = 0; + + if (cxf->aead != NULL) { + if (cxf->aead->algo != RTE_CRYPTO_AEAD_AES_GCM) + return -EINVAL; + sa->aad_len = cxf->aead->aad_length; + sa->icv_len = cxf->aead->digest_length; + sa->iv_len = cxf->aead->iv.length; + sa->iv_ptr = sa->iv.v; + sa->pad_align = 4; + } else { + sa->aad_len = 0; + sa->icv_len = cxf->auth->digest_length; + if (cxf->cipher->algo == RTE_CRYPTO_CIPHER_NULL) { + sa->pad_align = 4; + sa->iv_len = 0; + sa->iv_ptr = sa->iv.v; + } else if (cxf->cipher->algo == RTE_CRYPTO_CIPHER_AES_CBC) { + sa->pad_align = sizeof(sa->iv.v); + sa->iv_len = sizeof(sa->iv.v); + sa->iv_ptr = sa->iv.v; + memset(sa->iv.v, 0, sizeof(sa->iv.v)); + } else + return -EINVAL; + } + + sa->type = fill_sa_type(prm); + sa->flowid = prm->flowid; + sa->spi = rte_cpu_to_be_32(prm->ipsec_xform.spi); + sa->salt = prm->ipsec_xform.salt; + sa->sqn = 0; + + sa->proto = prm->tun.next_proto; + + if ((sa->type & RTE_IPSEC_SATP_DIR_MASK) == RTE_IPSEC_SATP_DIR_IB) + esp_inb_tun_init(sa); + else + esp_outb_tun_init(sa, prm); + + sa->ctp.type = RTE_CRYPTO_OP_TYPE_SYMMETRIC; + sa->ctp.status = RTE_CRYPTO_OP_STATUS_NOT_PROCESSED; + sa->ctp.sess_type = RTE_CRYPTO_OP_WITH_SESSION; + + /* pass info required for inline outbound */ + sa->session.sctx = prm->sec.sctx; + sa->session.sec = prm->sec.sses; + sa->session.ol_flags = prm->sec.ol_flags; + + if ((sa->type & RTE_IPSEC_SATP_USE_MASK) != RTE_IPSEC_SATP_USE_INLN) + rc = crypto_session_init(sa, prm); + return rc; +} + +int __rte_experimental +rte_ipsec_sa_init(struct rte_ipsec_sa *sa, const struct rte_ipsec_sa_prm *prm) +{ + int32_t rc; + struct crypto_xform cxf; + + if (sa == NULL || prm == NULL) + return -EINVAL; + + /* only esp inbound and outbound tunnel is supported right now */ + if (prm->ipsec_xform.proto != RTE_SECURITY_IPSEC_SA_PROTO_ESP || + prm->ipsec_xform.mode != + RTE_SECURITY_IPSEC_SA_MODE_TUNNEL) + return -EINVAL; + + /* only inline crypto or none action type are supported */ + if (!(prm->sec.type == RTE_SECURITY_ACTION_TYPE_INLINE_CRYPTO || + prm->sec.type == RTE_SECURITY_ACTION_TYPE_NONE)) + return -EINVAL; + + if (prm->tun.hdr_len > sizeof(sa->hdr)) + return -EINVAL; + + rc = fill_crypto_xform(&cxf, prm); + if (rc != 0) + return rc; + + rc = esp_tun_init(sa, prm, &cxf); + if (rc != 0) + rte_ipsec_sa_fini(sa); + + return rc; +} + +static inline void +esp_outb_tun_cop_prepare(struct rte_crypto_op *cop, + const struct rte_ipsec_sa *sa, struct rte_mbuf *mb, + const union sym_op_data *icv, uint32_t plen) +{ + struct rte_crypto_sym_op *sop; + + cop->type = sa->ctp.type; + cop->status = sa->ctp.status; + cop->sess_type = sa->ctp.sess_type; + + sop = cop->sym; + + /* fill sym op fields */ + sop->session = (void *)(uintptr_t)&sa->session.crypto; + sop->m_src = mb; + + sop->cipher.data.offset = sa->ctp.cipher.offset; + sop->cipher.data.length = sa->ctp.cipher.length + plen; + sop->auth.data.offset = sa->ctp.auth.offset; + sop->auth.data.length = sa->ctp.auth.length + plen; + sop->auth.digest.data = icv->va; + sop->auth.digest.phys_addr = icv->pa; + + /* !!! fill sym op aead fields */ +} + +static inline int32_t +esp_outb_tun_pkt_prepare(struct rte_ipsec_sa *sa, struct rte_mbuf *mb, + union sym_op_data *icv) +{ + uint32_t clen, hlen, pdlen, pdofs, tlen; + struct rte_mbuf *ml; + struct esp_hdr *esph; + struct esp_tail *espt; + char *ph, *pt; + uint64_t *iv; + + hlen = sa->hdr_len + sa->iv_len + sizeof(*esph); + /* calculate padding and tail space required */ + + /* number of bytes to encrypt */ + clen = mb->pkt_len + sizeof(*espt); + clen = RTE_ALIGN_CEIL(clen, sa->pad_align); + + /* pad length + esp tail */ + pdlen = clen - mb->pkt_len; + tlen = pdlen + sa->icv_len; + + /* do append and prepend */ + ml = rte_pktmbuf_lastseg(mb); + if (tlen + sa->aad_len > rte_pktmbuf_tailroom(ml)) + return -ENOSPC; + + /* prepend header */ + ph = rte_pktmbuf_prepend(mb, hlen); + if (ph == NULL) + return -ENOSPC; + + /* append tail */ + pdofs = ml->data_len; + ml->data_len += tlen; + mb->pkt_len += tlen; + pt = rte_pktmbuf_mtod_offset(ml, typeof(pt), pdofs); + + /* copy tunnel pkt header */ + rte_memcpy(ph, sa->hdr, sa->hdr_len); + + /* update original and new ip header fields */ + if (sa->type & RTE_IPSEC_SATP_MODE_TUNLV4) { + struct ipv4_hdr *l3h; + l3h = (struct ipv4_hdr *)(ph + sa->hdr_l3_off); + l3h->packet_id = rte_cpu_to_be_16(sa->sqn); + l3h->total_length = rte_cpu_to_be_16(mb->pkt_len - + sa->hdr_l3_off); + } else { + struct ipv6_hdr *l3h; + l3h = (struct ipv6_hdr *)(ph + sa->hdr_l3_off); + l3h->payload_len = rte_cpu_to_be_16(mb->pkt_len - + sa->hdr_l3_off - sizeof(*l3h)); + } + + /* update spi, seqn and iv */ + esph = (struct esp_hdr *)(ph + sa->hdr_len); + iv = (uint64_t *)(esph + 1); + + esph->spi = sa->spi; + esph->seq = rte_cpu_to_be_32(sa->sqn); + rte_memcpy(iv, sa->iv_ptr, sa->iv_len); + + /* offset for ICV */ + pdofs += pdlen; + + /* pad length */ + pdlen -= sizeof(*espt); + + /* copy padding data */ + rte_memcpy(pt, esp_pad_bytes, pdlen); + + /* update esp trailer */ + espt = (struct esp_tail *)(pt + pdlen); + espt->pad_len = pdlen; + espt->next_proto = sa->proto; + + /* !!! fill aad fields, if any (aad fields are placed after icv */ + + icv->va = rte_pktmbuf_mtod_offset(ml, void *, pdofs); + icv->pa = rte_pktmbuf_iova_offset(ml, pdofs); + + return clen; +} + +static inline uint32_t +esn_outb_check_sqn(struct rte_ipsec_sa *sa, uint32_t num) +{ + RTE_SET_USED(sa); + return num; +} + +static inline int +esn_inb_check_sqn(struct rte_ipsec_sa *sa, uint32_t sqn) +{ + RTE_SET_USED(sa); + RTE_SET_USED(sqn); + return 0; +} + +static inline uint16_t +esp_outb_tun_prepare(struct rte_ipsec_sa *sa, struct rte_mbuf *mb[], + struct rte_crypto_op *cop[], uint16_t num) +{ + int32_t rc; + uint32_t i, n; + union sym_op_data icv; + + n = esn_outb_check_sqn(sa, num); + + for (i = 0; i != n; i++) { + + sa->sqn++; + sa->iv.v8 = rte_cpu_to_be_64(sa->sqn); + + /* update the packet itself */ + rc = esp_outb_tun_pkt_prepare(sa, mb[i], &icv); + if (rc < 0) { + rte_errno = -rc; + break; + } + + /* update crypto op */ + esp_outb_tun_cop_prepare(cop[i], sa, mb[i], &icv, rc); + } + + return i; +} + +static inline int32_t +esp_inb_tun_cop_prepare(struct rte_crypto_op *cop, + const struct rte_ipsec_sa *sa, struct rte_mbuf *mb, + const union sym_op_data *icv, uint32_t pofs, uint32_t plen) +{ + struct rte_crypto_sym_op *sop; + uint64_t *ivc, *ivp; + uint32_t clen; + + clen = plen - sa->ctp.cipher.length; + if ((int32_t)clen < 0 || (clen & (sa->pad_align - 1)) != 0) + return -EINVAL; + + cop->type = sa->ctp.type; + cop->status = sa->ctp.status; + cop->sess_type = sa->ctp.sess_type; + + sop = cop->sym; + + /* fill sym op fields */ + sop->session = (void *)(uintptr_t)&sa->session.crypto; + sop->m_src = mb; + + sop->cipher.data.offset = pofs + sa->ctp.cipher.offset; + sop->cipher.data.length = clen; + sop->auth.data.offset = pofs + sa->ctp.auth.offset; + sop->auth.data.length = plen - sa->ctp.auth.length; + sop->auth.digest.data = icv->va; + sop->auth.digest.phys_addr = icv->pa; + + /* !!! fill sym op aead fields */ + + /* copy iv from the input packet to the cop */ + ivc = (uint64_t *)(sop + 1); + ivp = rte_pktmbuf_mtod_offset(mb, uint64_t *, + pofs + sizeof(struct esp_hdr)); + rte_memcpy(ivc, ivp, sa->iv_len); + return 0; +} + +static inline int32_t +esp_inb_tun_pkt_prepare(const struct rte_ipsec_sa *sa, struct rte_mbuf *mb, + uint32_t hlen, union sym_op_data *icv) +{ + struct rte_mbuf *ml; + uint32_t icv_ofs, plen; + + plen = mb->pkt_len; + plen = plen - hlen; + + ml = rte_pktmbuf_lastseg(mb); + icv_ofs = ml->data_len - sa->icv_len; + + icv->va = rte_pktmbuf_mtod_offset(ml, void *, icv_ofs); + icv->pa = rte_pktmbuf_iova_offset(ml, icv_ofs); + + return plen; +} + +static inline uint16_t +esp_inb_tun_prepare(struct rte_ipsec_sa *sa, struct rte_mbuf *mb[], + struct rte_crypto_op *cop[], uint16_t num) +{ + int32_t rc; + uint32_t i, hl; + union sym_op_data icv; + + for (i = 0; i != num; i++) { + + hl = mb[i]->l2_len + mb[i]->l3_len; + rc = esp_inb_tun_pkt_prepare(sa, mb[i], hl, &icv); + + if (rc >= 0) + rc = esp_inb_tun_cop_prepare(cop[i], sa, mb[i], &icv, + hl, rc); + if (rc < 0) { + rte_errno = -rc; + break; + } + } + + return i; +} + +uint16_t __rte_experimental +rte_ipsec_crypto_prepare(struct rte_ipsec_sa *sa, struct rte_mbuf *mb[], + struct rte_crypto_op *cop[], uint16_t num) +{ + static const uint64_t msk = RTE_IPSEC_SATP_DIR_MASK | + RTE_IPSEC_SATP_MODE_MASK; + + switch (sa->type & msk) { + case (RTE_IPSEC_SATP_DIR_IB | RTE_IPSEC_SATP_MODE_TUNLV4): + case (RTE_IPSEC_SATP_DIR_IB | RTE_IPSEC_SATP_MODE_TUNLV6): + return esp_inb_tun_prepare(sa, mb, cop, num); + case (RTE_IPSEC_SATP_DIR_OB | RTE_IPSEC_SATP_MODE_TUNLV4): + case (RTE_IPSEC_SATP_DIR_OB | RTE_IPSEC_SATP_MODE_TUNLV6): + return esp_outb_tun_prepare(sa, mb, cop, num); + default: + rte_errno = ENOTSUP; + return 0; + } +} + +/* + * !!! create something more generic (and smarter) + * ideally in librte_mbuf + */ +static inline void +free_mbuf_bulk(struct rte_mbuf *mb[], uint32_t num) +{ + uint32_t i; + + for (i = 0; i != num; i++) + rte_pktmbuf_free(mb[i]); +} + +/* exclude NULLs from the final list of packets. */ +static inline uint32_t +compress_pkt_list(struct rte_mbuf *pkt[], uint32_t nb_pkt, uint32_t nb_zero) +{ + uint32_t i, j, k, l; + + for (j = nb_pkt; nb_zero != 0 && j-- != 0; ) { + + /* found a hole. */ + if (pkt[j] == NULL) { + + /* find how big is it. */ + for (i = j; i-- != 0 && pkt[i] == NULL; ) + ; + /* fill the hole. */ + for (k = j + 1, l = i + 1; k != nb_pkt; k++, l++) + pkt[l] = pkt[k]; + + nb_pkt -= j - i; + nb_zero -= j - i; + j = i + 1; + } + } + + return nb_pkt; +} + +static inline int +esp_inb_tun_single_pkt_process(struct rte_ipsec_sa *sa, struct rte_mbuf *mb, + uint32_t icv_len) +{ + uint32_t hlen, tlen; + struct esp_hdr *esph; + struct esp_tail *espt; + struct rte_mbuf *ml; + char *pd; + + ml = rte_pktmbuf_lastseg(mb); + espt = rte_pktmbuf_mtod_offset(ml, struct esp_tail *, + ml->data_len - icv_len - sizeof(*espt)); + + /* cut of ICV, ESP tail and padding bytes */ + tlen = icv_len + sizeof(*espt) + espt->pad_len; + ml->data_len -= tlen; + mb->pkt_len -= tlen; + + /* cut of L2/L3 headers, ESP header and IV */ + hlen = mb->l2_len + mb->l3_len; + esph = rte_pktmbuf_mtod_offset(mb, struct esp_hdr *, hlen); + rte_pktmbuf_adj(mb, hlen + sa->ctp.cipher.offset); + + /* reset mbuf metatdata: L2/L3 len, packet type */ + mb->packet_type = RTE_PTYPE_UNKNOWN; + mb->l2_len = 0; + mb->l3_len = 0; + + /* clear the PKT_RX_SEC_OFFLOAD flag if set */ + mb->ol_flags &= ~(mb->ol_flags & PKT_RX_SEC_OFFLOAD); + + /* + * check spi, sqn, padding and next proto. + * drop packet if something is wrong. + * ??? consider move spi and sqn check to prepare. + */ + + pd = (char *)espt - espt->pad_len; + if (esph->spi != sa->spi || + esn_inb_check_sqn(sa, esph->seq) != 0 || + espt->next_proto != sa->proto || + memcmp(pd, esp_pad_bytes, espt->pad_len)) + return -EINVAL; + + return 0; +} + +static inline uint16_t +esp_inb_tun_pkt_process(struct rte_ipsec_sa *sa, struct rte_mbuf *mb[], + struct rte_mbuf *dr[], uint16_t num) +{ + uint32_t i, k, icv_len; + + icv_len = sa->icv_len; + + k = 0; + for (i = 0; i != num; i++) { + if (esp_inb_tun_single_pkt_process(sa, mb[i], icv_len)) { + dr[k++] = mb[i]; + mb[i] = NULL; + } + } + + if (k != 0) + compress_pkt_list(mb, num, k); + + return num - k; +} + +static inline uint16_t +esp_pkt_process(struct rte_ipsec_sa *sa, struct rte_mbuf *mb[], + struct rte_mbuf *dr[], uint16_t num) +{ + static const uint64_t msk = RTE_IPSEC_SATP_DIR_MASK | + RTE_IPSEC_SATP_MODE_MASK; + + switch (sa->type & msk) { + case (RTE_IPSEC_SATP_DIR_IB | RTE_IPSEC_SATP_MODE_TUNLV4): + case (RTE_IPSEC_SATP_DIR_IB | RTE_IPSEC_SATP_MODE_TUNLV6): + return esp_inb_tun_pkt_process(sa, mb, dr, num); + case (RTE_IPSEC_SATP_DIR_OB | RTE_IPSEC_SATP_MODE_TUNLV4): + case (RTE_IPSEC_SATP_DIR_OB | RTE_IPSEC_SATP_MODE_TUNLV6): + return num; + default: + return 0; + } +} + +static inline uint16_t +esp_process(const struct rte_crypto_op *cop[], struct rte_mbuf *mb[], + struct rte_ipsec_group grp[], uint16_t num) +{ + uint32_t cnt, i, j, k, n; + uintptr_t ns, ps; + struct rte_ipsec_sa *sa; + struct rte_mbuf *m, *dr[num]; + + j = 0; + k = 0; + n = 0; + ps = 0; + + for (i = 0; i != num; i++) { + + m = cop[i]->sym[0].m_src; + ns = (uintptr_t)cop[i]->sym[0].session; + + if (cop[i]->status != RTE_CRYPTO_OP_STATUS_SUCCESS) { + dr[k++] = m; + continue; + } + + if (ps != ns) { + + if (ps != 0) { + sa = cses2sa(ps); + + /* setup count for current group */ + grp[n].cnt = mb + j - grp[n].m; + + /* do SA type specific processing */ + cnt = esp_pkt_process(sa, grp[n].m, dr + k, + grp[n].cnt); + + /* some packets were dropped */ + cnt = grp[n].cnt - cnt; + if (cnt != 0) { + grp[n].cnt -= cnt; + j -= cnt; + k += cnt; + } + + /* open new group */ + n++; + } + + grp[n].flowid = cses2sa(ns)->flowid; + grp[n].m = mb + j; + ps = ns; + } + + mb[j++] = m; + } + + if (ps != 0) { + sa = cses2sa(ps); + grp[n].cnt = mb + j - grp[n].m; + cnt = esp_pkt_process(sa, grp[n].m, dr + k, grp[n].cnt); + cnt = grp[n].cnt - cnt; + if (cnt != 0) { + grp[n].cnt -= cnt; + j -= cnt; + k += cnt; + } + n++; + } + + if (k != 0) + free_mbuf_bulk(dr, k); + + return n; +} + +uint16_t __rte_experimental +rte_ipsec_crypto_process(const struct rte_crypto_op *cop[], + struct rte_mbuf *mb[], struct rte_ipsec_group grp[], uint16_t num) +{ + return esp_process(cop, mb, grp, num); +} + +static inline uint16_t +inline_outb_tun_pkt_process(struct rte_ipsec_sa *sa, struct rte_mbuf *mb[], + uint16_t num) +{ + uint32_t i; + struct rte_mbuf *m; + int rc; + union sym_op_data icv; + + for (i = 0; i != num; i++) { + m = mb[i]; + + sa->sqn++; + sa->iv.v8 = rte_cpu_to_be_64(sa->sqn); + + /* update the packet itself */ + rc = esp_outb_tun_pkt_prepare(sa, m, &icv); + if (rc < 0) { + rte_errno = -rc; + break; + } + + m->ol_flags |= PKT_TX_SEC_OFFLOAD; + + if (sa->session.ol_flags & RTE_SECURITY_TX_OLOAD_NEED_MDATA) + rte_security_set_pkt_metadata(sa->session.sctx, + sa->session.sec, m, NULL); + } + + return i; +} + +static inline uint16_t +inline_inb_tun_pkt_process(struct rte_ipsec_sa *sa, struct rte_mbuf *mb[], + uint16_t num) +{ + uint32_t i, icv_len; + int rc; + + icv_len = sa->icv_len; + + for (i = 0; i != num; i++) { + rc = esp_inb_tun_single_pkt_process(sa, mb[i], icv_len); + if (rc != 0) { + rte_errno = -rc; + break; + } + } + + return i; +} + +uint16_t __rte_experimental +rte_ipsec_inline_process(struct rte_ipsec_sa *sa, struct rte_mbuf *mb[], + uint16_t num) +{ + static const uint64_t msk = RTE_IPSEC_SATP_DIR_MASK | + RTE_IPSEC_SATP_MODE_MASK; + + switch (sa->type & msk) { + case (RTE_IPSEC_SATP_DIR_IB | RTE_IPSEC_SATP_MODE_TUNLV4): + case (RTE_IPSEC_SATP_DIR_IB | RTE_IPSEC_SATP_MODE_TUNLV6): + return inline_inb_tun_pkt_process(sa, mb, num); + case (RTE_IPSEC_SATP_DIR_OB | RTE_IPSEC_SATP_MODE_TUNLV4): + case (RTE_IPSEC_SATP_DIR_OB | RTE_IPSEC_SATP_MODE_TUNLV6): + return inline_outb_tun_pkt_process(sa, mb, num); + default: + rte_errno = ENOTSUP; + } + + return 0; +} diff --git a/lib/librte_net/rte_esp.h b/lib/librte_net/rte_esp.h index f77ec2eb2..8e1b3d2dd 100644 --- a/lib/librte_net/rte_esp.h +++ b/lib/librte_net/rte_esp.h @@ -11,7 +11,7 @@ * ESP-related defines */ -#include +#include #ifdef __cplusplus extern "C" { @@ -25,6 +25,14 @@ struct esp_hdr { rte_be32_t seq; /**< packet sequence number */ } __attribute__((__packed__)); +/** + * ESP Trailer + */ +struct esp_tail { + uint8_t pad_len; /**< number of pad bytes (0-255) */ + uint8_t next_proto; /**< IPv4 or IPv6 or next layer header */ +} __attribute__((__packed__)); + #ifdef __cplusplus } #endif diff --git a/lib/meson.build b/lib/meson.build index eb91f100b..bb07e67bd 100644 --- a/lib/meson.build +++ b/lib/meson.build @@ -21,6 +21,8 @@ libraries = [ 'compat', # just a header, used for versioning 'kni', 'latencystats', 'lpm', 'member', 'meter', 'power', 'pdump', 'rawdev', 'reorder', 'sched', 'security', 'vhost', + # ipsec lib depends on crypto and security + 'ipsec', # add pkt framework libs which use other libs from above 'port', 'table', 'pipeline', # flow_classify lib depends on pkt framework table lib diff --git a/mk/rte.app.mk b/mk/rte.app.mk index de33883be..7f4344ecd 100644 --- a/mk/rte.app.mk +++ b/mk/rte.app.mk @@ -62,6 +62,8 @@ ifeq ($(CONFIG_RTE_LIBRTE_BPF_ELF),y) _LDLIBS-$(CONFIG_RTE_LIBRTE_BPF) += -lelf endif +_LDLIBS-$(CONFIG_RTE_LIBRTE_IPSEC) += -lrte_ipsec + _LDLIBS-y += --whole-archive _LDLIBS-$(CONFIG_RTE_LIBRTE_CFGFILE) += -lrte_cfgfile -- 2.13.6