From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from dpdk.org (dpdk.org [92.243.14.124]) by inbox.dpdk.org (Postfix) with ESMTP id 0B2C8A0613 for ; Fri, 27 Sep 2019 01:20:18 +0200 (CEST) Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id 62A671DBD; Fri, 27 Sep 2019 01:20:16 +0200 (CEST) Received: from mga05.intel.com (mga05.intel.com [192.55.52.43]) by dpdk.org (Postfix) with ESMTP id 3E0D21DB9 for ; Fri, 27 Sep 2019 01:20:14 +0200 (CEST) X-Amp-Result: SKIPPED(no attachment in message) X-Amp-File-Uploaded: False Received: from orsmga008.jf.intel.com ([10.7.209.65]) by fmsmga105.fm.intel.com with ESMTP/TLS/DHE-RSA-AES256-GCM-SHA384; 26 Sep 2019 16:20:12 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.64,553,1559545200"; d="scan'208";a="183791422" Received: from irsmsx103.ger.corp.intel.com ([163.33.3.157]) by orsmga008.jf.intel.com with ESMTP; 26 Sep 2019 16:20:11 -0700 Received: from irsmsx105.ger.corp.intel.com ([169.254.7.164]) by IRSMSX103.ger.corp.intel.com ([169.254.3.36]) with mapi id 14.03.0439.000; Fri, 27 Sep 2019 00:20:10 +0100 From: "Ananyev, Konstantin" To: "Zhang, Roy Fan" , "dev@dpdk.org" CC: "Doherty, Declan" , "akhil.goyal@nxp.com" Thread-Topic: [PATCH 08/10] ipsec: add rte_security cpu_crypto action support Thread-Index: AQHVZLTz1xBXZzw+5EG4S9a5D69UjKc+KqyQ Date: Thu, 26 Sep 2019 23:20:09 +0000 Message-ID: <2601191342CEEE43887BDE71AB977258019196B6CC@irsmsx105.ger.corp.intel.com> References: <20190903154046.55992-1-roy.fan.zhang@intel.com> <20190906131330.40185-1-roy.fan.zhang@intel.com> <20190906131330.40185-9-roy.fan.zhang@intel.com> In-Reply-To: <20190906131330.40185-9-roy.fan.zhang@intel.com> Accept-Language: en-IE, en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-titus-metadata-40: eyJDYXRlZ29yeUxhYmVscyI6IiIsIk1ldGFkYXRhIjp7Im5zIjoiaHR0cDpcL1wvd3d3LnRpdHVzLmNvbVwvbnNcL0ludGVsMyIsImlkIjoiMzI2ZTBiMzYtMjU0Zi00ZDY4LTk1N2UtMWQ0ZDhhMDViYWQxIiwicHJvcHMiOlt7Im4iOiJDVFBDbGFzc2lmaWNhdGlvbiIsInZhbHMiOlt7InZhbHVlIjoiQ1RQX05UIn1dfV19LCJTdWJqZWN0TGFiZWxzIjpbXSwiVE1DVmVyc2lvbiI6IjE3LjEwLjE4MDQuNDkiLCJUcnVzdGVkTGFiZWxIYXNoIjoiN3dGNVg1OGdlSFNYbGMwazR0RDJRNHpCYXUrTW1QWXloUWdLVmdpOEdZeVBLMHRnbkZcL1N1UWMyZXdBSTF1d2oifQ== x-ctpclassification: CTP_NT dlp-product: dlpe-windows dlp-version: 11.2.0.6 dlp-reaction: no-action x-originating-ip: [163.33.239.182] Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Subject: Re: [dpdk-dev] [PATCH 08/10] ipsec: add rte_security cpu_crypto action support X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" Hi Fan, ... > diff --git a/lib/librte_ipsec/esp_outb.c b/lib/librte_ipsec/esp_outb.c > index 55799a867..097cb663f 100644 > --- a/lib/librte_ipsec/esp_outb.c > +++ b/lib/librte_ipsec/esp_outb.c > @@ -403,6 +403,292 @@ esp_outb_trs_prepare(const struct rte_ipsec_session= *ss, struct rte_mbuf *mb[], > return k; > } >=20 > + > +static inline int > +outb_sync_crypto_proc_prepare(struct rte_mbuf *m, const struct rte_ipsec= _sa *sa, > + const uint64_t ivp[IPSEC_MAX_IV_QWORD], > + const union sym_op_data *icv, uint32_t hlen, uint32_t plen, > + struct rte_security_vec *buf, struct iovec *cur_vec, void *iv, > + void **aad, void **digest) > +{ > + struct rte_mbuf *ms; > + struct aead_gcm_iv *gcm; > + struct aesctr_cnt_blk *ctr; > + struct iovec *vec =3D cur_vec; > + uint32_t left, off =3D 0, n_seg =3D 0; Please separate variable definition and value assignment. It makes it hard to read, plus we don't do that in the rest of the library, so better to follow rest of the code style.=20 > + uint32_t algo; > + > + algo =3D sa->algo_type; > + > + switch (algo) { > + case ALGO_TYPE_AES_GCM: > + gcm =3D iv; > + aead_gcm_iv_fill(gcm, ivp[0], sa->salt); > + *aad =3D (void *)(icv->va + sa->icv_len); Why do we want to allocate aad inside the packet at all? Why not just to do that on the stack instead? In that case you probably wouldn't need this icv stuff at all to be passed = to that function. > + off =3D sa->ctp.cipher.offset + hlen; > + break; > + case ALGO_TYPE_AES_CBC: > + case ALGO_TYPE_3DES_CBC: > + off =3D sa->ctp.auth.offset + hlen; > + break; > + case ALGO_TYPE_AES_CTR: > + ctr =3D iv; > + aes_ctr_cnt_blk_fill(ctr, ivp[0], sa->salt); > + break; > + case ALGO_TYPE_NULL: > + break; For latest two, why off is zero? Shouldn't it at least be 'hlen'? In fact, I think it needs to be: sa->ctp.auth.offset + hlen; > + } > + > + *digest =3D (void *)icv->va; Could be done in the upper layer function, together with aad assignment, I = think. Looking at this function, it seems to consist of 2 separate parts: 1. calculates offset and generates iv 2. setup iovec[]. Probably worth to split it into 2 separate functions like that. Would be much easier to read/understand. > + > + left =3D sa->ctp.cipher.length + plen; > + > + ms =3D mbuf_get_seg_ofs(m, &off); > + if (!ms) > + return -1; outb_tun_pkt_prepare() should already check that we have a valid packet. I don't think there is a need to check for any failure here. Another thing, our esp header will be in the first segment for sure, so do we need get_seg_ofs() here at all?=20 > + > + while (n_seg < RTE_LIBRTE_IP_FRAG_MAX_FRAG && left && ms) { I don't think this is right, we shouldn't impose additional limitations to the number of segments in the packet. > + uint32_t len =3D RTE_MIN(left, ms->data_len - off); > + > + vec->iov_base =3D rte_pktmbuf_mtod_offset(ms, void *, off); > + vec->iov_len =3D len; > + > + left -=3D len; > + vec++; > + n_seg++; > + ms =3D ms->next; > + off =3D 0; Whole construction seems a bit over-complicated here... Why just not have a separate function that would dill iovec[] from mbuf And return an error if there is not enough iovec[] entries? Something like: static inline int mbuf_to_iovec(const struct rte_mbuf *mb, uint32_t ofs, uint32_t len, struct= iovec vec[], uint32_t num) { uint32_t i; if (mb->nb_seg > num) return - mb->nb_seg; vec[0].iov_base =3D rte_pktmbuf_mtod_offset(mb, void *, off); vec[0].iov_len =3D mb->data_len - off; for (i =3D 1, ms =3D mb->next; mb !=3D NULL; ms =3D ms->next, i++) { vec[i].iov_base =3D rte_pktmbuf_mtod(ms); vec[i].iov_len =3D ms->data_len; } vec[i].iov_len -=3D mb->pkt_len - len; return i; } Then we can use that function to fill our iovec[] in a loop. > + } > + > + if (left) > + return -1; > + > + buf->vec =3D cur_vec; > + buf->num =3D n_seg; > + > + return n_seg; > +} > + > +/** > + * Local post process function prototype that same as process function p= rototype > + * as rte_ipsec_sa_pkt_func's process(). > + */ > +typedef uint16_t (*sync_crypto_post_process)(const struct rte_ipsec_sess= ion *ss, > + struct rte_mbuf *mb[], > + uint16_t num); Stylish thing: typdef newtype_t .... > +static uint16_t > +esp_outb_tun_sync_crypto_process(const struct rte_ipsec_session *ss, > + struct rte_mbuf *mb[], uint16_t num, > + sync_crypto_post_process post_process) > +{ > + uint64_t sqn; > + rte_be64_t sqc; > + struct rte_ipsec_sa *sa; > + struct rte_security_ctx *ctx; > + struct rte_security_session *rss; > + union sym_op_data icv; > + struct rte_security_vec buf[num]; > + struct iovec vec[RTE_LIBRTE_IP_FRAG_MAX_FRAG * num]; > + uint32_t vec_idx =3D 0; > + void *aad[num]; > + void *digest[num]; > + void *iv[num]; > + uint8_t ivs[num][IPSEC_MAX_IV_SIZE]; > + uint64_t ivp[IPSEC_MAX_IV_QWORD]; Why do we need both ivs and ivp? > + int status[num]; > + uint32_t dr[num]; > + uint32_t i, n, k; > + int32_t rc; > + > + sa =3D ss->sa; > + ctx =3D ss->security.ctx; > + rss =3D ss->security.ses; > + > + k =3D 0; > + n =3D num; > + sqn =3D esn_outb_update_sqn(sa, &n); > + if (n !=3D num) > + rte_errno =3D EOVERFLOW; > + > + for (i =3D 0; i !=3D n; i++) { > + sqc =3D rte_cpu_to_be_64(sqn + i); > + gen_iv(ivp, sqc); > + > + /* try to update the packet itself */ > + rc =3D outb_tun_pkt_prepare(sa, sqc, ivp, mb[i], &icv, > + sa->sqh_len); > + > + /* success, setup crypto op */ > + if (rc >=3D 0) { > + outb_pkt_xprepare(sa, sqc, &icv); We probably need something like outb_pkt_sync_xprepare(sa, sqc, &aad[i]); h= ere. To avoid using space in the packet for aad. > + > + iv[k] =3D (void *)ivs[k]; Do we really need type conversion here? > + rc =3D outb_sync_crypto_proc_prepare(mb[i], sa, ivp, &icv, > + 0, rc, &buf[k], &vec[vec_idx], iv[k], > + &aad[k], &digest[k]); > + if (rc < 0) { > + dr[i - k] =3D i; > + rte_errno =3D -rc; > + continue; > + } > + > + vec_idx +=3D rc; > + k++; > + /* failure, put packet into the death-row */ > + } else { > + dr[i - k] =3D i; > + rte_errno =3D -rc; > + } > + } > + > + /* copy not prepared mbufs beyond good ones */ > + if (k !=3D n && k !=3D 0) > + move_bad_mbufs(mb, dr, n, n - k); > + > + if (unlikely(k =3D=3D 0)) { I don't think 'unlikely' will make any difference here here. > + rte_errno =3D EBADMSG; > + return 0; > + } > + > + /* process the packets */ > + n =3D 0; > + rte_security_process_cpu_crypto_bulk(ctx, rss, buf, iv, aad, digest, > + status, k); Looking at the code below, I think it will be plausible to make=20 rte_security_process_cpu_crypto_bulk() to return number of failures (or number of succese). > + /* move failed process packets to dr */ > + for (i =3D 0; i < n; i++) { That loop will never be executed. Should be i < k. > + if (status[i]) > + dr[n++] =3D i; Forgot to set rte_errno. > + } > + > + if (n) if (n !=3D 0 && n !=3D k) > + move_bad_mbufs(mb, dr, k, n); > + > + return post_process(ss, mb, k - n); > +} > + > +static uint16_t > +esp_outb_trs_sync_crypto_process(const struct rte_ipsec_session *ss, > + struct rte_mbuf *mb[], uint16_t num, > + sync_crypto_post_process post_process) > + > +{ > + uint64_t sqn; > + rte_be64_t sqc; > + struct rte_ipsec_sa *sa; > + struct rte_security_ctx *ctx; > + struct rte_security_session *rss; > + union sym_op_data icv; > + struct rte_security_vec buf[num]; > + struct iovec vec[RTE_LIBRTE_IP_FRAG_MAX_FRAG * num]; > + uint32_t vec_idx =3D 0; > + void *aad[num]; > + void *digest[num]; > + uint8_t ivs[num][IPSEC_MAX_IV_SIZE]; > + void *iv[num]; > + int status[num]; > + uint64_t ivp[IPSEC_MAX_IV_QWORD]; > + uint32_t dr[num]; > + uint32_t i, n, k; > + uint32_t l2, l3; > + int32_t rc; > + > + sa =3D ss->sa; > + ctx =3D ss->security.ctx; > + rss =3D ss->security.ses; > + > + k =3D 0; > + n =3D num; > + sqn =3D esn_outb_update_sqn(sa, &n); > + if (n !=3D num) > + rte_errno =3D EOVERFLOW; > + > + for (i =3D 0; i !=3D n; i++) { > + l2 =3D mb[i]->l2_len; > + l3 =3D mb[i]->l3_len; > + > + sqc =3D rte_cpu_to_be_64(sqn + i); > + gen_iv(ivp, sqc); > + > + /* try to update the packet itself */ > + rc =3D outb_trs_pkt_prepare(sa, sqc, ivp, mb[i], l2, l3, &icv, > + sa->sqh_len); > + > + /* success, setup crypto op */ > + if (rc >=3D 0) { > + outb_pkt_xprepare(sa, sqc, &icv); > + > + iv[k] =3D (void *)ivs[k]; > + > + rc =3D outb_sync_crypto_proc_prepare(mb[i], sa, ivp, &icv, > + l2 + l3, rc, &buf[k], &vec[vec_idx], > + iv[k], &aad[k], &digest[k]); > + if (rc < 0) { > + dr[i - k] =3D i; > + rte_errno =3D -rc; > + continue; > + } > + > + vec_idx +=3D rc; > + k++; > + /* failure, put packet into the death-row */ > + } else { > + dr[i - k] =3D i; > + rte_errno =3D -rc; > + } > + } > + > + /* copy not prepared mbufs beyond good ones */ > + if (k !=3D n && k !=3D 0) > + move_bad_mbufs(mb, dr, n, n - k); You don't really need to do it here. Just one such thing at the very end should be enough. > + > + /* process the packets */ > + n =3D 0; > + rte_security_process_cpu_crypto_bulk(ctx, rss, buf, iv, aad, digest, > + status, k); > + /* move failed process packets to dr */ > + for (i =3D 0; i < k; i++) { > + if (status[i]) > + dr[n++] =3D i; > + } > + > + if (n) > + move_bad_mbufs(mb, dr, k, n); > + > + return post_process(ss, mb, k - n); > +} > + > +uint16_t > +esp_outb_tun_sync_crpyto_sqh_process(const struct rte_ipsec_session *ss, > + struct rte_mbuf *mb[], uint16_t num) > +{ > + return esp_outb_tun_sync_crypto_process(ss, mb, num, > + esp_outb_sqh_process); esp_outb_sqh_process() relies on PKT_RX_SEC_OFFLOAD_FAILED been set in mb->ol_flags for failed packets. At first for _sync_ case no-one will set it for you. Second - for _sync_ you don't really need that it is just an extra overhead= here. So I think you can't reuse this function without some modifications here. Probably easier to make a new one (and extract some common code into another helper function that esp_out_sqh_process and new one can call). > +} > + > +uint16_t > +esp_outb_tun_sync_crpyto_flag_process(const struct rte_ipsec_session *ss= , > + struct rte_mbuf *mb[], uint16_t num) > +{ > + return esp_outb_tun_sync_crypto_process(ss, mb, num, > + esp_outb_pkt_flag_process); Same as above, plus the fact that you made esp_outb_pkt_flag_process() is not a static one, so compiler wouldn't be able to inline it. > +} > + > +uint16_t > +esp_outb_trs_sync_crpyto_sqh_process(const struct rte_ipsec_session *ss, > + struct rte_mbuf *mb[], uint16_t num) > +{ > + return esp_outb_trs_sync_crypto_process(ss, mb, num, > + esp_outb_sqh_process); > +} > + > +uint16_t > +esp_outb_trs_sync_crpyto_flag_process(const struct rte_ipsec_session *ss= , > + struct rte_mbuf *mb[], uint16_t num) > +{ > + return esp_outb_trs_sync_crypto_process(ss, mb, num, > + esp_outb_pkt_flag_process); > +} > + > /* > * process outbound packets for SA with ESN support, > * for algorithms that require SQN.hibits to be implictly included > @@ -410,8 +696,8 @@ esp_outb_trs_prepare(const struct rte_ipsec_session *= ss, struct rte_mbuf *mb[], > * In that case we have to move ICV bytes back to their proper place. > */ > uint16_t > -esp_outb_sqh_process(const struct rte_ipsec_session *ss, struct rte_mbuf= *mb[], > - uint16_t num) > +esp_outb_sqh_process(const struct rte_ipsec_session *ss, > + struct rte_mbuf *mb[], uint16_t num) Any purpose for that change? > { > uint32_t i, k, icv_len, *icv; > struct rte_mbuf *ml; > diff --git a/lib/librte_ipsec/sa.c b/lib/librte_ipsec/sa.c > index 23d394b46..31ffbce2c 100644 > --- a/lib/librte_ipsec/sa.c > +++ b/lib/librte_ipsec/sa.c > @@ -544,9 +544,9 @@ lksd_proto_prepare(const struct rte_ipsec_session *ss= , > * - inbound/outbound for RTE_SECURITY_ACTION_TYPE_LOOKASIDE_PROTOCOL > * - outbound for RTE_SECURITY_ACTION_TYPE_NONE when ESN is disabled > */ > -static uint16_t > -pkt_flag_process(const struct rte_ipsec_session *ss, struct rte_mbuf *mb= [], > - uint16_t num) > +uint16_t > +esp_outb_pkt_flag_process(const struct rte_ipsec_session *ss, > + struct rte_mbuf *mb[], uint16_t num) Why to rename this function? As comment above it states, the function is used for both inbound and outbo= und code path.=20 Such renaming seems misleading to me. > { > uint32_t i, k; > uint32_t dr[num]; > @@ -599,12 +599,48 @@ lksd_none_pkt_func_select(const struct rte_ipsec_sa= *sa, > case (RTE_IPSEC_SATP_DIR_OB | RTE_IPSEC_SATP_MODE_TUNLV6): > pf->prepare =3D esp_outb_tun_prepare; > pf->process =3D (sa->sqh_len !=3D 0) ? > - esp_outb_sqh_process : pkt_flag_process; > + esp_outb_sqh_process : esp_outb_pkt_flag_process; > break; > case (RTE_IPSEC_SATP_DIR_OB | RTE_IPSEC_SATP_MODE_TRANS): > pf->prepare =3D esp_outb_trs_prepare; > pf->process =3D (sa->sqh_len !=3D 0) ? > - esp_outb_sqh_process : pkt_flag_process; > + esp_outb_sqh_process : esp_outb_pkt_flag_process; > + break; > + default: > + rc =3D -ENOTSUP; > + } > + > + return rc; > +} > + > +static int > +lksd_sync_crypto_pkt_func_select(const struct rte_ipsec_sa *sa, > + struct rte_ipsec_sa_pkt_func *pf) As a nit: probably no point to have lksd_prefix for _sync_ functions. > +{ > + int32_t rc; > + > + static const uint64_t msk =3D RTE_IPSEC_SATP_DIR_MASK | > + RTE_IPSEC_SATP_MODE_MASK; > + > + rc =3D 0; > + switch (sa->type & msk) { > + case (RTE_IPSEC_SATP_DIR_IB | RTE_IPSEC_SATP_MODE_TUNLV4): > + case (RTE_IPSEC_SATP_DIR_IB | RTE_IPSEC_SATP_MODE_TUNLV6): > + pf->process =3D esp_inb_tun_sync_crypto_pkt_process; > + break; > + case (RTE_IPSEC_SATP_DIR_IB | RTE_IPSEC_SATP_MODE_TRANS): > + pf->process =3D esp_inb_trs_sync_crypto_pkt_process; > + break; > + case (RTE_IPSEC_SATP_DIR_OB | RTE_IPSEC_SATP_MODE_TUNLV4): > + case (RTE_IPSEC_SATP_DIR_OB | RTE_IPSEC_SATP_MODE_TUNLV6): > + pf->process =3D (sa->sqh_len !=3D 0) ? > + esp_outb_tun_sync_crpyto_sqh_process : > + esp_outb_tun_sync_crpyto_flag_process; > + break; > + case (RTE_IPSEC_SATP_DIR_OB | RTE_IPSEC_SATP_MODE_TRANS): > + pf->process =3D (sa->sqh_len !=3D 0) ? > + esp_outb_trs_sync_crpyto_sqh_process : > + esp_outb_trs_sync_crpyto_flag_process; > break; > default: > rc =3D -ENOTSUP; > @@ -672,13 +708,16 @@ ipsec_sa_pkt_func_select(const struct rte_ipsec_ses= sion *ss, > case RTE_SECURITY_ACTION_TYPE_INLINE_PROTOCOL: > if ((sa->type & RTE_IPSEC_SATP_DIR_MASK) =3D=3D > RTE_IPSEC_SATP_DIR_IB) > - pf->process =3D pkt_flag_process; > + pf->process =3D esp_outb_pkt_flag_process; > else > pf->process =3D inline_proto_outb_pkt_process; > break; > case RTE_SECURITY_ACTION_TYPE_LOOKASIDE_PROTOCOL: > pf->prepare =3D lksd_proto_prepare; > - pf->process =3D pkt_flag_process; > + pf->process =3D esp_outb_pkt_flag_process; > + break; > + case RTE_SECURITY_ACTION_TYPE_CPU_CRYPTO: > + rc =3D lksd_sync_crypto_pkt_func_select(sa, pf); > break; > default: > rc =3D -ENOTSUP; > diff --git a/lib/librte_ipsec/sa.h b/lib/librte_ipsec/sa.h > index 51e69ad05..02c7abc60 100644 > --- a/lib/librte_ipsec/sa.h > +++ b/lib/librte_ipsec/sa.h > @@ -156,6 +156,14 @@ uint16_t > inline_inb_trs_pkt_process(const struct rte_ipsec_session *ss, > struct rte_mbuf *mb[], uint16_t num); >=20 > +uint16_t > +esp_inb_tun_sync_crypto_pkt_process(const struct rte_ipsec_session *ss, > + struct rte_mbuf *mb[], uint16_t num); > + > +uint16_t > +esp_inb_trs_sync_crypto_pkt_process(const struct rte_ipsec_session *ss, > + struct rte_mbuf *mb[], uint16_t num); > + > /* outbound processing */ >=20 > uint16_t > @@ -170,6 +178,10 @@ uint16_t > esp_outb_sqh_process(const struct rte_ipsec_session *ss, struct rte_mbuf= *mb[], > uint16_t num); >=20 > +uint16_t > +esp_outb_pkt_flag_process(const struct rte_ipsec_session *ss, > + struct rte_mbuf *mb[], uint16_t num); > + > uint16_t > inline_outb_tun_pkt_process(const struct rte_ipsec_session *ss, > struct rte_mbuf *mb[], uint16_t num); > @@ -182,4 +194,21 @@ uint16_t > inline_proto_outb_pkt_process(const struct rte_ipsec_session *ss, > struct rte_mbuf *mb[], uint16_t num); >=20 > +uint16_t > +esp_outb_tun_sync_crpyto_sqh_process(const struct rte_ipsec_session *ss, > + struct rte_mbuf *mb[], uint16_t num); > + > +uint16_t > +esp_outb_tun_sync_crpyto_flag_process(const struct rte_ipsec_session *ss= , > + struct rte_mbuf *mb[], uint16_t num); > + > +uint16_t > +esp_outb_trs_sync_crpyto_sqh_process(const struct rte_ipsec_session *ss, > + struct rte_mbuf *mb[], uint16_t num); > + > +uint16_t > +esp_outb_trs_sync_crpyto_flag_process(const struct rte_ipsec_session *ss= , > + struct rte_mbuf *mb[], uint16_t num); > + > + > #endif /* _SA_H_ */