From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mga01.intel.com (mga01.intel.com [192.55.52.88]) by dpdk.org (Postfix) with ESMTP id A70702B9F for ; Wed, 18 Oct 2017 23:29:41 +0200 (CEST) Received: from orsmga004.jf.intel.com ([10.7.209.38]) by fmsmga101.fm.intel.com with ESMTP/TLS/DHE-RSA-AES256-GCM-SHA384; 18 Oct 2017 14:29:37 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.43,398,1503385200"; d="scan'208";a="139796642" Received: from irsmsx154.ger.corp.intel.com ([163.33.192.96]) by orsmga004.jf.intel.com with ESMTP; 18 Oct 2017 14:29:33 -0700 Received: from irsmsx155.ger.corp.intel.com (163.33.192.3) by IRSMSX154.ger.corp.intel.com (163.33.192.96) with Microsoft SMTP Server (TLS) id 14.3.319.2; Wed, 18 Oct 2017 22:29:32 +0100 Received: from irsmsx103.ger.corp.intel.com ([169.254.3.49]) by irsmsx155.ger.corp.intel.com ([169.254.14.169]) with mapi id 14.03.0319.002; Wed, 18 Oct 2017 22:29:32 +0100 From: "Ananyev, Konstantin" To: Akhil Goyal , "dev@dpdk.org" CC: "Doherty, Declan" , "De Lara Guarch, Pablo" , "hemant.agrawal@nxp.com" , "Nicolau, Radu" , "borisp@mellanox.com" , "aviadye@mellanox.com" , "thomas@monjalon.net" , "sandeep.malik@nxp.com" , "jerin.jacob@caviumnetworks.com" , "Mcnamara, John" , "shahafs@mellanox.com" , "olivier.matz@6wind.com" Thread-Topic: [PATCH v4 10/12] net/ixgbe: enable inline ipsec Thread-Index: AQHTRTrLmmjEr5R+pU+nk9OQ+YL6eqLp8BWg Date: Wed, 18 Oct 2017 21:29:31 +0000 Message-ID: <2601191342CEEE43887BDE71AB9772585FAAAD3E@IRSMSX103.ger.corp.intel.com> References: <20171006181151.4758-1-akhil.goyal@nxp.com> <20171014221734.15511-1-akhil.goyal@nxp.com> <20171014221734.15511-11-akhil.goyal@nxp.com> In-Reply-To: <20171014221734.15511-11-akhil.goyal@nxp.com> Accept-Language: en-IE, en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-titus-metadata-40: eyJDYXRlZ29yeUxhYmVscyI6IiIsIk1ldGFkYXRhIjp7Im5zIjoiaHR0cDpcL1wvd3d3LnRpdHVzLmNvbVwvbnNcL0ludGVsMyIsImlkIjoiZTEwOWY1MTYtNDExZS00NjFlLWJhNDAtZGQyOTg5Zjc3YzMxIiwicHJvcHMiOlt7Im4iOiJDVFBDbGFzc2lmaWNhdGlvbiIsInZhbHMiOlt7InZhbHVlIjoiQ1RQX0lDIn1dfV19LCJTdWJqZWN0TGFiZWxzIjpbXSwiVE1DVmVyc2lvbiI6IjE2LjUuOS4zIiwiVHJ1c3RlZExhYmVsSGFzaCI6IitoeTRHSjhXSHlSc1g5YTN2anBhekJINmUrcUN4Q1VscjNrOVhVSk5BcEE9In0= x-ctpclassification: CTP_IC dlp-product: dlpe-windows dlp-version: 11.0.0.116 dlp-reaction: no-action x-originating-ip: [163.33.239.181] Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Subject: Re: [dpdk-dev] [PATCH v4 10/12] net/ixgbe: enable inline ipsec X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 18 Oct 2017 21:29:42 -0000 Hi Radu, Few comments from me below. Konstantin > -----Original Message----- > From: Akhil Goyal [mailto:akhil.goyal@nxp.com] > Sent: Saturday, October 14, 2017 11:18 PM > To: dev@dpdk.org > Cc: Doherty, Declan ; De Lara Guarch, Pablo ; hemant.agrawal@nxp.com; > Nicolau, Radu ; borisp@mellanox.com; aviadye@mell= anox.com; thomas@monjalon.net; > sandeep.malik@nxp.com; jerin.jacob@caviumnetworks.com; Mcnamara, John ; Ananyev, Konstantin > ; shahafs@mellanox.com; olivier.matz@6wind.= com > Subject: [PATCH v4 10/12] net/ixgbe: enable inline ipsec >=20 > From: Radu Nicolau >=20 > Signed-off-by: Radu Nicolau > Signed-off-by: Declan Doherty > --- > drivers/net/Makefile | 2 +- > drivers/net/ixgbe/Makefile | 2 +- > drivers/net/ixgbe/base/ixgbe_osdep.h | 8 + > drivers/net/ixgbe/ixgbe_ethdev.c | 19 + > drivers/net/ixgbe/ixgbe_ethdev.h | 6 +- > drivers/net/ixgbe/ixgbe_flow.c | 47 +++ > drivers/net/ixgbe/ixgbe_ipsec.c | 744 +++++++++++++++++++++++++++= ++++++ > drivers/net/ixgbe/ixgbe_ipsec.h | 147 +++++++ > drivers/net/ixgbe/ixgbe_rxtx.c | 53 ++- > drivers/net/ixgbe/ixgbe_rxtx.h | 11 +- > drivers/net/ixgbe/ixgbe_rxtx_vec_sse.c | 50 ++- > 11 files changed, 1079 insertions(+), 10 deletions(-) > create mode 100644 drivers/net/ixgbe/ixgbe_ipsec.c > create mode 100644 drivers/net/ixgbe/ixgbe_ipsec.h >=20 > diff --git a/drivers/net/Makefile b/drivers/net/Makefile > index 5d2ad2f..339ff36 100644 > --- a/drivers/net/Makefile > +++ b/drivers/net/Makefile > @@ -68,7 +68,7 @@ DEPDIRS-fm10k =3D $(core-libs) librte_hash > DIRS-$(CONFIG_RTE_LIBRTE_I40E_PMD) +=3D i40e > DEPDIRS-i40e =3D $(core-libs) librte_hash > DIRS-$(CONFIG_RTE_LIBRTE_IXGBE_PMD) +=3D ixgbe > -DEPDIRS-ixgbe =3D $(core-libs) librte_hash > +DEPDIRS-ixgbe =3D $(core-libs) librte_hash librte_security > DIRS-$(CONFIG_RTE_LIBRTE_LIO_PMD) +=3D liquidio > DEPDIRS-liquidio =3D $(core-libs) > DIRS-$(CONFIG_RTE_LIBRTE_MLX4_PMD) +=3D mlx4 > diff --git a/drivers/net/ixgbe/Makefile b/drivers/net/ixgbe/Makefile > index 95c806d..6e963c7 100644 > --- a/drivers/net/ixgbe/Makefile > +++ b/drivers/net/ixgbe/Makefile > @@ -118,11 +118,11 @@ SRCS-$(CONFIG_RTE_IXGBE_INC_VECTOR) +=3D ixgbe_rxtx= _vec_neon.c > else > SRCS-$(CONFIG_RTE_IXGBE_INC_VECTOR) +=3D ixgbe_rxtx_vec_sse.c > endif > - > ifeq ($(CONFIG_RTE_LIBRTE_IXGBE_BYPASS),y) > SRCS-$(CONFIG_RTE_LIBRTE_IXGBE_PMD) +=3D ixgbe_bypass.c > SRCS-$(CONFIG_RTE_LIBRTE_IXGBE_PMD) +=3D ixgbe_82599_bypass.c > endif > +SRCS-$(CONFIG_RTE_LIBRTE_IXGBE_PMD) +=3D ixgbe_ipsec.c > SRCS-$(CONFIG_RTE_LIBRTE_IXGBE_PMD) +=3D rte_pmd_ixgbe.c > SRCS-$(CONFIG_RTE_LIBRTE_IXGBE_PMD) +=3D ixgbe_tm.c >=20 > diff --git a/drivers/net/ixgbe/base/ixgbe_osdep.h b/drivers/net/ixgbe/bas= e/ixgbe_osdep.h > index 4aab278..b132a0f 100644 > --- a/drivers/net/ixgbe/base/ixgbe_osdep.h > +++ b/drivers/net/ixgbe/base/ixgbe_osdep.h > @@ -161,4 +161,12 @@ static inline uint32_t ixgbe_read_addr(volatile void= * addr) > #define IXGBE_WRITE_REG_ARRAY(hw, reg, index, value) \ > IXGBE_PCI_REG_WRITE(IXGBE_PCI_REG_ARRAY_ADDR((hw), (reg), (index)), (va= lue)) >=20 > +#define IXGBE_WRITE_REG_THEN_POLL_MASK(hw, reg, val, mask, poll_ms) \ > +{ \ > + uint32_t cnt =3D poll_ms; \ > + IXGBE_WRITE_REG(hw, (reg), (val)); \ > + while (((IXGBE_READ_REG(hw, (reg))) & (mask)) && (cnt--)) \ > + rte_delay_ms(1); \ > +} > + As you have a macro that consists from multiple statements, you'll need a d= o { ..} while (0) wrapper around it. Though I still suggest to make it an inline function - would be much better= . > #endif /* _IXGBE_OS_H_ */ > diff --git a/drivers/net/ixgbe/ixgbe_ethdev.c b/drivers/net/ixgbe/ixgbe_e= thdev.c > index 14b9c53..fcabd5e 100644 > --- a/drivers/net/ixgbe/ixgbe_ethdev.c > +++ b/drivers/net/ixgbe/ixgbe_ethdev.c > @@ -61,6 +61,7 @@ > #include > #include > #include > +#include >=20 > #include "ixgbe_logs.h" > #include "base/ixgbe_api.h" > @@ -1132,6 +1133,7 @@ eth_ixgbe_dev_init(struct rte_eth_dev *eth_dev) > IXGBE_DEV_PRIVATE_TO_FILTER_INFO(eth_dev->data->dev_private); > struct ixgbe_bw_conf *bw_conf =3D > IXGBE_DEV_PRIVATE_TO_BW_CONF(eth_dev->data->dev_private); > + struct rte_security_ctx *security_instance; > uint32_t ctrl_ext; > uint16_t csum; > int diag, i; > @@ -1139,6 +1141,17 @@ eth_ixgbe_dev_init(struct rte_eth_dev *eth_dev) > PMD_INIT_FUNC_TRACE(); >=20 > eth_dev->dev_ops =3D &ixgbe_eth_dev_ops; > + security_instance =3D rte_malloc("rte_security_instances_ops", > + sizeof(struct rte_security_ctx), 0); > + if (security_instance =3D=3D NULL) > + return -ENOMEM; > + security_instance->state =3D RTE_SECURITY_INSTANCE_VALID; > + security_instance->device =3D (void *)eth_dev; > + security_instance->ops =3D &ixgbe_security_ops; > + security_instance->sess_cnt =3D 0; > + > + eth_dev->data->security_ctx =3D security_instance; > + > eth_dev->rx_pkt_burst =3D &ixgbe_recv_pkts; > eth_dev->tx_pkt_burst =3D &ixgbe_xmit_pkts; > eth_dev->tx_pkt_prepare =3D &ixgbe_prep_pkts; > @@ -1169,6 +1182,7 @@ eth_ixgbe_dev_init(struct rte_eth_dev *eth_dev) >=20 > rte_eth_copy_pci_info(eth_dev, pci_dev); > eth_dev->data->dev_flags |=3D RTE_ETH_DEV_DETACHABLE; > + eth_dev->data->dev_flags |=3D RTE_ETH_DEV_SECURITY; >=20 > /* Vendor and Device ID need to be set before init of shared code */ > hw->device_id =3D pci_dev->id.device_id; > @@ -1401,6 +1415,8 @@ eth_ixgbe_dev_uninit(struct rte_eth_dev *eth_dev) > /* Remove all Traffic Manager configuration */ > ixgbe_tm_conf_uninit(eth_dev); >=20 > + rte_free(eth_dev->data->security_ctx); > + > return 0; > } >=20 > @@ -3695,6 +3711,9 @@ ixgbe_dev_info_get(struct rte_eth_dev *dev, struct = rte_eth_dev_info *dev_info) > hw->mac.type =3D=3D ixgbe_mac_X550EM_a) > dev_info->tx_offload_capa |=3D DEV_TX_OFFLOAD_OUTER_IPV4_CKSUM; >=20 > + dev_info->rx_offload_capa |=3D DEV_RX_OFFLOAD_SECURITY; > + dev_info->tx_offload_capa |=3D DEV_TX_OFFLOAD_SECURITY; > + > dev_info->default_rxconf =3D (struct rte_eth_rxconf) { > .rx_thresh =3D { > .pthresh =3D IXGBE_DEFAULT_RX_PTHRESH, > diff --git a/drivers/net/ixgbe/ixgbe_ethdev.h b/drivers/net/ixgbe/ixgbe_e= thdev.h > index e28c856..f5b52c4 100644 > --- a/drivers/net/ixgbe/ixgbe_ethdev.h > +++ b/drivers/net/ixgbe/ixgbe_ethdev.h > @@ -38,6 +38,7 @@ > #include "base/ixgbe_dcb_82599.h" > #include "base/ixgbe_dcb_82598.h" > #include "ixgbe_bypass.h" > +#include "ixgbe_ipsec.h" > #include > #include > #include > @@ -486,7 +487,7 @@ struct ixgbe_adapter { > struct ixgbe_filter_info filter; > struct ixgbe_l2_tn_info l2_tn; > struct ixgbe_bw_conf bw_conf; > - > + struct ixgbe_ipsec ipsec; > bool rx_bulk_alloc_allowed; > bool rx_vec_allowed; > struct rte_timecounter systime_tc; > @@ -543,6 +544,9 @@ struct ixgbe_adapter { > #define IXGBE_DEV_PRIVATE_TO_TM_CONF(adapter) \ > (&((struct ixgbe_adapter *)adapter)->tm_conf) >=20 > +#define IXGBE_DEV_PRIVATE_TO_IPSEC(adapter)\ > + (&((struct ixgbe_adapter *)adapter)->ipsec) > + > /* > * RX/TX function prototypes > */ > diff --git a/drivers/net/ixgbe/ixgbe_flow.c b/drivers/net/ixgbe/ixgbe_flo= w.c > index 904c146..13c8243 100644 > --- a/drivers/net/ixgbe/ixgbe_flow.c > +++ b/drivers/net/ixgbe/ixgbe_flow.c > @@ -187,6 +187,9 @@ const struct rte_flow_action *next_no_void_action( > * END > * other members in mask and spec should set to 0x00. > * item->last should be NULL. > + * > + * Special case for flow action type RTE_FLOW_ACTION_TYPE_SECURITY. > + * > */ > static int > cons_parse_ntuple_filter(const struct rte_flow_attr *attr, > @@ -226,6 +229,41 @@ cons_parse_ntuple_filter(const struct rte_flow_attr = *attr, > return -rte_errno; > } >=20 > + /** > + * Special case for flow action type RTE_FLOW_ACTION_TYPE_SECURITY > + */ > + act =3D next_no_void_action(actions, NULL); > + if (act->type =3D=3D RTE_FLOW_ACTION_TYPE_SECURITY) { > + const void *conf =3D act->conf; > + /* check if the next not void item is END */ > + act =3D next_no_void_action(actions, act); > + if (act->type !=3D RTE_FLOW_ACTION_TYPE_END) { > + memset(filter, 0, sizeof(struct rte_eth_ntuple_filter)); > + rte_flow_error_set(error, EINVAL, > + RTE_FLOW_ERROR_TYPE_ACTION, > + act, "Not supported action."); > + return -rte_errno; > + } > + > + /* get the IP pattern*/ > + item =3D next_no_void_pattern(pattern, NULL); > + while (item->type !=3D RTE_FLOW_ITEM_TYPE_IPV4 && > + item->type !=3D RTE_FLOW_ITEM_TYPE_IPV6) { > + if (item->last || > + item->type =3D=3D RTE_FLOW_ITEM_TYPE_END) { > + rte_flow_error_set(error, EINVAL, > + RTE_FLOW_ERROR_TYPE_ITEM, > + item, "IP pattern missing."); > + return -rte_errno; > + } > + item =3D next_no_void_pattern(pattern, item); > + } > + > + filter->proto =3D IPPROTO_ESP; > + return ixgbe_crypto_add_ingress_sa_from_flow(conf, item->spec, > + item->type =3D=3D RTE_FLOW_ITEM_TYPE_IPV6); > + } > + > /* the first not void item can be MAC or IPv4 */ > item =3D next_no_void_pattern(pattern, NULL); >=20 > @@ -519,6 +557,10 @@ ixgbe_parse_ntuple_filter(struct rte_eth_dev *dev, > if (ret) > return ret; >=20 > + /* ESP flow not really a flow*/ > + if (filter->proto =3D=3D IPPROTO_ESP) > + return 0; > + > /* Ixgbe doesn't support tcp flags. */ > if (filter->flags & RTE_NTUPLE_FLAGS_TCP_FLAG) { > memset(filter, 0, sizeof(struct rte_eth_ntuple_filter)); > @@ -2758,6 +2800,11 @@ ixgbe_flow_create(struct rte_eth_dev *dev, > memset(&ntuple_filter, 0, sizeof(struct rte_eth_ntuple_filter)); > ret =3D ixgbe_parse_ntuple_filter(dev, attr, pattern, > actions, &ntuple_filter, error); > + > + /* ESP flow not really a flow*/ > + if (ntuple_filter.proto =3D=3D IPPROTO_ESP) > + return flow; > + > if (!ret) { > ret =3D ixgbe_add_del_ntuple_filter(dev, &ntuple_filter, TRUE); > if (!ret) { > diff --git a/drivers/net/ixgbe/ixgbe_ipsec.c b/drivers/net/ixgbe/ixgbe_ip= sec.c > new file mode 100644 > index 0000000..6ace305 > --- /dev/null > +++ b/drivers/net/ixgbe/ixgbe_ipsec.c > @@ -0,0 +1,744 @@ > +/*- > + * BSD LICENSE > + * > + * Copyright(c) 2010-2017 Intel Corporation. All rights reserved. > + * All rights reserved. > + * > + * Redistribution and use in source and binary forms, with or without > + * modification, are permitted provided that the following conditions > + * are met: > + * > + * * Redistributions of source code must retain the above copyright > + * notice, this list of conditions and the following disclaimer. > + * * Redistributions in binary form must reproduce the above copyrig= ht > + * notice, this list of conditions and the following disclaimer in > + * the documentation and/or other materials provided with the > + * distribution. > + * * Neither the name of Intel Corporation nor the names of its > + * contributors may be used to endorse or promote products derived > + * from this software without specific prior written permission. > + * > + * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS > + * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT > + * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS F= OR > + * A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGH= T > + * OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTA= L, > + * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT > + * LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF US= E, > + * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON A= NY > + * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT > + * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE U= SE > + * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE= . > + */ > + > +#include > +#include > +#include > +#include > +#include > +#include > +#include > + > +#include "base/ixgbe_type.h" > +#include "base/ixgbe_api.h" > +#include "ixgbe_ethdev.h" > +#include "ixgbe_ipsec.h" > + > +#define RTE_IXGBE_REGISTER_POLL_WAIT_5_MS 5 > + > +#define IXGBE_WAIT_RREAD \ > + IXGBE_WRITE_REG_THEN_POLL_MASK(hw, IXGBE_IPSRXIDX, reg_val, \ > + IPSRXIDX_READ, RTE_IXGBE_REGISTER_POLL_WAIT_5_MS) > +#define IXGBE_WAIT_RWRITE \ > + IXGBE_WRITE_REG_THEN_POLL_MASK(hw, IXGBE_IPSRXIDX, reg_val, \ > + IPSRXIDX_WRITE, RTE_IXGBE_REGISTER_POLL_WAIT_5_MS) > +#define IXGBE_WAIT_TREAD \ > + IXGBE_WRITE_REG_THEN_POLL_MASK(hw, IXGBE_IPSTXIDX, reg_val, \ > + IPSRXIDX_READ, RTE_IXGBE_REGISTER_POLL_WAIT_5_MS) > +#define IXGBE_WAIT_TWRITE \ > + IXGBE_WRITE_REG_THEN_POLL_MASK(hw, IXGBE_IPSTXIDX, reg_val, \ > + IPSRXIDX_WRITE, RTE_IXGBE_REGISTER_POLL_WAIT_5_MS) > + > +#define CMP_IP(a, b) (\ > + (a).ipv6[0] =3D=3D (b).ipv6[0] && \ > + (a).ipv6[1] =3D=3D (b).ipv6[1] && \ > + (a).ipv6[2] =3D=3D (b).ipv6[2] && \ > + (a).ipv6[3] =3D=3D (b).ipv6[3]) > + > + > +static void > +ixgbe_crypto_clear_ipsec_tables(struct rte_eth_dev *dev) > +{ > + struct ixgbe_hw *hw =3D IXGBE_DEV_PRIVATE_TO_HW(dev->data->dev_private)= ; > + int i =3D 0; > + > + /* clear Rx IP table*/ > + for (i =3D 0; i < IPSEC_MAX_RX_IP_COUNT; i++) { > + uint16_t index =3D i << 3; > + uint32_t reg_val =3D IPSRXIDX_WRITE | IPSRXIDX_TABLE_IP | index; > + IXGBE_WRITE_REG(hw, IXGBE_IPSRXIPADDR(0), 0); > + IXGBE_WRITE_REG(hw, IXGBE_IPSRXIPADDR(1), 0); > + IXGBE_WRITE_REG(hw, IXGBE_IPSRXIPADDR(2), 0); > + IXGBE_WRITE_REG(hw, IXGBE_IPSRXIPADDR(3), 0); > + IXGBE_WAIT_RWRITE; > + } > + > + /* clear Rx SPI and Rx/Tx SA tables*/ > + for (i =3D 0; i < IPSEC_MAX_SA_COUNT; i++) { > + uint32_t index =3D i << 3; > + uint32_t reg_val =3D IPSRXIDX_WRITE | IPSRXIDX_TABLE_SPI | index; > + IXGBE_WRITE_REG(hw, IXGBE_IPSRXSPI, 0); > + IXGBE_WRITE_REG(hw, IXGBE_IPSRXIPIDX, 0); > + IXGBE_WAIT_RWRITE; > + reg_val =3D IPSRXIDX_WRITE | IPSRXIDX_TABLE_KEY | index; > + IXGBE_WRITE_REG(hw, IXGBE_IPSRXKEY(0), 0); > + IXGBE_WRITE_REG(hw, IXGBE_IPSRXKEY(1), 0); > + IXGBE_WRITE_REG(hw, IXGBE_IPSRXKEY(2), 0); > + IXGBE_WRITE_REG(hw, IXGBE_IPSRXKEY(3), 0); > + IXGBE_WRITE_REG(hw, IXGBE_IPSRXSALT, 0); > + IXGBE_WRITE_REG(hw, IXGBE_IPSRXMOD, 0); > + IXGBE_WAIT_RWRITE; > + reg_val =3D IPSRXIDX_WRITE | index; > + IXGBE_WRITE_REG(hw, IXGBE_IPSTXKEY(0), 0); > + IXGBE_WRITE_REG(hw, IXGBE_IPSTXKEY(1), 0); > + IXGBE_WRITE_REG(hw, IXGBE_IPSTXKEY(2), 0); > + IXGBE_WRITE_REG(hw, IXGBE_IPSTXKEY(3), 0); > + IXGBE_WRITE_REG(hw, IXGBE_IPSTXSALT, 0); > + IXGBE_WAIT_TWRITE; > + } > +} > + > +static int > +ixgbe_crypto_add_sa(struct ixgbe_crypto_session *ic_session) > +{ > + struct rte_eth_dev *dev =3D ic_session->dev; > + struct ixgbe_hw *hw =3D IXGBE_DEV_PRIVATE_TO_HW(dev->data->dev_private)= ; > + struct ixgbe_ipsec *priv =3D IXGBE_DEV_PRIVATE_TO_IPSEC( > + dev->data->dev_private); > + uint32_t reg_val; > + int sa_index =3D -1; > + > + if (ic_session->op =3D=3D IXGBE_OP_AUTHENTICATED_DECRYPTION) { > + int i, ip_index =3D -1; > + > + /* Find a match in the IP table*/ > + for (i =3D 0; i < IPSEC_MAX_RX_IP_COUNT; i++) { > + if (CMP_IP(priv->rx_ip_tbl[i].ip, > + ic_session->dst_ip)) { > + ip_index =3D i; > + break; > + } > + } > + /* If no match, find a free entry in the IP table*/ > + if (ip_index < 0) { > + for (i =3D 0; i < IPSEC_MAX_RX_IP_COUNT; i++) { > + if (priv->rx_ip_tbl[i].ref_count =3D=3D 0) { > + ip_index =3D i; > + break; > + } > + } > + } > + > + /* Fail if no match and no free entries*/ > + if (ip_index < 0) { > + PMD_DRV_LOG(ERR, > + "No free entry left in the Rx IP table\n"); > + return -1; > + } > + > + /* Find a free entry in the SA table*/ > + for (i =3D 0; i < IPSEC_MAX_SA_COUNT; i++) { > + if (priv->rx_sa_tbl[i].used =3D=3D 0) { > + sa_index =3D i; > + break; > + } > + } > + /* Fail if no free entries*/ > + if (sa_index < 0) { > + PMD_DRV_LOG(ERR, > + "No free entry left in the Rx SA table\n"); > + return -1; > + } > + > + priv->rx_ip_tbl[ip_index].ip.ipv6[0] =3D > + ic_session->dst_ip.ipv6[0]; > + priv->rx_ip_tbl[ip_index].ip.ipv6[1] =3D > + ic_session->dst_ip.ipv6[1]; > + priv->rx_ip_tbl[ip_index].ip.ipv6[2] =3D > + ic_session->dst_ip.ipv6[2]; > + priv->rx_ip_tbl[ip_index].ip.ipv6[3] =3D > + ic_session->dst_ip.ipv6[3]; > + priv->rx_ip_tbl[ip_index].ref_count++; > + > + priv->rx_sa_tbl[sa_index].spi =3D > + rte_cpu_to_be_32(ic_session->spi); > + priv->rx_sa_tbl[sa_index].ip_index =3D ip_index; > + priv->rx_sa_tbl[sa_index].key[3] =3D > + rte_cpu_to_be_32(*(uint32_t *)&ic_session->key[0]); > + priv->rx_sa_tbl[sa_index].key[2] =3D > + rte_cpu_to_be_32(*(uint32_t *)&ic_session->key[4]); > + priv->rx_sa_tbl[sa_index].key[1] =3D > + rte_cpu_to_be_32(*(uint32_t *)&ic_session->key[8]); > + priv->rx_sa_tbl[sa_index].key[0] =3D > + rte_cpu_to_be_32(*(uint32_t *)&ic_session->key[12]); > + priv->rx_sa_tbl[sa_index].salt =3D > + rte_cpu_to_be_32(ic_session->salt); > + priv->rx_sa_tbl[sa_index].mode =3D IPSRXMOD_VALID; > + if (ic_session->op =3D=3D IXGBE_OP_AUTHENTICATED_DECRYPTION) > + priv->rx_sa_tbl[sa_index].mode |=3D > + (IPSRXMOD_PROTO | IPSRXMOD_DECRYPT); > + if (ic_session->dst_ip.type =3D=3D IPv6) > + priv->rx_sa_tbl[sa_index].mode |=3D IPSRXMOD_IPV6; > + priv->rx_sa_tbl[sa_index].used =3D 1; > + > + /* write IP table entry*/ > + reg_val =3D IPSRXIDX_RX_EN | IPSRXIDX_WRITE | > + IPSRXIDX_TABLE_IP | (ip_index << 3); > + if (priv->rx_ip_tbl[ip_index].ip.type =3D=3D IPv4) { > + IXGBE_WRITE_REG(hw, IXGBE_IPSRXIPADDR(0), 0); > + IXGBE_WRITE_REG(hw, IXGBE_IPSRXIPADDR(1), 0); > + IXGBE_WRITE_REG(hw, IXGBE_IPSRXIPADDR(2), 0); > + IXGBE_WRITE_REG(hw, IXGBE_IPSRXIPADDR(3), > + priv->rx_ip_tbl[ip_index].ip.ipv4); > + } else { > + IXGBE_WRITE_REG(hw, IXGBE_IPSRXIPADDR(0), > + priv->rx_ip_tbl[ip_index].ip.ipv6[0]); > + IXGBE_WRITE_REG(hw, IXGBE_IPSRXIPADDR(1), > + priv->rx_ip_tbl[ip_index].ip.ipv6[1]); > + IXGBE_WRITE_REG(hw, IXGBE_IPSRXIPADDR(2), > + priv->rx_ip_tbl[ip_index].ip.ipv6[2]); > + IXGBE_WRITE_REG(hw, IXGBE_IPSRXIPADDR(3), > + priv->rx_ip_tbl[ip_index].ip.ipv6[3]); > + } > + IXGBE_WAIT_RWRITE; > + > + /* write SPI table entry*/ > + reg_val =3D IPSRXIDX_RX_EN | IPSRXIDX_WRITE | > + IPSRXIDX_TABLE_SPI | (sa_index << 3); > + IXGBE_WRITE_REG(hw, IXGBE_IPSRXSPI, > + priv->rx_sa_tbl[sa_index].spi); > + IXGBE_WRITE_REG(hw, IXGBE_IPSRXIPIDX, > + priv->rx_sa_tbl[sa_index].ip_index); > + IXGBE_WAIT_RWRITE; > + > + /* write Key table entry*/ > + reg_val =3D IPSRXIDX_RX_EN | IPSRXIDX_WRITE | > + IPSRXIDX_TABLE_KEY | (sa_index << 3); > + IXGBE_WRITE_REG(hw, IXGBE_IPSRXKEY(0), > + priv->rx_sa_tbl[sa_index].key[0]); > + IXGBE_WRITE_REG(hw, IXGBE_IPSRXKEY(1), > + priv->rx_sa_tbl[sa_index].key[1]); > + IXGBE_WRITE_REG(hw, IXGBE_IPSRXKEY(2), > + priv->rx_sa_tbl[sa_index].key[2]); > + IXGBE_WRITE_REG(hw, IXGBE_IPSRXKEY(3), > + priv->rx_sa_tbl[sa_index].key[3]); > + IXGBE_WRITE_REG(hw, IXGBE_IPSRXSALT, > + priv->rx_sa_tbl[sa_index].salt); > + IXGBE_WRITE_REG(hw, IXGBE_IPSRXMOD, > + priv->rx_sa_tbl[sa_index].mode); > + IXGBE_WAIT_RWRITE; > + > + } else { /* sess->dir =3D=3D RTE_CRYPTO_OUTBOUND */ > + int i; > + > + /* Find a free entry in the SA table*/ > + for (i =3D 0; i < IPSEC_MAX_SA_COUNT; i++) { > + if (priv->tx_sa_tbl[i].used =3D=3D 0) { > + sa_index =3D i; > + break; > + } > + } > + /* Fail if no free entries*/ > + if (sa_index < 0) { > + PMD_DRV_LOG(ERR, > + "No free entry left in the Tx SA table\n"); > + return -1; > + } > + > + priv->tx_sa_tbl[sa_index].spi =3D > + rte_cpu_to_be_32(ic_session->spi); > + priv->tx_sa_tbl[sa_index].key[3] =3D > + rte_cpu_to_be_32(*(uint32_t *)&ic_session->key[0]); > + priv->tx_sa_tbl[sa_index].key[2] =3D > + rte_cpu_to_be_32(*(uint32_t *)&ic_session->key[4]); > + priv->tx_sa_tbl[sa_index].key[1] =3D > + rte_cpu_to_be_32(*(uint32_t *)&ic_session->key[8]); > + priv->tx_sa_tbl[sa_index].key[0] =3D > + rte_cpu_to_be_32(*(uint32_t *)&ic_session->key[12]); > + priv->tx_sa_tbl[sa_index].salt =3D > + rte_cpu_to_be_32(ic_session->salt); > + > + reg_val =3D IPSRXIDX_RX_EN | IPSRXIDX_WRITE | (sa_index << 3); > + IXGBE_WRITE_REG(hw, IXGBE_IPSTXKEY(0), > + priv->tx_sa_tbl[sa_index].key[0]); > + IXGBE_WRITE_REG(hw, IXGBE_IPSTXKEY(1), > + priv->tx_sa_tbl[sa_index].key[1]); > + IXGBE_WRITE_REG(hw, IXGBE_IPSTXKEY(2), > + priv->tx_sa_tbl[sa_index].key[2]); > + IXGBE_WRITE_REG(hw, IXGBE_IPSTXKEY(3), > + priv->tx_sa_tbl[sa_index].key[3]); > + IXGBE_WRITE_REG(hw, IXGBE_IPSTXSALT, > + priv->tx_sa_tbl[sa_index].salt); > + IXGBE_WAIT_TWRITE; > + > + priv->tx_sa_tbl[i].used =3D 1; > + ic_session->sa_index =3D sa_index; > + } > + > + return 0; > +} > + > +static int > +ixgbe_crypto_remove_sa(struct rte_eth_dev *dev, > + struct ixgbe_crypto_session *ic_session) > +{ > + struct ixgbe_hw *hw =3D IXGBE_DEV_PRIVATE_TO_HW(dev->data->dev_private)= ; > + struct ixgbe_ipsec *priv =3D > + IXGBE_DEV_PRIVATE_TO_IPSEC(dev->data->dev_private); > + uint32_t reg_val; > + int sa_index =3D -1; > + > + if (ic_session->op =3D=3D IXGBE_OP_AUTHENTICATED_DECRYPTION) { > + int i, ip_index =3D -1; > + > + /* Find a match in the IP table*/ > + for (i =3D 0; i < IPSEC_MAX_RX_IP_COUNT; i++) { > + if (CMP_IP(priv->rx_ip_tbl[i].ip, ic_session->dst_ip)) { > + ip_index =3D i; > + break; > + } > + } > + > + /* Fail if no match*/ > + if (ip_index < 0) { > + PMD_DRV_LOG(ERR, > + "Entry not found in the Rx IP table\n"); > + return -1; > + } > + > + /* Find a free entry in the SA table*/ > + for (i =3D 0; i < IPSEC_MAX_SA_COUNT; i++) { > + if (priv->rx_sa_tbl[i].spi =3D=3D > + rte_cpu_to_be_32(ic_session->spi)) { > + sa_index =3D i; > + break; > + } > + } > + /* Fail if no match*/ > + if (sa_index < 0) { > + PMD_DRV_LOG(ERR, > + "Entry not found in the Rx SA table\n"); > + return -1; > + } > + > + /* Disable and clear Rx SPI and key table table entryes*/ > + reg_val =3D IPSRXIDX_WRITE | IPSRXIDX_TABLE_SPI | (sa_index << 3); > + IXGBE_WRITE_REG(hw, IXGBE_IPSRXSPI, 0); > + IXGBE_WRITE_REG(hw, IXGBE_IPSRXIPIDX, 0); > + IXGBE_WAIT_RWRITE; > + reg_val =3D IPSRXIDX_WRITE | IPSRXIDX_TABLE_KEY | (sa_index << 3); > + IXGBE_WRITE_REG(hw, IXGBE_IPSRXKEY(0), 0); > + IXGBE_WRITE_REG(hw, IXGBE_IPSRXKEY(1), 0); > + IXGBE_WRITE_REG(hw, IXGBE_IPSRXKEY(2), 0); > + IXGBE_WRITE_REG(hw, IXGBE_IPSRXKEY(3), 0); > + IXGBE_WRITE_REG(hw, IXGBE_IPSRXSALT, 0); > + IXGBE_WRITE_REG(hw, IXGBE_IPSRXMOD, 0); > + IXGBE_WAIT_RWRITE; > + priv->rx_sa_tbl[sa_index].used =3D 0; > + > + /* If last used then clear the IP table entry*/ > + priv->rx_ip_tbl[ip_index].ref_count--; > + if (priv->rx_ip_tbl[ip_index].ref_count =3D=3D 0) { > + reg_val =3D IPSRXIDX_WRITE | IPSRXIDX_TABLE_IP | > + (ip_index << 3); > + IXGBE_WRITE_REG(hw, IXGBE_IPSRXIPADDR(0), 0); > + IXGBE_WRITE_REG(hw, IXGBE_IPSRXIPADDR(1), 0); > + IXGBE_WRITE_REG(hw, IXGBE_IPSRXIPADDR(2), 0); > + IXGBE_WRITE_REG(hw, IXGBE_IPSRXIPADDR(3), 0); > + } > + } else { /* session->dir =3D=3D RTE_CRYPTO_OUTBOUND */ > + int i; > + > + /* Find a match in the SA table*/ > + for (i =3D 0; i < IPSEC_MAX_SA_COUNT; i++) { > + if (priv->tx_sa_tbl[i].spi =3D=3D > + rte_cpu_to_be_32(ic_session->spi)) { > + sa_index =3D i; > + break; > + } > + } > + /* Fail if no match entries*/ > + if (sa_index < 0) { > + PMD_DRV_LOG(ERR, > + "Entry not found in the Tx SA table\n"); > + return -1; > + } > + reg_val =3D IPSRXIDX_WRITE | (sa_index << 3); > + IXGBE_WRITE_REG(hw, IXGBE_IPSTXKEY(0), 0); > + IXGBE_WRITE_REG(hw, IXGBE_IPSTXKEY(1), 0); > + IXGBE_WRITE_REG(hw, IXGBE_IPSTXKEY(2), 0); > + IXGBE_WRITE_REG(hw, IXGBE_IPSTXKEY(3), 0); > + IXGBE_WRITE_REG(hw, IXGBE_IPSTXSALT, 0); > + IXGBE_WAIT_TWRITE; > + > + priv->tx_sa_tbl[sa_index].used =3D 0; > + } > + > + return 0; > +} > + > +static int > +ixgbe_crypto_create_session(void *device, > + struct rte_security_session_conf *conf, > + struct rte_security_session *session, > + struct rte_mempool *mempool) > +{ > + struct rte_eth_dev *eth_dev =3D (struct rte_eth_dev *)device; > + struct ixgbe_crypto_session *ic_session =3D NULL; > + struct rte_crypto_aead_xform *aead_xform; > + struct rte_eth_conf *dev_conf =3D ð_dev->data->dev_conf; > + > + if (rte_mempool_get(mempool, (void **)&ic_session)) { > + PMD_DRV_LOG(ERR, "Cannot get object from ic_session mempool"); > + return -ENOMEM; > + } > + > + if (conf->crypto_xform->type !=3D RTE_CRYPTO_SYM_XFORM_AEAD || > + conf->crypto_xform->aead.algo !=3D > + RTE_CRYPTO_AEAD_AES_GCM) { > + PMD_DRV_LOG(ERR, "Unsupported crypto transformation mode\n"); > + return -ENOTSUP; > + } > + aead_xform =3D &conf->crypto_xform->aead; > + > + if (conf->ipsec.direction =3D=3D RTE_SECURITY_IPSEC_SA_DIR_INGRESS) { > + if (dev_conf->rxmode.enable_sec) { > + ic_session->op =3D IXGBE_OP_AUTHENTICATED_DECRYPTION; > + } else { > + PMD_DRV_LOG(ERR, "IPsec decryption not enabled\n"); > + return -ENOTSUP; > + } > + } else { > + if (dev_conf->txmode.enable_sec) { > + ic_session->op =3D IXGBE_OP_AUTHENTICATED_ENCRYPTION; > + } else { > + PMD_DRV_LOG(ERR, "IPsec encryption not enabled\n"); > + return -ENOTSUP; > + } > + } > + > + ic_session->key =3D aead_xform->key.data; > + memcpy(&ic_session->salt, > + &aead_xform->key.data[aead_xform->key.length], 4); > + ic_session->spi =3D conf->ipsec.spi; > + ic_session->dev =3D eth_dev; > + > + set_sec_session_private_data(session, ic_session); > + > + if (ic_session->op =3D=3D IXGBE_OP_AUTHENTICATED_ENCRYPTION) { > + if (ixgbe_crypto_add_sa(ic_session)) { > + PMD_DRV_LOG(ERR, "Failed to add SA\n"); > + return -EPERM; > + } > + } > + > + return 0; > +} > + > +static int > +ixgbe_crypto_remove_session(void *device, > + struct rte_security_session *session) > +{ > + struct rte_eth_dev *eth_dev =3D device; > + struct ixgbe_crypto_session *ic_session =3D > + (struct ixgbe_crypto_session *) > + get_sec_session_private_data(session); > + struct rte_mempool *mempool =3D rte_mempool_from_obj(ic_session); > + > + if (eth_dev !=3D ic_session->dev) { > + PMD_DRV_LOG(ERR, "Session not bound to this device\n"); > + return -ENODEV; > + } > + > + if (ixgbe_crypto_remove_sa(eth_dev, ic_session)) { > + PMD_DRV_LOG(ERR, "Failed to remove session\n"); > + return -EFAULT; > + } > + > + rte_mempool_put(mempool, (void *)ic_session); > + > + return 0; > +} > + > +static int > +ixgbe_crypto_update_mb(void *device __rte_unused, > + struct rte_security_session *session, > + struct rte_mbuf *m, void *params __rte_unused) > +{ > + struct ixgbe_crypto_session *ic_session =3D > + get_sec_session_private_data(session); > + if (ic_session->op =3D=3D IXGBE_OP_AUTHENTICATED_ENCRYPTION) { > + struct ixgbe_crypto_tx_desc_md *mdata =3D > + (struct ixgbe_crypto_tx_desc_md *)&m->udata64; > + mdata->enc =3D 1; > + mdata->sa_idx =3D ic_session->sa_index; > + mdata->pad_len =3D *rte_pktmbuf_mtod_offset(m, > + uint8_t *, rte_pktmbuf_pkt_len(m) - 18) + 18; Could you explain what pad_len supposed to contain? Also what is a magical constant '18'? Could you create some macro if needed? > + } > + return 0; > +} > + > +struct rte_cryptodev_capabilities aes_gmac_crypto_capabilities[] =3D { > + { /* AES GMAC (128-bit) */ > + .op =3D RTE_CRYPTO_OP_TYPE_SYMMETRIC, > + {.sym =3D { > + .xform_type =3D RTE_CRYPTO_SYM_XFORM_AUTH, > + {.auth =3D { > + .algo =3D RTE_CRYPTO_AUTH_AES_GMAC, > + .block_size =3D 16, > + .key_size =3D { > + .min =3D 16, > + .max =3D 16, > + .increment =3D 0 > + }, > + .digest_size =3D { > + .min =3D 12, > + .max =3D 12, > + .increment =3D 0 > + }, > + .iv_size =3D { > + .min =3D 12, > + .max =3D 12, > + .increment =3D 0 > + } > + }, } > + }, } > + }, > + { > + .op =3D RTE_CRYPTO_OP_TYPE_UNDEFINED, > + {.sym =3D { > + .xform_type =3D RTE_CRYPTO_SYM_XFORM_NOT_SPECIFIED > + }, } > + }, > +}; > + > +struct rte_cryptodev_capabilities aes_gcm_gmac_crypto_capabilities[] =3D= { > + { /* AES GMAC (128-bit) */ > + .op =3D RTE_CRYPTO_OP_TYPE_SYMMETRIC, > + {.sym =3D { > + .xform_type =3D RTE_CRYPTO_SYM_XFORM_AUTH, > + {.auth =3D { > + .algo =3D RTE_CRYPTO_AUTH_AES_GMAC, > + .block_size =3D 16, > + .key_size =3D { > + .min =3D 16, > + .max =3D 16, > + .increment =3D 0 > + }, > + .digest_size =3D { > + .min =3D 12, > + .max =3D 12, > + .increment =3D 0 > + }, > + .iv_size =3D { > + .min =3D 12, > + .max =3D 12, > + .increment =3D 0 > + } > + }, } > + }, } > + }, > + { /* AES GCM (128-bit) */ > + .op =3D RTE_CRYPTO_OP_TYPE_SYMMETRIC, > + {.sym =3D { > + .xform_type =3D RTE_CRYPTO_SYM_XFORM_AEAD, > + {.aead =3D { > + .algo =3D RTE_CRYPTO_AEAD_AES_GCM, > + .block_size =3D 16, > + .key_size =3D { > + .min =3D 16, > + .max =3D 16, > + .increment =3D 0 > + }, > + .digest_size =3D { > + .min =3D 8, > + .max =3D 16, > + .increment =3D 4 > + }, > + .aad_size =3D { > + .min =3D 0, > + .max =3D 65535, > + .increment =3D 1 > + }, > + .iv_size =3D { > + .min =3D 12, > + .max =3D 12, > + .increment =3D 0 > + } > + }, } > + }, } > + }, > + { > + .op =3D RTE_CRYPTO_OP_TYPE_UNDEFINED, > + {.sym =3D { > + .xform_type =3D RTE_CRYPTO_SYM_XFORM_NOT_SPECIFIED > + }, } > + }, > +}; > + > +static const struct rte_security_capability ixgbe_security_capabilities[= ] =3D { > + { /* IPsec Inline Crypto ESP Transport Egress */ > + .action =3D RTE_SECURITY_ACTION_TYPE_INLINE_CRYPTO, > + .protocol =3D RTE_SECURITY_PROTOCOL_IPSEC, > + .ipsec =3D { > + .proto =3D RTE_SECURITY_IPSEC_SA_PROTO_ESP, > + .mode =3D RTE_SECURITY_IPSEC_SA_MODE_TRANSPORT, > + .direction =3D RTE_SECURITY_IPSEC_SA_DIR_EGRESS, > + .options =3D { 0 } > + }, > + .crypto_capabilities =3D aes_gcm_gmac_crypto_capabilities, > + .ol_flags =3D RTE_SECURITY_TX_OLOAD_NEED_MDATA > + }, > + { /* IPsec Inline Crypto ESP Transport Ingress */ > + .action =3D RTE_SECURITY_ACTION_TYPE_INLINE_CRYPTO, > + .protocol =3D RTE_SECURITY_PROTOCOL_IPSEC, > + .ipsec =3D { > + .proto =3D RTE_SECURITY_IPSEC_SA_PROTO_ESP, > + .mode =3D RTE_SECURITY_IPSEC_SA_MODE_TRANSPORT, > + .direction =3D RTE_SECURITY_IPSEC_SA_DIR_INGRESS, > + .options =3D { 0 } > + }, > + .crypto_capabilities =3D aes_gcm_gmac_crypto_capabilities, > + .ol_flags =3D 0 > + }, > + { /* IPsec Inline Crypto ESP Tunnel Egress */ > + .action =3D RTE_SECURITY_ACTION_TYPE_INLINE_CRYPTO, > + .protocol =3D RTE_SECURITY_PROTOCOL_IPSEC, > + .ipsec =3D { > + .proto =3D RTE_SECURITY_IPSEC_SA_PROTO_ESP, > + .mode =3D RTE_SECURITY_IPSEC_SA_MODE_TUNNEL, > + .direction =3D RTE_SECURITY_IPSEC_SA_DIR_EGRESS, > + .options =3D { 0 } > + }, > + .crypto_capabilities =3D aes_gcm_gmac_crypto_capabilities, > + .ol_flags =3D RTE_SECURITY_TX_OLOAD_NEED_MDATA > + }, > + { /* IPsec Inline Crypto ESP Tunnel Ingress */ > + .action =3D RTE_SECURITY_ACTION_TYPE_INLINE_CRYPTO, > + .protocol =3D RTE_SECURITY_PROTOCOL_IPSEC, > + .ipsec =3D { > + .proto =3D RTE_SECURITY_IPSEC_SA_PROTO_ESP, > + .mode =3D RTE_SECURITY_IPSEC_SA_MODE_TUNNEL, > + .direction =3D RTE_SECURITY_IPSEC_SA_DIR_INGRESS, > + .options =3D { 0 } > + }, > + .crypto_capabilities =3D aes_gcm_gmac_crypto_capabilities, > + .ol_flags =3D 0 > + }, > + { > + .action =3D RTE_SECURITY_ACTION_TYPE_NONE > + } > +}; > + > +static const struct rte_security_capability * > +ixgbe_crypto_capabilities_get(void *device __rte_unused) > +{ As a nit: if ixgbe_security_capabilities are not used in any other place -= =20 you can move its definition inside that function. > + return ixgbe_security_capabilities; > +} > + > + > +int > +ixgbe_crypto_enable_ipsec(struct rte_eth_dev *dev) > +{ > + struct ixgbe_hw *hw =3D IXGBE_DEV_PRIVATE_TO_HW(dev->data->dev_private)= ; > + uint32_t reg; > + > + /* sanity checks */ > + if (dev->data->dev_conf.rxmode.enable_lro) { > + PMD_DRV_LOG(ERR, "RSC and IPsec not supported"); > + return -1; > + } > + if (!dev->data->dev_conf.rxmode.hw_strip_crc) { > + PMD_DRV_LOG(ERR, "HW CRC strip needs to be enabled for IPsec"); > + return -1; > + } > + > + > + /* Set IXGBE_SECTXBUFFAF to 0x15 as required in the datasheet*/ > + IXGBE_WRITE_REG(hw, IXGBE_SECTXBUFFAF, 0x15); > + > + /* IFG needs to be set to 3 when we are using security. Otherwise a Tx > + * hang will occur with heavy traffic. > + */ > + reg =3D IXGBE_READ_REG(hw, IXGBE_SECTXMINIFG); > + reg =3D (reg & 0xFFFFFFF0) | 0x3; > + IXGBE_WRITE_REG(hw, IXGBE_SECTXMINIFG, reg); > + > + reg =3D IXGBE_READ_REG(hw, IXGBE_HLREG0); > + reg |=3D IXGBE_HLREG0_TXCRCEN | IXGBE_HLREG0_RXCRCSTRP; > + IXGBE_WRITE_REG(hw, IXGBE_HLREG0, reg); > + > + if (dev->data->dev_conf.rxmode.enable_sec) { > + IXGBE_WRITE_REG(hw, IXGBE_SECRXCTRL, 0); > + reg =3D IXGBE_READ_REG(hw, IXGBE_SECRXCTRL); > + if (reg !=3D 0) { > + PMD_DRV_LOG(ERR, "Error enabling Rx Crypto"); > + return -1; > + } > + } > + if (dev->data->dev_conf.txmode.enable_sec) { > + IXGBE_WRITE_REG(hw, IXGBE_SECTXCTRL, > + IXGBE_SECTXCTRL_STORE_FORWARD); > + reg =3D IXGBE_READ_REG(hw, IXGBE_SECTXCTRL); > + if (reg !=3D IXGBE_SECTXCTRL_STORE_FORWARD) { > + PMD_DRV_LOG(ERR, "Error enabling Rx Crypto"); > + return -1; > + } > + } > + > + ixgbe_crypto_clear_ipsec_tables(dev); > + > + return 0; > +} > + > +int > +ixgbe_crypto_add_ingress_sa_from_flow(const void *sess, > + const void *ip_spec, > + uint8_t is_ipv6) > +{ > + struct ixgbe_crypto_session *ic_session > + =3D get_sec_session_private_data(sess); > + > + if (ic_session->op =3D=3D IXGBE_OP_AUTHENTICATED_DECRYPTION) { > + if (is_ipv6) { > + const struct rte_flow_item_ipv6 *ipv6 =3D ip_spec; > + ic_session->src_ip.type =3D IPv6; > + ic_session->dst_ip.type =3D IPv6; > + rte_memcpy(ic_session->src_ip.ipv6, > + ipv6->hdr.src_addr, 16); > + rte_memcpy(ic_session->dst_ip.ipv6, > + ipv6->hdr.dst_addr, 16); > + } else { > + const struct rte_flow_item_ipv4 *ipv4 =3D ip_spec; > + ic_session->src_ip.type =3D IPv4; > + ic_session->dst_ip.type =3D IPv4; > + ic_session->src_ip.ipv4 =3D ipv4->hdr.src_addr; > + ic_session->dst_ip.ipv4 =3D ipv4->hdr.dst_addr; > + } > + return ixgbe_crypto_add_sa(ic_session); > + } > + > + return 0; > +} > + > + > +struct rte_security_ops ixgbe_security_ops =3D { > + .session_create =3D ixgbe_crypto_create_session, > + .session_update =3D NULL, > + .session_stats_get =3D NULL, > + .session_destroy =3D ixgbe_crypto_remove_session, > + > + .set_pkt_metadata =3D ixgbe_crypto_update_mb, > + > + .capabilities_get =3D ixgbe_crypto_capabilities_get > +}; > diff --git a/drivers/net/ixgbe/ixgbe_ipsec.h b/drivers/net/ixgbe/ixgbe_ip= sec.h > new file mode 100644 > index 0000000..9f06235 > --- /dev/null > +++ b/drivers/net/ixgbe/ixgbe_ipsec.h > @@ -0,0 +1,147 @@ > +/*- > + * BSD LICENSE > + * > + * Copyright(c) 2010-2017 Intel Corporation. All rights reserved. > + * All rights reserved. > + * > + * Redistribution and use in source and binary forms, with or without > + * modification, are permitted provided that the following conditions > + * are met: > + * > + * * Redistributions of source code must retain the above copyright > + * notice, this list of conditions and the following disclaimer. > + * * Redistributions in binary form must reproduce the above copyrig= ht > + * notice, this list of conditions and the following disclaimer in > + * the documentation and/or other materials provided with the > + * distribution. > + * * Neither the name of Intel Corporation nor the names of its > + * contributors may be used to endorse or promote products derived > + * from this software without specific prior written permission. > + * > + * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS > + * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT > + * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS F= OR > + * A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGH= T > + * OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTA= L, > + * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT > + * LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF US= E, > + * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON A= NY > + * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT > + * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE U= SE > + * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE= . > + */ > + > +#ifndef IXGBE_IPSEC_H_ > +#define IXGBE_IPSEC_H_ > + > +#include > + > +#define IPSRXIDX_RX_EN 0x00000001 > +#define IPSRXIDX_TABLE_IP 0x00000002 > +#define IPSRXIDX_TABLE_SPI 0x00000004 > +#define IPSRXIDX_TABLE_KEY 0x00000006 > +#define IPSRXIDX_WRITE 0x80000000 > +#define IPSRXIDX_READ 0x40000000 > +#define IPSRXMOD_VALID 0x00000001 > +#define IPSRXMOD_PROTO 0x00000004 > +#define IPSRXMOD_DECRYPT 0x00000008 > +#define IPSRXMOD_IPV6 0x00000010 > +#define IXGBE_ADVTXD_POPTS_IPSEC 0x00000400 > +#define IXGBE_ADVTXD_TUCMD_IPSEC_TYPE_ESP 0x00002000 > +#define IXGBE_ADVTXD_TUCMD_IPSEC_ENCRYPT_EN 0x00004000 > +#define IXGBE_RXDADV_IPSEC_STATUS_SECP 0x00020000 > +#define IXGBE_RXDADV_IPSEC_ERROR_BIT_MASK 0x18000000 > +#define IXGBE_RXDADV_IPSEC_ERROR_INVALID_PROTOCOL 0x08000000 > +#define IXGBE_RXDADV_IPSEC_ERROR_INVALID_LENGTH 0x10000000 > +#define IXGBE_RXDADV_IPSEC_ERROR_AUTHENTICATION_FAILED 0x18000000 > + > +#define IPSEC_MAX_RX_IP_COUNT 128 > +#define IPSEC_MAX_SA_COUNT 1024 > + > +enum ixgbe_operation { > + IXGBE_OP_AUTHENTICATED_ENCRYPTION, > + IXGBE_OP_AUTHENTICATED_DECRYPTION > +}; > + > +enum ixgbe_gcm_key { > + IXGBE_GCM_KEY_128, > + IXGBE_GCM_KEY_256 > +}; > + > +/** > + * Generic IP address structure > + * TODO: Find better location for this rte_net.h possibly. > + **/ > +struct ipaddr { > + enum ipaddr_type { > + IPv4, > + IPv6 > + } type; > + /**< IP Address Type - IPv4/IPv6 */ > + > + union { > + uint32_t ipv4; > + uint32_t ipv6[4]; > + }; > +}; > + > +/** inline crypto crypto private session structure */ > +struct ixgbe_crypto_session { > + enum ixgbe_operation op; > + uint8_t *key; > + uint32_t salt; > + uint32_t sa_index; > + uint32_t spi; > + struct ipaddr src_ip; > + struct ipaddr dst_ip; > + struct rte_eth_dev *dev; > +} __rte_cache_aligned; > + > +struct ixgbe_crypto_rx_ip_table { > + struct ipaddr ip; > + uint16_t ref_count; > +}; > +struct ixgbe_crypto_rx_sa_table { > + uint32_t spi; > + uint32_t ip_index; > + uint32_t key[4]; > + uint32_t salt; > + uint8_t mode; > + uint8_t used; > +}; > + > +struct ixgbe_crypto_tx_sa_table { > + uint32_t spi; > + uint32_t key[4]; > + uint32_t salt; > + uint8_t used; > +}; > + > +struct ixgbe_crypto_tx_desc_md { > + union { > + uint64_t data; > + struct { > + uint32_t sa_idx; > + uint8_t pad_len; > + uint8_t enc; > + }; > + }; > +}; Why just not:=20 union ixgbe_crypto_tx_desc_md { uint64_t data; struct {...}; }; ? > + > +struct ixgbe_ipsec { > + struct ixgbe_crypto_rx_ip_table rx_ip_tbl[IPSEC_MAX_RX_IP_COUNT]; > + struct ixgbe_crypto_rx_sa_table rx_sa_tbl[IPSEC_MAX_SA_COUNT]; > + struct ixgbe_crypto_tx_sa_table tx_sa_tbl[IPSEC_MAX_SA_COUNT]; > +}; > + > +extern struct rte_security_ops ixgbe_security_ops; > + > + > +int ixgbe_crypto_enable_ipsec(struct rte_eth_dev *dev); > +int ixgbe_crypto_add_ingress_sa_from_flow(const void *sess, > + const void *ip_spec, > + uint8_t is_ipv6); > + > + > + > +#endif /*IXGBE_IPSEC_H_*/ > diff --git a/drivers/net/ixgbe/ixgbe_rxtx.c b/drivers/net/ixgbe/ixgbe_rxt= x.c > index 0038dfb..279e3fa 100644 > --- a/drivers/net/ixgbe/ixgbe_rxtx.c > +++ b/drivers/net/ixgbe/ixgbe_rxtx.c > @@ -93,6 +93,7 @@ > PKT_TX_TCP_SEG | \ > PKT_TX_MACSEC | \ > PKT_TX_OUTER_IP_CKSUM | \ > + PKT_TX_SEC_OFFLOAD | \ > IXGBE_TX_IEEE1588_TMST) >=20 > #define IXGBE_TX_OFFLOAD_NOTSUP_MASK \ > @@ -395,7 +396,8 @@ ixgbe_xmit_pkts_vec(void *tx_queue, struct rte_mbuf *= *tx_pkts, > static inline void > ixgbe_set_xmit_ctx(struct ixgbe_tx_queue *txq, > volatile struct ixgbe_adv_tx_context_desc *ctx_txd, > - uint64_t ol_flags, union ixgbe_tx_offload tx_offload) > + uint64_t ol_flags, union ixgbe_tx_offload tx_offload, > + struct rte_mbuf *mb) I don't think you need to pass mb as a parameter to that function: you already have ol_flags as a parameter and all you need is just struct ix= gbe_crypto_tx_desc_md md here as an extra parameter. > { > uint32_t type_tucmd_mlhl; > uint32_t mss_l4len_idx =3D 0; > @@ -479,6 +481,18 @@ ixgbe_set_xmit_ctx(struct ixgbe_tx_queue *txq, > seqnum_seed |=3D tx_offload.l2_len > << IXGBE_ADVTXD_TUNNEL_LEN; > } > + if (mb->ol_flags & PKT_TX_SEC_OFFLOAD) { > + struct ixgbe_crypto_tx_desc_md *mdata =3D > + (struct ixgbe_crypto_tx_desc_md *) > + &mb->udata64; > + seqnum_seed |=3D > + (IXGBE_ADVTXD_IPSEC_SA_INDEX_MASK & mdata->sa_idx); > + type_tucmd_mlhl |=3D mdata->enc ? > + (IXGBE_ADVTXD_TUCMD_IPSEC_TYPE_ESP | > + IXGBE_ADVTXD_TUCMD_IPSEC_ENCRYPT_EN) : 0; > + type_tucmd_mlhl |=3D > + (mdata->pad_len & IXGBE_ADVTXD_IPSEC_ESP_LEN_MASK); Shouldn't we also update tx_offload_mask here? > + } >=20 > txq->ctx_cache[ctx_idx].flags =3D ol_flags; > txq->ctx_cache[ctx_idx].tx_offload.data[0] =3D > @@ -657,6 +671,7 @@ ixgbe_xmit_pkts(void *tx_queue, struct rte_mbuf **tx_= pkts, > uint32_t ctx =3D 0; > uint32_t new_ctx; > union ixgbe_tx_offload tx_offload; > + uint8_t use_ipsec; >=20 > tx_offload.data[0] =3D 0; > tx_offload.data[1] =3D 0; > @@ -684,6 +699,7 @@ ixgbe_xmit_pkts(void *tx_queue, struct rte_mbuf **tx_= pkts, > * are needed for offload functionality. > */ > ol_flags =3D tx_pkt->ol_flags; > + use_ipsec =3D txq->using_ipsec && (ol_flags & PKT_TX_SEC_OFFLOAD); >=20 > /* If hardware offload required */ > tx_ol_req =3D ol_flags & IXGBE_TX_OFFLOAD_MASK; > @@ -695,6 +711,13 @@ ixgbe_xmit_pkts(void *tx_queue, struct rte_mbuf **tx= _pkts, > tx_offload.tso_segsz =3D tx_pkt->tso_segsz; > tx_offload.outer_l2_len =3D tx_pkt->outer_l2_len; > tx_offload.outer_l3_len =3D tx_pkt->outer_l3_len; > + if (use_ipsec) { > + struct ixgbe_crypto_tx_desc_md *ipsec_mdata =3D > + (struct ixgbe_crypto_tx_desc_md *) > + &tx_pkt->udata64; > + tx_offload.sa_idx =3D ipsec_mdata->sa_idx; > + tx_offload.sec_pad_len =3D ipsec_mdata->pad_len; > + } >=20 > /* If new context need be built or reuse the exist ctx. */ > ctx =3D what_advctx_update(txq, tx_ol_req, > @@ -855,7 +878,7 @@ ixgbe_xmit_pkts(void *tx_queue, struct rte_mbuf **tx_= pkts, > } >=20 > ixgbe_set_xmit_ctx(txq, ctx_txd, tx_ol_req, > - tx_offload); > + tx_offload, tx_pkt); >=20 > txe->last_id =3D tx_last; > tx_id =3D txe->next_id; > @@ -873,6 +896,8 @@ ixgbe_xmit_pkts(void *tx_queue, struct rte_mbuf **tx_= pkts, > } >=20 > olinfo_status |=3D (pkt_len << IXGBE_ADVTXD_PAYLEN_SHIFT); > + if (use_ipsec) > + olinfo_status |=3D IXGBE_ADVTXD_POPTS_IPSEC; >=20 > m_seg =3D tx_pkt; > do { > @@ -1447,6 +1472,12 @@ rx_desc_error_to_pkt_flags(uint32_t rx_status) > pkt_flags |=3D PKT_RX_EIP_CKSUM_BAD; > } >=20 > + if (rx_status & IXGBE_RXD_STAT_SECP) { > + pkt_flags |=3D PKT_RX_SEC_OFFLOAD; > + if (rx_status & IXGBE_RXDADV_LNKSEC_ERROR_BAD_SIG) > + pkt_flags |=3D PKT_RX_SEC_OFFLOAD_FAILED; > + } > + > return pkt_flags; > } >=20 > @@ -2364,8 +2395,9 @@ void __attribute__((cold)) > ixgbe_set_tx_function(struct rte_eth_dev *dev, struct ixgbe_tx_queue *tx= q) > { > /* Use a simple Tx queue (no offloads, no multi segs) if possible */ > - if (((txq->txq_flags & IXGBE_SIMPLE_FLAGS) =3D=3D IXGBE_SIMPLE_FLAGS) > - && (txq->tx_rs_thresh >=3D RTE_PMD_IXGBE_TX_MAX_BURST)) { > + if (((txq->txq_flags & IXGBE_SIMPLE_FLAGS) =3D=3D IXGBE_SIMPLE_FLAGS) &= & > + (txq->tx_rs_thresh >=3D RTE_PMD_IXGBE_TX_MAX_BURST) && > + !(dev->data->dev_conf.txmode.enable_sec)) { > PMD_INIT_LOG(DEBUG, "Using simple tx code path"); > dev->tx_pkt_prepare =3D NULL; > #ifdef RTE_IXGBE_INC_VECTOR > @@ -2535,6 +2567,7 @@ ixgbe_dev_tx_queue_setup(struct rte_eth_dev *dev, > txq->txq_flags =3D tx_conf->txq_flags; > txq->ops =3D &def_txq_ops; > txq->tx_deferred_start =3D tx_conf->tx_deferred_start; > + txq->using_ipsec =3D dev->data->dev_conf.txmode.enable_sec; >=20 > /* > * Modification to set VFTDT for virtual function if vf is detected > @@ -4519,6 +4552,7 @@ ixgbe_set_rx_function(struct rte_eth_dev *dev) > struct ixgbe_rx_queue *rxq =3D dev->data->rx_queues[i]; >=20 > rxq->rx_using_sse =3D rx_using_sse; > + rxq->using_ipsec =3D dev->data->dev_conf.rxmode.enable_sec; > } > } >=20 > @@ -5006,6 +5040,17 @@ ixgbe_dev_rxtx_start(struct rte_eth_dev *dev) > dev->data->dev_conf.lpbk_mode =3D=3D IXGBE_LPBK_82599_TX_RX) > ixgbe_setup_loopback_link_82599(hw); >=20 > + if (dev->data->dev_conf.rxmode.enable_sec || > + dev->data->dev_conf.txmode.enable_sec) { > + ret =3D ixgbe_crypto_enable_ipsec(dev); > + if (ret !=3D 0) { > + PMD_DRV_LOG(ERR, > + "ixgbe_crypto_enable_ipsec fails with %d.", > + ret); > + return ret; > + } > + } > + > return 0; > } >=20 > diff --git a/drivers/net/ixgbe/ixgbe_rxtx.h b/drivers/net/ixgbe/ixgbe_rxt= x.h > index 81c527f..4017831 100644 > --- a/drivers/net/ixgbe/ixgbe_rxtx.h > +++ b/drivers/net/ixgbe/ixgbe_rxtx.h > @@ -138,8 +138,10 @@ struct ixgbe_rx_queue { > uint16_t rx_nb_avail; /**< nr of staged pkts ready to ret to app */ > uint16_t rx_next_avail; /**< idx of next staged pkt to ret to app */ > uint16_t rx_free_trigger; /**< triggers rx buffer allocation */ > - uint16_t rx_using_sse; > + uint8_t rx_using_sse; > /**< indicates that vector RX is in use */ > + uint8_t using_ipsec; > + /**< indicates that IPsec RX feature is in use */ > #ifdef RTE_IXGBE_INC_VECTOR > uint16_t rxrearm_nb; /**< number of remaining to be re-a= rmed */ > uint16_t rxrearm_start; /**< the idx we start the re-arming= from */ > @@ -183,6 +185,10 @@ union ixgbe_tx_offload { > /* fields for TX offloading of tunnels */ > uint64_t outer_l3_len:8; /**< Outer L3 (IP) Hdr Length. */ > uint64_t outer_l2_len:8; /**< Outer L2 (MAC) Hdr Length. */ > + > + /* inline ipsec related*/ > + uint64_t sa_idx:8; /**< TX SA database entry index */ > + uint64_t sec_pad_len:4; /**< padding length */ > }; > }; >=20 > @@ -247,6 +253,9 @@ struct ixgbe_tx_queue { > struct ixgbe_advctx_info ctx_cache[IXGBE_CTX_NUM]; > const struct ixgbe_txq_ops *ops; /**< txq ops */ > uint8_t tx_deferred_start; /**< not in global dev start. */ > + uint8_t using_ipsec; > + /**< indicates that IPsec TX feature is in use */ > + > }; >=20 > struct ixgbe_txq_ops { > diff --git a/drivers/net/ixgbe/ixgbe_rxtx_vec_sse.c b/drivers/net/ixgbe/i= xgbe_rxtx_vec_sse.c > index e704a7f..c9b1e2e 100644 > --- a/drivers/net/ixgbe/ixgbe_rxtx_vec_sse.c > +++ b/drivers/net/ixgbe/ixgbe_rxtx_vec_sse.c > @@ -124,10 +124,12 @@ ixgbe_rxq_rearm(struct ixgbe_rx_queue *rxq) >=20 > static inline void > desc_to_olflags_v(__m128i descs[4], __m128i mbuf_init, uint8_t vlan_flag= s, > - struct rte_mbuf **rx_pkts) > + struct rte_mbuf **rx_pkts, uint8_t use_ipsec) > { > __m128i ptype0, ptype1, vtag0, vtag1, csum; > __m128i rearm0, rearm1, rearm2, rearm3; > + __m128i sterr0, sterr1, sterr2, sterr3; > + __m128i tmp1, tmp2, tmp3, tmp4; >=20 > /* mask everything except rss type */ > const __m128i rsstype_msk =3D _mm_set_epi16( > @@ -174,6 +176,41 @@ desc_to_olflags_v(__m128i descs[4], __m128i mbuf_ini= t, uint8_t vlan_flags, > 0, PKT_RX_L4_CKSUM_GOOD >> sizeof(uint8_t), 0, > PKT_RX_L4_CKSUM_GOOD >> sizeof(uint8_t)); >=20 > + const __m128i ipsec_sterr_msk =3D _mm_set_epi32( > + 0, IXGBE_RXDADV_IPSEC_STATUS_SECP | > + IXGBE_RXDADV_IPSEC_ERROR_AUTH_FAILED, > + 0, 0); > + const __m128i ipsec_proc_msk =3D _mm_set_epi32( > + 0, IXGBE_RXDADV_IPSEC_STATUS_SECP, 0, 0); > + const __m128i ipsec_err_flag =3D _mm_set_epi32( > + 0, PKT_RX_SEC_OFFLOAD_FAILED | PKT_RX_SEC_OFFLOAD, > + 0, 0); > + const __m128i ipsec_proc_flag =3D _mm_set_epi32( > + 0, PKT_RX_SEC_OFFLOAD, 0, 0); > + > + if (use_ipsec) { > + sterr0 =3D _mm_and_si128(descs[0], ipsec_sterr_msk); > + sterr1 =3D _mm_and_si128(descs[1], ipsec_sterr_msk); > + sterr2 =3D _mm_and_si128(descs[2], ipsec_sterr_msk); > + sterr3 =3D _mm_and_si128(descs[3], ipsec_sterr_msk); > + tmp1 =3D _mm_cmpeq_epi32(sterr0, ipsec_sterr_msk); > + tmp2 =3D _mm_cmpeq_epi32(sterr0, ipsec_proc_msk); > + tmp3 =3D _mm_cmpeq_epi32(sterr1, ipsec_sterr_msk); > + tmp4 =3D _mm_cmpeq_epi32(sterr1, ipsec_proc_msk); > + sterr0 =3D _mm_or_si128(_mm_and_si128(tmp1, ipsec_err_flag), > + _mm_and_si128(tmp2, ipsec_proc_flag)); > + sterr1 =3D _mm_or_si128(_mm_and_si128(tmp3, ipsec_err_flag), > + _mm_and_si128(tmp4, ipsec_proc_flag)); > + tmp1 =3D _mm_cmpeq_epi32(sterr2, ipsec_sterr_msk); > + tmp2 =3D _mm_cmpeq_epi32(sterr2, ipsec_proc_msk); > + tmp3 =3D _mm_cmpeq_epi32(sterr3, ipsec_sterr_msk); > + tmp4 =3D _mm_cmpeq_epi32(sterr3, ipsec_proc_msk); > + sterr2 =3D _mm_or_si128(_mm_and_si128(tmp1, ipsec_err_flag), > + _mm_and_si128(tmp2, ipsec_proc_flag)); > + sterr3 =3D _mm_or_si128(_mm_and_si128(tmp3, ipsec_err_flag), > + _mm_and_si128(tmp4, ipsec_proc_flag)); > + } > + > ptype0 =3D _mm_unpacklo_epi16(descs[0], descs[1]); > ptype1 =3D _mm_unpacklo_epi16(descs[2], descs[3]); > vtag0 =3D _mm_unpackhi_epi16(descs[0], descs[1]); > @@ -221,6 +258,13 @@ desc_to_olflags_v(__m128i descs[4], __m128i mbuf_ini= t, uint8_t vlan_flags, > rearm2 =3D _mm_blend_epi16(mbuf_init, _mm_slli_si128(vtag1, 4), 0x10); > rearm3 =3D _mm_blend_epi16(mbuf_init, _mm_slli_si128(vtag1, 2), 0x10); >=20 > + if (use_ipsec) { > + rearm0 =3D _mm_or_si128(rearm0, sterr0); > + rearm1 =3D _mm_or_si128(rearm1, sterr1); > + rearm2 =3D _mm_or_si128(rearm2, sterr2); > + rearm3 =3D _mm_or_si128(rearm3, sterr3); > + } > + > /* write the rearm data and the olflags in one write */ > RTE_BUILD_BUG_ON(offsetof(struct rte_mbuf, ol_flags) !=3D > offsetof(struct rte_mbuf, rearm_data) + 8); > @@ -310,6 +354,7 @@ _recv_raw_pkts_vec(struct ixgbe_rx_queue *rxq, struct= rte_mbuf **rx_pkts, > volatile union ixgbe_adv_rx_desc *rxdp; > struct ixgbe_rx_entry *sw_ring; > uint16_t nb_pkts_recd; > + uint8_t use_ipsec =3D rxq->using_ipsec; > int pos; > uint64_t var; > __m128i shuf_msk; > @@ -471,7 +516,8 @@ _recv_raw_pkts_vec(struct ixgbe_rx_queue *rxq, struct= rte_mbuf **rx_pkts, > sterr_tmp1 =3D _mm_unpackhi_epi32(descs[1], descs[0]); >=20 > /* set ol_flags with vlan packet type */ > - desc_to_olflags_v(descs, mbuf_init, vlan_flags, &rx_pkts[pos]); > + desc_to_olflags_v(descs, mbuf_init, vlan_flags, > + &rx_pkts[pos], use_ipsec); >=20 > /* D.2 pkt 3,4 set in_port/nb_seg and remove crc */ > pkt_mb4 =3D _mm_add_epi16(pkt_mb4, crc_adjust); > -- > 2.9.3