From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from EUR04-DB3-obe.outbound.protection.outlook.com (mail-eopbgr60048.outbound.protection.outlook.com [40.107.6.48]) by dpdk.org (Postfix) with ESMTP id BB0265424 for ; Wed, 31 Oct 2018 16:10:00 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Mellanox.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=e5W6IryLJ7bAsHW0ADBG5Kl6R9Y4Ir8l3QN9CX1T3f0=; b=iSi01BjjwRY/elsyYxIX0l5trhE0oz7/A2Cld2HFQnQMCuqbQpIf/nAJMDYbRkPax1rSAtR9pAHUn2BJdBaKDovVXcrEtKsiNA6fVSs496nRUl7CIWsVjRkJzxMFs+OIBof5JN6+XVzIPiJB8X86KZvc8VXVjGwEdOXPilzis44= Received: from DB7PR05MB4426.eurprd05.prod.outlook.com (52.134.109.15) by DB7PR05MB4187.eurprd05.prod.outlook.com (52.134.107.156) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.1273.25; Wed, 31 Oct 2018 15:09:59 +0000 Received: from DB7PR05MB4426.eurprd05.prod.outlook.com ([fe80::80e:e6b:baf2:d973]) by DB7PR05MB4426.eurprd05.prod.outlook.com ([fe80::80e:e6b:baf2:d973%3]) with mapi id 15.20.1273.027; Wed, 31 Oct 2018 15:09:59 +0000 From: Shahaf Shuler To: Dekel Peled , Yongseok Koh CC: "dev@dpdk.org" , Ori Kam Thread-Topic: [dpdk-dev] [PATCH v7 7/7] net/mlx5: add caching of encap decap actions Thread-Index: AQHUcO/wTgz070NvcEWgRzBpx0C2yqU5U4rQ Date: Wed, 31 Oct 2018 15:09:59 +0000 Message-ID: References: <1540498108-18358-1-git-send-email-dekelp@mellanox.com> <1540969847-48919-8-git-send-email-dekelp@mellanox.com> In-Reply-To: <1540969847-48919-8-git-send-email-dekelp@mellanox.com> Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: authentication-results: spf=none (sender IP is ) smtp.mailfrom=shahafs@mellanox.com; x-originating-ip: [193.47.165.251] x-ms-publictraffictype: Email x-microsoft-exchange-diagnostics: 1; DB7PR05MB4187; 6:aYBknJD601eR3B0zFN1vxcT9Mb1lMa0NnYFkWXWfX6yw5ipFhsXWjGzlnWeWgF9SbZk72ywhTbvNhKg4Px5SntsnmUBzL1TfmJD6Qgmn9Gn1tH0JXfpozY48O3zAv8Iwg6KTl789yup88UmsIgyux3GRDkod7aXya6TsPumHrIpsnC5N+HLpCstDTni+YY+lp5evMuC8GBY0FtbsVIwG9VkVQjBSSwX+0sAcIxQaEngiI0t2aQrGep/u8PDPuxc/XMNLIPQuPAv8cXBVu8fEqyIEOufGPa7tMFb4XfDX1MCvTFF0AA5yO2nrkn6gUCadBEHPfw4E4uK/VO4xrQmz0k+6gINb40u+5QirgI17PdGn9OfdeRBRbuVfBG2mTPeqmuzuMHCC/gJc6XO0BhvFfrx1m3jP5TYbeQyYuNLrdWvSyMsCIEo4BE1XYuG0tjAmhZm3dFIkyWc2i3ZE8dZQ2Q==; 5:l0PtPx8rUh+cMNKR2dnaoaXBDL2ltD4FJ9ElSYvgnM72Z8bQYfYi3i9f83ZdTLlYLJiS44bZYwbs2deXnLwG/bgHcZ6SzIwADU4ruLm84jnx/cQk1wmLu/mtJ98YLKgBQckg6y0/YT9YZtZUGGE62/UnZXoLEY+TzGN0uG5ISxw=; 7:KwfnXK0OjAVweP13hOGbHN8Oid0PL11sHcIYC/biV+RXWiLv/fbEdrL8C0IbP0H1AqYAsMq6+AGkVosfcG/X7mslHhph4u47AO7YS8PQHG2s+mM0SPicy/W6NkiTBNeNsSaiQAzg8PLcfU+lBnmVHw== x-ms-exchange-antispam-srfa-diagnostics: SOS; x-ms-office365-filtering-correlation-id: bf270d8e-d171-4142-3d7f-08d63f42eceb x-ms-office365-filtering-ht: Tenant x-microsoft-antispam: BCL:0; PCL:0; RULEID:(7020095)(4652040)(8989299)(4534185)(4627221)(201703031133081)(201702281549075)(8990200)(5600074)(711020)(4618075)(2017052603328)(7153060)(7193020); SRVR:DB7PR05MB4187; x-ms-traffictypediagnostic: DB7PR05MB4187: x-microsoft-antispam-prvs: x-exchange-antispam-report-test: UriScan:; x-ms-exchange-senderadcheck: 1 x-exchange-antispam-report-cfa-test: BCL:0; PCL:0; RULEID:(8211001083)(6040522)(2401047)(8121501046)(5005006)(93006095)(93001095)(10201501046)(3002001)(3231382)(944501410)(52105095)(6055026)(148016)(149066)(150057)(6041310)(20161123564045)(20161123560045)(20161123562045)(20161123558120)(201703131423095)(201702281528075)(20161123555045)(201703061421075)(201703061406153)(201708071742011)(7699051)(76991095); SRVR:DB7PR05MB4187; BCL:0; PCL:0; RULEID:; SRVR:DB7PR05MB4187; x-forefront-prvs: 084285FC5C x-forefront-antispam-report: SFV:NSPM; SFS:(10009020)(376002)(346002)(136003)(396003)(366004)(39860400002)(199004)(189003)(476003)(86362001)(14454004)(478600001)(105586002)(102836004)(26005)(106356001)(186003)(6636002)(486006)(81156014)(81166006)(4326008)(2900100001)(110136005)(54906003)(25786009)(446003)(97736004)(14444005)(256004)(8936002)(11346002)(68736007)(4744004)(66066001)(2906002)(316002)(6116002)(71200400001)(33656002)(5250100002)(5660300001)(55016002)(53936002)(6246003)(107886003)(99286004)(3846002)(71190400001)(9686003)(53946003)(74316002)(305945005)(7736002)(76176011)(7696005)(6506007)(6436002)(229853002)(290074003); DIR:OUT; SFP:1101; SCL:1; SRVR:DB7PR05MB4187; H:DB7PR05MB4426.eurprd05.prod.outlook.com; FPR:; SPF:None; LANG:en; PTR:InfoNoRecords; A:1; MX:1; received-spf: None (protection.outlook.com: mellanox.com does not designate permitted sender hosts) x-microsoft-antispam-message-info: hoj4/Vw4mLsPlH71/pO9c5Djogrqf6Xy8epOsEHdKm/UkplhcQasw7d2rgxCg2F7CY9Pk1jANO2gWJnKfdNIvP/pOtAuva2qTJAdH7VUkx+ku/8ENueiPpeubqo1YJ8OwSEAPV1H6SSFH0mKp61oPExuQrcAmZKjkeMpwYLJyQjE/FNvbZI2oMjpnlGr39SkLkl59zSZsYx8RxmRVElNrB0m3EmJ/ohrPg+FnRKno6zresgYHggu66twQiHzcQnghlQ+jkVSDa2VcnHEL2djEIqhTMmgEN9h6hjY+VjpcHfOxDd8kGF5eeVCsZp3jB84Ff0rGCZtRYDxj1EiGQpMJcTYHD9j3u8jymiibpvWEkg= spamdiagnosticoutput: 1:99 spamdiagnosticmetadata: NSPM Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 X-OriginatorOrg: Mellanox.com X-MS-Exchange-CrossTenant-Network-Message-Id: bf270d8e-d171-4142-3d7f-08d63f42eceb X-MS-Exchange-CrossTenant-originalarrivaltime: 31 Oct 2018 15:09:59.3063 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Hosted X-MS-Exchange-CrossTenant-id: a652971c-7d2e-4d9b-a6a4-d149256f461b X-MS-Exchange-Transport-CrossTenantHeadersStamped: DB7PR05MB4187 Subject: Re: [dpdk-dev] [PATCH v7 7/7] net/mlx5: add caching of encap decap actions X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 31 Oct 2018 15:10:00 -0000 Wednesday, October 31, 2018 9:11 AM, Dekel Peled: > Subject: [dpdk-dev] [PATCH v7 7/7] net/mlx5: add caching of encap decap > actions >=20 > Make flow encap and decap Verbs actions cacheable resources. > Reuse 17.11 PR 876. No one knows what is PR 876 in the mailing list, also this code is not in 1= 7.11 community.=20 Need to make a proper commit log here explains how you do the caching and w= hy it is needed.=20 >=20 > Signed-off-by: Dekel Peled > --- > drivers/net/mlx5/mlx5.h | 1 + > drivers/net/mlx5/mlx5_flow.h | 18 ++- > drivers/net/mlx5/mlx5_flow_dv.c | 265 ++++++++++++++++++++++++++--- > ----------- > 3 files changed, 193 insertions(+), 91 deletions(-) >=20 > diff --git a/drivers/net/mlx5/mlx5.h b/drivers/net/mlx5/mlx5.h index > 74d87c0..0422803 100644 > --- a/drivers/net/mlx5/mlx5.h > +++ b/drivers/net/mlx5/mlx5.h > @@ -219,6 +219,7 @@ struct priv { > /* Verbs Indirection tables. */ > LIST_HEAD(ind_tables, mlx5_ind_table_ibv) ind_tbls; > LIST_HEAD(matchers, mlx5_flow_dv_matcher) matchers; > + LIST_HEAD(encap_decap, mlx5_flow_dv_encap_decap_resource) > +encaps_decaps; > uint32_t link_speed_capa; /* Link speed capabilities. */ > struct mlx5_xstats_ctrl xstats_ctrl; /* Extended stats control. */ > int primary_socket; /* Unix socket for primary process. */ diff --git > a/drivers/net/mlx5/mlx5_flow.h b/drivers/net/mlx5/mlx5_flow.h index > 908123f..25cd9c5 100644 > --- a/drivers/net/mlx5/mlx5_flow.h > +++ b/drivers/net/mlx5/mlx5_flow.h > @@ -170,6 +170,7 @@ struct mlx5_flow_dv_match_params { }; >=20 > #define MLX5_DV_MAX_NUMBER_OF_ACTIONS 8 > +#define MLX5_ENCAP_MAX_LEN 132 >=20 > /* Matcher structure. */ > struct mlx5_flow_dv_matcher { > @@ -183,6 +184,19 @@ struct mlx5_flow_dv_matcher { > struct mlx5_flow_dv_match_params mask; /**< Matcher mask. */ }; >=20 > +/* Encap/decap resource structure. */ > +struct mlx5_flow_dv_encap_decap_resource { > + LIST_ENTRY(mlx5_flow_dv_encap_decap_resource) next; > + /* Pointer to next element. */ > + rte_atomic32_t refcnt; /**< Reference counter. */ > + struct ibv_flow_action *verbs_action; > + /**< Verbs encap/decap action object. */ > + uint8_t buf[MLX5_ENCAP_MAX_LEN]; > + size_t size; > + uint8_t reformat_type; > + uint8_t ft_type; > +}; > + > /* DV flows structure. */ > struct mlx5_flow_dv { > uint64_t hash_fields; /**< Fields that participate in the hash. */ @@ > -191,12 +205,12 @@ struct mlx5_flow_dv { > struct mlx5_flow_dv_matcher *matcher; /**< Cache to matcher. */ > struct mlx5_flow_dv_match_params value; > /**< Holds the value that the packet is compared to. */ > + struct mlx5_flow_dv_encap_decap_resource *encap_decap; > + /**< Pointer to encap/decap resource in cache. */ > struct ibv_flow *flow; /**< Installed flow. */ #ifdef > HAVE_IBV_FLOW_DV_SUPPORT > struct mlx5dv_flow_action_attr > actions[MLX5_DV_MAX_NUMBER_OF_ACTIONS]; > /**< Action list. */ > - struct ibv_flow_action *encap_decap_verbs_action; > - /**< Verbs encap/decap object. */ > #endif > int actions_n; /**< number of actions. */ }; diff --git > a/drivers/net/mlx5/mlx5_flow_dv.c b/drivers/net/mlx5/mlx5_flow_dv.c > index d1c811f..818b30c 100644 > --- a/drivers/net/mlx5/mlx5_flow_dv.c > +++ b/drivers/net/mlx5/mlx5_flow_dv.c > @@ -34,8 +34,6 @@ >=20 > #ifdef HAVE_IBV_FLOW_DV_SUPPORT >=20 > -#define MLX5_ENCAP_MAX_LEN 132 > - > /** > * Validate META item. > * > @@ -271,6 +269,77 @@ > return 0; > } >=20 > + > +/** > + * Find existing encap/decap resource or create and register a new one. > + * > + * @param dev[in, out] > + * Pointer to rte_eth_dev structure. > + * @param[in, out] resource > + * Pointer to encap/decap resource. > + * @parm[in, out] dev_flow > + * Pointer to the dev_flow. > + * @param[out] error > + * pointer to error structure. > + * > + * @return > + * 0 on success otherwise -errno and errno is set. > + */ > +static int > +flow_dv_encap_decap_resource_register > + (struct rte_eth_dev *dev, > + struct mlx5_flow_dv_encap_decap_resource > *resource, > + struct mlx5_flow *dev_flow, > + struct rte_flow_error *error) > +{ > + struct priv *priv =3D dev->data->dev_private; > + struct mlx5_flow_dv_encap_decap_resource *cache_resource; > + > + /* Lookup a matching resource from cache. */ > + LIST_FOREACH(cache_resource, &priv->encaps_decaps, next) { > + if (resource->reformat_type =3D=3D cache_resource- > >reformat_type && > + resource->ft_type =3D=3D cache_resource->ft_type && > + resource->size =3D=3D cache_resource->size && > + !memcmp((const void *)resource->buf, > + (const void *)cache_resource->buf, > + resource->size)) { > + DRV_LOG(DEBUG, "encap/decap resource %p: refcnt > %d++", > + (void *)cache_resource, > + rte_atomic32_read(&cache_resource- > >refcnt)); > + rte_atomic32_inc(&cache_resource->refcnt); > + dev_flow->dv.encap_decap =3D cache_resource; > + return 0; > + } > + } > + /* Register new encap/decap resource. */ > + cache_resource =3D rte_calloc(__func__, 1, sizeof(*cache_resource), > 0); > + if (!cache_resource) > + return rte_flow_error_set(error, ENOMEM, > + > RTE_FLOW_ERROR_TYPE_UNSPECIFIED, NULL, > + "cannot allocate resource > memory"); > + *cache_resource =3D *resource; > + cache_resource->verbs_action =3D > + mlx5_glue->dv_create_flow_action_packet_reformat > + (priv->ctx, cache_resource->size, > + (cache_resource->size ? cache_resource->buf : > NULL), > + cache_resource->reformat_type, > + cache_resource->ft_type); > + if (!cache_resource->verbs_action) { > + rte_free(cache_resource); > + return rte_flow_error_set(error, ENOMEM, > + > RTE_FLOW_ERROR_TYPE_UNSPECIFIED, > + NULL, "cannot create action"); > + } > + rte_atomic32_init(&cache_resource->refcnt); > + rte_atomic32_inc(&cache_resource->refcnt); > + LIST_INSERT_HEAD(&priv->encaps_decaps, cache_resource, next); > + dev_flow->dv.encap_decap =3D cache_resource; > + DRV_LOG(DEBUG, "new encap/decap resource %p: refcnt %d++", > + (void *)cache_resource, > + rte_atomic32_read(&cache_resource->refcnt)); > + return 0; > +} > + > /** > * Get the size of specific rte_flow_item_type > * > @@ -505,31 +574,33 @@ > * Pointer to rte_eth_dev structure. > * @param[in] action > * Pointer to action structure. > + * @param[in, out] dev_flow > + * Pointer to the mlx5_flow. > * @param[out] error > * Pointer to the error structure. > * > * @return > - * Pointer to action on success, NULL otherwise and rte_errno is set. > + * 0 on success, a negative errno value otherwise and rte_errno is set= . > */ > -static struct ibv_flow_action * > +static int > flow_dv_create_action_l2_encap(struct rte_eth_dev *dev, > const struct rte_flow_action *action, > + struct mlx5_flow *dev_flow, > struct rte_flow_error *error) { > - struct ibv_flow_action *verbs_action =3D NULL; > const struct rte_flow_item *encap_data; > const struct rte_flow_action_raw_encap *raw_encap_data; > - struct priv *priv =3D dev->data->dev_private; > - uint8_t buf[MLX5_ENCAP_MAX_LEN]; > - uint8_t *buf_ptr =3D buf; > - size_t size =3D 0; > - int convert_result =3D 0; > + struct mlx5_flow_dv_encap_decap_resource res =3D { > + .reformat_type =3D > + > MLX5DV_FLOW_ACTION_PACKET_REFORMAT_TYPE_L2_TO_L2_TU > NNEL, > + .ft_type =3D MLX5DV_FLOW_TABLE_TYPE_NIC_TX, > + }; >=20 > if (action->type =3D=3D RTE_FLOW_ACTION_TYPE_RAW_ENCAP) { > raw_encap_data =3D > (const struct rte_flow_action_raw_encap *)action- > >conf; > - buf_ptr =3D raw_encap_data->data; > - size =3D raw_encap_data->size; > + res.size =3D raw_encap_data->size; > + memcpy(res.buf, raw_encap_data->data, res.size); > } else { > if (action->type =3D=3D > RTE_FLOW_ACTION_TYPE_VXLAN_ENCAP) > encap_data =3D > @@ -539,19 +610,15 @@ > encap_data =3D > ((const struct rte_flow_action_nvgre_encap > *) > action->conf)->definition; > - convert_result =3D flow_dv_convert_encap_data(encap_data, > buf, > - &size, error); > - if (convert_result) > - return NULL; > + if (flow_dv_convert_encap_data(encap_data, res.buf, > + &res.size, error)) > + return -rte_errno; > } > - verbs_action =3D mlx5_glue- > >dv_create_flow_action_packet_reformat > - (priv->ctx, size, buf_ptr, > - > MLX5DV_FLOW_ACTION_PACKET_REFORMAT_TYPE_L2_TO_L2_TUNNEL, > - MLX5DV_FLOW_TABLE_TYPE_NIC_TX); > - if (!verbs_action) > - rte_flow_error_set(error, EINVAL, > RTE_FLOW_ERROR_TYPE_ACTION, > - NULL, "cannot create L2 encap action"); > - return verbs_action; > + if (flow_dv_encap_decap_resource_register(dev, &res, dev_flow, > error)) > + return rte_flow_error_set(error, EINVAL, > + RTE_FLOW_ERROR_TYPE_ACTION, > + NULL, "can't create L2 encap > action"); > + return 0; > } >=20 > /** > @@ -559,27 +626,31 @@ > * > * @param[in] dev > * Pointer to rte_eth_dev structure. > + * @param[in, out] dev_flow > + * Pointer to the mlx5_flow. > * @param[out] error > * Pointer to the error structure. > * > * @return > - * Pointer to action on success, NULL otherwise and rte_errno is set. > + * 0 on success, a negative errno value otherwise and rte_errno is set= . > */ > -static struct ibv_flow_action * > +static int > flow_dv_create_action_l2_decap(struct rte_eth_dev *dev, > + struct mlx5_flow *dev_flow, > struct rte_flow_error *error) { > - struct ibv_flow_action *verbs_action =3D NULL; > - struct priv *priv =3D dev->data->dev_private; > + struct mlx5_flow_dv_encap_decap_resource res =3D { > + .size =3D 0, > + .reformat_type =3D > + > MLX5DV_FLOW_ACTION_PACKET_REFORMAT_TYPE_L2_TUNNEL_T > O_L2, > + .ft_type =3D MLX5DV_FLOW_TABLE_TYPE_NIC_RX, > + }; >=20 > - verbs_action =3D mlx5_glue- > >dv_create_flow_action_packet_reformat > - (priv->ctx, 0, NULL, > - > MLX5DV_FLOW_ACTION_PACKET_REFORMAT_TYPE_L2_TUNNEL_TO_L2, > - MLX5DV_FLOW_TABLE_TYPE_NIC_RX); > - if (!verbs_action) > - rte_flow_error_set(error, EINVAL, > RTE_FLOW_ERROR_TYPE_ACTION, > - NULL, "cannot create L2 decap action"); > - return verbs_action; > + if (flow_dv_encap_decap_resource_register(dev, &res, dev_flow, > error)) > + return rte_flow_error_set(error, EINVAL, > + RTE_FLOW_ERROR_TYPE_ACTION, > + NULL, "can't create L2 decap > action"); > + return 0; > } >=20 > /** > @@ -589,41 +660,39 @@ > * Pointer to rte_eth_dev structure. > * @param[in] action > * Pointer to action structure. > + * @param[in, out] dev_flow > + * Pointer to the mlx5_flow. > * @param[in] attr > * Pointer to the flow attributes. > * @param[out] error > * Pointer to the error structure. > * > * @return > - * Pointer to action on success, NULL otherwise and rte_errno is set. > + * 0 on success, a negative errno value otherwise and rte_errno is set= . > */ > -static struct ibv_flow_action * > +static int > flow_dv_create_action_raw_encap(struct rte_eth_dev *dev, > const struct rte_flow_action *action, > + struct mlx5_flow *dev_flow, > const struct rte_flow_attr *attr, > struct rte_flow_error *error) > { > - struct ibv_flow_action *verbs_action =3D NULL; > const struct rte_flow_action_raw_encap *encap_data; > - struct priv *priv =3D dev->data->dev_private; > - enum mlx5dv_flow_action_packet_reformat_type reformat_type; > - enum mlx5dv_flow_table_type ft_type; > + struct mlx5_flow_dv_encap_decap_resource res; >=20 > encap_data =3D (const struct rte_flow_action_raw_encap *)action- > >conf; > - reformat_type =3D attr->egress ? > + res.size =3D encap_data->size; > + memcpy(res.buf, encap_data->data, res.size); > + res.reformat_type =3D attr->egress ? >=20 > MLX5DV_FLOW_ACTION_PACKET_REFORMAT_TYPE_L2_TO_L3_TU > NNEL : >=20 > MLX5DV_FLOW_ACTION_PACKET_REFORMAT_TYPE_L3_TUNNEL_T > O_L2; > - ft_type =3D attr->egress ? > - MLX5DV_FLOW_TABLE_TYPE_NIC_TX : > - MLX5DV_FLOW_TABLE_TYPE_NIC_RX; > - verbs_action =3D mlx5_glue- > >dv_create_flow_action_packet_reformat > - (priv->ctx, encap_data->size, > - (encap_data->size ? encap_data->data : > NULL), > - reformat_type, ft_type); > - if (!verbs_action) > - rte_flow_error_set(error, EINVAL, > RTE_FLOW_ERROR_TYPE_ACTION, > - NULL, "cannot create encap action"); > - return verbs_action; > + res.ft_type =3D attr->egress ? MLX5DV_FLOW_TABLE_TYPE_NIC_TX : > + MLX5DV_FLOW_TABLE_TYPE_NIC_RX; > + if (flow_dv_encap_decap_resource_register(dev, &res, dev_flow, > error)) > + return rte_flow_error_set(error, EINVAL, > + RTE_FLOW_ERROR_TYPE_ACTION, > + NULL, "can't create encap action"); > + return 0; > } >=20 > /** > @@ -1689,15 +1758,13 @@ > break; > case RTE_FLOW_ACTION_TYPE_VXLAN_ENCAP: > case RTE_FLOW_ACTION_TYPE_NVGRE_ENCAP: > + if (flow_dv_create_action_l2_encap(dev, action, > + dev_flow, error)) > + return -rte_errno; > dev_flow->dv.actions[actions_n].type =3D > MLX5DV_FLOW_ACTION_IBV_FLOW_ACTION; > dev_flow->dv.actions[actions_n].action =3D > - flow_dv_create_action_l2_encap(dev, > action, > - error); > - if (!(dev_flow->dv.actions[actions_n].action)) > - return -rte_errno; > - dev_flow->dv.encap_decap_verbs_action =3D > - dev_flow->dv.actions[actions_n].action; > + dev_flow->dv.encap_decap->verbs_action; > flow->actions |=3D action->type =3D=3D > RTE_FLOW_ACTION_TYPE_VXLAN_ENCAP ? > MLX5_FLOW_ACTION_VXLAN_ENCAP : > @@ -1706,14 +1773,12 @@ > break; > case RTE_FLOW_ACTION_TYPE_VXLAN_DECAP: > case RTE_FLOW_ACTION_TYPE_NVGRE_DECAP: > + if (flow_dv_create_action_l2_decap(dev, dev_flow, error)) > + return -rte_errno; > dev_flow->dv.actions[actions_n].type =3D > MLX5DV_FLOW_ACTION_IBV_FLOW_ACTION; > dev_flow->dv.actions[actions_n].action =3D > - flow_dv_create_action_l2_decap(dev, > error); > - if (!(dev_flow->dv.actions[actions_n].action)) > - return -rte_errno; > - dev_flow->dv.encap_decap_verbs_action =3D > - dev_flow->dv.actions[actions_n].action; > + dev_flow->dv.encap_decap->verbs_action; > flow->actions |=3D action->type =3D=3D > RTE_FLOW_ACTION_TYPE_VXLAN_DECAP ? > MLX5_FLOW_ACTION_VXLAN_DECAP : > @@ -1723,27 +1788,23 @@ > case RTE_FLOW_ACTION_TYPE_RAW_ENCAP: > /* Handle encap action with preceding decap */ > if (flow->actions & MLX5_FLOW_ACTION_RAW_DECAP) { > + if (flow_dv_create_action_raw_encap(dev, action, > + dev_flow, > + attr, error)) > + return -rte_errno; > dev_flow->dv.actions[actions_n].type =3D >=20 > MLX5DV_FLOW_ACTION_IBV_FLOW_ACTION; > dev_flow->dv.actions[actions_n].action =3D > - flow_dv_create_action_raw_encap > - (dev, action, > - attr, error); > - if (!(dev_flow->dv.actions[actions_n].action)) > - return -rte_errno; > - dev_flow->dv.encap_decap_verbs_action =3D > - dev_flow->dv.actions[actions_n].action; > + dev_flow->dv.encap_decap- > >verbs_action; > } else { > /* Handle encap action without preceding decap */ > + if (flow_dv_create_action_l2_encap(dev, action, > + dev_flow, error)) > + return -rte_errno; > dev_flow->dv.actions[actions_n].type =3D >=20 > MLX5DV_FLOW_ACTION_IBV_FLOW_ACTION; > dev_flow->dv.actions[actions_n].action =3D > - flow_dv_create_action_l2_encap > - (dev, action, error); > - if (!(dev_flow->dv.actions[actions_n].action)) > - return -rte_errno; > - dev_flow->dv.encap_decap_verbs_action =3D > - dev_flow->dv.actions[actions_n].action; > + dev_flow->dv.encap_decap- > >verbs_action; > } > flow->actions |=3D MLX5_FLOW_ACTION_RAW_ENCAP; > actions_n++; > @@ -1756,15 +1817,13 @@ > } > /* Handle decap action only if it isn't followed by encap */ > if (action_ptr->type !=3D > RTE_FLOW_ACTION_TYPE_RAW_ENCAP) { > + if (flow_dv_create_action_l2_decap(dev, dev_flow, > + error)) > + return -rte_errno; > dev_flow->dv.actions[actions_n].type =3D >=20 > MLX5DV_FLOW_ACTION_IBV_FLOW_ACTION; > dev_flow->dv.actions[actions_n].action =3D > - > flow_dv_create_action_l2_decap(dev, > - error); > - if (!(dev_flow->dv.actions[actions_n].action)) > - return -rte_errno; > - dev_flow->dv.encap_decap_verbs_action =3D > - dev_flow->dv.actions[actions_n].action; > + dev_flow->dv.encap_decap- > >verbs_action; > actions_n++; > } > /* If decap is followed by encap, handle it at encap case. */ > @@ -2074,6 +2133,37 @@ } >=20 > /** > + * Release an encap/decap resource. > + * > + * @param flow > + * Pointer to mlx5_flow. > + * > + * @return > + * 1 while a reference on it exists, 0 when freed. > + */ > +static int > +flow_dv_encap_decap_resource_release(struct mlx5_flow *flow) { > + struct mlx5_flow_dv_encap_decap_resource *cache_resource =3D > + flow->dv.encap_decap; > + > + assert(cache_resource->verbs_action); > + DRV_LOG(DEBUG, "encap/decap resource %p: refcnt %d--", > + (void *)cache_resource, > + rte_atomic32_read(&cache_resource->refcnt)); > + if (rte_atomic32_dec_and_test(&cache_resource->refcnt)) { > + claim_zero(mlx5_glue->destroy_flow_action > + (cache_resource->verbs_action)); > + LIST_REMOVE(cache_resource, next); > + rte_free(cache_resource); > + DRV_LOG(DEBUG, "encap/decap resource %p: removed", > + cache_resource); > + return 0; > + } > + return 1; > +} > + > +/** > * Remove the flow from the NIC but keeps it in memory. > * > * @param[in] dev > @@ -2128,11 +2218,8 @@ > LIST_REMOVE(dev_flow, next); > if (dev_flow->dv.matcher) > flow_dv_matcher_release(dev, dev_flow); > - if (dev_flow->dv.encap_decap_verbs_action) { > - claim_zero(mlx5_glue->destroy_flow_action > - (dev_flow->dv.encap_decap_verbs_action)); > - dev_flow->dv.encap_decap_verbs_action =3D NULL; > - } > + if (dev_flow->dv.encap_decap) > + > flow_dv_encap_decap_resource_release(dev_flow); > rte_free(dev_flow); > } > } > -- > 1.8.3.1