From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from EUR01-DB5-obe.outbound.protection.outlook.com (mail-eopbgr150051.outbound.protection.outlook.com [40.107.15.51]) by dpdk.org (Postfix) with ESMTP id 90F781B3D8 for ; Thu, 28 Mar 2019 17:32:59 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Mellanox.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=K+zqPVmFPhlMY0o7ELrD2ESN/S3aS0wHdD/ZxMylFRo=; b=bhkUOzEQrVGxFiW/U97XMEvDy32PavrSDbNx/rBOMe5f64izM5cbwZi3JxYbdR6031pZQmXsmj4TuYyLicWaJg0OTBNhETy8DVXS5+Xk7GkXZPzl542e94E0RpaO2IwI+Q8ykyWgfQktei7OxW+7QaffA3oXbarG1dcfqsQmzuw= Received: from AM4PR05MB3425.eurprd05.prod.outlook.com (10.171.190.15) by AM4PR05MB3330.eurprd05.prod.outlook.com (10.171.191.28) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.1730.18; Thu, 28 Mar 2019 16:32:58 +0000 Received: from AM4PR05MB3425.eurprd05.prod.outlook.com ([fe80::5df0:22de:97f0:3827]) by AM4PR05MB3425.eurprd05.prod.outlook.com ([fe80::5df0:22de:97f0:3827%4]) with mapi id 15.20.1750.014; Thu, 28 Mar 2019 16:32:58 +0000 From: Ori Kam To: Matan Azrad , Yongseok Koh , Shahaf Shuler CC: "dev@dpdk.org" , Ori Kam , Slava Ovsiienko Thread-Topic: [PATCH v2 3/3] net/mlx5: add jump action support for NIC Thread-Index: AQHU5YPnhYdRQOEA1EaodH+rzfI8Pw== Date: Thu, 28 Mar 2019 16:32:58 +0000 Message-ID: <1553790741-69362-4-git-send-email-orika@mellanox.com> References: <1553096315-91832-1-git-send-email-orika@mellanox.com> <1553790741-69362-1-git-send-email-orika@mellanox.com> In-Reply-To: <1553790741-69362-1-git-send-email-orika@mellanox.com> Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-clientproxiedby: LO2P265CA0362.GBRP265.PROD.OUTLOOK.COM (2603:10a6:600:a3::14) To AM4PR05MB3425.eurprd05.prod.outlook.com (2603:10a6:205:b::15) authentication-results: spf=none (sender IP is ) smtp.mailfrom=orika@mellanox.com; x-ms-exchange-messagesentrepresentingtype: 1 x-mailer: git-send-email 1.8.3.1 x-originating-ip: [37.142.13.130] x-ms-publictraffictype: Email x-ms-office365-filtering-correlation-id: e40f3198-5b50-4c57-4d4c-08d6b39b0946 x-ms-office365-filtering-ht: Tenant x-microsoft-antispam: BCL:0; PCL:0; RULEID:(2390118)(7020095)(4652040)(8989299)(4534185)(4627221)(201703031133081)(201702281549075)(8990200)(5600127)(711020)(4605104)(4618075)(2017052603328)(7153060)(7193020); SRVR:AM4PR05MB3330; x-ms-traffictypediagnostic: AM4PR05MB3330: x-microsoft-antispam-prvs: x-forefront-prvs: 0990C54589 x-forefront-antispam-report: SFV:NSPM; SFS:(10009020)(136003)(366004)(376002)(346002)(39860400002)(396003)(199004)(189003)(256004)(4326008)(25786009)(2616005)(2906002)(68736007)(26005)(71200400001)(53946003)(476003)(446003)(6512007)(8936002)(71190400001)(14454004)(8676002)(102836004)(81156014)(3846002)(386003)(50226002)(11346002)(81166006)(36756003)(6116002)(30864003)(6436002)(86362001)(6636002)(99286004)(105586002)(5660300002)(6486002)(54906003)(97736004)(316002)(107886003)(52116002)(106356001)(66066001)(486006)(53936002)(478600001)(7736002)(6506007)(14444005)(4720700003)(76176011)(305945005)(110136005)(186003); DIR:OUT; SFP:1101; SCL:1; SRVR:AM4PR05MB3330; H:AM4PR05MB3425.eurprd05.prod.outlook.com; FPR:; SPF:None; LANG:en; PTR:InfoNoRecords; A:1; MX:1; received-spf: None (protection.outlook.com: mellanox.com does not designate permitted sender hosts) x-ms-exchange-senderadcheck: 1 x-microsoft-antispam-message-info: lEiUhC/Xck30/v9nNoITmcpSSt+OqOt1UOyQiBp+PmF6Ci5GhI73mdKTOjwgHK878sLs5eX307tgEB+FTdkRWqxeQa1Vp3CceyQuAbwhxaRGLbn+SukYEv9VDrou4sSBTqk8bA2boxlvKubsRyL6UgJTdac8qoG33ZiXm88QLfolvANihL5RV6SWyvl9EPo8jVY1F6FefPdYLjTbiSNDAmP+H33LLFkw7DftixAMlDr/0XqKXNvxQf7dh02cwvHuYzxf38m6T5sMbiTRD14HAT6j5ECAL0gaxhet+VmSSuVnb9JAsHDYyScqczZ8twKD5tGqlIChxW/u1Y1rUc113/yF3aOIOsdO7ENmrdmMCiD2hD/Oa8iIbWWO50bAmS4wsVyuIOjFpzyqJdfnZUyUezCHg9gKg6EDMhqNhaxJmkU= Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 X-OriginatorOrg: Mellanox.com X-MS-Exchange-CrossTenant-Network-Message-Id: e40f3198-5b50-4c57-4d4c-08d6b39b0946 X-MS-Exchange-CrossTenant-originalarrivaltime: 28 Mar 2019 16:32:58.1282 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Hosted X-MS-Exchange-CrossTenant-id: a652971c-7d2e-4d9b-a6a4-d149256f461b X-MS-Exchange-CrossTenant-mailboxtype: HOSTED X-MS-Exchange-Transport-CrossTenantHeadersStamped: AM4PR05MB3330 Subject: [dpdk-dev] [PATCH v2 3/3] net/mlx5: add jump action support for NIC X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 28 Mar 2019 16:33:00 -0000 When using Direct Rules we can add actions to jump between tables. This is extra useful since rule insertion rate is much higher on other tables compared to table zero. if no group is selected the rule is added to group 0. Signed-off-by: Ori Kam --- drivers/net/mlx5/mlx5.h | 6 + drivers/net/mlx5/mlx5_flow.h | 15 ++- drivers/net/mlx5/mlx5_flow_dv.c | 278 +++++++++++++++++++++++++++++++++++-= ---- drivers/net/mlx5/mlx5_glue.c | 13 ++ drivers/net/mlx5/mlx5_glue.h | 1 + 5 files changed, 282 insertions(+), 31 deletions(-) diff --git a/drivers/net/mlx5/mlx5.h b/drivers/net/mlx5/mlx5.h index 73f6f0d..a3d5f8e 100644 --- a/drivers/net/mlx5/mlx5.h +++ b/drivers/net/mlx5/mlx5.h @@ -279,6 +279,12 @@ struct mlx5_priv { LIST_HEAD(encap_decap, mlx5_flow_dv_encap_decap_resource) encaps_decaps; LIST_HEAD(modify_cmd, mlx5_flow_dv_modify_hdr_resource) modify_cmds; LIST_HEAD(tag, mlx5_flow_dv_tag_resource) tags; + LIST_HEAD(jump, mlx5_flow_dv_jump_tbl_resource) jump_tbl; + /* Pointer to next element. */ + rte_atomic32_t refcnt; /**< Reference counter. */ + struct ibv_flow_action *verbs_action; + /**< Verbs modify header action object. */ + uint8_t ft_type; /**< Flow table type, Rx or Tx. */ /* Tags resources cache. */ uint32_t link_speed_capa; /* Link speed capabilities. */ struct mlx5_xstats_ctrl xstats_ctrl; /* Extended stats control. */ diff --git a/drivers/net/mlx5/mlx5_flow.h b/drivers/net/mlx5/mlx5_flow.h index 8ba37a0..622e305 100644 --- a/drivers/net/mlx5/mlx5_flow.h +++ b/drivers/net/mlx5/mlx5_flow.h @@ -115,7 +115,8 @@ #define MLX5_FLOW_ACTION_RAW_DECAP (1u << 27) =20 #define MLX5_FLOW_FATE_ACTIONS \ - (MLX5_FLOW_ACTION_DROP | MLX5_FLOW_ACTION_QUEUE | MLX5_FLOW_ACTION_RSS) + (MLX5_FLOW_ACTION_DROP | MLX5_FLOW_ACTION_QUEUE | \ + MLX5_FLOW_ACTION_RSS | MLX5_FLOW_ACTION_JUMP) =20 #define MLX5_FLOW_ENCAP_ACTIONS (MLX5_FLOW_ACTION_VXLAN_ENCAP | \ MLX5_FLOW_ACTION_NVGRE_ENCAP | \ @@ -250,6 +251,16 @@ struct mlx5_flow_dv_modify_hdr_resource { /**< Modification actions. */ }; =20 +/* Jump action resource structure. */ +struct mlx5_flow_dv_jump_tbl_resource { + LIST_ENTRY(mlx5_flow_dv_jump_tbl_resource) next; + /* Pointer to next element. */ + rte_atomic32_t refcnt; /**< Reference counter. */ + void *action; /**< Pointer to the rdma core action. */ + uint8_t ft_type; /**< Flow table type, Rx or Tx. */ + struct mlx5_flow_tbl_resource *tbl; /**< The target table. */ +}; + /* * Max number of actions per DV flow. * See CREATE_FLOW_MAX_FLOW_ACTIONS_SUPPORTED @@ -270,6 +281,8 @@ struct mlx5_flow_dv { struct mlx5_flow_dv_modify_hdr_resource *modify_hdr; /**< Pointer to modify header resource in cache. */ struct ibv_flow *flow; /**< Installed flow. */ + struct mlx5_flow_dv_jump_tbl_resource *jump; + /**< Pointer to the jump action resource. */ #ifdef HAVE_IBV_FLOW_DV_SUPPORT void *actions[MLX5_DV_MAX_NUMBER_OF_ACTIONS]; /**< Action list. */ diff --git a/drivers/net/mlx5/mlx5_flow_dv.c b/drivers/net/mlx5/mlx5_flow_d= v.c index 6e4f6c4..6997dc6 100644 --- a/drivers/net/mlx5/mlx5_flow_dv.c +++ b/drivers/net/mlx5/mlx5_flow_dv.c @@ -861,6 +861,68 @@ struct field_modify_info modify_tcp[] =3D { } =20 /** + * Find existing table jump resource or create and register a new one. + * + * @param dev[in, out] + * Pointer to rte_eth_dev structure. + * @param[in, out] resource + * Pointer to jump table resource. + * @parm[in, out] dev_flow + * Pointer to the dev_flow. + * @param[out] error + * pointer to error structure. + * + * @return + * 0 on success otherwise -errno and errno is set. + */ +static int +flow_dv_jump_tbl_resource_register + (struct rte_eth_dev *dev, + struct mlx5_flow_dv_jump_tbl_resource *resource, + struct mlx5_flow *dev_flow, + struct rte_flow_error *error) +{ + struct mlx5_priv *priv =3D dev->data->dev_private; + struct mlx5_flow_dv_jump_tbl_resource *cache_resource; + + /* Lookup a matching resource from cache. */ + LIST_FOREACH(cache_resource, &priv->jump_tbl, next) { + if (resource->tbl =3D=3D cache_resource->tbl) { + DRV_LOG(DEBUG, "jump table resource resource %p: refcnt %d++", + (void *)cache_resource, + rte_atomic32_read(&cache_resource->refcnt)); + rte_atomic32_inc(&cache_resource->refcnt); + dev_flow->dv.jump =3D cache_resource; + return 0; + } + } + /* Register new jump table resource. */ + cache_resource =3D rte_calloc(__func__, 1, sizeof(*cache_resource), 0); + if (!cache_resource) + return rte_flow_error_set(error, ENOMEM, + RTE_FLOW_ERROR_TYPE_UNSPECIFIED, NULL, + "cannot allocate resource memory"); + *cache_resource =3D *resource; + cache_resource->action =3D + mlx5_glue->dr_create_flow_action_dest_flow_tbl + (resource->tbl->obj); + if (!cache_resource->action) { + rte_free(cache_resource); + return rte_flow_error_set(error, ENOMEM, + RTE_FLOW_ERROR_TYPE_UNSPECIFIED, + NULL, "cannot create action"); + } + rte_atomic32_init(&cache_resource->refcnt); + rte_atomic32_inc(&cache_resource->refcnt); + LIST_INSERT_HEAD(&priv->jump_tbl, cache_resource, next); + dev_flow->dv.jump =3D cache_resource; + DRV_LOG(DEBUG, "new jump table resource %p: refcnt %d++", + (void *)cache_resource, + rte_atomic32_read(&cache_resource->refcnt)); + return 0; +} + +/** * Get the size of specific rte_flow_item_type * * @param[in] item_type @@ -1423,6 +1485,37 @@ struct field_modify_info modify_tcp[] =3D { } =20 /** + * Validate jump action. + * + * @param[in] action + * Pointer to the modify action. + * @param[in] group + * The group of the current flow. + * @param[out] error + * Pointer to error structure. + * + * @return + * 0 on success, a negative errno value otherwise and rte_errno is set. + */ +static int +flow_dv_validate_action_jump(const struct rte_flow_action *action, + uint32_t group, + struct rte_flow_error *error) +{ + if (action->type !=3D RTE_FLOW_ACTION_TYPE_JUMP && !action->conf) + return rte_flow_error_set(error, EINVAL, + RTE_FLOW_ERROR_TYPE_ACTION_CONF, + NULL, "action configuration not set"); + if (group >=3D ((const struct rte_flow_action_jump *)action->conf)->group= ) + return rte_flow_error_set(error, EINVAL, + RTE_FLOW_ERROR_TYPE_ACTION, NULL, + "target group must be higher then" + " the current flow group"); + return 0; +} + + +/** * Find existing modify-header resource or create and register a new one. * * @param dev[in, out] @@ -1605,7 +1698,7 @@ struct field_modify_info modify_tcp[] =3D { struct mlx5_priv *priv =3D dev->data->dev_private; uint32_t priority_max =3D priv->config.flow_prio - 1; =20 -#ifdef HAVE_MLX5DV_DR +#ifndef HAVE_MLX5DV_DR if (attributes->group) return rte_flow_error_set(error, ENOTSUP, RTE_FLOW_ERROR_TYPE_ATTR_GROUP, @@ -1977,6 +2070,14 @@ struct field_modify_info modify_tcp[] =3D { MLX5_FLOW_ACTION_SET_TTL : MLX5_FLOW_ACTION_DEC_TTL; break; + case RTE_FLOW_ACTION_TYPE_JUMP: + ret =3D flow_dv_validate_action_jump(actions, + attr->group, error); + if (ret) + return ret; + ++actions_n; + action_flags |=3D MLX5_FLOW_ACTION_JUMP; + break; default: return rte_flow_error_set(error, ENOTSUP, RTE_FLOW_ERROR_TYPE_ACTION, @@ -2756,6 +2857,73 @@ struct field_modify_info modify_tcp[] =3D { return match_criteria_enable; } =20 + +/** + * Get a flow table. + * + * @param dev[in, out] + * Pointer to rte_eth_dev structure. + * @param[in] table_id + * Table id to use. + * @param[in] egress + * Direction of the table. + * @param[out] error + * pointer to error structure. + * + * @return + * Returns tables resource based on the index, NULL in case of failed. + */ +static struct mlx5_flow_tbl_resource * +flow_dv_tbl_resource_get(struct rte_eth_dev *dev, + uint32_t table_id, uint8_t egress, + struct rte_flow_error *error) +{ + struct mlx5_priv *priv =3D dev->data->dev_private; + struct mlx5_flow_tbl_resource *tbl; + + if (egress) { + tbl =3D &priv->tx_tbl[table_id]; + if (!tbl->obj) + tbl->obj =3D mlx5_glue->dr_create_flow_tbl + (priv->tx_ns, table_id); + } else { + tbl =3D &priv->rx_tbl[table_id]; + if (!tbl->obj) + tbl->obj =3D mlx5_glue->dr_create_flow_tbl + (priv->rx_ns, table_id); + } + if (!tbl->obj) { + rte_flow_error_set(error, ENOMEM, + RTE_FLOW_ERROR_TYPE_UNSPECIFIED, + NULL, "cannot create table"); + return NULL; + } + rte_atomic32_inc(&tbl->refcnt); + return tbl; +} + +/** + * Release a flow table. + * + * @param[in] tbl + * Table resource to be released. + * + * @return + * Returns 0 if table was released, else return 1; + */ +static int +flow_dv_tbl_resource_release(struct mlx5_flow_tbl_resource *tbl) +{ + if (!tbl) + return 0; + if (rte_atomic32_dec_and_test(&tbl->refcnt)) { + mlx5_glue->dr_destroy_flow_tbl(tbl->obj); + tbl->obj =3D NULL; + return 0; + } + return 1; +} + /** * Register the flow matcher. * @@ -2784,6 +2952,9 @@ struct field_modify_info modify_tcp[] =3D { .match_mask =3D (void *)&matcher->mask, }; struct mlx5_flow_tbl_resource *tbl =3D NULL; +#ifndef HAVE_MLX5DV_DR + struct mlx5_flow_tbl_resource tbl_tmp; +#endif =20 /* Lookup from cache. */ LIST_FOREACH(cache_matcher, &priv->matchers, next) { @@ -2805,33 +2976,24 @@ struct field_modify_info modify_tcp[] =3D { return 0; } } -#ifdef HAVE_MLX5DV_DR - if (matcher->egress) { - tbl =3D &priv->tx_tbl[matcher->group]; - if (!tbl->obj) - tbl->obj =3D mlx5_glue->dr_create_flow_tbl - (priv->tx_ns, - matcher->group * MLX5_GROUP_FACTOR); - } else { - tbl =3D &priv->rx_tbl[matcher->group]; - if (!tbl->obj) - tbl->obj =3D mlx5_glue->dr_create_flow_tbl - (priv->rx_ns, - matcher->group * MLX5_GROUP_FACTOR); - } - if (!tbl->obj) - return rte_flow_error_set(error, ENOMEM, - RTE_FLOW_ERROR_TYPE_UNSPECIFIED, - NULL, "cannot create table"); - - rte_atomic32_inc(&tbl->refcnt); -#endif /* Register new matcher. */ cache_matcher =3D rte_calloc(__func__, 1, sizeof(*cache_matcher), 0); if (!cache_matcher) return rte_flow_error_set(error, ENOMEM, RTE_FLOW_ERROR_TYPE_UNSPECIFIED, NULL, "cannot allocate matcher memory"); +#ifdef HAVE_MLX5DV_DR + tbl =3D flow_dv_tbl_resource_get(dev, matcher->group * MLX5_GROUP_FACTOR, + matcher->egress, error); + if (!tbl) { + rte_free(cache_matcher); + return rte_flow_error_set(error, ENOMEM, + RTE_FLOW_ERROR_TYPE_UNSPECIFIED, + NULL, "cannot create table"); + } +#else + tbl =3D &tbl_tmp; +#endif *cache_matcher =3D *matcher; dv_attr.match_criteria_enable =3D flow_dv_matcher_enable(cache_matcher->mask.buf); @@ -2844,10 +3006,7 @@ struct field_modify_info modify_tcp[] =3D { if (!cache_matcher->matcher_object) { rte_free(cache_matcher); #ifdef HAVE_MLX5DV_DR - if (rte_atomic32_dec_and_test(&tbl->refcnt)) { - mlx5_glue->dr_destroy_flow_tbl(tbl->obj); - tbl->obj =3D NULL; - } + flow_dv_tbl_resource_release(tbl); #endif return rte_flow_error_set(error, ENOMEM, RTE_FLOW_ERROR_TYPE_UNSPECIFIED, @@ -3033,6 +3192,9 @@ struct field_modify_info modify_tcp[] =3D { const struct rte_flow_action *action =3D actions; const struct rte_flow_action_count *count =3D action->conf; const uint8_t *rss_key; + const struct rte_flow_action_jump *jump_data; + struct mlx5_flow_dv_jump_tbl_resource jump_tbl_resource; + struct mlx5_flow_tbl_resource *tbl; =20 switch (actions->type) { case RTE_FLOW_ACTION_TYPE_VOID: @@ -3171,6 +3333,31 @@ struct field_modify_info modify_tcp[] =3D { /* If decap is followed by encap, handle it at encap. */ action_flags |=3D MLX5_FLOW_ACTION_RAW_DECAP; break; + case RTE_FLOW_ACTION_TYPE_JUMP: + jump_data =3D action->conf; + tbl =3D flow_dv_tbl_resource_get(dev, jump_data->group * + MLX5_GROUP_FACTOR, + attr->egress, error); + if (!tbl) + return rte_flow_error_set + (error, errno, + RTE_FLOW_ERROR_TYPE_ACTION, + NULL, + "cannot create jump action."); + jump_tbl_resource.tbl =3D tbl; + if (flow_dv_jump_tbl_resource_register + (dev, &jump_tbl_resource, dev_flow, error)) { + flow_dv_tbl_resource_release(tbl); + return rte_flow_error_set + (error, errno, + RTE_FLOW_ERROR_TYPE_ACTION, + NULL, + "cannot create jump action."); + } + dev_flow->dv.actions[actions_n++] =3D + dev_flow->dv.jump->action; + action_flags |=3D MLX5_FLOW_ACTION_JUMP; + break; case RTE_FLOW_ACTION_TYPE_SET_MAC_SRC: case RTE_FLOW_ACTION_TYPE_SET_MAC_DST: if (flow_dv_convert_action_modify_mac(&res, actions, @@ -3503,10 +3690,7 @@ struct field_modify_info modify_tcp[] =3D { tbl =3D &priv->tx_tbl[matcher->group]; else tbl =3D &priv->rx_tbl[matcher->group]; - if (rte_atomic32_dec_and_test(&tbl->refcnt)) { - mlx5_glue->dr_destroy_flow_tbl(tbl->obj); - tbl->obj =3D NULL; - } + flow_dv_tbl_resource_release(tbl); rte_free(matcher); DRV_LOG(DEBUG, "port %u matcher %p: removed", dev->data->port_id, (void *)matcher); @@ -3547,6 +3731,38 @@ struct field_modify_info modify_tcp[] =3D { } =20 /** + * Release an jump to table action resource. + * + * @param flow + * Pointer to mlx5_flow. + * + * @return + * 1 while a reference on it exists, 0 when freed. + */ +static int +flow_dv_jump_tbl_resource_release(struct mlx5_flow *flow) +{ + struct mlx5_flow_dv_jump_tbl_resource *cache_resource =3D + flow->dv.jump; + + assert(cache_resource->action); + DRV_LOG(DEBUG, "jump table resource %p: refcnt %d--", + (void *)cache_resource, + rte_atomic32_read(&cache_resource->refcnt)); + if (rte_atomic32_dec_and_test(&cache_resource->refcnt)) { + claim_zero(mlx5_glue->destroy_flow_action + (cache_resource->action)); + LIST_REMOVE(cache_resource, next); + flow_dv_tbl_resource_release(cache_resource->tbl); + rte_free(cache_resource); + DRV_LOG(DEBUG, "jump table resource %p: removed", + (void *)cache_resource); + return 0; + } + return 1; +} + +/** * Release a modify-header resource. * * @param flow @@ -3642,6 +3858,8 @@ struct field_modify_info modify_tcp[] =3D { flow_dv_encap_decap_resource_release(dev_flow); if (dev_flow->dv.modify_hdr) flow_dv_modify_hdr_resource_release(dev_flow); + if (dev_flow->dv.jump) + flow_dv_jump_tbl_resource_release(dev_flow); rte_free(dev_flow); } } diff --git a/drivers/net/mlx5/mlx5_glue.c b/drivers/net/mlx5/mlx5_glue.c index f509f85..fca34ca 100644 --- a/drivers/net/mlx5/mlx5_glue.c +++ b/drivers/net/mlx5/mlx5_glue.c @@ -370,6 +370,17 @@ } =20 static void * +mlx5_glue_dr_create_flow_action_dest_flow_tbl(void *tbl) +{ +#ifdef HAVE_MLX5DV_DR + return mlx5dv_dr_create_action_dest_flow_table(tbl); +#else + (void)tbl; + return NULL; +#endif +} + +static void * mlx5_glue_dr_create_flow_tbl(void *ns, uint32_t level) { #ifdef HAVE_MLX5DV_DR @@ -833,6 +844,8 @@ .get_async_event =3D mlx5_glue_get_async_event, .port_state_str =3D mlx5_glue_port_state_str, .cq_ex_to_cq =3D mlx5_glue_cq_ex_to_cq, + .dr_create_flow_action_dest_flow_tbl =3D + mlx5_glue_dr_create_flow_action_dest_flow_tbl, .dr_create_flow_tbl =3D mlx5_glue_dr_create_flow_tbl, .dr_destroy_flow_tbl =3D mlx5_glue_dr_destroy_flow_tbl, .dr_create_ns =3D mlx5_glue_dr_create_ns, diff --git a/drivers/net/mlx5/mlx5_glue.h b/drivers/net/mlx5/mlx5_glue.h index 7115575..16d5dd6 100644 --- a/drivers/net/mlx5/mlx5_glue.h +++ b/drivers/net/mlx5/mlx5_glue.h @@ -145,6 +145,7 @@ struct mlx5_glue { struct ibv_async_event *event); const char *(*port_state_str)(enum ibv_port_state port_state); struct ibv_cq *(*cq_ex_to_cq)(struct ibv_cq_ex *cq); + void *(*dr_create_flow_action_dest_flow_tbl)(void *tbl); void *(*dr_create_flow_tbl)(void *ns, uint32_t level); int (*dr_destroy_flow_tbl)(void *tbl); void *(*dr_create_ns)(struct ibv_context *ctx, --=20 1.8.3.1 From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from dpdk.org (dpdk.org [92.243.14.124]) by dpdk.space (Postfix) with ESMTP id 599DFA0679 for ; Thu, 28 Mar 2019 17:33:29 +0100 (CET) Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id 35BDE1B3FD; Thu, 28 Mar 2019 17:33:06 +0100 (CET) Received: from EUR01-DB5-obe.outbound.protection.outlook.com (mail-eopbgr150051.outbound.protection.outlook.com [40.107.15.51]) by dpdk.org (Postfix) with ESMTP id 90F781B3D8 for ; Thu, 28 Mar 2019 17:32:59 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Mellanox.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=K+zqPVmFPhlMY0o7ELrD2ESN/S3aS0wHdD/ZxMylFRo=; b=bhkUOzEQrVGxFiW/U97XMEvDy32PavrSDbNx/rBOMe5f64izM5cbwZi3JxYbdR6031pZQmXsmj4TuYyLicWaJg0OTBNhETy8DVXS5+Xk7GkXZPzl542e94E0RpaO2IwI+Q8ykyWgfQktei7OxW+7QaffA3oXbarG1dcfqsQmzuw= Received: from AM4PR05MB3425.eurprd05.prod.outlook.com (10.171.190.15) by AM4PR05MB3330.eurprd05.prod.outlook.com (10.171.191.28) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.1730.18; Thu, 28 Mar 2019 16:32:58 +0000 Received: from AM4PR05MB3425.eurprd05.prod.outlook.com ([fe80::5df0:22de:97f0:3827]) by AM4PR05MB3425.eurprd05.prod.outlook.com ([fe80::5df0:22de:97f0:3827%4]) with mapi id 15.20.1750.014; Thu, 28 Mar 2019 16:32:58 +0000 From: Ori Kam To: Matan Azrad , Yongseok Koh , Shahaf Shuler CC: "dev@dpdk.org" , Ori Kam , Slava Ovsiienko Thread-Topic: [PATCH v2 3/3] net/mlx5: add jump action support for NIC Thread-Index: AQHU5YPnhYdRQOEA1EaodH+rzfI8Pw== Date: Thu, 28 Mar 2019 16:32:58 +0000 Message-ID: <1553790741-69362-4-git-send-email-orika@mellanox.com> References: <1553096315-91832-1-git-send-email-orika@mellanox.com> <1553790741-69362-1-git-send-email-orika@mellanox.com> In-Reply-To: <1553790741-69362-1-git-send-email-orika@mellanox.com> Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-clientproxiedby: LO2P265CA0362.GBRP265.PROD.OUTLOOK.COM (2603:10a6:600:a3::14) To AM4PR05MB3425.eurprd05.prod.outlook.com (2603:10a6:205:b::15) authentication-results: spf=none (sender IP is ) smtp.mailfrom=orika@mellanox.com; x-ms-exchange-messagesentrepresentingtype: 1 x-mailer: git-send-email 1.8.3.1 x-originating-ip: [37.142.13.130] x-ms-publictraffictype: Email x-ms-office365-filtering-correlation-id: e40f3198-5b50-4c57-4d4c-08d6b39b0946 x-ms-office365-filtering-ht: Tenant x-microsoft-antispam: BCL:0; PCL:0; RULEID:(2390118)(7020095)(4652040)(8989299)(4534185)(4627221)(201703031133081)(201702281549075)(8990200)(5600127)(711020)(4605104)(4618075)(2017052603328)(7153060)(7193020); SRVR:AM4PR05MB3330; x-ms-traffictypediagnostic: AM4PR05MB3330: x-microsoft-antispam-prvs: x-forefront-prvs: 0990C54589 x-forefront-antispam-report: SFV:NSPM; SFS:(10009020)(136003)(366004)(376002)(346002)(39860400002)(396003)(199004)(189003)(256004)(4326008)(25786009)(2616005)(2906002)(68736007)(26005)(71200400001)(53946003)(476003)(446003)(6512007)(8936002)(71190400001)(14454004)(8676002)(102836004)(81156014)(3846002)(386003)(50226002)(11346002)(81166006)(36756003)(6116002)(30864003)(6436002)(86362001)(6636002)(99286004)(105586002)(5660300002)(6486002)(54906003)(97736004)(316002)(107886003)(52116002)(106356001)(66066001)(486006)(53936002)(478600001)(7736002)(6506007)(14444005)(4720700003)(76176011)(305945005)(110136005)(186003); DIR:OUT; SFP:1101; SCL:1; SRVR:AM4PR05MB3330; H:AM4PR05MB3425.eurprd05.prod.outlook.com; FPR:; SPF:None; LANG:en; PTR:InfoNoRecords; A:1; MX:1; received-spf: None (protection.outlook.com: mellanox.com does not designate permitted sender hosts) x-ms-exchange-senderadcheck: 1 x-microsoft-antispam-message-info: lEiUhC/Xck30/v9nNoITmcpSSt+OqOt1UOyQiBp+PmF6Ci5GhI73mdKTOjwgHK878sLs5eX307tgEB+FTdkRWqxeQa1Vp3CceyQuAbwhxaRGLbn+SukYEv9VDrou4sSBTqk8bA2boxlvKubsRyL6UgJTdac8qoG33ZiXm88QLfolvANihL5RV6SWyvl9EPo8jVY1F6FefPdYLjTbiSNDAmP+H33LLFkw7DftixAMlDr/0XqKXNvxQf7dh02cwvHuYzxf38m6T5sMbiTRD14HAT6j5ECAL0gaxhet+VmSSuVnb9JAsHDYyScqczZ8twKD5tGqlIChxW/u1Y1rUc113/yF3aOIOsdO7ENmrdmMCiD2hD/Oa8iIbWWO50bAmS4wsVyuIOjFpzyqJdfnZUyUezCHg9gKg6EDMhqNhaxJmkU= Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 X-OriginatorOrg: Mellanox.com X-MS-Exchange-CrossTenant-Network-Message-Id: e40f3198-5b50-4c57-4d4c-08d6b39b0946 X-MS-Exchange-CrossTenant-originalarrivaltime: 28 Mar 2019 16:32:58.1282 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Hosted X-MS-Exchange-CrossTenant-id: a652971c-7d2e-4d9b-a6a4-d149256f461b X-MS-Exchange-CrossTenant-mailboxtype: HOSTED X-MS-Exchange-Transport-CrossTenantHeadersStamped: AM4PR05MB3330 Subject: [dpdk-dev] [PATCH v2 3/3] net/mlx5: add jump action support for NIC X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" Message-ID: <20190328163258.HCVQP2XEz_TFg_h-SvvgRvvWm_MuqrUMuXurZuXD7sM@z> When using Direct Rules we can add actions to jump between tables. This is extra useful since rule insertion rate is much higher on other tables compared to table zero. if no group is selected the rule is added to group 0. Signed-off-by: Ori Kam --- drivers/net/mlx5/mlx5.h | 6 + drivers/net/mlx5/mlx5_flow.h | 15 ++- drivers/net/mlx5/mlx5_flow_dv.c | 278 +++++++++++++++++++++++++++++++++++-= ---- drivers/net/mlx5/mlx5_glue.c | 13 ++ drivers/net/mlx5/mlx5_glue.h | 1 + 5 files changed, 282 insertions(+), 31 deletions(-) diff --git a/drivers/net/mlx5/mlx5.h b/drivers/net/mlx5/mlx5.h index 73f6f0d..a3d5f8e 100644 --- a/drivers/net/mlx5/mlx5.h +++ b/drivers/net/mlx5/mlx5.h @@ -279,6 +279,12 @@ struct mlx5_priv { LIST_HEAD(encap_decap, mlx5_flow_dv_encap_decap_resource) encaps_decaps; LIST_HEAD(modify_cmd, mlx5_flow_dv_modify_hdr_resource) modify_cmds; LIST_HEAD(tag, mlx5_flow_dv_tag_resource) tags; + LIST_HEAD(jump, mlx5_flow_dv_jump_tbl_resource) jump_tbl; + /* Pointer to next element. */ + rte_atomic32_t refcnt; /**< Reference counter. */ + struct ibv_flow_action *verbs_action; + /**< Verbs modify header action object. */ + uint8_t ft_type; /**< Flow table type, Rx or Tx. */ /* Tags resources cache. */ uint32_t link_speed_capa; /* Link speed capabilities. */ struct mlx5_xstats_ctrl xstats_ctrl; /* Extended stats control. */ diff --git a/drivers/net/mlx5/mlx5_flow.h b/drivers/net/mlx5/mlx5_flow.h index 8ba37a0..622e305 100644 --- a/drivers/net/mlx5/mlx5_flow.h +++ b/drivers/net/mlx5/mlx5_flow.h @@ -115,7 +115,8 @@ #define MLX5_FLOW_ACTION_RAW_DECAP (1u << 27) =20 #define MLX5_FLOW_FATE_ACTIONS \ - (MLX5_FLOW_ACTION_DROP | MLX5_FLOW_ACTION_QUEUE | MLX5_FLOW_ACTION_RSS) + (MLX5_FLOW_ACTION_DROP | MLX5_FLOW_ACTION_QUEUE | \ + MLX5_FLOW_ACTION_RSS | MLX5_FLOW_ACTION_JUMP) =20 #define MLX5_FLOW_ENCAP_ACTIONS (MLX5_FLOW_ACTION_VXLAN_ENCAP | \ MLX5_FLOW_ACTION_NVGRE_ENCAP | \ @@ -250,6 +251,16 @@ struct mlx5_flow_dv_modify_hdr_resource { /**< Modification actions. */ }; =20 +/* Jump action resource structure. */ +struct mlx5_flow_dv_jump_tbl_resource { + LIST_ENTRY(mlx5_flow_dv_jump_tbl_resource) next; + /* Pointer to next element. */ + rte_atomic32_t refcnt; /**< Reference counter. */ + void *action; /**< Pointer to the rdma core action. */ + uint8_t ft_type; /**< Flow table type, Rx or Tx. */ + struct mlx5_flow_tbl_resource *tbl; /**< The target table. */ +}; + /* * Max number of actions per DV flow. * See CREATE_FLOW_MAX_FLOW_ACTIONS_SUPPORTED @@ -270,6 +281,8 @@ struct mlx5_flow_dv { struct mlx5_flow_dv_modify_hdr_resource *modify_hdr; /**< Pointer to modify header resource in cache. */ struct ibv_flow *flow; /**< Installed flow. */ + struct mlx5_flow_dv_jump_tbl_resource *jump; + /**< Pointer to the jump action resource. */ #ifdef HAVE_IBV_FLOW_DV_SUPPORT void *actions[MLX5_DV_MAX_NUMBER_OF_ACTIONS]; /**< Action list. */ diff --git a/drivers/net/mlx5/mlx5_flow_dv.c b/drivers/net/mlx5/mlx5_flow_d= v.c index 6e4f6c4..6997dc6 100644 --- a/drivers/net/mlx5/mlx5_flow_dv.c +++ b/drivers/net/mlx5/mlx5_flow_dv.c @@ -861,6 +861,68 @@ struct field_modify_info modify_tcp[] =3D { } =20 /** + * Find existing table jump resource or create and register a new one. + * + * @param dev[in, out] + * Pointer to rte_eth_dev structure. + * @param[in, out] resource + * Pointer to jump table resource. + * @parm[in, out] dev_flow + * Pointer to the dev_flow. + * @param[out] error + * pointer to error structure. + * + * @return + * 0 on success otherwise -errno and errno is set. + */ +static int +flow_dv_jump_tbl_resource_register + (struct rte_eth_dev *dev, + struct mlx5_flow_dv_jump_tbl_resource *resource, + struct mlx5_flow *dev_flow, + struct rte_flow_error *error) +{ + struct mlx5_priv *priv =3D dev->data->dev_private; + struct mlx5_flow_dv_jump_tbl_resource *cache_resource; + + /* Lookup a matching resource from cache. */ + LIST_FOREACH(cache_resource, &priv->jump_tbl, next) { + if (resource->tbl =3D=3D cache_resource->tbl) { + DRV_LOG(DEBUG, "jump table resource resource %p: refcnt %d++", + (void *)cache_resource, + rte_atomic32_read(&cache_resource->refcnt)); + rte_atomic32_inc(&cache_resource->refcnt); + dev_flow->dv.jump =3D cache_resource; + return 0; + } + } + /* Register new jump table resource. */ + cache_resource =3D rte_calloc(__func__, 1, sizeof(*cache_resource), 0); + if (!cache_resource) + return rte_flow_error_set(error, ENOMEM, + RTE_FLOW_ERROR_TYPE_UNSPECIFIED, NULL, + "cannot allocate resource memory"); + *cache_resource =3D *resource; + cache_resource->action =3D + mlx5_glue->dr_create_flow_action_dest_flow_tbl + (resource->tbl->obj); + if (!cache_resource->action) { + rte_free(cache_resource); + return rte_flow_error_set(error, ENOMEM, + RTE_FLOW_ERROR_TYPE_UNSPECIFIED, + NULL, "cannot create action"); + } + rte_atomic32_init(&cache_resource->refcnt); + rte_atomic32_inc(&cache_resource->refcnt); + LIST_INSERT_HEAD(&priv->jump_tbl, cache_resource, next); + dev_flow->dv.jump =3D cache_resource; + DRV_LOG(DEBUG, "new jump table resource %p: refcnt %d++", + (void *)cache_resource, + rte_atomic32_read(&cache_resource->refcnt)); + return 0; +} + +/** * Get the size of specific rte_flow_item_type * * @param[in] item_type @@ -1423,6 +1485,37 @@ struct field_modify_info modify_tcp[] =3D { } =20 /** + * Validate jump action. + * + * @param[in] action + * Pointer to the modify action. + * @param[in] group + * The group of the current flow. + * @param[out] error + * Pointer to error structure. + * + * @return + * 0 on success, a negative errno value otherwise and rte_errno is set. + */ +static int +flow_dv_validate_action_jump(const struct rte_flow_action *action, + uint32_t group, + struct rte_flow_error *error) +{ + if (action->type !=3D RTE_FLOW_ACTION_TYPE_JUMP && !action->conf) + return rte_flow_error_set(error, EINVAL, + RTE_FLOW_ERROR_TYPE_ACTION_CONF, + NULL, "action configuration not set"); + if (group >=3D ((const struct rte_flow_action_jump *)action->conf)->group= ) + return rte_flow_error_set(error, EINVAL, + RTE_FLOW_ERROR_TYPE_ACTION, NULL, + "target group must be higher then" + " the current flow group"); + return 0; +} + + +/** * Find existing modify-header resource or create and register a new one. * * @param dev[in, out] @@ -1605,7 +1698,7 @@ struct field_modify_info modify_tcp[] =3D { struct mlx5_priv *priv =3D dev->data->dev_private; uint32_t priority_max =3D priv->config.flow_prio - 1; =20 -#ifdef HAVE_MLX5DV_DR +#ifndef HAVE_MLX5DV_DR if (attributes->group) return rte_flow_error_set(error, ENOTSUP, RTE_FLOW_ERROR_TYPE_ATTR_GROUP, @@ -1977,6 +2070,14 @@ struct field_modify_info modify_tcp[] =3D { MLX5_FLOW_ACTION_SET_TTL : MLX5_FLOW_ACTION_DEC_TTL; break; + case RTE_FLOW_ACTION_TYPE_JUMP: + ret =3D flow_dv_validate_action_jump(actions, + attr->group, error); + if (ret) + return ret; + ++actions_n; + action_flags |=3D MLX5_FLOW_ACTION_JUMP; + break; default: return rte_flow_error_set(error, ENOTSUP, RTE_FLOW_ERROR_TYPE_ACTION, @@ -2756,6 +2857,73 @@ struct field_modify_info modify_tcp[] =3D { return match_criteria_enable; } =20 + +/** + * Get a flow table. + * + * @param dev[in, out] + * Pointer to rte_eth_dev structure. + * @param[in] table_id + * Table id to use. + * @param[in] egress + * Direction of the table. + * @param[out] error + * pointer to error structure. + * + * @return + * Returns tables resource based on the index, NULL in case of failed. + */ +static struct mlx5_flow_tbl_resource * +flow_dv_tbl_resource_get(struct rte_eth_dev *dev, + uint32_t table_id, uint8_t egress, + struct rte_flow_error *error) +{ + struct mlx5_priv *priv =3D dev->data->dev_private; + struct mlx5_flow_tbl_resource *tbl; + + if (egress) { + tbl =3D &priv->tx_tbl[table_id]; + if (!tbl->obj) + tbl->obj =3D mlx5_glue->dr_create_flow_tbl + (priv->tx_ns, table_id); + } else { + tbl =3D &priv->rx_tbl[table_id]; + if (!tbl->obj) + tbl->obj =3D mlx5_glue->dr_create_flow_tbl + (priv->rx_ns, table_id); + } + if (!tbl->obj) { + rte_flow_error_set(error, ENOMEM, + RTE_FLOW_ERROR_TYPE_UNSPECIFIED, + NULL, "cannot create table"); + return NULL; + } + rte_atomic32_inc(&tbl->refcnt); + return tbl; +} + +/** + * Release a flow table. + * + * @param[in] tbl + * Table resource to be released. + * + * @return + * Returns 0 if table was released, else return 1; + */ +static int +flow_dv_tbl_resource_release(struct mlx5_flow_tbl_resource *tbl) +{ + if (!tbl) + return 0; + if (rte_atomic32_dec_and_test(&tbl->refcnt)) { + mlx5_glue->dr_destroy_flow_tbl(tbl->obj); + tbl->obj =3D NULL; + return 0; + } + return 1; +} + /** * Register the flow matcher. * @@ -2784,6 +2952,9 @@ struct field_modify_info modify_tcp[] =3D { .match_mask =3D (void *)&matcher->mask, }; struct mlx5_flow_tbl_resource *tbl =3D NULL; +#ifndef HAVE_MLX5DV_DR + struct mlx5_flow_tbl_resource tbl_tmp; +#endif =20 /* Lookup from cache. */ LIST_FOREACH(cache_matcher, &priv->matchers, next) { @@ -2805,33 +2976,24 @@ struct field_modify_info modify_tcp[] =3D { return 0; } } -#ifdef HAVE_MLX5DV_DR - if (matcher->egress) { - tbl =3D &priv->tx_tbl[matcher->group]; - if (!tbl->obj) - tbl->obj =3D mlx5_glue->dr_create_flow_tbl - (priv->tx_ns, - matcher->group * MLX5_GROUP_FACTOR); - } else { - tbl =3D &priv->rx_tbl[matcher->group]; - if (!tbl->obj) - tbl->obj =3D mlx5_glue->dr_create_flow_tbl - (priv->rx_ns, - matcher->group * MLX5_GROUP_FACTOR); - } - if (!tbl->obj) - return rte_flow_error_set(error, ENOMEM, - RTE_FLOW_ERROR_TYPE_UNSPECIFIED, - NULL, "cannot create table"); - - rte_atomic32_inc(&tbl->refcnt); -#endif /* Register new matcher. */ cache_matcher =3D rte_calloc(__func__, 1, sizeof(*cache_matcher), 0); if (!cache_matcher) return rte_flow_error_set(error, ENOMEM, RTE_FLOW_ERROR_TYPE_UNSPECIFIED, NULL, "cannot allocate matcher memory"); +#ifdef HAVE_MLX5DV_DR + tbl =3D flow_dv_tbl_resource_get(dev, matcher->group * MLX5_GROUP_FACTOR, + matcher->egress, error); + if (!tbl) { + rte_free(cache_matcher); + return rte_flow_error_set(error, ENOMEM, + RTE_FLOW_ERROR_TYPE_UNSPECIFIED, + NULL, "cannot create table"); + } +#else + tbl =3D &tbl_tmp; +#endif *cache_matcher =3D *matcher; dv_attr.match_criteria_enable =3D flow_dv_matcher_enable(cache_matcher->mask.buf); @@ -2844,10 +3006,7 @@ struct field_modify_info modify_tcp[] =3D { if (!cache_matcher->matcher_object) { rte_free(cache_matcher); #ifdef HAVE_MLX5DV_DR - if (rte_atomic32_dec_and_test(&tbl->refcnt)) { - mlx5_glue->dr_destroy_flow_tbl(tbl->obj); - tbl->obj =3D NULL; - } + flow_dv_tbl_resource_release(tbl); #endif return rte_flow_error_set(error, ENOMEM, RTE_FLOW_ERROR_TYPE_UNSPECIFIED, @@ -3033,6 +3192,9 @@ struct field_modify_info modify_tcp[] =3D { const struct rte_flow_action *action =3D actions; const struct rte_flow_action_count *count =3D action->conf; const uint8_t *rss_key; + const struct rte_flow_action_jump *jump_data; + struct mlx5_flow_dv_jump_tbl_resource jump_tbl_resource; + struct mlx5_flow_tbl_resource *tbl; =20 switch (actions->type) { case RTE_FLOW_ACTION_TYPE_VOID: @@ -3171,6 +3333,31 @@ struct field_modify_info modify_tcp[] =3D { /* If decap is followed by encap, handle it at encap. */ action_flags |=3D MLX5_FLOW_ACTION_RAW_DECAP; break; + case RTE_FLOW_ACTION_TYPE_JUMP: + jump_data =3D action->conf; + tbl =3D flow_dv_tbl_resource_get(dev, jump_data->group * + MLX5_GROUP_FACTOR, + attr->egress, error); + if (!tbl) + return rte_flow_error_set + (error, errno, + RTE_FLOW_ERROR_TYPE_ACTION, + NULL, + "cannot create jump action."); + jump_tbl_resource.tbl =3D tbl; + if (flow_dv_jump_tbl_resource_register + (dev, &jump_tbl_resource, dev_flow, error)) { + flow_dv_tbl_resource_release(tbl); + return rte_flow_error_set + (error, errno, + RTE_FLOW_ERROR_TYPE_ACTION, + NULL, + "cannot create jump action."); + } + dev_flow->dv.actions[actions_n++] =3D + dev_flow->dv.jump->action; + action_flags |=3D MLX5_FLOW_ACTION_JUMP; + break; case RTE_FLOW_ACTION_TYPE_SET_MAC_SRC: case RTE_FLOW_ACTION_TYPE_SET_MAC_DST: if (flow_dv_convert_action_modify_mac(&res, actions, @@ -3503,10 +3690,7 @@ struct field_modify_info modify_tcp[] =3D { tbl =3D &priv->tx_tbl[matcher->group]; else tbl =3D &priv->rx_tbl[matcher->group]; - if (rte_atomic32_dec_and_test(&tbl->refcnt)) { - mlx5_glue->dr_destroy_flow_tbl(tbl->obj); - tbl->obj =3D NULL; - } + flow_dv_tbl_resource_release(tbl); rte_free(matcher); DRV_LOG(DEBUG, "port %u matcher %p: removed", dev->data->port_id, (void *)matcher); @@ -3547,6 +3731,38 @@ struct field_modify_info modify_tcp[] =3D { } =20 /** + * Release an jump to table action resource. + * + * @param flow + * Pointer to mlx5_flow. + * + * @return + * 1 while a reference on it exists, 0 when freed. + */ +static int +flow_dv_jump_tbl_resource_release(struct mlx5_flow *flow) +{ + struct mlx5_flow_dv_jump_tbl_resource *cache_resource =3D + flow->dv.jump; + + assert(cache_resource->action); + DRV_LOG(DEBUG, "jump table resource %p: refcnt %d--", + (void *)cache_resource, + rte_atomic32_read(&cache_resource->refcnt)); + if (rte_atomic32_dec_and_test(&cache_resource->refcnt)) { + claim_zero(mlx5_glue->destroy_flow_action + (cache_resource->action)); + LIST_REMOVE(cache_resource, next); + flow_dv_tbl_resource_release(cache_resource->tbl); + rte_free(cache_resource); + DRV_LOG(DEBUG, "jump table resource %p: removed", + (void *)cache_resource); + return 0; + } + return 1; +} + +/** * Release a modify-header resource. * * @param flow @@ -3642,6 +3858,8 @@ struct field_modify_info modify_tcp[] =3D { flow_dv_encap_decap_resource_release(dev_flow); if (dev_flow->dv.modify_hdr) flow_dv_modify_hdr_resource_release(dev_flow); + if (dev_flow->dv.jump) + flow_dv_jump_tbl_resource_release(dev_flow); rte_free(dev_flow); } } diff --git a/drivers/net/mlx5/mlx5_glue.c b/drivers/net/mlx5/mlx5_glue.c index f509f85..fca34ca 100644 --- a/drivers/net/mlx5/mlx5_glue.c +++ b/drivers/net/mlx5/mlx5_glue.c @@ -370,6 +370,17 @@ } =20 static void * +mlx5_glue_dr_create_flow_action_dest_flow_tbl(void *tbl) +{ +#ifdef HAVE_MLX5DV_DR + return mlx5dv_dr_create_action_dest_flow_table(tbl); +#else + (void)tbl; + return NULL; +#endif +} + +static void * mlx5_glue_dr_create_flow_tbl(void *ns, uint32_t level) { #ifdef HAVE_MLX5DV_DR @@ -833,6 +844,8 @@ .get_async_event =3D mlx5_glue_get_async_event, .port_state_str =3D mlx5_glue_port_state_str, .cq_ex_to_cq =3D mlx5_glue_cq_ex_to_cq, + .dr_create_flow_action_dest_flow_tbl =3D + mlx5_glue_dr_create_flow_action_dest_flow_tbl, .dr_create_flow_tbl =3D mlx5_glue_dr_create_flow_tbl, .dr_destroy_flow_tbl =3D mlx5_glue_dr_destroy_flow_tbl, .dr_create_ns =3D mlx5_glue_dr_create_ns, diff --git a/drivers/net/mlx5/mlx5_glue.h b/drivers/net/mlx5/mlx5_glue.h index 7115575..16d5dd6 100644 --- a/drivers/net/mlx5/mlx5_glue.h +++ b/drivers/net/mlx5/mlx5_glue.h @@ -145,6 +145,7 @@ struct mlx5_glue { struct ibv_async_event *event); const char *(*port_state_str)(enum ibv_port_state port_state); struct ibv_cq *(*cq_ex_to_cq)(struct ibv_cq_ex *cq); + void *(*dr_create_flow_action_dest_flow_tbl)(void *tbl); void *(*dr_create_flow_tbl)(void *ns, uint32_t level); int (*dr_destroy_flow_tbl)(void *tbl); void *(*dr_create_ns)(struct ibv_context *ctx, --=20 1.8.3.1