* [dpdk-dev] [PATCH 0/3]net/mlx5: Add Direct Rule support
@ 2019-03-20 15:38 Ori Kam
2019-03-20 15:38 ` Ori Kam
` (4 more replies)
0 siblings, 5 replies; 52+ messages in thread
From: Ori Kam @ 2019-03-20 15:38 UTC (permalink / raw)
To: Shahaf Shuler, Yongseok Koh, Matan Azrad; +Cc: dev, Ori Kam, Slava Ovsiienko
This patch set adds support for Direct Rules API for MLX5 driver.
As more and more applications like OVS are inserting and removing large
number of flows, the flow insertion rate become more critical.
In current MLX5 PMD rules are inserted to the device using the FW,
this limits the max insertion rate. The solution is using a new set of API
that are part of RDMA-Core. which enables bypassing the FW and writing
the flows directly to the device, which results in a much faster
insertion rate.
Ori Kam (3):
net/mlx5: prepare Direct Verbs for Direct Rule
net/mlx5: add Direct Rules API
net/mlx5: add jump action support for NIC
drivers/net/mlx5/Makefile | 5 +
drivers/net/mlx5/mlx5.c | 21 ++
drivers/net/mlx5/mlx5.h | 23 ++
drivers/net/mlx5/mlx5_flow.c | 1 +
drivers/net/mlx5/mlx5_flow.h | 38 +++-
drivers/net/mlx5/mlx5_flow_dv.c | 476 ++++++++++++++++++++++++++++++++++------
drivers/net/mlx5/mlx5_glue.c | 300 +++++++++++++++++++++----
drivers/net/mlx5/mlx5_glue.h | 50 +++--
drivers/net/mlx5/mlx5_prm.h | 24 +-
9 files changed, 810 insertions(+), 128 deletions(-)
--
1.8.3.1
^ permalink raw reply [flat|nested] 52+ messages in thread
* [dpdk-dev] [PATCH 0/3]net/mlx5: Add Direct Rule support
2019-03-20 15:38 [dpdk-dev] [PATCH 0/3]net/mlx5: Add Direct Rule support Ori Kam
@ 2019-03-20 15:38 ` Ori Kam
2019-03-20 15:39 ` [dpdk-dev] [PATCH 1/3] net/mlx5: prepare Direct Verbs for Direct Rule Ori Kam
` (3 subsequent siblings)
4 siblings, 0 replies; 52+ messages in thread
From: Ori Kam @ 2019-03-20 15:38 UTC (permalink / raw)
To: Shahaf Shuler, Yongseok Koh, Matan Azrad; +Cc: dev, Ori Kam, Slava Ovsiienko
This patch set adds support for Direct Rules API for MLX5 driver.
As more and more applications like OVS are inserting and removing large
number of flows, the flow insertion rate become more critical.
In current MLX5 PMD rules are inserted to the device using the FW,
this limits the max insertion rate. The solution is using a new set of API
that are part of RDMA-Core. which enables bypassing the FW and writing
the flows directly to the device, which results in a much faster
insertion rate.
Ori Kam (3):
net/mlx5: prepare Direct Verbs for Direct Rule
net/mlx5: add Direct Rules API
net/mlx5: add jump action support for NIC
drivers/net/mlx5/Makefile | 5 +
drivers/net/mlx5/mlx5.c | 21 ++
drivers/net/mlx5/mlx5.h | 23 ++
drivers/net/mlx5/mlx5_flow.c | 1 +
drivers/net/mlx5/mlx5_flow.h | 38 +++-
drivers/net/mlx5/mlx5_flow_dv.c | 476 ++++++++++++++++++++++++++++++++++------
drivers/net/mlx5/mlx5_glue.c | 300 +++++++++++++++++++++----
drivers/net/mlx5/mlx5_glue.h | 50 +++--
drivers/net/mlx5/mlx5_prm.h | 24 +-
9 files changed, 810 insertions(+), 128 deletions(-)
--
1.8.3.1
^ permalink raw reply [flat|nested] 52+ messages in thread
* [dpdk-dev] [PATCH 1/3] net/mlx5: prepare Direct Verbs for Direct Rule
2019-03-20 15:38 [dpdk-dev] [PATCH 0/3]net/mlx5: Add Direct Rule support Ori Kam
2019-03-20 15:38 ` Ori Kam
@ 2019-03-20 15:39 ` Ori Kam
2019-03-20 15:39 ` Ori Kam
2019-03-20 15:39 ` [dpdk-dev] [PATCH 2/3] net/mlx5: add Direct Rules API Ori Kam
` (2 subsequent siblings)
4 siblings, 1 reply; 52+ messages in thread
From: Ori Kam @ 2019-03-20 15:39 UTC (permalink / raw)
To: Shahaf Shuler, Yongseok Koh, Matan Azrad; +Cc: dev, Ori Kam, Slava Ovsiienko
This is the first patch of a series that is designed to enable the
Direct Rules API.
The main difference between Direct Verbs and Direct Rules from API
prespective, is that in Direct Rules each action has it's own create
function and the object itself is of type void.
In this patch I'm adding functions to generate actions that currenlty
are done without create action, and I'm changing the action type to be
void *, so in next patches only the glue functions will need to change.
Signed-off-by: Ori Kam <orika@mellanox.com>
---
drivers/net/mlx5/mlx5.h | 2 +
drivers/net/mlx5/mlx5_flow.h | 17 +++-
drivers/net/mlx5/mlx5_flow_dv.c | 188 ++++++++++++++++++++++++++++------------
drivers/net/mlx5/mlx5_glue.c | 141 ++++++++++++++++++++++++------
drivers/net/mlx5/mlx5_glue.h | 26 +++---
5 files changed, 278 insertions(+), 96 deletions(-)
diff --git a/drivers/net/mlx5/mlx5.h b/drivers/net/mlx5/mlx5.h
index 88ffb19..c0aa891 100644
--- a/drivers/net/mlx5/mlx5.h
+++ b/drivers/net/mlx5/mlx5.h
@@ -239,6 +239,8 @@ struct mlx5_priv {
LIST_HEAD(matchers, mlx5_flow_dv_matcher) matchers;
LIST_HEAD(encap_decap, mlx5_flow_dv_encap_decap_resource) encaps_decaps;
LIST_HEAD(modify_cmd, mlx5_flow_dv_modify_hdr_resource) modify_cmds;
+ LIST_HEAD(tag, mlx5_flow_dv_tag_resource) tags;
+ /* Tags resources cache. */
uint32_t link_speed_capa; /* Link speed capabilities. */
struct mlx5_xstats_ctrl xstats_ctrl; /* Extended stats control. */
struct mlx5_stats_ctrl stats_ctrl; /* Stats control. */
diff --git a/drivers/net/mlx5/mlx5_flow.h b/drivers/net/mlx5/mlx5_flow.h
index e1e798b..4f69ae2 100644
--- a/drivers/net/mlx5/mlx5_flow.h
+++ b/drivers/net/mlx5/mlx5_flow.h
@@ -214,7 +214,7 @@ struct mlx5_flow_dv_encap_decap_resource {
LIST_ENTRY(mlx5_flow_dv_encap_decap_resource) next;
/* Pointer to next element. */
rte_atomic32_t refcnt; /**< Reference counter. */
- struct ibv_flow_action *verbs_action;
+ void *verbs_action;
/**< Verbs encap/decap action object. */
uint8_t buf[MLX5_ENCAP_MAX_LEN];
size_t size;
@@ -222,6 +222,16 @@ struct mlx5_flow_dv_encap_decap_resource {
uint8_t ft_type;
};
+/* Tag resource structure. */
+struct mlx5_flow_dv_tag_resource {
+ LIST_ENTRY(mlx5_flow_dv_tag_resource) next;
+ /* Pointer to next element. */
+ rte_atomic32_t refcnt; /**< Reference counter. */
+ void *action;
+ /**< Verbs tag action object. */
+ uint32_t tag; /**< the tag value. */
+};
+
/* Number of modification commands. */
#define MLX5_MODIFY_NUM 8
@@ -259,7 +269,7 @@ struct mlx5_flow_dv {
/**< Pointer to modify header resource in cache. */
struct ibv_flow *flow; /**< Installed flow. */
#ifdef HAVE_IBV_FLOW_DV_SUPPORT
- struct mlx5dv_flow_action_attr actions[MLX5_DV_MAX_NUMBER_OF_ACTIONS];
+ void *actions[MLX5_DV_MAX_NUMBER_OF_ACTIONS];
/**< Action list. */
#endif
int actions_n; /**< number of actions. */
@@ -332,6 +342,7 @@ struct mlx5_flow_counter {
};
uint64_t hits; /**< Number of packets matched by the rule. */
uint64_t bytes; /**< Number of bytes matched by the rule. */
+ void *action; /**< Pointer to the dv action. */
};
/* Flow structure. */
@@ -339,6 +350,8 @@ struct rte_flow {
TAILQ_ENTRY(rte_flow) next; /**< Pointer to the next flow structure. */
enum mlx5_flow_drv_type drv_type; /**< Drvier type. */
struct mlx5_flow_counter *counter; /**< Holds flow counter. */
+ struct mlx5_flow_dv_tag_resource *tag_resource;
+ /**< pointer to the tag action. */
struct rte_flow_action_rss rss;/**< RSS context. */
uint8_t key[MLX5_RSS_HASH_KEY_LEN]; /**< RSS hash key. */
uint16_t (*queue)[]; /**< Destination queues to redirect traffic to. */
diff --git a/drivers/net/mlx5/mlx5_flow_dv.c b/drivers/net/mlx5/mlx5_flow_dv.c
index ebcdd15..0eb3a28 100644
--- a/drivers/net/mlx5/mlx5_flow_dv.c
+++ b/drivers/net/mlx5/mlx5_flow_dv.c
@@ -32,13 +32,8 @@
#include "mlx5_prm.h"
#include "mlx5_glue.h"
#include "mlx5_flow.h"
-
#ifdef HAVE_IBV_FLOW_DV_SUPPORT
-#ifndef HAVE_IBV_FLOW_DEVX_COUNTERS
-#define MLX5DV_FLOW_ACTION_COUNTER_DEVX 0
-#endif
-
union flow_dv_attr {
struct {
uint32_t valid:1;
@@ -1537,6 +1532,11 @@ struct field_modify_info modify_tcp[] = {
.id = id,
.dcs = dcs,
};
+ tmpl.action = mlx5_glue->dv_create_flow_action_counter(dcs->obj, 0);
+ if (!tmpl.action) {
+ ret = errno;
+ goto error_exit;
+ }
*cnt = tmpl;
LIST_INSERT_HEAD(&priv->flow_counters, cnt, next);
return cnt;
@@ -2805,6 +2805,97 @@ struct field_modify_info modify_tcp[] = {
}
/**
+ * Find existing tag resource or create and register a new one.
+ *
+ * @param dev[in, out]
+ * Pointer to rte_eth_dev structure.
+ * @param[in, out] resource
+ * Pointer to tag resource.
+ * @parm[in, out] dev_flow
+ * Pointer to the dev_flow.
+ * @param[out] error
+ * pointer to error structure.
+ *
+ * @return
+ * 0 on success otherwise -errno and errno is set.
+ */
+static int
+flow_dv_tag_resource_register
+ (struct rte_eth_dev *dev,
+ struct mlx5_flow_dv_tag_resource *resource,
+ struct mlx5_flow *dev_flow,
+ struct rte_flow_error *error)
+{
+ struct mlx5_priv *priv = dev->data->dev_private;
+ struct mlx5_flow_dv_tag_resource *cache_resource;
+
+ /* Lookup a matching resource from cache. */
+ LIST_FOREACH(cache_resource, &priv->tags, next) {
+ if (resource->tag == cache_resource->tag) {
+ DRV_LOG(DEBUG, "tag resource %p: refcnt %d++",
+ (void *)cache_resource,
+ rte_atomic32_read(&cache_resource->refcnt));
+ rte_atomic32_inc(&cache_resource->refcnt);
+ dev_flow->flow->tag_resource = cache_resource;
+ return 0;
+ }
+ }
+ /* Register new resource. */
+ cache_resource = rte_calloc(__func__, 1, sizeof(*cache_resource), 0);
+ if (!cache_resource)
+ return rte_flow_error_set(error, ENOMEM,
+ RTE_FLOW_ERROR_TYPE_UNSPECIFIED, NULL,
+ "cannot allocate resource memory");
+ *cache_resource = *resource;
+ cache_resource->action = mlx5_glue->dv_create_flow_action_tag
+ (resource->tag);
+ if (!cache_resource->action) {
+ rte_free(cache_resource);
+ return rte_flow_error_set(error, ENOMEM,
+ RTE_FLOW_ERROR_TYPE_UNSPECIFIED,
+ NULL, "cannot create action");
+ }
+ rte_atomic32_init(&cache_resource->refcnt);
+ rte_atomic32_inc(&cache_resource->refcnt);
+ LIST_INSERT_HEAD(&priv->tags, cache_resource, next);
+ dev_flow->flow->tag_resource = cache_resource;
+ DRV_LOG(DEBUG, "new tag resource %p: refcnt %d++",
+ (void *)cache_resource,
+ rte_atomic32_read(&cache_resource->refcnt));
+ return 0;
+}
+
+/**
+ * Release the tag.
+ *
+ * @param dev
+ * Pointer to Ethernet device.
+ * @param flow
+ * Pointer to mlx5_flow.
+ *
+ * @return
+ * 1 while a reference on it exists, 0 when freed.
+ */
+static int
+flow_dv_tag_release(struct rte_eth_dev *dev,
+ struct mlx5_flow_dv_tag_resource *tag)
+{
+ assert(tag);
+ DRV_LOG(DEBUG, "port %u tag %p: refcnt %d--",
+ dev->data->port_id, (void *)tag,
+ rte_atomic32_read(&tag->refcnt));
+ if (rte_atomic32_dec_and_test(&tag->refcnt)) {
+ claim_zero(mlx5_glue->destroy_flow_action(tag->action));
+ LIST_REMOVE(tag, next);
+ DRV_LOG(DEBUG, "port %u tag %p: removed",
+ dev->data->port_id, (void *)tag);
+ rte_free(tag);
+ return 0;
+ }
+ return 1;
+}
+
+/**
* Fill the flow with DV spec.
*
* @param[in] dev
@@ -2849,6 +2940,7 @@ struct field_modify_info modify_tcp[] = {
MLX5DV_FLOW_TABLE_TYPE_NIC_RX
};
union flow_dv_attr flow_attr = { .attr = 0 };
+ struct mlx5_flow_dv_tag_resource tag_resource;
if (priority == MLX5_FLOW_PRIO_RSVD)
priority = priv->config.flow_prio - 1;
@@ -2863,26 +2955,29 @@ struct field_modify_info modify_tcp[] = {
case RTE_FLOW_ACTION_TYPE_VOID:
break;
case RTE_FLOW_ACTION_TYPE_FLAG:
- dev_flow->dv.actions[actions_n].type =
- MLX5DV_FLOW_ACTION_TAG;
- dev_flow->dv.actions[actions_n].tag_value =
+ tag_resource.tag =
mlx5_flow_mark_set(MLX5_FLOW_MARK_DEFAULT);
- actions_n++;
+ if (!flow->tag_resource)
+ if (flow_dv_tag_resource_register
+ (dev, &tag_resource, dev_flow, error))
+ return errno;
+ dev_flow->dv.actions[actions_n++] =
+ flow->tag_resource->action;
action_flags |= MLX5_FLOW_ACTION_FLAG;
break;
case RTE_FLOW_ACTION_TYPE_MARK:
- dev_flow->dv.actions[actions_n].type =
- MLX5DV_FLOW_ACTION_TAG;
- dev_flow->dv.actions[actions_n].tag_value =
- mlx5_flow_mark_set
- (((const struct rte_flow_action_mark *)
- (actions->conf))->id);
- actions_n++;
+ tag_resource.tag = mlx5_flow_mark_set
+ (((const struct rte_flow_action_mark *)
+ (actions->conf))->id);
+ if (!flow->tag_resource)
+ if (flow_dv_tag_resource_register
+ (dev, &tag_resource, dev_flow, error))
+ return errno;
+ dev_flow->dv.actions[actions_n++] =
+ flow->tag_resource->action;
action_flags |= MLX5_FLOW_ACTION_MARK;
break;
case RTE_FLOW_ACTION_TYPE_DROP:
- dev_flow->dv.actions[actions_n].type =
- MLX5DV_FLOW_ACTION_DROP;
action_flags |= MLX5_FLOW_ACTION_DROP;
break;
case RTE_FLOW_ACTION_TYPE_QUEUE:
@@ -2910,17 +3005,13 @@ struct field_modify_info modify_tcp[] = {
rte_errno = ENOTSUP;
goto cnt_err;
}
- flow->counter =
- flow_dv_counter_new(dev,
- count->shared, count->id);
+ flow->counter = flow_dv_counter_new(dev, count->shared,
+ count->id);
if (flow->counter == NULL)
goto cnt_err;
- dev_flow->dv.actions[actions_n].type =
- MLX5DV_FLOW_ACTION_COUNTER_DEVX;
- dev_flow->dv.actions[actions_n].obj =
- flow->counter->dcs->obj;
+ dev_flow->dv.actions[actions_n++] =
+ flow->counter->action;
action_flags |= MLX5_FLOW_ACTION_COUNT;
- ++actions_n;
break;
cnt_err:
if (rte_errno == ENOTSUP)
@@ -2941,11 +3032,8 @@ struct field_modify_info modify_tcp[] = {
if (flow_dv_create_action_l2_encap(dev, actions,
dev_flow, error))
return -rte_errno;
- dev_flow->dv.actions[actions_n].type =
- MLX5DV_FLOW_ACTION_IBV_FLOW_ACTION;
- dev_flow->dv.actions[actions_n].action =
+ dev_flow->dv.actions[actions_n++] =
dev_flow->dv.encap_decap->verbs_action;
- actions_n++;
action_flags |= actions->type ==
RTE_FLOW_ACTION_TYPE_VXLAN_ENCAP ?
MLX5_FLOW_ACTION_VXLAN_ENCAP :
@@ -2956,11 +3044,8 @@ struct field_modify_info modify_tcp[] = {
if (flow_dv_create_action_l2_decap(dev, dev_flow,
error))
return -rte_errno;
- dev_flow->dv.actions[actions_n].type =
- MLX5DV_FLOW_ACTION_IBV_FLOW_ACTION;
- dev_flow->dv.actions[actions_n].action =
+ dev_flow->dv.actions[actions_n++] =
dev_flow->dv.encap_decap->verbs_action;
- actions_n++;
action_flags |= actions->type ==
RTE_FLOW_ACTION_TYPE_VXLAN_DECAP ?
MLX5_FLOW_ACTION_VXLAN_DECAP :
@@ -2972,9 +3057,7 @@ struct field_modify_info modify_tcp[] = {
if (flow_dv_create_action_raw_encap
(dev, actions, dev_flow, attr, error))
return -rte_errno;
- dev_flow->dv.actions[actions_n].type =
- MLX5DV_FLOW_ACTION_IBV_FLOW_ACTION;
- dev_flow->dv.actions[actions_n].action =
+ dev_flow->dv.actions[actions_n++] =
dev_flow->dv.encap_decap->verbs_action;
} else {
/* Handle encap without preceding decap. */
@@ -2982,12 +3065,9 @@ struct field_modify_info modify_tcp[] = {
dev_flow,
error))
return -rte_errno;
- dev_flow->dv.actions[actions_n].type =
- MLX5DV_FLOW_ACTION_IBV_FLOW_ACTION;
- dev_flow->dv.actions[actions_n].action =
+ dev_flow->dv.actions[actions_n++] =
dev_flow->dv.encap_decap->verbs_action;
}
- actions_n++;
action_flags |= MLX5_FLOW_ACTION_RAW_ENCAP;
break;
case RTE_FLOW_ACTION_TYPE_RAW_DECAP:
@@ -3002,11 +3082,8 @@ struct field_modify_info modify_tcp[] = {
dev_flow,
error))
return -rte_errno;
- dev_flow->dv.actions[actions_n].type =
- MLX5DV_FLOW_ACTION_IBV_FLOW_ACTION;
- dev_flow->dv.actions[actions_n].action =
+ dev_flow->dv.actions[actions_n++] =
dev_flow->dv.encap_decap->verbs_action;
- actions_n++;
}
/* If decap is followed by encap, handle it at encap. */
action_flags |= MLX5_FLOW_ACTION_RAW_DECAP;
@@ -3075,11 +3152,8 @@ struct field_modify_info modify_tcp[] = {
dev_flow,
error))
return -rte_errno;
- dev_flow->dv.actions[actions_n].type =
- MLX5DV_FLOW_ACTION_IBV_FLOW_ACTION;
- dev_flow->dv.actions[actions_n].action =
+ dev_flow->dv.actions[actions_n++] =
dev_flow->dv.modify_hdr->verbs_action;
- actions_n++;
}
break;
default:
@@ -3241,9 +3315,9 @@ struct field_modify_info modify_tcp[] = {
"cannot get drop hash queue");
goto error;
}
- dv->actions[n].type = MLX5DV_FLOW_ACTION_DEST_IBV_QP;
- dv->actions[n].qp = dv->hrxq->qp;
- n++;
+ dv->actions[n++] =
+ mlx5_glue->dv_create_flow_action_dest_ibv_qp
+ (dv->hrxq->qp);
} else if (flow->actions &
(MLX5_FLOW_ACTION_QUEUE | MLX5_FLOW_ACTION_RSS)) {
struct mlx5_hrxq *hrxq;
@@ -3268,9 +3342,9 @@ struct field_modify_info modify_tcp[] = {
goto error;
}
dv->hrxq = hrxq;
- dv->actions[n].type = MLX5DV_FLOW_ACTION_DEST_IBV_QP;
- dv->actions[n].qp = hrxq->qp;
- n++;
+ dv->actions[n++] =
+ mlx5_glue->dv_create_flow_action_dest_ibv_qp
+ (dv->hrxq->qp);
}
dv->flow =
mlx5_glue->dv_create_flow(dv->matcher->matcher_object,
@@ -3448,6 +3522,10 @@ struct field_modify_info modify_tcp[] = {
flow_dv_counter_release(flow->counter);
flow->counter = NULL;
}
+ if (flow->tag_resource) {
+ flow_dv_tag_release(dev, flow->tag_resource);
+ flow->tag_resource = NULL;
+ }
while (!LIST_EMPTY(&flow->dev_flows)) {
dev_flow = LIST_FIRST(&flow->dev_flows);
LIST_REMOVE(dev_flow, next);
diff --git a/drivers/net/mlx5/mlx5_glue.c b/drivers/net/mlx5/mlx5_glue.c
index c817d86..cfd939d 100644
--- a/drivers/net/mlx5/mlx5_glue.c
+++ b/drivers/net/mlx5/mlx5_glue.c
@@ -175,10 +175,20 @@
}
static int
-mlx5_glue_destroy_flow_action(struct ibv_flow_action *action)
+mlx5_glue_destroy_flow_action(void *action)
{
#ifdef HAVE_IBV_FLOW_DV_SUPPORT
- return ibv_destroy_flow_action(action);
+ struct mlx5dv_flow_action_attr *attr = action;
+ int res = 0;
+ switch (attr->type) {
+ case MLX5DV_FLOW_ACTION_TAG:
+ break;
+ default:
+ res = ibv_destroy_flow_action(attr->action);
+ break;
+ }
+ free(action);
+ return res;
#else
(void)action;
return ENOTSUP;
@@ -430,16 +440,23 @@
mlx5_glue_dv_create_flow(struct mlx5dv_flow_matcher *matcher,
struct mlx5dv_flow_match_parameters *match_value,
size_t num_actions,
- struct mlx5dv_flow_action_attr *actions_attr)
+ void *actions[])
{
#ifdef HAVE_IBV_FLOW_DV_SUPPORT
+ struct mlx5dv_flow_action_attr actions_attr[8];
+
+ if (num_actions > 8)
+ return NULL;
+ for (size_t i = 0; i < num_actions; i++)
+ actions_attr[i] =
+ *((struct mlx5dv_flow_action_attr *)(actions[i]));
return mlx5dv_create_flow(matcher, match_value,
num_actions, actions_attr);
#else
(void)matcher;
(void)match_value;
(void)num_actions;
- (void)actions_attr;
+ (void)actions;
return NULL;
#endif
}
@@ -455,31 +472,45 @@
#endif
}
-static struct ibv_flow_action *
-mlx5_glue_dv_create_flow_action_packet_reformat
- (struct ibv_context *ctx,
- size_t data_sz,
- void *data,
- enum mlx5dv_flow_action_packet_reformat_type reformat_type,
- enum mlx5dv_flow_table_type ft_type)
+static void *
+mlx5_glue_dv_create_flow_action_counter(void *counter_obj, uint32_t offset)
{
#ifdef HAVE_IBV_FLOW_DV_SUPPORT
- return mlx5dv_create_flow_action_packet_reformat(ctx,
- data_sz,
- data,
- reformat_type,
- ft_type);
+ struct mlx5dv_flow_action_attr *action;
+
+ (void)offset;
+ action = malloc(sizeof(*action));
+ if (!action)
+ return NULL;
+ action->type = MLX5DV_FLOW_ACTION_COUNTER_DEVX;
+ action->obj = counter_obj;
+ return action;
#else
- (void)ctx;
- (void)data_sz;
- (void)data;
- (void)reformat_type;
- (void)ft_type;
+ (void)counter_obj;
+ (void)offset;
return NULL;
#endif
}
-static struct ibv_flow_action *
+static void *
+mlx5_glue_dv_create_flow_action_dest_ibv_qp(void *qp)
+{
+#ifdef HAVE_IBV_FLOW_DV_SUPPORT
+ struct mlx5dv_flow_action_attr *action;
+
+ action = malloc(sizeof(*action));
+ if (!action)
+ return NULL;
+ action->type = MLX5DV_FLOW_ACTION_DEST_IBV_QP;
+ action->obj = qp;
+ return action;
+#else
+ (void)qp;
+ return NULL;
+#endif
+}
+
+static void *
mlx5_glue_dv_create_flow_action_modify_header
(struct ibv_context *ctx,
size_t actions_sz,
@@ -487,8 +518,15 @@
enum mlx5dv_flow_table_type ft_type)
{
#ifdef HAVE_IBV_FLOW_DV_SUPPORT
- return mlx5dv_create_flow_action_modify_header(ctx, actions_sz,
- actions, ft_type);
+ struct mlx5dv_flow_action_attr *action;
+
+ action = malloc(sizeof(*action));
+ if (!action)
+ return NULL;
+ action->type = MLX5DV_FLOW_ACTION_IBV_FLOW_ACTION;
+ action->action = mlx5dv_create_flow_action_modify_header
+ (ctx, actions_sz, actions, ft_type);
+ return action;
#else
(void)ctx;
(void)actions_sz;
@@ -498,6 +536,50 @@
#endif
}
+static void *
+mlx5_glue_dv_create_flow_action_packet_reformat
+ (struct ibv_context *ctx,
+ size_t data_sz,
+ void *data,
+ enum mlx5dv_flow_action_packet_reformat_type reformat_type,
+ enum mlx5dv_flow_table_type ft_type)
+{
+#ifdef HAVE_IBV_FLOW_DV_SUPPORT
+ struct mlx5dv_flow_action_attr *action;
+
+ action = malloc(sizeof(*action));
+ if (!action)
+ return NULL;
+ action->type = MLX5DV_FLOW_ACTION_IBV_FLOW_ACTION;
+ action->action = mlx5dv_create_flow_action_packet_reformat
+ (ctx, data_sz, data, reformat_type, ft_type);
+ return action;
+#else
+ (void)ctx;
+ (void)data_sz;
+ (void)data;
+ (void)reformat_type;
+ (void)ft_type;
+ return NULL;
+#endif
+}
+
+static void *
+mlx5_glue_dv_create_flow_action_tag(uint32_t tag)
+{
+#ifdef HAVE_IBV_FLOW_DV_SUPPORT
+ struct mlx5dv_flow_action_attr *action;
+ action = malloc(sizeof(*action));
+ if (!action)
+ return NULL;
+ action->type = MLX5DV_FLOW_ACTION_TAG;
+ action->tag_value = tag;
+ return action;
+#endif
+ (void)tag;
+ return NULL;
+}
+
static struct ibv_context *
mlx5_glue_dv_open_device(struct ibv_device *device)
{
@@ -645,10 +727,15 @@
.dv_create_flow_matcher = mlx5_glue_dv_create_flow_matcher,
.dv_destroy_flow_matcher = mlx5_glue_dv_destroy_flow_matcher,
.dv_create_flow = mlx5_glue_dv_create_flow,
- .dv_create_flow_action_packet_reformat =
- mlx5_glue_dv_create_flow_action_packet_reformat,
+ .dv_create_flow_action_counter =
+ mlx5_glue_dv_create_flow_action_counter,
+ .dv_create_flow_action_dest_ibv_qp =
+ mlx5_glue_dv_create_flow_action_dest_ibv_qp,
.dv_create_flow_action_modify_header =
- mlx5_glue_dv_create_flow_action_modify_header,
+ mlx5_glue_dv_create_flow_action_modify_header,
+ .dv_create_flow_action_packet_reformat =
+ mlx5_glue_dv_create_flow_action_packet_reformat,
+ .dv_create_flow_action_tag = mlx5_glue_dv_create_flow_action_tag,
.dv_open_device = mlx5_glue_dv_open_device,
.devx_obj_create = mlx5_glue_devx_obj_create,
.devx_obj_destroy = mlx5_glue_devx_obj_destroy,
diff --git a/drivers/net/mlx5/mlx5_glue.h b/drivers/net/mlx5/mlx5_glue.h
index b118960..167f1f7 100644
--- a/drivers/net/mlx5/mlx5_glue.h
+++ b/drivers/net/mlx5/mlx5_glue.h
@@ -55,6 +55,10 @@
enum mlx5dv_flow_table_type { flow_table_type = 0, };
#endif
+#ifndef HAVE_IBV_FLOW_DEVX_COUNTERS
+#define MLX5DV_FLOW_ACTION_COUNTER_DEVX 0
+#endif
+
#ifndef HAVE_IBV_DEVX_OBJ
struct mlx5dv_devx_obj;
#endif
@@ -98,7 +102,7 @@ struct mlx5_glue {
struct ibv_flow *(*create_flow)(struct ibv_qp *qp,
struct ibv_flow_attr *flow);
int (*destroy_flow)(struct ibv_flow *flow_id);
- int (*destroy_flow_action)(struct ibv_flow_action *action);
+ int (*destroy_flow_action)(void *action);
struct ibv_qp *(*create_qp)(struct ibv_pd *pd,
struct ibv_qp_init_attr *qp_init_attr);
struct ibv_qp *(*create_qp_ex)
@@ -160,19 +164,17 @@ struct mlx5_glue {
int (*dv_destroy_flow_matcher)(struct mlx5dv_flow_matcher *matcher);
struct ibv_flow *(*dv_create_flow)(struct mlx5dv_flow_matcher *matcher,
struct mlx5dv_flow_match_parameters *match_value,
- size_t num_actions,
- struct mlx5dv_flow_action_attr *actions_attr);
- struct ibv_flow_action *(*dv_create_flow_action_packet_reformat)
- (struct ibv_context *ctx,
- size_t data_sz,
- void *data,
+ size_t num_actions, void *actions[]);
+ void *(*dv_create_flow_action_counter)(void *obj, uint32_t offset);
+ void *(*dv_create_flow_action_dest_ibv_qp)(void *qp);
+ void *(*dv_create_flow_action_modify_header)
+ (struct ibv_context *ctx, size_t actions_sz, uint64_t actions[],
+ enum mlx5dv_flow_table_type ft_type);
+ void *(*dv_create_flow_action_packet_reformat)
+ (struct ibv_context *ctx, size_t data_sz, void *data,
enum mlx5dv_flow_action_packet_reformat_type reformat_type,
enum mlx5dv_flow_table_type ft_type);
- struct ibv_flow_action *(*dv_create_flow_action_modify_header)
- (struct ibv_context *ctx,
- size_t actions_sz,
- uint64_t actions[],
- enum mlx5dv_flow_table_type ft_type);
+ void *(*dv_create_flow_action_tag)(uint32_t tag);
struct ibv_context *(*dv_open_device)(struct ibv_device *device);
struct mlx5dv_devx_obj *(*devx_obj_create)
(struct ibv_context *ctx,
--
1.8.3.1
^ permalink raw reply [flat|nested] 52+ messages in thread
* [dpdk-dev] [PATCH 1/3] net/mlx5: prepare Direct Verbs for Direct Rule
2019-03-20 15:39 ` [dpdk-dev] [PATCH 1/3] net/mlx5: prepare Direct Verbs for Direct Rule Ori Kam
@ 2019-03-20 15:39 ` Ori Kam
0 siblings, 0 replies; 52+ messages in thread
From: Ori Kam @ 2019-03-20 15:39 UTC (permalink / raw)
To: Shahaf Shuler, Yongseok Koh, Matan Azrad; +Cc: dev, Ori Kam, Slava Ovsiienko
This is the first patch of a series that is designed to enable the
Direct Rules API.
The main difference between Direct Verbs and Direct Rules from API
prespective, is that in Direct Rules each action has it's own create
function and the object itself is of type void.
In this patch I'm adding functions to generate actions that currenlty
are done without create action, and I'm changing the action type to be
void *, so in next patches only the glue functions will need to change.
Signed-off-by: Ori Kam <orika@mellanox.com>
---
drivers/net/mlx5/mlx5.h | 2 +
drivers/net/mlx5/mlx5_flow.h | 17 +++-
drivers/net/mlx5/mlx5_flow_dv.c | 188 ++++++++++++++++++++++++++++------------
drivers/net/mlx5/mlx5_glue.c | 141 ++++++++++++++++++++++++------
drivers/net/mlx5/mlx5_glue.h | 26 +++---
5 files changed, 278 insertions(+), 96 deletions(-)
diff --git a/drivers/net/mlx5/mlx5.h b/drivers/net/mlx5/mlx5.h
index 88ffb19..c0aa891 100644
--- a/drivers/net/mlx5/mlx5.h
+++ b/drivers/net/mlx5/mlx5.h
@@ -239,6 +239,8 @@ struct mlx5_priv {
LIST_HEAD(matchers, mlx5_flow_dv_matcher) matchers;
LIST_HEAD(encap_decap, mlx5_flow_dv_encap_decap_resource) encaps_decaps;
LIST_HEAD(modify_cmd, mlx5_flow_dv_modify_hdr_resource) modify_cmds;
+ LIST_HEAD(tag, mlx5_flow_dv_tag_resource) tags;
+ /* Tags resources cache. */
uint32_t link_speed_capa; /* Link speed capabilities. */
struct mlx5_xstats_ctrl xstats_ctrl; /* Extended stats control. */
struct mlx5_stats_ctrl stats_ctrl; /* Stats control. */
diff --git a/drivers/net/mlx5/mlx5_flow.h b/drivers/net/mlx5/mlx5_flow.h
index e1e798b..4f69ae2 100644
--- a/drivers/net/mlx5/mlx5_flow.h
+++ b/drivers/net/mlx5/mlx5_flow.h
@@ -214,7 +214,7 @@ struct mlx5_flow_dv_encap_decap_resource {
LIST_ENTRY(mlx5_flow_dv_encap_decap_resource) next;
/* Pointer to next element. */
rte_atomic32_t refcnt; /**< Reference counter. */
- struct ibv_flow_action *verbs_action;
+ void *verbs_action;
/**< Verbs encap/decap action object. */
uint8_t buf[MLX5_ENCAP_MAX_LEN];
size_t size;
@@ -222,6 +222,16 @@ struct mlx5_flow_dv_encap_decap_resource {
uint8_t ft_type;
};
+/* Tag resource structure. */
+struct mlx5_flow_dv_tag_resource {
+ LIST_ENTRY(mlx5_flow_dv_tag_resource) next;
+ /* Pointer to next element. */
+ rte_atomic32_t refcnt; /**< Reference counter. */
+ void *action;
+ /**< Verbs tag action object. */
+ uint32_t tag; /**< the tag value. */
+};
+
/* Number of modification commands. */
#define MLX5_MODIFY_NUM 8
@@ -259,7 +269,7 @@ struct mlx5_flow_dv {
/**< Pointer to modify header resource in cache. */
struct ibv_flow *flow; /**< Installed flow. */
#ifdef HAVE_IBV_FLOW_DV_SUPPORT
- struct mlx5dv_flow_action_attr actions[MLX5_DV_MAX_NUMBER_OF_ACTIONS];
+ void *actions[MLX5_DV_MAX_NUMBER_OF_ACTIONS];
/**< Action list. */
#endif
int actions_n; /**< number of actions. */
@@ -332,6 +342,7 @@ struct mlx5_flow_counter {
};
uint64_t hits; /**< Number of packets matched by the rule. */
uint64_t bytes; /**< Number of bytes matched by the rule. */
+ void *action; /**< Pointer to the dv action. */
};
/* Flow structure. */
@@ -339,6 +350,8 @@ struct rte_flow {
TAILQ_ENTRY(rte_flow) next; /**< Pointer to the next flow structure. */
enum mlx5_flow_drv_type drv_type; /**< Drvier type. */
struct mlx5_flow_counter *counter; /**< Holds flow counter. */
+ struct mlx5_flow_dv_tag_resource *tag_resource;
+ /**< pointer to the tag action. */
struct rte_flow_action_rss rss;/**< RSS context. */
uint8_t key[MLX5_RSS_HASH_KEY_LEN]; /**< RSS hash key. */
uint16_t (*queue)[]; /**< Destination queues to redirect traffic to. */
diff --git a/drivers/net/mlx5/mlx5_flow_dv.c b/drivers/net/mlx5/mlx5_flow_dv.c
index ebcdd15..0eb3a28 100644
--- a/drivers/net/mlx5/mlx5_flow_dv.c
+++ b/drivers/net/mlx5/mlx5_flow_dv.c
@@ -32,13 +32,8 @@
#include "mlx5_prm.h"
#include "mlx5_glue.h"
#include "mlx5_flow.h"
-
#ifdef HAVE_IBV_FLOW_DV_SUPPORT
-#ifndef HAVE_IBV_FLOW_DEVX_COUNTERS
-#define MLX5DV_FLOW_ACTION_COUNTER_DEVX 0
-#endif
-
union flow_dv_attr {
struct {
uint32_t valid:1;
@@ -1537,6 +1532,11 @@ struct field_modify_info modify_tcp[] = {
.id = id,
.dcs = dcs,
};
+ tmpl.action = mlx5_glue->dv_create_flow_action_counter(dcs->obj, 0);
+ if (!tmpl.action) {
+ ret = errno;
+ goto error_exit;
+ }
*cnt = tmpl;
LIST_INSERT_HEAD(&priv->flow_counters, cnt, next);
return cnt;
@@ -2805,6 +2805,97 @@ struct field_modify_info modify_tcp[] = {
}
/**
+ * Find existing tag resource or create and register a new one.
+ *
+ * @param dev[in, out]
+ * Pointer to rte_eth_dev structure.
+ * @param[in, out] resource
+ * Pointer to tag resource.
+ * @parm[in, out] dev_flow
+ * Pointer to the dev_flow.
+ * @param[out] error
+ * pointer to error structure.
+ *
+ * @return
+ * 0 on success otherwise -errno and errno is set.
+ */
+static int
+flow_dv_tag_resource_register
+ (struct rte_eth_dev *dev,
+ struct mlx5_flow_dv_tag_resource *resource,
+ struct mlx5_flow *dev_flow,
+ struct rte_flow_error *error)
+{
+ struct mlx5_priv *priv = dev->data->dev_private;
+ struct mlx5_flow_dv_tag_resource *cache_resource;
+
+ /* Lookup a matching resource from cache. */
+ LIST_FOREACH(cache_resource, &priv->tags, next) {
+ if (resource->tag == cache_resource->tag) {
+ DRV_LOG(DEBUG, "tag resource %p: refcnt %d++",
+ (void *)cache_resource,
+ rte_atomic32_read(&cache_resource->refcnt));
+ rte_atomic32_inc(&cache_resource->refcnt);
+ dev_flow->flow->tag_resource = cache_resource;
+ return 0;
+ }
+ }
+ /* Register new resource. */
+ cache_resource = rte_calloc(__func__, 1, sizeof(*cache_resource), 0);
+ if (!cache_resource)
+ return rte_flow_error_set(error, ENOMEM,
+ RTE_FLOW_ERROR_TYPE_UNSPECIFIED, NULL,
+ "cannot allocate resource memory");
+ *cache_resource = *resource;
+ cache_resource->action = mlx5_glue->dv_create_flow_action_tag
+ (resource->tag);
+ if (!cache_resource->action) {
+ rte_free(cache_resource);
+ return rte_flow_error_set(error, ENOMEM,
+ RTE_FLOW_ERROR_TYPE_UNSPECIFIED,
+ NULL, "cannot create action");
+ }
+ rte_atomic32_init(&cache_resource->refcnt);
+ rte_atomic32_inc(&cache_resource->refcnt);
+ LIST_INSERT_HEAD(&priv->tags, cache_resource, next);
+ dev_flow->flow->tag_resource = cache_resource;
+ DRV_LOG(DEBUG, "new tag resource %p: refcnt %d++",
+ (void *)cache_resource,
+ rte_atomic32_read(&cache_resource->refcnt));
+ return 0;
+}
+
+/**
+ * Release the tag.
+ *
+ * @param dev
+ * Pointer to Ethernet device.
+ * @param flow
+ * Pointer to mlx5_flow.
+ *
+ * @return
+ * 1 while a reference on it exists, 0 when freed.
+ */
+static int
+flow_dv_tag_release(struct rte_eth_dev *dev,
+ struct mlx5_flow_dv_tag_resource *tag)
+{
+ assert(tag);
+ DRV_LOG(DEBUG, "port %u tag %p: refcnt %d--",
+ dev->data->port_id, (void *)tag,
+ rte_atomic32_read(&tag->refcnt));
+ if (rte_atomic32_dec_and_test(&tag->refcnt)) {
+ claim_zero(mlx5_glue->destroy_flow_action(tag->action));
+ LIST_REMOVE(tag, next);
+ DRV_LOG(DEBUG, "port %u tag %p: removed",
+ dev->data->port_id, (void *)tag);
+ rte_free(tag);
+ return 0;
+ }
+ return 1;
+}
+
+/**
* Fill the flow with DV spec.
*
* @param[in] dev
@@ -2849,6 +2940,7 @@ struct field_modify_info modify_tcp[] = {
MLX5DV_FLOW_TABLE_TYPE_NIC_RX
};
union flow_dv_attr flow_attr = { .attr = 0 };
+ struct mlx5_flow_dv_tag_resource tag_resource;
if (priority == MLX5_FLOW_PRIO_RSVD)
priority = priv->config.flow_prio - 1;
@@ -2863,26 +2955,29 @@ struct field_modify_info modify_tcp[] = {
case RTE_FLOW_ACTION_TYPE_VOID:
break;
case RTE_FLOW_ACTION_TYPE_FLAG:
- dev_flow->dv.actions[actions_n].type =
- MLX5DV_FLOW_ACTION_TAG;
- dev_flow->dv.actions[actions_n].tag_value =
+ tag_resource.tag =
mlx5_flow_mark_set(MLX5_FLOW_MARK_DEFAULT);
- actions_n++;
+ if (!flow->tag_resource)
+ if (flow_dv_tag_resource_register
+ (dev, &tag_resource, dev_flow, error))
+ return errno;
+ dev_flow->dv.actions[actions_n++] =
+ flow->tag_resource->action;
action_flags |= MLX5_FLOW_ACTION_FLAG;
break;
case RTE_FLOW_ACTION_TYPE_MARK:
- dev_flow->dv.actions[actions_n].type =
- MLX5DV_FLOW_ACTION_TAG;
- dev_flow->dv.actions[actions_n].tag_value =
- mlx5_flow_mark_set
- (((const struct rte_flow_action_mark *)
- (actions->conf))->id);
- actions_n++;
+ tag_resource.tag = mlx5_flow_mark_set
+ (((const struct rte_flow_action_mark *)
+ (actions->conf))->id);
+ if (!flow->tag_resource)
+ if (flow_dv_tag_resource_register
+ (dev, &tag_resource, dev_flow, error))
+ return errno;
+ dev_flow->dv.actions[actions_n++] =
+ flow->tag_resource->action;
action_flags |= MLX5_FLOW_ACTION_MARK;
break;
case RTE_FLOW_ACTION_TYPE_DROP:
- dev_flow->dv.actions[actions_n].type =
- MLX5DV_FLOW_ACTION_DROP;
action_flags |= MLX5_FLOW_ACTION_DROP;
break;
case RTE_FLOW_ACTION_TYPE_QUEUE:
@@ -2910,17 +3005,13 @@ struct field_modify_info modify_tcp[] = {
rte_errno = ENOTSUP;
goto cnt_err;
}
- flow->counter =
- flow_dv_counter_new(dev,
- count->shared, count->id);
+ flow->counter = flow_dv_counter_new(dev, count->shared,
+ count->id);
if (flow->counter == NULL)
goto cnt_err;
- dev_flow->dv.actions[actions_n].type =
- MLX5DV_FLOW_ACTION_COUNTER_DEVX;
- dev_flow->dv.actions[actions_n].obj =
- flow->counter->dcs->obj;
+ dev_flow->dv.actions[actions_n++] =
+ flow->counter->action;
action_flags |= MLX5_FLOW_ACTION_COUNT;
- ++actions_n;
break;
cnt_err:
if (rte_errno == ENOTSUP)
@@ -2941,11 +3032,8 @@ struct field_modify_info modify_tcp[] = {
if (flow_dv_create_action_l2_encap(dev, actions,
dev_flow, error))
return -rte_errno;
- dev_flow->dv.actions[actions_n].type =
- MLX5DV_FLOW_ACTION_IBV_FLOW_ACTION;
- dev_flow->dv.actions[actions_n].action =
+ dev_flow->dv.actions[actions_n++] =
dev_flow->dv.encap_decap->verbs_action;
- actions_n++;
action_flags |= actions->type ==
RTE_FLOW_ACTION_TYPE_VXLAN_ENCAP ?
MLX5_FLOW_ACTION_VXLAN_ENCAP :
@@ -2956,11 +3044,8 @@ struct field_modify_info modify_tcp[] = {
if (flow_dv_create_action_l2_decap(dev, dev_flow,
error))
return -rte_errno;
- dev_flow->dv.actions[actions_n].type =
- MLX5DV_FLOW_ACTION_IBV_FLOW_ACTION;
- dev_flow->dv.actions[actions_n].action =
+ dev_flow->dv.actions[actions_n++] =
dev_flow->dv.encap_decap->verbs_action;
- actions_n++;
action_flags |= actions->type ==
RTE_FLOW_ACTION_TYPE_VXLAN_DECAP ?
MLX5_FLOW_ACTION_VXLAN_DECAP :
@@ -2972,9 +3057,7 @@ struct field_modify_info modify_tcp[] = {
if (flow_dv_create_action_raw_encap
(dev, actions, dev_flow, attr, error))
return -rte_errno;
- dev_flow->dv.actions[actions_n].type =
- MLX5DV_FLOW_ACTION_IBV_FLOW_ACTION;
- dev_flow->dv.actions[actions_n].action =
+ dev_flow->dv.actions[actions_n++] =
dev_flow->dv.encap_decap->verbs_action;
} else {
/* Handle encap without preceding decap. */
@@ -2982,12 +3065,9 @@ struct field_modify_info modify_tcp[] = {
dev_flow,
error))
return -rte_errno;
- dev_flow->dv.actions[actions_n].type =
- MLX5DV_FLOW_ACTION_IBV_FLOW_ACTION;
- dev_flow->dv.actions[actions_n].action =
+ dev_flow->dv.actions[actions_n++] =
dev_flow->dv.encap_decap->verbs_action;
}
- actions_n++;
action_flags |= MLX5_FLOW_ACTION_RAW_ENCAP;
break;
case RTE_FLOW_ACTION_TYPE_RAW_DECAP:
@@ -3002,11 +3082,8 @@ struct field_modify_info modify_tcp[] = {
dev_flow,
error))
return -rte_errno;
- dev_flow->dv.actions[actions_n].type =
- MLX5DV_FLOW_ACTION_IBV_FLOW_ACTION;
- dev_flow->dv.actions[actions_n].action =
+ dev_flow->dv.actions[actions_n++] =
dev_flow->dv.encap_decap->verbs_action;
- actions_n++;
}
/* If decap is followed by encap, handle it at encap. */
action_flags |= MLX5_FLOW_ACTION_RAW_DECAP;
@@ -3075,11 +3152,8 @@ struct field_modify_info modify_tcp[] = {
dev_flow,
error))
return -rte_errno;
- dev_flow->dv.actions[actions_n].type =
- MLX5DV_FLOW_ACTION_IBV_FLOW_ACTION;
- dev_flow->dv.actions[actions_n].action =
+ dev_flow->dv.actions[actions_n++] =
dev_flow->dv.modify_hdr->verbs_action;
- actions_n++;
}
break;
default:
@@ -3241,9 +3315,9 @@ struct field_modify_info modify_tcp[] = {
"cannot get drop hash queue");
goto error;
}
- dv->actions[n].type = MLX5DV_FLOW_ACTION_DEST_IBV_QP;
- dv->actions[n].qp = dv->hrxq->qp;
- n++;
+ dv->actions[n++] =
+ mlx5_glue->dv_create_flow_action_dest_ibv_qp
+ (dv->hrxq->qp);
} else if (flow->actions &
(MLX5_FLOW_ACTION_QUEUE | MLX5_FLOW_ACTION_RSS)) {
struct mlx5_hrxq *hrxq;
@@ -3268,9 +3342,9 @@ struct field_modify_info modify_tcp[] = {
goto error;
}
dv->hrxq = hrxq;
- dv->actions[n].type = MLX5DV_FLOW_ACTION_DEST_IBV_QP;
- dv->actions[n].qp = hrxq->qp;
- n++;
+ dv->actions[n++] =
+ mlx5_glue->dv_create_flow_action_dest_ibv_qp
+ (dv->hrxq->qp);
}
dv->flow =
mlx5_glue->dv_create_flow(dv->matcher->matcher_object,
@@ -3448,6 +3522,10 @@ struct field_modify_info modify_tcp[] = {
flow_dv_counter_release(flow->counter);
flow->counter = NULL;
}
+ if (flow->tag_resource) {
+ flow_dv_tag_release(dev, flow->tag_resource);
+ flow->tag_resource = NULL;
+ }
while (!LIST_EMPTY(&flow->dev_flows)) {
dev_flow = LIST_FIRST(&flow->dev_flows);
LIST_REMOVE(dev_flow, next);
diff --git a/drivers/net/mlx5/mlx5_glue.c b/drivers/net/mlx5/mlx5_glue.c
index c817d86..cfd939d 100644
--- a/drivers/net/mlx5/mlx5_glue.c
+++ b/drivers/net/mlx5/mlx5_glue.c
@@ -175,10 +175,20 @@
}
static int
-mlx5_glue_destroy_flow_action(struct ibv_flow_action *action)
+mlx5_glue_destroy_flow_action(void *action)
{
#ifdef HAVE_IBV_FLOW_DV_SUPPORT
- return ibv_destroy_flow_action(action);
+ struct mlx5dv_flow_action_attr *attr = action;
+ int res = 0;
+ switch (attr->type) {
+ case MLX5DV_FLOW_ACTION_TAG:
+ break;
+ default:
+ res = ibv_destroy_flow_action(attr->action);
+ break;
+ }
+ free(action);
+ return res;
#else
(void)action;
return ENOTSUP;
@@ -430,16 +440,23 @@
mlx5_glue_dv_create_flow(struct mlx5dv_flow_matcher *matcher,
struct mlx5dv_flow_match_parameters *match_value,
size_t num_actions,
- struct mlx5dv_flow_action_attr *actions_attr)
+ void *actions[])
{
#ifdef HAVE_IBV_FLOW_DV_SUPPORT
+ struct mlx5dv_flow_action_attr actions_attr[8];
+
+ if (num_actions > 8)
+ return NULL;
+ for (size_t i = 0; i < num_actions; i++)
+ actions_attr[i] =
+ *((struct mlx5dv_flow_action_attr *)(actions[i]));
return mlx5dv_create_flow(matcher, match_value,
num_actions, actions_attr);
#else
(void)matcher;
(void)match_value;
(void)num_actions;
- (void)actions_attr;
+ (void)actions;
return NULL;
#endif
}
@@ -455,31 +472,45 @@
#endif
}
-static struct ibv_flow_action *
-mlx5_glue_dv_create_flow_action_packet_reformat
- (struct ibv_context *ctx,
- size_t data_sz,
- void *data,
- enum mlx5dv_flow_action_packet_reformat_type reformat_type,
- enum mlx5dv_flow_table_type ft_type)
+static void *
+mlx5_glue_dv_create_flow_action_counter(void *counter_obj, uint32_t offset)
{
#ifdef HAVE_IBV_FLOW_DV_SUPPORT
- return mlx5dv_create_flow_action_packet_reformat(ctx,
- data_sz,
- data,
- reformat_type,
- ft_type);
+ struct mlx5dv_flow_action_attr *action;
+
+ (void)offset;
+ action = malloc(sizeof(*action));
+ if (!action)
+ return NULL;
+ action->type = MLX5DV_FLOW_ACTION_COUNTER_DEVX;
+ action->obj = counter_obj;
+ return action;
#else
- (void)ctx;
- (void)data_sz;
- (void)data;
- (void)reformat_type;
- (void)ft_type;
+ (void)counter_obj;
+ (void)offset;
return NULL;
#endif
}
-static struct ibv_flow_action *
+static void *
+mlx5_glue_dv_create_flow_action_dest_ibv_qp(void *qp)
+{
+#ifdef HAVE_IBV_FLOW_DV_SUPPORT
+ struct mlx5dv_flow_action_attr *action;
+
+ action = malloc(sizeof(*action));
+ if (!action)
+ return NULL;
+ action->type = MLX5DV_FLOW_ACTION_DEST_IBV_QP;
+ action->obj = qp;
+ return action;
+#else
+ (void)qp;
+ return NULL;
+#endif
+}
+
+static void *
mlx5_glue_dv_create_flow_action_modify_header
(struct ibv_context *ctx,
size_t actions_sz,
@@ -487,8 +518,15 @@
enum mlx5dv_flow_table_type ft_type)
{
#ifdef HAVE_IBV_FLOW_DV_SUPPORT
- return mlx5dv_create_flow_action_modify_header(ctx, actions_sz,
- actions, ft_type);
+ struct mlx5dv_flow_action_attr *action;
+
+ action = malloc(sizeof(*action));
+ if (!action)
+ return NULL;
+ action->type = MLX5DV_FLOW_ACTION_IBV_FLOW_ACTION;
+ action->action = mlx5dv_create_flow_action_modify_header
+ (ctx, actions_sz, actions, ft_type);
+ return action;
#else
(void)ctx;
(void)actions_sz;
@@ -498,6 +536,50 @@
#endif
}
+static void *
+mlx5_glue_dv_create_flow_action_packet_reformat
+ (struct ibv_context *ctx,
+ size_t data_sz,
+ void *data,
+ enum mlx5dv_flow_action_packet_reformat_type reformat_type,
+ enum mlx5dv_flow_table_type ft_type)
+{
+#ifdef HAVE_IBV_FLOW_DV_SUPPORT
+ struct mlx5dv_flow_action_attr *action;
+
+ action = malloc(sizeof(*action));
+ if (!action)
+ return NULL;
+ action->type = MLX5DV_FLOW_ACTION_IBV_FLOW_ACTION;
+ action->action = mlx5dv_create_flow_action_packet_reformat
+ (ctx, data_sz, data, reformat_type, ft_type);
+ return action;
+#else
+ (void)ctx;
+ (void)data_sz;
+ (void)data;
+ (void)reformat_type;
+ (void)ft_type;
+ return NULL;
+#endif
+}
+
+static void *
+mlx5_glue_dv_create_flow_action_tag(uint32_t tag)
+{
+#ifdef HAVE_IBV_FLOW_DV_SUPPORT
+ struct mlx5dv_flow_action_attr *action;
+ action = malloc(sizeof(*action));
+ if (!action)
+ return NULL;
+ action->type = MLX5DV_FLOW_ACTION_TAG;
+ action->tag_value = tag;
+ return action;
+#endif
+ (void)tag;
+ return NULL;
+}
+
static struct ibv_context *
mlx5_glue_dv_open_device(struct ibv_device *device)
{
@@ -645,10 +727,15 @@
.dv_create_flow_matcher = mlx5_glue_dv_create_flow_matcher,
.dv_destroy_flow_matcher = mlx5_glue_dv_destroy_flow_matcher,
.dv_create_flow = mlx5_glue_dv_create_flow,
- .dv_create_flow_action_packet_reformat =
- mlx5_glue_dv_create_flow_action_packet_reformat,
+ .dv_create_flow_action_counter =
+ mlx5_glue_dv_create_flow_action_counter,
+ .dv_create_flow_action_dest_ibv_qp =
+ mlx5_glue_dv_create_flow_action_dest_ibv_qp,
.dv_create_flow_action_modify_header =
- mlx5_glue_dv_create_flow_action_modify_header,
+ mlx5_glue_dv_create_flow_action_modify_header,
+ .dv_create_flow_action_packet_reformat =
+ mlx5_glue_dv_create_flow_action_packet_reformat,
+ .dv_create_flow_action_tag = mlx5_glue_dv_create_flow_action_tag,
.dv_open_device = mlx5_glue_dv_open_device,
.devx_obj_create = mlx5_glue_devx_obj_create,
.devx_obj_destroy = mlx5_glue_devx_obj_destroy,
diff --git a/drivers/net/mlx5/mlx5_glue.h b/drivers/net/mlx5/mlx5_glue.h
index b118960..167f1f7 100644
--- a/drivers/net/mlx5/mlx5_glue.h
+++ b/drivers/net/mlx5/mlx5_glue.h
@@ -55,6 +55,10 @@
enum mlx5dv_flow_table_type { flow_table_type = 0, };
#endif
+#ifndef HAVE_IBV_FLOW_DEVX_COUNTERS
+#define MLX5DV_FLOW_ACTION_COUNTER_DEVX 0
+#endif
+
#ifndef HAVE_IBV_DEVX_OBJ
struct mlx5dv_devx_obj;
#endif
@@ -98,7 +102,7 @@ struct mlx5_glue {
struct ibv_flow *(*create_flow)(struct ibv_qp *qp,
struct ibv_flow_attr *flow);
int (*destroy_flow)(struct ibv_flow *flow_id);
- int (*destroy_flow_action)(struct ibv_flow_action *action);
+ int (*destroy_flow_action)(void *action);
struct ibv_qp *(*create_qp)(struct ibv_pd *pd,
struct ibv_qp_init_attr *qp_init_attr);
struct ibv_qp *(*create_qp_ex)
@@ -160,19 +164,17 @@ struct mlx5_glue {
int (*dv_destroy_flow_matcher)(struct mlx5dv_flow_matcher *matcher);
struct ibv_flow *(*dv_create_flow)(struct mlx5dv_flow_matcher *matcher,
struct mlx5dv_flow_match_parameters *match_value,
- size_t num_actions,
- struct mlx5dv_flow_action_attr *actions_attr);
- struct ibv_flow_action *(*dv_create_flow_action_packet_reformat)
- (struct ibv_context *ctx,
- size_t data_sz,
- void *data,
+ size_t num_actions, void *actions[]);
+ void *(*dv_create_flow_action_counter)(void *obj, uint32_t offset);
+ void *(*dv_create_flow_action_dest_ibv_qp)(void *qp);
+ void *(*dv_create_flow_action_modify_header)
+ (struct ibv_context *ctx, size_t actions_sz, uint64_t actions[],
+ enum mlx5dv_flow_table_type ft_type);
+ void *(*dv_create_flow_action_packet_reformat)
+ (struct ibv_context *ctx, size_t data_sz, void *data,
enum mlx5dv_flow_action_packet_reformat_type reformat_type,
enum mlx5dv_flow_table_type ft_type);
- struct ibv_flow_action *(*dv_create_flow_action_modify_header)
- (struct ibv_context *ctx,
- size_t actions_sz,
- uint64_t actions[],
- enum mlx5dv_flow_table_type ft_type);
+ void *(*dv_create_flow_action_tag)(uint32_t tag);
struct ibv_context *(*dv_open_device)(struct ibv_device *device);
struct mlx5dv_devx_obj *(*devx_obj_create)
(struct ibv_context *ctx,
--
1.8.3.1
^ permalink raw reply [flat|nested] 52+ messages in thread
* [dpdk-dev] [PATCH 2/3] net/mlx5: add Direct Rules API
2019-03-20 15:38 [dpdk-dev] [PATCH 0/3]net/mlx5: Add Direct Rule support Ori Kam
2019-03-20 15:38 ` Ori Kam
2019-03-20 15:39 ` [dpdk-dev] [PATCH 1/3] net/mlx5: prepare Direct Verbs for Direct Rule Ori Kam
@ 2019-03-20 15:39 ` Ori Kam
2019-03-20 15:39 ` Ori Kam
2019-03-20 15:39 ` [dpdk-dev] [PATCH 3/3] net/mlx5: add jump action support for NIC Ori Kam
2019-03-28 16:32 ` [dpdk-dev] [PATCH v2 0/3] net/mlx5: Add Direct Rule support Ori Kam
4 siblings, 1 reply; 52+ messages in thread
From: Ori Kam @ 2019-03-20 15:39 UTC (permalink / raw)
To: Shahaf Shuler, Yongseok Koh, Matan Azrad; +Cc: dev, Ori Kam, Slava Ovsiienko
Adds calls to the Direct Rules API inside the glue functions.
Due to difference in parameters between the Direct Rules and Direct
Verbs some of the glue functions API was updated.
Signed-off-by: Ori Kam <orika@mellanox.com>
---
drivers/net/mlx5/Makefile | 5 ++
drivers/net/mlx5/mlx5.c | 21 +++++
drivers/net/mlx5/mlx5.h | 15 ++++
drivers/net/mlx5/mlx5_flow.c | 1 +
drivers/net/mlx5/mlx5_flow.h | 6 +-
drivers/net/mlx5/mlx5_flow_dv.c | 81 ++++++++++++++++---
drivers/net/mlx5/mlx5_glue.c | 170 ++++++++++++++++++++++++++++++++++------
drivers/net/mlx5/mlx5_glue.h | 31 +++++---
drivers/net/mlx5/mlx5_prm.h | 24 +++++-
9 files changed, 307 insertions(+), 47 deletions(-)
diff --git a/drivers/net/mlx5/Makefile b/drivers/net/mlx5/Makefile
index 9a7da18..1ed299d 100644
--- a/drivers/net/mlx5/Makefile
+++ b/drivers/net/mlx5/Makefile
@@ -156,6 +156,11 @@ mlx5_autoconf.h.new: $(RTE_SDK)/buildtools/auto-config-h.sh
func mlx5dv_create_flow_action_packet_reformat \
$(AUTOCONF_OUTPUT)
$Q sh -- '$<' '$@' \
+ HAVE_MLX5DV_DR \
+ infiniband/mlx5dv.h \
+ enum MLX5DV_DR_NS_TYPE_TERMINATING \
+ $(AUTOCONF_OUTPUT)
+ $Q sh -- '$<' '$@' \
HAVE_IBV_DEVX_OBJ \
infiniband/mlx5dv.h \
func mlx5dv_devx_obj_create \
diff --git a/drivers/net/mlx5/mlx5.c b/drivers/net/mlx5/mlx5.c
index ae4b716..ad1975c 100644
--- a/drivers/net/mlx5/mlx5.c
+++ b/drivers/net/mlx5/mlx5.c
@@ -762,6 +762,9 @@
int own_domain_id = 0;
uint16_t port_id;
unsigned int i;
+#ifdef HAVE_MLX5DV_DR
+ void *ns;
+#endif
/* Determine if this port representor is supposed to be spawned. */
if (switch_info->representor && dpdk_dev->devargs) {
@@ -1219,6 +1222,24 @@
priv->tcf_context = NULL;
}
}
+#ifdef HAVE_MLX5DV_DR
+ ns = mlx5dv_dr_create_ns(ctx,
+ MLX5DV_DR_NS_DOMAIN_INGRESS_BYPASS);
+ if (ns == NULL) {
+ DRV_LOG(ERR, "mlx5dv_dr_create_ns failed");
+ err = errno;
+ goto error;
+ }
+ priv->rx_ns = ns;
+ ns = mlx5dv_dr_create_ns(ctx,
+ MLX5DV_DR_NS_DOMAIN_EGRESS_BYPASS);
+ if (ns == NULL) {
+ DRV_LOG(ERR, "mlx5dv_dr_create_ns failed");
+ err = errno;
+ goto error;
+ }
+ priv->tx_ns = ns;
+#endif
TAILQ_INIT(&priv->flows);
TAILQ_INIT(&priv->ctrl_flows);
/* Hint libmlx5 to use PMD allocator for data plane resources */
diff --git a/drivers/net/mlx5/mlx5.h b/drivers/net/mlx5/mlx5.h
index c0aa891..a0d4961 100644
--- a/drivers/net/mlx5/mlx5.h
+++ b/drivers/net/mlx5/mlx5.h
@@ -187,6 +187,15 @@ struct mlx5_drop {
struct mlx5_flow_tcf_context;
+/* Table structure. */
+struct mlx5_flow_tbl_resource {
+ void *obj; /**< Pointer to DR table object. */
+ rte_atomic32_t refcnt; /**< Reference counter. */
+};
+
+#define MLX5_MAX_TABLES 1024
+#define MLX5_GROUP_FACTOR 1
+
struct mlx5_priv {
LIST_ENTRY(mlx5_priv) mem_event_cb;
/**< Called by memory event callback. */
@@ -259,6 +268,12 @@ struct mlx5_priv {
/* UAR same-page access control required in 32bit implementations. */
#endif
struct mlx5_flow_tcf_context *tcf_context; /* TC flower context. */
+ void *rx_ns; /* RX Direct Rules name space handle. */
+ struct mlx5_flow_tbl_resource rx_tbl[MLX5_MAX_TABLES];
+ /* RX Direct Rules tables. */
+ void *tx_ns; /* TX Direct Rules name space handle. */
+ struct mlx5_flow_tbl_resource tx_tbl[MLX5_MAX_TABLES];
+ /* TX Direct Rules tables/ */
};
#define PORT_ID(priv) ((priv)->dev_data->port_id)
diff --git a/drivers/net/mlx5/mlx5_flow.c b/drivers/net/mlx5/mlx5_flow.c
index dea38e2..010d61a 100644
--- a/drivers/net/mlx5/mlx5_flow.c
+++ b/drivers/net/mlx5/mlx5_flow.c
@@ -2082,6 +2082,7 @@ uint32_t mlx5_flow_adjust_priority(struct rte_eth_dev *dev, int32_t priority,
flow_size += RTE_ALIGN_CEIL(sizeof(uint16_t), sizeof(void *));
flow = rte_calloc(__func__, 1, flow_size, 0);
flow->drv_type = flow_get_drv_type(dev, attr);
+ flow->ingress = attr->ingress;
assert(flow->drv_type > MLX5_FLOW_TYPE_MIN &&
flow->drv_type < MLX5_FLOW_TYPE_MAX);
flow->queue = (void *)(flow + 1);
diff --git a/drivers/net/mlx5/mlx5_flow.h b/drivers/net/mlx5/mlx5_flow.h
index 4f69ae2..8ba37a0 100644
--- a/drivers/net/mlx5/mlx5_flow.h
+++ b/drivers/net/mlx5/mlx5_flow.h
@@ -204,6 +204,7 @@ struct mlx5_flow_dv_matcher {
uint16_t crc; /**< CRC of key. */
uint16_t priority; /**< Priority of matcher. */
uint8_t egress; /**< Egress matcher. */
+ uint32_t group; /**< The matcher group. */
struct mlx5_flow_dv_match_params mask; /**< Matcher mask. */
};
@@ -220,6 +221,7 @@ struct mlx5_flow_dv_encap_decap_resource {
size_t size;
uint8_t reformat_type;
uint8_t ft_type;
+ uint64_t flags; /**< Flags for RDMA API. */
};
/* Tag resource structure. */
@@ -348,7 +350,7 @@ struct mlx5_flow_counter {
/* Flow structure. */
struct rte_flow {
TAILQ_ENTRY(rte_flow) next; /**< Pointer to the next flow structure. */
- enum mlx5_flow_drv_type drv_type; /**< Drvier type. */
+ enum mlx5_flow_drv_type drv_type; /**< Driver type. */
struct mlx5_flow_counter *counter; /**< Holds flow counter. */
struct mlx5_flow_dv_tag_resource *tag_resource;
/**< pointer to the tag action. */
@@ -360,6 +362,8 @@ struct rte_flow {
uint64_t actions;
/**< Bit-fields of detected actions, see MLX5_FLOW_ACTION_*. */
struct mlx5_fdir *fdir; /**< Pointer to associated FDIR if any. */
+ uint8_t ingress; /**< 1 if the flow is ingress. */
+ uint32_t group; /**< The group index. */
};
typedef int (*mlx5_flow_validate_t)(struct rte_eth_dev *dev,
diff --git a/drivers/net/mlx5/mlx5_flow_dv.c b/drivers/net/mlx5/mlx5_flow_dv.c
index 0eb3a28..228ba3f 100644
--- a/drivers/net/mlx5/mlx5_flow_dv.c
+++ b/drivers/net/mlx5/mlx5_flow_dv.c
@@ -805,11 +805,20 @@ struct field_modify_info modify_tcp[] = {
{
struct mlx5_priv *priv = dev->data->dev_private;
struct mlx5_flow_dv_encap_decap_resource *cache_resource;
+ struct rte_flow *flow = dev_flow->flow;
+ struct mlx5dv_dr_ns *ns;
+
+ resource->flags = flow->group ? 0 : 1;
+ if (flow->ingress)
+ ns = priv->rx_ns;
+ else
+ ns = priv->tx_ns;
/* Lookup a matching resource from cache. */
LIST_FOREACH(cache_resource, &priv->encaps_decaps, next) {
if (resource->reformat_type == cache_resource->reformat_type &&
resource->ft_type == cache_resource->ft_type &&
+ resource->flags == cache_resource->flags &&
resource->size == cache_resource->size &&
!memcmp((const void *)resource->buf,
(const void *)cache_resource->buf,
@@ -831,10 +840,10 @@ struct field_modify_info modify_tcp[] = {
*cache_resource = *resource;
cache_resource->verbs_action =
mlx5_glue->dv_create_flow_action_packet_reformat
- (priv->ctx, cache_resource->size,
- (cache_resource->size ? cache_resource->buf : NULL),
- cache_resource->reformat_type,
- cache_resource->ft_type);
+ (priv->ctx, cache_resource->reformat_type,
+ cache_resource->ft_type, ns, cache_resource->flags,
+ cache_resource->size,
+ (cache_resource->size ? cache_resource->buf : NULL));
if (!cache_resource->verbs_action) {
rte_free(cache_resource);
return rte_flow_error_set(error, ENOMEM,
@@ -1438,6 +1447,10 @@ struct field_modify_info modify_tcp[] = {
struct mlx5_priv *priv = dev->data->dev_private;
struct mlx5_flow_dv_modify_hdr_resource *cache_resource;
+ struct mlx5dv_dr_ns *ns =
+ resource->ft_type == MLX5DV_FLOW_TABLE_TYPE_NIC_TX ?
+ priv->tx_ns : priv->rx_ns;
+
/* Lookup a matching resource from cache. */
LIST_FOREACH(cache_resource, &priv->modify_cmds, next) {
if (resource->ft_type == cache_resource->ft_type &&
@@ -1463,11 +1476,11 @@ struct field_modify_info modify_tcp[] = {
*cache_resource = *resource;
cache_resource->verbs_action =
mlx5_glue->dv_create_flow_action_modify_header
- (priv->ctx,
+ (priv->ctx, cache_resource->ft_type,
+ ns, 0,
cache_resource->actions_num *
sizeof(cache_resource->actions[0]),
- (uint64_t *)cache_resource->actions,
- cache_resource->ft_type);
+ (uint64_t *)cache_resource->actions);
if (!cache_resource->verbs_action) {
rte_free(cache_resource);
return rte_flow_error_set(error, ENOMEM,
@@ -1592,11 +1605,13 @@ struct field_modify_info modify_tcp[] = {
struct mlx5_priv *priv = dev->data->dev_private;
uint32_t priority_max = priv->config.flow_prio - 1;
+#ifdef HAVE_MLX5DV_DR
if (attributes->group)
return rte_flow_error_set(error, ENOTSUP,
RTE_FLOW_ERROR_TYPE_ATTR_GROUP,
NULL,
"groups is not supported");
+#endif
if (attributes->priority != MLX5_FLOW_PRIO_RSVD &&
attributes->priority >= priority_max)
return rte_flow_error_set(error, ENOTSUP,
@@ -2723,7 +2738,11 @@ struct field_modify_info modify_tcp[] = {
match_criteria_enable |=
(!HEADER_IS_ZERO(match_criteria, misc_parameters_2)) <<
MLX5_MATCH_CRITERIA_ENABLE_MISC2_BIT;
-
+#ifdef HAVE_MLX5DV_DR
+ match_criteria_enable |=
+ (!HEADER_IS_ZERO(match_criteria, misc_parameters_3)) <<
+ MLX5_MATCH_CRITERIA_ENABLE_MISC3_BIT;
+#endif
return match_criteria_enable;
}
@@ -2754,12 +2773,14 @@ struct field_modify_info modify_tcp[] = {
.type = IBV_FLOW_ATTR_NORMAL,
.match_mask = (void *)&matcher->mask,
};
+ struct mlx5_flow_tbl_resource *tbl;
/* Lookup from cache. */
LIST_FOREACH(cache_matcher, &priv->matchers, next) {
if (matcher->crc == cache_matcher->crc &&
matcher->priority == cache_matcher->priority &&
matcher->egress == cache_matcher->egress &&
+ matcher->group == cache_matcher->group &&
!memcmp((const void *)matcher->mask.buf,
(const void *)cache_matcher->mask.buf,
cache_matcher->mask.size)) {
@@ -2774,6 +2795,27 @@ struct field_modify_info modify_tcp[] = {
return 0;
}
}
+#ifdef HAVE_MLX5DV_DR
+ if (matcher->egress) {
+ tbl = &priv->tx_tbl[matcher->group];
+ if (!tbl->obj)
+ tbl->obj = mlx5_glue->dr_create_flow_tbl
+ (priv->tx_ns,
+ matcher->group * MLX5_GROUP_FACTOR);
+ } else {
+ tbl = &priv->rx_tbl[matcher->group];
+ if (!tbl->obj)
+ tbl->obj = mlx5_glue->dr_create_flow_tbl
+ (priv->rx_ns,
+ matcher->group * MLX5_GROUP_FACTOR);
+ }
+ if (!tbl->obj)
+ return rte_flow_error_set(error, ENOMEM,
+ RTE_FLOW_ERROR_TYPE_UNSPECIFIED,
+ NULL, "cannot create table");
+
+ rte_atomic32_inc(&tbl->refcnt);
+#endif
/* Register new matcher. */
cache_matcher = rte_calloc(__func__, 1, sizeof(*cache_matcher), 0);
if (!cache_matcher)
@@ -2787,9 +2829,16 @@ struct field_modify_info modify_tcp[] = {
if (matcher->egress)
dv_attr.flags |= IBV_FLOW_ATTR_FLAGS_EGRESS;
cache_matcher->matcher_object =
- mlx5_glue->dv_create_flow_matcher(priv->ctx, &dv_attr);
+ mlx5_glue->dv_create_flow_matcher(priv->ctx, &dv_attr,
+ tbl->obj);
if (!cache_matcher->matcher_object) {
rte_free(cache_matcher);
+#ifdef HAVE_MLX5DV_DR
+ if (rte_atomic32_dec_and_test(&tbl->refcnt)) {
+ mlx5_glue->dr_destroy_flow_tbl(tbl->obj);
+ tbl->obj = NULL;
+ }
+#endif
return rte_flow_error_set(error, ENOMEM,
RTE_FLOW_ERROR_TYPE_UNSPECIFIED,
NULL, "cannot create matcher");
@@ -2801,6 +2850,7 @@ struct field_modify_info modify_tcp[] = {
cache_matcher->priority,
cache_matcher->egress ? "tx" : "rx", (void *)cache_matcher,
rte_atomic32_read(&cache_matcher->refcnt));
+ rte_atomic32_inc(&tbl->refcnt);
return 0;
}
@@ -3276,6 +3326,7 @@ struct field_modify_info modify_tcp[] = {
matcher.priority = mlx5_flow_adjust_priority(dev, priority,
matcher.priority);
matcher.egress = attr->egress;
+ matcher.group = attr->group;
if (flow_dv_matcher_register(dev, &matcher, dev_flow, error))
return -rte_errno;
return 0;
@@ -3391,6 +3442,8 @@ struct field_modify_info modify_tcp[] = {
struct mlx5_flow *flow)
{
struct mlx5_flow_dv_matcher *matcher = flow->dv.matcher;
+ struct mlx5_priv *priv = dev->data->dev_private;
+ struct mlx5_flow_tbl_resource *tbl;
assert(matcher->matcher_object);
DRV_LOG(DEBUG, "port %u matcher %p: refcnt %d--",
@@ -3400,6 +3453,14 @@ struct field_modify_info modify_tcp[] = {
claim_zero(mlx5_glue->dv_destroy_flow_matcher
(matcher->matcher_object));
LIST_REMOVE(matcher, next);
+ if (matcher->egress)
+ tbl = &priv->tx_tbl[matcher->group];
+ else
+ tbl = &priv->rx_tbl[matcher->group];
+ if (rte_atomic32_dec_and_test(&tbl->refcnt)) {
+ mlx5_glue->dr_destroy_flow_tbl(tbl->obj);
+ tbl->obj = NULL;
+ }
rte_free(matcher);
DRV_LOG(DEBUG, "port %u matcher %p: removed",
dev->data->port_id, (void *)matcher);
@@ -3489,7 +3550,7 @@ struct field_modify_info modify_tcp[] = {
LIST_FOREACH(dev_flow, &flow->dev_flows, next) {
dv = &dev_flow->dv;
if (dv->flow) {
- claim_zero(mlx5_glue->destroy_flow(dv->flow));
+ claim_zero(mlx5_glue->dv_destroy_flow(dv->flow));
dv->flow = NULL;
}
if (dv->hrxq) {
diff --git a/drivers/net/mlx5/mlx5_glue.c b/drivers/net/mlx5/mlx5_glue.c
index cfd939d..f509f85 100644
--- a/drivers/net/mlx5/mlx5_glue.c
+++ b/drivers/net/mlx5/mlx5_glue.c
@@ -178,6 +178,9 @@
mlx5_glue_destroy_flow_action(void *action)
{
#ifdef HAVE_IBV_FLOW_DV_SUPPORT
+#ifdef HAVE_MLX5DV_DR
+ return mlx5dv_dr_destroy_action(action);
+#else
struct mlx5dv_flow_action_attr *attr = action;
int res = 0;
switch (attr->type) {
@@ -189,6 +192,7 @@
}
free(action);
return res;
+#endif
#else
(void)action;
return ENOTSUP;
@@ -365,6 +369,53 @@
return ibv_cq_ex_to_cq(cq);
}
+static void *
+mlx5_glue_dr_create_flow_tbl(void *ns, uint32_t level)
+{
+#ifdef HAVE_MLX5DV_DR
+ return mlx5dv_dr_create_ft(ns, level);
+#else
+ (void)ns;
+ (void)level;
+ return NULL;
+#endif
+}
+
+static int
+mlx5_glue_dr_destroy_flow_tbl(void *tbl)
+{
+#ifdef HAVE_MLX5DV_DR
+ return mlx5dv_dr_destroy_ft(tbl);
+#else
+ (void)tbl;
+ return 0;
+#endif
+}
+
+static void *
+mlx5_glue_dr_create_ns(struct ibv_context *ctx,
+ enum mlx5dv_dr_ns_domain domain)
+{
+#ifdef HAVE_MLX5DV_DR
+ return mlx5dv_dr_create_ns(ctx, domain);
+#else
+ (void)ctx;
+ (void)domain;
+ return NULL;
+#endif
+}
+
+static int
+mlx5_glue_dr_destroy_ns(void *ns)
+{
+#ifdef HAVE_MLX5DV_DR
+ return mlx5dv_dr_destroy_ns(ns);
+#else
+ (void)ns;
+ return 0;
+#endif
+}
+
static struct ibv_cq_ex *
mlx5_glue_dv_create_cq(struct ibv_context *context,
struct ibv_cq_init_attr_ex *cq_attr,
@@ -423,26 +474,40 @@
#endif
}
-static struct mlx5dv_flow_matcher *
+static void *
mlx5_glue_dv_create_flow_matcher(struct ibv_context *context,
- struct mlx5dv_flow_matcher_attr *matcher_attr)
+ struct mlx5dv_flow_matcher_attr *matcher_attr,
+ void *tbl)
{
#ifdef HAVE_IBV_FLOW_DV_SUPPORT
+#ifdef HAVE_MLX5DV_DR
+ (void)context;
+ return mlx5dv_dr_create_matcher(tbl, matcher_attr->priority,
+ matcher_attr->match_criteria_enable,
+ matcher_attr->match_mask);
+#else
+ (void)tbl;
return mlx5dv_create_flow_matcher(context, matcher_attr);
+#endif
#else
(void)context;
(void)matcher_attr;
+ (void)tbl;
return NULL;
#endif
}
-static struct ibv_flow *
-mlx5_glue_dv_create_flow(struct mlx5dv_flow_matcher *matcher,
- struct mlx5dv_flow_match_parameters *match_value,
+static void *
+mlx5_glue_dv_create_flow(void *matcher,
+ void *match_value,
size_t num_actions,
void *actions[])
{
#ifdef HAVE_IBV_FLOW_DV_SUPPORT
+#ifdef HAVE_MLX5DV_DR
+ return mlx5dv_dr_create_rule(matcher, match_value, num_actions,
+ (struct mlx5dv_dr_action **)actions);
+#else
struct mlx5dv_flow_action_attr actions_attr[8];
if (num_actions > 8)
@@ -452,6 +517,7 @@
*((struct mlx5dv_flow_action_attr *)(actions[i]));
return mlx5dv_create_flow(matcher, match_value,
num_actions, actions_attr);
+#endif
#else
(void)matcher;
(void)match_value;
@@ -461,21 +527,13 @@
#endif
}
-static int
-mlx5_glue_dv_destroy_flow_matcher(struct mlx5dv_flow_matcher *matcher)
-{
-#ifdef HAVE_IBV_FLOW_DV_SUPPORT
- return mlx5dv_destroy_flow_matcher(matcher);
-#else
- (void)matcher;
- return 0;
-#endif
-}
-
static void *
mlx5_glue_dv_create_flow_action_counter(void *counter_obj, uint32_t offset)
{
#ifdef HAVE_IBV_FLOW_DV_SUPPORT
+#ifdef HAVE_MLX5DV_DR
+ return mlx5dv_dr_create_action_devx_counter(counter_obj, offset);
+#else
struct mlx5dv_flow_action_attr *action;
(void)offset;
@@ -485,6 +543,7 @@
action->type = MLX5DV_FLOW_ACTION_COUNTER_DEVX;
action->obj = counter_obj;
return action;
+#endif
#else
(void)counter_obj;
(void)offset;
@@ -496,6 +555,9 @@
mlx5_glue_dv_create_flow_action_dest_ibv_qp(void *qp)
{
#ifdef HAVE_IBV_FLOW_DV_SUPPORT
+#ifdef HAVE_MLX5DV_DR
+ return mlx5dv_dr_create_action_dest_ibv_qp(qp);
+#else
struct mlx5dv_flow_action_attr *action;
action = malloc(sizeof(*action));
@@ -504,6 +566,7 @@
action->type = MLX5DV_FLOW_ACTION_DEST_IBV_QP;
action->obj = qp;
return action;
+#endif
#else
(void)qp;
return NULL;
@@ -513,13 +576,22 @@
static void *
mlx5_glue_dv_create_flow_action_modify_header
(struct ibv_context *ctx,
+ enum mlx5dv_flow_table_type ft_type,
+ void *ns, uint64_t flags,
size_t actions_sz,
- uint64_t actions[],
- enum mlx5dv_flow_table_type ft_type)
+ uint64_t actions[])
{
#ifdef HAVE_IBV_FLOW_DV_SUPPORT
+#ifdef HAVE_MLX5DV_DR
+ (void)ctx;
+ (void)ft_type;
+ return mlx5dv_dr_create_action_modify_header(ns, flags, actions_sz,
+ actions);
+#else
struct mlx5dv_flow_action_attr *action;
+ (void)ns;
+ (void)flags;
action = malloc(sizeof(*action));
if (!action)
return NULL;
@@ -527,11 +599,14 @@
action->action = mlx5dv_create_flow_action_modify_header
(ctx, actions_sz, actions, ft_type);
return action;
+#endif
#else
(void)ctx;
+ (void)ft_type;
+ (void)ns;
+ (void)flags;
(void)actions_sz;
(void)actions;
- (void)ft_type;
return NULL;
#endif
}
@@ -539,12 +614,20 @@
static void *
mlx5_glue_dv_create_flow_action_packet_reformat
(struct ibv_context *ctx,
- size_t data_sz,
- void *data,
enum mlx5dv_flow_action_packet_reformat_type reformat_type,
- enum mlx5dv_flow_table_type ft_type)
+ enum mlx5dv_flow_table_type ft_type, struct mlx5dv_dr_ns *ns,
+ uint32_t flags, size_t data_sz, void *data)
{
#ifdef HAVE_IBV_FLOW_DV_SUPPORT
+#ifdef HAVE_MLX5DV_DR
+ (void)ctx;
+ (void)ft_type;
+ return mlx5dv_dr_create_action_packet_reformat(ns, flags,
+ reformat_type, data_sz,
+ data);
+#else
+ (void)ns;
+ (void)flags;
struct mlx5dv_flow_action_attr *action;
action = malloc(sizeof(*action));
@@ -554,12 +637,15 @@
action->action = mlx5dv_create_flow_action_packet_reformat
(ctx, data_sz, data, reformat_type, ft_type);
return action;
+#endif
#else
(void)ctx;
- (void)data_sz;
- (void)data;
(void)reformat_type;
(void)ft_type;
+ (void)ns;
+ (void)flags;
+ (void)data_sz;
+ (void)data;
return NULL;
#endif
}
@@ -568,6 +654,9 @@
mlx5_glue_dv_create_flow_action_tag(uint32_t tag)
{
#ifdef HAVE_IBV_FLOW_DV_SUPPORT
+#ifdef HAVE_MLX5DV_DR
+ return mlx5dv_dr_create_action_tag(tag);
+#else
struct mlx5dv_flow_action_attr *action;
action = malloc(sizeof(*action));
if (!action)
@@ -576,10 +665,36 @@
action->tag_value = tag;
return action;
#endif
+#endif
(void)tag;
return NULL;
}
+static int
+mlx5_glue_dv_destroy_flow(void *flow_id)
+{
+#ifdef HAVE_MLX5DV_DR
+ return mlx5dv_dr_destroy_rule(flow_id);
+#else
+ return ibv_destroy_flow(flow_id);
+#endif
+}
+
+static int
+mlx5_glue_dv_destroy_flow_matcher(void *matcher)
+{
+#ifdef HAVE_IBV_FLOW_DV_SUPPORT
+#ifdef HAVE_MLX5DV_DR
+ return mlx5dv_dr_destroy_matcher(matcher);
+#else
+ return mlx5dv_destroy_flow_matcher(matcher);
+#endif
+#else
+ (void)matcher;
+ return 0;
+#endif
+}
+
static struct ibv_context *
mlx5_glue_dv_open_device(struct ibv_device *device)
{
@@ -718,6 +833,10 @@
.get_async_event = mlx5_glue_get_async_event,
.port_state_str = mlx5_glue_port_state_str,
.cq_ex_to_cq = mlx5_glue_cq_ex_to_cq,
+ .dr_create_flow_tbl = mlx5_glue_dr_create_flow_tbl,
+ .dr_destroy_flow_tbl = mlx5_glue_dr_destroy_flow_tbl,
+ .dr_create_ns = mlx5_glue_dr_create_ns,
+ .dr_destroy_ns = mlx5_glue_dr_destroy_ns,
.dv_create_cq = mlx5_glue_dv_create_cq,
.dv_create_wq = mlx5_glue_dv_create_wq,
.dv_query_device = mlx5_glue_dv_query_device,
@@ -725,7 +844,6 @@
.dv_init_obj = mlx5_glue_dv_init_obj,
.dv_create_qp = mlx5_glue_dv_create_qp,
.dv_create_flow_matcher = mlx5_glue_dv_create_flow_matcher,
- .dv_destroy_flow_matcher = mlx5_glue_dv_destroy_flow_matcher,
.dv_create_flow = mlx5_glue_dv_create_flow,
.dv_create_flow_action_counter =
mlx5_glue_dv_create_flow_action_counter,
@@ -736,6 +854,8 @@
.dv_create_flow_action_packet_reformat =
mlx5_glue_dv_create_flow_action_packet_reformat,
.dv_create_flow_action_tag = mlx5_glue_dv_create_flow_action_tag,
+ .dv_destroy_flow = mlx5_glue_dv_destroy_flow,
+ .dv_destroy_flow_matcher = mlx5_glue_dv_destroy_flow_matcher,
.dv_open_device = mlx5_glue_dv_open_device,
.devx_obj_create = mlx5_glue_devx_obj_create,
.devx_obj_destroy = mlx5_glue_devx_obj_destroy,
diff --git a/drivers/net/mlx5/mlx5_glue.h b/drivers/net/mlx5/mlx5_glue.h
index 167f1f7..7115575 100644
--- a/drivers/net/mlx5/mlx5_glue.h
+++ b/drivers/net/mlx5/mlx5_glue.h
@@ -63,6 +63,11 @@
struct mlx5dv_devx_obj;
#endif
+#ifndef HAVE_MLX5DV_DR
+struct mlx5dv_dr_ns;
+enum mlx5dv_dr_ns_domain { unused, };
+#endif
+
/* LIB_GLUE_VERSION must be updated every time this structure is modified. */
struct mlx5_glue {
const char *version;
@@ -140,6 +145,11 @@ struct mlx5_glue {
struct ibv_async_event *event);
const char *(*port_state_str)(enum ibv_port_state port_state);
struct ibv_cq *(*cq_ex_to_cq)(struct ibv_cq_ex *cq);
+ void *(*dr_create_flow_tbl)(void *ns, uint32_t level);
+ int (*dr_destroy_flow_tbl)(void *tbl);
+ void *(*dr_create_ns)(struct ibv_context *ctx,
+ enum mlx5dv_dr_ns_domain domain);
+ int (*dr_destroy_ns)(void *ns);
struct ibv_cq_ex *(*dv_create_cq)
(struct ibv_context *context,
struct ibv_cq_init_attr_ex *cq_attr,
@@ -158,23 +168,26 @@ struct mlx5_glue {
(struct ibv_context *context,
struct ibv_qp_init_attr_ex *qp_init_attr_ex,
struct mlx5dv_qp_init_attr *dv_qp_init_attr);
- struct mlx5dv_flow_matcher *(*dv_create_flow_matcher)
+ void *(*dv_create_flow_matcher)
(struct ibv_context *context,
- struct mlx5dv_flow_matcher_attr *matcher_attr);
- int (*dv_destroy_flow_matcher)(struct mlx5dv_flow_matcher *matcher);
- struct ibv_flow *(*dv_create_flow)(struct mlx5dv_flow_matcher *matcher,
- struct mlx5dv_flow_match_parameters *match_value,
+ struct mlx5dv_flow_matcher_attr *matcher_attr,
+ void *tbl);
+ void *(*dv_create_flow)(void *matcher, void *match_value,
size_t num_actions, void *actions[]);
void *(*dv_create_flow_action_counter)(void *obj, uint32_t offset);
void *(*dv_create_flow_action_dest_ibv_qp)(void *qp);
void *(*dv_create_flow_action_modify_header)
- (struct ibv_context *ctx, size_t actions_sz, uint64_t actions[],
- enum mlx5dv_flow_table_type ft_type);
+ (struct ibv_context *ctx, enum mlx5dv_flow_table_type ft_type,
+ void *ns, uint64_t flags, size_t actions_sz,
+ uint64_t actions[]);
void *(*dv_create_flow_action_packet_reformat)
- (struct ibv_context *ctx, size_t data_sz, void *data,
+ (struct ibv_context *ctx,
enum mlx5dv_flow_action_packet_reformat_type reformat_type,
- enum mlx5dv_flow_table_type ft_type);
+ enum mlx5dv_flow_table_type ft_type, struct mlx5dv_dr_ns *ns,
+ uint32_t flags, size_t data_sz, void *data);
void *(*dv_create_flow_action_tag)(uint32_t tag);
+ int (*dv_destroy_flow)(void *flow);
+ int (*dv_destroy_flow_matcher)(void *matcher);
struct ibv_context *(*dv_open_device)(struct ibv_device *device);
struct mlx5dv_devx_obj *(*devx_obj_create)
(struct ibv_context *ctx,
diff --git a/drivers/net/mlx5/mlx5_prm.h b/drivers/net/mlx5/mlx5_prm.h
index da1219e..b15266f 100644
--- a/drivers/net/mlx5/mlx5_prm.h
+++ b/drivers/net/mlx5/mlx5_prm.h
@@ -492,20 +492,40 @@ struct mlx5_ifc_fte_match_set_misc2_bits {
u8 reserved_at_1a0[0x60];
};
+struct mlx5_ifc_fte_match_set_misc3_bits {
+ u8 inner_tcp_seq_num[0x20];
+ u8 outer_tcp_seq_num[0x20];
+ u8 inner_tcp_ack_num[0x20];
+ u8 outer_tcp_ack_num[0x20];
+ u8 reserved_at_auto1[0x8];
+ u8 outer_vxlan_gpe_vni[0x18];
+ u8 outer_vxlan_gpe_next_protocol[0x8];
+ u8 outer_vxlan_gpe_flags[0x8];
+ u8 reserved_at_a8[0x10];
+ u8 icmp_header_data[0x20];
+ u8 icmpv6_header_data[0x20];
+ u8 icmp_type[0x8];
+ u8 icmp_code[0x8];
+ u8 icmpv6_type[0x8];
+ u8 icmpv6_code[0x8];
+ u8 reserved_at_1a0[0xe0];
+};
+
/* Flow matcher. */
struct mlx5_ifc_fte_match_param_bits {
struct mlx5_ifc_fte_match_set_lyr_2_4_bits outer_headers;
struct mlx5_ifc_fte_match_set_misc_bits misc_parameters;
struct mlx5_ifc_fte_match_set_lyr_2_4_bits inner_headers;
struct mlx5_ifc_fte_match_set_misc2_bits misc_parameters_2;
- u8 reserved_at_800[0x800];
+ struct mlx5_ifc_fte_match_set_misc3_bits misc_parameters_3;
};
enum {
MLX5_MATCH_CRITERIA_ENABLE_OUTER_BIT,
MLX5_MATCH_CRITERIA_ENABLE_MISC_BIT,
MLX5_MATCH_CRITERIA_ENABLE_INNER_BIT,
- MLX5_MATCH_CRITERIA_ENABLE_MISC2_BIT
+ MLX5_MATCH_CRITERIA_ENABLE_MISC2_BIT,
+ MLX5_MATCH_CRITERIA_ENABLE_MISC3_BIT
};
enum {
--
1.8.3.1
^ permalink raw reply [flat|nested] 52+ messages in thread
* [dpdk-dev] [PATCH 2/3] net/mlx5: add Direct Rules API
2019-03-20 15:39 ` [dpdk-dev] [PATCH 2/3] net/mlx5: add Direct Rules API Ori Kam
@ 2019-03-20 15:39 ` Ori Kam
0 siblings, 0 replies; 52+ messages in thread
From: Ori Kam @ 2019-03-20 15:39 UTC (permalink / raw)
To: Shahaf Shuler, Yongseok Koh, Matan Azrad; +Cc: dev, Ori Kam, Slava Ovsiienko
Adds calls to the Direct Rules API inside the glue functions.
Due to difference in parameters between the Direct Rules and Direct
Verbs some of the glue functions API was updated.
Signed-off-by: Ori Kam <orika@mellanox.com>
---
drivers/net/mlx5/Makefile | 5 ++
drivers/net/mlx5/mlx5.c | 21 +++++
drivers/net/mlx5/mlx5.h | 15 ++++
drivers/net/mlx5/mlx5_flow.c | 1 +
drivers/net/mlx5/mlx5_flow.h | 6 +-
drivers/net/mlx5/mlx5_flow_dv.c | 81 ++++++++++++++++---
drivers/net/mlx5/mlx5_glue.c | 170 ++++++++++++++++++++++++++++++++++------
drivers/net/mlx5/mlx5_glue.h | 31 +++++---
drivers/net/mlx5/mlx5_prm.h | 24 +++++-
9 files changed, 307 insertions(+), 47 deletions(-)
diff --git a/drivers/net/mlx5/Makefile b/drivers/net/mlx5/Makefile
index 9a7da18..1ed299d 100644
--- a/drivers/net/mlx5/Makefile
+++ b/drivers/net/mlx5/Makefile
@@ -156,6 +156,11 @@ mlx5_autoconf.h.new: $(RTE_SDK)/buildtools/auto-config-h.sh
func mlx5dv_create_flow_action_packet_reformat \
$(AUTOCONF_OUTPUT)
$Q sh -- '$<' '$@' \
+ HAVE_MLX5DV_DR \
+ infiniband/mlx5dv.h \
+ enum MLX5DV_DR_NS_TYPE_TERMINATING \
+ $(AUTOCONF_OUTPUT)
+ $Q sh -- '$<' '$@' \
HAVE_IBV_DEVX_OBJ \
infiniband/mlx5dv.h \
func mlx5dv_devx_obj_create \
diff --git a/drivers/net/mlx5/mlx5.c b/drivers/net/mlx5/mlx5.c
index ae4b716..ad1975c 100644
--- a/drivers/net/mlx5/mlx5.c
+++ b/drivers/net/mlx5/mlx5.c
@@ -762,6 +762,9 @@
int own_domain_id = 0;
uint16_t port_id;
unsigned int i;
+#ifdef HAVE_MLX5DV_DR
+ void *ns;
+#endif
/* Determine if this port representor is supposed to be spawned. */
if (switch_info->representor && dpdk_dev->devargs) {
@@ -1219,6 +1222,24 @@
priv->tcf_context = NULL;
}
}
+#ifdef HAVE_MLX5DV_DR
+ ns = mlx5dv_dr_create_ns(ctx,
+ MLX5DV_DR_NS_DOMAIN_INGRESS_BYPASS);
+ if (ns == NULL) {
+ DRV_LOG(ERR, "mlx5dv_dr_create_ns failed");
+ err = errno;
+ goto error;
+ }
+ priv->rx_ns = ns;
+ ns = mlx5dv_dr_create_ns(ctx,
+ MLX5DV_DR_NS_DOMAIN_EGRESS_BYPASS);
+ if (ns == NULL) {
+ DRV_LOG(ERR, "mlx5dv_dr_create_ns failed");
+ err = errno;
+ goto error;
+ }
+ priv->tx_ns = ns;
+#endif
TAILQ_INIT(&priv->flows);
TAILQ_INIT(&priv->ctrl_flows);
/* Hint libmlx5 to use PMD allocator for data plane resources */
diff --git a/drivers/net/mlx5/mlx5.h b/drivers/net/mlx5/mlx5.h
index c0aa891..a0d4961 100644
--- a/drivers/net/mlx5/mlx5.h
+++ b/drivers/net/mlx5/mlx5.h
@@ -187,6 +187,15 @@ struct mlx5_drop {
struct mlx5_flow_tcf_context;
+/* Table structure. */
+struct mlx5_flow_tbl_resource {
+ void *obj; /**< Pointer to DR table object. */
+ rte_atomic32_t refcnt; /**< Reference counter. */
+};
+
+#define MLX5_MAX_TABLES 1024
+#define MLX5_GROUP_FACTOR 1
+
struct mlx5_priv {
LIST_ENTRY(mlx5_priv) mem_event_cb;
/**< Called by memory event callback. */
@@ -259,6 +268,12 @@ struct mlx5_priv {
/* UAR same-page access control required in 32bit implementations. */
#endif
struct mlx5_flow_tcf_context *tcf_context; /* TC flower context. */
+ void *rx_ns; /* RX Direct Rules name space handle. */
+ struct mlx5_flow_tbl_resource rx_tbl[MLX5_MAX_TABLES];
+ /* RX Direct Rules tables. */
+ void *tx_ns; /* TX Direct Rules name space handle. */
+ struct mlx5_flow_tbl_resource tx_tbl[MLX5_MAX_TABLES];
+ /* TX Direct Rules tables/ */
};
#define PORT_ID(priv) ((priv)->dev_data->port_id)
diff --git a/drivers/net/mlx5/mlx5_flow.c b/drivers/net/mlx5/mlx5_flow.c
index dea38e2..010d61a 100644
--- a/drivers/net/mlx5/mlx5_flow.c
+++ b/drivers/net/mlx5/mlx5_flow.c
@@ -2082,6 +2082,7 @@ uint32_t mlx5_flow_adjust_priority(struct rte_eth_dev *dev, int32_t priority,
flow_size += RTE_ALIGN_CEIL(sizeof(uint16_t), sizeof(void *));
flow = rte_calloc(__func__, 1, flow_size, 0);
flow->drv_type = flow_get_drv_type(dev, attr);
+ flow->ingress = attr->ingress;
assert(flow->drv_type > MLX5_FLOW_TYPE_MIN &&
flow->drv_type < MLX5_FLOW_TYPE_MAX);
flow->queue = (void *)(flow + 1);
diff --git a/drivers/net/mlx5/mlx5_flow.h b/drivers/net/mlx5/mlx5_flow.h
index 4f69ae2..8ba37a0 100644
--- a/drivers/net/mlx5/mlx5_flow.h
+++ b/drivers/net/mlx5/mlx5_flow.h
@@ -204,6 +204,7 @@ struct mlx5_flow_dv_matcher {
uint16_t crc; /**< CRC of key. */
uint16_t priority; /**< Priority of matcher. */
uint8_t egress; /**< Egress matcher. */
+ uint32_t group; /**< The matcher group. */
struct mlx5_flow_dv_match_params mask; /**< Matcher mask. */
};
@@ -220,6 +221,7 @@ struct mlx5_flow_dv_encap_decap_resource {
size_t size;
uint8_t reformat_type;
uint8_t ft_type;
+ uint64_t flags; /**< Flags for RDMA API. */
};
/* Tag resource structure. */
@@ -348,7 +350,7 @@ struct mlx5_flow_counter {
/* Flow structure. */
struct rte_flow {
TAILQ_ENTRY(rte_flow) next; /**< Pointer to the next flow structure. */
- enum mlx5_flow_drv_type drv_type; /**< Drvier type. */
+ enum mlx5_flow_drv_type drv_type; /**< Driver type. */
struct mlx5_flow_counter *counter; /**< Holds flow counter. */
struct mlx5_flow_dv_tag_resource *tag_resource;
/**< pointer to the tag action. */
@@ -360,6 +362,8 @@ struct rte_flow {
uint64_t actions;
/**< Bit-fields of detected actions, see MLX5_FLOW_ACTION_*. */
struct mlx5_fdir *fdir; /**< Pointer to associated FDIR if any. */
+ uint8_t ingress; /**< 1 if the flow is ingress. */
+ uint32_t group; /**< The group index. */
};
typedef int (*mlx5_flow_validate_t)(struct rte_eth_dev *dev,
diff --git a/drivers/net/mlx5/mlx5_flow_dv.c b/drivers/net/mlx5/mlx5_flow_dv.c
index 0eb3a28..228ba3f 100644
--- a/drivers/net/mlx5/mlx5_flow_dv.c
+++ b/drivers/net/mlx5/mlx5_flow_dv.c
@@ -805,11 +805,20 @@ struct field_modify_info modify_tcp[] = {
{
struct mlx5_priv *priv = dev->data->dev_private;
struct mlx5_flow_dv_encap_decap_resource *cache_resource;
+ struct rte_flow *flow = dev_flow->flow;
+ struct mlx5dv_dr_ns *ns;
+
+ resource->flags = flow->group ? 0 : 1;
+ if (flow->ingress)
+ ns = priv->rx_ns;
+ else
+ ns = priv->tx_ns;
/* Lookup a matching resource from cache. */
LIST_FOREACH(cache_resource, &priv->encaps_decaps, next) {
if (resource->reformat_type == cache_resource->reformat_type &&
resource->ft_type == cache_resource->ft_type &&
+ resource->flags == cache_resource->flags &&
resource->size == cache_resource->size &&
!memcmp((const void *)resource->buf,
(const void *)cache_resource->buf,
@@ -831,10 +840,10 @@ struct field_modify_info modify_tcp[] = {
*cache_resource = *resource;
cache_resource->verbs_action =
mlx5_glue->dv_create_flow_action_packet_reformat
- (priv->ctx, cache_resource->size,
- (cache_resource->size ? cache_resource->buf : NULL),
- cache_resource->reformat_type,
- cache_resource->ft_type);
+ (priv->ctx, cache_resource->reformat_type,
+ cache_resource->ft_type, ns, cache_resource->flags,
+ cache_resource->size,
+ (cache_resource->size ? cache_resource->buf : NULL));
if (!cache_resource->verbs_action) {
rte_free(cache_resource);
return rte_flow_error_set(error, ENOMEM,
@@ -1438,6 +1447,10 @@ struct field_modify_info modify_tcp[] = {
struct mlx5_priv *priv = dev->data->dev_private;
struct mlx5_flow_dv_modify_hdr_resource *cache_resource;
+ struct mlx5dv_dr_ns *ns =
+ resource->ft_type == MLX5DV_FLOW_TABLE_TYPE_NIC_TX ?
+ priv->tx_ns : priv->rx_ns;
+
/* Lookup a matching resource from cache. */
LIST_FOREACH(cache_resource, &priv->modify_cmds, next) {
if (resource->ft_type == cache_resource->ft_type &&
@@ -1463,11 +1476,11 @@ struct field_modify_info modify_tcp[] = {
*cache_resource = *resource;
cache_resource->verbs_action =
mlx5_glue->dv_create_flow_action_modify_header
- (priv->ctx,
+ (priv->ctx, cache_resource->ft_type,
+ ns, 0,
cache_resource->actions_num *
sizeof(cache_resource->actions[0]),
- (uint64_t *)cache_resource->actions,
- cache_resource->ft_type);
+ (uint64_t *)cache_resource->actions);
if (!cache_resource->verbs_action) {
rte_free(cache_resource);
return rte_flow_error_set(error, ENOMEM,
@@ -1592,11 +1605,13 @@ struct field_modify_info modify_tcp[] = {
struct mlx5_priv *priv = dev->data->dev_private;
uint32_t priority_max = priv->config.flow_prio - 1;
+#ifdef HAVE_MLX5DV_DR
if (attributes->group)
return rte_flow_error_set(error, ENOTSUP,
RTE_FLOW_ERROR_TYPE_ATTR_GROUP,
NULL,
"groups is not supported");
+#endif
if (attributes->priority != MLX5_FLOW_PRIO_RSVD &&
attributes->priority >= priority_max)
return rte_flow_error_set(error, ENOTSUP,
@@ -2723,7 +2738,11 @@ struct field_modify_info modify_tcp[] = {
match_criteria_enable |=
(!HEADER_IS_ZERO(match_criteria, misc_parameters_2)) <<
MLX5_MATCH_CRITERIA_ENABLE_MISC2_BIT;
-
+#ifdef HAVE_MLX5DV_DR
+ match_criteria_enable |=
+ (!HEADER_IS_ZERO(match_criteria, misc_parameters_3)) <<
+ MLX5_MATCH_CRITERIA_ENABLE_MISC3_BIT;
+#endif
return match_criteria_enable;
}
@@ -2754,12 +2773,14 @@ struct field_modify_info modify_tcp[] = {
.type = IBV_FLOW_ATTR_NORMAL,
.match_mask = (void *)&matcher->mask,
};
+ struct mlx5_flow_tbl_resource *tbl;
/* Lookup from cache. */
LIST_FOREACH(cache_matcher, &priv->matchers, next) {
if (matcher->crc == cache_matcher->crc &&
matcher->priority == cache_matcher->priority &&
matcher->egress == cache_matcher->egress &&
+ matcher->group == cache_matcher->group &&
!memcmp((const void *)matcher->mask.buf,
(const void *)cache_matcher->mask.buf,
cache_matcher->mask.size)) {
@@ -2774,6 +2795,27 @@ struct field_modify_info modify_tcp[] = {
return 0;
}
}
+#ifdef HAVE_MLX5DV_DR
+ if (matcher->egress) {
+ tbl = &priv->tx_tbl[matcher->group];
+ if (!tbl->obj)
+ tbl->obj = mlx5_glue->dr_create_flow_tbl
+ (priv->tx_ns,
+ matcher->group * MLX5_GROUP_FACTOR);
+ } else {
+ tbl = &priv->rx_tbl[matcher->group];
+ if (!tbl->obj)
+ tbl->obj = mlx5_glue->dr_create_flow_tbl
+ (priv->rx_ns,
+ matcher->group * MLX5_GROUP_FACTOR);
+ }
+ if (!tbl->obj)
+ return rte_flow_error_set(error, ENOMEM,
+ RTE_FLOW_ERROR_TYPE_UNSPECIFIED,
+ NULL, "cannot create table");
+
+ rte_atomic32_inc(&tbl->refcnt);
+#endif
/* Register new matcher. */
cache_matcher = rte_calloc(__func__, 1, sizeof(*cache_matcher), 0);
if (!cache_matcher)
@@ -2787,9 +2829,16 @@ struct field_modify_info modify_tcp[] = {
if (matcher->egress)
dv_attr.flags |= IBV_FLOW_ATTR_FLAGS_EGRESS;
cache_matcher->matcher_object =
- mlx5_glue->dv_create_flow_matcher(priv->ctx, &dv_attr);
+ mlx5_glue->dv_create_flow_matcher(priv->ctx, &dv_attr,
+ tbl->obj);
if (!cache_matcher->matcher_object) {
rte_free(cache_matcher);
+#ifdef HAVE_MLX5DV_DR
+ if (rte_atomic32_dec_and_test(&tbl->refcnt)) {
+ mlx5_glue->dr_destroy_flow_tbl(tbl->obj);
+ tbl->obj = NULL;
+ }
+#endif
return rte_flow_error_set(error, ENOMEM,
RTE_FLOW_ERROR_TYPE_UNSPECIFIED,
NULL, "cannot create matcher");
@@ -2801,6 +2850,7 @@ struct field_modify_info modify_tcp[] = {
cache_matcher->priority,
cache_matcher->egress ? "tx" : "rx", (void *)cache_matcher,
rte_atomic32_read(&cache_matcher->refcnt));
+ rte_atomic32_inc(&tbl->refcnt);
return 0;
}
@@ -3276,6 +3326,7 @@ struct field_modify_info modify_tcp[] = {
matcher.priority = mlx5_flow_adjust_priority(dev, priority,
matcher.priority);
matcher.egress = attr->egress;
+ matcher.group = attr->group;
if (flow_dv_matcher_register(dev, &matcher, dev_flow, error))
return -rte_errno;
return 0;
@@ -3391,6 +3442,8 @@ struct field_modify_info modify_tcp[] = {
struct mlx5_flow *flow)
{
struct mlx5_flow_dv_matcher *matcher = flow->dv.matcher;
+ struct mlx5_priv *priv = dev->data->dev_private;
+ struct mlx5_flow_tbl_resource *tbl;
assert(matcher->matcher_object);
DRV_LOG(DEBUG, "port %u matcher %p: refcnt %d--",
@@ -3400,6 +3453,14 @@ struct field_modify_info modify_tcp[] = {
claim_zero(mlx5_glue->dv_destroy_flow_matcher
(matcher->matcher_object));
LIST_REMOVE(matcher, next);
+ if (matcher->egress)
+ tbl = &priv->tx_tbl[matcher->group];
+ else
+ tbl = &priv->rx_tbl[matcher->group];
+ if (rte_atomic32_dec_and_test(&tbl->refcnt)) {
+ mlx5_glue->dr_destroy_flow_tbl(tbl->obj);
+ tbl->obj = NULL;
+ }
rte_free(matcher);
DRV_LOG(DEBUG, "port %u matcher %p: removed",
dev->data->port_id, (void *)matcher);
@@ -3489,7 +3550,7 @@ struct field_modify_info modify_tcp[] = {
LIST_FOREACH(dev_flow, &flow->dev_flows, next) {
dv = &dev_flow->dv;
if (dv->flow) {
- claim_zero(mlx5_glue->destroy_flow(dv->flow));
+ claim_zero(mlx5_glue->dv_destroy_flow(dv->flow));
dv->flow = NULL;
}
if (dv->hrxq) {
diff --git a/drivers/net/mlx5/mlx5_glue.c b/drivers/net/mlx5/mlx5_glue.c
index cfd939d..f509f85 100644
--- a/drivers/net/mlx5/mlx5_glue.c
+++ b/drivers/net/mlx5/mlx5_glue.c
@@ -178,6 +178,9 @@
mlx5_glue_destroy_flow_action(void *action)
{
#ifdef HAVE_IBV_FLOW_DV_SUPPORT
+#ifdef HAVE_MLX5DV_DR
+ return mlx5dv_dr_destroy_action(action);
+#else
struct mlx5dv_flow_action_attr *attr = action;
int res = 0;
switch (attr->type) {
@@ -189,6 +192,7 @@
}
free(action);
return res;
+#endif
#else
(void)action;
return ENOTSUP;
@@ -365,6 +369,53 @@
return ibv_cq_ex_to_cq(cq);
}
+static void *
+mlx5_glue_dr_create_flow_tbl(void *ns, uint32_t level)
+{
+#ifdef HAVE_MLX5DV_DR
+ return mlx5dv_dr_create_ft(ns, level);
+#else
+ (void)ns;
+ (void)level;
+ return NULL;
+#endif
+}
+
+static int
+mlx5_glue_dr_destroy_flow_tbl(void *tbl)
+{
+#ifdef HAVE_MLX5DV_DR
+ return mlx5dv_dr_destroy_ft(tbl);
+#else
+ (void)tbl;
+ return 0;
+#endif
+}
+
+static void *
+mlx5_glue_dr_create_ns(struct ibv_context *ctx,
+ enum mlx5dv_dr_ns_domain domain)
+{
+#ifdef HAVE_MLX5DV_DR
+ return mlx5dv_dr_create_ns(ctx, domain);
+#else
+ (void)ctx;
+ (void)domain;
+ return NULL;
+#endif
+}
+
+static int
+mlx5_glue_dr_destroy_ns(void *ns)
+{
+#ifdef HAVE_MLX5DV_DR
+ return mlx5dv_dr_destroy_ns(ns);
+#else
+ (void)ns;
+ return 0;
+#endif
+}
+
static struct ibv_cq_ex *
mlx5_glue_dv_create_cq(struct ibv_context *context,
struct ibv_cq_init_attr_ex *cq_attr,
@@ -423,26 +474,40 @@
#endif
}
-static struct mlx5dv_flow_matcher *
+static void *
mlx5_glue_dv_create_flow_matcher(struct ibv_context *context,
- struct mlx5dv_flow_matcher_attr *matcher_attr)
+ struct mlx5dv_flow_matcher_attr *matcher_attr,
+ void *tbl)
{
#ifdef HAVE_IBV_FLOW_DV_SUPPORT
+#ifdef HAVE_MLX5DV_DR
+ (void)context;
+ return mlx5dv_dr_create_matcher(tbl, matcher_attr->priority,
+ matcher_attr->match_criteria_enable,
+ matcher_attr->match_mask);
+#else
+ (void)tbl;
return mlx5dv_create_flow_matcher(context, matcher_attr);
+#endif
#else
(void)context;
(void)matcher_attr;
+ (void)tbl;
return NULL;
#endif
}
-static struct ibv_flow *
-mlx5_glue_dv_create_flow(struct mlx5dv_flow_matcher *matcher,
- struct mlx5dv_flow_match_parameters *match_value,
+static void *
+mlx5_glue_dv_create_flow(void *matcher,
+ void *match_value,
size_t num_actions,
void *actions[])
{
#ifdef HAVE_IBV_FLOW_DV_SUPPORT
+#ifdef HAVE_MLX5DV_DR
+ return mlx5dv_dr_create_rule(matcher, match_value, num_actions,
+ (struct mlx5dv_dr_action **)actions);
+#else
struct mlx5dv_flow_action_attr actions_attr[8];
if (num_actions > 8)
@@ -452,6 +517,7 @@
*((struct mlx5dv_flow_action_attr *)(actions[i]));
return mlx5dv_create_flow(matcher, match_value,
num_actions, actions_attr);
+#endif
#else
(void)matcher;
(void)match_value;
@@ -461,21 +527,13 @@
#endif
}
-static int
-mlx5_glue_dv_destroy_flow_matcher(struct mlx5dv_flow_matcher *matcher)
-{
-#ifdef HAVE_IBV_FLOW_DV_SUPPORT
- return mlx5dv_destroy_flow_matcher(matcher);
-#else
- (void)matcher;
- return 0;
-#endif
-}
-
static void *
mlx5_glue_dv_create_flow_action_counter(void *counter_obj, uint32_t offset)
{
#ifdef HAVE_IBV_FLOW_DV_SUPPORT
+#ifdef HAVE_MLX5DV_DR
+ return mlx5dv_dr_create_action_devx_counter(counter_obj, offset);
+#else
struct mlx5dv_flow_action_attr *action;
(void)offset;
@@ -485,6 +543,7 @@
action->type = MLX5DV_FLOW_ACTION_COUNTER_DEVX;
action->obj = counter_obj;
return action;
+#endif
#else
(void)counter_obj;
(void)offset;
@@ -496,6 +555,9 @@
mlx5_glue_dv_create_flow_action_dest_ibv_qp(void *qp)
{
#ifdef HAVE_IBV_FLOW_DV_SUPPORT
+#ifdef HAVE_MLX5DV_DR
+ return mlx5dv_dr_create_action_dest_ibv_qp(qp);
+#else
struct mlx5dv_flow_action_attr *action;
action = malloc(sizeof(*action));
@@ -504,6 +566,7 @@
action->type = MLX5DV_FLOW_ACTION_DEST_IBV_QP;
action->obj = qp;
return action;
+#endif
#else
(void)qp;
return NULL;
@@ -513,13 +576,22 @@
static void *
mlx5_glue_dv_create_flow_action_modify_header
(struct ibv_context *ctx,
+ enum mlx5dv_flow_table_type ft_type,
+ void *ns, uint64_t flags,
size_t actions_sz,
- uint64_t actions[],
- enum mlx5dv_flow_table_type ft_type)
+ uint64_t actions[])
{
#ifdef HAVE_IBV_FLOW_DV_SUPPORT
+#ifdef HAVE_MLX5DV_DR
+ (void)ctx;
+ (void)ft_type;
+ return mlx5dv_dr_create_action_modify_header(ns, flags, actions_sz,
+ actions);
+#else
struct mlx5dv_flow_action_attr *action;
+ (void)ns;
+ (void)flags;
action = malloc(sizeof(*action));
if (!action)
return NULL;
@@ -527,11 +599,14 @@
action->action = mlx5dv_create_flow_action_modify_header
(ctx, actions_sz, actions, ft_type);
return action;
+#endif
#else
(void)ctx;
+ (void)ft_type;
+ (void)ns;
+ (void)flags;
(void)actions_sz;
(void)actions;
- (void)ft_type;
return NULL;
#endif
}
@@ -539,12 +614,20 @@
static void *
mlx5_glue_dv_create_flow_action_packet_reformat
(struct ibv_context *ctx,
- size_t data_sz,
- void *data,
enum mlx5dv_flow_action_packet_reformat_type reformat_type,
- enum mlx5dv_flow_table_type ft_type)
+ enum mlx5dv_flow_table_type ft_type, struct mlx5dv_dr_ns *ns,
+ uint32_t flags, size_t data_sz, void *data)
{
#ifdef HAVE_IBV_FLOW_DV_SUPPORT
+#ifdef HAVE_MLX5DV_DR
+ (void)ctx;
+ (void)ft_type;
+ return mlx5dv_dr_create_action_packet_reformat(ns, flags,
+ reformat_type, data_sz,
+ data);
+#else
+ (void)ns;
+ (void)flags;
struct mlx5dv_flow_action_attr *action;
action = malloc(sizeof(*action));
@@ -554,12 +637,15 @@
action->action = mlx5dv_create_flow_action_packet_reformat
(ctx, data_sz, data, reformat_type, ft_type);
return action;
+#endif
#else
(void)ctx;
- (void)data_sz;
- (void)data;
(void)reformat_type;
(void)ft_type;
+ (void)ns;
+ (void)flags;
+ (void)data_sz;
+ (void)data;
return NULL;
#endif
}
@@ -568,6 +654,9 @@
mlx5_glue_dv_create_flow_action_tag(uint32_t tag)
{
#ifdef HAVE_IBV_FLOW_DV_SUPPORT
+#ifdef HAVE_MLX5DV_DR
+ return mlx5dv_dr_create_action_tag(tag);
+#else
struct mlx5dv_flow_action_attr *action;
action = malloc(sizeof(*action));
if (!action)
@@ -576,10 +665,36 @@
action->tag_value = tag;
return action;
#endif
+#endif
(void)tag;
return NULL;
}
+static int
+mlx5_glue_dv_destroy_flow(void *flow_id)
+{
+#ifdef HAVE_MLX5DV_DR
+ return mlx5dv_dr_destroy_rule(flow_id);
+#else
+ return ibv_destroy_flow(flow_id);
+#endif
+}
+
+static int
+mlx5_glue_dv_destroy_flow_matcher(void *matcher)
+{
+#ifdef HAVE_IBV_FLOW_DV_SUPPORT
+#ifdef HAVE_MLX5DV_DR
+ return mlx5dv_dr_destroy_matcher(matcher);
+#else
+ return mlx5dv_destroy_flow_matcher(matcher);
+#endif
+#else
+ (void)matcher;
+ return 0;
+#endif
+}
+
static struct ibv_context *
mlx5_glue_dv_open_device(struct ibv_device *device)
{
@@ -718,6 +833,10 @@
.get_async_event = mlx5_glue_get_async_event,
.port_state_str = mlx5_glue_port_state_str,
.cq_ex_to_cq = mlx5_glue_cq_ex_to_cq,
+ .dr_create_flow_tbl = mlx5_glue_dr_create_flow_tbl,
+ .dr_destroy_flow_tbl = mlx5_glue_dr_destroy_flow_tbl,
+ .dr_create_ns = mlx5_glue_dr_create_ns,
+ .dr_destroy_ns = mlx5_glue_dr_destroy_ns,
.dv_create_cq = mlx5_glue_dv_create_cq,
.dv_create_wq = mlx5_glue_dv_create_wq,
.dv_query_device = mlx5_glue_dv_query_device,
@@ -725,7 +844,6 @@
.dv_init_obj = mlx5_glue_dv_init_obj,
.dv_create_qp = mlx5_glue_dv_create_qp,
.dv_create_flow_matcher = mlx5_glue_dv_create_flow_matcher,
- .dv_destroy_flow_matcher = mlx5_glue_dv_destroy_flow_matcher,
.dv_create_flow = mlx5_glue_dv_create_flow,
.dv_create_flow_action_counter =
mlx5_glue_dv_create_flow_action_counter,
@@ -736,6 +854,8 @@
.dv_create_flow_action_packet_reformat =
mlx5_glue_dv_create_flow_action_packet_reformat,
.dv_create_flow_action_tag = mlx5_glue_dv_create_flow_action_tag,
+ .dv_destroy_flow = mlx5_glue_dv_destroy_flow,
+ .dv_destroy_flow_matcher = mlx5_glue_dv_destroy_flow_matcher,
.dv_open_device = mlx5_glue_dv_open_device,
.devx_obj_create = mlx5_glue_devx_obj_create,
.devx_obj_destroy = mlx5_glue_devx_obj_destroy,
diff --git a/drivers/net/mlx5/mlx5_glue.h b/drivers/net/mlx5/mlx5_glue.h
index 167f1f7..7115575 100644
--- a/drivers/net/mlx5/mlx5_glue.h
+++ b/drivers/net/mlx5/mlx5_glue.h
@@ -63,6 +63,11 @@
struct mlx5dv_devx_obj;
#endif
+#ifndef HAVE_MLX5DV_DR
+struct mlx5dv_dr_ns;
+enum mlx5dv_dr_ns_domain { unused, };
+#endif
+
/* LIB_GLUE_VERSION must be updated every time this structure is modified. */
struct mlx5_glue {
const char *version;
@@ -140,6 +145,11 @@ struct mlx5_glue {
struct ibv_async_event *event);
const char *(*port_state_str)(enum ibv_port_state port_state);
struct ibv_cq *(*cq_ex_to_cq)(struct ibv_cq_ex *cq);
+ void *(*dr_create_flow_tbl)(void *ns, uint32_t level);
+ int (*dr_destroy_flow_tbl)(void *tbl);
+ void *(*dr_create_ns)(struct ibv_context *ctx,
+ enum mlx5dv_dr_ns_domain domain);
+ int (*dr_destroy_ns)(void *ns);
struct ibv_cq_ex *(*dv_create_cq)
(struct ibv_context *context,
struct ibv_cq_init_attr_ex *cq_attr,
@@ -158,23 +168,26 @@ struct mlx5_glue {
(struct ibv_context *context,
struct ibv_qp_init_attr_ex *qp_init_attr_ex,
struct mlx5dv_qp_init_attr *dv_qp_init_attr);
- struct mlx5dv_flow_matcher *(*dv_create_flow_matcher)
+ void *(*dv_create_flow_matcher)
(struct ibv_context *context,
- struct mlx5dv_flow_matcher_attr *matcher_attr);
- int (*dv_destroy_flow_matcher)(struct mlx5dv_flow_matcher *matcher);
- struct ibv_flow *(*dv_create_flow)(struct mlx5dv_flow_matcher *matcher,
- struct mlx5dv_flow_match_parameters *match_value,
+ struct mlx5dv_flow_matcher_attr *matcher_attr,
+ void *tbl);
+ void *(*dv_create_flow)(void *matcher, void *match_value,
size_t num_actions, void *actions[]);
void *(*dv_create_flow_action_counter)(void *obj, uint32_t offset);
void *(*dv_create_flow_action_dest_ibv_qp)(void *qp);
void *(*dv_create_flow_action_modify_header)
- (struct ibv_context *ctx, size_t actions_sz, uint64_t actions[],
- enum mlx5dv_flow_table_type ft_type);
+ (struct ibv_context *ctx, enum mlx5dv_flow_table_type ft_type,
+ void *ns, uint64_t flags, size_t actions_sz,
+ uint64_t actions[]);
void *(*dv_create_flow_action_packet_reformat)
- (struct ibv_context *ctx, size_t data_sz, void *data,
+ (struct ibv_context *ctx,
enum mlx5dv_flow_action_packet_reformat_type reformat_type,
- enum mlx5dv_flow_table_type ft_type);
+ enum mlx5dv_flow_table_type ft_type, struct mlx5dv_dr_ns *ns,
+ uint32_t flags, size_t data_sz, void *data);
void *(*dv_create_flow_action_tag)(uint32_t tag);
+ int (*dv_destroy_flow)(void *flow);
+ int (*dv_destroy_flow_matcher)(void *matcher);
struct ibv_context *(*dv_open_device)(struct ibv_device *device);
struct mlx5dv_devx_obj *(*devx_obj_create)
(struct ibv_context *ctx,
diff --git a/drivers/net/mlx5/mlx5_prm.h b/drivers/net/mlx5/mlx5_prm.h
index da1219e..b15266f 100644
--- a/drivers/net/mlx5/mlx5_prm.h
+++ b/drivers/net/mlx5/mlx5_prm.h
@@ -492,20 +492,40 @@ struct mlx5_ifc_fte_match_set_misc2_bits {
u8 reserved_at_1a0[0x60];
};
+struct mlx5_ifc_fte_match_set_misc3_bits {
+ u8 inner_tcp_seq_num[0x20];
+ u8 outer_tcp_seq_num[0x20];
+ u8 inner_tcp_ack_num[0x20];
+ u8 outer_tcp_ack_num[0x20];
+ u8 reserved_at_auto1[0x8];
+ u8 outer_vxlan_gpe_vni[0x18];
+ u8 outer_vxlan_gpe_next_protocol[0x8];
+ u8 outer_vxlan_gpe_flags[0x8];
+ u8 reserved_at_a8[0x10];
+ u8 icmp_header_data[0x20];
+ u8 icmpv6_header_data[0x20];
+ u8 icmp_type[0x8];
+ u8 icmp_code[0x8];
+ u8 icmpv6_type[0x8];
+ u8 icmpv6_code[0x8];
+ u8 reserved_at_1a0[0xe0];
+};
+
/* Flow matcher. */
struct mlx5_ifc_fte_match_param_bits {
struct mlx5_ifc_fte_match_set_lyr_2_4_bits outer_headers;
struct mlx5_ifc_fte_match_set_misc_bits misc_parameters;
struct mlx5_ifc_fte_match_set_lyr_2_4_bits inner_headers;
struct mlx5_ifc_fte_match_set_misc2_bits misc_parameters_2;
- u8 reserved_at_800[0x800];
+ struct mlx5_ifc_fte_match_set_misc3_bits misc_parameters_3;
};
enum {
MLX5_MATCH_CRITERIA_ENABLE_OUTER_BIT,
MLX5_MATCH_CRITERIA_ENABLE_MISC_BIT,
MLX5_MATCH_CRITERIA_ENABLE_INNER_BIT,
- MLX5_MATCH_CRITERIA_ENABLE_MISC2_BIT
+ MLX5_MATCH_CRITERIA_ENABLE_MISC2_BIT,
+ MLX5_MATCH_CRITERIA_ENABLE_MISC3_BIT
};
enum {
--
1.8.3.1
^ permalink raw reply [flat|nested] 52+ messages in thread
* [dpdk-dev] [PATCH 3/3] net/mlx5: add jump action support for NIC
2019-03-20 15:38 [dpdk-dev] [PATCH 0/3]net/mlx5: Add Direct Rule support Ori Kam
` (2 preceding siblings ...)
2019-03-20 15:39 ` [dpdk-dev] [PATCH 2/3] net/mlx5: add Direct Rules API Ori Kam
@ 2019-03-20 15:39 ` Ori Kam
2019-03-20 15:39 ` Ori Kam
2019-03-28 16:32 ` [dpdk-dev] [PATCH v2 0/3] net/mlx5: Add Direct Rule support Ori Kam
4 siblings, 1 reply; 52+ messages in thread
From: Ori Kam @ 2019-03-20 15:39 UTC (permalink / raw)
To: Shahaf Shuler, Yongseok Koh, Matan Azrad; +Cc: dev, Ori Kam, Slava Ovsiienko
When using Direct Rules we can add actions to jump between tables.
This is extra useful since rule insertion rate is much higher on other
tables compared to table zero.
if no group is selected the rule is added to group 0.
Signed-off-by: Ori Kam <orika@mellanox.com>
---
drivers/net/mlx5/mlx5.h | 6 +
drivers/net/mlx5/mlx5_flow.h | 15 ++-
drivers/net/mlx5/mlx5_flow_dv.c | 271 +++++++++++++++++++++++++++++++++++-----
drivers/net/mlx5/mlx5_glue.c | 13 ++
drivers/net/mlx5/mlx5_glue.h | 1 +
5 files changed, 273 insertions(+), 33 deletions(-)
diff --git a/drivers/net/mlx5/mlx5.h b/drivers/net/mlx5/mlx5.h
index a0d4961..a88cb4a 100644
--- a/drivers/net/mlx5/mlx5.h
+++ b/drivers/net/mlx5/mlx5.h
@@ -249,6 +249,12 @@ struct mlx5_priv {
LIST_HEAD(encap_decap, mlx5_flow_dv_encap_decap_resource) encaps_decaps;
LIST_HEAD(modify_cmd, mlx5_flow_dv_modify_hdr_resource) modify_cmds;
LIST_HEAD(tag, mlx5_flow_dv_tag_resource) tags;
+ LIST_HEAD(jump, mlx5_flow_dv_jump_tbl_resource) jump_tbl;
+ /* Pointer to next element. */
+ rte_atomic32_t refcnt; /**< Reference counter. */
+ struct ibv_flow_action *verbs_action;
+ /**< Verbs modify header action object. */
+ uint8_t ft_type; /**< Flow table type, Rx or Tx. */
/* Tags resources cache. */
uint32_t link_speed_capa; /* Link speed capabilities. */
struct mlx5_xstats_ctrl xstats_ctrl; /* Extended stats control. */
diff --git a/drivers/net/mlx5/mlx5_flow.h b/drivers/net/mlx5/mlx5_flow.h
index 8ba37a0..7f421fb 100644
--- a/drivers/net/mlx5/mlx5_flow.h
+++ b/drivers/net/mlx5/mlx5_flow.h
@@ -115,7 +115,8 @@
#define MLX5_FLOW_ACTION_RAW_DECAP (1u << 27)
#define MLX5_FLOW_FATE_ACTIONS \
- (MLX5_FLOW_ACTION_DROP | MLX5_FLOW_ACTION_QUEUE | MLX5_FLOW_ACTION_RSS)
+ (MLX5_FLOW_ACTION_DROP | MLX5_FLOW_ACTION_QUEUE | \
+ MLX5_FLOW_ACTION_RSS | MLX5_FLOW_ACTION_JUMP)
#define MLX5_FLOW_ENCAP_ACTIONS (MLX5_FLOW_ACTION_VXLAN_ENCAP | \
MLX5_FLOW_ACTION_NVGRE_ENCAP | \
@@ -250,6 +251,16 @@ struct mlx5_flow_dv_modify_hdr_resource {
/**< Modification actions. */
};
+/* Jump action resource structure. */
+struct mlx5_flow_dv_jump_tbl_resource {
+ LIST_ENTRY(mlx5_flow_dv_modify_hdr_resource) next;
+ /* Pointer to next element. */
+ rte_atomic32_t refcnt; /**< Reference counter. */
+ void *action; /**< Pointer to the rdma core action. */
+ uint8_t ft_type; /**< Flow table type, Rx or Tx. */
+ struct mlx5_flow_tbl_resource *tbl; /**< The target table. */
+};
+
/*
* Max number of actions per DV flow.
* See CREATE_FLOW_MAX_FLOW_ACTIONS_SUPPORTED
@@ -270,6 +281,8 @@ struct mlx5_flow_dv {
struct mlx5_flow_dv_modify_hdr_resource *modify_hdr;
/**< Pointer to modify header resource in cache. */
struct ibv_flow *flow; /**< Installed flow. */
+ struct mlx5_flow_dv_jump_tbl_resource *jump;
+ /**< Pointer to the jump action resource. */
#ifdef HAVE_IBV_FLOW_DV_SUPPORT
void *actions[MLX5_DV_MAX_NUMBER_OF_ACTIONS];
/**< Action list. */
diff --git a/drivers/net/mlx5/mlx5_flow_dv.c b/drivers/net/mlx5/mlx5_flow_dv.c
index 228ba3f..44b36ee 100644
--- a/drivers/net/mlx5/mlx5_flow_dv.c
+++ b/drivers/net/mlx5/mlx5_flow_dv.c
@@ -861,6 +861,68 @@ struct field_modify_info modify_tcp[] = {
}
/**
+ * Find existing table jump resource or create and register a new one.
+ *
+ * @param dev[in, out]
+ * Pointer to rte_eth_dev structure.
+ * @param[in, out] resource
+ * Pointer to jump table resource.
+ * @parm[in, out] dev_flow
+ * Pointer to the dev_flow.
+ * @param[out] error
+ * pointer to error structure.
+ *
+ * @return
+ * 0 on success otherwise -errno and errno is set.
+ */
+static int
+flow_dv_jump_tbl_resource_register
+ (struct rte_eth_dev *dev,
+ struct mlx5_flow_dv_jump_tbl_resource *resource,
+ struct mlx5_flow *dev_flow,
+ struct rte_flow_error *error)
+{
+ struct mlx5_priv *priv = dev->data->dev_private;
+ struct mlx5_flow_dv_jump_tbl_resource *cache_resource;
+
+ /* Lookup a matching resource from cache. */
+ LIST_FOREACH(cache_resource, &priv->jump_tbl, next) {
+ if (resource->tbl == cache_resource->tbl) {
+ DRV_LOG(DEBUG, "jump table resource resource %p: refcnt %d++",
+ (void *)cache_resource,
+ rte_atomic32_read(&cache_resource->refcnt));
+ rte_atomic32_inc(&cache_resource->refcnt);
+ dev_flow->dv.jump = cache_resource;
+ return 0;
+ }
+ }
+ /* Register new jump table resource. */
+ cache_resource = rte_calloc(__func__, 1, sizeof(*cache_resource), 0);
+ if (!cache_resource)
+ return rte_flow_error_set(error, ENOMEM,
+ RTE_FLOW_ERROR_TYPE_UNSPECIFIED, NULL,
+ "cannot allocate resource memory");
+ *cache_resource = *resource;
+ cache_resource->action =
+ mlx5_glue->dr_create_flow_action_dest_flow_tbl
+ (resource->tbl->obj);
+ if (!cache_resource->action) {
+ rte_free(cache_resource);
+ return rte_flow_error_set(error, ENOMEM,
+ RTE_FLOW_ERROR_TYPE_UNSPECIFIED,
+ NULL, "cannot create action");
+ }
+ rte_atomic32_init(&cache_resource->refcnt);
+ rte_atomic32_inc(&cache_resource->refcnt);
+ LIST_INSERT_HEAD(&priv->jump_tbl, cache_resource, next);
+ dev_flow->dv.jump = cache_resource;
+ DRV_LOG(DEBUG, "new jump table resource %p: refcnt %d++",
+ (void *)cache_resource,
+ rte_atomic32_read(&cache_resource->refcnt));
+ return 0;
+}
+
+/**
* Get the size of specific rte_flow_item_type
*
* @param[in] item_type
@@ -1423,6 +1485,37 @@ struct field_modify_info modify_tcp[] = {
}
/**
+ * Validate jump action.
+ *
+ * @param[in] action
+ * Pointer to the modify action.
+ * @param[in] group
+ * The group of the current flow.
+ * @param[out] error
+ * Pointer to error structure.
+ *
+ * @return
+ * 0 on success, a negative errno value otherwise and rte_errno is set.
+ */
+static int
+flow_dv_validate_action_jump(const struct rte_flow_action *action,
+ uint32_t group,
+ struct rte_flow_error *error)
+{
+ if (action->type != RTE_FLOW_ACTION_TYPE_JUMP && !action->conf)
+ return rte_flow_error_set(error, EINVAL,
+ RTE_FLOW_ERROR_TYPE_ACTION_CONF,
+ NULL, "action configuration not set");
+ if (group >= ((const struct rte_flow_action_jump *)action->conf)->group)
+ return rte_flow_error_set(error, EINVAL,
+ RTE_FLOW_ERROR_TYPE_ACTION, NULL,
+ "target group must be higher then"
+ " the current flow group");
+ return 0;
+}
+
+
+/**
* Find existing modify-header resource or create and register a new one.
*
* @param dev[in, out]
@@ -1605,7 +1698,7 @@ struct field_modify_info modify_tcp[] = {
struct mlx5_priv *priv = dev->data->dev_private;
uint32_t priority_max = priv->config.flow_prio - 1;
-#ifdef HAVE_MLX5DV_DR
+#ifndef HAVE_MLX5DV_DR
if (attributes->group)
return rte_flow_error_set(error, ENOTSUP,
RTE_FLOW_ERROR_TYPE_ATTR_GROUP,
@@ -1977,6 +2070,14 @@ struct field_modify_info modify_tcp[] = {
MLX5_FLOW_ACTION_SET_TTL :
MLX5_FLOW_ACTION_DEC_TTL;
break;
+ case RTE_FLOW_ACTION_TYPE_JUMP:
+ ret = flow_dv_validate_action_jump(actions,
+ attr->group, error);
+ if (ret)
+ return ret;
+ ++actions_n;
+ action_flags |= MLX5_FLOW_ACTION_JUMP;
+ break;
default:
return rte_flow_error_set(error, ENOTSUP,
RTE_FLOW_ERROR_TYPE_ACTION,
@@ -2746,6 +2847,71 @@ struct field_modify_info modify_tcp[] = {
return match_criteria_enable;
}
+
+/**
+ * Get a flow table.
+ *
+ * @param dev[in, out]
+ * Pointer to rte_eth_dev structure.
+ * @param[in] table_id
+ * Table id to use.
+ * @param[in] egress
+ * Direction of the table.
+ * @param[out] error
+ * pointer to error structure.
+ *
+ * @return
+ * Returns tables resource based on the index, NULL in case of failed.
+ */
+static struct mlx5_flow_tbl_resource *
+flow_dv_tbl_resource_get(struct rte_eth_dev *dev,
+ uint32_t table_id, uint8_t egress,
+ struct rte_flow_error *error)
+{
+ struct mlx5_priv *priv = dev->data->dev_private;
+ struct mlx5_flow_tbl_resource *tbl;
+
+ if (egress) {
+ tbl = &priv->tx_tbl[table_id];
+ if (!tbl->obj)
+ tbl->obj = mlx5_glue->dr_create_flow_tbl
+ (priv->tx_ns, table_id);
+ } else {
+ tbl = &priv->rx_tbl[table_id];
+ if (!tbl->obj)
+ tbl->obj = mlx5_glue->dr_create_flow_tbl
+ (priv->rx_ns, table_id);
+ }
+ if (!tbl->obj) {
+ rte_flow_error_set(error, ENOMEM,
+ RTE_FLOW_ERROR_TYPE_UNSPECIFIED,
+ NULL, "cannot create table");
+ return NULL;
+ }
+ rte_atomic32_inc(&tbl->refcnt);
+ return tbl;
+}
+
+/**
+ * Release a flow table.
+ *
+ * @param[in] tbl
+ * Table resource to be released.
+ *
+ * @return
+ * Returns 0 if table was released, else return 1;
+ */
+static int
+flow_dv_tbl_resource_release(struct mlx5_flow_tbl_resource *tbl)
+{
+ if (rte_atomic32_dec_and_test(&tbl->refcnt)) {
+ mlx5_glue->dr_destroy_flow_tbl(tbl->obj);
+ tbl->obj = NULL;
+ return 0;
+ }
+ return 1;
+}
+
/**
* Register the flow matcher.
*
@@ -2795,33 +2961,20 @@ struct field_modify_info modify_tcp[] = {
return 0;
}
}
-#ifdef HAVE_MLX5DV_DR
- if (matcher->egress) {
- tbl = &priv->tx_tbl[matcher->group];
- if (!tbl->obj)
- tbl->obj = mlx5_glue->dr_create_flow_tbl
- (priv->tx_ns,
- matcher->group * MLX5_GROUP_FACTOR);
- } else {
- tbl = &priv->rx_tbl[matcher->group];
- if (!tbl->obj)
- tbl->obj = mlx5_glue->dr_create_flow_tbl
- (priv->rx_ns,
- matcher->group * MLX5_GROUP_FACTOR);
- }
- if (!tbl->obj)
- return rte_flow_error_set(error, ENOMEM,
- RTE_FLOW_ERROR_TYPE_UNSPECIFIED,
- NULL, "cannot create table");
-
- rte_atomic32_inc(&tbl->refcnt);
-#endif
/* Register new matcher. */
cache_matcher = rte_calloc(__func__, 1, sizeof(*cache_matcher), 0);
if (!cache_matcher)
return rte_flow_error_set(error, ENOMEM,
RTE_FLOW_ERROR_TYPE_UNSPECIFIED, NULL,
"cannot allocate matcher memory");
+ tbl = flow_dv_tbl_resource_get(dev, matcher->group * MLX5_GROUP_FACTOR,
+ matcher->egress, error);
+ if (!tbl) {
+ rte_free(cache_matcher);
+ return rte_flow_error_set(error, ENOMEM,
+ RTE_FLOW_ERROR_TYPE_UNSPECIFIED,
+ NULL, "cannot create table");
+ }
*cache_matcher = *matcher;
dv_attr.match_criteria_enable =
flow_dv_matcher_enable(cache_matcher->mask.buf);
@@ -2833,12 +2986,7 @@ struct field_modify_info modify_tcp[] = {
tbl->obj);
if (!cache_matcher->matcher_object) {
rte_free(cache_matcher);
-#ifdef HAVE_MLX5DV_DR
- if (rte_atomic32_dec_and_test(&tbl->refcnt)) {
- mlx5_glue->dr_destroy_flow_tbl(tbl->obj);
- tbl->obj = NULL;
- }
-#endif
+ flow_dv_tbl_resource_release(tbl);
return rte_flow_error_set(error, ENOMEM,
RTE_FLOW_ERROR_TYPE_UNSPECIFIED,
NULL, "cannot create matcher");
@@ -3000,6 +3148,9 @@ struct field_modify_info modify_tcp[] = {
const struct rte_flow_action *action = actions;
const struct rte_flow_action_count *count = action->conf;
const uint8_t *rss_key;
+ const struct rte_flow_action_jump *jump_data;
+ struct mlx5_flow_dv_jump_tbl_resource jump_tbl_resource;
+ struct mlx5_flow_tbl_resource *tbl;
switch (actions->type) {
case RTE_FLOW_ACTION_TYPE_VOID:
@@ -3138,6 +3289,31 @@ struct field_modify_info modify_tcp[] = {
/* If decap is followed by encap, handle it at encap. */
action_flags |= MLX5_FLOW_ACTION_RAW_DECAP;
break;
+ case RTE_FLOW_ACTION_TYPE_JUMP:
+ jump_data = action->conf;
+ tbl = flow_dv_tbl_resource_get(dev, jump_data->group *
+ MLX5_GROUP_FACTOR,
+ attr->egress, error);
+ if (!tbl)
+ return rte_flow_error_set
+ (error, errno,
+ RTE_FLOW_ERROR_TYPE_ACTION,
+ NULL,
+ "cannot create jump action.");
+ jump_tbl_resource.tbl = tbl;
+ if (flow_dv_jump_tbl_resource_register
+ (dev, &jump_tbl_resource, dev_flow, error)) {
+ flow_dv_tbl_resource_release(tbl);
+ return rte_flow_error_set
+ (error, errno,
+ RTE_FLOW_ERROR_TYPE_ACTION,
+ NULL,
+ "cannot create jump action.");
+ }
+ dev_flow->dv.actions[actions_n++] =
+ dev_flow->dv.jump->action;
+ action_flags |= MLX5_FLOW_ACTION_JUMP;
+ break;
case RTE_FLOW_ACTION_TYPE_SET_MAC_SRC:
case RTE_FLOW_ACTION_TYPE_SET_MAC_DST:
if (flow_dv_convert_action_modify_mac(&res, actions,
@@ -3457,10 +3633,7 @@ struct field_modify_info modify_tcp[] = {
tbl = &priv->tx_tbl[matcher->group];
else
tbl = &priv->rx_tbl[matcher->group];
- if (rte_atomic32_dec_and_test(&tbl->refcnt)) {
- mlx5_glue->dr_destroy_flow_tbl(tbl->obj);
- tbl->obj = NULL;
- }
+ flow_dv_tbl_resource_release(tbl);
rte_free(matcher);
DRV_LOG(DEBUG, "port %u matcher %p: removed",
dev->data->port_id, (void *)matcher);
@@ -3501,6 +3674,38 @@ struct field_modify_info modify_tcp[] = {
}
/**
+ * Release an jump to table action resource.
+ *
+ * @param flow
+ * Pointer to mlx5_flow.
+ *
+ * @return
+ * 1 while a reference on it exists, 0 when freed.
+ */
+static int
+flow_dv_jump_tbl_resource_release(struct mlx5_flow *flow)
+{
+ struct mlx5_flow_dv_jump_tbl_resource *cache_resource =
+ flow->dv.jump;
+
+ assert(cache_resource->action);
+ DRV_LOG(DEBUG, "jump table resource %p: refcnt %d--",
+ (void *)cache_resource,
+ rte_atomic32_read(&cache_resource->refcnt));
+ if (rte_atomic32_dec_and_test(&cache_resource->refcnt)) {
+ claim_zero(mlx5_glue->destroy_flow_action
+ (cache_resource->action));
+ LIST_REMOVE(cache_resource, next);
+ flow_dv_tbl_resource_release(cache_resource->tbl);
+ rte_free(cache_resource);
+ DRV_LOG(DEBUG, "jump table resource %p: removed",
+ (void *)cache_resource);
+ return 0;
+ }
+ return 1;
+}
+
+/**
* Release a modify-header resource.
*
* @param flow
@@ -3596,6 +3801,8 @@ struct field_modify_info modify_tcp[] = {
flow_dv_encap_decap_resource_release(dev_flow);
if (dev_flow->dv.modify_hdr)
flow_dv_modify_hdr_resource_release(dev_flow);
+ if (dev_flow->dv.jump)
+ flow_dv_jump_tbl_resource_release(dev_flow);
rte_free(dev_flow);
}
}
diff --git a/drivers/net/mlx5/mlx5_glue.c b/drivers/net/mlx5/mlx5_glue.c
index f509f85..fca34ca 100644
--- a/drivers/net/mlx5/mlx5_glue.c
+++ b/drivers/net/mlx5/mlx5_glue.c
@@ -370,6 +370,17 @@
}
static void *
+mlx5_glue_dr_create_flow_action_dest_flow_tbl(void *tbl)
+{
+#ifdef HAVE_MLX5DV_DR
+ return mlx5dv_dr_create_action_dest_flow_table(tbl);
+#else
+ (void)tbl;
+ return NULL;
+#endif
+}
+
+static void *
mlx5_glue_dr_create_flow_tbl(void *ns, uint32_t level)
{
#ifdef HAVE_MLX5DV_DR
@@ -833,6 +844,8 @@
.get_async_event = mlx5_glue_get_async_event,
.port_state_str = mlx5_glue_port_state_str,
.cq_ex_to_cq = mlx5_glue_cq_ex_to_cq,
+ .dr_create_flow_action_dest_flow_tbl =
+ mlx5_glue_dr_create_flow_action_dest_flow_tbl,
.dr_create_flow_tbl = mlx5_glue_dr_create_flow_tbl,
.dr_destroy_flow_tbl = mlx5_glue_dr_destroy_flow_tbl,
.dr_create_ns = mlx5_glue_dr_create_ns,
diff --git a/drivers/net/mlx5/mlx5_glue.h b/drivers/net/mlx5/mlx5_glue.h
index 7115575..16d5dd6 100644
--- a/drivers/net/mlx5/mlx5_glue.h
+++ b/drivers/net/mlx5/mlx5_glue.h
@@ -145,6 +145,7 @@ struct mlx5_glue {
struct ibv_async_event *event);
const char *(*port_state_str)(enum ibv_port_state port_state);
struct ibv_cq *(*cq_ex_to_cq)(struct ibv_cq_ex *cq);
+ void *(*dr_create_flow_action_dest_flow_tbl)(void *tbl);
void *(*dr_create_flow_tbl)(void *ns, uint32_t level);
int (*dr_destroy_flow_tbl)(void *tbl);
void *(*dr_create_ns)(struct ibv_context *ctx,
--
1.8.3.1
^ permalink raw reply [flat|nested] 52+ messages in thread
* [dpdk-dev] [PATCH 3/3] net/mlx5: add jump action support for NIC
2019-03-20 15:39 ` [dpdk-dev] [PATCH 3/3] net/mlx5: add jump action support for NIC Ori Kam
@ 2019-03-20 15:39 ` Ori Kam
0 siblings, 0 replies; 52+ messages in thread
From: Ori Kam @ 2019-03-20 15:39 UTC (permalink / raw)
To: Shahaf Shuler, Yongseok Koh, Matan Azrad; +Cc: dev, Ori Kam, Slava Ovsiienko
When using Direct Rules we can add actions to jump between tables.
This is extra useful since rule insertion rate is much higher on other
tables compared to table zero.
if no group is selected the rule is added to group 0.
Signed-off-by: Ori Kam <orika@mellanox.com>
---
drivers/net/mlx5/mlx5.h | 6 +
drivers/net/mlx5/mlx5_flow.h | 15 ++-
drivers/net/mlx5/mlx5_flow_dv.c | 271 +++++++++++++++++++++++++++++++++++-----
drivers/net/mlx5/mlx5_glue.c | 13 ++
drivers/net/mlx5/mlx5_glue.h | 1 +
5 files changed, 273 insertions(+), 33 deletions(-)
diff --git a/drivers/net/mlx5/mlx5.h b/drivers/net/mlx5/mlx5.h
index a0d4961..a88cb4a 100644
--- a/drivers/net/mlx5/mlx5.h
+++ b/drivers/net/mlx5/mlx5.h
@@ -249,6 +249,12 @@ struct mlx5_priv {
LIST_HEAD(encap_decap, mlx5_flow_dv_encap_decap_resource) encaps_decaps;
LIST_HEAD(modify_cmd, mlx5_flow_dv_modify_hdr_resource) modify_cmds;
LIST_HEAD(tag, mlx5_flow_dv_tag_resource) tags;
+ LIST_HEAD(jump, mlx5_flow_dv_jump_tbl_resource) jump_tbl;
+ /* Pointer to next element. */
+ rte_atomic32_t refcnt; /**< Reference counter. */
+ struct ibv_flow_action *verbs_action;
+ /**< Verbs modify header action object. */
+ uint8_t ft_type; /**< Flow table type, Rx or Tx. */
/* Tags resources cache. */
uint32_t link_speed_capa; /* Link speed capabilities. */
struct mlx5_xstats_ctrl xstats_ctrl; /* Extended stats control. */
diff --git a/drivers/net/mlx5/mlx5_flow.h b/drivers/net/mlx5/mlx5_flow.h
index 8ba37a0..7f421fb 100644
--- a/drivers/net/mlx5/mlx5_flow.h
+++ b/drivers/net/mlx5/mlx5_flow.h
@@ -115,7 +115,8 @@
#define MLX5_FLOW_ACTION_RAW_DECAP (1u << 27)
#define MLX5_FLOW_FATE_ACTIONS \
- (MLX5_FLOW_ACTION_DROP | MLX5_FLOW_ACTION_QUEUE | MLX5_FLOW_ACTION_RSS)
+ (MLX5_FLOW_ACTION_DROP | MLX5_FLOW_ACTION_QUEUE | \
+ MLX5_FLOW_ACTION_RSS | MLX5_FLOW_ACTION_JUMP)
#define MLX5_FLOW_ENCAP_ACTIONS (MLX5_FLOW_ACTION_VXLAN_ENCAP | \
MLX5_FLOW_ACTION_NVGRE_ENCAP | \
@@ -250,6 +251,16 @@ struct mlx5_flow_dv_modify_hdr_resource {
/**< Modification actions. */
};
+/* Jump action resource structure. */
+struct mlx5_flow_dv_jump_tbl_resource {
+ LIST_ENTRY(mlx5_flow_dv_modify_hdr_resource) next;
+ /* Pointer to next element. */
+ rte_atomic32_t refcnt; /**< Reference counter. */
+ void *action; /**< Pointer to the rdma core action. */
+ uint8_t ft_type; /**< Flow table type, Rx or Tx. */
+ struct mlx5_flow_tbl_resource *tbl; /**< The target table. */
+};
+
/*
* Max number of actions per DV flow.
* See CREATE_FLOW_MAX_FLOW_ACTIONS_SUPPORTED
@@ -270,6 +281,8 @@ struct mlx5_flow_dv {
struct mlx5_flow_dv_modify_hdr_resource *modify_hdr;
/**< Pointer to modify header resource in cache. */
struct ibv_flow *flow; /**< Installed flow. */
+ struct mlx5_flow_dv_jump_tbl_resource *jump;
+ /**< Pointer to the jump action resource. */
#ifdef HAVE_IBV_FLOW_DV_SUPPORT
void *actions[MLX5_DV_MAX_NUMBER_OF_ACTIONS];
/**< Action list. */
diff --git a/drivers/net/mlx5/mlx5_flow_dv.c b/drivers/net/mlx5/mlx5_flow_dv.c
index 228ba3f..44b36ee 100644
--- a/drivers/net/mlx5/mlx5_flow_dv.c
+++ b/drivers/net/mlx5/mlx5_flow_dv.c
@@ -861,6 +861,68 @@ struct field_modify_info modify_tcp[] = {
}
/**
+ * Find existing table jump resource or create and register a new one.
+ *
+ * @param dev[in, out]
+ * Pointer to rte_eth_dev structure.
+ * @param[in, out] resource
+ * Pointer to jump table resource.
+ * @parm[in, out] dev_flow
+ * Pointer to the dev_flow.
+ * @param[out] error
+ * pointer to error structure.
+ *
+ * @return
+ * 0 on success otherwise -errno and errno is set.
+ */
+static int
+flow_dv_jump_tbl_resource_register
+ (struct rte_eth_dev *dev,
+ struct mlx5_flow_dv_jump_tbl_resource *resource,
+ struct mlx5_flow *dev_flow,
+ struct rte_flow_error *error)
+{
+ struct mlx5_priv *priv = dev->data->dev_private;
+ struct mlx5_flow_dv_jump_tbl_resource *cache_resource;
+
+ /* Lookup a matching resource from cache. */
+ LIST_FOREACH(cache_resource, &priv->jump_tbl, next) {
+ if (resource->tbl == cache_resource->tbl) {
+ DRV_LOG(DEBUG, "jump table resource resource %p: refcnt %d++",
+ (void *)cache_resource,
+ rte_atomic32_read(&cache_resource->refcnt));
+ rte_atomic32_inc(&cache_resource->refcnt);
+ dev_flow->dv.jump = cache_resource;
+ return 0;
+ }
+ }
+ /* Register new jump table resource. */
+ cache_resource = rte_calloc(__func__, 1, sizeof(*cache_resource), 0);
+ if (!cache_resource)
+ return rte_flow_error_set(error, ENOMEM,
+ RTE_FLOW_ERROR_TYPE_UNSPECIFIED, NULL,
+ "cannot allocate resource memory");
+ *cache_resource = *resource;
+ cache_resource->action =
+ mlx5_glue->dr_create_flow_action_dest_flow_tbl
+ (resource->tbl->obj);
+ if (!cache_resource->action) {
+ rte_free(cache_resource);
+ return rte_flow_error_set(error, ENOMEM,
+ RTE_FLOW_ERROR_TYPE_UNSPECIFIED,
+ NULL, "cannot create action");
+ }
+ rte_atomic32_init(&cache_resource->refcnt);
+ rte_atomic32_inc(&cache_resource->refcnt);
+ LIST_INSERT_HEAD(&priv->jump_tbl, cache_resource, next);
+ dev_flow->dv.jump = cache_resource;
+ DRV_LOG(DEBUG, "new jump table resource %p: refcnt %d++",
+ (void *)cache_resource,
+ rte_atomic32_read(&cache_resource->refcnt));
+ return 0;
+}
+
+/**
* Get the size of specific rte_flow_item_type
*
* @param[in] item_type
@@ -1423,6 +1485,37 @@ struct field_modify_info modify_tcp[] = {
}
/**
+ * Validate jump action.
+ *
+ * @param[in] action
+ * Pointer to the modify action.
+ * @param[in] group
+ * The group of the current flow.
+ * @param[out] error
+ * Pointer to error structure.
+ *
+ * @return
+ * 0 on success, a negative errno value otherwise and rte_errno is set.
+ */
+static int
+flow_dv_validate_action_jump(const struct rte_flow_action *action,
+ uint32_t group,
+ struct rte_flow_error *error)
+{
+ if (action->type != RTE_FLOW_ACTION_TYPE_JUMP && !action->conf)
+ return rte_flow_error_set(error, EINVAL,
+ RTE_FLOW_ERROR_TYPE_ACTION_CONF,
+ NULL, "action configuration not set");
+ if (group >= ((const struct rte_flow_action_jump *)action->conf)->group)
+ return rte_flow_error_set(error, EINVAL,
+ RTE_FLOW_ERROR_TYPE_ACTION, NULL,
+ "target group must be higher then"
+ " the current flow group");
+ return 0;
+}
+
+
+/**
* Find existing modify-header resource or create and register a new one.
*
* @param dev[in, out]
@@ -1605,7 +1698,7 @@ struct field_modify_info modify_tcp[] = {
struct mlx5_priv *priv = dev->data->dev_private;
uint32_t priority_max = priv->config.flow_prio - 1;
-#ifdef HAVE_MLX5DV_DR
+#ifndef HAVE_MLX5DV_DR
if (attributes->group)
return rte_flow_error_set(error, ENOTSUP,
RTE_FLOW_ERROR_TYPE_ATTR_GROUP,
@@ -1977,6 +2070,14 @@ struct field_modify_info modify_tcp[] = {
MLX5_FLOW_ACTION_SET_TTL :
MLX5_FLOW_ACTION_DEC_TTL;
break;
+ case RTE_FLOW_ACTION_TYPE_JUMP:
+ ret = flow_dv_validate_action_jump(actions,
+ attr->group, error);
+ if (ret)
+ return ret;
+ ++actions_n;
+ action_flags |= MLX5_FLOW_ACTION_JUMP;
+ break;
default:
return rte_flow_error_set(error, ENOTSUP,
RTE_FLOW_ERROR_TYPE_ACTION,
@@ -2746,6 +2847,71 @@ struct field_modify_info modify_tcp[] = {
return match_criteria_enable;
}
+
+/**
+ * Get a flow table.
+ *
+ * @param dev[in, out]
+ * Pointer to rte_eth_dev structure.
+ * @param[in] table_id
+ * Table id to use.
+ * @param[in] egress
+ * Direction of the table.
+ * @param[out] error
+ * pointer to error structure.
+ *
+ * @return
+ * Returns tables resource based on the index, NULL in case of failed.
+ */
+static struct mlx5_flow_tbl_resource *
+flow_dv_tbl_resource_get(struct rte_eth_dev *dev,
+ uint32_t table_id, uint8_t egress,
+ struct rte_flow_error *error)
+{
+ struct mlx5_priv *priv = dev->data->dev_private;
+ struct mlx5_flow_tbl_resource *tbl;
+
+ if (egress) {
+ tbl = &priv->tx_tbl[table_id];
+ if (!tbl->obj)
+ tbl->obj = mlx5_glue->dr_create_flow_tbl
+ (priv->tx_ns, table_id);
+ } else {
+ tbl = &priv->rx_tbl[table_id];
+ if (!tbl->obj)
+ tbl->obj = mlx5_glue->dr_create_flow_tbl
+ (priv->rx_ns, table_id);
+ }
+ if (!tbl->obj) {
+ rte_flow_error_set(error, ENOMEM,
+ RTE_FLOW_ERROR_TYPE_UNSPECIFIED,
+ NULL, "cannot create table");
+ return NULL;
+ }
+ rte_atomic32_inc(&tbl->refcnt);
+ return tbl;
+}
+
+/**
+ * Release a flow table.
+ *
+ * @param[in] tbl
+ * Table resource to be released.
+ *
+ * @return
+ * Returns 0 if table was released, else return 1;
+ */
+static int
+flow_dv_tbl_resource_release(struct mlx5_flow_tbl_resource *tbl)
+{
+ if (rte_atomic32_dec_and_test(&tbl->refcnt)) {
+ mlx5_glue->dr_destroy_flow_tbl(tbl->obj);
+ tbl->obj = NULL;
+ return 0;
+ }
+ return 1;
+}
+
/**
* Register the flow matcher.
*
@@ -2795,33 +2961,20 @@ struct field_modify_info modify_tcp[] = {
return 0;
}
}
-#ifdef HAVE_MLX5DV_DR
- if (matcher->egress) {
- tbl = &priv->tx_tbl[matcher->group];
- if (!tbl->obj)
- tbl->obj = mlx5_glue->dr_create_flow_tbl
- (priv->tx_ns,
- matcher->group * MLX5_GROUP_FACTOR);
- } else {
- tbl = &priv->rx_tbl[matcher->group];
- if (!tbl->obj)
- tbl->obj = mlx5_glue->dr_create_flow_tbl
- (priv->rx_ns,
- matcher->group * MLX5_GROUP_FACTOR);
- }
- if (!tbl->obj)
- return rte_flow_error_set(error, ENOMEM,
- RTE_FLOW_ERROR_TYPE_UNSPECIFIED,
- NULL, "cannot create table");
-
- rte_atomic32_inc(&tbl->refcnt);
-#endif
/* Register new matcher. */
cache_matcher = rte_calloc(__func__, 1, sizeof(*cache_matcher), 0);
if (!cache_matcher)
return rte_flow_error_set(error, ENOMEM,
RTE_FLOW_ERROR_TYPE_UNSPECIFIED, NULL,
"cannot allocate matcher memory");
+ tbl = flow_dv_tbl_resource_get(dev, matcher->group * MLX5_GROUP_FACTOR,
+ matcher->egress, error);
+ if (!tbl) {
+ rte_free(cache_matcher);
+ return rte_flow_error_set(error, ENOMEM,
+ RTE_FLOW_ERROR_TYPE_UNSPECIFIED,
+ NULL, "cannot create table");
+ }
*cache_matcher = *matcher;
dv_attr.match_criteria_enable =
flow_dv_matcher_enable(cache_matcher->mask.buf);
@@ -2833,12 +2986,7 @@ struct field_modify_info modify_tcp[] = {
tbl->obj);
if (!cache_matcher->matcher_object) {
rte_free(cache_matcher);
-#ifdef HAVE_MLX5DV_DR
- if (rte_atomic32_dec_and_test(&tbl->refcnt)) {
- mlx5_glue->dr_destroy_flow_tbl(tbl->obj);
- tbl->obj = NULL;
- }
-#endif
+ flow_dv_tbl_resource_release(tbl);
return rte_flow_error_set(error, ENOMEM,
RTE_FLOW_ERROR_TYPE_UNSPECIFIED,
NULL, "cannot create matcher");
@@ -3000,6 +3148,9 @@ struct field_modify_info modify_tcp[] = {
const struct rte_flow_action *action = actions;
const struct rte_flow_action_count *count = action->conf;
const uint8_t *rss_key;
+ const struct rte_flow_action_jump *jump_data;
+ struct mlx5_flow_dv_jump_tbl_resource jump_tbl_resource;
+ struct mlx5_flow_tbl_resource *tbl;
switch (actions->type) {
case RTE_FLOW_ACTION_TYPE_VOID:
@@ -3138,6 +3289,31 @@ struct field_modify_info modify_tcp[] = {
/* If decap is followed by encap, handle it at encap. */
action_flags |= MLX5_FLOW_ACTION_RAW_DECAP;
break;
+ case RTE_FLOW_ACTION_TYPE_JUMP:
+ jump_data = action->conf;
+ tbl = flow_dv_tbl_resource_get(dev, jump_data->group *
+ MLX5_GROUP_FACTOR,
+ attr->egress, error);
+ if (!tbl)
+ return rte_flow_error_set
+ (error, errno,
+ RTE_FLOW_ERROR_TYPE_ACTION,
+ NULL,
+ "cannot create jump action.");
+ jump_tbl_resource.tbl = tbl;
+ if (flow_dv_jump_tbl_resource_register
+ (dev, &jump_tbl_resource, dev_flow, error)) {
+ flow_dv_tbl_resource_release(tbl);
+ return rte_flow_error_set
+ (error, errno,
+ RTE_FLOW_ERROR_TYPE_ACTION,
+ NULL,
+ "cannot create jump action.");
+ }
+ dev_flow->dv.actions[actions_n++] =
+ dev_flow->dv.jump->action;
+ action_flags |= MLX5_FLOW_ACTION_JUMP;
+ break;
case RTE_FLOW_ACTION_TYPE_SET_MAC_SRC:
case RTE_FLOW_ACTION_TYPE_SET_MAC_DST:
if (flow_dv_convert_action_modify_mac(&res, actions,
@@ -3457,10 +3633,7 @@ struct field_modify_info modify_tcp[] = {
tbl = &priv->tx_tbl[matcher->group];
else
tbl = &priv->rx_tbl[matcher->group];
- if (rte_atomic32_dec_and_test(&tbl->refcnt)) {
- mlx5_glue->dr_destroy_flow_tbl(tbl->obj);
- tbl->obj = NULL;
- }
+ flow_dv_tbl_resource_release(tbl);
rte_free(matcher);
DRV_LOG(DEBUG, "port %u matcher %p: removed",
dev->data->port_id, (void *)matcher);
@@ -3501,6 +3674,38 @@ struct field_modify_info modify_tcp[] = {
}
/**
+ * Release an jump to table action resource.
+ *
+ * @param flow
+ * Pointer to mlx5_flow.
+ *
+ * @return
+ * 1 while a reference on it exists, 0 when freed.
+ */
+static int
+flow_dv_jump_tbl_resource_release(struct mlx5_flow *flow)
+{
+ struct mlx5_flow_dv_jump_tbl_resource *cache_resource =
+ flow->dv.jump;
+
+ assert(cache_resource->action);
+ DRV_LOG(DEBUG, "jump table resource %p: refcnt %d--",
+ (void *)cache_resource,
+ rte_atomic32_read(&cache_resource->refcnt));
+ if (rte_atomic32_dec_and_test(&cache_resource->refcnt)) {
+ claim_zero(mlx5_glue->destroy_flow_action
+ (cache_resource->action));
+ LIST_REMOVE(cache_resource, next);
+ flow_dv_tbl_resource_release(cache_resource->tbl);
+ rte_free(cache_resource);
+ DRV_LOG(DEBUG, "jump table resource %p: removed",
+ (void *)cache_resource);
+ return 0;
+ }
+ return 1;
+}
+
+/**
* Release a modify-header resource.
*
* @param flow
@@ -3596,6 +3801,8 @@ struct field_modify_info modify_tcp[] = {
flow_dv_encap_decap_resource_release(dev_flow);
if (dev_flow->dv.modify_hdr)
flow_dv_modify_hdr_resource_release(dev_flow);
+ if (dev_flow->dv.jump)
+ flow_dv_jump_tbl_resource_release(dev_flow);
rte_free(dev_flow);
}
}
diff --git a/drivers/net/mlx5/mlx5_glue.c b/drivers/net/mlx5/mlx5_glue.c
index f509f85..fca34ca 100644
--- a/drivers/net/mlx5/mlx5_glue.c
+++ b/drivers/net/mlx5/mlx5_glue.c
@@ -370,6 +370,17 @@
}
static void *
+mlx5_glue_dr_create_flow_action_dest_flow_tbl(void *tbl)
+{
+#ifdef HAVE_MLX5DV_DR
+ return mlx5dv_dr_create_action_dest_flow_table(tbl);
+#else
+ (void)tbl;
+ return NULL;
+#endif
+}
+
+static void *
mlx5_glue_dr_create_flow_tbl(void *ns, uint32_t level)
{
#ifdef HAVE_MLX5DV_DR
@@ -833,6 +844,8 @@
.get_async_event = mlx5_glue_get_async_event,
.port_state_str = mlx5_glue_port_state_str,
.cq_ex_to_cq = mlx5_glue_cq_ex_to_cq,
+ .dr_create_flow_action_dest_flow_tbl =
+ mlx5_glue_dr_create_flow_action_dest_flow_tbl,
.dr_create_flow_tbl = mlx5_glue_dr_create_flow_tbl,
.dr_destroy_flow_tbl = mlx5_glue_dr_destroy_flow_tbl,
.dr_create_ns = mlx5_glue_dr_create_ns,
diff --git a/drivers/net/mlx5/mlx5_glue.h b/drivers/net/mlx5/mlx5_glue.h
index 7115575..16d5dd6 100644
--- a/drivers/net/mlx5/mlx5_glue.h
+++ b/drivers/net/mlx5/mlx5_glue.h
@@ -145,6 +145,7 @@ struct mlx5_glue {
struct ibv_async_event *event);
const char *(*port_state_str)(enum ibv_port_state port_state);
struct ibv_cq *(*cq_ex_to_cq)(struct ibv_cq_ex *cq);
+ void *(*dr_create_flow_action_dest_flow_tbl)(void *tbl);
void *(*dr_create_flow_tbl)(void *ns, uint32_t level);
int (*dr_destroy_flow_tbl)(void *tbl);
void *(*dr_create_ns)(struct ibv_context *ctx,
--
1.8.3.1
^ permalink raw reply [flat|nested] 52+ messages in thread
* [dpdk-dev] [PATCH v2 0/3] net/mlx5: Add Direct Rule support
2019-03-20 15:38 [dpdk-dev] [PATCH 0/3]net/mlx5: Add Direct Rule support Ori Kam
` (3 preceding siblings ...)
2019-03-20 15:39 ` [dpdk-dev] [PATCH 3/3] net/mlx5: add jump action support for NIC Ori Kam
@ 2019-03-28 16:32 ` Ori Kam
2019-03-28 16:32 ` Ori Kam
` (6 more replies)
4 siblings, 7 replies; 52+ messages in thread
From: Ori Kam @ 2019-03-28 16:32 UTC (permalink / raw)
To: Matan Azrad, Yongseok Koh, Shahaf Shuler; +Cc: dev, Ori Kam, Slava Ovsiienko
This patch set adds support for Direct Rules API for MLX5 driver.
As more and more applications like OVS are inserting and removing large
number of flows, the flow insertion rate become more critical.
In current MLX5 PMD rules are inserted to the device using the FW,
this limits the max insertion rate. The solution is using a new set of API
that are part of RDMA-Core. which enables bypassing the FW and writing
the flows directly to the device, which results in a much faster
insertion rate.
v2:
* Small issues fixed.
* Rebased.
Ori Kam (3):
net/mlx5: prepare Direct Verbs for Direct Rule
net/mlx5: add Direct Rules API
net/mlx5: add jump action support for NIC
drivers/net/mlx5/Makefile | 5 +
drivers/net/mlx5/mlx5.c | 16 ++
drivers/net/mlx5/mlx5.h | 23 ++
drivers/net/mlx5/mlx5_flow.c | 1 +
drivers/net/mlx5/mlx5_flow.h | 38 ++-
drivers/net/mlx5/mlx5_flow_dv.c | 509 ++++++++++++++++++++++++++++++++++------
drivers/net/mlx5/mlx5_glue.c | 300 +++++++++++++++++++----
drivers/net/mlx5/mlx5_glue.h | 50 ++--
drivers/net/mlx5/mlx5_prm.h | 24 +-
9 files changed, 832 insertions(+), 134 deletions(-)
--
1.8.3.1
^ permalink raw reply [flat|nested] 52+ messages in thread
* [dpdk-dev] [PATCH v2 0/3] net/mlx5: Add Direct Rule support
2019-03-28 16:32 ` [dpdk-dev] [PATCH v2 0/3] net/mlx5: Add Direct Rule support Ori Kam
@ 2019-03-28 16:32 ` Ori Kam
2019-03-28 16:32 ` [dpdk-dev] [PATCH v2 1/3] net/mlx5: prepare Direct Verbs for Direct Rule Ori Kam
` (5 subsequent siblings)
6 siblings, 0 replies; 52+ messages in thread
From: Ori Kam @ 2019-03-28 16:32 UTC (permalink / raw)
To: Matan Azrad, Yongseok Koh, Shahaf Shuler; +Cc: dev, Ori Kam, Slava Ovsiienko
This patch set adds support for Direct Rules API for MLX5 driver.
As more and more applications like OVS are inserting and removing large
number of flows, the flow insertion rate become more critical.
In current MLX5 PMD rules are inserted to the device using the FW,
this limits the max insertion rate. The solution is using a new set of API
that are part of RDMA-Core. which enables bypassing the FW and writing
the flows directly to the device, which results in a much faster
insertion rate.
v2:
* Small issues fixed.
* Rebased.
Ori Kam (3):
net/mlx5: prepare Direct Verbs for Direct Rule
net/mlx5: add Direct Rules API
net/mlx5: add jump action support for NIC
drivers/net/mlx5/Makefile | 5 +
drivers/net/mlx5/mlx5.c | 16 ++
drivers/net/mlx5/mlx5.h | 23 ++
drivers/net/mlx5/mlx5_flow.c | 1 +
drivers/net/mlx5/mlx5_flow.h | 38 ++-
drivers/net/mlx5/mlx5_flow_dv.c | 509 ++++++++++++++++++++++++++++++++++------
drivers/net/mlx5/mlx5_glue.c | 300 +++++++++++++++++++----
drivers/net/mlx5/mlx5_glue.h | 50 ++--
drivers/net/mlx5/mlx5_prm.h | 24 +-
9 files changed, 832 insertions(+), 134 deletions(-)
--
1.8.3.1
^ permalink raw reply [flat|nested] 52+ messages in thread
* [dpdk-dev] [PATCH v2 1/3] net/mlx5: prepare Direct Verbs for Direct Rule
2019-03-28 16:32 ` [dpdk-dev] [PATCH v2 0/3] net/mlx5: Add Direct Rule support Ori Kam
2019-03-28 16:32 ` Ori Kam
@ 2019-03-28 16:32 ` Ori Kam
2019-03-28 16:32 ` Ori Kam
` (2 more replies)
2019-03-28 16:32 ` [dpdk-dev] [PATCH v2 2/3] net/mlx5: add Direct Rules API Ori Kam
` (4 subsequent siblings)
6 siblings, 3 replies; 52+ messages in thread
From: Ori Kam @ 2019-03-28 16:32 UTC (permalink / raw)
To: Matan Azrad, Yongseok Koh, Shahaf Shuler; +Cc: dev, Ori Kam, Slava Ovsiienko
This is the first patch of a series that is designed to enable the
Direct Rules API.
The main difference between Direct Verbs and Direct Rules from API
prespective, is that in Direct Rules each action has it's own create
function and the object itself is of type void.
In this patch I'm adding functions to generate actions that currenlty
are done without create action, and I'm changing the action type to be
void *, so in next patches only the glue functions will need to change.
Signed-off-by: Ori Kam <orika@mellanox.com>
---
drivers/net/mlx5/mlx5.h | 2 +
drivers/net/mlx5/mlx5_flow.h | 17 +++-
drivers/net/mlx5/mlx5_flow_dv.c | 188 ++++++++++++++++++++++++++++------------
drivers/net/mlx5/mlx5_glue.c | 141 ++++++++++++++++++++++++------
drivers/net/mlx5/mlx5_glue.h | 26 +++---
5 files changed, 278 insertions(+), 96 deletions(-)
diff --git a/drivers/net/mlx5/mlx5.h b/drivers/net/mlx5/mlx5.h
index 7402798..b587359 100644
--- a/drivers/net/mlx5/mlx5.h
+++ b/drivers/net/mlx5/mlx5.h
@@ -269,6 +269,8 @@ struct mlx5_priv {
LIST_HEAD(matchers, mlx5_flow_dv_matcher) matchers;
LIST_HEAD(encap_decap, mlx5_flow_dv_encap_decap_resource) encaps_decaps;
LIST_HEAD(modify_cmd, mlx5_flow_dv_modify_hdr_resource) modify_cmds;
+ LIST_HEAD(tag, mlx5_flow_dv_tag_resource) tags;
+ /* Tags resources cache. */
uint32_t link_speed_capa; /* Link speed capabilities. */
struct mlx5_xstats_ctrl xstats_ctrl; /* Extended stats control. */
struct mlx5_stats_ctrl stats_ctrl; /* Stats control. */
diff --git a/drivers/net/mlx5/mlx5_flow.h b/drivers/net/mlx5/mlx5_flow.h
index e1e798b..4f69ae2 100644
--- a/drivers/net/mlx5/mlx5_flow.h
+++ b/drivers/net/mlx5/mlx5_flow.h
@@ -214,7 +214,7 @@ struct mlx5_flow_dv_encap_decap_resource {
LIST_ENTRY(mlx5_flow_dv_encap_decap_resource) next;
/* Pointer to next element. */
rte_atomic32_t refcnt; /**< Reference counter. */
- struct ibv_flow_action *verbs_action;
+ void *verbs_action;
/**< Verbs encap/decap action object. */
uint8_t buf[MLX5_ENCAP_MAX_LEN];
size_t size;
@@ -222,6 +222,16 @@ struct mlx5_flow_dv_encap_decap_resource {
uint8_t ft_type;
};
+/* Tag resource structure. */
+struct mlx5_flow_dv_tag_resource {
+ LIST_ENTRY(mlx5_flow_dv_tag_resource) next;
+ /* Pointer to next element. */
+ rte_atomic32_t refcnt; /**< Reference counter. */
+ void *action;
+ /**< Verbs tag action object. */
+ uint32_t tag; /**< the tag value. */
+};
+
/* Number of modification commands. */
#define MLX5_MODIFY_NUM 8
@@ -259,7 +269,7 @@ struct mlx5_flow_dv {
/**< Pointer to modify header resource in cache. */
struct ibv_flow *flow; /**< Installed flow. */
#ifdef HAVE_IBV_FLOW_DV_SUPPORT
- struct mlx5dv_flow_action_attr actions[MLX5_DV_MAX_NUMBER_OF_ACTIONS];
+ void *actions[MLX5_DV_MAX_NUMBER_OF_ACTIONS];
/**< Action list. */
#endif
int actions_n; /**< number of actions. */
@@ -332,6 +342,7 @@ struct mlx5_flow_counter {
};
uint64_t hits; /**< Number of packets matched by the rule. */
uint64_t bytes; /**< Number of bytes matched by the rule. */
+ void *action; /**< Pointer to the dv action. */
};
/* Flow structure. */
@@ -339,6 +350,8 @@ struct rte_flow {
TAILQ_ENTRY(rte_flow) next; /**< Pointer to the next flow structure. */
enum mlx5_flow_drv_type drv_type; /**< Drvier type. */
struct mlx5_flow_counter *counter; /**< Holds flow counter. */
+ struct mlx5_flow_dv_tag_resource *tag_resource;
+ /**< pointer to the tag action. */
struct rte_flow_action_rss rss;/**< RSS context. */
uint8_t key[MLX5_RSS_HASH_KEY_LEN]; /**< RSS hash key. */
uint16_t (*queue)[]; /**< Destination queues to redirect traffic to. */
diff --git a/drivers/net/mlx5/mlx5_flow_dv.c b/drivers/net/mlx5/mlx5_flow_dv.c
index 57847fb..b0c50a7 100644
--- a/drivers/net/mlx5/mlx5_flow_dv.c
+++ b/drivers/net/mlx5/mlx5_flow_dv.c
@@ -32,13 +32,8 @@
#include "mlx5_prm.h"
#include "mlx5_glue.h"
#include "mlx5_flow.h"
-
#ifdef HAVE_IBV_FLOW_DV_SUPPORT
-#ifndef HAVE_IBV_FLOW_DEVX_COUNTERS
-#define MLX5DV_FLOW_ACTION_COUNTER_DEVX 0
-#endif
-
union flow_dv_attr {
struct {
uint32_t valid:1;
@@ -1537,6 +1532,11 @@ struct field_modify_info modify_tcp[] = {
.id = id,
.dcs = dcs,
};
+ tmpl.action = mlx5_glue->dv_create_flow_action_counter(dcs->obj, 0);
+ if (!tmpl.action) {
+ ret = errno;
+ goto error_exit;
+ }
*cnt = tmpl;
LIST_INSERT_HEAD(&priv->flow_counters, cnt, next);
return cnt;
@@ -2828,6 +2828,97 @@ struct field_modify_info modify_tcp[] = {
}
/**
+ * Find existing tag resource or create and register a new one.
+ *
+ * @param dev[in, out]
+ * Pointer to rte_eth_dev structure.
+ * @param[in, out] resource
+ * Pointer to tag resource.
+ * @parm[in, out] dev_flow
+ * Pointer to the dev_flow.
+ * @param[out] error
+ * pointer to error structure.
+ *
+ * @return
+ * 0 on success otherwise -errno and errno is set.
+ */
+static int
+flow_dv_tag_resource_register
+ (struct rte_eth_dev *dev,
+ struct mlx5_flow_dv_tag_resource *resource,
+ struct mlx5_flow *dev_flow,
+ struct rte_flow_error *error)
+{
+ struct mlx5_priv *priv = dev->data->dev_private;
+ struct mlx5_flow_dv_tag_resource *cache_resource;
+
+ /* Lookup a matching resource from cache. */
+ LIST_FOREACH(cache_resource, &priv->tags, next) {
+ if (resource->tag == cache_resource->tag) {
+ DRV_LOG(DEBUG, "tag resource %p: refcnt %d++",
+ (void *)cache_resource,
+ rte_atomic32_read(&cache_resource->refcnt));
+ rte_atomic32_inc(&cache_resource->refcnt);
+ dev_flow->flow->tag_resource = cache_resource;
+ return 0;
+ }
+ }
+ /* Register new resource. */
+ cache_resource = rte_calloc(__func__, 1, sizeof(*cache_resource), 0);
+ if (!cache_resource)
+ return rte_flow_error_set(error, ENOMEM,
+ RTE_FLOW_ERROR_TYPE_UNSPECIFIED, NULL,
+ "cannot allocate resource memory");
+ *cache_resource = *resource;
+ cache_resource->action = mlx5_glue->dv_create_flow_action_tag
+ (resource->tag);
+ if (!cache_resource->action) {
+ rte_free(cache_resource);
+ return rte_flow_error_set(error, ENOMEM,
+ RTE_FLOW_ERROR_TYPE_UNSPECIFIED,
+ NULL, "cannot create action");
+ }
+ rte_atomic32_init(&cache_resource->refcnt);
+ rte_atomic32_inc(&cache_resource->refcnt);
+ LIST_INSERT_HEAD(&priv->tags, cache_resource, next);
+ dev_flow->flow->tag_resource = cache_resource;
+ DRV_LOG(DEBUG, "new tag resource %p: refcnt %d++",
+ (void *)cache_resource,
+ rte_atomic32_read(&cache_resource->refcnt));
+ return 0;
+}
+
+/**
+ * Release the tag.
+ *
+ * @param dev
+ * Pointer to Ethernet device.
+ * @param flow
+ * Pointer to mlx5_flow.
+ *
+ * @return
+ * 1 while a reference on it exists, 0 when freed.
+ */
+static int
+flow_dv_tag_release(struct rte_eth_dev *dev,
+ struct mlx5_flow_dv_tag_resource *tag)
+{
+ assert(tag);
+ DRV_LOG(DEBUG, "port %u tag %p: refcnt %d--",
+ dev->data->port_id, (void *)tag,
+ rte_atomic32_read(&tag->refcnt));
+ if (rte_atomic32_dec_and_test(&tag->refcnt)) {
+ claim_zero(mlx5_glue->destroy_flow_action(tag->action));
+ LIST_REMOVE(tag, next);
+ DRV_LOG(DEBUG, "port %u tag %p: removed",
+ dev->data->port_id, (void *)tag);
+ rte_free(tag);
+ return 0;
+ }
+ return 1;
+}
+
+/**
* Fill the flow with DV spec.
*
* @param[in] dev
@@ -2872,6 +2963,7 @@ struct field_modify_info modify_tcp[] = {
MLX5DV_FLOW_TABLE_TYPE_NIC_RX
};
union flow_dv_attr flow_attr = { .attr = 0 };
+ struct mlx5_flow_dv_tag_resource tag_resource;
if (priority == MLX5_FLOW_PRIO_RSVD)
priority = priv->config.flow_prio - 1;
@@ -2886,26 +2978,29 @@ struct field_modify_info modify_tcp[] = {
case RTE_FLOW_ACTION_TYPE_VOID:
break;
case RTE_FLOW_ACTION_TYPE_FLAG:
- dev_flow->dv.actions[actions_n].type =
- MLX5DV_FLOW_ACTION_TAG;
- dev_flow->dv.actions[actions_n].tag_value =
+ tag_resource.tag =
mlx5_flow_mark_set(MLX5_FLOW_MARK_DEFAULT);
- actions_n++;
+ if (!flow->tag_resource)
+ if (flow_dv_tag_resource_register
+ (dev, &tag_resource, dev_flow, error))
+ return errno;
+ dev_flow->dv.actions[actions_n++] =
+ flow->tag_resource->action;
action_flags |= MLX5_FLOW_ACTION_FLAG;
break;
case RTE_FLOW_ACTION_TYPE_MARK:
- dev_flow->dv.actions[actions_n].type =
- MLX5DV_FLOW_ACTION_TAG;
- dev_flow->dv.actions[actions_n].tag_value =
- mlx5_flow_mark_set
- (((const struct rte_flow_action_mark *)
- (actions->conf))->id);
- actions_n++;
+ tag_resource.tag = mlx5_flow_mark_set
+ (((const struct rte_flow_action_mark *)
+ (actions->conf))->id);
+ if (!flow->tag_resource)
+ if (flow_dv_tag_resource_register
+ (dev, &tag_resource, dev_flow, error))
+ return errno;
+ dev_flow->dv.actions[actions_n++] =
+ flow->tag_resource->action;
action_flags |= MLX5_FLOW_ACTION_MARK;
break;
case RTE_FLOW_ACTION_TYPE_DROP:
- dev_flow->dv.actions[actions_n].type =
- MLX5DV_FLOW_ACTION_DROP;
action_flags |= MLX5_FLOW_ACTION_DROP;
break;
case RTE_FLOW_ACTION_TYPE_QUEUE:
@@ -2933,17 +3028,13 @@ struct field_modify_info modify_tcp[] = {
rte_errno = ENOTSUP;
goto cnt_err;
}
- flow->counter =
- flow_dv_counter_new(dev,
- count->shared, count->id);
+ flow->counter = flow_dv_counter_new(dev, count->shared,
+ count->id);
if (flow->counter == NULL)
goto cnt_err;
- dev_flow->dv.actions[actions_n].type =
- MLX5DV_FLOW_ACTION_COUNTER_DEVX;
- dev_flow->dv.actions[actions_n].obj =
- flow->counter->dcs->obj;
+ dev_flow->dv.actions[actions_n++] =
+ flow->counter->action;
action_flags |= MLX5_FLOW_ACTION_COUNT;
- ++actions_n;
break;
cnt_err:
if (rte_errno == ENOTSUP)
@@ -2964,11 +3055,8 @@ struct field_modify_info modify_tcp[] = {
if (flow_dv_create_action_l2_encap(dev, actions,
dev_flow, error))
return -rte_errno;
- dev_flow->dv.actions[actions_n].type =
- MLX5DV_FLOW_ACTION_IBV_FLOW_ACTION;
- dev_flow->dv.actions[actions_n].action =
+ dev_flow->dv.actions[actions_n++] =
dev_flow->dv.encap_decap->verbs_action;
- actions_n++;
action_flags |= actions->type ==
RTE_FLOW_ACTION_TYPE_VXLAN_ENCAP ?
MLX5_FLOW_ACTION_VXLAN_ENCAP :
@@ -2979,11 +3067,8 @@ struct field_modify_info modify_tcp[] = {
if (flow_dv_create_action_l2_decap(dev, dev_flow,
error))
return -rte_errno;
- dev_flow->dv.actions[actions_n].type =
- MLX5DV_FLOW_ACTION_IBV_FLOW_ACTION;
- dev_flow->dv.actions[actions_n].action =
+ dev_flow->dv.actions[actions_n++] =
dev_flow->dv.encap_decap->verbs_action;
- actions_n++;
action_flags |= actions->type ==
RTE_FLOW_ACTION_TYPE_VXLAN_DECAP ?
MLX5_FLOW_ACTION_VXLAN_DECAP :
@@ -2995,9 +3080,7 @@ struct field_modify_info modify_tcp[] = {
if (flow_dv_create_action_raw_encap
(dev, actions, dev_flow, attr, error))
return -rte_errno;
- dev_flow->dv.actions[actions_n].type =
- MLX5DV_FLOW_ACTION_IBV_FLOW_ACTION;
- dev_flow->dv.actions[actions_n].action =
+ dev_flow->dv.actions[actions_n++] =
dev_flow->dv.encap_decap->verbs_action;
} else {
/* Handle encap without preceding decap. */
@@ -3005,12 +3088,9 @@ struct field_modify_info modify_tcp[] = {
dev_flow,
error))
return -rte_errno;
- dev_flow->dv.actions[actions_n].type =
- MLX5DV_FLOW_ACTION_IBV_FLOW_ACTION;
- dev_flow->dv.actions[actions_n].action =
+ dev_flow->dv.actions[actions_n++] =
dev_flow->dv.encap_decap->verbs_action;
}
- actions_n++;
action_flags |= MLX5_FLOW_ACTION_RAW_ENCAP;
break;
case RTE_FLOW_ACTION_TYPE_RAW_DECAP:
@@ -3025,11 +3105,8 @@ struct field_modify_info modify_tcp[] = {
dev_flow,
error))
return -rte_errno;
- dev_flow->dv.actions[actions_n].type =
- MLX5DV_FLOW_ACTION_IBV_FLOW_ACTION;
- dev_flow->dv.actions[actions_n].action =
+ dev_flow->dv.actions[actions_n++] =
dev_flow->dv.encap_decap->verbs_action;
- actions_n++;
}
/* If decap is followed by encap, handle it at encap. */
action_flags |= MLX5_FLOW_ACTION_RAW_DECAP;
@@ -3098,11 +3175,8 @@ struct field_modify_info modify_tcp[] = {
dev_flow,
error))
return -rte_errno;
- dev_flow->dv.actions[actions_n].type =
- MLX5DV_FLOW_ACTION_IBV_FLOW_ACTION;
- dev_flow->dv.actions[actions_n].action =
+ dev_flow->dv.actions[actions_n++] =
dev_flow->dv.modify_hdr->verbs_action;
- actions_n++;
}
break;
default:
@@ -3277,9 +3351,9 @@ struct field_modify_info modify_tcp[] = {
"cannot get drop hash queue");
goto error;
}
- dv->actions[n].type = MLX5DV_FLOW_ACTION_DEST_IBV_QP;
- dv->actions[n].qp = dv->hrxq->qp;
- n++;
+ dv->actions[n++] =
+ mlx5_glue->dv_create_flow_action_dest_ibv_qp
+ (dv->hrxq->qp);
} else if (flow->actions &
(MLX5_FLOW_ACTION_QUEUE | MLX5_FLOW_ACTION_RSS)) {
struct mlx5_hrxq *hrxq;
@@ -3304,9 +3378,9 @@ struct field_modify_info modify_tcp[] = {
goto error;
}
dv->hrxq = hrxq;
- dv->actions[n].type = MLX5DV_FLOW_ACTION_DEST_IBV_QP;
- dv->actions[n].qp = hrxq->qp;
- n++;
+ dv->actions[n++] =
+ mlx5_glue->dv_create_flow_action_dest_ibv_qp
+ (dv->hrxq->qp);
}
dv->flow =
mlx5_glue->dv_create_flow(dv->matcher->matcher_object,
@@ -3484,6 +3558,10 @@ struct field_modify_info modify_tcp[] = {
flow_dv_counter_release(flow->counter);
flow->counter = NULL;
}
+ if (flow->tag_resource) {
+ flow_dv_tag_release(dev, flow->tag_resource);
+ flow->tag_resource = NULL;
+ }
while (!LIST_EMPTY(&flow->dev_flows)) {
dev_flow = LIST_FIRST(&flow->dev_flows);
LIST_REMOVE(dev_flow, next);
diff --git a/drivers/net/mlx5/mlx5_glue.c b/drivers/net/mlx5/mlx5_glue.c
index c817d86..cfd939d 100644
--- a/drivers/net/mlx5/mlx5_glue.c
+++ b/drivers/net/mlx5/mlx5_glue.c
@@ -175,10 +175,20 @@
}
static int
-mlx5_glue_destroy_flow_action(struct ibv_flow_action *action)
+mlx5_glue_destroy_flow_action(void *action)
{
#ifdef HAVE_IBV_FLOW_DV_SUPPORT
- return ibv_destroy_flow_action(action);
+ struct mlx5dv_flow_action_attr *attr = action;
+ int res = 0;
+ switch (attr->type) {
+ case MLX5DV_FLOW_ACTION_TAG:
+ break;
+ default:
+ res = ibv_destroy_flow_action(attr->action);
+ break;
+ }
+ free(action);
+ return res;
#else
(void)action;
return ENOTSUP;
@@ -430,16 +440,23 @@
mlx5_glue_dv_create_flow(struct mlx5dv_flow_matcher *matcher,
struct mlx5dv_flow_match_parameters *match_value,
size_t num_actions,
- struct mlx5dv_flow_action_attr *actions_attr)
+ void *actions[])
{
#ifdef HAVE_IBV_FLOW_DV_SUPPORT
+ struct mlx5dv_flow_action_attr actions_attr[8];
+
+ if (num_actions > 8)
+ return NULL;
+ for (size_t i = 0; i < num_actions; i++)
+ actions_attr[i] =
+ *((struct mlx5dv_flow_action_attr *)(actions[i]));
return mlx5dv_create_flow(matcher, match_value,
num_actions, actions_attr);
#else
(void)matcher;
(void)match_value;
(void)num_actions;
- (void)actions_attr;
+ (void)actions;
return NULL;
#endif
}
@@ -455,31 +472,45 @@
#endif
}
-static struct ibv_flow_action *
-mlx5_glue_dv_create_flow_action_packet_reformat
- (struct ibv_context *ctx,
- size_t data_sz,
- void *data,
- enum mlx5dv_flow_action_packet_reformat_type reformat_type,
- enum mlx5dv_flow_table_type ft_type)
+static void *
+mlx5_glue_dv_create_flow_action_counter(void *counter_obj, uint32_t offset)
{
#ifdef HAVE_IBV_FLOW_DV_SUPPORT
- return mlx5dv_create_flow_action_packet_reformat(ctx,
- data_sz,
- data,
- reformat_type,
- ft_type);
+ struct mlx5dv_flow_action_attr *action;
+
+ (void)offset;
+ action = malloc(sizeof(*action));
+ if (!action)
+ return NULL;
+ action->type = MLX5DV_FLOW_ACTION_COUNTER_DEVX;
+ action->obj = counter_obj;
+ return action;
#else
- (void)ctx;
- (void)data_sz;
- (void)data;
- (void)reformat_type;
- (void)ft_type;
+ (void)counter_obj;
+ (void)offset;
return NULL;
#endif
}
-static struct ibv_flow_action *
+static void *
+mlx5_glue_dv_create_flow_action_dest_ibv_qp(void *qp)
+{
+#ifdef HAVE_IBV_FLOW_DV_SUPPORT
+ struct mlx5dv_flow_action_attr *action;
+
+ action = malloc(sizeof(*action));
+ if (!action)
+ return NULL;
+ action->type = MLX5DV_FLOW_ACTION_DEST_IBV_QP;
+ action->obj = qp;
+ return action;
+#else
+ (void)qp;
+ return NULL;
+#endif
+}
+
+static void *
mlx5_glue_dv_create_flow_action_modify_header
(struct ibv_context *ctx,
size_t actions_sz,
@@ -487,8 +518,15 @@
enum mlx5dv_flow_table_type ft_type)
{
#ifdef HAVE_IBV_FLOW_DV_SUPPORT
- return mlx5dv_create_flow_action_modify_header(ctx, actions_sz,
- actions, ft_type);
+ struct mlx5dv_flow_action_attr *action;
+
+ action = malloc(sizeof(*action));
+ if (!action)
+ return NULL;
+ action->type = MLX5DV_FLOW_ACTION_IBV_FLOW_ACTION;
+ action->action = mlx5dv_create_flow_action_modify_header
+ (ctx, actions_sz, actions, ft_type);
+ return action;
#else
(void)ctx;
(void)actions_sz;
@@ -498,6 +536,50 @@
#endif
}
+static void *
+mlx5_glue_dv_create_flow_action_packet_reformat
+ (struct ibv_context *ctx,
+ size_t data_sz,
+ void *data,
+ enum mlx5dv_flow_action_packet_reformat_type reformat_type,
+ enum mlx5dv_flow_table_type ft_type)
+{
+#ifdef HAVE_IBV_FLOW_DV_SUPPORT
+ struct mlx5dv_flow_action_attr *action;
+
+ action = malloc(sizeof(*action));
+ if (!action)
+ return NULL;
+ action->type = MLX5DV_FLOW_ACTION_IBV_FLOW_ACTION;
+ action->action = mlx5dv_create_flow_action_packet_reformat
+ (ctx, data_sz, data, reformat_type, ft_type);
+ return action;
+#else
+ (void)ctx;
+ (void)data_sz;
+ (void)data;
+ (void)reformat_type;
+ (void)ft_type;
+ return NULL;
+#endif
+}
+
+static void *
+mlx5_glue_dv_create_flow_action_tag(uint32_t tag)
+{
+#ifdef HAVE_IBV_FLOW_DV_SUPPORT
+ struct mlx5dv_flow_action_attr *action;
+ action = malloc(sizeof(*action));
+ if (!action)
+ return NULL;
+ action->type = MLX5DV_FLOW_ACTION_TAG;
+ action->tag_value = tag;
+ return action;
+#endif
+ (void)tag;
+ return NULL;
+}
+
static struct ibv_context *
mlx5_glue_dv_open_device(struct ibv_device *device)
{
@@ -645,10 +727,15 @@
.dv_create_flow_matcher = mlx5_glue_dv_create_flow_matcher,
.dv_destroy_flow_matcher = mlx5_glue_dv_destroy_flow_matcher,
.dv_create_flow = mlx5_glue_dv_create_flow,
- .dv_create_flow_action_packet_reformat =
- mlx5_glue_dv_create_flow_action_packet_reformat,
+ .dv_create_flow_action_counter =
+ mlx5_glue_dv_create_flow_action_counter,
+ .dv_create_flow_action_dest_ibv_qp =
+ mlx5_glue_dv_create_flow_action_dest_ibv_qp,
.dv_create_flow_action_modify_header =
- mlx5_glue_dv_create_flow_action_modify_header,
+ mlx5_glue_dv_create_flow_action_modify_header,
+ .dv_create_flow_action_packet_reformat =
+ mlx5_glue_dv_create_flow_action_packet_reformat,
+ .dv_create_flow_action_tag = mlx5_glue_dv_create_flow_action_tag,
.dv_open_device = mlx5_glue_dv_open_device,
.devx_obj_create = mlx5_glue_devx_obj_create,
.devx_obj_destroy = mlx5_glue_devx_obj_destroy,
diff --git a/drivers/net/mlx5/mlx5_glue.h b/drivers/net/mlx5/mlx5_glue.h
index b118960..167f1f7 100644
--- a/drivers/net/mlx5/mlx5_glue.h
+++ b/drivers/net/mlx5/mlx5_glue.h
@@ -55,6 +55,10 @@
enum mlx5dv_flow_table_type { flow_table_type = 0, };
#endif
+#ifndef HAVE_IBV_FLOW_DEVX_COUNTERS
+#define MLX5DV_FLOW_ACTION_COUNTER_DEVX 0
+#endif
+
#ifndef HAVE_IBV_DEVX_OBJ
struct mlx5dv_devx_obj;
#endif
@@ -98,7 +102,7 @@ struct mlx5_glue {
struct ibv_flow *(*create_flow)(struct ibv_qp *qp,
struct ibv_flow_attr *flow);
int (*destroy_flow)(struct ibv_flow *flow_id);
- int (*destroy_flow_action)(struct ibv_flow_action *action);
+ int (*destroy_flow_action)(void *action);
struct ibv_qp *(*create_qp)(struct ibv_pd *pd,
struct ibv_qp_init_attr *qp_init_attr);
struct ibv_qp *(*create_qp_ex)
@@ -160,19 +164,17 @@ struct mlx5_glue {
int (*dv_destroy_flow_matcher)(struct mlx5dv_flow_matcher *matcher);
struct ibv_flow *(*dv_create_flow)(struct mlx5dv_flow_matcher *matcher,
struct mlx5dv_flow_match_parameters *match_value,
- size_t num_actions,
- struct mlx5dv_flow_action_attr *actions_attr);
- struct ibv_flow_action *(*dv_create_flow_action_packet_reformat)
- (struct ibv_context *ctx,
- size_t data_sz,
- void *data,
+ size_t num_actions, void *actions[]);
+ void *(*dv_create_flow_action_counter)(void *obj, uint32_t offset);
+ void *(*dv_create_flow_action_dest_ibv_qp)(void *qp);
+ void *(*dv_create_flow_action_modify_header)
+ (struct ibv_context *ctx, size_t actions_sz, uint64_t actions[],
+ enum mlx5dv_flow_table_type ft_type);
+ void *(*dv_create_flow_action_packet_reformat)
+ (struct ibv_context *ctx, size_t data_sz, void *data,
enum mlx5dv_flow_action_packet_reformat_type reformat_type,
enum mlx5dv_flow_table_type ft_type);
- struct ibv_flow_action *(*dv_create_flow_action_modify_header)
- (struct ibv_context *ctx,
- size_t actions_sz,
- uint64_t actions[],
- enum mlx5dv_flow_table_type ft_type);
+ void *(*dv_create_flow_action_tag)(uint32_t tag);
struct ibv_context *(*dv_open_device)(struct ibv_device *device);
struct mlx5dv_devx_obj *(*devx_obj_create)
(struct ibv_context *ctx,
--
1.8.3.1
^ permalink raw reply [flat|nested] 52+ messages in thread
* [dpdk-dev] [PATCH v2 1/3] net/mlx5: prepare Direct Verbs for Direct Rule
2019-03-28 16:32 ` [dpdk-dev] [PATCH v2 1/3] net/mlx5: prepare Direct Verbs for Direct Rule Ori Kam
@ 2019-03-28 16:32 ` Ori Kam
2019-04-01 14:38 ` Slava Ovsiienko
2019-04-03 10:15 ` Shahaf Shuler
2 siblings, 0 replies; 52+ messages in thread
From: Ori Kam @ 2019-03-28 16:32 UTC (permalink / raw)
To: Matan Azrad, Yongseok Koh, Shahaf Shuler; +Cc: dev, Ori Kam, Slava Ovsiienko
This is the first patch of a series that is designed to enable the
Direct Rules API.
The main difference between Direct Verbs and Direct Rules from API
prespective, is that in Direct Rules each action has it's own create
function and the object itself is of type void.
In this patch I'm adding functions to generate actions that currenlty
are done without create action, and I'm changing the action type to be
void *, so in next patches only the glue functions will need to change.
Signed-off-by: Ori Kam <orika@mellanox.com>
---
drivers/net/mlx5/mlx5.h | 2 +
drivers/net/mlx5/mlx5_flow.h | 17 +++-
drivers/net/mlx5/mlx5_flow_dv.c | 188 ++++++++++++++++++++++++++++------------
drivers/net/mlx5/mlx5_glue.c | 141 ++++++++++++++++++++++++------
drivers/net/mlx5/mlx5_glue.h | 26 +++---
5 files changed, 278 insertions(+), 96 deletions(-)
diff --git a/drivers/net/mlx5/mlx5.h b/drivers/net/mlx5/mlx5.h
index 7402798..b587359 100644
--- a/drivers/net/mlx5/mlx5.h
+++ b/drivers/net/mlx5/mlx5.h
@@ -269,6 +269,8 @@ struct mlx5_priv {
LIST_HEAD(matchers, mlx5_flow_dv_matcher) matchers;
LIST_HEAD(encap_decap, mlx5_flow_dv_encap_decap_resource) encaps_decaps;
LIST_HEAD(modify_cmd, mlx5_flow_dv_modify_hdr_resource) modify_cmds;
+ LIST_HEAD(tag, mlx5_flow_dv_tag_resource) tags;
+ /* Tags resources cache. */
uint32_t link_speed_capa; /* Link speed capabilities. */
struct mlx5_xstats_ctrl xstats_ctrl; /* Extended stats control. */
struct mlx5_stats_ctrl stats_ctrl; /* Stats control. */
diff --git a/drivers/net/mlx5/mlx5_flow.h b/drivers/net/mlx5/mlx5_flow.h
index e1e798b..4f69ae2 100644
--- a/drivers/net/mlx5/mlx5_flow.h
+++ b/drivers/net/mlx5/mlx5_flow.h
@@ -214,7 +214,7 @@ struct mlx5_flow_dv_encap_decap_resource {
LIST_ENTRY(mlx5_flow_dv_encap_decap_resource) next;
/* Pointer to next element. */
rte_atomic32_t refcnt; /**< Reference counter. */
- struct ibv_flow_action *verbs_action;
+ void *verbs_action;
/**< Verbs encap/decap action object. */
uint8_t buf[MLX5_ENCAP_MAX_LEN];
size_t size;
@@ -222,6 +222,16 @@ struct mlx5_flow_dv_encap_decap_resource {
uint8_t ft_type;
};
+/* Tag resource structure. */
+struct mlx5_flow_dv_tag_resource {
+ LIST_ENTRY(mlx5_flow_dv_tag_resource) next;
+ /* Pointer to next element. */
+ rte_atomic32_t refcnt; /**< Reference counter. */
+ void *action;
+ /**< Verbs tag action object. */
+ uint32_t tag; /**< the tag value. */
+};
+
/* Number of modification commands. */
#define MLX5_MODIFY_NUM 8
@@ -259,7 +269,7 @@ struct mlx5_flow_dv {
/**< Pointer to modify header resource in cache. */
struct ibv_flow *flow; /**< Installed flow. */
#ifdef HAVE_IBV_FLOW_DV_SUPPORT
- struct mlx5dv_flow_action_attr actions[MLX5_DV_MAX_NUMBER_OF_ACTIONS];
+ void *actions[MLX5_DV_MAX_NUMBER_OF_ACTIONS];
/**< Action list. */
#endif
int actions_n; /**< number of actions. */
@@ -332,6 +342,7 @@ struct mlx5_flow_counter {
};
uint64_t hits; /**< Number of packets matched by the rule. */
uint64_t bytes; /**< Number of bytes matched by the rule. */
+ void *action; /**< Pointer to the dv action. */
};
/* Flow structure. */
@@ -339,6 +350,8 @@ struct rte_flow {
TAILQ_ENTRY(rte_flow) next; /**< Pointer to the next flow structure. */
enum mlx5_flow_drv_type drv_type; /**< Drvier type. */
struct mlx5_flow_counter *counter; /**< Holds flow counter. */
+ struct mlx5_flow_dv_tag_resource *tag_resource;
+ /**< pointer to the tag action. */
struct rte_flow_action_rss rss;/**< RSS context. */
uint8_t key[MLX5_RSS_HASH_KEY_LEN]; /**< RSS hash key. */
uint16_t (*queue)[]; /**< Destination queues to redirect traffic to. */
diff --git a/drivers/net/mlx5/mlx5_flow_dv.c b/drivers/net/mlx5/mlx5_flow_dv.c
index 57847fb..b0c50a7 100644
--- a/drivers/net/mlx5/mlx5_flow_dv.c
+++ b/drivers/net/mlx5/mlx5_flow_dv.c
@@ -32,13 +32,8 @@
#include "mlx5_prm.h"
#include "mlx5_glue.h"
#include "mlx5_flow.h"
-
#ifdef HAVE_IBV_FLOW_DV_SUPPORT
-#ifndef HAVE_IBV_FLOW_DEVX_COUNTERS
-#define MLX5DV_FLOW_ACTION_COUNTER_DEVX 0
-#endif
-
union flow_dv_attr {
struct {
uint32_t valid:1;
@@ -1537,6 +1532,11 @@ struct field_modify_info modify_tcp[] = {
.id = id,
.dcs = dcs,
};
+ tmpl.action = mlx5_glue->dv_create_flow_action_counter(dcs->obj, 0);
+ if (!tmpl.action) {
+ ret = errno;
+ goto error_exit;
+ }
*cnt = tmpl;
LIST_INSERT_HEAD(&priv->flow_counters, cnt, next);
return cnt;
@@ -2828,6 +2828,97 @@ struct field_modify_info modify_tcp[] = {
}
/**
+ * Find existing tag resource or create and register a new one.
+ *
+ * @param dev[in, out]
+ * Pointer to rte_eth_dev structure.
+ * @param[in, out] resource
+ * Pointer to tag resource.
+ * @parm[in, out] dev_flow
+ * Pointer to the dev_flow.
+ * @param[out] error
+ * pointer to error structure.
+ *
+ * @return
+ * 0 on success otherwise -errno and errno is set.
+ */
+static int
+flow_dv_tag_resource_register
+ (struct rte_eth_dev *dev,
+ struct mlx5_flow_dv_tag_resource *resource,
+ struct mlx5_flow *dev_flow,
+ struct rte_flow_error *error)
+{
+ struct mlx5_priv *priv = dev->data->dev_private;
+ struct mlx5_flow_dv_tag_resource *cache_resource;
+
+ /* Lookup a matching resource from cache. */
+ LIST_FOREACH(cache_resource, &priv->tags, next) {
+ if (resource->tag == cache_resource->tag) {
+ DRV_LOG(DEBUG, "tag resource %p: refcnt %d++",
+ (void *)cache_resource,
+ rte_atomic32_read(&cache_resource->refcnt));
+ rte_atomic32_inc(&cache_resource->refcnt);
+ dev_flow->flow->tag_resource = cache_resource;
+ return 0;
+ }
+ }
+ /* Register new resource. */
+ cache_resource = rte_calloc(__func__, 1, sizeof(*cache_resource), 0);
+ if (!cache_resource)
+ return rte_flow_error_set(error, ENOMEM,
+ RTE_FLOW_ERROR_TYPE_UNSPECIFIED, NULL,
+ "cannot allocate resource memory");
+ *cache_resource = *resource;
+ cache_resource->action = mlx5_glue->dv_create_flow_action_tag
+ (resource->tag);
+ if (!cache_resource->action) {
+ rte_free(cache_resource);
+ return rte_flow_error_set(error, ENOMEM,
+ RTE_FLOW_ERROR_TYPE_UNSPECIFIED,
+ NULL, "cannot create action");
+ }
+ rte_atomic32_init(&cache_resource->refcnt);
+ rte_atomic32_inc(&cache_resource->refcnt);
+ LIST_INSERT_HEAD(&priv->tags, cache_resource, next);
+ dev_flow->flow->tag_resource = cache_resource;
+ DRV_LOG(DEBUG, "new tag resource %p: refcnt %d++",
+ (void *)cache_resource,
+ rte_atomic32_read(&cache_resource->refcnt));
+ return 0;
+}
+
+/**
+ * Release the tag.
+ *
+ * @param dev
+ * Pointer to Ethernet device.
+ * @param flow
+ * Pointer to mlx5_flow.
+ *
+ * @return
+ * 1 while a reference on it exists, 0 when freed.
+ */
+static int
+flow_dv_tag_release(struct rte_eth_dev *dev,
+ struct mlx5_flow_dv_tag_resource *tag)
+{
+ assert(tag);
+ DRV_LOG(DEBUG, "port %u tag %p: refcnt %d--",
+ dev->data->port_id, (void *)tag,
+ rte_atomic32_read(&tag->refcnt));
+ if (rte_atomic32_dec_and_test(&tag->refcnt)) {
+ claim_zero(mlx5_glue->destroy_flow_action(tag->action));
+ LIST_REMOVE(tag, next);
+ DRV_LOG(DEBUG, "port %u tag %p: removed",
+ dev->data->port_id, (void *)tag);
+ rte_free(tag);
+ return 0;
+ }
+ return 1;
+}
+
+/**
* Fill the flow with DV spec.
*
* @param[in] dev
@@ -2872,6 +2963,7 @@ struct field_modify_info modify_tcp[] = {
MLX5DV_FLOW_TABLE_TYPE_NIC_RX
};
union flow_dv_attr flow_attr = { .attr = 0 };
+ struct mlx5_flow_dv_tag_resource tag_resource;
if (priority == MLX5_FLOW_PRIO_RSVD)
priority = priv->config.flow_prio - 1;
@@ -2886,26 +2978,29 @@ struct field_modify_info modify_tcp[] = {
case RTE_FLOW_ACTION_TYPE_VOID:
break;
case RTE_FLOW_ACTION_TYPE_FLAG:
- dev_flow->dv.actions[actions_n].type =
- MLX5DV_FLOW_ACTION_TAG;
- dev_flow->dv.actions[actions_n].tag_value =
+ tag_resource.tag =
mlx5_flow_mark_set(MLX5_FLOW_MARK_DEFAULT);
- actions_n++;
+ if (!flow->tag_resource)
+ if (flow_dv_tag_resource_register
+ (dev, &tag_resource, dev_flow, error))
+ return errno;
+ dev_flow->dv.actions[actions_n++] =
+ flow->tag_resource->action;
action_flags |= MLX5_FLOW_ACTION_FLAG;
break;
case RTE_FLOW_ACTION_TYPE_MARK:
- dev_flow->dv.actions[actions_n].type =
- MLX5DV_FLOW_ACTION_TAG;
- dev_flow->dv.actions[actions_n].tag_value =
- mlx5_flow_mark_set
- (((const struct rte_flow_action_mark *)
- (actions->conf))->id);
- actions_n++;
+ tag_resource.tag = mlx5_flow_mark_set
+ (((const struct rte_flow_action_mark *)
+ (actions->conf))->id);
+ if (!flow->tag_resource)
+ if (flow_dv_tag_resource_register
+ (dev, &tag_resource, dev_flow, error))
+ return errno;
+ dev_flow->dv.actions[actions_n++] =
+ flow->tag_resource->action;
action_flags |= MLX5_FLOW_ACTION_MARK;
break;
case RTE_FLOW_ACTION_TYPE_DROP:
- dev_flow->dv.actions[actions_n].type =
- MLX5DV_FLOW_ACTION_DROP;
action_flags |= MLX5_FLOW_ACTION_DROP;
break;
case RTE_FLOW_ACTION_TYPE_QUEUE:
@@ -2933,17 +3028,13 @@ struct field_modify_info modify_tcp[] = {
rte_errno = ENOTSUP;
goto cnt_err;
}
- flow->counter =
- flow_dv_counter_new(dev,
- count->shared, count->id);
+ flow->counter = flow_dv_counter_new(dev, count->shared,
+ count->id);
if (flow->counter == NULL)
goto cnt_err;
- dev_flow->dv.actions[actions_n].type =
- MLX5DV_FLOW_ACTION_COUNTER_DEVX;
- dev_flow->dv.actions[actions_n].obj =
- flow->counter->dcs->obj;
+ dev_flow->dv.actions[actions_n++] =
+ flow->counter->action;
action_flags |= MLX5_FLOW_ACTION_COUNT;
- ++actions_n;
break;
cnt_err:
if (rte_errno == ENOTSUP)
@@ -2964,11 +3055,8 @@ struct field_modify_info modify_tcp[] = {
if (flow_dv_create_action_l2_encap(dev, actions,
dev_flow, error))
return -rte_errno;
- dev_flow->dv.actions[actions_n].type =
- MLX5DV_FLOW_ACTION_IBV_FLOW_ACTION;
- dev_flow->dv.actions[actions_n].action =
+ dev_flow->dv.actions[actions_n++] =
dev_flow->dv.encap_decap->verbs_action;
- actions_n++;
action_flags |= actions->type ==
RTE_FLOW_ACTION_TYPE_VXLAN_ENCAP ?
MLX5_FLOW_ACTION_VXLAN_ENCAP :
@@ -2979,11 +3067,8 @@ struct field_modify_info modify_tcp[] = {
if (flow_dv_create_action_l2_decap(dev, dev_flow,
error))
return -rte_errno;
- dev_flow->dv.actions[actions_n].type =
- MLX5DV_FLOW_ACTION_IBV_FLOW_ACTION;
- dev_flow->dv.actions[actions_n].action =
+ dev_flow->dv.actions[actions_n++] =
dev_flow->dv.encap_decap->verbs_action;
- actions_n++;
action_flags |= actions->type ==
RTE_FLOW_ACTION_TYPE_VXLAN_DECAP ?
MLX5_FLOW_ACTION_VXLAN_DECAP :
@@ -2995,9 +3080,7 @@ struct field_modify_info modify_tcp[] = {
if (flow_dv_create_action_raw_encap
(dev, actions, dev_flow, attr, error))
return -rte_errno;
- dev_flow->dv.actions[actions_n].type =
- MLX5DV_FLOW_ACTION_IBV_FLOW_ACTION;
- dev_flow->dv.actions[actions_n].action =
+ dev_flow->dv.actions[actions_n++] =
dev_flow->dv.encap_decap->verbs_action;
} else {
/* Handle encap without preceding decap. */
@@ -3005,12 +3088,9 @@ struct field_modify_info modify_tcp[] = {
dev_flow,
error))
return -rte_errno;
- dev_flow->dv.actions[actions_n].type =
- MLX5DV_FLOW_ACTION_IBV_FLOW_ACTION;
- dev_flow->dv.actions[actions_n].action =
+ dev_flow->dv.actions[actions_n++] =
dev_flow->dv.encap_decap->verbs_action;
}
- actions_n++;
action_flags |= MLX5_FLOW_ACTION_RAW_ENCAP;
break;
case RTE_FLOW_ACTION_TYPE_RAW_DECAP:
@@ -3025,11 +3105,8 @@ struct field_modify_info modify_tcp[] = {
dev_flow,
error))
return -rte_errno;
- dev_flow->dv.actions[actions_n].type =
- MLX5DV_FLOW_ACTION_IBV_FLOW_ACTION;
- dev_flow->dv.actions[actions_n].action =
+ dev_flow->dv.actions[actions_n++] =
dev_flow->dv.encap_decap->verbs_action;
- actions_n++;
}
/* If decap is followed by encap, handle it at encap. */
action_flags |= MLX5_FLOW_ACTION_RAW_DECAP;
@@ -3098,11 +3175,8 @@ struct field_modify_info modify_tcp[] = {
dev_flow,
error))
return -rte_errno;
- dev_flow->dv.actions[actions_n].type =
- MLX5DV_FLOW_ACTION_IBV_FLOW_ACTION;
- dev_flow->dv.actions[actions_n].action =
+ dev_flow->dv.actions[actions_n++] =
dev_flow->dv.modify_hdr->verbs_action;
- actions_n++;
}
break;
default:
@@ -3277,9 +3351,9 @@ struct field_modify_info modify_tcp[] = {
"cannot get drop hash queue");
goto error;
}
- dv->actions[n].type = MLX5DV_FLOW_ACTION_DEST_IBV_QP;
- dv->actions[n].qp = dv->hrxq->qp;
- n++;
+ dv->actions[n++] =
+ mlx5_glue->dv_create_flow_action_dest_ibv_qp
+ (dv->hrxq->qp);
} else if (flow->actions &
(MLX5_FLOW_ACTION_QUEUE | MLX5_FLOW_ACTION_RSS)) {
struct mlx5_hrxq *hrxq;
@@ -3304,9 +3378,9 @@ struct field_modify_info modify_tcp[] = {
goto error;
}
dv->hrxq = hrxq;
- dv->actions[n].type = MLX5DV_FLOW_ACTION_DEST_IBV_QP;
- dv->actions[n].qp = hrxq->qp;
- n++;
+ dv->actions[n++] =
+ mlx5_glue->dv_create_flow_action_dest_ibv_qp
+ (dv->hrxq->qp);
}
dv->flow =
mlx5_glue->dv_create_flow(dv->matcher->matcher_object,
@@ -3484,6 +3558,10 @@ struct field_modify_info modify_tcp[] = {
flow_dv_counter_release(flow->counter);
flow->counter = NULL;
}
+ if (flow->tag_resource) {
+ flow_dv_tag_release(dev, flow->tag_resource);
+ flow->tag_resource = NULL;
+ }
while (!LIST_EMPTY(&flow->dev_flows)) {
dev_flow = LIST_FIRST(&flow->dev_flows);
LIST_REMOVE(dev_flow, next);
diff --git a/drivers/net/mlx5/mlx5_glue.c b/drivers/net/mlx5/mlx5_glue.c
index c817d86..cfd939d 100644
--- a/drivers/net/mlx5/mlx5_glue.c
+++ b/drivers/net/mlx5/mlx5_glue.c
@@ -175,10 +175,20 @@
}
static int
-mlx5_glue_destroy_flow_action(struct ibv_flow_action *action)
+mlx5_glue_destroy_flow_action(void *action)
{
#ifdef HAVE_IBV_FLOW_DV_SUPPORT
- return ibv_destroy_flow_action(action);
+ struct mlx5dv_flow_action_attr *attr = action;
+ int res = 0;
+ switch (attr->type) {
+ case MLX5DV_FLOW_ACTION_TAG:
+ break;
+ default:
+ res = ibv_destroy_flow_action(attr->action);
+ break;
+ }
+ free(action);
+ return res;
#else
(void)action;
return ENOTSUP;
@@ -430,16 +440,23 @@
mlx5_glue_dv_create_flow(struct mlx5dv_flow_matcher *matcher,
struct mlx5dv_flow_match_parameters *match_value,
size_t num_actions,
- struct mlx5dv_flow_action_attr *actions_attr)
+ void *actions[])
{
#ifdef HAVE_IBV_FLOW_DV_SUPPORT
+ struct mlx5dv_flow_action_attr actions_attr[8];
+
+ if (num_actions > 8)
+ return NULL;
+ for (size_t i = 0; i < num_actions; i++)
+ actions_attr[i] =
+ *((struct mlx5dv_flow_action_attr *)(actions[i]));
return mlx5dv_create_flow(matcher, match_value,
num_actions, actions_attr);
#else
(void)matcher;
(void)match_value;
(void)num_actions;
- (void)actions_attr;
+ (void)actions;
return NULL;
#endif
}
@@ -455,31 +472,45 @@
#endif
}
-static struct ibv_flow_action *
-mlx5_glue_dv_create_flow_action_packet_reformat
- (struct ibv_context *ctx,
- size_t data_sz,
- void *data,
- enum mlx5dv_flow_action_packet_reformat_type reformat_type,
- enum mlx5dv_flow_table_type ft_type)
+static void *
+mlx5_glue_dv_create_flow_action_counter(void *counter_obj, uint32_t offset)
{
#ifdef HAVE_IBV_FLOW_DV_SUPPORT
- return mlx5dv_create_flow_action_packet_reformat(ctx,
- data_sz,
- data,
- reformat_type,
- ft_type);
+ struct mlx5dv_flow_action_attr *action;
+
+ (void)offset;
+ action = malloc(sizeof(*action));
+ if (!action)
+ return NULL;
+ action->type = MLX5DV_FLOW_ACTION_COUNTER_DEVX;
+ action->obj = counter_obj;
+ return action;
#else
- (void)ctx;
- (void)data_sz;
- (void)data;
- (void)reformat_type;
- (void)ft_type;
+ (void)counter_obj;
+ (void)offset;
return NULL;
#endif
}
-static struct ibv_flow_action *
+static void *
+mlx5_glue_dv_create_flow_action_dest_ibv_qp(void *qp)
+{
+#ifdef HAVE_IBV_FLOW_DV_SUPPORT
+ struct mlx5dv_flow_action_attr *action;
+
+ action = malloc(sizeof(*action));
+ if (!action)
+ return NULL;
+ action->type = MLX5DV_FLOW_ACTION_DEST_IBV_QP;
+ action->obj = qp;
+ return action;
+#else
+ (void)qp;
+ return NULL;
+#endif
+}
+
+static void *
mlx5_glue_dv_create_flow_action_modify_header
(struct ibv_context *ctx,
size_t actions_sz,
@@ -487,8 +518,15 @@
enum mlx5dv_flow_table_type ft_type)
{
#ifdef HAVE_IBV_FLOW_DV_SUPPORT
- return mlx5dv_create_flow_action_modify_header(ctx, actions_sz,
- actions, ft_type);
+ struct mlx5dv_flow_action_attr *action;
+
+ action = malloc(sizeof(*action));
+ if (!action)
+ return NULL;
+ action->type = MLX5DV_FLOW_ACTION_IBV_FLOW_ACTION;
+ action->action = mlx5dv_create_flow_action_modify_header
+ (ctx, actions_sz, actions, ft_type);
+ return action;
#else
(void)ctx;
(void)actions_sz;
@@ -498,6 +536,50 @@
#endif
}
+static void *
+mlx5_glue_dv_create_flow_action_packet_reformat
+ (struct ibv_context *ctx,
+ size_t data_sz,
+ void *data,
+ enum mlx5dv_flow_action_packet_reformat_type reformat_type,
+ enum mlx5dv_flow_table_type ft_type)
+{
+#ifdef HAVE_IBV_FLOW_DV_SUPPORT
+ struct mlx5dv_flow_action_attr *action;
+
+ action = malloc(sizeof(*action));
+ if (!action)
+ return NULL;
+ action->type = MLX5DV_FLOW_ACTION_IBV_FLOW_ACTION;
+ action->action = mlx5dv_create_flow_action_packet_reformat
+ (ctx, data_sz, data, reformat_type, ft_type);
+ return action;
+#else
+ (void)ctx;
+ (void)data_sz;
+ (void)data;
+ (void)reformat_type;
+ (void)ft_type;
+ return NULL;
+#endif
+}
+
+static void *
+mlx5_glue_dv_create_flow_action_tag(uint32_t tag)
+{
+#ifdef HAVE_IBV_FLOW_DV_SUPPORT
+ struct mlx5dv_flow_action_attr *action;
+ action = malloc(sizeof(*action));
+ if (!action)
+ return NULL;
+ action->type = MLX5DV_FLOW_ACTION_TAG;
+ action->tag_value = tag;
+ return action;
+#endif
+ (void)tag;
+ return NULL;
+}
+
static struct ibv_context *
mlx5_glue_dv_open_device(struct ibv_device *device)
{
@@ -645,10 +727,15 @@
.dv_create_flow_matcher = mlx5_glue_dv_create_flow_matcher,
.dv_destroy_flow_matcher = mlx5_glue_dv_destroy_flow_matcher,
.dv_create_flow = mlx5_glue_dv_create_flow,
- .dv_create_flow_action_packet_reformat =
- mlx5_glue_dv_create_flow_action_packet_reformat,
+ .dv_create_flow_action_counter =
+ mlx5_glue_dv_create_flow_action_counter,
+ .dv_create_flow_action_dest_ibv_qp =
+ mlx5_glue_dv_create_flow_action_dest_ibv_qp,
.dv_create_flow_action_modify_header =
- mlx5_glue_dv_create_flow_action_modify_header,
+ mlx5_glue_dv_create_flow_action_modify_header,
+ .dv_create_flow_action_packet_reformat =
+ mlx5_glue_dv_create_flow_action_packet_reformat,
+ .dv_create_flow_action_tag = mlx5_glue_dv_create_flow_action_tag,
.dv_open_device = mlx5_glue_dv_open_device,
.devx_obj_create = mlx5_glue_devx_obj_create,
.devx_obj_destroy = mlx5_glue_devx_obj_destroy,
diff --git a/drivers/net/mlx5/mlx5_glue.h b/drivers/net/mlx5/mlx5_glue.h
index b118960..167f1f7 100644
--- a/drivers/net/mlx5/mlx5_glue.h
+++ b/drivers/net/mlx5/mlx5_glue.h
@@ -55,6 +55,10 @@
enum mlx5dv_flow_table_type { flow_table_type = 0, };
#endif
+#ifndef HAVE_IBV_FLOW_DEVX_COUNTERS
+#define MLX5DV_FLOW_ACTION_COUNTER_DEVX 0
+#endif
+
#ifndef HAVE_IBV_DEVX_OBJ
struct mlx5dv_devx_obj;
#endif
@@ -98,7 +102,7 @@ struct mlx5_glue {
struct ibv_flow *(*create_flow)(struct ibv_qp *qp,
struct ibv_flow_attr *flow);
int (*destroy_flow)(struct ibv_flow *flow_id);
- int (*destroy_flow_action)(struct ibv_flow_action *action);
+ int (*destroy_flow_action)(void *action);
struct ibv_qp *(*create_qp)(struct ibv_pd *pd,
struct ibv_qp_init_attr *qp_init_attr);
struct ibv_qp *(*create_qp_ex)
@@ -160,19 +164,17 @@ struct mlx5_glue {
int (*dv_destroy_flow_matcher)(struct mlx5dv_flow_matcher *matcher);
struct ibv_flow *(*dv_create_flow)(struct mlx5dv_flow_matcher *matcher,
struct mlx5dv_flow_match_parameters *match_value,
- size_t num_actions,
- struct mlx5dv_flow_action_attr *actions_attr);
- struct ibv_flow_action *(*dv_create_flow_action_packet_reformat)
- (struct ibv_context *ctx,
- size_t data_sz,
- void *data,
+ size_t num_actions, void *actions[]);
+ void *(*dv_create_flow_action_counter)(void *obj, uint32_t offset);
+ void *(*dv_create_flow_action_dest_ibv_qp)(void *qp);
+ void *(*dv_create_flow_action_modify_header)
+ (struct ibv_context *ctx, size_t actions_sz, uint64_t actions[],
+ enum mlx5dv_flow_table_type ft_type);
+ void *(*dv_create_flow_action_packet_reformat)
+ (struct ibv_context *ctx, size_t data_sz, void *data,
enum mlx5dv_flow_action_packet_reformat_type reformat_type,
enum mlx5dv_flow_table_type ft_type);
- struct ibv_flow_action *(*dv_create_flow_action_modify_header)
- (struct ibv_context *ctx,
- size_t actions_sz,
- uint64_t actions[],
- enum mlx5dv_flow_table_type ft_type);
+ void *(*dv_create_flow_action_tag)(uint32_t tag);
struct ibv_context *(*dv_open_device)(struct ibv_device *device);
struct mlx5dv_devx_obj *(*devx_obj_create)
(struct ibv_context *ctx,
--
1.8.3.1
^ permalink raw reply [flat|nested] 52+ messages in thread
* [dpdk-dev] [PATCH v2 2/3] net/mlx5: add Direct Rules API
2019-03-28 16:32 ` [dpdk-dev] [PATCH v2 0/3] net/mlx5: Add Direct Rule support Ori Kam
2019-03-28 16:32 ` Ori Kam
2019-03-28 16:32 ` [dpdk-dev] [PATCH v2 1/3] net/mlx5: prepare Direct Verbs for Direct Rule Ori Kam
@ 2019-03-28 16:32 ` Ori Kam
2019-03-28 16:32 ` Ori Kam
2019-04-01 14:38 ` Slava Ovsiienko
2019-03-28 16:32 ` [dpdk-dev] [PATCH v2 3/3] net/mlx5: add jump action support for NIC Ori Kam
` (3 subsequent siblings)
6 siblings, 2 replies; 52+ messages in thread
From: Ori Kam @ 2019-03-28 16:32 UTC (permalink / raw)
To: Matan Azrad, Yongseok Koh, Shahaf Shuler; +Cc: dev, Ori Kam, Slava Ovsiienko
Adds calls to the Direct Rules API inside the glue functions.
Due to difference in parameters between the Direct Rules and Direct
Verbs some of the glue functions API was updated.
Signed-off-by: Ori Kam <orika@mellanox.com>
---
drivers/net/mlx5/Makefile | 5 ++
drivers/net/mlx5/mlx5.c | 16 ++++
drivers/net/mlx5/mlx5.h | 15 ++++
drivers/net/mlx5/mlx5_flow.c | 1 +
drivers/net/mlx5/mlx5_flow.h | 6 +-
drivers/net/mlx5/mlx5_flow_dv.c | 103 ++++++++++++++++++++----
drivers/net/mlx5/mlx5_glue.c | 170 ++++++++++++++++++++++++++++++++++------
drivers/net/mlx5/mlx5_glue.h | 31 +++++---
drivers/net/mlx5/mlx5_prm.h | 24 +++++-
9 files changed, 318 insertions(+), 53 deletions(-)
diff --git a/drivers/net/mlx5/Makefile b/drivers/net/mlx5/Makefile
index c326494..3dd7e38 100644
--- a/drivers/net/mlx5/Makefile
+++ b/drivers/net/mlx5/Makefile
@@ -156,6 +156,11 @@ mlx5_autoconf.h.new: $(RTE_SDK)/buildtools/auto-config-h.sh
func mlx5dv_create_flow_action_packet_reformat \
$(AUTOCONF_OUTPUT)
$Q sh -- '$<' '$@' \
+ HAVE_MLX5DV_DR \
+ infiniband/mlx5dv.h \
+ enum MLX5DV_DR_NS_TYPE_TERMINATING \
+ $(AUTOCONF_OUTPUT)
+ $Q sh -- '$<' '$@' \
HAVE_IBV_DEVX_OBJ \
infiniband/mlx5dv.h \
func mlx5dv_devx_obj_create \
diff --git a/drivers/net/mlx5/mlx5.c b/drivers/net/mlx5/mlx5.c
index 7d1a21e..70e37b5 100644
--- a/drivers/net/mlx5/mlx5.c
+++ b/drivers/net/mlx5/mlx5.c
@@ -1362,6 +1362,22 @@ struct mlx5_dev_spawn_data {
priv->tcf_context = NULL;
}
}
+#ifdef HAVE_MLX5DV_DR
+ priv->rx_ns = mlx5dv_dr_create_ns
+ (sh->ctx, MLX5DV_DR_NS_DOMAIN_INGRESS_BYPASS);
+ if (priv->rx_ns == NULL) {
+ DRV_LOG(ERR, "mlx5dv_dr_create_ns failed");
+ err = errno;
+ goto error;
+ }
+ priv->tx_ns = mlx5dv_dr_create_ns(sh->ctx,
+ MLX5DV_DR_NS_DOMAIN_EGRESS_BYPASS);
+ if (priv->tx_ns == NULL) {
+ DRV_LOG(ERR, "mlx5dv_dr_create_ns failed");
+ err = errno;
+ goto error;
+ }
+#endif
TAILQ_INIT(&priv->flows);
TAILQ_INIT(&priv->ctrl_flows);
/* Hint libmlx5 to use PMD allocator for data plane resources */
diff --git a/drivers/net/mlx5/mlx5.h b/drivers/net/mlx5/mlx5.h
index b587359..73f6f0d 100644
--- a/drivers/net/mlx5/mlx5.h
+++ b/drivers/net/mlx5/mlx5.h
@@ -219,6 +219,15 @@ struct mlx5_ibv_shared {
struct mlx5_ibv_shared_port port[]; /* per device port data array. */
};
+/* Table structure. */
+struct mlx5_flow_tbl_resource {
+ void *obj; /**< Pointer to DR table object. */
+ rte_atomic32_t refcnt; /**< Reference counter. */
+};
+
+#define MLX5_MAX_TABLES 1024
+#define MLX5_GROUP_FACTOR 1
+
struct mlx5_priv {
LIST_ENTRY(mlx5_priv) mem_event_cb;
/**< Called by memory event callback. */
@@ -289,6 +298,12 @@ struct mlx5_priv {
/* UAR same-page access control required in 32bit implementations. */
#endif
struct mlx5_flow_tcf_context *tcf_context; /* TC flower context. */
+ void *rx_ns; /* RX Direct Rules name space handle. */
+ struct mlx5_flow_tbl_resource rx_tbl[MLX5_MAX_TABLES];
+ /* RX Direct Rules tables. */
+ void *tx_ns; /* TX Direct Rules name space handle. */
+ struct mlx5_flow_tbl_resource tx_tbl[MLX5_MAX_TABLES];
+ /* TX Direct Rules tables/ */
};
#define PORT_ID(priv) ((priv)->dev_data->port_id)
diff --git a/drivers/net/mlx5/mlx5_flow.c b/drivers/net/mlx5/mlx5_flow.c
index d09fdff..40d6818 100644
--- a/drivers/net/mlx5/mlx5_flow.c
+++ b/drivers/net/mlx5/mlx5_flow.c
@@ -2084,6 +2084,7 @@ uint32_t mlx5_flow_adjust_priority(struct rte_eth_dev *dev, int32_t priority,
flow_size += RTE_ALIGN_CEIL(sizeof(uint16_t), sizeof(void *));
flow = rte_calloc(__func__, 1, flow_size, 0);
flow->drv_type = flow_get_drv_type(dev, attr);
+ flow->ingress = attr->ingress;
assert(flow->drv_type > MLX5_FLOW_TYPE_MIN &&
flow->drv_type < MLX5_FLOW_TYPE_MAX);
flow->queue = (void *)(flow + 1);
diff --git a/drivers/net/mlx5/mlx5_flow.h b/drivers/net/mlx5/mlx5_flow.h
index 4f69ae2..8ba37a0 100644
--- a/drivers/net/mlx5/mlx5_flow.h
+++ b/drivers/net/mlx5/mlx5_flow.h
@@ -204,6 +204,7 @@ struct mlx5_flow_dv_matcher {
uint16_t crc; /**< CRC of key. */
uint16_t priority; /**< Priority of matcher. */
uint8_t egress; /**< Egress matcher. */
+ uint32_t group; /**< The matcher group. */
struct mlx5_flow_dv_match_params mask; /**< Matcher mask. */
};
@@ -220,6 +221,7 @@ struct mlx5_flow_dv_encap_decap_resource {
size_t size;
uint8_t reformat_type;
uint8_t ft_type;
+ uint64_t flags; /**< Flags for RDMA API. */
};
/* Tag resource structure. */
@@ -348,7 +350,7 @@ struct mlx5_flow_counter {
/* Flow structure. */
struct rte_flow {
TAILQ_ENTRY(rte_flow) next; /**< Pointer to the next flow structure. */
- enum mlx5_flow_drv_type drv_type; /**< Drvier type. */
+ enum mlx5_flow_drv_type drv_type; /**< Driver type. */
struct mlx5_flow_counter *counter; /**< Holds flow counter. */
struct mlx5_flow_dv_tag_resource *tag_resource;
/**< pointer to the tag action. */
@@ -360,6 +362,8 @@ struct rte_flow {
uint64_t actions;
/**< Bit-fields of detected actions, see MLX5_FLOW_ACTION_*. */
struct mlx5_fdir *fdir; /**< Pointer to associated FDIR if any. */
+ uint8_t ingress; /**< 1 if the flow is ingress. */
+ uint32_t group; /**< The group index. */
};
typedef int (*mlx5_flow_validate_t)(struct rte_eth_dev *dev,
diff --git a/drivers/net/mlx5/mlx5_flow_dv.c b/drivers/net/mlx5/mlx5_flow_dv.c
index b0c50a7..6e4f6c4 100644
--- a/drivers/net/mlx5/mlx5_flow_dv.c
+++ b/drivers/net/mlx5/mlx5_flow_dv.c
@@ -805,11 +805,20 @@ struct field_modify_info modify_tcp[] = {
{
struct mlx5_priv *priv = dev->data->dev_private;
struct mlx5_flow_dv_encap_decap_resource *cache_resource;
+ struct rte_flow *flow = dev_flow->flow;
+ struct mlx5dv_dr_ns *ns;
+
+ resource->flags = flow->group ? 0 : 1;
+ if (flow->ingress)
+ ns = priv->rx_ns;
+ else
+ ns = priv->tx_ns;
/* Lookup a matching resource from cache. */
LIST_FOREACH(cache_resource, &priv->encaps_decaps, next) {
if (resource->reformat_type == cache_resource->reformat_type &&
resource->ft_type == cache_resource->ft_type &&
+ resource->flags == cache_resource->flags &&
resource->size == cache_resource->size &&
!memcmp((const void *)resource->buf,
(const void *)cache_resource->buf,
@@ -831,10 +840,10 @@ struct field_modify_info modify_tcp[] = {
*cache_resource = *resource;
cache_resource->verbs_action =
mlx5_glue->dv_create_flow_action_packet_reformat
- (priv->sh->ctx, cache_resource->size,
- (cache_resource->size ? cache_resource->buf : NULL),
- cache_resource->reformat_type,
- cache_resource->ft_type);
+ (priv->sh->ctx, cache_resource->reformat_type,
+ cache_resource->ft_type, ns, cache_resource->flags,
+ cache_resource->size,
+ (cache_resource->size ? cache_resource->buf : NULL));
if (!cache_resource->verbs_action) {
rte_free(cache_resource);
return rte_flow_error_set(error, ENOMEM,
@@ -1438,6 +1447,10 @@ struct field_modify_info modify_tcp[] = {
struct mlx5_priv *priv = dev->data->dev_private;
struct mlx5_flow_dv_modify_hdr_resource *cache_resource;
+ struct mlx5dv_dr_ns *ns =
+ resource->ft_type == MLX5DV_FLOW_TABLE_TYPE_NIC_TX ?
+ priv->tx_ns : priv->rx_ns;
+
/* Lookup a matching resource from cache. */
LIST_FOREACH(cache_resource, &priv->modify_cmds, next) {
if (resource->ft_type == cache_resource->ft_type &&
@@ -1463,11 +1476,11 @@ struct field_modify_info modify_tcp[] = {
*cache_resource = *resource;
cache_resource->verbs_action =
mlx5_glue->dv_create_flow_action_modify_header
- (priv->sh->ctx,
+ (priv->sh->ctx, cache_resource->ft_type,
+ ns, 0,
cache_resource->actions_num *
sizeof(cache_resource->actions[0]),
- (uint64_t *)cache_resource->actions,
- cache_resource->ft_type);
+ (uint64_t *)cache_resource->actions);
if (!cache_resource->verbs_action) {
rte_free(cache_resource);
return rte_flow_error_set(error, ENOMEM,
@@ -1592,11 +1605,13 @@ struct field_modify_info modify_tcp[] = {
struct mlx5_priv *priv = dev->data->dev_private;
uint32_t priority_max = priv->config.flow_prio - 1;
+#ifdef HAVE_MLX5DV_DR
if (attributes->group)
return rte_flow_error_set(error, ENOTSUP,
RTE_FLOW_ERROR_TYPE_ATTR_GROUP,
NULL,
"groups is not supported");
+#endif
if (attributes->priority != MLX5_FLOW_PRIO_RSVD &&
attributes->priority >= priority_max)
return rte_flow_error_set(error, ENOTSUP,
@@ -2169,11 +2184,13 @@ struct field_modify_info modify_tcp[] = {
* Flow pattern to translate.
* @param[in] inner
* Item is inner pattern.
+ * @param[in] group
+ * The group to insert the rule.
*/
static void
flow_dv_translate_item_ipv4(void *matcher, void *key,
const struct rte_flow_item *item,
- int inner)
+ int inner, uint32_t group)
{
const struct rte_flow_item_ipv4 *ipv4_m = item->mask;
const struct rte_flow_item_ipv4 *ipv4_v = item->spec;
@@ -2200,7 +2217,10 @@ struct field_modify_info modify_tcp[] = {
outer_headers);
headers_v = MLX5_ADDR_OF(fte_match_param, key, outer_headers);
}
- MLX5_SET(fte_match_set_lyr_2_4, headers_m, ip_version, 0xf);
+ if (group == 0)
+ MLX5_SET(fte_match_set_lyr_2_4, headers_m, ip_version, 0xf);
+ else
+ MLX5_SET(fte_match_set_lyr_2_4, headers_m, ip_version, 0x4);
MLX5_SET(fte_match_set_lyr_2_4, headers_v, ip_version, 4);
if (!ipv4_v)
return;
@@ -2242,11 +2262,13 @@ struct field_modify_info modify_tcp[] = {
* Flow pattern to translate.
* @param[in] inner
* Item is inner pattern.
+ * @param[in] group
+ * The group to insert the rule.
*/
static void
flow_dv_translate_item_ipv6(void *matcher, void *key,
const struct rte_flow_item *item,
- int inner)
+ int inner, uint32_t group)
{
const struct rte_flow_item_ipv6 *ipv6_m = item->mask;
const struct rte_flow_item_ipv6 *ipv6_v = item->spec;
@@ -2283,7 +2305,10 @@ struct field_modify_info modify_tcp[] = {
outer_headers);
headers_v = MLX5_ADDR_OF(fte_match_param, key, outer_headers);
}
- MLX5_SET(fte_match_set_lyr_2_4, headers_m, ip_version, 0xf);
+ if (group == 0)
+ MLX5_SET(fte_match_set_lyr_2_4, headers_m, ip_version, 0xf);
+ else
+ MLX5_SET(fte_match_set_lyr_2_4, headers_m, ip_version, 0x6);
MLX5_SET(fte_match_set_lyr_2_4, headers_v, ip_version, 6);
if (!ipv6_v)
return;
@@ -2723,7 +2748,11 @@ struct field_modify_info modify_tcp[] = {
match_criteria_enable |=
(!HEADER_IS_ZERO(match_criteria, misc_parameters_2)) <<
MLX5_MATCH_CRITERIA_ENABLE_MISC2_BIT;
-
+#ifdef HAVE_MLX5DV_DR
+ match_criteria_enable |=
+ (!HEADER_IS_ZERO(match_criteria, misc_parameters_3)) <<
+ MLX5_MATCH_CRITERIA_ENABLE_MISC3_BIT;
+#endif
return match_criteria_enable;
}
@@ -2754,12 +2783,14 @@ struct field_modify_info modify_tcp[] = {
.type = IBV_FLOW_ATTR_NORMAL,
.match_mask = (void *)&matcher->mask,
};
+ struct mlx5_flow_tbl_resource *tbl = NULL;
/* Lookup from cache. */
LIST_FOREACH(cache_matcher, &priv->matchers, next) {
if (matcher->crc == cache_matcher->crc &&
matcher->priority == cache_matcher->priority &&
matcher->egress == cache_matcher->egress &&
+ matcher->group == cache_matcher->group &&
!memcmp((const void *)matcher->mask.buf,
(const void *)cache_matcher->mask.buf,
cache_matcher->mask.size)) {
@@ -2774,6 +2805,27 @@ struct field_modify_info modify_tcp[] = {
return 0;
}
}
+#ifdef HAVE_MLX5DV_DR
+ if (matcher->egress) {
+ tbl = &priv->tx_tbl[matcher->group];
+ if (!tbl->obj)
+ tbl->obj = mlx5_glue->dr_create_flow_tbl
+ (priv->tx_ns,
+ matcher->group * MLX5_GROUP_FACTOR);
+ } else {
+ tbl = &priv->rx_tbl[matcher->group];
+ if (!tbl->obj)
+ tbl->obj = mlx5_glue->dr_create_flow_tbl
+ (priv->rx_ns,
+ matcher->group * MLX5_GROUP_FACTOR);
+ }
+ if (!tbl->obj)
+ return rte_flow_error_set(error, ENOMEM,
+ RTE_FLOW_ERROR_TYPE_UNSPECIFIED,
+ NULL, "cannot create table");
+
+ rte_atomic32_inc(&tbl->refcnt);
+#endif
/* Register new matcher. */
cache_matcher = rte_calloc(__func__, 1, sizeof(*cache_matcher), 0);
if (!cache_matcher)
@@ -2787,9 +2839,16 @@ struct field_modify_info modify_tcp[] = {
if (matcher->egress)
dv_attr.flags |= IBV_FLOW_ATTR_FLAGS_EGRESS;
cache_matcher->matcher_object =
- mlx5_glue->dv_create_flow_matcher(priv->sh->ctx, &dv_attr);
+ mlx5_glue->dv_create_flow_matcher(priv->sh->ctx, &dv_attr,
+ tbl->obj);
if (!cache_matcher->matcher_object) {
rte_free(cache_matcher);
+#ifdef HAVE_MLX5DV_DR
+ if (rte_atomic32_dec_and_test(&tbl->refcnt)) {
+ mlx5_glue->dr_destroy_flow_tbl(tbl->obj);
+ tbl->obj = NULL;
+ }
+#endif
return rte_flow_error_set(error, ENOMEM,
RTE_FLOW_ERROR_TYPE_UNSPECIFIED,
NULL, "cannot create matcher");
@@ -2801,6 +2860,7 @@ struct field_modify_info modify_tcp[] = {
cache_matcher->priority,
cache_matcher->egress ? "tx" : "rx", (void *)cache_matcher,
rte_atomic32_read(&cache_matcher->refcnt));
+ rte_atomic32_inc(&tbl->refcnt);
return 0;
}
@@ -3222,7 +3282,7 @@ struct field_modify_info modify_tcp[] = {
break;
case RTE_FLOW_ITEM_TYPE_IPV4:
flow_dv_translate_item_ipv4(match_mask, match_value,
- items, tunnel);
+ items, tunnel, attr->group);
matcher.priority = MLX5_PRIORITY_MAP_L3;
dev_flow->dv.hash_fields |=
mlx5_flow_hashfields_adjust
@@ -3234,7 +3294,7 @@ struct field_modify_info modify_tcp[] = {
break;
case RTE_FLOW_ITEM_TYPE_IPV6:
flow_dv_translate_item_ipv6(match_mask, match_value,
- items, tunnel);
+ items, tunnel, attr->group);
matcher.priority = MLX5_PRIORITY_MAP_L3;
dev_flow->dv.hash_fields |=
mlx5_flow_hashfields_adjust
@@ -3312,6 +3372,7 @@ struct field_modify_info modify_tcp[] = {
matcher.priority = mlx5_flow_adjust_priority(dev, priority,
matcher.priority);
matcher.egress = attr->egress;
+ matcher.group = attr->group;
if (flow_dv_matcher_register(dev, &matcher, dev_flow, error))
return -rte_errno;
return 0;
@@ -3427,6 +3488,8 @@ struct field_modify_info modify_tcp[] = {
struct mlx5_flow *flow)
{
struct mlx5_flow_dv_matcher *matcher = flow->dv.matcher;
+ struct mlx5_priv *priv = dev->data->dev_private;
+ struct mlx5_flow_tbl_resource *tbl;
assert(matcher->matcher_object);
DRV_LOG(DEBUG, "port %u matcher %p: refcnt %d--",
@@ -3436,6 +3499,14 @@ struct field_modify_info modify_tcp[] = {
claim_zero(mlx5_glue->dv_destroy_flow_matcher
(matcher->matcher_object));
LIST_REMOVE(matcher, next);
+ if (matcher->egress)
+ tbl = &priv->tx_tbl[matcher->group];
+ else
+ tbl = &priv->rx_tbl[matcher->group];
+ if (rte_atomic32_dec_and_test(&tbl->refcnt)) {
+ mlx5_glue->dr_destroy_flow_tbl(tbl->obj);
+ tbl->obj = NULL;
+ }
rte_free(matcher);
DRV_LOG(DEBUG, "port %u matcher %p: removed",
dev->data->port_id, (void *)matcher);
@@ -3525,7 +3596,7 @@ struct field_modify_info modify_tcp[] = {
LIST_FOREACH(dev_flow, &flow->dev_flows, next) {
dv = &dev_flow->dv;
if (dv->flow) {
- claim_zero(mlx5_glue->destroy_flow(dv->flow));
+ claim_zero(mlx5_glue->dv_destroy_flow(dv->flow));
dv->flow = NULL;
}
if (dv->hrxq) {
diff --git a/drivers/net/mlx5/mlx5_glue.c b/drivers/net/mlx5/mlx5_glue.c
index cfd939d..f509f85 100644
--- a/drivers/net/mlx5/mlx5_glue.c
+++ b/drivers/net/mlx5/mlx5_glue.c
@@ -178,6 +178,9 @@
mlx5_glue_destroy_flow_action(void *action)
{
#ifdef HAVE_IBV_FLOW_DV_SUPPORT
+#ifdef HAVE_MLX5DV_DR
+ return mlx5dv_dr_destroy_action(action);
+#else
struct mlx5dv_flow_action_attr *attr = action;
int res = 0;
switch (attr->type) {
@@ -189,6 +192,7 @@
}
free(action);
return res;
+#endif
#else
(void)action;
return ENOTSUP;
@@ -365,6 +369,53 @@
return ibv_cq_ex_to_cq(cq);
}
+static void *
+mlx5_glue_dr_create_flow_tbl(void *ns, uint32_t level)
+{
+#ifdef HAVE_MLX5DV_DR
+ return mlx5dv_dr_create_ft(ns, level);
+#else
+ (void)ns;
+ (void)level;
+ return NULL;
+#endif
+}
+
+static int
+mlx5_glue_dr_destroy_flow_tbl(void *tbl)
+{
+#ifdef HAVE_MLX5DV_DR
+ return mlx5dv_dr_destroy_ft(tbl);
+#else
+ (void)tbl;
+ return 0;
+#endif
+}
+
+static void *
+mlx5_glue_dr_create_ns(struct ibv_context *ctx,
+ enum mlx5dv_dr_ns_domain domain)
+{
+#ifdef HAVE_MLX5DV_DR
+ return mlx5dv_dr_create_ns(ctx, domain);
+#else
+ (void)ctx;
+ (void)domain;
+ return NULL;
+#endif
+}
+
+static int
+mlx5_glue_dr_destroy_ns(void *ns)
+{
+#ifdef HAVE_MLX5DV_DR
+ return mlx5dv_dr_destroy_ns(ns);
+#else
+ (void)ns;
+ return 0;
+#endif
+}
+
static struct ibv_cq_ex *
mlx5_glue_dv_create_cq(struct ibv_context *context,
struct ibv_cq_init_attr_ex *cq_attr,
@@ -423,26 +474,40 @@
#endif
}
-static struct mlx5dv_flow_matcher *
+static void *
mlx5_glue_dv_create_flow_matcher(struct ibv_context *context,
- struct mlx5dv_flow_matcher_attr *matcher_attr)
+ struct mlx5dv_flow_matcher_attr *matcher_attr,
+ void *tbl)
{
#ifdef HAVE_IBV_FLOW_DV_SUPPORT
+#ifdef HAVE_MLX5DV_DR
+ (void)context;
+ return mlx5dv_dr_create_matcher(tbl, matcher_attr->priority,
+ matcher_attr->match_criteria_enable,
+ matcher_attr->match_mask);
+#else
+ (void)tbl;
return mlx5dv_create_flow_matcher(context, matcher_attr);
+#endif
#else
(void)context;
(void)matcher_attr;
+ (void)tbl;
return NULL;
#endif
}
-static struct ibv_flow *
-mlx5_glue_dv_create_flow(struct mlx5dv_flow_matcher *matcher,
- struct mlx5dv_flow_match_parameters *match_value,
+static void *
+mlx5_glue_dv_create_flow(void *matcher,
+ void *match_value,
size_t num_actions,
void *actions[])
{
#ifdef HAVE_IBV_FLOW_DV_SUPPORT
+#ifdef HAVE_MLX5DV_DR
+ return mlx5dv_dr_create_rule(matcher, match_value, num_actions,
+ (struct mlx5dv_dr_action **)actions);
+#else
struct mlx5dv_flow_action_attr actions_attr[8];
if (num_actions > 8)
@@ -452,6 +517,7 @@
*((struct mlx5dv_flow_action_attr *)(actions[i]));
return mlx5dv_create_flow(matcher, match_value,
num_actions, actions_attr);
+#endif
#else
(void)matcher;
(void)match_value;
@@ -461,21 +527,13 @@
#endif
}
-static int
-mlx5_glue_dv_destroy_flow_matcher(struct mlx5dv_flow_matcher *matcher)
-{
-#ifdef HAVE_IBV_FLOW_DV_SUPPORT
- return mlx5dv_destroy_flow_matcher(matcher);
-#else
- (void)matcher;
- return 0;
-#endif
-}
-
static void *
mlx5_glue_dv_create_flow_action_counter(void *counter_obj, uint32_t offset)
{
#ifdef HAVE_IBV_FLOW_DV_SUPPORT
+#ifdef HAVE_MLX5DV_DR
+ return mlx5dv_dr_create_action_devx_counter(counter_obj, offset);
+#else
struct mlx5dv_flow_action_attr *action;
(void)offset;
@@ -485,6 +543,7 @@
action->type = MLX5DV_FLOW_ACTION_COUNTER_DEVX;
action->obj = counter_obj;
return action;
+#endif
#else
(void)counter_obj;
(void)offset;
@@ -496,6 +555,9 @@
mlx5_glue_dv_create_flow_action_dest_ibv_qp(void *qp)
{
#ifdef HAVE_IBV_FLOW_DV_SUPPORT
+#ifdef HAVE_MLX5DV_DR
+ return mlx5dv_dr_create_action_dest_ibv_qp(qp);
+#else
struct mlx5dv_flow_action_attr *action;
action = malloc(sizeof(*action));
@@ -504,6 +566,7 @@
action->type = MLX5DV_FLOW_ACTION_DEST_IBV_QP;
action->obj = qp;
return action;
+#endif
#else
(void)qp;
return NULL;
@@ -513,13 +576,22 @@
static void *
mlx5_glue_dv_create_flow_action_modify_header
(struct ibv_context *ctx,
+ enum mlx5dv_flow_table_type ft_type,
+ void *ns, uint64_t flags,
size_t actions_sz,
- uint64_t actions[],
- enum mlx5dv_flow_table_type ft_type)
+ uint64_t actions[])
{
#ifdef HAVE_IBV_FLOW_DV_SUPPORT
+#ifdef HAVE_MLX5DV_DR
+ (void)ctx;
+ (void)ft_type;
+ return mlx5dv_dr_create_action_modify_header(ns, flags, actions_sz,
+ actions);
+#else
struct mlx5dv_flow_action_attr *action;
+ (void)ns;
+ (void)flags;
action = malloc(sizeof(*action));
if (!action)
return NULL;
@@ -527,11 +599,14 @@
action->action = mlx5dv_create_flow_action_modify_header
(ctx, actions_sz, actions, ft_type);
return action;
+#endif
#else
(void)ctx;
+ (void)ft_type;
+ (void)ns;
+ (void)flags;
(void)actions_sz;
(void)actions;
- (void)ft_type;
return NULL;
#endif
}
@@ -539,12 +614,20 @@
static void *
mlx5_glue_dv_create_flow_action_packet_reformat
(struct ibv_context *ctx,
- size_t data_sz,
- void *data,
enum mlx5dv_flow_action_packet_reformat_type reformat_type,
- enum mlx5dv_flow_table_type ft_type)
+ enum mlx5dv_flow_table_type ft_type, struct mlx5dv_dr_ns *ns,
+ uint32_t flags, size_t data_sz, void *data)
{
#ifdef HAVE_IBV_FLOW_DV_SUPPORT
+#ifdef HAVE_MLX5DV_DR
+ (void)ctx;
+ (void)ft_type;
+ return mlx5dv_dr_create_action_packet_reformat(ns, flags,
+ reformat_type, data_sz,
+ data);
+#else
+ (void)ns;
+ (void)flags;
struct mlx5dv_flow_action_attr *action;
action = malloc(sizeof(*action));
@@ -554,12 +637,15 @@
action->action = mlx5dv_create_flow_action_packet_reformat
(ctx, data_sz, data, reformat_type, ft_type);
return action;
+#endif
#else
(void)ctx;
- (void)data_sz;
- (void)data;
(void)reformat_type;
(void)ft_type;
+ (void)ns;
+ (void)flags;
+ (void)data_sz;
+ (void)data;
return NULL;
#endif
}
@@ -568,6 +654,9 @@
mlx5_glue_dv_create_flow_action_tag(uint32_t tag)
{
#ifdef HAVE_IBV_FLOW_DV_SUPPORT
+#ifdef HAVE_MLX5DV_DR
+ return mlx5dv_dr_create_action_tag(tag);
+#else
struct mlx5dv_flow_action_attr *action;
action = malloc(sizeof(*action));
if (!action)
@@ -576,10 +665,36 @@
action->tag_value = tag;
return action;
#endif
+#endif
(void)tag;
return NULL;
}
+static int
+mlx5_glue_dv_destroy_flow(void *flow_id)
+{
+#ifdef HAVE_MLX5DV_DR
+ return mlx5dv_dr_destroy_rule(flow_id);
+#else
+ return ibv_destroy_flow(flow_id);
+#endif
+}
+
+static int
+mlx5_glue_dv_destroy_flow_matcher(void *matcher)
+{
+#ifdef HAVE_IBV_FLOW_DV_SUPPORT
+#ifdef HAVE_MLX5DV_DR
+ return mlx5dv_dr_destroy_matcher(matcher);
+#else
+ return mlx5dv_destroy_flow_matcher(matcher);
+#endif
+#else
+ (void)matcher;
+ return 0;
+#endif
+}
+
static struct ibv_context *
mlx5_glue_dv_open_device(struct ibv_device *device)
{
@@ -718,6 +833,10 @@
.get_async_event = mlx5_glue_get_async_event,
.port_state_str = mlx5_glue_port_state_str,
.cq_ex_to_cq = mlx5_glue_cq_ex_to_cq,
+ .dr_create_flow_tbl = mlx5_glue_dr_create_flow_tbl,
+ .dr_destroy_flow_tbl = mlx5_glue_dr_destroy_flow_tbl,
+ .dr_create_ns = mlx5_glue_dr_create_ns,
+ .dr_destroy_ns = mlx5_glue_dr_destroy_ns,
.dv_create_cq = mlx5_glue_dv_create_cq,
.dv_create_wq = mlx5_glue_dv_create_wq,
.dv_query_device = mlx5_glue_dv_query_device,
@@ -725,7 +844,6 @@
.dv_init_obj = mlx5_glue_dv_init_obj,
.dv_create_qp = mlx5_glue_dv_create_qp,
.dv_create_flow_matcher = mlx5_glue_dv_create_flow_matcher,
- .dv_destroy_flow_matcher = mlx5_glue_dv_destroy_flow_matcher,
.dv_create_flow = mlx5_glue_dv_create_flow,
.dv_create_flow_action_counter =
mlx5_glue_dv_create_flow_action_counter,
@@ -736,6 +854,8 @@
.dv_create_flow_action_packet_reformat =
mlx5_glue_dv_create_flow_action_packet_reformat,
.dv_create_flow_action_tag = mlx5_glue_dv_create_flow_action_tag,
+ .dv_destroy_flow = mlx5_glue_dv_destroy_flow,
+ .dv_destroy_flow_matcher = mlx5_glue_dv_destroy_flow_matcher,
.dv_open_device = mlx5_glue_dv_open_device,
.devx_obj_create = mlx5_glue_devx_obj_create,
.devx_obj_destroy = mlx5_glue_devx_obj_destroy,
diff --git a/drivers/net/mlx5/mlx5_glue.h b/drivers/net/mlx5/mlx5_glue.h
index 167f1f7..7115575 100644
--- a/drivers/net/mlx5/mlx5_glue.h
+++ b/drivers/net/mlx5/mlx5_glue.h
@@ -63,6 +63,11 @@
struct mlx5dv_devx_obj;
#endif
+#ifndef HAVE_MLX5DV_DR
+struct mlx5dv_dr_ns;
+enum mlx5dv_dr_ns_domain { unused, };
+#endif
+
/* LIB_GLUE_VERSION must be updated every time this structure is modified. */
struct mlx5_glue {
const char *version;
@@ -140,6 +145,11 @@ struct mlx5_glue {
struct ibv_async_event *event);
const char *(*port_state_str)(enum ibv_port_state port_state);
struct ibv_cq *(*cq_ex_to_cq)(struct ibv_cq_ex *cq);
+ void *(*dr_create_flow_tbl)(void *ns, uint32_t level);
+ int (*dr_destroy_flow_tbl)(void *tbl);
+ void *(*dr_create_ns)(struct ibv_context *ctx,
+ enum mlx5dv_dr_ns_domain domain);
+ int (*dr_destroy_ns)(void *ns);
struct ibv_cq_ex *(*dv_create_cq)
(struct ibv_context *context,
struct ibv_cq_init_attr_ex *cq_attr,
@@ -158,23 +168,26 @@ struct mlx5_glue {
(struct ibv_context *context,
struct ibv_qp_init_attr_ex *qp_init_attr_ex,
struct mlx5dv_qp_init_attr *dv_qp_init_attr);
- struct mlx5dv_flow_matcher *(*dv_create_flow_matcher)
+ void *(*dv_create_flow_matcher)
(struct ibv_context *context,
- struct mlx5dv_flow_matcher_attr *matcher_attr);
- int (*dv_destroy_flow_matcher)(struct mlx5dv_flow_matcher *matcher);
- struct ibv_flow *(*dv_create_flow)(struct mlx5dv_flow_matcher *matcher,
- struct mlx5dv_flow_match_parameters *match_value,
+ struct mlx5dv_flow_matcher_attr *matcher_attr,
+ void *tbl);
+ void *(*dv_create_flow)(void *matcher, void *match_value,
size_t num_actions, void *actions[]);
void *(*dv_create_flow_action_counter)(void *obj, uint32_t offset);
void *(*dv_create_flow_action_dest_ibv_qp)(void *qp);
void *(*dv_create_flow_action_modify_header)
- (struct ibv_context *ctx, size_t actions_sz, uint64_t actions[],
- enum mlx5dv_flow_table_type ft_type);
+ (struct ibv_context *ctx, enum mlx5dv_flow_table_type ft_type,
+ void *ns, uint64_t flags, size_t actions_sz,
+ uint64_t actions[]);
void *(*dv_create_flow_action_packet_reformat)
- (struct ibv_context *ctx, size_t data_sz, void *data,
+ (struct ibv_context *ctx,
enum mlx5dv_flow_action_packet_reformat_type reformat_type,
- enum mlx5dv_flow_table_type ft_type);
+ enum mlx5dv_flow_table_type ft_type, struct mlx5dv_dr_ns *ns,
+ uint32_t flags, size_t data_sz, void *data);
void *(*dv_create_flow_action_tag)(uint32_t tag);
+ int (*dv_destroy_flow)(void *flow);
+ int (*dv_destroy_flow_matcher)(void *matcher);
struct ibv_context *(*dv_open_device)(struct ibv_device *device);
struct mlx5dv_devx_obj *(*devx_obj_create)
(struct ibv_context *ctx,
diff --git a/drivers/net/mlx5/mlx5_prm.h b/drivers/net/mlx5/mlx5_prm.h
index da1219e..b15266f 100644
--- a/drivers/net/mlx5/mlx5_prm.h
+++ b/drivers/net/mlx5/mlx5_prm.h
@@ -492,20 +492,40 @@ struct mlx5_ifc_fte_match_set_misc2_bits {
u8 reserved_at_1a0[0x60];
};
+struct mlx5_ifc_fte_match_set_misc3_bits {
+ u8 inner_tcp_seq_num[0x20];
+ u8 outer_tcp_seq_num[0x20];
+ u8 inner_tcp_ack_num[0x20];
+ u8 outer_tcp_ack_num[0x20];
+ u8 reserved_at_auto1[0x8];
+ u8 outer_vxlan_gpe_vni[0x18];
+ u8 outer_vxlan_gpe_next_protocol[0x8];
+ u8 outer_vxlan_gpe_flags[0x8];
+ u8 reserved_at_a8[0x10];
+ u8 icmp_header_data[0x20];
+ u8 icmpv6_header_data[0x20];
+ u8 icmp_type[0x8];
+ u8 icmp_code[0x8];
+ u8 icmpv6_type[0x8];
+ u8 icmpv6_code[0x8];
+ u8 reserved_at_1a0[0xe0];
+};
+
/* Flow matcher. */
struct mlx5_ifc_fte_match_param_bits {
struct mlx5_ifc_fte_match_set_lyr_2_4_bits outer_headers;
struct mlx5_ifc_fte_match_set_misc_bits misc_parameters;
struct mlx5_ifc_fte_match_set_lyr_2_4_bits inner_headers;
struct mlx5_ifc_fte_match_set_misc2_bits misc_parameters_2;
- u8 reserved_at_800[0x800];
+ struct mlx5_ifc_fte_match_set_misc3_bits misc_parameters_3;
};
enum {
MLX5_MATCH_CRITERIA_ENABLE_OUTER_BIT,
MLX5_MATCH_CRITERIA_ENABLE_MISC_BIT,
MLX5_MATCH_CRITERIA_ENABLE_INNER_BIT,
- MLX5_MATCH_CRITERIA_ENABLE_MISC2_BIT
+ MLX5_MATCH_CRITERIA_ENABLE_MISC2_BIT,
+ MLX5_MATCH_CRITERIA_ENABLE_MISC3_BIT
};
enum {
--
1.8.3.1
^ permalink raw reply [flat|nested] 52+ messages in thread
* [dpdk-dev] [PATCH v2 2/3] net/mlx5: add Direct Rules API
2019-03-28 16:32 ` [dpdk-dev] [PATCH v2 2/3] net/mlx5: add Direct Rules API Ori Kam
@ 2019-03-28 16:32 ` Ori Kam
2019-04-01 14:38 ` Slava Ovsiienko
1 sibling, 0 replies; 52+ messages in thread
From: Ori Kam @ 2019-03-28 16:32 UTC (permalink / raw)
To: Matan Azrad, Yongseok Koh, Shahaf Shuler; +Cc: dev, Ori Kam, Slava Ovsiienko
Adds calls to the Direct Rules API inside the glue functions.
Due to difference in parameters between the Direct Rules and Direct
Verbs some of the glue functions API was updated.
Signed-off-by: Ori Kam <orika@mellanox.com>
---
drivers/net/mlx5/Makefile | 5 ++
drivers/net/mlx5/mlx5.c | 16 ++++
drivers/net/mlx5/mlx5.h | 15 ++++
drivers/net/mlx5/mlx5_flow.c | 1 +
drivers/net/mlx5/mlx5_flow.h | 6 +-
drivers/net/mlx5/mlx5_flow_dv.c | 103 ++++++++++++++++++++----
drivers/net/mlx5/mlx5_glue.c | 170 ++++++++++++++++++++++++++++++++++------
drivers/net/mlx5/mlx5_glue.h | 31 +++++---
drivers/net/mlx5/mlx5_prm.h | 24 +++++-
9 files changed, 318 insertions(+), 53 deletions(-)
diff --git a/drivers/net/mlx5/Makefile b/drivers/net/mlx5/Makefile
index c326494..3dd7e38 100644
--- a/drivers/net/mlx5/Makefile
+++ b/drivers/net/mlx5/Makefile
@@ -156,6 +156,11 @@ mlx5_autoconf.h.new: $(RTE_SDK)/buildtools/auto-config-h.sh
func mlx5dv_create_flow_action_packet_reformat \
$(AUTOCONF_OUTPUT)
$Q sh -- '$<' '$@' \
+ HAVE_MLX5DV_DR \
+ infiniband/mlx5dv.h \
+ enum MLX5DV_DR_NS_TYPE_TERMINATING \
+ $(AUTOCONF_OUTPUT)
+ $Q sh -- '$<' '$@' \
HAVE_IBV_DEVX_OBJ \
infiniband/mlx5dv.h \
func mlx5dv_devx_obj_create \
diff --git a/drivers/net/mlx5/mlx5.c b/drivers/net/mlx5/mlx5.c
index 7d1a21e..70e37b5 100644
--- a/drivers/net/mlx5/mlx5.c
+++ b/drivers/net/mlx5/mlx5.c
@@ -1362,6 +1362,22 @@ struct mlx5_dev_spawn_data {
priv->tcf_context = NULL;
}
}
+#ifdef HAVE_MLX5DV_DR
+ priv->rx_ns = mlx5dv_dr_create_ns
+ (sh->ctx, MLX5DV_DR_NS_DOMAIN_INGRESS_BYPASS);
+ if (priv->rx_ns == NULL) {
+ DRV_LOG(ERR, "mlx5dv_dr_create_ns failed");
+ err = errno;
+ goto error;
+ }
+ priv->tx_ns = mlx5dv_dr_create_ns(sh->ctx,
+ MLX5DV_DR_NS_DOMAIN_EGRESS_BYPASS);
+ if (priv->tx_ns == NULL) {
+ DRV_LOG(ERR, "mlx5dv_dr_create_ns failed");
+ err = errno;
+ goto error;
+ }
+#endif
TAILQ_INIT(&priv->flows);
TAILQ_INIT(&priv->ctrl_flows);
/* Hint libmlx5 to use PMD allocator for data plane resources */
diff --git a/drivers/net/mlx5/mlx5.h b/drivers/net/mlx5/mlx5.h
index b587359..73f6f0d 100644
--- a/drivers/net/mlx5/mlx5.h
+++ b/drivers/net/mlx5/mlx5.h
@@ -219,6 +219,15 @@ struct mlx5_ibv_shared {
struct mlx5_ibv_shared_port port[]; /* per device port data array. */
};
+/* Table structure. */
+struct mlx5_flow_tbl_resource {
+ void *obj; /**< Pointer to DR table object. */
+ rte_atomic32_t refcnt; /**< Reference counter. */
+};
+
+#define MLX5_MAX_TABLES 1024
+#define MLX5_GROUP_FACTOR 1
+
struct mlx5_priv {
LIST_ENTRY(mlx5_priv) mem_event_cb;
/**< Called by memory event callback. */
@@ -289,6 +298,12 @@ struct mlx5_priv {
/* UAR same-page access control required in 32bit implementations. */
#endif
struct mlx5_flow_tcf_context *tcf_context; /* TC flower context. */
+ void *rx_ns; /* RX Direct Rules name space handle. */
+ struct mlx5_flow_tbl_resource rx_tbl[MLX5_MAX_TABLES];
+ /* RX Direct Rules tables. */
+ void *tx_ns; /* TX Direct Rules name space handle. */
+ struct mlx5_flow_tbl_resource tx_tbl[MLX5_MAX_TABLES];
+ /* TX Direct Rules tables/ */
};
#define PORT_ID(priv) ((priv)->dev_data->port_id)
diff --git a/drivers/net/mlx5/mlx5_flow.c b/drivers/net/mlx5/mlx5_flow.c
index d09fdff..40d6818 100644
--- a/drivers/net/mlx5/mlx5_flow.c
+++ b/drivers/net/mlx5/mlx5_flow.c
@@ -2084,6 +2084,7 @@ uint32_t mlx5_flow_adjust_priority(struct rte_eth_dev *dev, int32_t priority,
flow_size += RTE_ALIGN_CEIL(sizeof(uint16_t), sizeof(void *));
flow = rte_calloc(__func__, 1, flow_size, 0);
flow->drv_type = flow_get_drv_type(dev, attr);
+ flow->ingress = attr->ingress;
assert(flow->drv_type > MLX5_FLOW_TYPE_MIN &&
flow->drv_type < MLX5_FLOW_TYPE_MAX);
flow->queue = (void *)(flow + 1);
diff --git a/drivers/net/mlx5/mlx5_flow.h b/drivers/net/mlx5/mlx5_flow.h
index 4f69ae2..8ba37a0 100644
--- a/drivers/net/mlx5/mlx5_flow.h
+++ b/drivers/net/mlx5/mlx5_flow.h
@@ -204,6 +204,7 @@ struct mlx5_flow_dv_matcher {
uint16_t crc; /**< CRC of key. */
uint16_t priority; /**< Priority of matcher. */
uint8_t egress; /**< Egress matcher. */
+ uint32_t group; /**< The matcher group. */
struct mlx5_flow_dv_match_params mask; /**< Matcher mask. */
};
@@ -220,6 +221,7 @@ struct mlx5_flow_dv_encap_decap_resource {
size_t size;
uint8_t reformat_type;
uint8_t ft_type;
+ uint64_t flags; /**< Flags for RDMA API. */
};
/* Tag resource structure. */
@@ -348,7 +350,7 @@ struct mlx5_flow_counter {
/* Flow structure. */
struct rte_flow {
TAILQ_ENTRY(rte_flow) next; /**< Pointer to the next flow structure. */
- enum mlx5_flow_drv_type drv_type; /**< Drvier type. */
+ enum mlx5_flow_drv_type drv_type; /**< Driver type. */
struct mlx5_flow_counter *counter; /**< Holds flow counter. */
struct mlx5_flow_dv_tag_resource *tag_resource;
/**< pointer to the tag action. */
@@ -360,6 +362,8 @@ struct rte_flow {
uint64_t actions;
/**< Bit-fields of detected actions, see MLX5_FLOW_ACTION_*. */
struct mlx5_fdir *fdir; /**< Pointer to associated FDIR if any. */
+ uint8_t ingress; /**< 1 if the flow is ingress. */
+ uint32_t group; /**< The group index. */
};
typedef int (*mlx5_flow_validate_t)(struct rte_eth_dev *dev,
diff --git a/drivers/net/mlx5/mlx5_flow_dv.c b/drivers/net/mlx5/mlx5_flow_dv.c
index b0c50a7..6e4f6c4 100644
--- a/drivers/net/mlx5/mlx5_flow_dv.c
+++ b/drivers/net/mlx5/mlx5_flow_dv.c
@@ -805,11 +805,20 @@ struct field_modify_info modify_tcp[] = {
{
struct mlx5_priv *priv = dev->data->dev_private;
struct mlx5_flow_dv_encap_decap_resource *cache_resource;
+ struct rte_flow *flow = dev_flow->flow;
+ struct mlx5dv_dr_ns *ns;
+
+ resource->flags = flow->group ? 0 : 1;
+ if (flow->ingress)
+ ns = priv->rx_ns;
+ else
+ ns = priv->tx_ns;
/* Lookup a matching resource from cache. */
LIST_FOREACH(cache_resource, &priv->encaps_decaps, next) {
if (resource->reformat_type == cache_resource->reformat_type &&
resource->ft_type == cache_resource->ft_type &&
+ resource->flags == cache_resource->flags &&
resource->size == cache_resource->size &&
!memcmp((const void *)resource->buf,
(const void *)cache_resource->buf,
@@ -831,10 +840,10 @@ struct field_modify_info modify_tcp[] = {
*cache_resource = *resource;
cache_resource->verbs_action =
mlx5_glue->dv_create_flow_action_packet_reformat
- (priv->sh->ctx, cache_resource->size,
- (cache_resource->size ? cache_resource->buf : NULL),
- cache_resource->reformat_type,
- cache_resource->ft_type);
+ (priv->sh->ctx, cache_resource->reformat_type,
+ cache_resource->ft_type, ns, cache_resource->flags,
+ cache_resource->size,
+ (cache_resource->size ? cache_resource->buf : NULL));
if (!cache_resource->verbs_action) {
rte_free(cache_resource);
return rte_flow_error_set(error, ENOMEM,
@@ -1438,6 +1447,10 @@ struct field_modify_info modify_tcp[] = {
struct mlx5_priv *priv = dev->data->dev_private;
struct mlx5_flow_dv_modify_hdr_resource *cache_resource;
+ struct mlx5dv_dr_ns *ns =
+ resource->ft_type == MLX5DV_FLOW_TABLE_TYPE_NIC_TX ?
+ priv->tx_ns : priv->rx_ns;
+
/* Lookup a matching resource from cache. */
LIST_FOREACH(cache_resource, &priv->modify_cmds, next) {
if (resource->ft_type == cache_resource->ft_type &&
@@ -1463,11 +1476,11 @@ struct field_modify_info modify_tcp[] = {
*cache_resource = *resource;
cache_resource->verbs_action =
mlx5_glue->dv_create_flow_action_modify_header
- (priv->sh->ctx,
+ (priv->sh->ctx, cache_resource->ft_type,
+ ns, 0,
cache_resource->actions_num *
sizeof(cache_resource->actions[0]),
- (uint64_t *)cache_resource->actions,
- cache_resource->ft_type);
+ (uint64_t *)cache_resource->actions);
if (!cache_resource->verbs_action) {
rte_free(cache_resource);
return rte_flow_error_set(error, ENOMEM,
@@ -1592,11 +1605,13 @@ struct field_modify_info modify_tcp[] = {
struct mlx5_priv *priv = dev->data->dev_private;
uint32_t priority_max = priv->config.flow_prio - 1;
+#ifdef HAVE_MLX5DV_DR
if (attributes->group)
return rte_flow_error_set(error, ENOTSUP,
RTE_FLOW_ERROR_TYPE_ATTR_GROUP,
NULL,
"groups is not supported");
+#endif
if (attributes->priority != MLX5_FLOW_PRIO_RSVD &&
attributes->priority >= priority_max)
return rte_flow_error_set(error, ENOTSUP,
@@ -2169,11 +2184,13 @@ struct field_modify_info modify_tcp[] = {
* Flow pattern to translate.
* @param[in] inner
* Item is inner pattern.
+ * @param[in] group
+ * The group to insert the rule.
*/
static void
flow_dv_translate_item_ipv4(void *matcher, void *key,
const struct rte_flow_item *item,
- int inner)
+ int inner, uint32_t group)
{
const struct rte_flow_item_ipv4 *ipv4_m = item->mask;
const struct rte_flow_item_ipv4 *ipv4_v = item->spec;
@@ -2200,7 +2217,10 @@ struct field_modify_info modify_tcp[] = {
outer_headers);
headers_v = MLX5_ADDR_OF(fte_match_param, key, outer_headers);
}
- MLX5_SET(fte_match_set_lyr_2_4, headers_m, ip_version, 0xf);
+ if (group == 0)
+ MLX5_SET(fte_match_set_lyr_2_4, headers_m, ip_version, 0xf);
+ else
+ MLX5_SET(fte_match_set_lyr_2_4, headers_m, ip_version, 0x4);
MLX5_SET(fte_match_set_lyr_2_4, headers_v, ip_version, 4);
if (!ipv4_v)
return;
@@ -2242,11 +2262,13 @@ struct field_modify_info modify_tcp[] = {
* Flow pattern to translate.
* @param[in] inner
* Item is inner pattern.
+ * @param[in] group
+ * The group to insert the rule.
*/
static void
flow_dv_translate_item_ipv6(void *matcher, void *key,
const struct rte_flow_item *item,
- int inner)
+ int inner, uint32_t group)
{
const struct rte_flow_item_ipv6 *ipv6_m = item->mask;
const struct rte_flow_item_ipv6 *ipv6_v = item->spec;
@@ -2283,7 +2305,10 @@ struct field_modify_info modify_tcp[] = {
outer_headers);
headers_v = MLX5_ADDR_OF(fte_match_param, key, outer_headers);
}
- MLX5_SET(fte_match_set_lyr_2_4, headers_m, ip_version, 0xf);
+ if (group == 0)
+ MLX5_SET(fte_match_set_lyr_2_4, headers_m, ip_version, 0xf);
+ else
+ MLX5_SET(fte_match_set_lyr_2_4, headers_m, ip_version, 0x6);
MLX5_SET(fte_match_set_lyr_2_4, headers_v, ip_version, 6);
if (!ipv6_v)
return;
@@ -2723,7 +2748,11 @@ struct field_modify_info modify_tcp[] = {
match_criteria_enable |=
(!HEADER_IS_ZERO(match_criteria, misc_parameters_2)) <<
MLX5_MATCH_CRITERIA_ENABLE_MISC2_BIT;
-
+#ifdef HAVE_MLX5DV_DR
+ match_criteria_enable |=
+ (!HEADER_IS_ZERO(match_criteria, misc_parameters_3)) <<
+ MLX5_MATCH_CRITERIA_ENABLE_MISC3_BIT;
+#endif
return match_criteria_enable;
}
@@ -2754,12 +2783,14 @@ struct field_modify_info modify_tcp[] = {
.type = IBV_FLOW_ATTR_NORMAL,
.match_mask = (void *)&matcher->mask,
};
+ struct mlx5_flow_tbl_resource *tbl = NULL;
/* Lookup from cache. */
LIST_FOREACH(cache_matcher, &priv->matchers, next) {
if (matcher->crc == cache_matcher->crc &&
matcher->priority == cache_matcher->priority &&
matcher->egress == cache_matcher->egress &&
+ matcher->group == cache_matcher->group &&
!memcmp((const void *)matcher->mask.buf,
(const void *)cache_matcher->mask.buf,
cache_matcher->mask.size)) {
@@ -2774,6 +2805,27 @@ struct field_modify_info modify_tcp[] = {
return 0;
}
}
+#ifdef HAVE_MLX5DV_DR
+ if (matcher->egress) {
+ tbl = &priv->tx_tbl[matcher->group];
+ if (!tbl->obj)
+ tbl->obj = mlx5_glue->dr_create_flow_tbl
+ (priv->tx_ns,
+ matcher->group * MLX5_GROUP_FACTOR);
+ } else {
+ tbl = &priv->rx_tbl[matcher->group];
+ if (!tbl->obj)
+ tbl->obj = mlx5_glue->dr_create_flow_tbl
+ (priv->rx_ns,
+ matcher->group * MLX5_GROUP_FACTOR);
+ }
+ if (!tbl->obj)
+ return rte_flow_error_set(error, ENOMEM,
+ RTE_FLOW_ERROR_TYPE_UNSPECIFIED,
+ NULL, "cannot create table");
+
+ rte_atomic32_inc(&tbl->refcnt);
+#endif
/* Register new matcher. */
cache_matcher = rte_calloc(__func__, 1, sizeof(*cache_matcher), 0);
if (!cache_matcher)
@@ -2787,9 +2839,16 @@ struct field_modify_info modify_tcp[] = {
if (matcher->egress)
dv_attr.flags |= IBV_FLOW_ATTR_FLAGS_EGRESS;
cache_matcher->matcher_object =
- mlx5_glue->dv_create_flow_matcher(priv->sh->ctx, &dv_attr);
+ mlx5_glue->dv_create_flow_matcher(priv->sh->ctx, &dv_attr,
+ tbl->obj);
if (!cache_matcher->matcher_object) {
rte_free(cache_matcher);
+#ifdef HAVE_MLX5DV_DR
+ if (rte_atomic32_dec_and_test(&tbl->refcnt)) {
+ mlx5_glue->dr_destroy_flow_tbl(tbl->obj);
+ tbl->obj = NULL;
+ }
+#endif
return rte_flow_error_set(error, ENOMEM,
RTE_FLOW_ERROR_TYPE_UNSPECIFIED,
NULL, "cannot create matcher");
@@ -2801,6 +2860,7 @@ struct field_modify_info modify_tcp[] = {
cache_matcher->priority,
cache_matcher->egress ? "tx" : "rx", (void *)cache_matcher,
rte_atomic32_read(&cache_matcher->refcnt));
+ rte_atomic32_inc(&tbl->refcnt);
return 0;
}
@@ -3222,7 +3282,7 @@ struct field_modify_info modify_tcp[] = {
break;
case RTE_FLOW_ITEM_TYPE_IPV4:
flow_dv_translate_item_ipv4(match_mask, match_value,
- items, tunnel);
+ items, tunnel, attr->group);
matcher.priority = MLX5_PRIORITY_MAP_L3;
dev_flow->dv.hash_fields |=
mlx5_flow_hashfields_adjust
@@ -3234,7 +3294,7 @@ struct field_modify_info modify_tcp[] = {
break;
case RTE_FLOW_ITEM_TYPE_IPV6:
flow_dv_translate_item_ipv6(match_mask, match_value,
- items, tunnel);
+ items, tunnel, attr->group);
matcher.priority = MLX5_PRIORITY_MAP_L3;
dev_flow->dv.hash_fields |=
mlx5_flow_hashfields_adjust
@@ -3312,6 +3372,7 @@ struct field_modify_info modify_tcp[] = {
matcher.priority = mlx5_flow_adjust_priority(dev, priority,
matcher.priority);
matcher.egress = attr->egress;
+ matcher.group = attr->group;
if (flow_dv_matcher_register(dev, &matcher, dev_flow, error))
return -rte_errno;
return 0;
@@ -3427,6 +3488,8 @@ struct field_modify_info modify_tcp[] = {
struct mlx5_flow *flow)
{
struct mlx5_flow_dv_matcher *matcher = flow->dv.matcher;
+ struct mlx5_priv *priv = dev->data->dev_private;
+ struct mlx5_flow_tbl_resource *tbl;
assert(matcher->matcher_object);
DRV_LOG(DEBUG, "port %u matcher %p: refcnt %d--",
@@ -3436,6 +3499,14 @@ struct field_modify_info modify_tcp[] = {
claim_zero(mlx5_glue->dv_destroy_flow_matcher
(matcher->matcher_object));
LIST_REMOVE(matcher, next);
+ if (matcher->egress)
+ tbl = &priv->tx_tbl[matcher->group];
+ else
+ tbl = &priv->rx_tbl[matcher->group];
+ if (rte_atomic32_dec_and_test(&tbl->refcnt)) {
+ mlx5_glue->dr_destroy_flow_tbl(tbl->obj);
+ tbl->obj = NULL;
+ }
rte_free(matcher);
DRV_LOG(DEBUG, "port %u matcher %p: removed",
dev->data->port_id, (void *)matcher);
@@ -3525,7 +3596,7 @@ struct field_modify_info modify_tcp[] = {
LIST_FOREACH(dev_flow, &flow->dev_flows, next) {
dv = &dev_flow->dv;
if (dv->flow) {
- claim_zero(mlx5_glue->destroy_flow(dv->flow));
+ claim_zero(mlx5_glue->dv_destroy_flow(dv->flow));
dv->flow = NULL;
}
if (dv->hrxq) {
diff --git a/drivers/net/mlx5/mlx5_glue.c b/drivers/net/mlx5/mlx5_glue.c
index cfd939d..f509f85 100644
--- a/drivers/net/mlx5/mlx5_glue.c
+++ b/drivers/net/mlx5/mlx5_glue.c
@@ -178,6 +178,9 @@
mlx5_glue_destroy_flow_action(void *action)
{
#ifdef HAVE_IBV_FLOW_DV_SUPPORT
+#ifdef HAVE_MLX5DV_DR
+ return mlx5dv_dr_destroy_action(action);
+#else
struct mlx5dv_flow_action_attr *attr = action;
int res = 0;
switch (attr->type) {
@@ -189,6 +192,7 @@
}
free(action);
return res;
+#endif
#else
(void)action;
return ENOTSUP;
@@ -365,6 +369,53 @@
return ibv_cq_ex_to_cq(cq);
}
+static void *
+mlx5_glue_dr_create_flow_tbl(void *ns, uint32_t level)
+{
+#ifdef HAVE_MLX5DV_DR
+ return mlx5dv_dr_create_ft(ns, level);
+#else
+ (void)ns;
+ (void)level;
+ return NULL;
+#endif
+}
+
+static int
+mlx5_glue_dr_destroy_flow_tbl(void *tbl)
+{
+#ifdef HAVE_MLX5DV_DR
+ return mlx5dv_dr_destroy_ft(tbl);
+#else
+ (void)tbl;
+ return 0;
+#endif
+}
+
+static void *
+mlx5_glue_dr_create_ns(struct ibv_context *ctx,
+ enum mlx5dv_dr_ns_domain domain)
+{
+#ifdef HAVE_MLX5DV_DR
+ return mlx5dv_dr_create_ns(ctx, domain);
+#else
+ (void)ctx;
+ (void)domain;
+ return NULL;
+#endif
+}
+
+static int
+mlx5_glue_dr_destroy_ns(void *ns)
+{
+#ifdef HAVE_MLX5DV_DR
+ return mlx5dv_dr_destroy_ns(ns);
+#else
+ (void)ns;
+ return 0;
+#endif
+}
+
static struct ibv_cq_ex *
mlx5_glue_dv_create_cq(struct ibv_context *context,
struct ibv_cq_init_attr_ex *cq_attr,
@@ -423,26 +474,40 @@
#endif
}
-static struct mlx5dv_flow_matcher *
+static void *
mlx5_glue_dv_create_flow_matcher(struct ibv_context *context,
- struct mlx5dv_flow_matcher_attr *matcher_attr)
+ struct mlx5dv_flow_matcher_attr *matcher_attr,
+ void *tbl)
{
#ifdef HAVE_IBV_FLOW_DV_SUPPORT
+#ifdef HAVE_MLX5DV_DR
+ (void)context;
+ return mlx5dv_dr_create_matcher(tbl, matcher_attr->priority,
+ matcher_attr->match_criteria_enable,
+ matcher_attr->match_mask);
+#else
+ (void)tbl;
return mlx5dv_create_flow_matcher(context, matcher_attr);
+#endif
#else
(void)context;
(void)matcher_attr;
+ (void)tbl;
return NULL;
#endif
}
-static struct ibv_flow *
-mlx5_glue_dv_create_flow(struct mlx5dv_flow_matcher *matcher,
- struct mlx5dv_flow_match_parameters *match_value,
+static void *
+mlx5_glue_dv_create_flow(void *matcher,
+ void *match_value,
size_t num_actions,
void *actions[])
{
#ifdef HAVE_IBV_FLOW_DV_SUPPORT
+#ifdef HAVE_MLX5DV_DR
+ return mlx5dv_dr_create_rule(matcher, match_value, num_actions,
+ (struct mlx5dv_dr_action **)actions);
+#else
struct mlx5dv_flow_action_attr actions_attr[8];
if (num_actions > 8)
@@ -452,6 +517,7 @@
*((struct mlx5dv_flow_action_attr *)(actions[i]));
return mlx5dv_create_flow(matcher, match_value,
num_actions, actions_attr);
+#endif
#else
(void)matcher;
(void)match_value;
@@ -461,21 +527,13 @@
#endif
}
-static int
-mlx5_glue_dv_destroy_flow_matcher(struct mlx5dv_flow_matcher *matcher)
-{
-#ifdef HAVE_IBV_FLOW_DV_SUPPORT
- return mlx5dv_destroy_flow_matcher(matcher);
-#else
- (void)matcher;
- return 0;
-#endif
-}
-
static void *
mlx5_glue_dv_create_flow_action_counter(void *counter_obj, uint32_t offset)
{
#ifdef HAVE_IBV_FLOW_DV_SUPPORT
+#ifdef HAVE_MLX5DV_DR
+ return mlx5dv_dr_create_action_devx_counter(counter_obj, offset);
+#else
struct mlx5dv_flow_action_attr *action;
(void)offset;
@@ -485,6 +543,7 @@
action->type = MLX5DV_FLOW_ACTION_COUNTER_DEVX;
action->obj = counter_obj;
return action;
+#endif
#else
(void)counter_obj;
(void)offset;
@@ -496,6 +555,9 @@
mlx5_glue_dv_create_flow_action_dest_ibv_qp(void *qp)
{
#ifdef HAVE_IBV_FLOW_DV_SUPPORT
+#ifdef HAVE_MLX5DV_DR
+ return mlx5dv_dr_create_action_dest_ibv_qp(qp);
+#else
struct mlx5dv_flow_action_attr *action;
action = malloc(sizeof(*action));
@@ -504,6 +566,7 @@
action->type = MLX5DV_FLOW_ACTION_DEST_IBV_QP;
action->obj = qp;
return action;
+#endif
#else
(void)qp;
return NULL;
@@ -513,13 +576,22 @@
static void *
mlx5_glue_dv_create_flow_action_modify_header
(struct ibv_context *ctx,
+ enum mlx5dv_flow_table_type ft_type,
+ void *ns, uint64_t flags,
size_t actions_sz,
- uint64_t actions[],
- enum mlx5dv_flow_table_type ft_type)
+ uint64_t actions[])
{
#ifdef HAVE_IBV_FLOW_DV_SUPPORT
+#ifdef HAVE_MLX5DV_DR
+ (void)ctx;
+ (void)ft_type;
+ return mlx5dv_dr_create_action_modify_header(ns, flags, actions_sz,
+ actions);
+#else
struct mlx5dv_flow_action_attr *action;
+ (void)ns;
+ (void)flags;
action = malloc(sizeof(*action));
if (!action)
return NULL;
@@ -527,11 +599,14 @@
action->action = mlx5dv_create_flow_action_modify_header
(ctx, actions_sz, actions, ft_type);
return action;
+#endif
#else
(void)ctx;
+ (void)ft_type;
+ (void)ns;
+ (void)flags;
(void)actions_sz;
(void)actions;
- (void)ft_type;
return NULL;
#endif
}
@@ -539,12 +614,20 @@
static void *
mlx5_glue_dv_create_flow_action_packet_reformat
(struct ibv_context *ctx,
- size_t data_sz,
- void *data,
enum mlx5dv_flow_action_packet_reformat_type reformat_type,
- enum mlx5dv_flow_table_type ft_type)
+ enum mlx5dv_flow_table_type ft_type, struct mlx5dv_dr_ns *ns,
+ uint32_t flags, size_t data_sz, void *data)
{
#ifdef HAVE_IBV_FLOW_DV_SUPPORT
+#ifdef HAVE_MLX5DV_DR
+ (void)ctx;
+ (void)ft_type;
+ return mlx5dv_dr_create_action_packet_reformat(ns, flags,
+ reformat_type, data_sz,
+ data);
+#else
+ (void)ns;
+ (void)flags;
struct mlx5dv_flow_action_attr *action;
action = malloc(sizeof(*action));
@@ -554,12 +637,15 @@
action->action = mlx5dv_create_flow_action_packet_reformat
(ctx, data_sz, data, reformat_type, ft_type);
return action;
+#endif
#else
(void)ctx;
- (void)data_sz;
- (void)data;
(void)reformat_type;
(void)ft_type;
+ (void)ns;
+ (void)flags;
+ (void)data_sz;
+ (void)data;
return NULL;
#endif
}
@@ -568,6 +654,9 @@
mlx5_glue_dv_create_flow_action_tag(uint32_t tag)
{
#ifdef HAVE_IBV_FLOW_DV_SUPPORT
+#ifdef HAVE_MLX5DV_DR
+ return mlx5dv_dr_create_action_tag(tag);
+#else
struct mlx5dv_flow_action_attr *action;
action = malloc(sizeof(*action));
if (!action)
@@ -576,10 +665,36 @@
action->tag_value = tag;
return action;
#endif
+#endif
(void)tag;
return NULL;
}
+static int
+mlx5_glue_dv_destroy_flow(void *flow_id)
+{
+#ifdef HAVE_MLX5DV_DR
+ return mlx5dv_dr_destroy_rule(flow_id);
+#else
+ return ibv_destroy_flow(flow_id);
+#endif
+}
+
+static int
+mlx5_glue_dv_destroy_flow_matcher(void *matcher)
+{
+#ifdef HAVE_IBV_FLOW_DV_SUPPORT
+#ifdef HAVE_MLX5DV_DR
+ return mlx5dv_dr_destroy_matcher(matcher);
+#else
+ return mlx5dv_destroy_flow_matcher(matcher);
+#endif
+#else
+ (void)matcher;
+ return 0;
+#endif
+}
+
static struct ibv_context *
mlx5_glue_dv_open_device(struct ibv_device *device)
{
@@ -718,6 +833,10 @@
.get_async_event = mlx5_glue_get_async_event,
.port_state_str = mlx5_glue_port_state_str,
.cq_ex_to_cq = mlx5_glue_cq_ex_to_cq,
+ .dr_create_flow_tbl = mlx5_glue_dr_create_flow_tbl,
+ .dr_destroy_flow_tbl = mlx5_glue_dr_destroy_flow_tbl,
+ .dr_create_ns = mlx5_glue_dr_create_ns,
+ .dr_destroy_ns = mlx5_glue_dr_destroy_ns,
.dv_create_cq = mlx5_glue_dv_create_cq,
.dv_create_wq = mlx5_glue_dv_create_wq,
.dv_query_device = mlx5_glue_dv_query_device,
@@ -725,7 +844,6 @@
.dv_init_obj = mlx5_glue_dv_init_obj,
.dv_create_qp = mlx5_glue_dv_create_qp,
.dv_create_flow_matcher = mlx5_glue_dv_create_flow_matcher,
- .dv_destroy_flow_matcher = mlx5_glue_dv_destroy_flow_matcher,
.dv_create_flow = mlx5_glue_dv_create_flow,
.dv_create_flow_action_counter =
mlx5_glue_dv_create_flow_action_counter,
@@ -736,6 +854,8 @@
.dv_create_flow_action_packet_reformat =
mlx5_glue_dv_create_flow_action_packet_reformat,
.dv_create_flow_action_tag = mlx5_glue_dv_create_flow_action_tag,
+ .dv_destroy_flow = mlx5_glue_dv_destroy_flow,
+ .dv_destroy_flow_matcher = mlx5_glue_dv_destroy_flow_matcher,
.dv_open_device = mlx5_glue_dv_open_device,
.devx_obj_create = mlx5_glue_devx_obj_create,
.devx_obj_destroy = mlx5_glue_devx_obj_destroy,
diff --git a/drivers/net/mlx5/mlx5_glue.h b/drivers/net/mlx5/mlx5_glue.h
index 167f1f7..7115575 100644
--- a/drivers/net/mlx5/mlx5_glue.h
+++ b/drivers/net/mlx5/mlx5_glue.h
@@ -63,6 +63,11 @@
struct mlx5dv_devx_obj;
#endif
+#ifndef HAVE_MLX5DV_DR
+struct mlx5dv_dr_ns;
+enum mlx5dv_dr_ns_domain { unused, };
+#endif
+
/* LIB_GLUE_VERSION must be updated every time this structure is modified. */
struct mlx5_glue {
const char *version;
@@ -140,6 +145,11 @@ struct mlx5_glue {
struct ibv_async_event *event);
const char *(*port_state_str)(enum ibv_port_state port_state);
struct ibv_cq *(*cq_ex_to_cq)(struct ibv_cq_ex *cq);
+ void *(*dr_create_flow_tbl)(void *ns, uint32_t level);
+ int (*dr_destroy_flow_tbl)(void *tbl);
+ void *(*dr_create_ns)(struct ibv_context *ctx,
+ enum mlx5dv_dr_ns_domain domain);
+ int (*dr_destroy_ns)(void *ns);
struct ibv_cq_ex *(*dv_create_cq)
(struct ibv_context *context,
struct ibv_cq_init_attr_ex *cq_attr,
@@ -158,23 +168,26 @@ struct mlx5_glue {
(struct ibv_context *context,
struct ibv_qp_init_attr_ex *qp_init_attr_ex,
struct mlx5dv_qp_init_attr *dv_qp_init_attr);
- struct mlx5dv_flow_matcher *(*dv_create_flow_matcher)
+ void *(*dv_create_flow_matcher)
(struct ibv_context *context,
- struct mlx5dv_flow_matcher_attr *matcher_attr);
- int (*dv_destroy_flow_matcher)(struct mlx5dv_flow_matcher *matcher);
- struct ibv_flow *(*dv_create_flow)(struct mlx5dv_flow_matcher *matcher,
- struct mlx5dv_flow_match_parameters *match_value,
+ struct mlx5dv_flow_matcher_attr *matcher_attr,
+ void *tbl);
+ void *(*dv_create_flow)(void *matcher, void *match_value,
size_t num_actions, void *actions[]);
void *(*dv_create_flow_action_counter)(void *obj, uint32_t offset);
void *(*dv_create_flow_action_dest_ibv_qp)(void *qp);
void *(*dv_create_flow_action_modify_header)
- (struct ibv_context *ctx, size_t actions_sz, uint64_t actions[],
- enum mlx5dv_flow_table_type ft_type);
+ (struct ibv_context *ctx, enum mlx5dv_flow_table_type ft_type,
+ void *ns, uint64_t flags, size_t actions_sz,
+ uint64_t actions[]);
void *(*dv_create_flow_action_packet_reformat)
- (struct ibv_context *ctx, size_t data_sz, void *data,
+ (struct ibv_context *ctx,
enum mlx5dv_flow_action_packet_reformat_type reformat_type,
- enum mlx5dv_flow_table_type ft_type);
+ enum mlx5dv_flow_table_type ft_type, struct mlx5dv_dr_ns *ns,
+ uint32_t flags, size_t data_sz, void *data);
void *(*dv_create_flow_action_tag)(uint32_t tag);
+ int (*dv_destroy_flow)(void *flow);
+ int (*dv_destroy_flow_matcher)(void *matcher);
struct ibv_context *(*dv_open_device)(struct ibv_device *device);
struct mlx5dv_devx_obj *(*devx_obj_create)
(struct ibv_context *ctx,
diff --git a/drivers/net/mlx5/mlx5_prm.h b/drivers/net/mlx5/mlx5_prm.h
index da1219e..b15266f 100644
--- a/drivers/net/mlx5/mlx5_prm.h
+++ b/drivers/net/mlx5/mlx5_prm.h
@@ -492,20 +492,40 @@ struct mlx5_ifc_fte_match_set_misc2_bits {
u8 reserved_at_1a0[0x60];
};
+struct mlx5_ifc_fte_match_set_misc3_bits {
+ u8 inner_tcp_seq_num[0x20];
+ u8 outer_tcp_seq_num[0x20];
+ u8 inner_tcp_ack_num[0x20];
+ u8 outer_tcp_ack_num[0x20];
+ u8 reserved_at_auto1[0x8];
+ u8 outer_vxlan_gpe_vni[0x18];
+ u8 outer_vxlan_gpe_next_protocol[0x8];
+ u8 outer_vxlan_gpe_flags[0x8];
+ u8 reserved_at_a8[0x10];
+ u8 icmp_header_data[0x20];
+ u8 icmpv6_header_data[0x20];
+ u8 icmp_type[0x8];
+ u8 icmp_code[0x8];
+ u8 icmpv6_type[0x8];
+ u8 icmpv6_code[0x8];
+ u8 reserved_at_1a0[0xe0];
+};
+
/* Flow matcher. */
struct mlx5_ifc_fte_match_param_bits {
struct mlx5_ifc_fte_match_set_lyr_2_4_bits outer_headers;
struct mlx5_ifc_fte_match_set_misc_bits misc_parameters;
struct mlx5_ifc_fte_match_set_lyr_2_4_bits inner_headers;
struct mlx5_ifc_fte_match_set_misc2_bits misc_parameters_2;
- u8 reserved_at_800[0x800];
+ struct mlx5_ifc_fte_match_set_misc3_bits misc_parameters_3;
};
enum {
MLX5_MATCH_CRITERIA_ENABLE_OUTER_BIT,
MLX5_MATCH_CRITERIA_ENABLE_MISC_BIT,
MLX5_MATCH_CRITERIA_ENABLE_INNER_BIT,
- MLX5_MATCH_CRITERIA_ENABLE_MISC2_BIT
+ MLX5_MATCH_CRITERIA_ENABLE_MISC2_BIT,
+ MLX5_MATCH_CRITERIA_ENABLE_MISC3_BIT
};
enum {
--
1.8.3.1
^ permalink raw reply [flat|nested] 52+ messages in thread
* [dpdk-dev] [PATCH v2 3/3] net/mlx5: add jump action support for NIC
2019-03-28 16:32 ` [dpdk-dev] [PATCH v2 0/3] net/mlx5: Add Direct Rule support Ori Kam
` (2 preceding siblings ...)
2019-03-28 16:32 ` [dpdk-dev] [PATCH v2 2/3] net/mlx5: add Direct Rules API Ori Kam
@ 2019-03-28 16:32 ` Ori Kam
2019-03-28 16:32 ` Ori Kam
` (2 more replies)
2019-04-03 10:17 ` [dpdk-dev] [PATCH v2 0/3] net/mlx5: Add Direct Rule support Shahaf Shuler
` (2 subsequent siblings)
6 siblings, 3 replies; 52+ messages in thread
From: Ori Kam @ 2019-03-28 16:32 UTC (permalink / raw)
To: Matan Azrad, Yongseok Koh, Shahaf Shuler; +Cc: dev, Ori Kam, Slava Ovsiienko
When using Direct Rules we can add actions to jump between tables.
This is extra useful since rule insertion rate is much higher on other
tables compared to table zero.
if no group is selected the rule is added to group 0.
Signed-off-by: Ori Kam <orika@mellanox.com>
---
drivers/net/mlx5/mlx5.h | 6 +
drivers/net/mlx5/mlx5_flow.h | 15 ++-
drivers/net/mlx5/mlx5_flow_dv.c | 278 +++++++++++++++++++++++++++++++++++-----
drivers/net/mlx5/mlx5_glue.c | 13 ++
drivers/net/mlx5/mlx5_glue.h | 1 +
5 files changed, 282 insertions(+), 31 deletions(-)
diff --git a/drivers/net/mlx5/mlx5.h b/drivers/net/mlx5/mlx5.h
index 73f6f0d..a3d5f8e 100644
--- a/drivers/net/mlx5/mlx5.h
+++ b/drivers/net/mlx5/mlx5.h
@@ -279,6 +279,12 @@ struct mlx5_priv {
LIST_HEAD(encap_decap, mlx5_flow_dv_encap_decap_resource) encaps_decaps;
LIST_HEAD(modify_cmd, mlx5_flow_dv_modify_hdr_resource) modify_cmds;
LIST_HEAD(tag, mlx5_flow_dv_tag_resource) tags;
+ LIST_HEAD(jump, mlx5_flow_dv_jump_tbl_resource) jump_tbl;
+ /* Pointer to next element. */
+ rte_atomic32_t refcnt; /**< Reference counter. */
+ struct ibv_flow_action *verbs_action;
+ /**< Verbs modify header action object. */
+ uint8_t ft_type; /**< Flow table type, Rx or Tx. */
/* Tags resources cache. */
uint32_t link_speed_capa; /* Link speed capabilities. */
struct mlx5_xstats_ctrl xstats_ctrl; /* Extended stats control. */
diff --git a/drivers/net/mlx5/mlx5_flow.h b/drivers/net/mlx5/mlx5_flow.h
index 8ba37a0..622e305 100644
--- a/drivers/net/mlx5/mlx5_flow.h
+++ b/drivers/net/mlx5/mlx5_flow.h
@@ -115,7 +115,8 @@
#define MLX5_FLOW_ACTION_RAW_DECAP (1u << 27)
#define MLX5_FLOW_FATE_ACTIONS \
- (MLX5_FLOW_ACTION_DROP | MLX5_FLOW_ACTION_QUEUE | MLX5_FLOW_ACTION_RSS)
+ (MLX5_FLOW_ACTION_DROP | MLX5_FLOW_ACTION_QUEUE | \
+ MLX5_FLOW_ACTION_RSS | MLX5_FLOW_ACTION_JUMP)
#define MLX5_FLOW_ENCAP_ACTIONS (MLX5_FLOW_ACTION_VXLAN_ENCAP | \
MLX5_FLOW_ACTION_NVGRE_ENCAP | \
@@ -250,6 +251,16 @@ struct mlx5_flow_dv_modify_hdr_resource {
/**< Modification actions. */
};
+/* Jump action resource structure. */
+struct mlx5_flow_dv_jump_tbl_resource {
+ LIST_ENTRY(mlx5_flow_dv_jump_tbl_resource) next;
+ /* Pointer to next element. */
+ rte_atomic32_t refcnt; /**< Reference counter. */
+ void *action; /**< Pointer to the rdma core action. */
+ uint8_t ft_type; /**< Flow table type, Rx or Tx. */
+ struct mlx5_flow_tbl_resource *tbl; /**< The target table. */
+};
+
/*
* Max number of actions per DV flow.
* See CREATE_FLOW_MAX_FLOW_ACTIONS_SUPPORTED
@@ -270,6 +281,8 @@ struct mlx5_flow_dv {
struct mlx5_flow_dv_modify_hdr_resource *modify_hdr;
/**< Pointer to modify header resource in cache. */
struct ibv_flow *flow; /**< Installed flow. */
+ struct mlx5_flow_dv_jump_tbl_resource *jump;
+ /**< Pointer to the jump action resource. */
#ifdef HAVE_IBV_FLOW_DV_SUPPORT
void *actions[MLX5_DV_MAX_NUMBER_OF_ACTIONS];
/**< Action list. */
diff --git a/drivers/net/mlx5/mlx5_flow_dv.c b/drivers/net/mlx5/mlx5_flow_dv.c
index 6e4f6c4..6997dc6 100644
--- a/drivers/net/mlx5/mlx5_flow_dv.c
+++ b/drivers/net/mlx5/mlx5_flow_dv.c
@@ -861,6 +861,68 @@ struct field_modify_info modify_tcp[] = {
}
/**
+ * Find existing table jump resource or create and register a new one.
+ *
+ * @param dev[in, out]
+ * Pointer to rte_eth_dev structure.
+ * @param[in, out] resource
+ * Pointer to jump table resource.
+ * @parm[in, out] dev_flow
+ * Pointer to the dev_flow.
+ * @param[out] error
+ * pointer to error structure.
+ *
+ * @return
+ * 0 on success otherwise -errno and errno is set.
+ */
+static int
+flow_dv_jump_tbl_resource_register
+ (struct rte_eth_dev *dev,
+ struct mlx5_flow_dv_jump_tbl_resource *resource,
+ struct mlx5_flow *dev_flow,
+ struct rte_flow_error *error)
+{
+ struct mlx5_priv *priv = dev->data->dev_private;
+ struct mlx5_flow_dv_jump_tbl_resource *cache_resource;
+
+ /* Lookup a matching resource from cache. */
+ LIST_FOREACH(cache_resource, &priv->jump_tbl, next) {
+ if (resource->tbl == cache_resource->tbl) {
+ DRV_LOG(DEBUG, "jump table resource resource %p: refcnt %d++",
+ (void *)cache_resource,
+ rte_atomic32_read(&cache_resource->refcnt));
+ rte_atomic32_inc(&cache_resource->refcnt);
+ dev_flow->dv.jump = cache_resource;
+ return 0;
+ }
+ }
+ /* Register new jump table resource. */
+ cache_resource = rte_calloc(__func__, 1, sizeof(*cache_resource), 0);
+ if (!cache_resource)
+ return rte_flow_error_set(error, ENOMEM,
+ RTE_FLOW_ERROR_TYPE_UNSPECIFIED, NULL,
+ "cannot allocate resource memory");
+ *cache_resource = *resource;
+ cache_resource->action =
+ mlx5_glue->dr_create_flow_action_dest_flow_tbl
+ (resource->tbl->obj);
+ if (!cache_resource->action) {
+ rte_free(cache_resource);
+ return rte_flow_error_set(error, ENOMEM,
+ RTE_FLOW_ERROR_TYPE_UNSPECIFIED,
+ NULL, "cannot create action");
+ }
+ rte_atomic32_init(&cache_resource->refcnt);
+ rte_atomic32_inc(&cache_resource->refcnt);
+ LIST_INSERT_HEAD(&priv->jump_tbl, cache_resource, next);
+ dev_flow->dv.jump = cache_resource;
+ DRV_LOG(DEBUG, "new jump table resource %p: refcnt %d++",
+ (void *)cache_resource,
+ rte_atomic32_read(&cache_resource->refcnt));
+ return 0;
+}
+
+/**
* Get the size of specific rte_flow_item_type
*
* @param[in] item_type
@@ -1423,6 +1485,37 @@ struct field_modify_info modify_tcp[] = {
}
/**
+ * Validate jump action.
+ *
+ * @param[in] action
+ * Pointer to the modify action.
+ * @param[in] group
+ * The group of the current flow.
+ * @param[out] error
+ * Pointer to error structure.
+ *
+ * @return
+ * 0 on success, a negative errno value otherwise and rte_errno is set.
+ */
+static int
+flow_dv_validate_action_jump(const struct rte_flow_action *action,
+ uint32_t group,
+ struct rte_flow_error *error)
+{
+ if (action->type != RTE_FLOW_ACTION_TYPE_JUMP && !action->conf)
+ return rte_flow_error_set(error, EINVAL,
+ RTE_FLOW_ERROR_TYPE_ACTION_CONF,
+ NULL, "action configuration not set");
+ if (group >= ((const struct rte_flow_action_jump *)action->conf)->group)
+ return rte_flow_error_set(error, EINVAL,
+ RTE_FLOW_ERROR_TYPE_ACTION, NULL,
+ "target group must be higher then"
+ " the current flow group");
+ return 0;
+}
+
+
+/**
* Find existing modify-header resource or create and register a new one.
*
* @param dev[in, out]
@@ -1605,7 +1698,7 @@ struct field_modify_info modify_tcp[] = {
struct mlx5_priv *priv = dev->data->dev_private;
uint32_t priority_max = priv->config.flow_prio - 1;
-#ifdef HAVE_MLX5DV_DR
+#ifndef HAVE_MLX5DV_DR
if (attributes->group)
return rte_flow_error_set(error, ENOTSUP,
RTE_FLOW_ERROR_TYPE_ATTR_GROUP,
@@ -1977,6 +2070,14 @@ struct field_modify_info modify_tcp[] = {
MLX5_FLOW_ACTION_SET_TTL :
MLX5_FLOW_ACTION_DEC_TTL;
break;
+ case RTE_FLOW_ACTION_TYPE_JUMP:
+ ret = flow_dv_validate_action_jump(actions,
+ attr->group, error);
+ if (ret)
+ return ret;
+ ++actions_n;
+ action_flags |= MLX5_FLOW_ACTION_JUMP;
+ break;
default:
return rte_flow_error_set(error, ENOTSUP,
RTE_FLOW_ERROR_TYPE_ACTION,
@@ -2756,6 +2857,73 @@ struct field_modify_info modify_tcp[] = {
return match_criteria_enable;
}
+
+/**
+ * Get a flow table.
+ *
+ * @param dev[in, out]
+ * Pointer to rte_eth_dev structure.
+ * @param[in] table_id
+ * Table id to use.
+ * @param[in] egress
+ * Direction of the table.
+ * @param[out] error
+ * pointer to error structure.
+ *
+ * @return
+ * Returns tables resource based on the index, NULL in case of failed.
+ */
+static struct mlx5_flow_tbl_resource *
+flow_dv_tbl_resource_get(struct rte_eth_dev *dev,
+ uint32_t table_id, uint8_t egress,
+ struct rte_flow_error *error)
+{
+ struct mlx5_priv *priv = dev->data->dev_private;
+ struct mlx5_flow_tbl_resource *tbl;
+
+ if (egress) {
+ tbl = &priv->tx_tbl[table_id];
+ if (!tbl->obj)
+ tbl->obj = mlx5_glue->dr_create_flow_tbl
+ (priv->tx_ns, table_id);
+ } else {
+ tbl = &priv->rx_tbl[table_id];
+ if (!tbl->obj)
+ tbl->obj = mlx5_glue->dr_create_flow_tbl
+ (priv->rx_ns, table_id);
+ }
+ if (!tbl->obj) {
+ rte_flow_error_set(error, ENOMEM,
+ RTE_FLOW_ERROR_TYPE_UNSPECIFIED,
+ NULL, "cannot create table");
+ return NULL;
+ }
+ rte_atomic32_inc(&tbl->refcnt);
+ return tbl;
+}
+
+/**
+ * Release a flow table.
+ *
+ * @param[in] tbl
+ * Table resource to be released.
+ *
+ * @return
+ * Returns 0 if table was released, else return 1;
+ */
+static int
+flow_dv_tbl_resource_release(struct mlx5_flow_tbl_resource *tbl)
+{
+ if (!tbl)
+ return 0;
+ if (rte_atomic32_dec_and_test(&tbl->refcnt)) {
+ mlx5_glue->dr_destroy_flow_tbl(tbl->obj);
+ tbl->obj = NULL;
+ return 0;
+ }
+ return 1;
+}
+
/**
* Register the flow matcher.
*
@@ -2784,6 +2952,9 @@ struct field_modify_info modify_tcp[] = {
.match_mask = (void *)&matcher->mask,
};
struct mlx5_flow_tbl_resource *tbl = NULL;
+#ifndef HAVE_MLX5DV_DR
+ struct mlx5_flow_tbl_resource tbl_tmp;
+#endif
/* Lookup from cache. */
LIST_FOREACH(cache_matcher, &priv->matchers, next) {
@@ -2805,33 +2976,24 @@ struct field_modify_info modify_tcp[] = {
return 0;
}
}
-#ifdef HAVE_MLX5DV_DR
- if (matcher->egress) {
- tbl = &priv->tx_tbl[matcher->group];
- if (!tbl->obj)
- tbl->obj = mlx5_glue->dr_create_flow_tbl
- (priv->tx_ns,
- matcher->group * MLX5_GROUP_FACTOR);
- } else {
- tbl = &priv->rx_tbl[matcher->group];
- if (!tbl->obj)
- tbl->obj = mlx5_glue->dr_create_flow_tbl
- (priv->rx_ns,
- matcher->group * MLX5_GROUP_FACTOR);
- }
- if (!tbl->obj)
- return rte_flow_error_set(error, ENOMEM,
- RTE_FLOW_ERROR_TYPE_UNSPECIFIED,
- NULL, "cannot create table");
-
- rte_atomic32_inc(&tbl->refcnt);
-#endif
/* Register new matcher. */
cache_matcher = rte_calloc(__func__, 1, sizeof(*cache_matcher), 0);
if (!cache_matcher)
return rte_flow_error_set(error, ENOMEM,
RTE_FLOW_ERROR_TYPE_UNSPECIFIED, NULL,
"cannot allocate matcher memory");
+#ifdef HAVE_MLX5DV_DR
+ tbl = flow_dv_tbl_resource_get(dev, matcher->group * MLX5_GROUP_FACTOR,
+ matcher->egress, error);
+ if (!tbl) {
+ rte_free(cache_matcher);
+ return rte_flow_error_set(error, ENOMEM,
+ RTE_FLOW_ERROR_TYPE_UNSPECIFIED,
+ NULL, "cannot create table");
+ }
+#else
+ tbl = &tbl_tmp;
+#endif
*cache_matcher = *matcher;
dv_attr.match_criteria_enable =
flow_dv_matcher_enable(cache_matcher->mask.buf);
@@ -2844,10 +3006,7 @@ struct field_modify_info modify_tcp[] = {
if (!cache_matcher->matcher_object) {
rte_free(cache_matcher);
#ifdef HAVE_MLX5DV_DR
- if (rte_atomic32_dec_and_test(&tbl->refcnt)) {
- mlx5_glue->dr_destroy_flow_tbl(tbl->obj);
- tbl->obj = NULL;
- }
+ flow_dv_tbl_resource_release(tbl);
#endif
return rte_flow_error_set(error, ENOMEM,
RTE_FLOW_ERROR_TYPE_UNSPECIFIED,
@@ -3033,6 +3192,9 @@ struct field_modify_info modify_tcp[] = {
const struct rte_flow_action *action = actions;
const struct rte_flow_action_count *count = action->conf;
const uint8_t *rss_key;
+ const struct rte_flow_action_jump *jump_data;
+ struct mlx5_flow_dv_jump_tbl_resource jump_tbl_resource;
+ struct mlx5_flow_tbl_resource *tbl;
switch (actions->type) {
case RTE_FLOW_ACTION_TYPE_VOID:
@@ -3171,6 +3333,31 @@ struct field_modify_info modify_tcp[] = {
/* If decap is followed by encap, handle it at encap. */
action_flags |= MLX5_FLOW_ACTION_RAW_DECAP;
break;
+ case RTE_FLOW_ACTION_TYPE_JUMP:
+ jump_data = action->conf;
+ tbl = flow_dv_tbl_resource_get(dev, jump_data->group *
+ MLX5_GROUP_FACTOR,
+ attr->egress, error);
+ if (!tbl)
+ return rte_flow_error_set
+ (error, errno,
+ RTE_FLOW_ERROR_TYPE_ACTION,
+ NULL,
+ "cannot create jump action.");
+ jump_tbl_resource.tbl = tbl;
+ if (flow_dv_jump_tbl_resource_register
+ (dev, &jump_tbl_resource, dev_flow, error)) {
+ flow_dv_tbl_resource_release(tbl);
+ return rte_flow_error_set
+ (error, errno,
+ RTE_FLOW_ERROR_TYPE_ACTION,
+ NULL,
+ "cannot create jump action.");
+ }
+ dev_flow->dv.actions[actions_n++] =
+ dev_flow->dv.jump->action;
+ action_flags |= MLX5_FLOW_ACTION_JUMP;
+ break;
case RTE_FLOW_ACTION_TYPE_SET_MAC_SRC:
case RTE_FLOW_ACTION_TYPE_SET_MAC_DST:
if (flow_dv_convert_action_modify_mac(&res, actions,
@@ -3503,10 +3690,7 @@ struct field_modify_info modify_tcp[] = {
tbl = &priv->tx_tbl[matcher->group];
else
tbl = &priv->rx_tbl[matcher->group];
- if (rte_atomic32_dec_and_test(&tbl->refcnt)) {
- mlx5_glue->dr_destroy_flow_tbl(tbl->obj);
- tbl->obj = NULL;
- }
+ flow_dv_tbl_resource_release(tbl);
rte_free(matcher);
DRV_LOG(DEBUG, "port %u matcher %p: removed",
dev->data->port_id, (void *)matcher);
@@ -3547,6 +3731,38 @@ struct field_modify_info modify_tcp[] = {
}
/**
+ * Release an jump to table action resource.
+ *
+ * @param flow
+ * Pointer to mlx5_flow.
+ *
+ * @return
+ * 1 while a reference on it exists, 0 when freed.
+ */
+static int
+flow_dv_jump_tbl_resource_release(struct mlx5_flow *flow)
+{
+ struct mlx5_flow_dv_jump_tbl_resource *cache_resource =
+ flow->dv.jump;
+
+ assert(cache_resource->action);
+ DRV_LOG(DEBUG, "jump table resource %p: refcnt %d--",
+ (void *)cache_resource,
+ rte_atomic32_read(&cache_resource->refcnt));
+ if (rte_atomic32_dec_and_test(&cache_resource->refcnt)) {
+ claim_zero(mlx5_glue->destroy_flow_action
+ (cache_resource->action));
+ LIST_REMOVE(cache_resource, next);
+ flow_dv_tbl_resource_release(cache_resource->tbl);
+ rte_free(cache_resource);
+ DRV_LOG(DEBUG, "jump table resource %p: removed",
+ (void *)cache_resource);
+ return 0;
+ }
+ return 1;
+}
+
+/**
* Release a modify-header resource.
*
* @param flow
@@ -3642,6 +3858,8 @@ struct field_modify_info modify_tcp[] = {
flow_dv_encap_decap_resource_release(dev_flow);
if (dev_flow->dv.modify_hdr)
flow_dv_modify_hdr_resource_release(dev_flow);
+ if (dev_flow->dv.jump)
+ flow_dv_jump_tbl_resource_release(dev_flow);
rte_free(dev_flow);
}
}
diff --git a/drivers/net/mlx5/mlx5_glue.c b/drivers/net/mlx5/mlx5_glue.c
index f509f85..fca34ca 100644
--- a/drivers/net/mlx5/mlx5_glue.c
+++ b/drivers/net/mlx5/mlx5_glue.c
@@ -370,6 +370,17 @@
}
static void *
+mlx5_glue_dr_create_flow_action_dest_flow_tbl(void *tbl)
+{
+#ifdef HAVE_MLX5DV_DR
+ return mlx5dv_dr_create_action_dest_flow_table(tbl);
+#else
+ (void)tbl;
+ return NULL;
+#endif
+}
+
+static void *
mlx5_glue_dr_create_flow_tbl(void *ns, uint32_t level)
{
#ifdef HAVE_MLX5DV_DR
@@ -833,6 +844,8 @@
.get_async_event = mlx5_glue_get_async_event,
.port_state_str = mlx5_glue_port_state_str,
.cq_ex_to_cq = mlx5_glue_cq_ex_to_cq,
+ .dr_create_flow_action_dest_flow_tbl =
+ mlx5_glue_dr_create_flow_action_dest_flow_tbl,
.dr_create_flow_tbl = mlx5_glue_dr_create_flow_tbl,
.dr_destroy_flow_tbl = mlx5_glue_dr_destroy_flow_tbl,
.dr_create_ns = mlx5_glue_dr_create_ns,
diff --git a/drivers/net/mlx5/mlx5_glue.h b/drivers/net/mlx5/mlx5_glue.h
index 7115575..16d5dd6 100644
--- a/drivers/net/mlx5/mlx5_glue.h
+++ b/drivers/net/mlx5/mlx5_glue.h
@@ -145,6 +145,7 @@ struct mlx5_glue {
struct ibv_async_event *event);
const char *(*port_state_str)(enum ibv_port_state port_state);
struct ibv_cq *(*cq_ex_to_cq)(struct ibv_cq_ex *cq);
+ void *(*dr_create_flow_action_dest_flow_tbl)(void *tbl);
void *(*dr_create_flow_tbl)(void *ns, uint32_t level);
int (*dr_destroy_flow_tbl)(void *tbl);
void *(*dr_create_ns)(struct ibv_context *ctx,
--
1.8.3.1
^ permalink raw reply [flat|nested] 52+ messages in thread
* [dpdk-dev] [PATCH v2 3/3] net/mlx5: add jump action support for NIC
2019-03-28 16:32 ` [dpdk-dev] [PATCH v2 3/3] net/mlx5: add jump action support for NIC Ori Kam
@ 2019-03-28 16:32 ` Ori Kam
2019-04-01 14:38 ` Slava Ovsiienko
2019-04-03 10:16 ` Shahaf Shuler
2 siblings, 0 replies; 52+ messages in thread
From: Ori Kam @ 2019-03-28 16:32 UTC (permalink / raw)
To: Matan Azrad, Yongseok Koh, Shahaf Shuler; +Cc: dev, Ori Kam, Slava Ovsiienko
When using Direct Rules we can add actions to jump between tables.
This is extra useful since rule insertion rate is much higher on other
tables compared to table zero.
if no group is selected the rule is added to group 0.
Signed-off-by: Ori Kam <orika@mellanox.com>
---
drivers/net/mlx5/mlx5.h | 6 +
drivers/net/mlx5/mlx5_flow.h | 15 ++-
drivers/net/mlx5/mlx5_flow_dv.c | 278 +++++++++++++++++++++++++++++++++++-----
drivers/net/mlx5/mlx5_glue.c | 13 ++
drivers/net/mlx5/mlx5_glue.h | 1 +
5 files changed, 282 insertions(+), 31 deletions(-)
diff --git a/drivers/net/mlx5/mlx5.h b/drivers/net/mlx5/mlx5.h
index 73f6f0d..a3d5f8e 100644
--- a/drivers/net/mlx5/mlx5.h
+++ b/drivers/net/mlx5/mlx5.h
@@ -279,6 +279,12 @@ struct mlx5_priv {
LIST_HEAD(encap_decap, mlx5_flow_dv_encap_decap_resource) encaps_decaps;
LIST_HEAD(modify_cmd, mlx5_flow_dv_modify_hdr_resource) modify_cmds;
LIST_HEAD(tag, mlx5_flow_dv_tag_resource) tags;
+ LIST_HEAD(jump, mlx5_flow_dv_jump_tbl_resource) jump_tbl;
+ /* Pointer to next element. */
+ rte_atomic32_t refcnt; /**< Reference counter. */
+ struct ibv_flow_action *verbs_action;
+ /**< Verbs modify header action object. */
+ uint8_t ft_type; /**< Flow table type, Rx or Tx. */
/* Tags resources cache. */
uint32_t link_speed_capa; /* Link speed capabilities. */
struct mlx5_xstats_ctrl xstats_ctrl; /* Extended stats control. */
diff --git a/drivers/net/mlx5/mlx5_flow.h b/drivers/net/mlx5/mlx5_flow.h
index 8ba37a0..622e305 100644
--- a/drivers/net/mlx5/mlx5_flow.h
+++ b/drivers/net/mlx5/mlx5_flow.h
@@ -115,7 +115,8 @@
#define MLX5_FLOW_ACTION_RAW_DECAP (1u << 27)
#define MLX5_FLOW_FATE_ACTIONS \
- (MLX5_FLOW_ACTION_DROP | MLX5_FLOW_ACTION_QUEUE | MLX5_FLOW_ACTION_RSS)
+ (MLX5_FLOW_ACTION_DROP | MLX5_FLOW_ACTION_QUEUE | \
+ MLX5_FLOW_ACTION_RSS | MLX5_FLOW_ACTION_JUMP)
#define MLX5_FLOW_ENCAP_ACTIONS (MLX5_FLOW_ACTION_VXLAN_ENCAP | \
MLX5_FLOW_ACTION_NVGRE_ENCAP | \
@@ -250,6 +251,16 @@ struct mlx5_flow_dv_modify_hdr_resource {
/**< Modification actions. */
};
+/* Jump action resource structure. */
+struct mlx5_flow_dv_jump_tbl_resource {
+ LIST_ENTRY(mlx5_flow_dv_jump_tbl_resource) next;
+ /* Pointer to next element. */
+ rte_atomic32_t refcnt; /**< Reference counter. */
+ void *action; /**< Pointer to the rdma core action. */
+ uint8_t ft_type; /**< Flow table type, Rx or Tx. */
+ struct mlx5_flow_tbl_resource *tbl; /**< The target table. */
+};
+
/*
* Max number of actions per DV flow.
* See CREATE_FLOW_MAX_FLOW_ACTIONS_SUPPORTED
@@ -270,6 +281,8 @@ struct mlx5_flow_dv {
struct mlx5_flow_dv_modify_hdr_resource *modify_hdr;
/**< Pointer to modify header resource in cache. */
struct ibv_flow *flow; /**< Installed flow. */
+ struct mlx5_flow_dv_jump_tbl_resource *jump;
+ /**< Pointer to the jump action resource. */
#ifdef HAVE_IBV_FLOW_DV_SUPPORT
void *actions[MLX5_DV_MAX_NUMBER_OF_ACTIONS];
/**< Action list. */
diff --git a/drivers/net/mlx5/mlx5_flow_dv.c b/drivers/net/mlx5/mlx5_flow_dv.c
index 6e4f6c4..6997dc6 100644
--- a/drivers/net/mlx5/mlx5_flow_dv.c
+++ b/drivers/net/mlx5/mlx5_flow_dv.c
@@ -861,6 +861,68 @@ struct field_modify_info modify_tcp[] = {
}
/**
+ * Find existing table jump resource or create and register a new one.
+ *
+ * @param dev[in, out]
+ * Pointer to rte_eth_dev structure.
+ * @param[in, out] resource
+ * Pointer to jump table resource.
+ * @parm[in, out] dev_flow
+ * Pointer to the dev_flow.
+ * @param[out] error
+ * pointer to error structure.
+ *
+ * @return
+ * 0 on success otherwise -errno and errno is set.
+ */
+static int
+flow_dv_jump_tbl_resource_register
+ (struct rte_eth_dev *dev,
+ struct mlx5_flow_dv_jump_tbl_resource *resource,
+ struct mlx5_flow *dev_flow,
+ struct rte_flow_error *error)
+{
+ struct mlx5_priv *priv = dev->data->dev_private;
+ struct mlx5_flow_dv_jump_tbl_resource *cache_resource;
+
+ /* Lookup a matching resource from cache. */
+ LIST_FOREACH(cache_resource, &priv->jump_tbl, next) {
+ if (resource->tbl == cache_resource->tbl) {
+ DRV_LOG(DEBUG, "jump table resource resource %p: refcnt %d++",
+ (void *)cache_resource,
+ rte_atomic32_read(&cache_resource->refcnt));
+ rte_atomic32_inc(&cache_resource->refcnt);
+ dev_flow->dv.jump = cache_resource;
+ return 0;
+ }
+ }
+ /* Register new jump table resource. */
+ cache_resource = rte_calloc(__func__, 1, sizeof(*cache_resource), 0);
+ if (!cache_resource)
+ return rte_flow_error_set(error, ENOMEM,
+ RTE_FLOW_ERROR_TYPE_UNSPECIFIED, NULL,
+ "cannot allocate resource memory");
+ *cache_resource = *resource;
+ cache_resource->action =
+ mlx5_glue->dr_create_flow_action_dest_flow_tbl
+ (resource->tbl->obj);
+ if (!cache_resource->action) {
+ rte_free(cache_resource);
+ return rte_flow_error_set(error, ENOMEM,
+ RTE_FLOW_ERROR_TYPE_UNSPECIFIED,
+ NULL, "cannot create action");
+ }
+ rte_atomic32_init(&cache_resource->refcnt);
+ rte_atomic32_inc(&cache_resource->refcnt);
+ LIST_INSERT_HEAD(&priv->jump_tbl, cache_resource, next);
+ dev_flow->dv.jump = cache_resource;
+ DRV_LOG(DEBUG, "new jump table resource %p: refcnt %d++",
+ (void *)cache_resource,
+ rte_atomic32_read(&cache_resource->refcnt));
+ return 0;
+}
+
+/**
* Get the size of specific rte_flow_item_type
*
* @param[in] item_type
@@ -1423,6 +1485,37 @@ struct field_modify_info modify_tcp[] = {
}
/**
+ * Validate jump action.
+ *
+ * @param[in] action
+ * Pointer to the modify action.
+ * @param[in] group
+ * The group of the current flow.
+ * @param[out] error
+ * Pointer to error structure.
+ *
+ * @return
+ * 0 on success, a negative errno value otherwise and rte_errno is set.
+ */
+static int
+flow_dv_validate_action_jump(const struct rte_flow_action *action,
+ uint32_t group,
+ struct rte_flow_error *error)
+{
+ if (action->type != RTE_FLOW_ACTION_TYPE_JUMP && !action->conf)
+ return rte_flow_error_set(error, EINVAL,
+ RTE_FLOW_ERROR_TYPE_ACTION_CONF,
+ NULL, "action configuration not set");
+ if (group >= ((const struct rte_flow_action_jump *)action->conf)->group)
+ return rte_flow_error_set(error, EINVAL,
+ RTE_FLOW_ERROR_TYPE_ACTION, NULL,
+ "target group must be higher then"
+ " the current flow group");
+ return 0;
+}
+
+
+/**
* Find existing modify-header resource or create and register a new one.
*
* @param dev[in, out]
@@ -1605,7 +1698,7 @@ struct field_modify_info modify_tcp[] = {
struct mlx5_priv *priv = dev->data->dev_private;
uint32_t priority_max = priv->config.flow_prio - 1;
-#ifdef HAVE_MLX5DV_DR
+#ifndef HAVE_MLX5DV_DR
if (attributes->group)
return rte_flow_error_set(error, ENOTSUP,
RTE_FLOW_ERROR_TYPE_ATTR_GROUP,
@@ -1977,6 +2070,14 @@ struct field_modify_info modify_tcp[] = {
MLX5_FLOW_ACTION_SET_TTL :
MLX5_FLOW_ACTION_DEC_TTL;
break;
+ case RTE_FLOW_ACTION_TYPE_JUMP:
+ ret = flow_dv_validate_action_jump(actions,
+ attr->group, error);
+ if (ret)
+ return ret;
+ ++actions_n;
+ action_flags |= MLX5_FLOW_ACTION_JUMP;
+ break;
default:
return rte_flow_error_set(error, ENOTSUP,
RTE_FLOW_ERROR_TYPE_ACTION,
@@ -2756,6 +2857,73 @@ struct field_modify_info modify_tcp[] = {
return match_criteria_enable;
}
+
+/**
+ * Get a flow table.
+ *
+ * @param dev[in, out]
+ * Pointer to rte_eth_dev structure.
+ * @param[in] table_id
+ * Table id to use.
+ * @param[in] egress
+ * Direction of the table.
+ * @param[out] error
+ * pointer to error structure.
+ *
+ * @return
+ * Returns tables resource based on the index, NULL in case of failed.
+ */
+static struct mlx5_flow_tbl_resource *
+flow_dv_tbl_resource_get(struct rte_eth_dev *dev,
+ uint32_t table_id, uint8_t egress,
+ struct rte_flow_error *error)
+{
+ struct mlx5_priv *priv = dev->data->dev_private;
+ struct mlx5_flow_tbl_resource *tbl;
+
+ if (egress) {
+ tbl = &priv->tx_tbl[table_id];
+ if (!tbl->obj)
+ tbl->obj = mlx5_glue->dr_create_flow_tbl
+ (priv->tx_ns, table_id);
+ } else {
+ tbl = &priv->rx_tbl[table_id];
+ if (!tbl->obj)
+ tbl->obj = mlx5_glue->dr_create_flow_tbl
+ (priv->rx_ns, table_id);
+ }
+ if (!tbl->obj) {
+ rte_flow_error_set(error, ENOMEM,
+ RTE_FLOW_ERROR_TYPE_UNSPECIFIED,
+ NULL, "cannot create table");
+ return NULL;
+ }
+ rte_atomic32_inc(&tbl->refcnt);
+ return tbl;
+}
+
+/**
+ * Release a flow table.
+ *
+ * @param[in] tbl
+ * Table resource to be released.
+ *
+ * @return
+ * Returns 0 if table was released, else return 1;
+ */
+static int
+flow_dv_tbl_resource_release(struct mlx5_flow_tbl_resource *tbl)
+{
+ if (!tbl)
+ return 0;
+ if (rte_atomic32_dec_and_test(&tbl->refcnt)) {
+ mlx5_glue->dr_destroy_flow_tbl(tbl->obj);
+ tbl->obj = NULL;
+ return 0;
+ }
+ return 1;
+}
+
/**
* Register the flow matcher.
*
@@ -2784,6 +2952,9 @@ struct field_modify_info modify_tcp[] = {
.match_mask = (void *)&matcher->mask,
};
struct mlx5_flow_tbl_resource *tbl = NULL;
+#ifndef HAVE_MLX5DV_DR
+ struct mlx5_flow_tbl_resource tbl_tmp;
+#endif
/* Lookup from cache. */
LIST_FOREACH(cache_matcher, &priv->matchers, next) {
@@ -2805,33 +2976,24 @@ struct field_modify_info modify_tcp[] = {
return 0;
}
}
-#ifdef HAVE_MLX5DV_DR
- if (matcher->egress) {
- tbl = &priv->tx_tbl[matcher->group];
- if (!tbl->obj)
- tbl->obj = mlx5_glue->dr_create_flow_tbl
- (priv->tx_ns,
- matcher->group * MLX5_GROUP_FACTOR);
- } else {
- tbl = &priv->rx_tbl[matcher->group];
- if (!tbl->obj)
- tbl->obj = mlx5_glue->dr_create_flow_tbl
- (priv->rx_ns,
- matcher->group * MLX5_GROUP_FACTOR);
- }
- if (!tbl->obj)
- return rte_flow_error_set(error, ENOMEM,
- RTE_FLOW_ERROR_TYPE_UNSPECIFIED,
- NULL, "cannot create table");
-
- rte_atomic32_inc(&tbl->refcnt);
-#endif
/* Register new matcher. */
cache_matcher = rte_calloc(__func__, 1, sizeof(*cache_matcher), 0);
if (!cache_matcher)
return rte_flow_error_set(error, ENOMEM,
RTE_FLOW_ERROR_TYPE_UNSPECIFIED, NULL,
"cannot allocate matcher memory");
+#ifdef HAVE_MLX5DV_DR
+ tbl = flow_dv_tbl_resource_get(dev, matcher->group * MLX5_GROUP_FACTOR,
+ matcher->egress, error);
+ if (!tbl) {
+ rte_free(cache_matcher);
+ return rte_flow_error_set(error, ENOMEM,
+ RTE_FLOW_ERROR_TYPE_UNSPECIFIED,
+ NULL, "cannot create table");
+ }
+#else
+ tbl = &tbl_tmp;
+#endif
*cache_matcher = *matcher;
dv_attr.match_criteria_enable =
flow_dv_matcher_enable(cache_matcher->mask.buf);
@@ -2844,10 +3006,7 @@ struct field_modify_info modify_tcp[] = {
if (!cache_matcher->matcher_object) {
rte_free(cache_matcher);
#ifdef HAVE_MLX5DV_DR
- if (rte_atomic32_dec_and_test(&tbl->refcnt)) {
- mlx5_glue->dr_destroy_flow_tbl(tbl->obj);
- tbl->obj = NULL;
- }
+ flow_dv_tbl_resource_release(tbl);
#endif
return rte_flow_error_set(error, ENOMEM,
RTE_FLOW_ERROR_TYPE_UNSPECIFIED,
@@ -3033,6 +3192,9 @@ struct field_modify_info modify_tcp[] = {
const struct rte_flow_action *action = actions;
const struct rte_flow_action_count *count = action->conf;
const uint8_t *rss_key;
+ const struct rte_flow_action_jump *jump_data;
+ struct mlx5_flow_dv_jump_tbl_resource jump_tbl_resource;
+ struct mlx5_flow_tbl_resource *tbl;
switch (actions->type) {
case RTE_FLOW_ACTION_TYPE_VOID:
@@ -3171,6 +3333,31 @@ struct field_modify_info modify_tcp[] = {
/* If decap is followed by encap, handle it at encap. */
action_flags |= MLX5_FLOW_ACTION_RAW_DECAP;
break;
+ case RTE_FLOW_ACTION_TYPE_JUMP:
+ jump_data = action->conf;
+ tbl = flow_dv_tbl_resource_get(dev, jump_data->group *
+ MLX5_GROUP_FACTOR,
+ attr->egress, error);
+ if (!tbl)
+ return rte_flow_error_set
+ (error, errno,
+ RTE_FLOW_ERROR_TYPE_ACTION,
+ NULL,
+ "cannot create jump action.");
+ jump_tbl_resource.tbl = tbl;
+ if (flow_dv_jump_tbl_resource_register
+ (dev, &jump_tbl_resource, dev_flow, error)) {
+ flow_dv_tbl_resource_release(tbl);
+ return rte_flow_error_set
+ (error, errno,
+ RTE_FLOW_ERROR_TYPE_ACTION,
+ NULL,
+ "cannot create jump action.");
+ }
+ dev_flow->dv.actions[actions_n++] =
+ dev_flow->dv.jump->action;
+ action_flags |= MLX5_FLOW_ACTION_JUMP;
+ break;
case RTE_FLOW_ACTION_TYPE_SET_MAC_SRC:
case RTE_FLOW_ACTION_TYPE_SET_MAC_DST:
if (flow_dv_convert_action_modify_mac(&res, actions,
@@ -3503,10 +3690,7 @@ struct field_modify_info modify_tcp[] = {
tbl = &priv->tx_tbl[matcher->group];
else
tbl = &priv->rx_tbl[matcher->group];
- if (rte_atomic32_dec_and_test(&tbl->refcnt)) {
- mlx5_glue->dr_destroy_flow_tbl(tbl->obj);
- tbl->obj = NULL;
- }
+ flow_dv_tbl_resource_release(tbl);
rte_free(matcher);
DRV_LOG(DEBUG, "port %u matcher %p: removed",
dev->data->port_id, (void *)matcher);
@@ -3547,6 +3731,38 @@ struct field_modify_info modify_tcp[] = {
}
/**
+ * Release an jump to table action resource.
+ *
+ * @param flow
+ * Pointer to mlx5_flow.
+ *
+ * @return
+ * 1 while a reference on it exists, 0 when freed.
+ */
+static int
+flow_dv_jump_tbl_resource_release(struct mlx5_flow *flow)
+{
+ struct mlx5_flow_dv_jump_tbl_resource *cache_resource =
+ flow->dv.jump;
+
+ assert(cache_resource->action);
+ DRV_LOG(DEBUG, "jump table resource %p: refcnt %d--",
+ (void *)cache_resource,
+ rte_atomic32_read(&cache_resource->refcnt));
+ if (rte_atomic32_dec_and_test(&cache_resource->refcnt)) {
+ claim_zero(mlx5_glue->destroy_flow_action
+ (cache_resource->action));
+ LIST_REMOVE(cache_resource, next);
+ flow_dv_tbl_resource_release(cache_resource->tbl);
+ rte_free(cache_resource);
+ DRV_LOG(DEBUG, "jump table resource %p: removed",
+ (void *)cache_resource);
+ return 0;
+ }
+ return 1;
+}
+
+/**
* Release a modify-header resource.
*
* @param flow
@@ -3642,6 +3858,8 @@ struct field_modify_info modify_tcp[] = {
flow_dv_encap_decap_resource_release(dev_flow);
if (dev_flow->dv.modify_hdr)
flow_dv_modify_hdr_resource_release(dev_flow);
+ if (dev_flow->dv.jump)
+ flow_dv_jump_tbl_resource_release(dev_flow);
rte_free(dev_flow);
}
}
diff --git a/drivers/net/mlx5/mlx5_glue.c b/drivers/net/mlx5/mlx5_glue.c
index f509f85..fca34ca 100644
--- a/drivers/net/mlx5/mlx5_glue.c
+++ b/drivers/net/mlx5/mlx5_glue.c
@@ -370,6 +370,17 @@
}
static void *
+mlx5_glue_dr_create_flow_action_dest_flow_tbl(void *tbl)
+{
+#ifdef HAVE_MLX5DV_DR
+ return mlx5dv_dr_create_action_dest_flow_table(tbl);
+#else
+ (void)tbl;
+ return NULL;
+#endif
+}
+
+static void *
mlx5_glue_dr_create_flow_tbl(void *ns, uint32_t level)
{
#ifdef HAVE_MLX5DV_DR
@@ -833,6 +844,8 @@
.get_async_event = mlx5_glue_get_async_event,
.port_state_str = mlx5_glue_port_state_str,
.cq_ex_to_cq = mlx5_glue_cq_ex_to_cq,
+ .dr_create_flow_action_dest_flow_tbl =
+ mlx5_glue_dr_create_flow_action_dest_flow_tbl,
.dr_create_flow_tbl = mlx5_glue_dr_create_flow_tbl,
.dr_destroy_flow_tbl = mlx5_glue_dr_destroy_flow_tbl,
.dr_create_ns = mlx5_glue_dr_create_ns,
diff --git a/drivers/net/mlx5/mlx5_glue.h b/drivers/net/mlx5/mlx5_glue.h
index 7115575..16d5dd6 100644
--- a/drivers/net/mlx5/mlx5_glue.h
+++ b/drivers/net/mlx5/mlx5_glue.h
@@ -145,6 +145,7 @@ struct mlx5_glue {
struct ibv_async_event *event);
const char *(*port_state_str)(enum ibv_port_state port_state);
struct ibv_cq *(*cq_ex_to_cq)(struct ibv_cq_ex *cq);
+ void *(*dr_create_flow_action_dest_flow_tbl)(void *tbl);
void *(*dr_create_flow_tbl)(void *ns, uint32_t level);
int (*dr_destroy_flow_tbl)(void *tbl);
void *(*dr_create_ns)(struct ibv_context *ctx,
--
1.8.3.1
^ permalink raw reply [flat|nested] 52+ messages in thread
* Re: [dpdk-dev] [PATCH v2 1/3] net/mlx5: prepare Direct Verbs for Direct Rule
2019-03-28 16:32 ` [dpdk-dev] [PATCH v2 1/3] net/mlx5: prepare Direct Verbs for Direct Rule Ori Kam
2019-03-28 16:32 ` Ori Kam
@ 2019-04-01 14:38 ` Slava Ovsiienko
2019-04-01 14:38 ` Slava Ovsiienko
2019-04-03 10:15 ` Shahaf Shuler
2 siblings, 1 reply; 52+ messages in thread
From: Slava Ovsiienko @ 2019-04-01 14:38 UTC (permalink / raw)
To: Ori Kam, Matan Azrad, Yongseok Koh, Shahaf Shuler; +Cc: dev
> -----Original Message-----
> From: Ori Kam
> Sent: Thursday, March 28, 2019 18:33
> To: Matan Azrad <matan@mellanox.com>; Yongseok Koh
> <yskoh@mellanox.com>; Shahaf Shuler <shahafs@mellanox.com>
> Cc: dev@dpdk.org; Ori Kam <orika@mellanox.com>; Slava Ovsiienko
> <viacheslavo@mellanox.com>
> Subject: [PATCH v2 1/3] net/mlx5: prepare Direct Verbs for Direct Rule
>
> This is the first patch of a series that is designed to enable the Direct Rules
> API.
>
> The main difference between Direct Verbs and Direct Rules from API
> prespective, is that in Direct Rules each action has it's own create function
> and the object itself is of type void.
>
> In this patch I'm adding functions to generate actions that currenlty are done
> without create action, and I'm changing the action type to be void *, so in
> next patches only the glue functions will need to change.
>
> Signed-off-by: Ori Kam <orika@mellanox.com>
Acked-by: Viacheslav Ovsiienko <viacheslavo@mellanox.com>
>
> ---
> drivers/net/mlx5/mlx5.h | 2 +
> drivers/net/mlx5/mlx5_flow.h | 17 +++-
> drivers/net/mlx5/mlx5_flow_dv.c | 188 ++++++++++++++++++++++++++++---
> ---------
> drivers/net/mlx5/mlx5_glue.c | 141 ++++++++++++++++++++++++------
> drivers/net/mlx5/mlx5_glue.h | 26 +++---
> 5 files changed, 278 insertions(+), 96 deletions(-)
>
> diff --git a/drivers/net/mlx5/mlx5.h b/drivers/net/mlx5/mlx5.h index
> 7402798..b587359 100644
> --- a/drivers/net/mlx5/mlx5.h
> +++ b/drivers/net/mlx5/mlx5.h
> @@ -269,6 +269,8 @@ struct mlx5_priv {
> LIST_HEAD(matchers, mlx5_flow_dv_matcher) matchers;
> LIST_HEAD(encap_decap, mlx5_flow_dv_encap_decap_resource)
> encaps_decaps;
> LIST_HEAD(modify_cmd, mlx5_flow_dv_modify_hdr_resource)
> modify_cmds;
> + LIST_HEAD(tag, mlx5_flow_dv_tag_resource) tags;
> + /* Tags resources cache. */
> uint32_t link_speed_capa; /* Link speed capabilities. */
> struct mlx5_xstats_ctrl xstats_ctrl; /* Extended stats control. */
> struct mlx5_stats_ctrl stats_ctrl; /* Stats control. */ diff --git
> a/drivers/net/mlx5/mlx5_flow.h b/drivers/net/mlx5/mlx5_flow.h index
> e1e798b..4f69ae2 100644
> --- a/drivers/net/mlx5/mlx5_flow.h
> +++ b/drivers/net/mlx5/mlx5_flow.h
> @@ -214,7 +214,7 @@ struct mlx5_flow_dv_encap_decap_resource {
> LIST_ENTRY(mlx5_flow_dv_encap_decap_resource) next;
> /* Pointer to next element. */
> rte_atomic32_t refcnt; /**< Reference counter. */
> - struct ibv_flow_action *verbs_action;
> + void *verbs_action;
> /**< Verbs encap/decap action object. */
> uint8_t buf[MLX5_ENCAP_MAX_LEN];
> size_t size;
> @@ -222,6 +222,16 @@ struct mlx5_flow_dv_encap_decap_resource {
> uint8_t ft_type;
> };
>
> +/* Tag resource structure. */
> +struct mlx5_flow_dv_tag_resource {
> + LIST_ENTRY(mlx5_flow_dv_tag_resource) next;
> + /* Pointer to next element. */
> + rte_atomic32_t refcnt; /**< Reference counter. */
> + void *action;
> + /**< Verbs tag action object. */
> + uint32_t tag; /**< the tag value. */
> +};
> +
> /* Number of modification commands. */
> #define MLX5_MODIFY_NUM 8
>
> @@ -259,7 +269,7 @@ struct mlx5_flow_dv {
> /**< Pointer to modify header resource in cache. */
> struct ibv_flow *flow; /**< Installed flow. */ #ifdef
> HAVE_IBV_FLOW_DV_SUPPORT
> - struct mlx5dv_flow_action_attr
> actions[MLX5_DV_MAX_NUMBER_OF_ACTIONS];
> + void *actions[MLX5_DV_MAX_NUMBER_OF_ACTIONS];
> /**< Action list. */
> #endif
> int actions_n; /**< number of actions. */ @@ -332,6 +342,7 @@
> struct mlx5_flow_counter {
> };
> uint64_t hits; /**< Number of packets matched by the rule. */
> uint64_t bytes; /**< Number of bytes matched by the rule. */
> + void *action; /**< Pointer to the dv action. */
> };
>
> /* Flow structure. */
> @@ -339,6 +350,8 @@ struct rte_flow {
> TAILQ_ENTRY(rte_flow) next; /**< Pointer to the next flow structure.
> */
> enum mlx5_flow_drv_type drv_type; /**< Drvier type. */
> struct mlx5_flow_counter *counter; /**< Holds flow counter. */
> + struct mlx5_flow_dv_tag_resource *tag_resource;
> + /**< pointer to the tag action. */
> struct rte_flow_action_rss rss;/**< RSS context. */
> uint8_t key[MLX5_RSS_HASH_KEY_LEN]; /**< RSS hash key. */
> uint16_t (*queue)[]; /**< Destination queues to redirect traffic to. */
> diff --git a/drivers/net/mlx5/mlx5_flow_dv.c
> b/drivers/net/mlx5/mlx5_flow_dv.c index 57847fb..b0c50a7 100644
> --- a/drivers/net/mlx5/mlx5_flow_dv.c
> +++ b/drivers/net/mlx5/mlx5_flow_dv.c
> @@ -32,13 +32,8 @@
> #include "mlx5_prm.h"
> #include "mlx5_glue.h"
> #include "mlx5_flow.h"
> -
> #ifdef HAVE_IBV_FLOW_DV_SUPPORT
>
> -#ifndef HAVE_IBV_FLOW_DEVX_COUNTERS
> -#define MLX5DV_FLOW_ACTION_COUNTER_DEVX 0 -#endif
> -
> union flow_dv_attr {
> struct {
> uint32_t valid:1;
> @@ -1537,6 +1532,11 @@ struct field_modify_info modify_tcp[] = {
> .id = id,
> .dcs = dcs,
> };
> + tmpl.action = mlx5_glue->dv_create_flow_action_counter(dcs->obj,
> 0);
> + if (!tmpl.action) {
> + ret = errno;
> + goto error_exit;
> + }
> *cnt = tmpl;
> LIST_INSERT_HEAD(&priv->flow_counters, cnt, next);
> return cnt;
> @@ -2828,6 +2828,97 @@ struct field_modify_info modify_tcp[] = { }
>
> /**
> + * Find existing tag resource or create and register a new one.
> + *
> + * @param dev[in, out]
> + * Pointer to rte_eth_dev structure.
> + * @param[in, out] resource
> + * Pointer to tag resource.
> + * @parm[in, out] dev_flow
> + * Pointer to the dev_flow.
> + * @param[out] error
> + * pointer to error structure.
> + *
> + * @return
> + * 0 on success otherwise -errno and errno is set.
> + */
> +static int
> +flow_dv_tag_resource_register
> + (struct rte_eth_dev *dev,
> + struct mlx5_flow_dv_tag_resource *resource,
> + struct mlx5_flow *dev_flow,
> + struct rte_flow_error *error)
> +{
> + struct mlx5_priv *priv = dev->data->dev_private;
> + struct mlx5_flow_dv_tag_resource *cache_resource;
> +
> + /* Lookup a matching resource from cache. */
> + LIST_FOREACH(cache_resource, &priv->tags, next) {
> + if (resource->tag == cache_resource->tag) {
> + DRV_LOG(DEBUG, "tag resource %p: refcnt %d++",
> + (void *)cache_resource,
> + rte_atomic32_read(&cache_resource-
> >refcnt));
> + rte_atomic32_inc(&cache_resource->refcnt);
> + dev_flow->flow->tag_resource = cache_resource;
> + return 0;
> + }
> + }
> + /* Register new resource. */
> + cache_resource = rte_calloc(__func__, 1, sizeof(*cache_resource), 0);
> + if (!cache_resource)
> + return rte_flow_error_set(error, ENOMEM,
> +
> RTE_FLOW_ERROR_TYPE_UNSPECIFIED, NULL,
> + "cannot allocate resource
> memory");
> + *cache_resource = *resource;
> + cache_resource->action = mlx5_glue->dv_create_flow_action_tag
> + (resource->tag);
> + if (!cache_resource->action) {
> + rte_free(cache_resource);
> + return rte_flow_error_set(error, ENOMEM,
> +
> RTE_FLOW_ERROR_TYPE_UNSPECIFIED,
> + NULL, "cannot create action");
> + }
> + rte_atomic32_init(&cache_resource->refcnt);
> + rte_atomic32_inc(&cache_resource->refcnt);
> + LIST_INSERT_HEAD(&priv->tags, cache_resource, next);
> + dev_flow->flow->tag_resource = cache_resource;
> + DRV_LOG(DEBUG, "new tag resource %p: refcnt %d++",
> + (void *)cache_resource,
> + rte_atomic32_read(&cache_resource->refcnt));
> + return 0;
> +}
> +
> +/**
> + * Release the tag.
> + *
> + * @param dev
> + * Pointer to Ethernet device.
> + * @param flow
> + * Pointer to mlx5_flow.
> + *
> + * @return
> + * 1 while a reference on it exists, 0 when freed.
> + */
> +static int
> +flow_dv_tag_release(struct rte_eth_dev *dev,
> + struct mlx5_flow_dv_tag_resource *tag) {
> + assert(tag);
> + DRV_LOG(DEBUG, "port %u tag %p: refcnt %d--",
> + dev->data->port_id, (void *)tag,
> + rte_atomic32_read(&tag->refcnt));
> + if (rte_atomic32_dec_and_test(&tag->refcnt)) {
> + claim_zero(mlx5_glue->destroy_flow_action(tag->action));
> + LIST_REMOVE(tag, next);
> + DRV_LOG(DEBUG, "port %u tag %p: removed",
> + dev->data->port_id, (void *)tag);
> + rte_free(tag);
> + return 0;
> + }
> + return 1;
> +}
> +
> +/**
> * Fill the flow with DV spec.
> *
> * @param[in] dev
> @@ -2872,6 +2963,7 @@ struct field_modify_info modify_tcp[] = {
>
> MLX5DV_FLOW_TABLE_TYPE_NIC_RX
> };
> union flow_dv_attr flow_attr = { .attr = 0 };
> + struct mlx5_flow_dv_tag_resource tag_resource;
>
> if (priority == MLX5_FLOW_PRIO_RSVD)
> priority = priv->config.flow_prio - 1; @@ -2886,26 +2978,29
> @@ struct field_modify_info modify_tcp[] = {
> case RTE_FLOW_ACTION_TYPE_VOID:
> break;
> case RTE_FLOW_ACTION_TYPE_FLAG:
> - dev_flow->dv.actions[actions_n].type =
> - MLX5DV_FLOW_ACTION_TAG;
> - dev_flow->dv.actions[actions_n].tag_value =
> + tag_resource.tag =
>
> mlx5_flow_mark_set(MLX5_FLOW_MARK_DEFAULT);
> - actions_n++;
> + if (!flow->tag_resource)
> + if (flow_dv_tag_resource_register
> + (dev, &tag_resource, dev_flow, error))
> + return errno;
> + dev_flow->dv.actions[actions_n++] =
> + flow->tag_resource->action;
> action_flags |= MLX5_FLOW_ACTION_FLAG;
> break;
> case RTE_FLOW_ACTION_TYPE_MARK:
> - dev_flow->dv.actions[actions_n].type =
> - MLX5DV_FLOW_ACTION_TAG;
> - dev_flow->dv.actions[actions_n].tag_value =
> - mlx5_flow_mark_set
> - (((const struct rte_flow_action_mark *)
> - (actions->conf))->id);
> - actions_n++;
> + tag_resource.tag = mlx5_flow_mark_set
> + (((const struct rte_flow_action_mark *)
> + (actions->conf))->id);
> + if (!flow->tag_resource)
> + if (flow_dv_tag_resource_register
> + (dev, &tag_resource, dev_flow, error))
> + return errno;
> + dev_flow->dv.actions[actions_n++] =
> + flow->tag_resource->action;
> action_flags |= MLX5_FLOW_ACTION_MARK;
> break;
> case RTE_FLOW_ACTION_TYPE_DROP:
> - dev_flow->dv.actions[actions_n].type =
> - MLX5DV_FLOW_ACTION_DROP;
> action_flags |= MLX5_FLOW_ACTION_DROP;
> break;
> case RTE_FLOW_ACTION_TYPE_QUEUE:
> @@ -2933,17 +3028,13 @@ struct field_modify_info modify_tcp[] = {
> rte_errno = ENOTSUP;
> goto cnt_err;
> }
> - flow->counter =
> - flow_dv_counter_new(dev,
> - count->shared, count->id);
> + flow->counter = flow_dv_counter_new(dev, count-
> >shared,
> + count->id);
> if (flow->counter == NULL)
> goto cnt_err;
> - dev_flow->dv.actions[actions_n].type =
> -
> MLX5DV_FLOW_ACTION_COUNTER_DEVX;
> - dev_flow->dv.actions[actions_n].obj =
> - flow->counter->dcs->obj;
> + dev_flow->dv.actions[actions_n++] =
> + flow->counter->action;
> action_flags |= MLX5_FLOW_ACTION_COUNT;
> - ++actions_n;
> break;
> cnt_err:
> if (rte_errno == ENOTSUP)
> @@ -2964,11 +3055,8 @@ struct field_modify_info modify_tcp[] = {
> if (flow_dv_create_action_l2_encap(dev, actions,
> dev_flow, error))
> return -rte_errno;
> - dev_flow->dv.actions[actions_n].type =
> -
> MLX5DV_FLOW_ACTION_IBV_FLOW_ACTION;
> - dev_flow->dv.actions[actions_n].action =
> + dev_flow->dv.actions[actions_n++] =
> dev_flow->dv.encap_decap->verbs_action;
> - actions_n++;
> action_flags |= actions->type ==
>
> RTE_FLOW_ACTION_TYPE_VXLAN_ENCAP ?
> MLX5_FLOW_ACTION_VXLAN_ENCAP
> :
> @@ -2979,11 +3067,8 @@ struct field_modify_info modify_tcp[] = {
> if (flow_dv_create_action_l2_decap(dev, dev_flow,
> error))
> return -rte_errno;
> - dev_flow->dv.actions[actions_n].type =
> -
> MLX5DV_FLOW_ACTION_IBV_FLOW_ACTION;
> - dev_flow->dv.actions[actions_n].action =
> + dev_flow->dv.actions[actions_n++] =
> dev_flow->dv.encap_decap->verbs_action;
> - actions_n++;
> action_flags |= actions->type ==
>
> RTE_FLOW_ACTION_TYPE_VXLAN_DECAP ?
> MLX5_FLOW_ACTION_VXLAN_DECAP
> :
> @@ -2995,9 +3080,7 @@ struct field_modify_info modify_tcp[] = {
> if (flow_dv_create_action_raw_encap
> (dev, actions, dev_flow, attr, error))
> return -rte_errno;
> - dev_flow->dv.actions[actions_n].type =
> -
> MLX5DV_FLOW_ACTION_IBV_FLOW_ACTION;
> - dev_flow->dv.actions[actions_n].action =
> + dev_flow->dv.actions[actions_n++] =
> dev_flow->dv.encap_decap-
> >verbs_action;
> } else {
> /* Handle encap without preceding decap. */
> @@ -3005,12 +3088,9 @@ struct field_modify_info modify_tcp[] = {
> dev_flow,
> error))
> return -rte_errno;
> - dev_flow->dv.actions[actions_n].type =
> -
> MLX5DV_FLOW_ACTION_IBV_FLOW_ACTION;
> - dev_flow->dv.actions[actions_n].action =
> + dev_flow->dv.actions[actions_n++] =
> dev_flow->dv.encap_decap-
> >verbs_action;
> }
> - actions_n++;
> action_flags |= MLX5_FLOW_ACTION_RAW_ENCAP;
> break;
> case RTE_FLOW_ACTION_TYPE_RAW_DECAP:
> @@ -3025,11 +3105,8 @@ struct field_modify_info modify_tcp[] = {
> dev_flow,
> error))
> return -rte_errno;
> - dev_flow->dv.actions[actions_n].type =
> -
> MLX5DV_FLOW_ACTION_IBV_FLOW_ACTION;
> - dev_flow->dv.actions[actions_n].action =
> + dev_flow->dv.actions[actions_n++] =
> dev_flow->dv.encap_decap-
> >verbs_action;
> - actions_n++;
> }
> /* If decap is followed by encap, handle it at encap.
> */
> action_flags |= MLX5_FLOW_ACTION_RAW_DECAP;
> @@ -3098,11 +3175,8 @@ struct field_modify_info modify_tcp[] = {
> dev_flow,
> error))
> return -rte_errno;
> - dev_flow->dv.actions[actions_n].type =
> -
> MLX5DV_FLOW_ACTION_IBV_FLOW_ACTION;
> - dev_flow->dv.actions[actions_n].action =
> + dev_flow->dv.actions[actions_n++] =
> dev_flow->dv.modify_hdr-
> >verbs_action;
> - actions_n++;
> }
> break;
> default:
> @@ -3277,9 +3351,9 @@ struct field_modify_info modify_tcp[] = {
> "cannot get drop hash queue");
> goto error;
> }
> - dv->actions[n].type =
> MLX5DV_FLOW_ACTION_DEST_IBV_QP;
> - dv->actions[n].qp = dv->hrxq->qp;
> - n++;
> + dv->actions[n++] =
> + mlx5_glue-
> >dv_create_flow_action_dest_ibv_qp
> + (dv->hrxq->qp);
> } else if (flow->actions &
> (MLX5_FLOW_ACTION_QUEUE |
> MLX5_FLOW_ACTION_RSS)) {
> struct mlx5_hrxq *hrxq;
> @@ -3304,9 +3378,9 @@ struct field_modify_info modify_tcp[] = {
> goto error;
> }
> dv->hrxq = hrxq;
> - dv->actions[n].type =
> MLX5DV_FLOW_ACTION_DEST_IBV_QP;
> - dv->actions[n].qp = hrxq->qp;
> - n++;
> + dv->actions[n++] =
> + mlx5_glue-
> >dv_create_flow_action_dest_ibv_qp
> + (dv->hrxq->qp);
> }
> dv->flow =
> mlx5_glue->dv_create_flow(dv->matcher-
> >matcher_object,
> @@ -3484,6 +3558,10 @@ struct field_modify_info modify_tcp[] = {
> flow_dv_counter_release(flow->counter);
> flow->counter = NULL;
> }
> + if (flow->tag_resource) {
> + flow_dv_tag_release(dev, flow->tag_resource);
> + flow->tag_resource = NULL;
> + }
> while (!LIST_EMPTY(&flow->dev_flows)) {
> dev_flow = LIST_FIRST(&flow->dev_flows);
> LIST_REMOVE(dev_flow, next);
> diff --git a/drivers/net/mlx5/mlx5_glue.c b/drivers/net/mlx5/mlx5_glue.c
> index c817d86..cfd939d 100644
> --- a/drivers/net/mlx5/mlx5_glue.c
> +++ b/drivers/net/mlx5/mlx5_glue.c
> @@ -175,10 +175,20 @@
> }
>
> static int
> -mlx5_glue_destroy_flow_action(struct ibv_flow_action *action)
> +mlx5_glue_destroy_flow_action(void *action)
> {
> #ifdef HAVE_IBV_FLOW_DV_SUPPORT
> - return ibv_destroy_flow_action(action);
> + struct mlx5dv_flow_action_attr *attr = action;
> + int res = 0;
> + switch (attr->type) {
> + case MLX5DV_FLOW_ACTION_TAG:
> + break;
> + default:
> + res = ibv_destroy_flow_action(attr->action);
> + break;
> + }
> + free(action);
> + return res;
> #else
> (void)action;
> return ENOTSUP;
> @@ -430,16 +440,23 @@
> mlx5_glue_dv_create_flow(struct mlx5dv_flow_matcher *matcher,
> struct mlx5dv_flow_match_parameters
> *match_value,
> size_t num_actions,
> - struct mlx5dv_flow_action_attr *actions_attr)
> + void *actions[])
> {
> #ifdef HAVE_IBV_FLOW_DV_SUPPORT
> + struct mlx5dv_flow_action_attr actions_attr[8];
> +
> + if (num_actions > 8)
> + return NULL;
> + for (size_t i = 0; i < num_actions; i++)
> + actions_attr[i] =
> + *((struct mlx5dv_flow_action_attr *)(actions[i]));
> return mlx5dv_create_flow(matcher, match_value,
> num_actions, actions_attr);
> #else
> (void)matcher;
> (void)match_value;
> (void)num_actions;
> - (void)actions_attr;
> + (void)actions;
> return NULL;
> #endif
> }
> @@ -455,31 +472,45 @@
> #endif
> }
>
> -static struct ibv_flow_action *
> -mlx5_glue_dv_create_flow_action_packet_reformat
> - (struct ibv_context *ctx,
> - size_t data_sz,
> - void *data,
> - enum mlx5dv_flow_action_packet_reformat_type
> reformat_type,
> - enum mlx5dv_flow_table_type ft_type)
> +static void *
> +mlx5_glue_dv_create_flow_action_counter(void *counter_obj, uint32_t
> +offset)
> {
> #ifdef HAVE_IBV_FLOW_DV_SUPPORT
> - return mlx5dv_create_flow_action_packet_reformat(ctx,
> - data_sz,
> - data,
> - reformat_type,
> - ft_type);
> + struct mlx5dv_flow_action_attr *action;
> +
> + (void)offset;
> + action = malloc(sizeof(*action));
> + if (!action)
> + return NULL;
> + action->type = MLX5DV_FLOW_ACTION_COUNTER_DEVX;
> + action->obj = counter_obj;
> + return action;
> #else
> - (void)ctx;
> - (void)data_sz;
> - (void)data;
> - (void)reformat_type;
> - (void)ft_type;
> + (void)counter_obj;
> + (void)offset;
> return NULL;
> #endif
> }
>
> -static struct ibv_flow_action *
> +static void *
> +mlx5_glue_dv_create_flow_action_dest_ibv_qp(void *qp) { #ifdef
> +HAVE_IBV_FLOW_DV_SUPPORT
> + struct mlx5dv_flow_action_attr *action;
> +
> + action = malloc(sizeof(*action));
> + if (!action)
> + return NULL;
> + action->type = MLX5DV_FLOW_ACTION_DEST_IBV_QP;
> + action->obj = qp;
> + return action;
> +#else
> + (void)qp;
> + return NULL;
> +#endif
> +}
> +
> +static void *
> mlx5_glue_dv_create_flow_action_modify_header
> (struct ibv_context *ctx,
> size_t actions_sz,
> @@ -487,8 +518,15 @@
> enum mlx5dv_flow_table_type
> ft_type) { #ifdef HAVE_IBV_FLOW_DV_SUPPORT
> - return mlx5dv_create_flow_action_modify_header(ctx, actions_sz,
> - actions, ft_type);
> + struct mlx5dv_flow_action_attr *action;
> +
> + action = malloc(sizeof(*action));
> + if (!action)
> + return NULL;
> + action->type = MLX5DV_FLOW_ACTION_IBV_FLOW_ACTION;
> + action->action = mlx5dv_create_flow_action_modify_header
> + (ctx, actions_sz, actions, ft_type);
> + return action;
> #else
> (void)ctx;
> (void)actions_sz;
> @@ -498,6 +536,50 @@
> #endif
> }
>
> +static void *
> +mlx5_glue_dv_create_flow_action_packet_reformat
> + (struct ibv_context *ctx,
> + size_t data_sz,
> + void *data,
> + enum mlx5dv_flow_action_packet_reformat_type
> reformat_type,
> + enum mlx5dv_flow_table_type ft_type) { #ifdef
> +HAVE_IBV_FLOW_DV_SUPPORT
> + struct mlx5dv_flow_action_attr *action;
> +
> + action = malloc(sizeof(*action));
> + if (!action)
> + return NULL;
> + action->type = MLX5DV_FLOW_ACTION_IBV_FLOW_ACTION;
> + action->action = mlx5dv_create_flow_action_packet_reformat
> + (ctx, data_sz, data, reformat_type, ft_type);
> + return action;
> +#else
> + (void)ctx;
> + (void)data_sz;
> + (void)data;
> + (void)reformat_type;
> + (void)ft_type;
> + return NULL;
> +#endif
> +}
> +
> +static void *
> +mlx5_glue_dv_create_flow_action_tag(uint32_t tag) { #ifdef
> +HAVE_IBV_FLOW_DV_SUPPORT
> + struct mlx5dv_flow_action_attr *action;
> + action = malloc(sizeof(*action));
> + if (!action)
> + return NULL;
> + action->type = MLX5DV_FLOW_ACTION_TAG;
> + action->tag_value = tag;
> + return action;
> +#endif
> + (void)tag;
> + return NULL;
> +}
> +
> static struct ibv_context *
> mlx5_glue_dv_open_device(struct ibv_device *device) { @@ -645,10
> +727,15 @@
> .dv_create_flow_matcher = mlx5_glue_dv_create_flow_matcher,
> .dv_destroy_flow_matcher = mlx5_glue_dv_destroy_flow_matcher,
> .dv_create_flow = mlx5_glue_dv_create_flow,
> - .dv_create_flow_action_packet_reformat =
> - mlx5_glue_dv_create_flow_action_packet_reformat,
> + .dv_create_flow_action_counter =
> + mlx5_glue_dv_create_flow_action_counter,
> + .dv_create_flow_action_dest_ibv_qp =
> + mlx5_glue_dv_create_flow_action_dest_ibv_qp,
> .dv_create_flow_action_modify_header =
> - mlx5_glue_dv_create_flow_action_modify_header,
> + mlx5_glue_dv_create_flow_action_modify_header,
> + .dv_create_flow_action_packet_reformat =
> + mlx5_glue_dv_create_flow_action_packet_reformat,
> + .dv_create_flow_action_tag =
> mlx5_glue_dv_create_flow_action_tag,
> .dv_open_device = mlx5_glue_dv_open_device,
> .devx_obj_create = mlx5_glue_devx_obj_create,
> .devx_obj_destroy = mlx5_glue_devx_obj_destroy, diff --git
> a/drivers/net/mlx5/mlx5_glue.h b/drivers/net/mlx5/mlx5_glue.h index
> b118960..167f1f7 100644
> --- a/drivers/net/mlx5/mlx5_glue.h
> +++ b/drivers/net/mlx5/mlx5_glue.h
> @@ -55,6 +55,10 @@
> enum mlx5dv_flow_table_type { flow_table_type = 0, }; #endif
>
> +#ifndef HAVE_IBV_FLOW_DEVX_COUNTERS
> +#define MLX5DV_FLOW_ACTION_COUNTER_DEVX 0 #endif
> +
> #ifndef HAVE_IBV_DEVX_OBJ
> struct mlx5dv_devx_obj;
> #endif
> @@ -98,7 +102,7 @@ struct mlx5_glue {
> struct ibv_flow *(*create_flow)(struct ibv_qp *qp,
> struct ibv_flow_attr *flow);
> int (*destroy_flow)(struct ibv_flow *flow_id);
> - int (*destroy_flow_action)(struct ibv_flow_action *action);
> + int (*destroy_flow_action)(void *action);
> struct ibv_qp *(*create_qp)(struct ibv_pd *pd,
> struct ibv_qp_init_attr *qp_init_attr);
> struct ibv_qp *(*create_qp_ex)
> @@ -160,19 +164,17 @@ struct mlx5_glue {
> int (*dv_destroy_flow_matcher)(struct mlx5dv_flow_matcher
> *matcher);
> struct ibv_flow *(*dv_create_flow)(struct mlx5dv_flow_matcher
> *matcher,
> struct mlx5dv_flow_match_parameters
> *match_value,
> - size_t num_actions,
> - struct mlx5dv_flow_action_attr *actions_attr);
> - struct ibv_flow_action *(*dv_create_flow_action_packet_reformat)
> - (struct ibv_context *ctx,
> - size_t data_sz,
> - void *data,
> + size_t num_actions, void *actions[]);
> + void *(*dv_create_flow_action_counter)(void *obj, uint32_t offset);
> + void *(*dv_create_flow_action_dest_ibv_qp)(void *qp);
> + void *(*dv_create_flow_action_modify_header)
> + (struct ibv_context *ctx, size_t actions_sz, uint64_t actions[],
> + enum mlx5dv_flow_table_type ft_type);
> + void *(*dv_create_flow_action_packet_reformat)
> + (struct ibv_context *ctx, size_t data_sz, void *data,
> enum mlx5dv_flow_action_packet_reformat_type
> reformat_type,
> enum mlx5dv_flow_table_type ft_type);
> - struct ibv_flow_action *(*dv_create_flow_action_modify_header)
> - (struct ibv_context *ctx,
> - size_t actions_sz,
> - uint64_t actions[],
> - enum mlx5dv_flow_table_type
> ft_type);
> + void *(*dv_create_flow_action_tag)(uint32_t tag);
> struct ibv_context *(*dv_open_device)(struct ibv_device *device);
> struct mlx5dv_devx_obj *(*devx_obj_create)
> (struct ibv_context *ctx,
> --
> 1.8.3.1
^ permalink raw reply [flat|nested] 52+ messages in thread
* Re: [dpdk-dev] [PATCH v2 1/3] net/mlx5: prepare Direct Verbs for Direct Rule
2019-04-01 14:38 ` Slava Ovsiienko
@ 2019-04-01 14:38 ` Slava Ovsiienko
0 siblings, 0 replies; 52+ messages in thread
From: Slava Ovsiienko @ 2019-04-01 14:38 UTC (permalink / raw)
To: Ori Kam, Matan Azrad, Yongseok Koh, Shahaf Shuler; +Cc: dev
> -----Original Message-----
> From: Ori Kam
> Sent: Thursday, March 28, 2019 18:33
> To: Matan Azrad <matan@mellanox.com>; Yongseok Koh
> <yskoh@mellanox.com>; Shahaf Shuler <shahafs@mellanox.com>
> Cc: dev@dpdk.org; Ori Kam <orika@mellanox.com>; Slava Ovsiienko
> <viacheslavo@mellanox.com>
> Subject: [PATCH v2 1/3] net/mlx5: prepare Direct Verbs for Direct Rule
>
> This is the first patch of a series that is designed to enable the Direct Rules
> API.
>
> The main difference between Direct Verbs and Direct Rules from API
> prespective, is that in Direct Rules each action has it's own create function
> and the object itself is of type void.
>
> In this patch I'm adding functions to generate actions that currenlty are done
> without create action, and I'm changing the action type to be void *, so in
> next patches only the glue functions will need to change.
>
> Signed-off-by: Ori Kam <orika@mellanox.com>
Acked-by: Viacheslav Ovsiienko <viacheslavo@mellanox.com>
>
> ---
> drivers/net/mlx5/mlx5.h | 2 +
> drivers/net/mlx5/mlx5_flow.h | 17 +++-
> drivers/net/mlx5/mlx5_flow_dv.c | 188 ++++++++++++++++++++++++++++---
> ---------
> drivers/net/mlx5/mlx5_glue.c | 141 ++++++++++++++++++++++++------
> drivers/net/mlx5/mlx5_glue.h | 26 +++---
> 5 files changed, 278 insertions(+), 96 deletions(-)
>
> diff --git a/drivers/net/mlx5/mlx5.h b/drivers/net/mlx5/mlx5.h index
> 7402798..b587359 100644
> --- a/drivers/net/mlx5/mlx5.h
> +++ b/drivers/net/mlx5/mlx5.h
> @@ -269,6 +269,8 @@ struct mlx5_priv {
> LIST_HEAD(matchers, mlx5_flow_dv_matcher) matchers;
> LIST_HEAD(encap_decap, mlx5_flow_dv_encap_decap_resource)
> encaps_decaps;
> LIST_HEAD(modify_cmd, mlx5_flow_dv_modify_hdr_resource)
> modify_cmds;
> + LIST_HEAD(tag, mlx5_flow_dv_tag_resource) tags;
> + /* Tags resources cache. */
> uint32_t link_speed_capa; /* Link speed capabilities. */
> struct mlx5_xstats_ctrl xstats_ctrl; /* Extended stats control. */
> struct mlx5_stats_ctrl stats_ctrl; /* Stats control. */ diff --git
> a/drivers/net/mlx5/mlx5_flow.h b/drivers/net/mlx5/mlx5_flow.h index
> e1e798b..4f69ae2 100644
> --- a/drivers/net/mlx5/mlx5_flow.h
> +++ b/drivers/net/mlx5/mlx5_flow.h
> @@ -214,7 +214,7 @@ struct mlx5_flow_dv_encap_decap_resource {
> LIST_ENTRY(mlx5_flow_dv_encap_decap_resource) next;
> /* Pointer to next element. */
> rte_atomic32_t refcnt; /**< Reference counter. */
> - struct ibv_flow_action *verbs_action;
> + void *verbs_action;
> /**< Verbs encap/decap action object. */
> uint8_t buf[MLX5_ENCAP_MAX_LEN];
> size_t size;
> @@ -222,6 +222,16 @@ struct mlx5_flow_dv_encap_decap_resource {
> uint8_t ft_type;
> };
>
> +/* Tag resource structure. */
> +struct mlx5_flow_dv_tag_resource {
> + LIST_ENTRY(mlx5_flow_dv_tag_resource) next;
> + /* Pointer to next element. */
> + rte_atomic32_t refcnt; /**< Reference counter. */
> + void *action;
> + /**< Verbs tag action object. */
> + uint32_t tag; /**< the tag value. */
> +};
> +
> /* Number of modification commands. */
> #define MLX5_MODIFY_NUM 8
>
> @@ -259,7 +269,7 @@ struct mlx5_flow_dv {
> /**< Pointer to modify header resource in cache. */
> struct ibv_flow *flow; /**< Installed flow. */ #ifdef
> HAVE_IBV_FLOW_DV_SUPPORT
> - struct mlx5dv_flow_action_attr
> actions[MLX5_DV_MAX_NUMBER_OF_ACTIONS];
> + void *actions[MLX5_DV_MAX_NUMBER_OF_ACTIONS];
> /**< Action list. */
> #endif
> int actions_n; /**< number of actions. */ @@ -332,6 +342,7 @@
> struct mlx5_flow_counter {
> };
> uint64_t hits; /**< Number of packets matched by the rule. */
> uint64_t bytes; /**< Number of bytes matched by the rule. */
> + void *action; /**< Pointer to the dv action. */
> };
>
> /* Flow structure. */
> @@ -339,6 +350,8 @@ struct rte_flow {
> TAILQ_ENTRY(rte_flow) next; /**< Pointer to the next flow structure.
> */
> enum mlx5_flow_drv_type drv_type; /**< Drvier type. */
> struct mlx5_flow_counter *counter; /**< Holds flow counter. */
> + struct mlx5_flow_dv_tag_resource *tag_resource;
> + /**< pointer to the tag action. */
> struct rte_flow_action_rss rss;/**< RSS context. */
> uint8_t key[MLX5_RSS_HASH_KEY_LEN]; /**< RSS hash key. */
> uint16_t (*queue)[]; /**< Destination queues to redirect traffic to. */
> diff --git a/drivers/net/mlx5/mlx5_flow_dv.c
> b/drivers/net/mlx5/mlx5_flow_dv.c index 57847fb..b0c50a7 100644
> --- a/drivers/net/mlx5/mlx5_flow_dv.c
> +++ b/drivers/net/mlx5/mlx5_flow_dv.c
> @@ -32,13 +32,8 @@
> #include "mlx5_prm.h"
> #include "mlx5_glue.h"
> #include "mlx5_flow.h"
> -
> #ifdef HAVE_IBV_FLOW_DV_SUPPORT
>
> -#ifndef HAVE_IBV_FLOW_DEVX_COUNTERS
> -#define MLX5DV_FLOW_ACTION_COUNTER_DEVX 0 -#endif
> -
> union flow_dv_attr {
> struct {
> uint32_t valid:1;
> @@ -1537,6 +1532,11 @@ struct field_modify_info modify_tcp[] = {
> .id = id,
> .dcs = dcs,
> };
> + tmpl.action = mlx5_glue->dv_create_flow_action_counter(dcs->obj,
> 0);
> + if (!tmpl.action) {
> + ret = errno;
> + goto error_exit;
> + }
> *cnt = tmpl;
> LIST_INSERT_HEAD(&priv->flow_counters, cnt, next);
> return cnt;
> @@ -2828,6 +2828,97 @@ struct field_modify_info modify_tcp[] = { }
>
> /**
> + * Find existing tag resource or create and register a new one.
> + *
> + * @param dev[in, out]
> + * Pointer to rte_eth_dev structure.
> + * @param[in, out] resource
> + * Pointer to tag resource.
> + * @parm[in, out] dev_flow
> + * Pointer to the dev_flow.
> + * @param[out] error
> + * pointer to error structure.
> + *
> + * @return
> + * 0 on success otherwise -errno and errno is set.
> + */
> +static int
> +flow_dv_tag_resource_register
> + (struct rte_eth_dev *dev,
> + struct mlx5_flow_dv_tag_resource *resource,
> + struct mlx5_flow *dev_flow,
> + struct rte_flow_error *error)
> +{
> + struct mlx5_priv *priv = dev->data->dev_private;
> + struct mlx5_flow_dv_tag_resource *cache_resource;
> +
> + /* Lookup a matching resource from cache. */
> + LIST_FOREACH(cache_resource, &priv->tags, next) {
> + if (resource->tag == cache_resource->tag) {
> + DRV_LOG(DEBUG, "tag resource %p: refcnt %d++",
> + (void *)cache_resource,
> + rte_atomic32_read(&cache_resource-
> >refcnt));
> + rte_atomic32_inc(&cache_resource->refcnt);
> + dev_flow->flow->tag_resource = cache_resource;
> + return 0;
> + }
> + }
> + /* Register new resource. */
> + cache_resource = rte_calloc(__func__, 1, sizeof(*cache_resource), 0);
> + if (!cache_resource)
> + return rte_flow_error_set(error, ENOMEM,
> +
> RTE_FLOW_ERROR_TYPE_UNSPECIFIED, NULL,
> + "cannot allocate resource
> memory");
> + *cache_resource = *resource;
> + cache_resource->action = mlx5_glue->dv_create_flow_action_tag
> + (resource->tag);
> + if (!cache_resource->action) {
> + rte_free(cache_resource);
> + return rte_flow_error_set(error, ENOMEM,
> +
> RTE_FLOW_ERROR_TYPE_UNSPECIFIED,
> + NULL, "cannot create action");
> + }
> + rte_atomic32_init(&cache_resource->refcnt);
> + rte_atomic32_inc(&cache_resource->refcnt);
> + LIST_INSERT_HEAD(&priv->tags, cache_resource, next);
> + dev_flow->flow->tag_resource = cache_resource;
> + DRV_LOG(DEBUG, "new tag resource %p: refcnt %d++",
> + (void *)cache_resource,
> + rte_atomic32_read(&cache_resource->refcnt));
> + return 0;
> +}
> +
> +/**
> + * Release the tag.
> + *
> + * @param dev
> + * Pointer to Ethernet device.
> + * @param flow
> + * Pointer to mlx5_flow.
> + *
> + * @return
> + * 1 while a reference on it exists, 0 when freed.
> + */
> +static int
> +flow_dv_tag_release(struct rte_eth_dev *dev,
> + struct mlx5_flow_dv_tag_resource *tag) {
> + assert(tag);
> + DRV_LOG(DEBUG, "port %u tag %p: refcnt %d--",
> + dev->data->port_id, (void *)tag,
> + rte_atomic32_read(&tag->refcnt));
> + if (rte_atomic32_dec_and_test(&tag->refcnt)) {
> + claim_zero(mlx5_glue->destroy_flow_action(tag->action));
> + LIST_REMOVE(tag, next);
> + DRV_LOG(DEBUG, "port %u tag %p: removed",
> + dev->data->port_id, (void *)tag);
> + rte_free(tag);
> + return 0;
> + }
> + return 1;
> +}
> +
> +/**
> * Fill the flow with DV spec.
> *
> * @param[in] dev
> @@ -2872,6 +2963,7 @@ struct field_modify_info modify_tcp[] = {
>
> MLX5DV_FLOW_TABLE_TYPE_NIC_RX
> };
> union flow_dv_attr flow_attr = { .attr = 0 };
> + struct mlx5_flow_dv_tag_resource tag_resource;
>
> if (priority == MLX5_FLOW_PRIO_RSVD)
> priority = priv->config.flow_prio - 1; @@ -2886,26 +2978,29
> @@ struct field_modify_info modify_tcp[] = {
> case RTE_FLOW_ACTION_TYPE_VOID:
> break;
> case RTE_FLOW_ACTION_TYPE_FLAG:
> - dev_flow->dv.actions[actions_n].type =
> - MLX5DV_FLOW_ACTION_TAG;
> - dev_flow->dv.actions[actions_n].tag_value =
> + tag_resource.tag =
>
> mlx5_flow_mark_set(MLX5_FLOW_MARK_DEFAULT);
> - actions_n++;
> + if (!flow->tag_resource)
> + if (flow_dv_tag_resource_register
> + (dev, &tag_resource, dev_flow, error))
> + return errno;
> + dev_flow->dv.actions[actions_n++] =
> + flow->tag_resource->action;
> action_flags |= MLX5_FLOW_ACTION_FLAG;
> break;
> case RTE_FLOW_ACTION_TYPE_MARK:
> - dev_flow->dv.actions[actions_n].type =
> - MLX5DV_FLOW_ACTION_TAG;
> - dev_flow->dv.actions[actions_n].tag_value =
> - mlx5_flow_mark_set
> - (((const struct rte_flow_action_mark *)
> - (actions->conf))->id);
> - actions_n++;
> + tag_resource.tag = mlx5_flow_mark_set
> + (((const struct rte_flow_action_mark *)
> + (actions->conf))->id);
> + if (!flow->tag_resource)
> + if (flow_dv_tag_resource_register
> + (dev, &tag_resource, dev_flow, error))
> + return errno;
> + dev_flow->dv.actions[actions_n++] =
> + flow->tag_resource->action;
> action_flags |= MLX5_FLOW_ACTION_MARK;
> break;
> case RTE_FLOW_ACTION_TYPE_DROP:
> - dev_flow->dv.actions[actions_n].type =
> - MLX5DV_FLOW_ACTION_DROP;
> action_flags |= MLX5_FLOW_ACTION_DROP;
> break;
> case RTE_FLOW_ACTION_TYPE_QUEUE:
> @@ -2933,17 +3028,13 @@ struct field_modify_info modify_tcp[] = {
> rte_errno = ENOTSUP;
> goto cnt_err;
> }
> - flow->counter =
> - flow_dv_counter_new(dev,
> - count->shared, count->id);
> + flow->counter = flow_dv_counter_new(dev, count-
> >shared,
> + count->id);
> if (flow->counter == NULL)
> goto cnt_err;
> - dev_flow->dv.actions[actions_n].type =
> -
> MLX5DV_FLOW_ACTION_COUNTER_DEVX;
> - dev_flow->dv.actions[actions_n].obj =
> - flow->counter->dcs->obj;
> + dev_flow->dv.actions[actions_n++] =
> + flow->counter->action;
> action_flags |= MLX5_FLOW_ACTION_COUNT;
> - ++actions_n;
> break;
> cnt_err:
> if (rte_errno == ENOTSUP)
> @@ -2964,11 +3055,8 @@ struct field_modify_info modify_tcp[] = {
> if (flow_dv_create_action_l2_encap(dev, actions,
> dev_flow, error))
> return -rte_errno;
> - dev_flow->dv.actions[actions_n].type =
> -
> MLX5DV_FLOW_ACTION_IBV_FLOW_ACTION;
> - dev_flow->dv.actions[actions_n].action =
> + dev_flow->dv.actions[actions_n++] =
> dev_flow->dv.encap_decap->verbs_action;
> - actions_n++;
> action_flags |= actions->type ==
>
> RTE_FLOW_ACTION_TYPE_VXLAN_ENCAP ?
> MLX5_FLOW_ACTION_VXLAN_ENCAP
> :
> @@ -2979,11 +3067,8 @@ struct field_modify_info modify_tcp[] = {
> if (flow_dv_create_action_l2_decap(dev, dev_flow,
> error))
> return -rte_errno;
> - dev_flow->dv.actions[actions_n].type =
> -
> MLX5DV_FLOW_ACTION_IBV_FLOW_ACTION;
> - dev_flow->dv.actions[actions_n].action =
> + dev_flow->dv.actions[actions_n++] =
> dev_flow->dv.encap_decap->verbs_action;
> - actions_n++;
> action_flags |= actions->type ==
>
> RTE_FLOW_ACTION_TYPE_VXLAN_DECAP ?
> MLX5_FLOW_ACTION_VXLAN_DECAP
> :
> @@ -2995,9 +3080,7 @@ struct field_modify_info modify_tcp[] = {
> if (flow_dv_create_action_raw_encap
> (dev, actions, dev_flow, attr, error))
> return -rte_errno;
> - dev_flow->dv.actions[actions_n].type =
> -
> MLX5DV_FLOW_ACTION_IBV_FLOW_ACTION;
> - dev_flow->dv.actions[actions_n].action =
> + dev_flow->dv.actions[actions_n++] =
> dev_flow->dv.encap_decap-
> >verbs_action;
> } else {
> /* Handle encap without preceding decap. */
> @@ -3005,12 +3088,9 @@ struct field_modify_info modify_tcp[] = {
> dev_flow,
> error))
> return -rte_errno;
> - dev_flow->dv.actions[actions_n].type =
> -
> MLX5DV_FLOW_ACTION_IBV_FLOW_ACTION;
> - dev_flow->dv.actions[actions_n].action =
> + dev_flow->dv.actions[actions_n++] =
> dev_flow->dv.encap_decap-
> >verbs_action;
> }
> - actions_n++;
> action_flags |= MLX5_FLOW_ACTION_RAW_ENCAP;
> break;
> case RTE_FLOW_ACTION_TYPE_RAW_DECAP:
> @@ -3025,11 +3105,8 @@ struct field_modify_info modify_tcp[] = {
> dev_flow,
> error))
> return -rte_errno;
> - dev_flow->dv.actions[actions_n].type =
> -
> MLX5DV_FLOW_ACTION_IBV_FLOW_ACTION;
> - dev_flow->dv.actions[actions_n].action =
> + dev_flow->dv.actions[actions_n++] =
> dev_flow->dv.encap_decap-
> >verbs_action;
> - actions_n++;
> }
> /* If decap is followed by encap, handle it at encap.
> */
> action_flags |= MLX5_FLOW_ACTION_RAW_DECAP;
> @@ -3098,11 +3175,8 @@ struct field_modify_info modify_tcp[] = {
> dev_flow,
> error))
> return -rte_errno;
> - dev_flow->dv.actions[actions_n].type =
> -
> MLX5DV_FLOW_ACTION_IBV_FLOW_ACTION;
> - dev_flow->dv.actions[actions_n].action =
> + dev_flow->dv.actions[actions_n++] =
> dev_flow->dv.modify_hdr-
> >verbs_action;
> - actions_n++;
> }
> break;
> default:
> @@ -3277,9 +3351,9 @@ struct field_modify_info modify_tcp[] = {
> "cannot get drop hash queue");
> goto error;
> }
> - dv->actions[n].type =
> MLX5DV_FLOW_ACTION_DEST_IBV_QP;
> - dv->actions[n].qp = dv->hrxq->qp;
> - n++;
> + dv->actions[n++] =
> + mlx5_glue-
> >dv_create_flow_action_dest_ibv_qp
> + (dv->hrxq->qp);
> } else if (flow->actions &
> (MLX5_FLOW_ACTION_QUEUE |
> MLX5_FLOW_ACTION_RSS)) {
> struct mlx5_hrxq *hrxq;
> @@ -3304,9 +3378,9 @@ struct field_modify_info modify_tcp[] = {
> goto error;
> }
> dv->hrxq = hrxq;
> - dv->actions[n].type =
> MLX5DV_FLOW_ACTION_DEST_IBV_QP;
> - dv->actions[n].qp = hrxq->qp;
> - n++;
> + dv->actions[n++] =
> + mlx5_glue-
> >dv_create_flow_action_dest_ibv_qp
> + (dv->hrxq->qp);
> }
> dv->flow =
> mlx5_glue->dv_create_flow(dv->matcher-
> >matcher_object,
> @@ -3484,6 +3558,10 @@ struct field_modify_info modify_tcp[] = {
> flow_dv_counter_release(flow->counter);
> flow->counter = NULL;
> }
> + if (flow->tag_resource) {
> + flow_dv_tag_release(dev, flow->tag_resource);
> + flow->tag_resource = NULL;
> + }
> while (!LIST_EMPTY(&flow->dev_flows)) {
> dev_flow = LIST_FIRST(&flow->dev_flows);
> LIST_REMOVE(dev_flow, next);
> diff --git a/drivers/net/mlx5/mlx5_glue.c b/drivers/net/mlx5/mlx5_glue.c
> index c817d86..cfd939d 100644
> --- a/drivers/net/mlx5/mlx5_glue.c
> +++ b/drivers/net/mlx5/mlx5_glue.c
> @@ -175,10 +175,20 @@
> }
>
> static int
> -mlx5_glue_destroy_flow_action(struct ibv_flow_action *action)
> +mlx5_glue_destroy_flow_action(void *action)
> {
> #ifdef HAVE_IBV_FLOW_DV_SUPPORT
> - return ibv_destroy_flow_action(action);
> + struct mlx5dv_flow_action_attr *attr = action;
> + int res = 0;
> + switch (attr->type) {
> + case MLX5DV_FLOW_ACTION_TAG:
> + break;
> + default:
> + res = ibv_destroy_flow_action(attr->action);
> + break;
> + }
> + free(action);
> + return res;
> #else
> (void)action;
> return ENOTSUP;
> @@ -430,16 +440,23 @@
> mlx5_glue_dv_create_flow(struct mlx5dv_flow_matcher *matcher,
> struct mlx5dv_flow_match_parameters
> *match_value,
> size_t num_actions,
> - struct mlx5dv_flow_action_attr *actions_attr)
> + void *actions[])
> {
> #ifdef HAVE_IBV_FLOW_DV_SUPPORT
> + struct mlx5dv_flow_action_attr actions_attr[8];
> +
> + if (num_actions > 8)
> + return NULL;
> + for (size_t i = 0; i < num_actions; i++)
> + actions_attr[i] =
> + *((struct mlx5dv_flow_action_attr *)(actions[i]));
> return mlx5dv_create_flow(matcher, match_value,
> num_actions, actions_attr);
> #else
> (void)matcher;
> (void)match_value;
> (void)num_actions;
> - (void)actions_attr;
> + (void)actions;
> return NULL;
> #endif
> }
> @@ -455,31 +472,45 @@
> #endif
> }
>
> -static struct ibv_flow_action *
> -mlx5_glue_dv_create_flow_action_packet_reformat
> - (struct ibv_context *ctx,
> - size_t data_sz,
> - void *data,
> - enum mlx5dv_flow_action_packet_reformat_type
> reformat_type,
> - enum mlx5dv_flow_table_type ft_type)
> +static void *
> +mlx5_glue_dv_create_flow_action_counter(void *counter_obj, uint32_t
> +offset)
> {
> #ifdef HAVE_IBV_FLOW_DV_SUPPORT
> - return mlx5dv_create_flow_action_packet_reformat(ctx,
> - data_sz,
> - data,
> - reformat_type,
> - ft_type);
> + struct mlx5dv_flow_action_attr *action;
> +
> + (void)offset;
> + action = malloc(sizeof(*action));
> + if (!action)
> + return NULL;
> + action->type = MLX5DV_FLOW_ACTION_COUNTER_DEVX;
> + action->obj = counter_obj;
> + return action;
> #else
> - (void)ctx;
> - (void)data_sz;
> - (void)data;
> - (void)reformat_type;
> - (void)ft_type;
> + (void)counter_obj;
> + (void)offset;
> return NULL;
> #endif
> }
>
> -static struct ibv_flow_action *
> +static void *
> +mlx5_glue_dv_create_flow_action_dest_ibv_qp(void *qp) { #ifdef
> +HAVE_IBV_FLOW_DV_SUPPORT
> + struct mlx5dv_flow_action_attr *action;
> +
> + action = malloc(sizeof(*action));
> + if (!action)
> + return NULL;
> + action->type = MLX5DV_FLOW_ACTION_DEST_IBV_QP;
> + action->obj = qp;
> + return action;
> +#else
> + (void)qp;
> + return NULL;
> +#endif
> +}
> +
> +static void *
> mlx5_glue_dv_create_flow_action_modify_header
> (struct ibv_context *ctx,
> size_t actions_sz,
> @@ -487,8 +518,15 @@
> enum mlx5dv_flow_table_type
> ft_type) { #ifdef HAVE_IBV_FLOW_DV_SUPPORT
> - return mlx5dv_create_flow_action_modify_header(ctx, actions_sz,
> - actions, ft_type);
> + struct mlx5dv_flow_action_attr *action;
> +
> + action = malloc(sizeof(*action));
> + if (!action)
> + return NULL;
> + action->type = MLX5DV_FLOW_ACTION_IBV_FLOW_ACTION;
> + action->action = mlx5dv_create_flow_action_modify_header
> + (ctx, actions_sz, actions, ft_type);
> + return action;
> #else
> (void)ctx;
> (void)actions_sz;
> @@ -498,6 +536,50 @@
> #endif
> }
>
> +static void *
> +mlx5_glue_dv_create_flow_action_packet_reformat
> + (struct ibv_context *ctx,
> + size_t data_sz,
> + void *data,
> + enum mlx5dv_flow_action_packet_reformat_type
> reformat_type,
> + enum mlx5dv_flow_table_type ft_type) { #ifdef
> +HAVE_IBV_FLOW_DV_SUPPORT
> + struct mlx5dv_flow_action_attr *action;
> +
> + action = malloc(sizeof(*action));
> + if (!action)
> + return NULL;
> + action->type = MLX5DV_FLOW_ACTION_IBV_FLOW_ACTION;
> + action->action = mlx5dv_create_flow_action_packet_reformat
> + (ctx, data_sz, data, reformat_type, ft_type);
> + return action;
> +#else
> + (void)ctx;
> + (void)data_sz;
> + (void)data;
> + (void)reformat_type;
> + (void)ft_type;
> + return NULL;
> +#endif
> +}
> +
> +static void *
> +mlx5_glue_dv_create_flow_action_tag(uint32_t tag) { #ifdef
> +HAVE_IBV_FLOW_DV_SUPPORT
> + struct mlx5dv_flow_action_attr *action;
> + action = malloc(sizeof(*action));
> + if (!action)
> + return NULL;
> + action->type = MLX5DV_FLOW_ACTION_TAG;
> + action->tag_value = tag;
> + return action;
> +#endif
> + (void)tag;
> + return NULL;
> +}
> +
> static struct ibv_context *
> mlx5_glue_dv_open_device(struct ibv_device *device) { @@ -645,10
> +727,15 @@
> .dv_create_flow_matcher = mlx5_glue_dv_create_flow_matcher,
> .dv_destroy_flow_matcher = mlx5_glue_dv_destroy_flow_matcher,
> .dv_create_flow = mlx5_glue_dv_create_flow,
> - .dv_create_flow_action_packet_reformat =
> - mlx5_glue_dv_create_flow_action_packet_reformat,
> + .dv_create_flow_action_counter =
> + mlx5_glue_dv_create_flow_action_counter,
> + .dv_create_flow_action_dest_ibv_qp =
> + mlx5_glue_dv_create_flow_action_dest_ibv_qp,
> .dv_create_flow_action_modify_header =
> - mlx5_glue_dv_create_flow_action_modify_header,
> + mlx5_glue_dv_create_flow_action_modify_header,
> + .dv_create_flow_action_packet_reformat =
> + mlx5_glue_dv_create_flow_action_packet_reformat,
> + .dv_create_flow_action_tag =
> mlx5_glue_dv_create_flow_action_tag,
> .dv_open_device = mlx5_glue_dv_open_device,
> .devx_obj_create = mlx5_glue_devx_obj_create,
> .devx_obj_destroy = mlx5_glue_devx_obj_destroy, diff --git
> a/drivers/net/mlx5/mlx5_glue.h b/drivers/net/mlx5/mlx5_glue.h index
> b118960..167f1f7 100644
> --- a/drivers/net/mlx5/mlx5_glue.h
> +++ b/drivers/net/mlx5/mlx5_glue.h
> @@ -55,6 +55,10 @@
> enum mlx5dv_flow_table_type { flow_table_type = 0, }; #endif
>
> +#ifndef HAVE_IBV_FLOW_DEVX_COUNTERS
> +#define MLX5DV_FLOW_ACTION_COUNTER_DEVX 0 #endif
> +
> #ifndef HAVE_IBV_DEVX_OBJ
> struct mlx5dv_devx_obj;
> #endif
> @@ -98,7 +102,7 @@ struct mlx5_glue {
> struct ibv_flow *(*create_flow)(struct ibv_qp *qp,
> struct ibv_flow_attr *flow);
> int (*destroy_flow)(struct ibv_flow *flow_id);
> - int (*destroy_flow_action)(struct ibv_flow_action *action);
> + int (*destroy_flow_action)(void *action);
> struct ibv_qp *(*create_qp)(struct ibv_pd *pd,
> struct ibv_qp_init_attr *qp_init_attr);
> struct ibv_qp *(*create_qp_ex)
> @@ -160,19 +164,17 @@ struct mlx5_glue {
> int (*dv_destroy_flow_matcher)(struct mlx5dv_flow_matcher
> *matcher);
> struct ibv_flow *(*dv_create_flow)(struct mlx5dv_flow_matcher
> *matcher,
> struct mlx5dv_flow_match_parameters
> *match_value,
> - size_t num_actions,
> - struct mlx5dv_flow_action_attr *actions_attr);
> - struct ibv_flow_action *(*dv_create_flow_action_packet_reformat)
> - (struct ibv_context *ctx,
> - size_t data_sz,
> - void *data,
> + size_t num_actions, void *actions[]);
> + void *(*dv_create_flow_action_counter)(void *obj, uint32_t offset);
> + void *(*dv_create_flow_action_dest_ibv_qp)(void *qp);
> + void *(*dv_create_flow_action_modify_header)
> + (struct ibv_context *ctx, size_t actions_sz, uint64_t actions[],
> + enum mlx5dv_flow_table_type ft_type);
> + void *(*dv_create_flow_action_packet_reformat)
> + (struct ibv_context *ctx, size_t data_sz, void *data,
> enum mlx5dv_flow_action_packet_reformat_type
> reformat_type,
> enum mlx5dv_flow_table_type ft_type);
> - struct ibv_flow_action *(*dv_create_flow_action_modify_header)
> - (struct ibv_context *ctx,
> - size_t actions_sz,
> - uint64_t actions[],
> - enum mlx5dv_flow_table_type
> ft_type);
> + void *(*dv_create_flow_action_tag)(uint32_t tag);
> struct ibv_context *(*dv_open_device)(struct ibv_device *device);
> struct mlx5dv_devx_obj *(*devx_obj_create)
> (struct ibv_context *ctx,
> --
> 1.8.3.1
^ permalink raw reply [flat|nested] 52+ messages in thread
* Re: [dpdk-dev] [PATCH v2 2/3] net/mlx5: add Direct Rules API
2019-03-28 16:32 ` [dpdk-dev] [PATCH v2 2/3] net/mlx5: add Direct Rules API Ori Kam
2019-03-28 16:32 ` Ori Kam
@ 2019-04-01 14:38 ` Slava Ovsiienko
2019-04-01 14:38 ` Slava Ovsiienko
1 sibling, 1 reply; 52+ messages in thread
From: Slava Ovsiienko @ 2019-04-01 14:38 UTC (permalink / raw)
To: Ori Kam, Matan Azrad, Yongseok Koh, Shahaf Shuler; +Cc: dev
> -----Original Message-----
> From: Ori Kam
> Sent: Thursday, March 28, 2019 18:33
> To: Matan Azrad <matan@mellanox.com>; Yongseok Koh
> <yskoh@mellanox.com>; Shahaf Shuler <shahafs@mellanox.com>
> Cc: dev@dpdk.org; Ori Kam <orika@mellanox.com>; Slava Ovsiienko
> <viacheslavo@mellanox.com>
> Subject: [PATCH v2 2/3] net/mlx5: add Direct Rules API
>
> Adds calls to the Direct Rules API inside the glue functions.
> Due to difference in parameters between the Direct Rules and Direct Verbs
> some of the glue functions API was updated.
>
> Signed-off-by: Ori Kam <orika@mellanox.com>
Acked-by: Viacheslav Ovsiienko <viacheslavo@mellanox.com>
>
> ---
> drivers/net/mlx5/Makefile | 5 ++
> drivers/net/mlx5/mlx5.c | 16 ++++
> drivers/net/mlx5/mlx5.h | 15 ++++
> drivers/net/mlx5/mlx5_flow.c | 1 +
> drivers/net/mlx5/mlx5_flow.h | 6 +-
> drivers/net/mlx5/mlx5_flow_dv.c | 103 ++++++++++++++++++++----
> drivers/net/mlx5/mlx5_glue.c | 170
> ++++++++++++++++++++++++++++++++++------
> drivers/net/mlx5/mlx5_glue.h | 31 +++++---
> drivers/net/mlx5/mlx5_prm.h | 24 +++++-
> 9 files changed, 318 insertions(+), 53 deletions(-)
>
> diff --git a/drivers/net/mlx5/Makefile b/drivers/net/mlx5/Makefile index
> c326494..3dd7e38 100644
> --- a/drivers/net/mlx5/Makefile
> +++ b/drivers/net/mlx5/Makefile
> @@ -156,6 +156,11 @@ mlx5_autoconf.h.new: $(RTE_SDK)/buildtools/auto-
> config-h.sh
> func mlx5dv_create_flow_action_packet_reformat \
> $(AUTOCONF_OUTPUT)
> $Q sh -- '$<' '$@' \
> + HAVE_MLX5DV_DR \
> + infiniband/mlx5dv.h \
> + enum MLX5DV_DR_NS_TYPE_TERMINATING \
> + $(AUTOCONF_OUTPUT)
> + $Q sh -- '$<' '$@' \
> HAVE_IBV_DEVX_OBJ \
> infiniband/mlx5dv.h \
> func mlx5dv_devx_obj_create \
> diff --git a/drivers/net/mlx5/mlx5.c b/drivers/net/mlx5/mlx5.c index
> 7d1a21e..70e37b5 100644
> --- a/drivers/net/mlx5/mlx5.c
> +++ b/drivers/net/mlx5/mlx5.c
> @@ -1362,6 +1362,22 @@ struct mlx5_dev_spawn_data {
> priv->tcf_context = NULL;
> }
> }
> +#ifdef HAVE_MLX5DV_DR
> + priv->rx_ns = mlx5dv_dr_create_ns
> + (sh->ctx,
> MLX5DV_DR_NS_DOMAIN_INGRESS_BYPASS);
> + if (priv->rx_ns == NULL) {
> + DRV_LOG(ERR, "mlx5dv_dr_create_ns failed");
> + err = errno;
> + goto error;
> + }
> + priv->tx_ns = mlx5dv_dr_create_ns(sh->ctx,
> +
> MLX5DV_DR_NS_DOMAIN_EGRESS_BYPASS);
> + if (priv->tx_ns == NULL) {
> + DRV_LOG(ERR, "mlx5dv_dr_create_ns failed");
> + err = errno;
> + goto error;
> + }
> +#endif
> TAILQ_INIT(&priv->flows);
> TAILQ_INIT(&priv->ctrl_flows);
> /* Hint libmlx5 to use PMD allocator for data plane resources */ diff -
> -git a/drivers/net/mlx5/mlx5.h b/drivers/net/mlx5/mlx5.h index
> b587359..73f6f0d 100644
> --- a/drivers/net/mlx5/mlx5.h
> +++ b/drivers/net/mlx5/mlx5.h
> @@ -219,6 +219,15 @@ struct mlx5_ibv_shared {
> struct mlx5_ibv_shared_port port[]; /* per device port data array. */
> };
>
> +/* Table structure. */
> +struct mlx5_flow_tbl_resource {
> + void *obj; /**< Pointer to DR table object. */
> + rte_atomic32_t refcnt; /**< Reference counter. */ };
> +
> +#define MLX5_MAX_TABLES 1024
> +#define MLX5_GROUP_FACTOR 1
> +
> struct mlx5_priv {
> LIST_ENTRY(mlx5_priv) mem_event_cb;
> /**< Called by memory event callback. */ @@ -289,6 +298,12 @@
> struct mlx5_priv {
> /* UAR same-page access control required in 32bit implementations.
> */ #endif
> struct mlx5_flow_tcf_context *tcf_context; /* TC flower context. */
> + void *rx_ns; /* RX Direct Rules name space handle. */
> + struct mlx5_flow_tbl_resource rx_tbl[MLX5_MAX_TABLES];
> + /* RX Direct Rules tables. */
> + void *tx_ns; /* TX Direct Rules name space handle. */
> + struct mlx5_flow_tbl_resource tx_tbl[MLX5_MAX_TABLES];
> + /* TX Direct Rules tables/ */
> };
>
> #define PORT_ID(priv) ((priv)->dev_data->port_id) diff --git
> a/drivers/net/mlx5/mlx5_flow.c b/drivers/net/mlx5/mlx5_flow.c index
> d09fdff..40d6818 100644
> --- a/drivers/net/mlx5/mlx5_flow.c
> +++ b/drivers/net/mlx5/mlx5_flow.c
> @@ -2084,6 +2084,7 @@ uint32_t mlx5_flow_adjust_priority(struct
> rte_eth_dev *dev, int32_t priority,
> flow_size += RTE_ALIGN_CEIL(sizeof(uint16_t), sizeof(void
> *));
> flow = rte_calloc(__func__, 1, flow_size, 0);
> flow->drv_type = flow_get_drv_type(dev, attr);
> + flow->ingress = attr->ingress;
> assert(flow->drv_type > MLX5_FLOW_TYPE_MIN &&
> flow->drv_type < MLX5_FLOW_TYPE_MAX);
> flow->queue = (void *)(flow + 1);
> diff --git a/drivers/net/mlx5/mlx5_flow.h b/drivers/net/mlx5/mlx5_flow.h
> index 4f69ae2..8ba37a0 100644
> --- a/drivers/net/mlx5/mlx5_flow.h
> +++ b/drivers/net/mlx5/mlx5_flow.h
> @@ -204,6 +204,7 @@ struct mlx5_flow_dv_matcher {
> uint16_t crc; /**< CRC of key. */
> uint16_t priority; /**< Priority of matcher. */
> uint8_t egress; /**< Egress matcher. */
> + uint32_t group; /**< The matcher group. */
> struct mlx5_flow_dv_match_params mask; /**< Matcher mask. */ };
>
> @@ -220,6 +221,7 @@ struct mlx5_flow_dv_encap_decap_resource {
> size_t size;
> uint8_t reformat_type;
> uint8_t ft_type;
> + uint64_t flags; /**< Flags for RDMA API. */
> };
>
> /* Tag resource structure. */
> @@ -348,7 +350,7 @@ struct mlx5_flow_counter {
> /* Flow structure. */
> struct rte_flow {
> TAILQ_ENTRY(rte_flow) next; /**< Pointer to the next flow structure.
> */
> - enum mlx5_flow_drv_type drv_type; /**< Drvier type. */
> + enum mlx5_flow_drv_type drv_type; /**< Driver type. */
> struct mlx5_flow_counter *counter; /**< Holds flow counter. */
> struct mlx5_flow_dv_tag_resource *tag_resource;
> /**< pointer to the tag action. */
> @@ -360,6 +362,8 @@ struct rte_flow {
> uint64_t actions;
> /**< Bit-fields of detected actions, see MLX5_FLOW_ACTION_*. */
> struct mlx5_fdir *fdir; /**< Pointer to associated FDIR if any. */
> + uint8_t ingress; /**< 1 if the flow is ingress. */
> + uint32_t group; /**< The group index. */
> };
>
> typedef int (*mlx5_flow_validate_t)(struct rte_eth_dev *dev, diff --git
> a/drivers/net/mlx5/mlx5_flow_dv.c b/drivers/net/mlx5/mlx5_flow_dv.c
> index b0c50a7..6e4f6c4 100644
> --- a/drivers/net/mlx5/mlx5_flow_dv.c
> +++ b/drivers/net/mlx5/mlx5_flow_dv.c
> @@ -805,11 +805,20 @@ struct field_modify_info modify_tcp[] = { {
> struct mlx5_priv *priv = dev->data->dev_private;
> struct mlx5_flow_dv_encap_decap_resource *cache_resource;
> + struct rte_flow *flow = dev_flow->flow;
> + struct mlx5dv_dr_ns *ns;
> +
> + resource->flags = flow->group ? 0 : 1;
> + if (flow->ingress)
> + ns = priv->rx_ns;
> + else
> + ns = priv->tx_ns;
>
> /* Lookup a matching resource from cache. */
> LIST_FOREACH(cache_resource, &priv->encaps_decaps, next) {
> if (resource->reformat_type == cache_resource-
> >reformat_type &&
> resource->ft_type == cache_resource->ft_type &&
> + resource->flags == cache_resource->flags &&
> resource->size == cache_resource->size &&
> !memcmp((const void *)resource->buf,
> (const void *)cache_resource->buf, @@ -831,10
> +840,10 @@ struct field_modify_info modify_tcp[] = {
> *cache_resource = *resource;
> cache_resource->verbs_action =
> mlx5_glue->dv_create_flow_action_packet_reformat
> - (priv->sh->ctx, cache_resource->size,
> - (cache_resource->size ? cache_resource->buf :
> NULL),
> - cache_resource->reformat_type,
> - cache_resource->ft_type);
> + (priv->sh->ctx, cache_resource->reformat_type,
> + cache_resource->ft_type, ns, cache_resource->flags,
> + cache_resource->size,
> + (cache_resource->size ? cache_resource->buf :
> NULL));
> if (!cache_resource->verbs_action) {
> rte_free(cache_resource);
> return rte_flow_error_set(error, ENOMEM, @@ -1438,6
> +1447,10 @@ struct field_modify_info modify_tcp[] = {
> struct mlx5_priv *priv = dev->data->dev_private;
> struct mlx5_flow_dv_modify_hdr_resource *cache_resource;
>
> + struct mlx5dv_dr_ns *ns =
> + resource->ft_type == MLX5DV_FLOW_TABLE_TYPE_NIC_TX ?
> + priv->tx_ns : priv->rx_ns;
> +
> /* Lookup a matching resource from cache. */
> LIST_FOREACH(cache_resource, &priv->modify_cmds, next) {
> if (resource->ft_type == cache_resource->ft_type && @@ -
> 1463,11 +1476,11 @@ struct field_modify_info modify_tcp[] = {
> *cache_resource = *resource;
> cache_resource->verbs_action =
> mlx5_glue->dv_create_flow_action_modify_header
> - (priv->sh->ctx,
> + (priv->sh->ctx, cache_resource-
> >ft_type,
> + ns, 0,
> cache_resource->actions_num *
> sizeof(cache_resource->actions[0]),
> - (uint64_t *)cache_resource->actions,
> - cache_resource->ft_type);
> + (uint64_t *)cache_resource-
> >actions);
> if (!cache_resource->verbs_action) {
> rte_free(cache_resource);
> return rte_flow_error_set(error, ENOMEM, @@ -1592,11
> +1605,13 @@ struct field_modify_info modify_tcp[] = {
> struct mlx5_priv *priv = dev->data->dev_private;
> uint32_t priority_max = priv->config.flow_prio - 1;
>
> +#ifdef HAVE_MLX5DV_DR
> if (attributes->group)
> return rte_flow_error_set(error, ENOTSUP,
>
> RTE_FLOW_ERROR_TYPE_ATTR_GROUP,
> NULL,
> "groups is not supported");
> +#endif
> if (attributes->priority != MLX5_FLOW_PRIO_RSVD &&
> attributes->priority >= priority_max)
> return rte_flow_error_set(error, ENOTSUP, @@ -2169,11
> +2184,13 @@ struct field_modify_info modify_tcp[] = {
> * Flow pattern to translate.
> * @param[in] inner
> * Item is inner pattern.
> + * @param[in] group
> + * The group to insert the rule.
> */
> static void
> flow_dv_translate_item_ipv4(void *matcher, void *key,
> const struct rte_flow_item *item,
> - int inner)
> + int inner, uint32_t group)
> {
> const struct rte_flow_item_ipv4 *ipv4_m = item->mask;
> const struct rte_flow_item_ipv4 *ipv4_v = item->spec; @@ -2200,7
> +2217,10 @@ struct field_modify_info modify_tcp[] = {
> outer_headers);
> headers_v = MLX5_ADDR_OF(fte_match_param, key,
> outer_headers);
> }
> - MLX5_SET(fte_match_set_lyr_2_4, headers_m, ip_version, 0xf);
> + if (group == 0)
> + MLX5_SET(fte_match_set_lyr_2_4, headers_m, ip_version,
> 0xf);
> + else
> + MLX5_SET(fte_match_set_lyr_2_4, headers_m, ip_version,
> 0x4);
> MLX5_SET(fte_match_set_lyr_2_4, headers_v, ip_version, 4);
> if (!ipv4_v)
> return;
> @@ -2242,11 +2262,13 @@ struct field_modify_info modify_tcp[] = {
> * Flow pattern to translate.
> * @param[in] inner
> * Item is inner pattern.
> + * @param[in] group
> + * The group to insert the rule.
> */
> static void
> flow_dv_translate_item_ipv6(void *matcher, void *key,
> const struct rte_flow_item *item,
> - int inner)
> + int inner, uint32_t group)
> {
> const struct rte_flow_item_ipv6 *ipv6_m = item->mask;
> const struct rte_flow_item_ipv6 *ipv6_v = item->spec; @@ -2283,7
> +2305,10 @@ struct field_modify_info modify_tcp[] = {
> outer_headers);
> headers_v = MLX5_ADDR_OF(fte_match_param, key,
> outer_headers);
> }
> - MLX5_SET(fte_match_set_lyr_2_4, headers_m, ip_version, 0xf);
> + if (group == 0)
> + MLX5_SET(fte_match_set_lyr_2_4, headers_m, ip_version,
> 0xf);
> + else
> + MLX5_SET(fte_match_set_lyr_2_4, headers_m, ip_version,
> 0x6);
> MLX5_SET(fte_match_set_lyr_2_4, headers_v, ip_version, 6);
> if (!ipv6_v)
> return;
> @@ -2723,7 +2748,11 @@ struct field_modify_info modify_tcp[] = {
> match_criteria_enable |=
> (!HEADER_IS_ZERO(match_criteria, misc_parameters_2)) <<
> MLX5_MATCH_CRITERIA_ENABLE_MISC2_BIT;
> -
> +#ifdef HAVE_MLX5DV_DR
> + match_criteria_enable |=
> + (!HEADER_IS_ZERO(match_criteria, misc_parameters_3)) <<
> + MLX5_MATCH_CRITERIA_ENABLE_MISC3_BIT;
> +#endif
> return match_criteria_enable;
> }
>
> @@ -2754,12 +2783,14 @@ struct field_modify_info modify_tcp[] = {
> .type = IBV_FLOW_ATTR_NORMAL,
> .match_mask = (void *)&matcher->mask,
> };
> + struct mlx5_flow_tbl_resource *tbl = NULL;
>
> /* Lookup from cache. */
> LIST_FOREACH(cache_matcher, &priv->matchers, next) {
> if (matcher->crc == cache_matcher->crc &&
> matcher->priority == cache_matcher->priority &&
> matcher->egress == cache_matcher->egress &&
> + matcher->group == cache_matcher->group &&
> !memcmp((const void *)matcher->mask.buf,
> (const void *)cache_matcher->mask.buf,
> cache_matcher->mask.size)) {
> @@ -2774,6 +2805,27 @@ struct field_modify_info modify_tcp[] = {
> return 0;
> }
> }
> +#ifdef HAVE_MLX5DV_DR
> + if (matcher->egress) {
> + tbl = &priv->tx_tbl[matcher->group];
> + if (!tbl->obj)
> + tbl->obj = mlx5_glue->dr_create_flow_tbl
> + (priv->tx_ns,
> + matcher->group * MLX5_GROUP_FACTOR);
> + } else {
> + tbl = &priv->rx_tbl[matcher->group];
> + if (!tbl->obj)
> + tbl->obj = mlx5_glue->dr_create_flow_tbl
> + (priv->rx_ns,
> + matcher->group * MLX5_GROUP_FACTOR);
> + }
> + if (!tbl->obj)
> + return rte_flow_error_set(error, ENOMEM,
> +
> RTE_FLOW_ERROR_TYPE_UNSPECIFIED,
> + NULL, "cannot create table");
> +
> + rte_atomic32_inc(&tbl->refcnt);
> +#endif
> /* Register new matcher. */
> cache_matcher = rte_calloc(__func__, 1, sizeof(*cache_matcher), 0);
> if (!cache_matcher)
> @@ -2787,9 +2839,16 @@ struct field_modify_info modify_tcp[] = {
> if (matcher->egress)
> dv_attr.flags |= IBV_FLOW_ATTR_FLAGS_EGRESS;
> cache_matcher->matcher_object =
> - mlx5_glue->dv_create_flow_matcher(priv->sh->ctx,
> &dv_attr);
> + mlx5_glue->dv_create_flow_matcher(priv->sh->ctx,
> &dv_attr,
> + tbl->obj);
> if (!cache_matcher->matcher_object) {
> rte_free(cache_matcher);
> +#ifdef HAVE_MLX5DV_DR
> + if (rte_atomic32_dec_and_test(&tbl->refcnt)) {
> + mlx5_glue->dr_destroy_flow_tbl(tbl->obj);
> + tbl->obj = NULL;
> + }
> +#endif
> return rte_flow_error_set(error, ENOMEM,
>
> RTE_FLOW_ERROR_TYPE_UNSPECIFIED,
> NULL, "cannot create matcher");
> @@ -2801,6 +2860,7 @@ struct field_modify_info modify_tcp[] = {
> cache_matcher->priority,
> cache_matcher->egress ? "tx" : "rx", (void *)cache_matcher,
> rte_atomic32_read(&cache_matcher->refcnt));
> + rte_atomic32_inc(&tbl->refcnt);
> return 0;
> }
>
> @@ -3222,7 +3282,7 @@ struct field_modify_info modify_tcp[] = {
> break;
> case RTE_FLOW_ITEM_TYPE_IPV4:
> flow_dv_translate_item_ipv4(match_mask,
> match_value,
> - items, tunnel);
> + items, tunnel, attr->group);
> matcher.priority = MLX5_PRIORITY_MAP_L3;
> dev_flow->dv.hash_fields |=
> mlx5_flow_hashfields_adjust
> @@ -3234,7 +3294,7 @@ struct field_modify_info modify_tcp[] = {
> break;
> case RTE_FLOW_ITEM_TYPE_IPV6:
> flow_dv_translate_item_ipv6(match_mask,
> match_value,
> - items, tunnel);
> + items, tunnel, attr->group);
> matcher.priority = MLX5_PRIORITY_MAP_L3;
> dev_flow->dv.hash_fields |=
> mlx5_flow_hashfields_adjust
> @@ -3312,6 +3372,7 @@ struct field_modify_info modify_tcp[] = {
> matcher.priority = mlx5_flow_adjust_priority(dev, priority,
> matcher.priority);
> matcher.egress = attr->egress;
> + matcher.group = attr->group;
> if (flow_dv_matcher_register(dev, &matcher, dev_flow, error))
> return -rte_errno;
> return 0;
> @@ -3427,6 +3488,8 @@ struct field_modify_info modify_tcp[] = {
> struct mlx5_flow *flow)
> {
> struct mlx5_flow_dv_matcher *matcher = flow->dv.matcher;
> + struct mlx5_priv *priv = dev->data->dev_private;
> + struct mlx5_flow_tbl_resource *tbl;
>
> assert(matcher->matcher_object);
> DRV_LOG(DEBUG, "port %u matcher %p: refcnt %d--", @@ -3436,6
> +3499,14 @@ struct field_modify_info modify_tcp[] = {
> claim_zero(mlx5_glue->dv_destroy_flow_matcher
> (matcher->matcher_object));
> LIST_REMOVE(matcher, next);
> + if (matcher->egress)
> + tbl = &priv->tx_tbl[matcher->group];
> + else
> + tbl = &priv->rx_tbl[matcher->group];
> + if (rte_atomic32_dec_and_test(&tbl->refcnt)) {
> + mlx5_glue->dr_destroy_flow_tbl(tbl->obj);
> + tbl->obj = NULL;
> + }
> rte_free(matcher);
> DRV_LOG(DEBUG, "port %u matcher %p: removed",
> dev->data->port_id, (void *)matcher); @@ -3525,7
> +3596,7 @@ struct field_modify_info modify_tcp[] = {
> LIST_FOREACH(dev_flow, &flow->dev_flows, next) {
> dv = &dev_flow->dv;
> if (dv->flow) {
> - claim_zero(mlx5_glue->destroy_flow(dv->flow));
> + claim_zero(mlx5_glue->dv_destroy_flow(dv->flow));
> dv->flow = NULL;
> }
> if (dv->hrxq) {
> diff --git a/drivers/net/mlx5/mlx5_glue.c b/drivers/net/mlx5/mlx5_glue.c
> index cfd939d..f509f85 100644
> --- a/drivers/net/mlx5/mlx5_glue.c
> +++ b/drivers/net/mlx5/mlx5_glue.c
> @@ -178,6 +178,9 @@
> mlx5_glue_destroy_flow_action(void *action) { #ifdef
> HAVE_IBV_FLOW_DV_SUPPORT
> +#ifdef HAVE_MLX5DV_DR
> + return mlx5dv_dr_destroy_action(action); #else
> struct mlx5dv_flow_action_attr *attr = action;
> int res = 0;
> switch (attr->type) {
> @@ -189,6 +192,7 @@
> }
> free(action);
> return res;
> +#endif
> #else
> (void)action;
> return ENOTSUP;
> @@ -365,6 +369,53 @@
> return ibv_cq_ex_to_cq(cq);
> }
>
> +static void *
> +mlx5_glue_dr_create_flow_tbl(void *ns, uint32_t level) { #ifdef
> +HAVE_MLX5DV_DR
> + return mlx5dv_dr_create_ft(ns, level); #else
> + (void)ns;
> + (void)level;
> + return NULL;
> +#endif
> +}
> +
> +static int
> +mlx5_glue_dr_destroy_flow_tbl(void *tbl) { #ifdef HAVE_MLX5DV_DR
> + return mlx5dv_dr_destroy_ft(tbl);
> +#else
> + (void)tbl;
> + return 0;
> +#endif
> +}
> +
> +static void *
> +mlx5_glue_dr_create_ns(struct ibv_context *ctx,
> + enum mlx5dv_dr_ns_domain domain) { #ifdef
> HAVE_MLX5DV_DR
> + return mlx5dv_dr_create_ns(ctx, domain); #else
> + (void)ctx;
> + (void)domain;
> + return NULL;
> +#endif
> +}
> +
> +static int
> +mlx5_glue_dr_destroy_ns(void *ns)
> +{
> +#ifdef HAVE_MLX5DV_DR
> + return mlx5dv_dr_destroy_ns(ns);
> +#else
> + (void)ns;
> + return 0;
> +#endif
> +}
> +
> static struct ibv_cq_ex *
> mlx5_glue_dv_create_cq(struct ibv_context *context,
> struct ibv_cq_init_attr_ex *cq_attr, @@ -423,26 +474,40
> @@ #endif }
>
> -static struct mlx5dv_flow_matcher *
> +static void *
> mlx5_glue_dv_create_flow_matcher(struct ibv_context *context,
> - struct mlx5dv_flow_matcher_attr
> *matcher_attr)
> + struct mlx5dv_flow_matcher_attr
> *matcher_attr,
> + void *tbl)
> {
> #ifdef HAVE_IBV_FLOW_DV_SUPPORT
> +#ifdef HAVE_MLX5DV_DR
> + (void)context;
> + return mlx5dv_dr_create_matcher(tbl, matcher_attr->priority,
> + matcher_attr->match_criteria_enable,
> + matcher_attr->match_mask);
> +#else
> + (void)tbl;
> return mlx5dv_create_flow_matcher(context, matcher_attr);
> +#endif
> #else
> (void)context;
> (void)matcher_attr;
> + (void)tbl;
> return NULL;
> #endif
> }
>
> -static struct ibv_flow *
> -mlx5_glue_dv_create_flow(struct mlx5dv_flow_matcher *matcher,
> - struct mlx5dv_flow_match_parameters
> *match_value,
> +static void *
> +mlx5_glue_dv_create_flow(void *matcher,
> + void *match_value,
> size_t num_actions,
> void *actions[])
> {
> #ifdef HAVE_IBV_FLOW_DV_SUPPORT
> +#ifdef HAVE_MLX5DV_DR
> + return mlx5dv_dr_create_rule(matcher, match_value, num_actions,
> + (struct mlx5dv_dr_action **)actions); #else
> struct mlx5dv_flow_action_attr actions_attr[8];
>
> if (num_actions > 8)
> @@ -452,6 +517,7 @@
> *((struct mlx5dv_flow_action_attr *)(actions[i]));
> return mlx5dv_create_flow(matcher, match_value,
> num_actions, actions_attr);
> +#endif
> #else
> (void)matcher;
> (void)match_value;
> @@ -461,21 +527,13 @@
> #endif
> }
>
> -static int
> -mlx5_glue_dv_destroy_flow_matcher(struct mlx5dv_flow_matcher
> *matcher) -{ -#ifdef HAVE_IBV_FLOW_DV_SUPPORT
> - return mlx5dv_destroy_flow_matcher(matcher);
> -#else
> - (void)matcher;
> - return 0;
> -#endif
> -}
> -
> static void *
> mlx5_glue_dv_create_flow_action_counter(void *counter_obj, uint32_t
> offset) { #ifdef HAVE_IBV_FLOW_DV_SUPPORT
> +#ifdef HAVE_MLX5DV_DR
> + return mlx5dv_dr_create_action_devx_counter(counter_obj, offset);
> +#else
> struct mlx5dv_flow_action_attr *action;
>
> (void)offset;
> @@ -485,6 +543,7 @@
> action->type = MLX5DV_FLOW_ACTION_COUNTER_DEVX;
> action->obj = counter_obj;
> return action;
> +#endif
> #else
> (void)counter_obj;
> (void)offset;
> @@ -496,6 +555,9 @@
> mlx5_glue_dv_create_flow_action_dest_ibv_qp(void *qp) { #ifdef
> HAVE_IBV_FLOW_DV_SUPPORT
> +#ifdef HAVE_MLX5DV_DR
> + return mlx5dv_dr_create_action_dest_ibv_qp(qp);
> +#else
> struct mlx5dv_flow_action_attr *action;
>
> action = malloc(sizeof(*action));
> @@ -504,6 +566,7 @@
> action->type = MLX5DV_FLOW_ACTION_DEST_IBV_QP;
> action->obj = qp;
> return action;
> +#endif
> #else
> (void)qp;
> return NULL;
> @@ -513,13 +576,22 @@
> static void *
> mlx5_glue_dv_create_flow_action_modify_header
> (struct ibv_context *ctx,
> + enum mlx5dv_flow_table_type
> ft_type,
> + void *ns, uint64_t flags,
> size_t actions_sz,
> - uint64_t actions[],
> - enum mlx5dv_flow_table_type
> ft_type)
> + uint64_t actions[])
> {
> #ifdef HAVE_IBV_FLOW_DV_SUPPORT
> +#ifdef HAVE_MLX5DV_DR
> + (void)ctx;
> + (void)ft_type;
> + return mlx5dv_dr_create_action_modify_header(ns, flags,
> actions_sz,
> + actions);
> +#else
> struct mlx5dv_flow_action_attr *action;
>
> + (void)ns;
> + (void)flags;
> action = malloc(sizeof(*action));
> if (!action)
> return NULL;
> @@ -527,11 +599,14 @@
> action->action = mlx5dv_create_flow_action_modify_header
> (ctx, actions_sz, actions, ft_type);
> return action;
> +#endif
> #else
> (void)ctx;
> + (void)ft_type;
> + (void)ns;
> + (void)flags;
> (void)actions_sz;
> (void)actions;
> - (void)ft_type;
> return NULL;
> #endif
> }
> @@ -539,12 +614,20 @@
> static void *
> mlx5_glue_dv_create_flow_action_packet_reformat
> (struct ibv_context *ctx,
> - size_t data_sz,
> - void *data,
> enum mlx5dv_flow_action_packet_reformat_type
> reformat_type,
> - enum mlx5dv_flow_table_type ft_type)
> + enum mlx5dv_flow_table_type ft_type, struct mlx5dv_dr_ns
> *ns,
> + uint32_t flags, size_t data_sz, void *data)
> {
> #ifdef HAVE_IBV_FLOW_DV_SUPPORT
> +#ifdef HAVE_MLX5DV_DR
> + (void)ctx;
> + (void)ft_type;
> + return mlx5dv_dr_create_action_packet_reformat(ns, flags,
> + reformat_type, data_sz,
> + data);
> +#else
> + (void)ns;
> + (void)flags;
> struct mlx5dv_flow_action_attr *action;
>
> action = malloc(sizeof(*action));
> @@ -554,12 +637,15 @@
> action->action = mlx5dv_create_flow_action_packet_reformat
> (ctx, data_sz, data, reformat_type, ft_type);
> return action;
> +#endif
> #else
> (void)ctx;
> - (void)data_sz;
> - (void)data;
> (void)reformat_type;
> (void)ft_type;
> + (void)ns;
> + (void)flags;
> + (void)data_sz;
> + (void)data;
> return NULL;
> #endif
> }
> @@ -568,6 +654,9 @@
> mlx5_glue_dv_create_flow_action_tag(uint32_t tag) { #ifdef
> HAVE_IBV_FLOW_DV_SUPPORT
> +#ifdef HAVE_MLX5DV_DR
> + return mlx5dv_dr_create_action_tag(tag); #else
> struct mlx5dv_flow_action_attr *action;
> action = malloc(sizeof(*action));
> if (!action)
> @@ -576,10 +665,36 @@
> action->tag_value = tag;
> return action;
> #endif
> +#endif
> (void)tag;
> return NULL;
> }
>
> +static int
> +mlx5_glue_dv_destroy_flow(void *flow_id) { #ifdef HAVE_MLX5DV_DR
> + return mlx5dv_dr_destroy_rule(flow_id); #else
> + return ibv_destroy_flow(flow_id);
> +#endif
> +}
> +
> +static int
> +mlx5_glue_dv_destroy_flow_matcher(void *matcher) { #ifdef
> +HAVE_IBV_FLOW_DV_SUPPORT #ifdef HAVE_MLX5DV_DR
> + return mlx5dv_dr_destroy_matcher(matcher);
> +#else
> + return mlx5dv_destroy_flow_matcher(matcher);
> +#endif
> +#else
> + (void)matcher;
> + return 0;
> +#endif
> +}
> +
> static struct ibv_context *
> mlx5_glue_dv_open_device(struct ibv_device *device) { @@ -718,6 +833,10
> @@
> .get_async_event = mlx5_glue_get_async_event,
> .port_state_str = mlx5_glue_port_state_str,
> .cq_ex_to_cq = mlx5_glue_cq_ex_to_cq,
> + .dr_create_flow_tbl = mlx5_glue_dr_create_flow_tbl,
> + .dr_destroy_flow_tbl = mlx5_glue_dr_destroy_flow_tbl,
> + .dr_create_ns = mlx5_glue_dr_create_ns,
> + .dr_destroy_ns = mlx5_glue_dr_destroy_ns,
> .dv_create_cq = mlx5_glue_dv_create_cq,
> .dv_create_wq = mlx5_glue_dv_create_wq,
> .dv_query_device = mlx5_glue_dv_query_device, @@ -725,7 +844,6
> @@
> .dv_init_obj = mlx5_glue_dv_init_obj,
> .dv_create_qp = mlx5_glue_dv_create_qp,
> .dv_create_flow_matcher = mlx5_glue_dv_create_flow_matcher,
> - .dv_destroy_flow_matcher = mlx5_glue_dv_destroy_flow_matcher,
> .dv_create_flow = mlx5_glue_dv_create_flow,
> .dv_create_flow_action_counter =
> mlx5_glue_dv_create_flow_action_counter,
> @@ -736,6 +854,8 @@
> .dv_create_flow_action_packet_reformat =
> mlx5_glue_dv_create_flow_action_packet_reformat,
> .dv_create_flow_action_tag =
> mlx5_glue_dv_create_flow_action_tag,
> + .dv_destroy_flow = mlx5_glue_dv_destroy_flow,
> + .dv_destroy_flow_matcher = mlx5_glue_dv_destroy_flow_matcher,
> .dv_open_device = mlx5_glue_dv_open_device,
> .devx_obj_create = mlx5_glue_devx_obj_create,
> .devx_obj_destroy = mlx5_glue_devx_obj_destroy, diff --git
> a/drivers/net/mlx5/mlx5_glue.h b/drivers/net/mlx5/mlx5_glue.h index
> 167f1f7..7115575 100644
> --- a/drivers/net/mlx5/mlx5_glue.h
> +++ b/drivers/net/mlx5/mlx5_glue.h
> @@ -63,6 +63,11 @@
> struct mlx5dv_devx_obj;
> #endif
>
> +#ifndef HAVE_MLX5DV_DR
> +struct mlx5dv_dr_ns;
> +enum mlx5dv_dr_ns_domain { unused, };
> +#endif
> +
> /* LIB_GLUE_VERSION must be updated every time this structure is
> modified. */ struct mlx5_glue {
> const char *version;
> @@ -140,6 +145,11 @@ struct mlx5_glue {
> struct ibv_async_event *event);
> const char *(*port_state_str)(enum ibv_port_state port_state);
> struct ibv_cq *(*cq_ex_to_cq)(struct ibv_cq_ex *cq);
> + void *(*dr_create_flow_tbl)(void *ns, uint32_t level);
> + int (*dr_destroy_flow_tbl)(void *tbl);
> + void *(*dr_create_ns)(struct ibv_context *ctx,
> + enum mlx5dv_dr_ns_domain domain);
> + int (*dr_destroy_ns)(void *ns);
> struct ibv_cq_ex *(*dv_create_cq)
> (struct ibv_context *context,
> struct ibv_cq_init_attr_ex *cq_attr, @@ -158,23 +168,26
> @@ struct mlx5_glue {
> (struct ibv_context *context,
> struct ibv_qp_init_attr_ex *qp_init_attr_ex,
> struct mlx5dv_qp_init_attr *dv_qp_init_attr);
> - struct mlx5dv_flow_matcher *(*dv_create_flow_matcher)
> + void *(*dv_create_flow_matcher)
> (struct ibv_context *context,
> - struct mlx5dv_flow_matcher_attr *matcher_attr);
> - int (*dv_destroy_flow_matcher)(struct mlx5dv_flow_matcher
> *matcher);
> - struct ibv_flow *(*dv_create_flow)(struct mlx5dv_flow_matcher
> *matcher,
> - struct mlx5dv_flow_match_parameters
> *match_value,
> + struct mlx5dv_flow_matcher_attr *matcher_attr,
> + void *tbl);
> + void *(*dv_create_flow)(void *matcher, void *match_value,
> size_t num_actions, void *actions[]);
> void *(*dv_create_flow_action_counter)(void *obj, uint32_t offset);
> void *(*dv_create_flow_action_dest_ibv_qp)(void *qp);
> void *(*dv_create_flow_action_modify_header)
> - (struct ibv_context *ctx, size_t actions_sz, uint64_t actions[],
> - enum mlx5dv_flow_table_type ft_type);
> + (struct ibv_context *ctx, enum mlx5dv_flow_table_type
> ft_type,
> + void *ns, uint64_t flags, size_t actions_sz,
> + uint64_t actions[]);
> void *(*dv_create_flow_action_packet_reformat)
> - (struct ibv_context *ctx, size_t data_sz, void *data,
> + (struct ibv_context *ctx,
> enum mlx5dv_flow_action_packet_reformat_type
> reformat_type,
> - enum mlx5dv_flow_table_type ft_type);
> + enum mlx5dv_flow_table_type ft_type, struct mlx5dv_dr_ns
> *ns,
> + uint32_t flags, size_t data_sz, void *data);
> void *(*dv_create_flow_action_tag)(uint32_t tag);
> + int (*dv_destroy_flow)(void *flow);
> + int (*dv_destroy_flow_matcher)(void *matcher);
> struct ibv_context *(*dv_open_device)(struct ibv_device *device);
> struct mlx5dv_devx_obj *(*devx_obj_create)
> (struct ibv_context *ctx,
> diff --git a/drivers/net/mlx5/mlx5_prm.h b/drivers/net/mlx5/mlx5_prm.h
> index da1219e..b15266f 100644
> --- a/drivers/net/mlx5/mlx5_prm.h
> +++ b/drivers/net/mlx5/mlx5_prm.h
> @@ -492,20 +492,40 @@ struct mlx5_ifc_fte_match_set_misc2_bits {
> u8 reserved_at_1a0[0x60];
> };
>
> +struct mlx5_ifc_fte_match_set_misc3_bits {
> + u8 inner_tcp_seq_num[0x20];
> + u8 outer_tcp_seq_num[0x20];
> + u8 inner_tcp_ack_num[0x20];
> + u8 outer_tcp_ack_num[0x20];
> + u8 reserved_at_auto1[0x8];
> + u8 outer_vxlan_gpe_vni[0x18];
> + u8 outer_vxlan_gpe_next_protocol[0x8];
> + u8 outer_vxlan_gpe_flags[0x8];
> + u8 reserved_at_a8[0x10];
> + u8 icmp_header_data[0x20];
> + u8 icmpv6_header_data[0x20];
> + u8 icmp_type[0x8];
> + u8 icmp_code[0x8];
> + u8 icmpv6_type[0x8];
> + u8 icmpv6_code[0x8];
> + u8 reserved_at_1a0[0xe0];
> +};
> +
> /* Flow matcher. */
> struct mlx5_ifc_fte_match_param_bits {
> struct mlx5_ifc_fte_match_set_lyr_2_4_bits outer_headers;
> struct mlx5_ifc_fte_match_set_misc_bits misc_parameters;
> struct mlx5_ifc_fte_match_set_lyr_2_4_bits inner_headers;
> struct mlx5_ifc_fte_match_set_misc2_bits misc_parameters_2;
> - u8 reserved_at_800[0x800];
> + struct mlx5_ifc_fte_match_set_misc3_bits misc_parameters_3;
> };
>
> enum {
> MLX5_MATCH_CRITERIA_ENABLE_OUTER_BIT,
> MLX5_MATCH_CRITERIA_ENABLE_MISC_BIT,
> MLX5_MATCH_CRITERIA_ENABLE_INNER_BIT,
> - MLX5_MATCH_CRITERIA_ENABLE_MISC2_BIT
> + MLX5_MATCH_CRITERIA_ENABLE_MISC2_BIT,
> + MLX5_MATCH_CRITERIA_ENABLE_MISC3_BIT
> };
>
> enum {
> --
> 1.8.3.1
^ permalink raw reply [flat|nested] 52+ messages in thread
* Re: [dpdk-dev] [PATCH v2 2/3] net/mlx5: add Direct Rules API
2019-04-01 14:38 ` Slava Ovsiienko
@ 2019-04-01 14:38 ` Slava Ovsiienko
0 siblings, 0 replies; 52+ messages in thread
From: Slava Ovsiienko @ 2019-04-01 14:38 UTC (permalink / raw)
To: Ori Kam, Matan Azrad, Yongseok Koh, Shahaf Shuler; +Cc: dev
> -----Original Message-----
> From: Ori Kam
> Sent: Thursday, March 28, 2019 18:33
> To: Matan Azrad <matan@mellanox.com>; Yongseok Koh
> <yskoh@mellanox.com>; Shahaf Shuler <shahafs@mellanox.com>
> Cc: dev@dpdk.org; Ori Kam <orika@mellanox.com>; Slava Ovsiienko
> <viacheslavo@mellanox.com>
> Subject: [PATCH v2 2/3] net/mlx5: add Direct Rules API
>
> Adds calls to the Direct Rules API inside the glue functions.
> Due to difference in parameters between the Direct Rules and Direct Verbs
> some of the glue functions API was updated.
>
> Signed-off-by: Ori Kam <orika@mellanox.com>
Acked-by: Viacheslav Ovsiienko <viacheslavo@mellanox.com>
>
> ---
> drivers/net/mlx5/Makefile | 5 ++
> drivers/net/mlx5/mlx5.c | 16 ++++
> drivers/net/mlx5/mlx5.h | 15 ++++
> drivers/net/mlx5/mlx5_flow.c | 1 +
> drivers/net/mlx5/mlx5_flow.h | 6 +-
> drivers/net/mlx5/mlx5_flow_dv.c | 103 ++++++++++++++++++++----
> drivers/net/mlx5/mlx5_glue.c | 170
> ++++++++++++++++++++++++++++++++++------
> drivers/net/mlx5/mlx5_glue.h | 31 +++++---
> drivers/net/mlx5/mlx5_prm.h | 24 +++++-
> 9 files changed, 318 insertions(+), 53 deletions(-)
>
> diff --git a/drivers/net/mlx5/Makefile b/drivers/net/mlx5/Makefile index
> c326494..3dd7e38 100644
> --- a/drivers/net/mlx5/Makefile
> +++ b/drivers/net/mlx5/Makefile
> @@ -156,6 +156,11 @@ mlx5_autoconf.h.new: $(RTE_SDK)/buildtools/auto-
> config-h.sh
> func mlx5dv_create_flow_action_packet_reformat \
> $(AUTOCONF_OUTPUT)
> $Q sh -- '$<' '$@' \
> + HAVE_MLX5DV_DR \
> + infiniband/mlx5dv.h \
> + enum MLX5DV_DR_NS_TYPE_TERMINATING \
> + $(AUTOCONF_OUTPUT)
> + $Q sh -- '$<' '$@' \
> HAVE_IBV_DEVX_OBJ \
> infiniband/mlx5dv.h \
> func mlx5dv_devx_obj_create \
> diff --git a/drivers/net/mlx5/mlx5.c b/drivers/net/mlx5/mlx5.c index
> 7d1a21e..70e37b5 100644
> --- a/drivers/net/mlx5/mlx5.c
> +++ b/drivers/net/mlx5/mlx5.c
> @@ -1362,6 +1362,22 @@ struct mlx5_dev_spawn_data {
> priv->tcf_context = NULL;
> }
> }
> +#ifdef HAVE_MLX5DV_DR
> + priv->rx_ns = mlx5dv_dr_create_ns
> + (sh->ctx,
> MLX5DV_DR_NS_DOMAIN_INGRESS_BYPASS);
> + if (priv->rx_ns == NULL) {
> + DRV_LOG(ERR, "mlx5dv_dr_create_ns failed");
> + err = errno;
> + goto error;
> + }
> + priv->tx_ns = mlx5dv_dr_create_ns(sh->ctx,
> +
> MLX5DV_DR_NS_DOMAIN_EGRESS_BYPASS);
> + if (priv->tx_ns == NULL) {
> + DRV_LOG(ERR, "mlx5dv_dr_create_ns failed");
> + err = errno;
> + goto error;
> + }
> +#endif
> TAILQ_INIT(&priv->flows);
> TAILQ_INIT(&priv->ctrl_flows);
> /* Hint libmlx5 to use PMD allocator for data plane resources */ diff -
> -git a/drivers/net/mlx5/mlx5.h b/drivers/net/mlx5/mlx5.h index
> b587359..73f6f0d 100644
> --- a/drivers/net/mlx5/mlx5.h
> +++ b/drivers/net/mlx5/mlx5.h
> @@ -219,6 +219,15 @@ struct mlx5_ibv_shared {
> struct mlx5_ibv_shared_port port[]; /* per device port data array. */
> };
>
> +/* Table structure. */
> +struct mlx5_flow_tbl_resource {
> + void *obj; /**< Pointer to DR table object. */
> + rte_atomic32_t refcnt; /**< Reference counter. */ };
> +
> +#define MLX5_MAX_TABLES 1024
> +#define MLX5_GROUP_FACTOR 1
> +
> struct mlx5_priv {
> LIST_ENTRY(mlx5_priv) mem_event_cb;
> /**< Called by memory event callback. */ @@ -289,6 +298,12 @@
> struct mlx5_priv {
> /* UAR same-page access control required in 32bit implementations.
> */ #endif
> struct mlx5_flow_tcf_context *tcf_context; /* TC flower context. */
> + void *rx_ns; /* RX Direct Rules name space handle. */
> + struct mlx5_flow_tbl_resource rx_tbl[MLX5_MAX_TABLES];
> + /* RX Direct Rules tables. */
> + void *tx_ns; /* TX Direct Rules name space handle. */
> + struct mlx5_flow_tbl_resource tx_tbl[MLX5_MAX_TABLES];
> + /* TX Direct Rules tables/ */
> };
>
> #define PORT_ID(priv) ((priv)->dev_data->port_id) diff --git
> a/drivers/net/mlx5/mlx5_flow.c b/drivers/net/mlx5/mlx5_flow.c index
> d09fdff..40d6818 100644
> --- a/drivers/net/mlx5/mlx5_flow.c
> +++ b/drivers/net/mlx5/mlx5_flow.c
> @@ -2084,6 +2084,7 @@ uint32_t mlx5_flow_adjust_priority(struct
> rte_eth_dev *dev, int32_t priority,
> flow_size += RTE_ALIGN_CEIL(sizeof(uint16_t), sizeof(void
> *));
> flow = rte_calloc(__func__, 1, flow_size, 0);
> flow->drv_type = flow_get_drv_type(dev, attr);
> + flow->ingress = attr->ingress;
> assert(flow->drv_type > MLX5_FLOW_TYPE_MIN &&
> flow->drv_type < MLX5_FLOW_TYPE_MAX);
> flow->queue = (void *)(flow + 1);
> diff --git a/drivers/net/mlx5/mlx5_flow.h b/drivers/net/mlx5/mlx5_flow.h
> index 4f69ae2..8ba37a0 100644
> --- a/drivers/net/mlx5/mlx5_flow.h
> +++ b/drivers/net/mlx5/mlx5_flow.h
> @@ -204,6 +204,7 @@ struct mlx5_flow_dv_matcher {
> uint16_t crc; /**< CRC of key. */
> uint16_t priority; /**< Priority of matcher. */
> uint8_t egress; /**< Egress matcher. */
> + uint32_t group; /**< The matcher group. */
> struct mlx5_flow_dv_match_params mask; /**< Matcher mask. */ };
>
> @@ -220,6 +221,7 @@ struct mlx5_flow_dv_encap_decap_resource {
> size_t size;
> uint8_t reformat_type;
> uint8_t ft_type;
> + uint64_t flags; /**< Flags for RDMA API. */
> };
>
> /* Tag resource structure. */
> @@ -348,7 +350,7 @@ struct mlx5_flow_counter {
> /* Flow structure. */
> struct rte_flow {
> TAILQ_ENTRY(rte_flow) next; /**< Pointer to the next flow structure.
> */
> - enum mlx5_flow_drv_type drv_type; /**< Drvier type. */
> + enum mlx5_flow_drv_type drv_type; /**< Driver type. */
> struct mlx5_flow_counter *counter; /**< Holds flow counter. */
> struct mlx5_flow_dv_tag_resource *tag_resource;
> /**< pointer to the tag action. */
> @@ -360,6 +362,8 @@ struct rte_flow {
> uint64_t actions;
> /**< Bit-fields of detected actions, see MLX5_FLOW_ACTION_*. */
> struct mlx5_fdir *fdir; /**< Pointer to associated FDIR if any. */
> + uint8_t ingress; /**< 1 if the flow is ingress. */
> + uint32_t group; /**< The group index. */
> };
>
> typedef int (*mlx5_flow_validate_t)(struct rte_eth_dev *dev, diff --git
> a/drivers/net/mlx5/mlx5_flow_dv.c b/drivers/net/mlx5/mlx5_flow_dv.c
> index b0c50a7..6e4f6c4 100644
> --- a/drivers/net/mlx5/mlx5_flow_dv.c
> +++ b/drivers/net/mlx5/mlx5_flow_dv.c
> @@ -805,11 +805,20 @@ struct field_modify_info modify_tcp[] = { {
> struct mlx5_priv *priv = dev->data->dev_private;
> struct mlx5_flow_dv_encap_decap_resource *cache_resource;
> + struct rte_flow *flow = dev_flow->flow;
> + struct mlx5dv_dr_ns *ns;
> +
> + resource->flags = flow->group ? 0 : 1;
> + if (flow->ingress)
> + ns = priv->rx_ns;
> + else
> + ns = priv->tx_ns;
>
> /* Lookup a matching resource from cache. */
> LIST_FOREACH(cache_resource, &priv->encaps_decaps, next) {
> if (resource->reformat_type == cache_resource-
> >reformat_type &&
> resource->ft_type == cache_resource->ft_type &&
> + resource->flags == cache_resource->flags &&
> resource->size == cache_resource->size &&
> !memcmp((const void *)resource->buf,
> (const void *)cache_resource->buf, @@ -831,10
> +840,10 @@ struct field_modify_info modify_tcp[] = {
> *cache_resource = *resource;
> cache_resource->verbs_action =
> mlx5_glue->dv_create_flow_action_packet_reformat
> - (priv->sh->ctx, cache_resource->size,
> - (cache_resource->size ? cache_resource->buf :
> NULL),
> - cache_resource->reformat_type,
> - cache_resource->ft_type);
> + (priv->sh->ctx, cache_resource->reformat_type,
> + cache_resource->ft_type, ns, cache_resource->flags,
> + cache_resource->size,
> + (cache_resource->size ? cache_resource->buf :
> NULL));
> if (!cache_resource->verbs_action) {
> rte_free(cache_resource);
> return rte_flow_error_set(error, ENOMEM, @@ -1438,6
> +1447,10 @@ struct field_modify_info modify_tcp[] = {
> struct mlx5_priv *priv = dev->data->dev_private;
> struct mlx5_flow_dv_modify_hdr_resource *cache_resource;
>
> + struct mlx5dv_dr_ns *ns =
> + resource->ft_type == MLX5DV_FLOW_TABLE_TYPE_NIC_TX ?
> + priv->tx_ns : priv->rx_ns;
> +
> /* Lookup a matching resource from cache. */
> LIST_FOREACH(cache_resource, &priv->modify_cmds, next) {
> if (resource->ft_type == cache_resource->ft_type && @@ -
> 1463,11 +1476,11 @@ struct field_modify_info modify_tcp[] = {
> *cache_resource = *resource;
> cache_resource->verbs_action =
> mlx5_glue->dv_create_flow_action_modify_header
> - (priv->sh->ctx,
> + (priv->sh->ctx, cache_resource-
> >ft_type,
> + ns, 0,
> cache_resource->actions_num *
> sizeof(cache_resource->actions[0]),
> - (uint64_t *)cache_resource->actions,
> - cache_resource->ft_type);
> + (uint64_t *)cache_resource-
> >actions);
> if (!cache_resource->verbs_action) {
> rte_free(cache_resource);
> return rte_flow_error_set(error, ENOMEM, @@ -1592,11
> +1605,13 @@ struct field_modify_info modify_tcp[] = {
> struct mlx5_priv *priv = dev->data->dev_private;
> uint32_t priority_max = priv->config.flow_prio - 1;
>
> +#ifdef HAVE_MLX5DV_DR
> if (attributes->group)
> return rte_flow_error_set(error, ENOTSUP,
>
> RTE_FLOW_ERROR_TYPE_ATTR_GROUP,
> NULL,
> "groups is not supported");
> +#endif
> if (attributes->priority != MLX5_FLOW_PRIO_RSVD &&
> attributes->priority >= priority_max)
> return rte_flow_error_set(error, ENOTSUP, @@ -2169,11
> +2184,13 @@ struct field_modify_info modify_tcp[] = {
> * Flow pattern to translate.
> * @param[in] inner
> * Item is inner pattern.
> + * @param[in] group
> + * The group to insert the rule.
> */
> static void
> flow_dv_translate_item_ipv4(void *matcher, void *key,
> const struct rte_flow_item *item,
> - int inner)
> + int inner, uint32_t group)
> {
> const struct rte_flow_item_ipv4 *ipv4_m = item->mask;
> const struct rte_flow_item_ipv4 *ipv4_v = item->spec; @@ -2200,7
> +2217,10 @@ struct field_modify_info modify_tcp[] = {
> outer_headers);
> headers_v = MLX5_ADDR_OF(fte_match_param, key,
> outer_headers);
> }
> - MLX5_SET(fte_match_set_lyr_2_4, headers_m, ip_version, 0xf);
> + if (group == 0)
> + MLX5_SET(fte_match_set_lyr_2_4, headers_m, ip_version,
> 0xf);
> + else
> + MLX5_SET(fte_match_set_lyr_2_4, headers_m, ip_version,
> 0x4);
> MLX5_SET(fte_match_set_lyr_2_4, headers_v, ip_version, 4);
> if (!ipv4_v)
> return;
> @@ -2242,11 +2262,13 @@ struct field_modify_info modify_tcp[] = {
> * Flow pattern to translate.
> * @param[in] inner
> * Item is inner pattern.
> + * @param[in] group
> + * The group to insert the rule.
> */
> static void
> flow_dv_translate_item_ipv6(void *matcher, void *key,
> const struct rte_flow_item *item,
> - int inner)
> + int inner, uint32_t group)
> {
> const struct rte_flow_item_ipv6 *ipv6_m = item->mask;
> const struct rte_flow_item_ipv6 *ipv6_v = item->spec; @@ -2283,7
> +2305,10 @@ struct field_modify_info modify_tcp[] = {
> outer_headers);
> headers_v = MLX5_ADDR_OF(fte_match_param, key,
> outer_headers);
> }
> - MLX5_SET(fte_match_set_lyr_2_4, headers_m, ip_version, 0xf);
> + if (group == 0)
> + MLX5_SET(fte_match_set_lyr_2_4, headers_m, ip_version,
> 0xf);
> + else
> + MLX5_SET(fte_match_set_lyr_2_4, headers_m, ip_version,
> 0x6);
> MLX5_SET(fte_match_set_lyr_2_4, headers_v, ip_version, 6);
> if (!ipv6_v)
> return;
> @@ -2723,7 +2748,11 @@ struct field_modify_info modify_tcp[] = {
> match_criteria_enable |=
> (!HEADER_IS_ZERO(match_criteria, misc_parameters_2)) <<
> MLX5_MATCH_CRITERIA_ENABLE_MISC2_BIT;
> -
> +#ifdef HAVE_MLX5DV_DR
> + match_criteria_enable |=
> + (!HEADER_IS_ZERO(match_criteria, misc_parameters_3)) <<
> + MLX5_MATCH_CRITERIA_ENABLE_MISC3_BIT;
> +#endif
> return match_criteria_enable;
> }
>
> @@ -2754,12 +2783,14 @@ struct field_modify_info modify_tcp[] = {
> .type = IBV_FLOW_ATTR_NORMAL,
> .match_mask = (void *)&matcher->mask,
> };
> + struct mlx5_flow_tbl_resource *tbl = NULL;
>
> /* Lookup from cache. */
> LIST_FOREACH(cache_matcher, &priv->matchers, next) {
> if (matcher->crc == cache_matcher->crc &&
> matcher->priority == cache_matcher->priority &&
> matcher->egress == cache_matcher->egress &&
> + matcher->group == cache_matcher->group &&
> !memcmp((const void *)matcher->mask.buf,
> (const void *)cache_matcher->mask.buf,
> cache_matcher->mask.size)) {
> @@ -2774,6 +2805,27 @@ struct field_modify_info modify_tcp[] = {
> return 0;
> }
> }
> +#ifdef HAVE_MLX5DV_DR
> + if (matcher->egress) {
> + tbl = &priv->tx_tbl[matcher->group];
> + if (!tbl->obj)
> + tbl->obj = mlx5_glue->dr_create_flow_tbl
> + (priv->tx_ns,
> + matcher->group * MLX5_GROUP_FACTOR);
> + } else {
> + tbl = &priv->rx_tbl[matcher->group];
> + if (!tbl->obj)
> + tbl->obj = mlx5_glue->dr_create_flow_tbl
> + (priv->rx_ns,
> + matcher->group * MLX5_GROUP_FACTOR);
> + }
> + if (!tbl->obj)
> + return rte_flow_error_set(error, ENOMEM,
> +
> RTE_FLOW_ERROR_TYPE_UNSPECIFIED,
> + NULL, "cannot create table");
> +
> + rte_atomic32_inc(&tbl->refcnt);
> +#endif
> /* Register new matcher. */
> cache_matcher = rte_calloc(__func__, 1, sizeof(*cache_matcher), 0);
> if (!cache_matcher)
> @@ -2787,9 +2839,16 @@ struct field_modify_info modify_tcp[] = {
> if (matcher->egress)
> dv_attr.flags |= IBV_FLOW_ATTR_FLAGS_EGRESS;
> cache_matcher->matcher_object =
> - mlx5_glue->dv_create_flow_matcher(priv->sh->ctx,
> &dv_attr);
> + mlx5_glue->dv_create_flow_matcher(priv->sh->ctx,
> &dv_attr,
> + tbl->obj);
> if (!cache_matcher->matcher_object) {
> rte_free(cache_matcher);
> +#ifdef HAVE_MLX5DV_DR
> + if (rte_atomic32_dec_and_test(&tbl->refcnt)) {
> + mlx5_glue->dr_destroy_flow_tbl(tbl->obj);
> + tbl->obj = NULL;
> + }
> +#endif
> return rte_flow_error_set(error, ENOMEM,
>
> RTE_FLOW_ERROR_TYPE_UNSPECIFIED,
> NULL, "cannot create matcher");
> @@ -2801,6 +2860,7 @@ struct field_modify_info modify_tcp[] = {
> cache_matcher->priority,
> cache_matcher->egress ? "tx" : "rx", (void *)cache_matcher,
> rte_atomic32_read(&cache_matcher->refcnt));
> + rte_atomic32_inc(&tbl->refcnt);
> return 0;
> }
>
> @@ -3222,7 +3282,7 @@ struct field_modify_info modify_tcp[] = {
> break;
> case RTE_FLOW_ITEM_TYPE_IPV4:
> flow_dv_translate_item_ipv4(match_mask,
> match_value,
> - items, tunnel);
> + items, tunnel, attr->group);
> matcher.priority = MLX5_PRIORITY_MAP_L3;
> dev_flow->dv.hash_fields |=
> mlx5_flow_hashfields_adjust
> @@ -3234,7 +3294,7 @@ struct field_modify_info modify_tcp[] = {
> break;
> case RTE_FLOW_ITEM_TYPE_IPV6:
> flow_dv_translate_item_ipv6(match_mask,
> match_value,
> - items, tunnel);
> + items, tunnel, attr->group);
> matcher.priority = MLX5_PRIORITY_MAP_L3;
> dev_flow->dv.hash_fields |=
> mlx5_flow_hashfields_adjust
> @@ -3312,6 +3372,7 @@ struct field_modify_info modify_tcp[] = {
> matcher.priority = mlx5_flow_adjust_priority(dev, priority,
> matcher.priority);
> matcher.egress = attr->egress;
> + matcher.group = attr->group;
> if (flow_dv_matcher_register(dev, &matcher, dev_flow, error))
> return -rte_errno;
> return 0;
> @@ -3427,6 +3488,8 @@ struct field_modify_info modify_tcp[] = {
> struct mlx5_flow *flow)
> {
> struct mlx5_flow_dv_matcher *matcher = flow->dv.matcher;
> + struct mlx5_priv *priv = dev->data->dev_private;
> + struct mlx5_flow_tbl_resource *tbl;
>
> assert(matcher->matcher_object);
> DRV_LOG(DEBUG, "port %u matcher %p: refcnt %d--", @@ -3436,6
> +3499,14 @@ struct field_modify_info modify_tcp[] = {
> claim_zero(mlx5_glue->dv_destroy_flow_matcher
> (matcher->matcher_object));
> LIST_REMOVE(matcher, next);
> + if (matcher->egress)
> + tbl = &priv->tx_tbl[matcher->group];
> + else
> + tbl = &priv->rx_tbl[matcher->group];
> + if (rte_atomic32_dec_and_test(&tbl->refcnt)) {
> + mlx5_glue->dr_destroy_flow_tbl(tbl->obj);
> + tbl->obj = NULL;
> + }
> rte_free(matcher);
> DRV_LOG(DEBUG, "port %u matcher %p: removed",
> dev->data->port_id, (void *)matcher); @@ -3525,7
> +3596,7 @@ struct field_modify_info modify_tcp[] = {
> LIST_FOREACH(dev_flow, &flow->dev_flows, next) {
> dv = &dev_flow->dv;
> if (dv->flow) {
> - claim_zero(mlx5_glue->destroy_flow(dv->flow));
> + claim_zero(mlx5_glue->dv_destroy_flow(dv->flow));
> dv->flow = NULL;
> }
> if (dv->hrxq) {
> diff --git a/drivers/net/mlx5/mlx5_glue.c b/drivers/net/mlx5/mlx5_glue.c
> index cfd939d..f509f85 100644
> --- a/drivers/net/mlx5/mlx5_glue.c
> +++ b/drivers/net/mlx5/mlx5_glue.c
> @@ -178,6 +178,9 @@
> mlx5_glue_destroy_flow_action(void *action) { #ifdef
> HAVE_IBV_FLOW_DV_SUPPORT
> +#ifdef HAVE_MLX5DV_DR
> + return mlx5dv_dr_destroy_action(action); #else
> struct mlx5dv_flow_action_attr *attr = action;
> int res = 0;
> switch (attr->type) {
> @@ -189,6 +192,7 @@
> }
> free(action);
> return res;
> +#endif
> #else
> (void)action;
> return ENOTSUP;
> @@ -365,6 +369,53 @@
> return ibv_cq_ex_to_cq(cq);
> }
>
> +static void *
> +mlx5_glue_dr_create_flow_tbl(void *ns, uint32_t level) { #ifdef
> +HAVE_MLX5DV_DR
> + return mlx5dv_dr_create_ft(ns, level); #else
> + (void)ns;
> + (void)level;
> + return NULL;
> +#endif
> +}
> +
> +static int
> +mlx5_glue_dr_destroy_flow_tbl(void *tbl) { #ifdef HAVE_MLX5DV_DR
> + return mlx5dv_dr_destroy_ft(tbl);
> +#else
> + (void)tbl;
> + return 0;
> +#endif
> +}
> +
> +static void *
> +mlx5_glue_dr_create_ns(struct ibv_context *ctx,
> + enum mlx5dv_dr_ns_domain domain) { #ifdef
> HAVE_MLX5DV_DR
> + return mlx5dv_dr_create_ns(ctx, domain); #else
> + (void)ctx;
> + (void)domain;
> + return NULL;
> +#endif
> +}
> +
> +static int
> +mlx5_glue_dr_destroy_ns(void *ns)
> +{
> +#ifdef HAVE_MLX5DV_DR
> + return mlx5dv_dr_destroy_ns(ns);
> +#else
> + (void)ns;
> + return 0;
> +#endif
> +}
> +
> static struct ibv_cq_ex *
> mlx5_glue_dv_create_cq(struct ibv_context *context,
> struct ibv_cq_init_attr_ex *cq_attr, @@ -423,26 +474,40
> @@ #endif }
>
> -static struct mlx5dv_flow_matcher *
> +static void *
> mlx5_glue_dv_create_flow_matcher(struct ibv_context *context,
> - struct mlx5dv_flow_matcher_attr
> *matcher_attr)
> + struct mlx5dv_flow_matcher_attr
> *matcher_attr,
> + void *tbl)
> {
> #ifdef HAVE_IBV_FLOW_DV_SUPPORT
> +#ifdef HAVE_MLX5DV_DR
> + (void)context;
> + return mlx5dv_dr_create_matcher(tbl, matcher_attr->priority,
> + matcher_attr->match_criteria_enable,
> + matcher_attr->match_mask);
> +#else
> + (void)tbl;
> return mlx5dv_create_flow_matcher(context, matcher_attr);
> +#endif
> #else
> (void)context;
> (void)matcher_attr;
> + (void)tbl;
> return NULL;
> #endif
> }
>
> -static struct ibv_flow *
> -mlx5_glue_dv_create_flow(struct mlx5dv_flow_matcher *matcher,
> - struct mlx5dv_flow_match_parameters
> *match_value,
> +static void *
> +mlx5_glue_dv_create_flow(void *matcher,
> + void *match_value,
> size_t num_actions,
> void *actions[])
> {
> #ifdef HAVE_IBV_FLOW_DV_SUPPORT
> +#ifdef HAVE_MLX5DV_DR
> + return mlx5dv_dr_create_rule(matcher, match_value, num_actions,
> + (struct mlx5dv_dr_action **)actions); #else
> struct mlx5dv_flow_action_attr actions_attr[8];
>
> if (num_actions > 8)
> @@ -452,6 +517,7 @@
> *((struct mlx5dv_flow_action_attr *)(actions[i]));
> return mlx5dv_create_flow(matcher, match_value,
> num_actions, actions_attr);
> +#endif
> #else
> (void)matcher;
> (void)match_value;
> @@ -461,21 +527,13 @@
> #endif
> }
>
> -static int
> -mlx5_glue_dv_destroy_flow_matcher(struct mlx5dv_flow_matcher
> *matcher) -{ -#ifdef HAVE_IBV_FLOW_DV_SUPPORT
> - return mlx5dv_destroy_flow_matcher(matcher);
> -#else
> - (void)matcher;
> - return 0;
> -#endif
> -}
> -
> static void *
> mlx5_glue_dv_create_flow_action_counter(void *counter_obj, uint32_t
> offset) { #ifdef HAVE_IBV_FLOW_DV_SUPPORT
> +#ifdef HAVE_MLX5DV_DR
> + return mlx5dv_dr_create_action_devx_counter(counter_obj, offset);
> +#else
> struct mlx5dv_flow_action_attr *action;
>
> (void)offset;
> @@ -485,6 +543,7 @@
> action->type = MLX5DV_FLOW_ACTION_COUNTER_DEVX;
> action->obj = counter_obj;
> return action;
> +#endif
> #else
> (void)counter_obj;
> (void)offset;
> @@ -496,6 +555,9 @@
> mlx5_glue_dv_create_flow_action_dest_ibv_qp(void *qp) { #ifdef
> HAVE_IBV_FLOW_DV_SUPPORT
> +#ifdef HAVE_MLX5DV_DR
> + return mlx5dv_dr_create_action_dest_ibv_qp(qp);
> +#else
> struct mlx5dv_flow_action_attr *action;
>
> action = malloc(sizeof(*action));
> @@ -504,6 +566,7 @@
> action->type = MLX5DV_FLOW_ACTION_DEST_IBV_QP;
> action->obj = qp;
> return action;
> +#endif
> #else
> (void)qp;
> return NULL;
> @@ -513,13 +576,22 @@
> static void *
> mlx5_glue_dv_create_flow_action_modify_header
> (struct ibv_context *ctx,
> + enum mlx5dv_flow_table_type
> ft_type,
> + void *ns, uint64_t flags,
> size_t actions_sz,
> - uint64_t actions[],
> - enum mlx5dv_flow_table_type
> ft_type)
> + uint64_t actions[])
> {
> #ifdef HAVE_IBV_FLOW_DV_SUPPORT
> +#ifdef HAVE_MLX5DV_DR
> + (void)ctx;
> + (void)ft_type;
> + return mlx5dv_dr_create_action_modify_header(ns, flags,
> actions_sz,
> + actions);
> +#else
> struct mlx5dv_flow_action_attr *action;
>
> + (void)ns;
> + (void)flags;
> action = malloc(sizeof(*action));
> if (!action)
> return NULL;
> @@ -527,11 +599,14 @@
> action->action = mlx5dv_create_flow_action_modify_header
> (ctx, actions_sz, actions, ft_type);
> return action;
> +#endif
> #else
> (void)ctx;
> + (void)ft_type;
> + (void)ns;
> + (void)flags;
> (void)actions_sz;
> (void)actions;
> - (void)ft_type;
> return NULL;
> #endif
> }
> @@ -539,12 +614,20 @@
> static void *
> mlx5_glue_dv_create_flow_action_packet_reformat
> (struct ibv_context *ctx,
> - size_t data_sz,
> - void *data,
> enum mlx5dv_flow_action_packet_reformat_type
> reformat_type,
> - enum mlx5dv_flow_table_type ft_type)
> + enum mlx5dv_flow_table_type ft_type, struct mlx5dv_dr_ns
> *ns,
> + uint32_t flags, size_t data_sz, void *data)
> {
> #ifdef HAVE_IBV_FLOW_DV_SUPPORT
> +#ifdef HAVE_MLX5DV_DR
> + (void)ctx;
> + (void)ft_type;
> + return mlx5dv_dr_create_action_packet_reformat(ns, flags,
> + reformat_type, data_sz,
> + data);
> +#else
> + (void)ns;
> + (void)flags;
> struct mlx5dv_flow_action_attr *action;
>
> action = malloc(sizeof(*action));
> @@ -554,12 +637,15 @@
> action->action = mlx5dv_create_flow_action_packet_reformat
> (ctx, data_sz, data, reformat_type, ft_type);
> return action;
> +#endif
> #else
> (void)ctx;
> - (void)data_sz;
> - (void)data;
> (void)reformat_type;
> (void)ft_type;
> + (void)ns;
> + (void)flags;
> + (void)data_sz;
> + (void)data;
> return NULL;
> #endif
> }
> @@ -568,6 +654,9 @@
> mlx5_glue_dv_create_flow_action_tag(uint32_t tag) { #ifdef
> HAVE_IBV_FLOW_DV_SUPPORT
> +#ifdef HAVE_MLX5DV_DR
> + return mlx5dv_dr_create_action_tag(tag); #else
> struct mlx5dv_flow_action_attr *action;
> action = malloc(sizeof(*action));
> if (!action)
> @@ -576,10 +665,36 @@
> action->tag_value = tag;
> return action;
> #endif
> +#endif
> (void)tag;
> return NULL;
> }
>
> +static int
> +mlx5_glue_dv_destroy_flow(void *flow_id) { #ifdef HAVE_MLX5DV_DR
> + return mlx5dv_dr_destroy_rule(flow_id); #else
> + return ibv_destroy_flow(flow_id);
> +#endif
> +}
> +
> +static int
> +mlx5_glue_dv_destroy_flow_matcher(void *matcher) { #ifdef
> +HAVE_IBV_FLOW_DV_SUPPORT #ifdef HAVE_MLX5DV_DR
> + return mlx5dv_dr_destroy_matcher(matcher);
> +#else
> + return mlx5dv_destroy_flow_matcher(matcher);
> +#endif
> +#else
> + (void)matcher;
> + return 0;
> +#endif
> +}
> +
> static struct ibv_context *
> mlx5_glue_dv_open_device(struct ibv_device *device) { @@ -718,6 +833,10
> @@
> .get_async_event = mlx5_glue_get_async_event,
> .port_state_str = mlx5_glue_port_state_str,
> .cq_ex_to_cq = mlx5_glue_cq_ex_to_cq,
> + .dr_create_flow_tbl = mlx5_glue_dr_create_flow_tbl,
> + .dr_destroy_flow_tbl = mlx5_glue_dr_destroy_flow_tbl,
> + .dr_create_ns = mlx5_glue_dr_create_ns,
> + .dr_destroy_ns = mlx5_glue_dr_destroy_ns,
> .dv_create_cq = mlx5_glue_dv_create_cq,
> .dv_create_wq = mlx5_glue_dv_create_wq,
> .dv_query_device = mlx5_glue_dv_query_device, @@ -725,7 +844,6
> @@
> .dv_init_obj = mlx5_glue_dv_init_obj,
> .dv_create_qp = mlx5_glue_dv_create_qp,
> .dv_create_flow_matcher = mlx5_glue_dv_create_flow_matcher,
> - .dv_destroy_flow_matcher = mlx5_glue_dv_destroy_flow_matcher,
> .dv_create_flow = mlx5_glue_dv_create_flow,
> .dv_create_flow_action_counter =
> mlx5_glue_dv_create_flow_action_counter,
> @@ -736,6 +854,8 @@
> .dv_create_flow_action_packet_reformat =
> mlx5_glue_dv_create_flow_action_packet_reformat,
> .dv_create_flow_action_tag =
> mlx5_glue_dv_create_flow_action_tag,
> + .dv_destroy_flow = mlx5_glue_dv_destroy_flow,
> + .dv_destroy_flow_matcher = mlx5_glue_dv_destroy_flow_matcher,
> .dv_open_device = mlx5_glue_dv_open_device,
> .devx_obj_create = mlx5_glue_devx_obj_create,
> .devx_obj_destroy = mlx5_glue_devx_obj_destroy, diff --git
> a/drivers/net/mlx5/mlx5_glue.h b/drivers/net/mlx5/mlx5_glue.h index
> 167f1f7..7115575 100644
> --- a/drivers/net/mlx5/mlx5_glue.h
> +++ b/drivers/net/mlx5/mlx5_glue.h
> @@ -63,6 +63,11 @@
> struct mlx5dv_devx_obj;
> #endif
>
> +#ifndef HAVE_MLX5DV_DR
> +struct mlx5dv_dr_ns;
> +enum mlx5dv_dr_ns_domain { unused, };
> +#endif
> +
> /* LIB_GLUE_VERSION must be updated every time this structure is
> modified. */ struct mlx5_glue {
> const char *version;
> @@ -140,6 +145,11 @@ struct mlx5_glue {
> struct ibv_async_event *event);
> const char *(*port_state_str)(enum ibv_port_state port_state);
> struct ibv_cq *(*cq_ex_to_cq)(struct ibv_cq_ex *cq);
> + void *(*dr_create_flow_tbl)(void *ns, uint32_t level);
> + int (*dr_destroy_flow_tbl)(void *tbl);
> + void *(*dr_create_ns)(struct ibv_context *ctx,
> + enum mlx5dv_dr_ns_domain domain);
> + int (*dr_destroy_ns)(void *ns);
> struct ibv_cq_ex *(*dv_create_cq)
> (struct ibv_context *context,
> struct ibv_cq_init_attr_ex *cq_attr, @@ -158,23 +168,26
> @@ struct mlx5_glue {
> (struct ibv_context *context,
> struct ibv_qp_init_attr_ex *qp_init_attr_ex,
> struct mlx5dv_qp_init_attr *dv_qp_init_attr);
> - struct mlx5dv_flow_matcher *(*dv_create_flow_matcher)
> + void *(*dv_create_flow_matcher)
> (struct ibv_context *context,
> - struct mlx5dv_flow_matcher_attr *matcher_attr);
> - int (*dv_destroy_flow_matcher)(struct mlx5dv_flow_matcher
> *matcher);
> - struct ibv_flow *(*dv_create_flow)(struct mlx5dv_flow_matcher
> *matcher,
> - struct mlx5dv_flow_match_parameters
> *match_value,
> + struct mlx5dv_flow_matcher_attr *matcher_attr,
> + void *tbl);
> + void *(*dv_create_flow)(void *matcher, void *match_value,
> size_t num_actions, void *actions[]);
> void *(*dv_create_flow_action_counter)(void *obj, uint32_t offset);
> void *(*dv_create_flow_action_dest_ibv_qp)(void *qp);
> void *(*dv_create_flow_action_modify_header)
> - (struct ibv_context *ctx, size_t actions_sz, uint64_t actions[],
> - enum mlx5dv_flow_table_type ft_type);
> + (struct ibv_context *ctx, enum mlx5dv_flow_table_type
> ft_type,
> + void *ns, uint64_t flags, size_t actions_sz,
> + uint64_t actions[]);
> void *(*dv_create_flow_action_packet_reformat)
> - (struct ibv_context *ctx, size_t data_sz, void *data,
> + (struct ibv_context *ctx,
> enum mlx5dv_flow_action_packet_reformat_type
> reformat_type,
> - enum mlx5dv_flow_table_type ft_type);
> + enum mlx5dv_flow_table_type ft_type, struct mlx5dv_dr_ns
> *ns,
> + uint32_t flags, size_t data_sz, void *data);
> void *(*dv_create_flow_action_tag)(uint32_t tag);
> + int (*dv_destroy_flow)(void *flow);
> + int (*dv_destroy_flow_matcher)(void *matcher);
> struct ibv_context *(*dv_open_device)(struct ibv_device *device);
> struct mlx5dv_devx_obj *(*devx_obj_create)
> (struct ibv_context *ctx,
> diff --git a/drivers/net/mlx5/mlx5_prm.h b/drivers/net/mlx5/mlx5_prm.h
> index da1219e..b15266f 100644
> --- a/drivers/net/mlx5/mlx5_prm.h
> +++ b/drivers/net/mlx5/mlx5_prm.h
> @@ -492,20 +492,40 @@ struct mlx5_ifc_fte_match_set_misc2_bits {
> u8 reserved_at_1a0[0x60];
> };
>
> +struct mlx5_ifc_fte_match_set_misc3_bits {
> + u8 inner_tcp_seq_num[0x20];
> + u8 outer_tcp_seq_num[0x20];
> + u8 inner_tcp_ack_num[0x20];
> + u8 outer_tcp_ack_num[0x20];
> + u8 reserved_at_auto1[0x8];
> + u8 outer_vxlan_gpe_vni[0x18];
> + u8 outer_vxlan_gpe_next_protocol[0x8];
> + u8 outer_vxlan_gpe_flags[0x8];
> + u8 reserved_at_a8[0x10];
> + u8 icmp_header_data[0x20];
> + u8 icmpv6_header_data[0x20];
> + u8 icmp_type[0x8];
> + u8 icmp_code[0x8];
> + u8 icmpv6_type[0x8];
> + u8 icmpv6_code[0x8];
> + u8 reserved_at_1a0[0xe0];
> +};
> +
> /* Flow matcher. */
> struct mlx5_ifc_fte_match_param_bits {
> struct mlx5_ifc_fte_match_set_lyr_2_4_bits outer_headers;
> struct mlx5_ifc_fte_match_set_misc_bits misc_parameters;
> struct mlx5_ifc_fte_match_set_lyr_2_4_bits inner_headers;
> struct mlx5_ifc_fte_match_set_misc2_bits misc_parameters_2;
> - u8 reserved_at_800[0x800];
> + struct mlx5_ifc_fte_match_set_misc3_bits misc_parameters_3;
> };
>
> enum {
> MLX5_MATCH_CRITERIA_ENABLE_OUTER_BIT,
> MLX5_MATCH_CRITERIA_ENABLE_MISC_BIT,
> MLX5_MATCH_CRITERIA_ENABLE_INNER_BIT,
> - MLX5_MATCH_CRITERIA_ENABLE_MISC2_BIT
> + MLX5_MATCH_CRITERIA_ENABLE_MISC2_BIT,
> + MLX5_MATCH_CRITERIA_ENABLE_MISC3_BIT
> };
>
> enum {
> --
> 1.8.3.1
^ permalink raw reply [flat|nested] 52+ messages in thread
* Re: [dpdk-dev] [PATCH v2 3/3] net/mlx5: add jump action support for NIC
2019-03-28 16:32 ` [dpdk-dev] [PATCH v2 3/3] net/mlx5: add jump action support for NIC Ori Kam
2019-03-28 16:32 ` Ori Kam
@ 2019-04-01 14:38 ` Slava Ovsiienko
2019-04-01 14:38 ` Slava Ovsiienko
2019-04-03 10:16 ` Shahaf Shuler
2 siblings, 1 reply; 52+ messages in thread
From: Slava Ovsiienko @ 2019-04-01 14:38 UTC (permalink / raw)
To: Ori Kam, Matan Azrad, Yongseok Koh, Shahaf Shuler; +Cc: dev
> -----Original Message-----
> From: Ori Kam
> Sent: Thursday, March 28, 2019 18:33
> To: Matan Azrad <matan@mellanox.com>; Yongseok Koh
> <yskoh@mellanox.com>; Shahaf Shuler <shahafs@mellanox.com>
> Cc: dev@dpdk.org; Ori Kam <orika@mellanox.com>; Slava Ovsiienko
> <viacheslavo@mellanox.com>
> Subject: [PATCH v2 3/3] net/mlx5: add jump action support for NIC
>
> When using Direct Rules we can add actions to jump between tables.
> This is extra useful since rule insertion rate is much higher on other tables
> compared to table zero.
>
> if no group is selected the rule is added to group 0.
>
> Signed-off-by: Ori Kam <orika@mellanox.com>
Acked-by: Viacheslav Ovsiienko <viacheslavo@mellanox.com>
> ---
> drivers/net/mlx5/mlx5.h | 6 +
> drivers/net/mlx5/mlx5_flow.h | 15 ++-
> drivers/net/mlx5/mlx5_flow_dv.c | 278
> +++++++++++++++++++++++++++++++++++-----
> drivers/net/mlx5/mlx5_glue.c | 13 ++
> drivers/net/mlx5/mlx5_glue.h | 1 +
> 5 files changed, 282 insertions(+), 31 deletions(-)
>
> diff --git a/drivers/net/mlx5/mlx5.h b/drivers/net/mlx5/mlx5.h index
> 73f6f0d..a3d5f8e 100644
> --- a/drivers/net/mlx5/mlx5.h
> +++ b/drivers/net/mlx5/mlx5.h
> @@ -279,6 +279,12 @@ struct mlx5_priv {
> LIST_HEAD(encap_decap, mlx5_flow_dv_encap_decap_resource)
> encaps_decaps;
> LIST_HEAD(modify_cmd, mlx5_flow_dv_modify_hdr_resource)
> modify_cmds;
> LIST_HEAD(tag, mlx5_flow_dv_tag_resource) tags;
> + LIST_HEAD(jump, mlx5_flow_dv_jump_tbl_resource) jump_tbl;
> + /* Pointer to next element. */
> + rte_atomic32_t refcnt; /**< Reference counter. */
> + struct ibv_flow_action *verbs_action;
> + /**< Verbs modify header action object. */
> + uint8_t ft_type; /**< Flow table type, Rx or Tx. */
> /* Tags resources cache. */
> uint32_t link_speed_capa; /* Link speed capabilities. */
> struct mlx5_xstats_ctrl xstats_ctrl; /* Extended stats control. */ diff --
> git a/drivers/net/mlx5/mlx5_flow.h b/drivers/net/mlx5/mlx5_flow.h index
> 8ba37a0..622e305 100644
> --- a/drivers/net/mlx5/mlx5_flow.h
> +++ b/drivers/net/mlx5/mlx5_flow.h
> @@ -115,7 +115,8 @@
> #define MLX5_FLOW_ACTION_RAW_DECAP (1u << 27)
>
> #define MLX5_FLOW_FATE_ACTIONS \
> - (MLX5_FLOW_ACTION_DROP | MLX5_FLOW_ACTION_QUEUE |
> MLX5_FLOW_ACTION_RSS)
> + (MLX5_FLOW_ACTION_DROP | MLX5_FLOW_ACTION_QUEUE | \
> + MLX5_FLOW_ACTION_RSS | MLX5_FLOW_ACTION_JUMP)
>
> #define MLX5_FLOW_ENCAP_ACTIONS
> (MLX5_FLOW_ACTION_VXLAN_ENCAP | \
> MLX5_FLOW_ACTION_NVGRE_ENCAP | \
> @@ -250,6 +251,16 @@ struct mlx5_flow_dv_modify_hdr_resource {
> /**< Modification actions. */
> };
>
> +/* Jump action resource structure. */
> +struct mlx5_flow_dv_jump_tbl_resource {
> + LIST_ENTRY(mlx5_flow_dv_jump_tbl_resource) next;
> + /* Pointer to next element. */
> + rte_atomic32_t refcnt; /**< Reference counter. */
> + void *action; /**< Pointer to the rdma core action. */
> + uint8_t ft_type; /**< Flow table type, Rx or Tx. */
> + struct mlx5_flow_tbl_resource *tbl; /**< The target table. */ };
> +
> /*
> * Max number of actions per DV flow.
> * See CREATE_FLOW_MAX_FLOW_ACTIONS_SUPPORTED
> @@ -270,6 +281,8 @@ struct mlx5_flow_dv {
> struct mlx5_flow_dv_modify_hdr_resource *modify_hdr;
> /**< Pointer to modify header resource in cache. */
> struct ibv_flow *flow; /**< Installed flow. */
> + struct mlx5_flow_dv_jump_tbl_resource *jump;
> + /**< Pointer to the jump action resource. */
> #ifdef HAVE_IBV_FLOW_DV_SUPPORT
> void *actions[MLX5_DV_MAX_NUMBER_OF_ACTIONS];
> /**< Action list. */
> diff --git a/drivers/net/mlx5/mlx5_flow_dv.c
> b/drivers/net/mlx5/mlx5_flow_dv.c index 6e4f6c4..6997dc6 100644
> --- a/drivers/net/mlx5/mlx5_flow_dv.c
> +++ b/drivers/net/mlx5/mlx5_flow_dv.c
> @@ -861,6 +861,68 @@ struct field_modify_info modify_tcp[] = { }
>
> /**
> + * Find existing table jump resource or create and register a new one.
> + *
> + * @param dev[in, out]
> + * Pointer to rte_eth_dev structure.
> + * @param[in, out] resource
> + * Pointer to jump table resource.
> + * @parm[in, out] dev_flow
> + * Pointer to the dev_flow.
> + * @param[out] error
> + * pointer to error structure.
> + *
> + * @return
> + * 0 on success otherwise -errno and errno is set.
> + */
> +static int
> +flow_dv_jump_tbl_resource_register
> + (struct rte_eth_dev *dev,
> + struct mlx5_flow_dv_jump_tbl_resource *resource,
> + struct mlx5_flow *dev_flow,
> + struct rte_flow_error *error)
> +{
> + struct mlx5_priv *priv = dev->data->dev_private;
> + struct mlx5_flow_dv_jump_tbl_resource *cache_resource;
> +
> + /* Lookup a matching resource from cache. */
> + LIST_FOREACH(cache_resource, &priv->jump_tbl, next) {
> + if (resource->tbl == cache_resource->tbl) {
> + DRV_LOG(DEBUG, "jump table resource resource %p:
> refcnt %d++",
> + (void *)cache_resource,
> + rte_atomic32_read(&cache_resource-
> >refcnt));
> + rte_atomic32_inc(&cache_resource->refcnt);
> + dev_flow->dv.jump = cache_resource;
> + return 0;
> + }
> + }
> + /* Register new jump table resource. */
> + cache_resource = rte_calloc(__func__, 1, sizeof(*cache_resource), 0);
> + if (!cache_resource)
> + return rte_flow_error_set(error, ENOMEM,
> +
> RTE_FLOW_ERROR_TYPE_UNSPECIFIED, NULL,
> + "cannot allocate resource
> memory");
> + *cache_resource = *resource;
> + cache_resource->action =
> + mlx5_glue->dr_create_flow_action_dest_flow_tbl
> + (resource->tbl->obj);
> + if (!cache_resource->action) {
> + rte_free(cache_resource);
> + return rte_flow_error_set(error, ENOMEM,
> +
> RTE_FLOW_ERROR_TYPE_UNSPECIFIED,
> + NULL, "cannot create action");
> + }
> + rte_atomic32_init(&cache_resource->refcnt);
> + rte_atomic32_inc(&cache_resource->refcnt);
> + LIST_INSERT_HEAD(&priv->jump_tbl, cache_resource, next);
> + dev_flow->dv.jump = cache_resource;
> + DRV_LOG(DEBUG, "new jump table resource %p: refcnt %d++",
> + (void *)cache_resource,
> + rte_atomic32_read(&cache_resource->refcnt));
> + return 0;
> +}
> +
> +/**
> * Get the size of specific rte_flow_item_type
> *
> * @param[in] item_type
> @@ -1423,6 +1485,37 @@ struct field_modify_info modify_tcp[] = { }
>
> /**
> + * Validate jump action.
> + *
> + * @param[in] action
> + * Pointer to the modify action.
> + * @param[in] group
> + * The group of the current flow.
> + * @param[out] error
> + * Pointer to error structure.
> + *
> + * @return
> + * 0 on success, a negative errno value otherwise and rte_errno is set.
> + */
> +static int
> +flow_dv_validate_action_jump(const struct rte_flow_action *action,
> + uint32_t group,
> + struct rte_flow_error *error)
> +{
> + if (action->type != RTE_FLOW_ACTION_TYPE_JUMP && !action->conf)
> + return rte_flow_error_set(error, EINVAL,
> +
> RTE_FLOW_ERROR_TYPE_ACTION_CONF,
> + NULL, "action configuration not
> set");
> + if (group >= ((const struct rte_flow_action_jump *)action->conf)-
> >group)
> + return rte_flow_error_set(error, EINVAL,
> + RTE_FLOW_ERROR_TYPE_ACTION,
> NULL,
> + "target group must be higher then"
> + " the current flow group");
> + return 0;
> +}
> +
> +
> +/**
> * Find existing modify-header resource or create and register a new one.
> *
> * @param dev[in, out]
> @@ -1605,7 +1698,7 @@ struct field_modify_info modify_tcp[] = {
> struct mlx5_priv *priv = dev->data->dev_private;
> uint32_t priority_max = priv->config.flow_prio - 1;
>
> -#ifdef HAVE_MLX5DV_DR
> +#ifndef HAVE_MLX5DV_DR
> if (attributes->group)
> return rte_flow_error_set(error, ENOTSUP,
>
> RTE_FLOW_ERROR_TYPE_ATTR_GROUP,
> @@ -1977,6 +2070,14 @@ struct field_modify_info modify_tcp[] = {
>
> MLX5_FLOW_ACTION_SET_TTL :
>
> MLX5_FLOW_ACTION_DEC_TTL;
> break;
> + case RTE_FLOW_ACTION_TYPE_JUMP:
> + ret = flow_dv_validate_action_jump(actions,
> + attr->group, error);
> + if (ret)
> + return ret;
> + ++actions_n;
> + action_flags |= MLX5_FLOW_ACTION_JUMP;
> + break;
> default:
> return rte_flow_error_set(error, ENOTSUP,
>
> RTE_FLOW_ERROR_TYPE_ACTION,
> @@ -2756,6 +2857,73 @@ struct field_modify_info modify_tcp[] = {
> return match_criteria_enable;
> }
>
> +
> +/**
> + * Get a flow table.
> + *
> + * @param dev[in, out]
> + * Pointer to rte_eth_dev structure.
> + * @param[in] table_id
> + * Table id to use.
> + * @param[in] egress
> + * Direction of the table.
> + * @param[out] error
> + * pointer to error structure.
> + *
> + * @return
> + * Returns tables resource based on the index, NULL in case of failed.
> + */
> +static struct mlx5_flow_tbl_resource *
> +flow_dv_tbl_resource_get(struct rte_eth_dev *dev,
> + uint32_t table_id, uint8_t egress,
> + struct rte_flow_error *error)
> +{
> + struct mlx5_priv *priv = dev->data->dev_private;
> + struct mlx5_flow_tbl_resource *tbl;
> +
> + if (egress) {
> + tbl = &priv->tx_tbl[table_id];
> + if (!tbl->obj)
> + tbl->obj = mlx5_glue->dr_create_flow_tbl
> + (priv->tx_ns, table_id);
> + } else {
> + tbl = &priv->rx_tbl[table_id];
> + if (!tbl->obj)
> + tbl->obj = mlx5_glue->dr_create_flow_tbl
> + (priv->rx_ns, table_id);
> + }
> + if (!tbl->obj) {
> + rte_flow_error_set(error, ENOMEM,
> + RTE_FLOW_ERROR_TYPE_UNSPECIFIED,
> + NULL, "cannot create table");
> + return NULL;
> + }
> + rte_atomic32_inc(&tbl->refcnt);
> + return tbl;
> +}
> +
> +/**
> + * Release a flow table.
> + *
> + * @param[in] tbl
> + * Table resource to be released.
> + *
> + * @return
> + * Returns 0 if table was released, else return 1;
> + */
> +static int
> +flow_dv_tbl_resource_release(struct mlx5_flow_tbl_resource *tbl) {
> + if (!tbl)
> + return 0;
> + if (rte_atomic32_dec_and_test(&tbl->refcnt)) {
> + mlx5_glue->dr_destroy_flow_tbl(tbl->obj);
> + tbl->obj = NULL;
> + return 0;
> + }
> + return 1;
> +}
> +
> /**
> * Register the flow matcher.
> *
> @@ -2784,6 +2952,9 @@ struct field_modify_info modify_tcp[] = {
> .match_mask = (void *)&matcher->mask,
> };
> struct mlx5_flow_tbl_resource *tbl = NULL;
> +#ifndef HAVE_MLX5DV_DR
> + struct mlx5_flow_tbl_resource tbl_tmp; #endif
>
> /* Lookup from cache. */
> LIST_FOREACH(cache_matcher, &priv->matchers, next) { @@ -
> 2805,33 +2976,24 @@ struct field_modify_info modify_tcp[] = {
> return 0;
> }
> }
> -#ifdef HAVE_MLX5DV_DR
> - if (matcher->egress) {
> - tbl = &priv->tx_tbl[matcher->group];
> - if (!tbl->obj)
> - tbl->obj = mlx5_glue->dr_create_flow_tbl
> - (priv->tx_ns,
> - matcher->group * MLX5_GROUP_FACTOR);
> - } else {
> - tbl = &priv->rx_tbl[matcher->group];
> - if (!tbl->obj)
> - tbl->obj = mlx5_glue->dr_create_flow_tbl
> - (priv->rx_ns,
> - matcher->group * MLX5_GROUP_FACTOR);
> - }
> - if (!tbl->obj)
> - return rte_flow_error_set(error, ENOMEM,
> -
> RTE_FLOW_ERROR_TYPE_UNSPECIFIED,
> - NULL, "cannot create table");
> -
> - rte_atomic32_inc(&tbl->refcnt);
> -#endif
> /* Register new matcher. */
> cache_matcher = rte_calloc(__func__, 1, sizeof(*cache_matcher), 0);
> if (!cache_matcher)
> return rte_flow_error_set(error, ENOMEM,
>
> RTE_FLOW_ERROR_TYPE_UNSPECIFIED, NULL,
> "cannot allocate matcher memory");
> +#ifdef HAVE_MLX5DV_DR
> + tbl = flow_dv_tbl_resource_get(dev, matcher->group *
> MLX5_GROUP_FACTOR,
> + matcher->egress, error);
> + if (!tbl) {
> + rte_free(cache_matcher);
> + return rte_flow_error_set(error, ENOMEM,
> +
> RTE_FLOW_ERROR_TYPE_UNSPECIFIED,
> + NULL, "cannot create table");
> + }
> +#else
> + tbl = &tbl_tmp;
> +#endif
> *cache_matcher = *matcher;
> dv_attr.match_criteria_enable =
> flow_dv_matcher_enable(cache_matcher->mask.buf);
> @@ -2844,10 +3006,7 @@ struct field_modify_info modify_tcp[] = {
> if (!cache_matcher->matcher_object) {
> rte_free(cache_matcher);
> #ifdef HAVE_MLX5DV_DR
> - if (rte_atomic32_dec_and_test(&tbl->refcnt)) {
> - mlx5_glue->dr_destroy_flow_tbl(tbl->obj);
> - tbl->obj = NULL;
> - }
> + flow_dv_tbl_resource_release(tbl);
> #endif
> return rte_flow_error_set(error, ENOMEM,
>
> RTE_FLOW_ERROR_TYPE_UNSPECIFIED, @@ -3033,6 +3192,9 @@ struct
> field_modify_info modify_tcp[] = {
> const struct rte_flow_action *action = actions;
> const struct rte_flow_action_count *count = action->conf;
> const uint8_t *rss_key;
> + const struct rte_flow_action_jump *jump_data;
> + struct mlx5_flow_dv_jump_tbl_resource jump_tbl_resource;
> + struct mlx5_flow_tbl_resource *tbl;
>
> switch (actions->type) {
> case RTE_FLOW_ACTION_TYPE_VOID:
> @@ -3171,6 +3333,31 @@ struct field_modify_info modify_tcp[] = {
> /* If decap is followed by encap, handle it at encap.
> */
> action_flags |= MLX5_FLOW_ACTION_RAW_DECAP;
> break;
> + case RTE_FLOW_ACTION_TYPE_JUMP:
> + jump_data = action->conf;
> + tbl = flow_dv_tbl_resource_get(dev, jump_data-
> >group *
> + MLX5_GROUP_FACTOR,
> + attr->egress, error);
> + if (!tbl)
> + return rte_flow_error_set
> + (error, errno,
> +
> RTE_FLOW_ERROR_TYPE_ACTION,
> + NULL,
> + "cannot create jump
> action.");
> + jump_tbl_resource.tbl = tbl;
> + if (flow_dv_jump_tbl_resource_register
> + (dev, &jump_tbl_resource, dev_flow, error)) {
> + flow_dv_tbl_resource_release(tbl);
> + return rte_flow_error_set
> + (error, errno,
> +
> RTE_FLOW_ERROR_TYPE_ACTION,
> + NULL,
> + "cannot create jump
> action.");
> + }
> + dev_flow->dv.actions[actions_n++] =
> + dev_flow->dv.jump->action;
> + action_flags |= MLX5_FLOW_ACTION_JUMP;
> + break;
> case RTE_FLOW_ACTION_TYPE_SET_MAC_SRC:
> case RTE_FLOW_ACTION_TYPE_SET_MAC_DST:
> if (flow_dv_convert_action_modify_mac(&res,
> actions, @@ -3503,10 +3690,7 @@ struct field_modify_info modify_tcp[] = {
> tbl = &priv->tx_tbl[matcher->group];
> else
> tbl = &priv->rx_tbl[matcher->group];
> - if (rte_atomic32_dec_and_test(&tbl->refcnt)) {
> - mlx5_glue->dr_destroy_flow_tbl(tbl->obj);
> - tbl->obj = NULL;
> - }
> + flow_dv_tbl_resource_release(tbl);
> rte_free(matcher);
> DRV_LOG(DEBUG, "port %u matcher %p: removed",
> dev->data->port_id, (void *)matcher); @@ -3547,6
> +3731,38 @@ struct field_modify_info modify_tcp[] = { }
>
> /**
> + * Release an jump to table action resource.
> + *
> + * @param flow
> + * Pointer to mlx5_flow.
> + *
> + * @return
> + * 1 while a reference on it exists, 0 when freed.
> + */
> +static int
> +flow_dv_jump_tbl_resource_release(struct mlx5_flow *flow) {
> + struct mlx5_flow_dv_jump_tbl_resource *cache_resource =
> + flow->dv.jump;
> +
> + assert(cache_resource->action);
> + DRV_LOG(DEBUG, "jump table resource %p: refcnt %d--",
> + (void *)cache_resource,
> + rte_atomic32_read(&cache_resource->refcnt));
> + if (rte_atomic32_dec_and_test(&cache_resource->refcnt)) {
> + claim_zero(mlx5_glue->destroy_flow_action
> + (cache_resource->action));
> + LIST_REMOVE(cache_resource, next);
> + flow_dv_tbl_resource_release(cache_resource->tbl);
> + rte_free(cache_resource);
> + DRV_LOG(DEBUG, "jump table resource %p: removed",
> + (void *)cache_resource);
> + return 0;
> + }
> + return 1;
> +}
> +
> +/**
> * Release a modify-header resource.
> *
> * @param flow
> @@ -3642,6 +3858,8 @@ struct field_modify_info modify_tcp[] = {
> flow_dv_encap_decap_resource_release(dev_flow);
> if (dev_flow->dv.modify_hdr)
> flow_dv_modify_hdr_resource_release(dev_flow);
> + if (dev_flow->dv.jump)
> + flow_dv_jump_tbl_resource_release(dev_flow);
> rte_free(dev_flow);
> }
> }
> diff --git a/drivers/net/mlx5/mlx5_glue.c b/drivers/net/mlx5/mlx5_glue.c
> index f509f85..fca34ca 100644
> --- a/drivers/net/mlx5/mlx5_glue.c
> +++ b/drivers/net/mlx5/mlx5_glue.c
> @@ -370,6 +370,17 @@
> }
>
> static void *
> +mlx5_glue_dr_create_flow_action_dest_flow_tbl(void *tbl) { #ifdef
> +HAVE_MLX5DV_DR
> + return mlx5dv_dr_create_action_dest_flow_table(tbl);
> +#else
> + (void)tbl;
> + return NULL;
> +#endif
> +}
> +
> +static void *
> mlx5_glue_dr_create_flow_tbl(void *ns, uint32_t level) { #ifdef
> HAVE_MLX5DV_DR @@ -833,6 +844,8 @@
> .get_async_event = mlx5_glue_get_async_event,
> .port_state_str = mlx5_glue_port_state_str,
> .cq_ex_to_cq = mlx5_glue_cq_ex_to_cq,
> + .dr_create_flow_action_dest_flow_tbl =
> + mlx5_glue_dr_create_flow_action_dest_flow_tbl,
> .dr_create_flow_tbl = mlx5_glue_dr_create_flow_tbl,
> .dr_destroy_flow_tbl = mlx5_glue_dr_destroy_flow_tbl,
> .dr_create_ns = mlx5_glue_dr_create_ns, diff --git
> a/drivers/net/mlx5/mlx5_glue.h b/drivers/net/mlx5/mlx5_glue.h index
> 7115575..16d5dd6 100644
> --- a/drivers/net/mlx5/mlx5_glue.h
> +++ b/drivers/net/mlx5/mlx5_glue.h
> @@ -145,6 +145,7 @@ struct mlx5_glue {
> struct ibv_async_event *event);
> const char *(*port_state_str)(enum ibv_port_state port_state);
> struct ibv_cq *(*cq_ex_to_cq)(struct ibv_cq_ex *cq);
> + void *(*dr_create_flow_action_dest_flow_tbl)(void *tbl);
> void *(*dr_create_flow_tbl)(void *ns, uint32_t level);
> int (*dr_destroy_flow_tbl)(void *tbl);
> void *(*dr_create_ns)(struct ibv_context *ctx,
> --
> 1.8.3.1
^ permalink raw reply [flat|nested] 52+ messages in thread
* Re: [dpdk-dev] [PATCH v2 3/3] net/mlx5: add jump action support for NIC
2019-04-01 14:38 ` Slava Ovsiienko
@ 2019-04-01 14:38 ` Slava Ovsiienko
0 siblings, 0 replies; 52+ messages in thread
From: Slava Ovsiienko @ 2019-04-01 14:38 UTC (permalink / raw)
To: Ori Kam, Matan Azrad, Yongseok Koh, Shahaf Shuler; +Cc: dev
> -----Original Message-----
> From: Ori Kam
> Sent: Thursday, March 28, 2019 18:33
> To: Matan Azrad <matan@mellanox.com>; Yongseok Koh
> <yskoh@mellanox.com>; Shahaf Shuler <shahafs@mellanox.com>
> Cc: dev@dpdk.org; Ori Kam <orika@mellanox.com>; Slava Ovsiienko
> <viacheslavo@mellanox.com>
> Subject: [PATCH v2 3/3] net/mlx5: add jump action support for NIC
>
> When using Direct Rules we can add actions to jump between tables.
> This is extra useful since rule insertion rate is much higher on other tables
> compared to table zero.
>
> if no group is selected the rule is added to group 0.
>
> Signed-off-by: Ori Kam <orika@mellanox.com>
Acked-by: Viacheslav Ovsiienko <viacheslavo@mellanox.com>
> ---
> drivers/net/mlx5/mlx5.h | 6 +
> drivers/net/mlx5/mlx5_flow.h | 15 ++-
> drivers/net/mlx5/mlx5_flow_dv.c | 278
> +++++++++++++++++++++++++++++++++++-----
> drivers/net/mlx5/mlx5_glue.c | 13 ++
> drivers/net/mlx5/mlx5_glue.h | 1 +
> 5 files changed, 282 insertions(+), 31 deletions(-)
>
> diff --git a/drivers/net/mlx5/mlx5.h b/drivers/net/mlx5/mlx5.h index
> 73f6f0d..a3d5f8e 100644
> --- a/drivers/net/mlx5/mlx5.h
> +++ b/drivers/net/mlx5/mlx5.h
> @@ -279,6 +279,12 @@ struct mlx5_priv {
> LIST_HEAD(encap_decap, mlx5_flow_dv_encap_decap_resource)
> encaps_decaps;
> LIST_HEAD(modify_cmd, mlx5_flow_dv_modify_hdr_resource)
> modify_cmds;
> LIST_HEAD(tag, mlx5_flow_dv_tag_resource) tags;
> + LIST_HEAD(jump, mlx5_flow_dv_jump_tbl_resource) jump_tbl;
> + /* Pointer to next element. */
> + rte_atomic32_t refcnt; /**< Reference counter. */
> + struct ibv_flow_action *verbs_action;
> + /**< Verbs modify header action object. */
> + uint8_t ft_type; /**< Flow table type, Rx or Tx. */
> /* Tags resources cache. */
> uint32_t link_speed_capa; /* Link speed capabilities. */
> struct mlx5_xstats_ctrl xstats_ctrl; /* Extended stats control. */ diff --
> git a/drivers/net/mlx5/mlx5_flow.h b/drivers/net/mlx5/mlx5_flow.h index
> 8ba37a0..622e305 100644
> --- a/drivers/net/mlx5/mlx5_flow.h
> +++ b/drivers/net/mlx5/mlx5_flow.h
> @@ -115,7 +115,8 @@
> #define MLX5_FLOW_ACTION_RAW_DECAP (1u << 27)
>
> #define MLX5_FLOW_FATE_ACTIONS \
> - (MLX5_FLOW_ACTION_DROP | MLX5_FLOW_ACTION_QUEUE |
> MLX5_FLOW_ACTION_RSS)
> + (MLX5_FLOW_ACTION_DROP | MLX5_FLOW_ACTION_QUEUE | \
> + MLX5_FLOW_ACTION_RSS | MLX5_FLOW_ACTION_JUMP)
>
> #define MLX5_FLOW_ENCAP_ACTIONS
> (MLX5_FLOW_ACTION_VXLAN_ENCAP | \
> MLX5_FLOW_ACTION_NVGRE_ENCAP | \
> @@ -250,6 +251,16 @@ struct mlx5_flow_dv_modify_hdr_resource {
> /**< Modification actions. */
> };
>
> +/* Jump action resource structure. */
> +struct mlx5_flow_dv_jump_tbl_resource {
> + LIST_ENTRY(mlx5_flow_dv_jump_tbl_resource) next;
> + /* Pointer to next element. */
> + rte_atomic32_t refcnt; /**< Reference counter. */
> + void *action; /**< Pointer to the rdma core action. */
> + uint8_t ft_type; /**< Flow table type, Rx or Tx. */
> + struct mlx5_flow_tbl_resource *tbl; /**< The target table. */ };
> +
> /*
> * Max number of actions per DV flow.
> * See CREATE_FLOW_MAX_FLOW_ACTIONS_SUPPORTED
> @@ -270,6 +281,8 @@ struct mlx5_flow_dv {
> struct mlx5_flow_dv_modify_hdr_resource *modify_hdr;
> /**< Pointer to modify header resource in cache. */
> struct ibv_flow *flow; /**< Installed flow. */
> + struct mlx5_flow_dv_jump_tbl_resource *jump;
> + /**< Pointer to the jump action resource. */
> #ifdef HAVE_IBV_FLOW_DV_SUPPORT
> void *actions[MLX5_DV_MAX_NUMBER_OF_ACTIONS];
> /**< Action list. */
> diff --git a/drivers/net/mlx5/mlx5_flow_dv.c
> b/drivers/net/mlx5/mlx5_flow_dv.c index 6e4f6c4..6997dc6 100644
> --- a/drivers/net/mlx5/mlx5_flow_dv.c
> +++ b/drivers/net/mlx5/mlx5_flow_dv.c
> @@ -861,6 +861,68 @@ struct field_modify_info modify_tcp[] = { }
>
> /**
> + * Find existing table jump resource or create and register a new one.
> + *
> + * @param dev[in, out]
> + * Pointer to rte_eth_dev structure.
> + * @param[in, out] resource
> + * Pointer to jump table resource.
> + * @parm[in, out] dev_flow
> + * Pointer to the dev_flow.
> + * @param[out] error
> + * pointer to error structure.
> + *
> + * @return
> + * 0 on success otherwise -errno and errno is set.
> + */
> +static int
> +flow_dv_jump_tbl_resource_register
> + (struct rte_eth_dev *dev,
> + struct mlx5_flow_dv_jump_tbl_resource *resource,
> + struct mlx5_flow *dev_flow,
> + struct rte_flow_error *error)
> +{
> + struct mlx5_priv *priv = dev->data->dev_private;
> + struct mlx5_flow_dv_jump_tbl_resource *cache_resource;
> +
> + /* Lookup a matching resource from cache. */
> + LIST_FOREACH(cache_resource, &priv->jump_tbl, next) {
> + if (resource->tbl == cache_resource->tbl) {
> + DRV_LOG(DEBUG, "jump table resource resource %p:
> refcnt %d++",
> + (void *)cache_resource,
> + rte_atomic32_read(&cache_resource-
> >refcnt));
> + rte_atomic32_inc(&cache_resource->refcnt);
> + dev_flow->dv.jump = cache_resource;
> + return 0;
> + }
> + }
> + /* Register new jump table resource. */
> + cache_resource = rte_calloc(__func__, 1, sizeof(*cache_resource), 0);
> + if (!cache_resource)
> + return rte_flow_error_set(error, ENOMEM,
> +
> RTE_FLOW_ERROR_TYPE_UNSPECIFIED, NULL,
> + "cannot allocate resource
> memory");
> + *cache_resource = *resource;
> + cache_resource->action =
> + mlx5_glue->dr_create_flow_action_dest_flow_tbl
> + (resource->tbl->obj);
> + if (!cache_resource->action) {
> + rte_free(cache_resource);
> + return rte_flow_error_set(error, ENOMEM,
> +
> RTE_FLOW_ERROR_TYPE_UNSPECIFIED,
> + NULL, "cannot create action");
> + }
> + rte_atomic32_init(&cache_resource->refcnt);
> + rte_atomic32_inc(&cache_resource->refcnt);
> + LIST_INSERT_HEAD(&priv->jump_tbl, cache_resource, next);
> + dev_flow->dv.jump = cache_resource;
> + DRV_LOG(DEBUG, "new jump table resource %p: refcnt %d++",
> + (void *)cache_resource,
> + rte_atomic32_read(&cache_resource->refcnt));
> + return 0;
> +}
> +
> +/**
> * Get the size of specific rte_flow_item_type
> *
> * @param[in] item_type
> @@ -1423,6 +1485,37 @@ struct field_modify_info modify_tcp[] = { }
>
> /**
> + * Validate jump action.
> + *
> + * @param[in] action
> + * Pointer to the modify action.
> + * @param[in] group
> + * The group of the current flow.
> + * @param[out] error
> + * Pointer to error structure.
> + *
> + * @return
> + * 0 on success, a negative errno value otherwise and rte_errno is set.
> + */
> +static int
> +flow_dv_validate_action_jump(const struct rte_flow_action *action,
> + uint32_t group,
> + struct rte_flow_error *error)
> +{
> + if (action->type != RTE_FLOW_ACTION_TYPE_JUMP && !action->conf)
> + return rte_flow_error_set(error, EINVAL,
> +
> RTE_FLOW_ERROR_TYPE_ACTION_CONF,
> + NULL, "action configuration not
> set");
> + if (group >= ((const struct rte_flow_action_jump *)action->conf)-
> >group)
> + return rte_flow_error_set(error, EINVAL,
> + RTE_FLOW_ERROR_TYPE_ACTION,
> NULL,
> + "target group must be higher then"
> + " the current flow group");
> + return 0;
> +}
> +
> +
> +/**
> * Find existing modify-header resource or create and register a new one.
> *
> * @param dev[in, out]
> @@ -1605,7 +1698,7 @@ struct field_modify_info modify_tcp[] = {
> struct mlx5_priv *priv = dev->data->dev_private;
> uint32_t priority_max = priv->config.flow_prio - 1;
>
> -#ifdef HAVE_MLX5DV_DR
> +#ifndef HAVE_MLX5DV_DR
> if (attributes->group)
> return rte_flow_error_set(error, ENOTSUP,
>
> RTE_FLOW_ERROR_TYPE_ATTR_GROUP,
> @@ -1977,6 +2070,14 @@ struct field_modify_info modify_tcp[] = {
>
> MLX5_FLOW_ACTION_SET_TTL :
>
> MLX5_FLOW_ACTION_DEC_TTL;
> break;
> + case RTE_FLOW_ACTION_TYPE_JUMP:
> + ret = flow_dv_validate_action_jump(actions,
> + attr->group, error);
> + if (ret)
> + return ret;
> + ++actions_n;
> + action_flags |= MLX5_FLOW_ACTION_JUMP;
> + break;
> default:
> return rte_flow_error_set(error, ENOTSUP,
>
> RTE_FLOW_ERROR_TYPE_ACTION,
> @@ -2756,6 +2857,73 @@ struct field_modify_info modify_tcp[] = {
> return match_criteria_enable;
> }
>
> +
> +/**
> + * Get a flow table.
> + *
> + * @param dev[in, out]
> + * Pointer to rte_eth_dev structure.
> + * @param[in] table_id
> + * Table id to use.
> + * @param[in] egress
> + * Direction of the table.
> + * @param[out] error
> + * pointer to error structure.
> + *
> + * @return
> + * Returns tables resource based on the index, NULL in case of failed.
> + */
> +static struct mlx5_flow_tbl_resource *
> +flow_dv_tbl_resource_get(struct rte_eth_dev *dev,
> + uint32_t table_id, uint8_t egress,
> + struct rte_flow_error *error)
> +{
> + struct mlx5_priv *priv = dev->data->dev_private;
> + struct mlx5_flow_tbl_resource *tbl;
> +
> + if (egress) {
> + tbl = &priv->tx_tbl[table_id];
> + if (!tbl->obj)
> + tbl->obj = mlx5_glue->dr_create_flow_tbl
> + (priv->tx_ns, table_id);
> + } else {
> + tbl = &priv->rx_tbl[table_id];
> + if (!tbl->obj)
> + tbl->obj = mlx5_glue->dr_create_flow_tbl
> + (priv->rx_ns, table_id);
> + }
> + if (!tbl->obj) {
> + rte_flow_error_set(error, ENOMEM,
> + RTE_FLOW_ERROR_TYPE_UNSPECIFIED,
> + NULL, "cannot create table");
> + return NULL;
> + }
> + rte_atomic32_inc(&tbl->refcnt);
> + return tbl;
> +}
> +
> +/**
> + * Release a flow table.
> + *
> + * @param[in] tbl
> + * Table resource to be released.
> + *
> + * @return
> + * Returns 0 if table was released, else return 1;
> + */
> +static int
> +flow_dv_tbl_resource_release(struct mlx5_flow_tbl_resource *tbl) {
> + if (!tbl)
> + return 0;
> + if (rte_atomic32_dec_and_test(&tbl->refcnt)) {
> + mlx5_glue->dr_destroy_flow_tbl(tbl->obj);
> + tbl->obj = NULL;
> + return 0;
> + }
> + return 1;
> +}
> +
> /**
> * Register the flow matcher.
> *
> @@ -2784,6 +2952,9 @@ struct field_modify_info modify_tcp[] = {
> .match_mask = (void *)&matcher->mask,
> };
> struct mlx5_flow_tbl_resource *tbl = NULL;
> +#ifndef HAVE_MLX5DV_DR
> + struct mlx5_flow_tbl_resource tbl_tmp; #endif
>
> /* Lookup from cache. */
> LIST_FOREACH(cache_matcher, &priv->matchers, next) { @@ -
> 2805,33 +2976,24 @@ struct field_modify_info modify_tcp[] = {
> return 0;
> }
> }
> -#ifdef HAVE_MLX5DV_DR
> - if (matcher->egress) {
> - tbl = &priv->tx_tbl[matcher->group];
> - if (!tbl->obj)
> - tbl->obj = mlx5_glue->dr_create_flow_tbl
> - (priv->tx_ns,
> - matcher->group * MLX5_GROUP_FACTOR);
> - } else {
> - tbl = &priv->rx_tbl[matcher->group];
> - if (!tbl->obj)
> - tbl->obj = mlx5_glue->dr_create_flow_tbl
> - (priv->rx_ns,
> - matcher->group * MLX5_GROUP_FACTOR);
> - }
> - if (!tbl->obj)
> - return rte_flow_error_set(error, ENOMEM,
> -
> RTE_FLOW_ERROR_TYPE_UNSPECIFIED,
> - NULL, "cannot create table");
> -
> - rte_atomic32_inc(&tbl->refcnt);
> -#endif
> /* Register new matcher. */
> cache_matcher = rte_calloc(__func__, 1, sizeof(*cache_matcher), 0);
> if (!cache_matcher)
> return rte_flow_error_set(error, ENOMEM,
>
> RTE_FLOW_ERROR_TYPE_UNSPECIFIED, NULL,
> "cannot allocate matcher memory");
> +#ifdef HAVE_MLX5DV_DR
> + tbl = flow_dv_tbl_resource_get(dev, matcher->group *
> MLX5_GROUP_FACTOR,
> + matcher->egress, error);
> + if (!tbl) {
> + rte_free(cache_matcher);
> + return rte_flow_error_set(error, ENOMEM,
> +
> RTE_FLOW_ERROR_TYPE_UNSPECIFIED,
> + NULL, "cannot create table");
> + }
> +#else
> + tbl = &tbl_tmp;
> +#endif
> *cache_matcher = *matcher;
> dv_attr.match_criteria_enable =
> flow_dv_matcher_enable(cache_matcher->mask.buf);
> @@ -2844,10 +3006,7 @@ struct field_modify_info modify_tcp[] = {
> if (!cache_matcher->matcher_object) {
> rte_free(cache_matcher);
> #ifdef HAVE_MLX5DV_DR
> - if (rte_atomic32_dec_and_test(&tbl->refcnt)) {
> - mlx5_glue->dr_destroy_flow_tbl(tbl->obj);
> - tbl->obj = NULL;
> - }
> + flow_dv_tbl_resource_release(tbl);
> #endif
> return rte_flow_error_set(error, ENOMEM,
>
> RTE_FLOW_ERROR_TYPE_UNSPECIFIED, @@ -3033,6 +3192,9 @@ struct
> field_modify_info modify_tcp[] = {
> const struct rte_flow_action *action = actions;
> const struct rte_flow_action_count *count = action->conf;
> const uint8_t *rss_key;
> + const struct rte_flow_action_jump *jump_data;
> + struct mlx5_flow_dv_jump_tbl_resource jump_tbl_resource;
> + struct mlx5_flow_tbl_resource *tbl;
>
> switch (actions->type) {
> case RTE_FLOW_ACTION_TYPE_VOID:
> @@ -3171,6 +3333,31 @@ struct field_modify_info modify_tcp[] = {
> /* If decap is followed by encap, handle it at encap.
> */
> action_flags |= MLX5_FLOW_ACTION_RAW_DECAP;
> break;
> + case RTE_FLOW_ACTION_TYPE_JUMP:
> + jump_data = action->conf;
> + tbl = flow_dv_tbl_resource_get(dev, jump_data-
> >group *
> + MLX5_GROUP_FACTOR,
> + attr->egress, error);
> + if (!tbl)
> + return rte_flow_error_set
> + (error, errno,
> +
> RTE_FLOW_ERROR_TYPE_ACTION,
> + NULL,
> + "cannot create jump
> action.");
> + jump_tbl_resource.tbl = tbl;
> + if (flow_dv_jump_tbl_resource_register
> + (dev, &jump_tbl_resource, dev_flow, error)) {
> + flow_dv_tbl_resource_release(tbl);
> + return rte_flow_error_set
> + (error, errno,
> +
> RTE_FLOW_ERROR_TYPE_ACTION,
> + NULL,
> + "cannot create jump
> action.");
> + }
> + dev_flow->dv.actions[actions_n++] =
> + dev_flow->dv.jump->action;
> + action_flags |= MLX5_FLOW_ACTION_JUMP;
> + break;
> case RTE_FLOW_ACTION_TYPE_SET_MAC_SRC:
> case RTE_FLOW_ACTION_TYPE_SET_MAC_DST:
> if (flow_dv_convert_action_modify_mac(&res,
> actions, @@ -3503,10 +3690,7 @@ struct field_modify_info modify_tcp[] = {
> tbl = &priv->tx_tbl[matcher->group];
> else
> tbl = &priv->rx_tbl[matcher->group];
> - if (rte_atomic32_dec_and_test(&tbl->refcnt)) {
> - mlx5_glue->dr_destroy_flow_tbl(tbl->obj);
> - tbl->obj = NULL;
> - }
> + flow_dv_tbl_resource_release(tbl);
> rte_free(matcher);
> DRV_LOG(DEBUG, "port %u matcher %p: removed",
> dev->data->port_id, (void *)matcher); @@ -3547,6
> +3731,38 @@ struct field_modify_info modify_tcp[] = { }
>
> /**
> + * Release an jump to table action resource.
> + *
> + * @param flow
> + * Pointer to mlx5_flow.
> + *
> + * @return
> + * 1 while a reference on it exists, 0 when freed.
> + */
> +static int
> +flow_dv_jump_tbl_resource_release(struct mlx5_flow *flow) {
> + struct mlx5_flow_dv_jump_tbl_resource *cache_resource =
> + flow->dv.jump;
> +
> + assert(cache_resource->action);
> + DRV_LOG(DEBUG, "jump table resource %p: refcnt %d--",
> + (void *)cache_resource,
> + rte_atomic32_read(&cache_resource->refcnt));
> + if (rte_atomic32_dec_and_test(&cache_resource->refcnt)) {
> + claim_zero(mlx5_glue->destroy_flow_action
> + (cache_resource->action));
> + LIST_REMOVE(cache_resource, next);
> + flow_dv_tbl_resource_release(cache_resource->tbl);
> + rte_free(cache_resource);
> + DRV_LOG(DEBUG, "jump table resource %p: removed",
> + (void *)cache_resource);
> + return 0;
> + }
> + return 1;
> +}
> +
> +/**
> * Release a modify-header resource.
> *
> * @param flow
> @@ -3642,6 +3858,8 @@ struct field_modify_info modify_tcp[] = {
> flow_dv_encap_decap_resource_release(dev_flow);
> if (dev_flow->dv.modify_hdr)
> flow_dv_modify_hdr_resource_release(dev_flow);
> + if (dev_flow->dv.jump)
> + flow_dv_jump_tbl_resource_release(dev_flow);
> rte_free(dev_flow);
> }
> }
> diff --git a/drivers/net/mlx5/mlx5_glue.c b/drivers/net/mlx5/mlx5_glue.c
> index f509f85..fca34ca 100644
> --- a/drivers/net/mlx5/mlx5_glue.c
> +++ b/drivers/net/mlx5/mlx5_glue.c
> @@ -370,6 +370,17 @@
> }
>
> static void *
> +mlx5_glue_dr_create_flow_action_dest_flow_tbl(void *tbl) { #ifdef
> +HAVE_MLX5DV_DR
> + return mlx5dv_dr_create_action_dest_flow_table(tbl);
> +#else
> + (void)tbl;
> + return NULL;
> +#endif
> +}
> +
> +static void *
> mlx5_glue_dr_create_flow_tbl(void *ns, uint32_t level) { #ifdef
> HAVE_MLX5DV_DR @@ -833,6 +844,8 @@
> .get_async_event = mlx5_glue_get_async_event,
> .port_state_str = mlx5_glue_port_state_str,
> .cq_ex_to_cq = mlx5_glue_cq_ex_to_cq,
> + .dr_create_flow_action_dest_flow_tbl =
> + mlx5_glue_dr_create_flow_action_dest_flow_tbl,
> .dr_create_flow_tbl = mlx5_glue_dr_create_flow_tbl,
> .dr_destroy_flow_tbl = mlx5_glue_dr_destroy_flow_tbl,
> .dr_create_ns = mlx5_glue_dr_create_ns, diff --git
> a/drivers/net/mlx5/mlx5_glue.h b/drivers/net/mlx5/mlx5_glue.h index
> 7115575..16d5dd6 100644
> --- a/drivers/net/mlx5/mlx5_glue.h
> +++ b/drivers/net/mlx5/mlx5_glue.h
> @@ -145,6 +145,7 @@ struct mlx5_glue {
> struct ibv_async_event *event);
> const char *(*port_state_str)(enum ibv_port_state port_state);
> struct ibv_cq *(*cq_ex_to_cq)(struct ibv_cq_ex *cq);
> + void *(*dr_create_flow_action_dest_flow_tbl)(void *tbl);
> void *(*dr_create_flow_tbl)(void *ns, uint32_t level);
> int (*dr_destroy_flow_tbl)(void *tbl);
> void *(*dr_create_ns)(struct ibv_context *ctx,
> --
> 1.8.3.1
^ permalink raw reply [flat|nested] 52+ messages in thread
* Re: [dpdk-dev] [PATCH v2 1/3] net/mlx5: prepare Direct Verbs for Direct Rule
2019-03-28 16:32 ` [dpdk-dev] [PATCH v2 1/3] net/mlx5: prepare Direct Verbs for Direct Rule Ori Kam
2019-03-28 16:32 ` Ori Kam
2019-04-01 14:38 ` Slava Ovsiienko
@ 2019-04-03 10:15 ` Shahaf Shuler
2019-04-03 10:15 ` Shahaf Shuler
2 siblings, 1 reply; 52+ messages in thread
From: Shahaf Shuler @ 2019-04-03 10:15 UTC (permalink / raw)
To: Ori Kam, Matan Azrad, Yongseok Koh; +Cc: dev, Slava Ovsiienko
Thursday, March 28, 2019 6:33 PM, Ori Kam:
> Subject: [PATCH v2 1/3] net/mlx5: prepare Direct Verbs for Direct Rule
> --- a/drivers/net/mlx5/mlx5_glue.h
> +++ b/drivers/net/mlx5/mlx5_glue.h
> @@ -55,6 +55,10 @@
[...]
> enum mlx5dv_flow_table_type { flow_table_type = 0, }; #endif
>
> +#ifndef HAVE_IBV_FLOW_DEVX_COUNTERS
> +#define MLX5DV_FLOW_ACTION_COUNTER_DEVX 0 #endif
> +
> #ifndef HAVE_IBV_DEVX_OBJ
> struct mlx5dv_devx_obj;
> #endif
> @@ -98,7 +102,7 @@ struct mlx5_glue {
> struct ibv_flow *(*create_flow)(struct ibv_qp *qp,
> struct ibv_flow_attr *flow);
> int (*destroy_flow)(struct ibv_flow *flow_id);
> - int (*destroy_flow_action)(struct ibv_flow_action *action);
> + int (*destroy_flow_action)(void *action);
> struct ibv_qp *(*create_qp)(struct ibv_pd *pd,
> struct ibv_qp_init_attr *qp_init_attr);
> struct ibv_qp *(*create_qp_ex)
> @@ -160,19 +164,17 @@ struct mlx5_glue {
> int (*dv_destroy_flow_matcher)(struct mlx5dv_flow_matcher
> *matcher);
> struct ibv_flow *(*dv_create_flow)(struct mlx5dv_flow_matcher
> *matcher,
> struct mlx5dv_flow_match_parameters
> *match_value,
> - size_t num_actions,
> - struct mlx5dv_flow_action_attr *actions_attr);
> - struct ibv_flow_action *(*dv_create_flow_action_packet_reformat)
> - (struct ibv_context *ctx,
> - size_t data_sz,
> - void *data,
> + size_t num_actions, void *actions[]);
> + void *(*dv_create_flow_action_counter)(void *obj, uint32_t
> offset);
> + void *(*dv_create_flow_action_dest_ibv_qp)(void *qp);
> + void *(*dv_create_flow_action_modify_header)
> + (struct ibv_context *ctx, size_t actions_sz, uint64_t actions[],
> + enum mlx5dv_flow_table_type ft_type);
> + void *(*dv_create_flow_action_packet_reformat)
> + (struct ibv_context *ctx, size_t data_sz, void *data,
> enum mlx5dv_flow_action_packet_reformat_type
> reformat_type,
> enum mlx5dv_flow_table_type ft_type);
> - struct ibv_flow_action *(*dv_create_flow_action_modify_header)
> - (struct ibv_context *ctx,
> - size_t actions_sz,
> - uint64_t actions[],
> - enum mlx5dv_flow_table_type
> ft_type);
> + void *(*dv_create_flow_action_tag)(uint32_t tag);
> struct ibv_context *(*dv_open_device)(struct ibv_device *device);
> struct mlx5dv_devx_obj *(*devx_obj_create)
> (struct ibv_context *ctx,
Due to those changes you must bump up the LIB_GLUE_VERSION .
> --
> 1.8.3.1
^ permalink raw reply [flat|nested] 52+ messages in thread
* Re: [dpdk-dev] [PATCH v2 1/3] net/mlx5: prepare Direct Verbs for Direct Rule
2019-04-03 10:15 ` Shahaf Shuler
@ 2019-04-03 10:15 ` Shahaf Shuler
0 siblings, 0 replies; 52+ messages in thread
From: Shahaf Shuler @ 2019-04-03 10:15 UTC (permalink / raw)
To: Ori Kam, Matan Azrad, Yongseok Koh; +Cc: dev, Slava Ovsiienko
Thursday, March 28, 2019 6:33 PM, Ori Kam:
> Subject: [PATCH v2 1/3] net/mlx5: prepare Direct Verbs for Direct Rule
> --- a/drivers/net/mlx5/mlx5_glue.h
> +++ b/drivers/net/mlx5/mlx5_glue.h
> @@ -55,6 +55,10 @@
[...]
> enum mlx5dv_flow_table_type { flow_table_type = 0, }; #endif
>
> +#ifndef HAVE_IBV_FLOW_DEVX_COUNTERS
> +#define MLX5DV_FLOW_ACTION_COUNTER_DEVX 0 #endif
> +
> #ifndef HAVE_IBV_DEVX_OBJ
> struct mlx5dv_devx_obj;
> #endif
> @@ -98,7 +102,7 @@ struct mlx5_glue {
> struct ibv_flow *(*create_flow)(struct ibv_qp *qp,
> struct ibv_flow_attr *flow);
> int (*destroy_flow)(struct ibv_flow *flow_id);
> - int (*destroy_flow_action)(struct ibv_flow_action *action);
> + int (*destroy_flow_action)(void *action);
> struct ibv_qp *(*create_qp)(struct ibv_pd *pd,
> struct ibv_qp_init_attr *qp_init_attr);
> struct ibv_qp *(*create_qp_ex)
> @@ -160,19 +164,17 @@ struct mlx5_glue {
> int (*dv_destroy_flow_matcher)(struct mlx5dv_flow_matcher
> *matcher);
> struct ibv_flow *(*dv_create_flow)(struct mlx5dv_flow_matcher
> *matcher,
> struct mlx5dv_flow_match_parameters
> *match_value,
> - size_t num_actions,
> - struct mlx5dv_flow_action_attr *actions_attr);
> - struct ibv_flow_action *(*dv_create_flow_action_packet_reformat)
> - (struct ibv_context *ctx,
> - size_t data_sz,
> - void *data,
> + size_t num_actions, void *actions[]);
> + void *(*dv_create_flow_action_counter)(void *obj, uint32_t
> offset);
> + void *(*dv_create_flow_action_dest_ibv_qp)(void *qp);
> + void *(*dv_create_flow_action_modify_header)
> + (struct ibv_context *ctx, size_t actions_sz, uint64_t actions[],
> + enum mlx5dv_flow_table_type ft_type);
> + void *(*dv_create_flow_action_packet_reformat)
> + (struct ibv_context *ctx, size_t data_sz, void *data,
> enum mlx5dv_flow_action_packet_reformat_type
> reformat_type,
> enum mlx5dv_flow_table_type ft_type);
> - struct ibv_flow_action *(*dv_create_flow_action_modify_header)
> - (struct ibv_context *ctx,
> - size_t actions_sz,
> - uint64_t actions[],
> - enum mlx5dv_flow_table_type
> ft_type);
> + void *(*dv_create_flow_action_tag)(uint32_t tag);
> struct ibv_context *(*dv_open_device)(struct ibv_device *device);
> struct mlx5dv_devx_obj *(*devx_obj_create)
> (struct ibv_context *ctx,
Due to those changes you must bump up the LIB_GLUE_VERSION .
> --
> 1.8.3.1
^ permalink raw reply [flat|nested] 52+ messages in thread
* Re: [dpdk-dev] [PATCH v2 3/3] net/mlx5: add jump action support for NIC
2019-03-28 16:32 ` [dpdk-dev] [PATCH v2 3/3] net/mlx5: add jump action support for NIC Ori Kam
2019-03-28 16:32 ` Ori Kam
2019-04-01 14:38 ` Slava Ovsiienko
@ 2019-04-03 10:16 ` Shahaf Shuler
2019-04-03 10:16 ` Shahaf Shuler
2 siblings, 1 reply; 52+ messages in thread
From: Shahaf Shuler @ 2019-04-03 10:16 UTC (permalink / raw)
To: Ori Kam, Matan Azrad, Yongseok Koh; +Cc: dev, Slava Ovsiienko
Thursday, March 28, 2019 6:33 PM, Ori Kam:
> Subject: [PATCH v2 3/3] net/mlx5: add jump action support for NIC
>
> When using Direct Rules we can add actions to jump between tables.
> This is extra useful since rule insertion rate is much higher on other tables
> compared to table zero.
>
> if no group is selected the rule is added to group 0.
>
> Signed-off-by: Ori Kam <orika@mellanox.com>
> ---
> drivers/net/mlx5/mlx5.h | 6 +
> drivers/net/mlx5/mlx5_flow.h | 15 ++-
> drivers/net/mlx5/mlx5_flow_dv.c | 278
> +++++++++++++++++++++++++++++++++++-----
> drivers/net/mlx5/mlx5_glue.c | 13 ++
> drivers/net/mlx5/mlx5_glue.h | 1 +
> 5 files changed, 282 insertions(+), 31 deletions(-)
>
[...]
> /**
> * Register the flow matcher.
> *
> @@ -2784,6 +2952,9 @@ struct field_modify_info modify_tcp[] = {
> .match_mask = (void *)&matcher->mask,
> };
> struct mlx5_flow_tbl_resource *tbl = NULL;
> +#ifndef HAVE_MLX5DV_DR
> + struct mlx5_flow_tbl_resource tbl_tmp; #endif
>
> /* Lookup from cache. */
> LIST_FOREACH(cache_matcher, &priv->matchers, next) { @@ -
> 2805,33 +2976,24 @@ struct field_modify_info modify_tcp[] = {
> return 0;
> }
> }
> -#ifdef HAVE_MLX5DV_DR
> - if (matcher->egress) {
> - tbl = &priv->tx_tbl[matcher->group];
> - if (!tbl->obj)
> - tbl->obj = mlx5_glue->dr_create_flow_tbl
> - (priv->tx_ns,
> - matcher->group * MLX5_GROUP_FACTOR);
> - } else {
> - tbl = &priv->rx_tbl[matcher->group];
> - if (!tbl->obj)
> - tbl->obj = mlx5_glue->dr_create_flow_tbl
> - (priv->rx_ns,
> - matcher->group * MLX5_GROUP_FACTOR);
> - }
> - if (!tbl->obj)
> - return rte_flow_error_set(error, ENOMEM,
> -
> RTE_FLOW_ERROR_TYPE_UNSPECIFIED,
> - NULL, "cannot create table");
> -
> - rte_atomic32_inc(&tbl->refcnt);
> -#endif
> /* Register new matcher. */
> cache_matcher = rte_calloc(__func__, 1, sizeof(*cache_matcher),
> 0);
> if (!cache_matcher)
> return rte_flow_error_set(error, ENOMEM,
>
> RTE_FLOW_ERROR_TYPE_UNSPECIFIED, NULL,
> "cannot allocate matcher memory");
> +#ifdef HAVE_MLX5DV_DR
> + tbl = flow_dv_tbl_resource_get(dev, matcher->group *
> MLX5_GROUP_FACTOR,
> + matcher->egress, error);
> + if (!tbl) {
> + rte_free(cache_matcher);
> + return rte_flow_error_set(error, ENOMEM,
> +
> RTE_FLOW_ERROR_TYPE_UNSPECIFIED,
> + NULL, "cannot create table");
> + }
> +#else
> + tbl = &tbl_tmp;
> +#endif
I suggest to have the HAVE_MLX5DV_DR inside the flow_dv_tbl_resource_get (return NULL if no DR support).
Then we can have a cleaner code of this function.
^ permalink raw reply [flat|nested] 52+ messages in thread
* Re: [dpdk-dev] [PATCH v2 3/3] net/mlx5: add jump action support for NIC
2019-04-03 10:16 ` Shahaf Shuler
@ 2019-04-03 10:16 ` Shahaf Shuler
0 siblings, 0 replies; 52+ messages in thread
From: Shahaf Shuler @ 2019-04-03 10:16 UTC (permalink / raw)
To: Ori Kam, Matan Azrad, Yongseok Koh; +Cc: dev, Slava Ovsiienko
Thursday, March 28, 2019 6:33 PM, Ori Kam:
> Subject: [PATCH v2 3/3] net/mlx5: add jump action support for NIC
>
> When using Direct Rules we can add actions to jump between tables.
> This is extra useful since rule insertion rate is much higher on other tables
> compared to table zero.
>
> if no group is selected the rule is added to group 0.
>
> Signed-off-by: Ori Kam <orika@mellanox.com>
> ---
> drivers/net/mlx5/mlx5.h | 6 +
> drivers/net/mlx5/mlx5_flow.h | 15 ++-
> drivers/net/mlx5/mlx5_flow_dv.c | 278
> +++++++++++++++++++++++++++++++++++-----
> drivers/net/mlx5/mlx5_glue.c | 13 ++
> drivers/net/mlx5/mlx5_glue.h | 1 +
> 5 files changed, 282 insertions(+), 31 deletions(-)
>
[...]
> /**
> * Register the flow matcher.
> *
> @@ -2784,6 +2952,9 @@ struct field_modify_info modify_tcp[] = {
> .match_mask = (void *)&matcher->mask,
> };
> struct mlx5_flow_tbl_resource *tbl = NULL;
> +#ifndef HAVE_MLX5DV_DR
> + struct mlx5_flow_tbl_resource tbl_tmp; #endif
>
> /* Lookup from cache. */
> LIST_FOREACH(cache_matcher, &priv->matchers, next) { @@ -
> 2805,33 +2976,24 @@ struct field_modify_info modify_tcp[] = {
> return 0;
> }
> }
> -#ifdef HAVE_MLX5DV_DR
> - if (matcher->egress) {
> - tbl = &priv->tx_tbl[matcher->group];
> - if (!tbl->obj)
> - tbl->obj = mlx5_glue->dr_create_flow_tbl
> - (priv->tx_ns,
> - matcher->group * MLX5_GROUP_FACTOR);
> - } else {
> - tbl = &priv->rx_tbl[matcher->group];
> - if (!tbl->obj)
> - tbl->obj = mlx5_glue->dr_create_flow_tbl
> - (priv->rx_ns,
> - matcher->group * MLX5_GROUP_FACTOR);
> - }
> - if (!tbl->obj)
> - return rte_flow_error_set(error, ENOMEM,
> -
> RTE_FLOW_ERROR_TYPE_UNSPECIFIED,
> - NULL, "cannot create table");
> -
> - rte_atomic32_inc(&tbl->refcnt);
> -#endif
> /* Register new matcher. */
> cache_matcher = rte_calloc(__func__, 1, sizeof(*cache_matcher),
> 0);
> if (!cache_matcher)
> return rte_flow_error_set(error, ENOMEM,
>
> RTE_FLOW_ERROR_TYPE_UNSPECIFIED, NULL,
> "cannot allocate matcher memory");
> +#ifdef HAVE_MLX5DV_DR
> + tbl = flow_dv_tbl_resource_get(dev, matcher->group *
> MLX5_GROUP_FACTOR,
> + matcher->egress, error);
> + if (!tbl) {
> + rte_free(cache_matcher);
> + return rte_flow_error_set(error, ENOMEM,
> +
> RTE_FLOW_ERROR_TYPE_UNSPECIFIED,
> + NULL, "cannot create table");
> + }
> +#else
> + tbl = &tbl_tmp;
> +#endif
I suggest to have the HAVE_MLX5DV_DR inside the flow_dv_tbl_resource_get (return NULL if no DR support).
Then we can have a cleaner code of this function.
^ permalink raw reply [flat|nested] 52+ messages in thread
* Re: [dpdk-dev] [PATCH v2 0/3] net/mlx5: Add Direct Rule support
2019-03-28 16:32 ` [dpdk-dev] [PATCH v2 0/3] net/mlx5: Add Direct Rule support Ori Kam
` (3 preceding siblings ...)
2019-03-28 16:32 ` [dpdk-dev] [PATCH v2 3/3] net/mlx5: add jump action support for NIC Ori Kam
@ 2019-04-03 10:17 ` Shahaf Shuler
2019-04-03 10:17 ` Shahaf Shuler
2019-04-03 13:21 ` [dpdk-dev] [PATCH v3 " Ori Kam
2019-04-04 9:54 ` [dpdk-dev] [PATCH v4 0/3] net/mlx5: Add Direct Rule support Ori Kam
6 siblings, 1 reply; 52+ messages in thread
From: Shahaf Shuler @ 2019-04-03 10:17 UTC (permalink / raw)
To: Ori Kam, Matan Azrad, Yongseok Koh; +Cc: dev, Slava Ovsiienko
Hi Ori,
Thursday, March 28, 2019 6:33 PM, Ori Kam:
> Subject: [PATCH v2 0/3] net/mlx5: Add Direct Rule support
>
> This patch set adds support for Direct Rules API for MLX5 driver.
>
> As more and more applications like OVS are inserting and removing large
> number of flows, the flow insertion rate become more critical.
>
> In current MLX5 PMD rules are inserted to the device using the FW, this limits
> the max insertion rate. The solution is using a new set of API that are part of
> RDMA-Core. which enables bypassing the FW and writing the flows directly
> to the device, which results in a much faster insertion rate.
See small comments on the series.
Once addressed you can add my acked-by.
>
> v2:
> * Small issues fixed.
> * Rebased.
>
>
> Ori Kam (3):
> net/mlx5: prepare Direct Verbs for Direct Rule
> net/mlx5: add Direct Rules API
> net/mlx5: add jump action support for NIC
>
> drivers/net/mlx5/Makefile | 5 +
> drivers/net/mlx5/mlx5.c | 16 ++
> drivers/net/mlx5/mlx5.h | 23 ++
> drivers/net/mlx5/mlx5_flow.c | 1 +
> drivers/net/mlx5/mlx5_flow.h | 38 ++-
> drivers/net/mlx5/mlx5_flow_dv.c | 509
> ++++++++++++++++++++++++++++++++++------
> drivers/net/mlx5/mlx5_glue.c | 300 +++++++++++++++++++----
> drivers/net/mlx5/mlx5_glue.h | 50 ++--
> drivers/net/mlx5/mlx5_prm.h | 24 +-
> 9 files changed, 832 insertions(+), 134 deletions(-)
>
> --
> 1.8.3.1
^ permalink raw reply [flat|nested] 52+ messages in thread
* Re: [dpdk-dev] [PATCH v2 0/3] net/mlx5: Add Direct Rule support
2019-04-03 10:17 ` [dpdk-dev] [PATCH v2 0/3] net/mlx5: Add Direct Rule support Shahaf Shuler
@ 2019-04-03 10:17 ` Shahaf Shuler
0 siblings, 0 replies; 52+ messages in thread
From: Shahaf Shuler @ 2019-04-03 10:17 UTC (permalink / raw)
To: Ori Kam, Matan Azrad, Yongseok Koh; +Cc: dev, Slava Ovsiienko
Hi Ori,
Thursday, March 28, 2019 6:33 PM, Ori Kam:
> Subject: [PATCH v2 0/3] net/mlx5: Add Direct Rule support
>
> This patch set adds support for Direct Rules API for MLX5 driver.
>
> As more and more applications like OVS are inserting and removing large
> number of flows, the flow insertion rate become more critical.
>
> In current MLX5 PMD rules are inserted to the device using the FW, this limits
> the max insertion rate. The solution is using a new set of API that are part of
> RDMA-Core. which enables bypassing the FW and writing the flows directly
> to the device, which results in a much faster insertion rate.
See small comments on the series.
Once addressed you can add my acked-by.
>
> v2:
> * Small issues fixed.
> * Rebased.
>
>
> Ori Kam (3):
> net/mlx5: prepare Direct Verbs for Direct Rule
> net/mlx5: add Direct Rules API
> net/mlx5: add jump action support for NIC
>
> drivers/net/mlx5/Makefile | 5 +
> drivers/net/mlx5/mlx5.c | 16 ++
> drivers/net/mlx5/mlx5.h | 23 ++
> drivers/net/mlx5/mlx5_flow.c | 1 +
> drivers/net/mlx5/mlx5_flow.h | 38 ++-
> drivers/net/mlx5/mlx5_flow_dv.c | 509
> ++++++++++++++++++++++++++++++++++------
> drivers/net/mlx5/mlx5_glue.c | 300 +++++++++++++++++++----
> drivers/net/mlx5/mlx5_glue.h | 50 ++--
> drivers/net/mlx5/mlx5_prm.h | 24 +-
> 9 files changed, 832 insertions(+), 134 deletions(-)
>
> --
> 1.8.3.1
^ permalink raw reply [flat|nested] 52+ messages in thread
* [dpdk-dev] [PATCH v3 0/3] net/mlx5: Add Direct Rule support
2019-03-28 16:32 ` [dpdk-dev] [PATCH v2 0/3] net/mlx5: Add Direct Rule support Ori Kam
` (4 preceding siblings ...)
2019-04-03 10:17 ` [dpdk-dev] [PATCH v2 0/3] net/mlx5: Add Direct Rule support Shahaf Shuler
@ 2019-04-03 13:21 ` Ori Kam
2019-04-03 13:21 ` Ori Kam
` (3 more replies)
2019-04-04 9:54 ` [dpdk-dev] [PATCH v4 0/3] net/mlx5: Add Direct Rule support Ori Kam
6 siblings, 4 replies; 52+ messages in thread
From: Ori Kam @ 2019-04-03 13:21 UTC (permalink / raw)
To: Matan Azrad, Yongseok Koh, Shahaf Shuler; +Cc: dev, Ori Kam, Slava Ovsiienko
This patch set adds support for Direct Rules API for MLX5 driver.
As more and more applications like OVS are inserting and removing large
number of flows, the flow insertion rate become more critical.
In current MLX5 PMD rules are inserted to the device using the FW,
this limits the max insertion rate. The solution is using a new set of API
that are part of RDMA-Core. which enables bypassing the FW and writing
the flows directly to the device, which results in a much faster
insertion rate.
v3:
* Address comments from ML.
* rebase.
v2:
* Small issues fixed.
* Rebased.
Ori Kam (3):
net/mlx5: prepare Direct Verbs for Direct Rule
net/mlx5: add Direct Rules API
net/mlx5: add jump action support for NIC
drivers/net/mlx5/Makefile | 7 +-
drivers/net/mlx5/meson.build | 2 +-
drivers/net/mlx5/mlx5.c | 16 ++
drivers/net/mlx5/mlx5.h | 23 ++
drivers/net/mlx5/mlx5_flow.c | 1 +
drivers/net/mlx5/mlx5_flow.h | 38 ++-
drivers/net/mlx5/mlx5_flow_dv.c | 506 ++++++++++++++++++++++++++++++++++------
drivers/net/mlx5/mlx5_glue.c | 300 ++++++++++++++++++++----
drivers/net/mlx5/mlx5_glue.h | 50 ++--
drivers/net/mlx5/mlx5_prm.h | 24 +-
10 files changed, 835 insertions(+), 132 deletions(-)
--
1.8.3.1
^ permalink raw reply [flat|nested] 52+ messages in thread
* [dpdk-dev] [PATCH v3 0/3] net/mlx5: Add Direct Rule support
2019-04-03 13:21 ` [dpdk-dev] [PATCH v3 " Ori Kam
@ 2019-04-03 13:21 ` Ori Kam
2019-04-03 13:21 ` [dpdk-dev] [PATCH v3 1/3] net/mlx5: prepare Direct Verbs for Direct Rule Ori Kam
` (2 subsequent siblings)
3 siblings, 0 replies; 52+ messages in thread
From: Ori Kam @ 2019-04-03 13:21 UTC (permalink / raw)
To: Matan Azrad, Yongseok Koh, Shahaf Shuler; +Cc: dev, Ori Kam, Slava Ovsiienko
This patch set adds support for Direct Rules API for MLX5 driver.
As more and more applications like OVS are inserting and removing large
number of flows, the flow insertion rate become more critical.
In current MLX5 PMD rules are inserted to the device using the FW,
this limits the max insertion rate. The solution is using a new set of API
that are part of RDMA-Core. which enables bypassing the FW and writing
the flows directly to the device, which results in a much faster
insertion rate.
v3:
* Address comments from ML.
* rebase.
v2:
* Small issues fixed.
* Rebased.
Ori Kam (3):
net/mlx5: prepare Direct Verbs for Direct Rule
net/mlx5: add Direct Rules API
net/mlx5: add jump action support for NIC
drivers/net/mlx5/Makefile | 7 +-
drivers/net/mlx5/meson.build | 2 +-
drivers/net/mlx5/mlx5.c | 16 ++
drivers/net/mlx5/mlx5.h | 23 ++
drivers/net/mlx5/mlx5_flow.c | 1 +
drivers/net/mlx5/mlx5_flow.h | 38 ++-
drivers/net/mlx5/mlx5_flow_dv.c | 506 ++++++++++++++++++++++++++++++++++------
drivers/net/mlx5/mlx5_glue.c | 300 ++++++++++++++++++++----
drivers/net/mlx5/mlx5_glue.h | 50 ++--
drivers/net/mlx5/mlx5_prm.h | 24 +-
10 files changed, 835 insertions(+), 132 deletions(-)
--
1.8.3.1
^ permalink raw reply [flat|nested] 52+ messages in thread
* [dpdk-dev] [PATCH v3 1/3] net/mlx5: prepare Direct Verbs for Direct Rule
2019-04-03 13:21 ` [dpdk-dev] [PATCH v3 " Ori Kam
2019-04-03 13:21 ` Ori Kam
@ 2019-04-03 13:21 ` Ori Kam
2019-04-03 13:21 ` Ori Kam
2019-04-03 13:21 ` [dpdk-dev] [PATCH v3 2/3] net/mlx5: add Direct Rules API Ori Kam
2019-04-03 13:21 ` [dpdk-dev] [PATCH v3 3/3] net/mlx5: add jump action support for NIC Ori Kam
3 siblings, 1 reply; 52+ messages in thread
From: Ori Kam @ 2019-04-03 13:21 UTC (permalink / raw)
To: Matan Azrad, Yongseok Koh, Shahaf Shuler; +Cc: dev, Ori Kam, Slava Ovsiienko
This is the first patch of a series that is designed to enable the
Direct Rules API.
The main difference between Direct Verbs and Direct Rules from API
prespective, is that in Direct Rules each action has it's own create
function and the object itself is of type void.
In this patch I'm adding functions to generate actions that currenlty
are done without create action, and I'm changing the action type to be
void *, so in next patches only the glue functions will need to change.
Signed-off-by: Ori Kam <orika@mellanox.com>
Acked-by: Shahaf Shuler <shahafs@mellanox.com>
---
drivers/net/mlx5/Makefile | 2 +-
drivers/net/mlx5/meson.build | 2 +-
drivers/net/mlx5/mlx5.h | 2 +
drivers/net/mlx5/mlx5_flow.h | 17 +++-
drivers/net/mlx5/mlx5_flow_dv.c | 184 +++++++++++++++++++++++++++++-----------
drivers/net/mlx5/mlx5_glue.c | 141 ++++++++++++++++++++++++------
drivers/net/mlx5/mlx5_glue.h | 26 +++---
7 files changed, 280 insertions(+), 94 deletions(-)
diff --git a/drivers/net/mlx5/Makefile b/drivers/net/mlx5/Makefile
index ccd179c..0d20f0f 100644
--- a/drivers/net/mlx5/Makefile
+++ b/drivers/net/mlx5/Makefile
@@ -8,7 +8,7 @@ include $(RTE_SDK)/mk/rte.vars.mk
LIB = librte_pmd_mlx5.a
LIB_GLUE = $(LIB_GLUE_BASE).$(LIB_GLUE_VERSION)
LIB_GLUE_BASE = librte_pmd_mlx5_glue.so
-LIB_GLUE_VERSION = 19.02.0
+LIB_GLUE_VERSION = 19.05.0
# Sources.
SRCS-$(CONFIG_RTE_LIBRTE_MLX5_PMD) += mlx5.c
diff --git a/drivers/net/mlx5/meson.build b/drivers/net/mlx5/meson.build
index a429655..a4c684e 100644
--- a/drivers/net/mlx5/meson.build
+++ b/drivers/net/mlx5/meson.build
@@ -4,7 +4,7 @@
pmd_dlopen = (get_option('ibverbs_link') == 'dlopen')
LIB_GLUE_BASE = 'librte_pmd_mlx5_glue.so'
-LIB_GLUE_VERSION = '19.02.0'
+LIB_GLUE_VERSION = '19.05.0'
LIB_GLUE = LIB_GLUE_BASE + '.' + LIB_GLUE_VERSION
if pmd_dlopen
dpdk_conf.set('RTE_IBVERBS_LINK_DLOPEN', 1)
diff --git a/drivers/net/mlx5/mlx5.h b/drivers/net/mlx5/mlx5.h
index 47a7d75..784bf9b 100644
--- a/drivers/net/mlx5/mlx5.h
+++ b/drivers/net/mlx5/mlx5.h
@@ -309,6 +309,8 @@ struct mlx5_priv {
LIST_HEAD(matchers, mlx5_flow_dv_matcher) matchers;
LIST_HEAD(encap_decap, mlx5_flow_dv_encap_decap_resource) encaps_decaps;
LIST_HEAD(modify_cmd, mlx5_flow_dv_modify_hdr_resource) modify_cmds;
+ LIST_HEAD(tag, mlx5_flow_dv_tag_resource) tags;
+ /* Tags resources cache. */
uint32_t link_speed_capa; /* Link speed capabilities. */
struct mlx5_xstats_ctrl xstats_ctrl; /* Extended stats control. */
struct mlx5_stats_ctrl stats_ctrl; /* Stats control. */
diff --git a/drivers/net/mlx5/mlx5_flow.h b/drivers/net/mlx5/mlx5_flow.h
index e1e798b..4f69ae2 100644
--- a/drivers/net/mlx5/mlx5_flow.h
+++ b/drivers/net/mlx5/mlx5_flow.h
@@ -214,7 +214,7 @@ struct mlx5_flow_dv_encap_decap_resource {
LIST_ENTRY(mlx5_flow_dv_encap_decap_resource) next;
/* Pointer to next element. */
rte_atomic32_t refcnt; /**< Reference counter. */
- struct ibv_flow_action *verbs_action;
+ void *verbs_action;
/**< Verbs encap/decap action object. */
uint8_t buf[MLX5_ENCAP_MAX_LEN];
size_t size;
@@ -222,6 +222,16 @@ struct mlx5_flow_dv_encap_decap_resource {
uint8_t ft_type;
};
+/* Tag resource structure. */
+struct mlx5_flow_dv_tag_resource {
+ LIST_ENTRY(mlx5_flow_dv_tag_resource) next;
+ /* Pointer to next element. */
+ rte_atomic32_t refcnt; /**< Reference counter. */
+ void *action;
+ /**< Verbs tag action object. */
+ uint32_t tag; /**< the tag value. */
+};
+
/* Number of modification commands. */
#define MLX5_MODIFY_NUM 8
@@ -259,7 +269,7 @@ struct mlx5_flow_dv {
/**< Pointer to modify header resource in cache. */
struct ibv_flow *flow; /**< Installed flow. */
#ifdef HAVE_IBV_FLOW_DV_SUPPORT
- struct mlx5dv_flow_action_attr actions[MLX5_DV_MAX_NUMBER_OF_ACTIONS];
+ void *actions[MLX5_DV_MAX_NUMBER_OF_ACTIONS];
/**< Action list. */
#endif
int actions_n; /**< number of actions. */
@@ -332,6 +342,7 @@ struct mlx5_flow_counter {
};
uint64_t hits; /**< Number of packets matched by the rule. */
uint64_t bytes; /**< Number of bytes matched by the rule. */
+ void *action; /**< Pointer to the dv action. */
};
/* Flow structure. */
@@ -339,6 +350,8 @@ struct rte_flow {
TAILQ_ENTRY(rte_flow) next; /**< Pointer to the next flow structure. */
enum mlx5_flow_drv_type drv_type; /**< Drvier type. */
struct mlx5_flow_counter *counter; /**< Holds flow counter. */
+ struct mlx5_flow_dv_tag_resource *tag_resource;
+ /**< pointer to the tag action. */
struct rte_flow_action_rss rss;/**< RSS context. */
uint8_t key[MLX5_RSS_HASH_KEY_LEN]; /**< RSS hash key. */
uint16_t (*queue)[]; /**< Destination queues to redirect traffic to. */
diff --git a/drivers/net/mlx5/mlx5_flow_dv.c b/drivers/net/mlx5/mlx5_flow_dv.c
index ad84dea..d9e2ac9 100644
--- a/drivers/net/mlx5/mlx5_flow_dv.c
+++ b/drivers/net/mlx5/mlx5_flow_dv.c
@@ -32,7 +32,6 @@
#include "mlx5_prm.h"
#include "mlx5_glue.h"
#include "mlx5_flow.h"
-
#ifdef HAVE_IBV_FLOW_DV_SUPPORT
#ifndef HAVE_IBV_FLOW_DEVX_COUNTERS
@@ -1537,6 +1536,11 @@ struct field_modify_info modify_tcp[] = {
.id = id,
.dcs = dcs,
};
+ tmpl.action = mlx5_glue->dv_create_flow_action_counter(dcs->obj, 0);
+ if (!tmpl.action) {
+ ret = errno;
+ goto error_exit;
+ }
*cnt = tmpl;
LIST_INSERT_HEAD(&priv->flow_counters, cnt, next);
return cnt;
@@ -2828,6 +2832,97 @@ struct field_modify_info modify_tcp[] = {
}
/**
+ * Find existing tag resource or create and register a new one.
+ *
+ * @param dev[in, out]
+ * Pointer to rte_eth_dev structure.
+ * @param[in, out] resource
+ * Pointer to tag resource.
+ * @parm[in, out] dev_flow
+ * Pointer to the dev_flow.
+ * @param[out] error
+ * pointer to error structure.
+ *
+ * @return
+ * 0 on success otherwise -errno and errno is set.
+ */
+static int
+flow_dv_tag_resource_register
+ (struct rte_eth_dev *dev,
+ struct mlx5_flow_dv_tag_resource *resource,
+ struct mlx5_flow *dev_flow,
+ struct rte_flow_error *error)
+{
+ struct mlx5_priv *priv = dev->data->dev_private;
+ struct mlx5_flow_dv_tag_resource *cache_resource;
+
+ /* Lookup a matching resource from cache. */
+ LIST_FOREACH(cache_resource, &priv->tags, next) {
+ if (resource->tag == cache_resource->tag) {
+ DRV_LOG(DEBUG, "tag resource %p: refcnt %d++",
+ (void *)cache_resource,
+ rte_atomic32_read(&cache_resource->refcnt));
+ rte_atomic32_inc(&cache_resource->refcnt);
+ dev_flow->flow->tag_resource = cache_resource;
+ return 0;
+ }
+ }
+ /* Register new resource. */
+ cache_resource = rte_calloc(__func__, 1, sizeof(*cache_resource), 0);
+ if (!cache_resource)
+ return rte_flow_error_set(error, ENOMEM,
+ RTE_FLOW_ERROR_TYPE_UNSPECIFIED, NULL,
+ "cannot allocate resource memory");
+ *cache_resource = *resource;
+ cache_resource->action = mlx5_glue->dv_create_flow_action_tag
+ (resource->tag);
+ if (!cache_resource->action) {
+ rte_free(cache_resource);
+ return rte_flow_error_set(error, ENOMEM,
+ RTE_FLOW_ERROR_TYPE_UNSPECIFIED,
+ NULL, "cannot create action");
+ }
+ rte_atomic32_init(&cache_resource->refcnt);
+ rte_atomic32_inc(&cache_resource->refcnt);
+ LIST_INSERT_HEAD(&priv->tags, cache_resource, next);
+ dev_flow->flow->tag_resource = cache_resource;
+ DRV_LOG(DEBUG, "new tag resource %p: refcnt %d++",
+ (void *)cache_resource,
+ rte_atomic32_read(&cache_resource->refcnt));
+ return 0;
+}
+
+/**
+ * Release the tag.
+ *
+ * @param dev
+ * Pointer to Ethernet device.
+ * @param flow
+ * Pointer to mlx5_flow.
+ *
+ * @return
+ * 1 while a reference on it exists, 0 when freed.
+ */
+static int
+flow_dv_tag_release(struct rte_eth_dev *dev,
+ struct mlx5_flow_dv_tag_resource *tag)
+{
+ assert(tag);
+ DRV_LOG(DEBUG, "port %u tag %p: refcnt %d--",
+ dev->data->port_id, (void *)tag,
+ rte_atomic32_read(&tag->refcnt));
+ if (rte_atomic32_dec_and_test(&tag->refcnt)) {
+ claim_zero(mlx5_glue->destroy_flow_action(tag->action));
+ LIST_REMOVE(tag, next);
+ DRV_LOG(DEBUG, "port %u tag %p: removed",
+ dev->data->port_id, (void *)tag);
+ rte_free(tag);
+ return 0;
+ }
+ return 1;
+}
+
+/**
* Fill the flow with DV spec.
*
* @param[in] dev
@@ -2872,6 +2967,7 @@ struct field_modify_info modify_tcp[] = {
MLX5DV_FLOW_TABLE_TYPE_NIC_RX
};
union flow_dv_attr flow_attr = { .attr = 0 };
+ struct mlx5_flow_dv_tag_resource tag_resource;
if (priority == MLX5_FLOW_PRIO_RSVD)
priority = priv->config.flow_prio - 1;
@@ -2886,26 +2982,29 @@ struct field_modify_info modify_tcp[] = {
case RTE_FLOW_ACTION_TYPE_VOID:
break;
case RTE_FLOW_ACTION_TYPE_FLAG:
- dev_flow->dv.actions[actions_n].type =
- MLX5DV_FLOW_ACTION_TAG;
- dev_flow->dv.actions[actions_n].tag_value =
+ tag_resource.tag =
mlx5_flow_mark_set(MLX5_FLOW_MARK_DEFAULT);
- actions_n++;
+ if (!flow->tag_resource)
+ if (flow_dv_tag_resource_register
+ (dev, &tag_resource, dev_flow, error))
+ return errno;
+ dev_flow->dv.actions[actions_n++] =
+ flow->tag_resource->action;
action_flags |= MLX5_FLOW_ACTION_FLAG;
break;
case RTE_FLOW_ACTION_TYPE_MARK:
- dev_flow->dv.actions[actions_n].type =
- MLX5DV_FLOW_ACTION_TAG;
- dev_flow->dv.actions[actions_n].tag_value =
- mlx5_flow_mark_set
- (((const struct rte_flow_action_mark *)
- (actions->conf))->id);
- actions_n++;
+ tag_resource.tag = mlx5_flow_mark_set
+ (((const struct rte_flow_action_mark *)
+ (actions->conf))->id);
+ if (!flow->tag_resource)
+ if (flow_dv_tag_resource_register
+ (dev, &tag_resource, dev_flow, error))
+ return errno;
+ dev_flow->dv.actions[actions_n++] =
+ flow->tag_resource->action;
action_flags |= MLX5_FLOW_ACTION_MARK;
break;
case RTE_FLOW_ACTION_TYPE_DROP:
- dev_flow->dv.actions[actions_n].type =
- MLX5DV_FLOW_ACTION_DROP;
action_flags |= MLX5_FLOW_ACTION_DROP;
break;
case RTE_FLOW_ACTION_TYPE_QUEUE:
@@ -2933,17 +3032,13 @@ struct field_modify_info modify_tcp[] = {
rte_errno = ENOTSUP;
goto cnt_err;
}
- flow->counter =
- flow_dv_counter_new(dev,
- count->shared, count->id);
+ flow->counter = flow_dv_counter_new(dev, count->shared,
+ count->id);
if (flow->counter == NULL)
goto cnt_err;
- dev_flow->dv.actions[actions_n].type =
- MLX5DV_FLOW_ACTION_COUNTERS_DEVX;
- dev_flow->dv.actions[actions_n].obj =
- flow->counter->dcs->obj;
+ dev_flow->dv.actions[actions_n++] =
+ flow->counter->action;
action_flags |= MLX5_FLOW_ACTION_COUNT;
- ++actions_n;
break;
cnt_err:
if (rte_errno == ENOTSUP)
@@ -2964,11 +3059,8 @@ struct field_modify_info modify_tcp[] = {
if (flow_dv_create_action_l2_encap(dev, actions,
dev_flow, error))
return -rte_errno;
- dev_flow->dv.actions[actions_n].type =
- MLX5DV_FLOW_ACTION_IBV_FLOW_ACTION;
- dev_flow->dv.actions[actions_n].action =
+ dev_flow->dv.actions[actions_n++] =
dev_flow->dv.encap_decap->verbs_action;
- actions_n++;
action_flags |= actions->type ==
RTE_FLOW_ACTION_TYPE_VXLAN_ENCAP ?
MLX5_FLOW_ACTION_VXLAN_ENCAP :
@@ -2979,11 +3071,8 @@ struct field_modify_info modify_tcp[] = {
if (flow_dv_create_action_l2_decap(dev, dev_flow,
error))
return -rte_errno;
- dev_flow->dv.actions[actions_n].type =
- MLX5DV_FLOW_ACTION_IBV_FLOW_ACTION;
- dev_flow->dv.actions[actions_n].action =
+ dev_flow->dv.actions[actions_n++] =
dev_flow->dv.encap_decap->verbs_action;
- actions_n++;
action_flags |= actions->type ==
RTE_FLOW_ACTION_TYPE_VXLAN_DECAP ?
MLX5_FLOW_ACTION_VXLAN_DECAP :
@@ -2995,9 +3084,7 @@ struct field_modify_info modify_tcp[] = {
if (flow_dv_create_action_raw_encap
(dev, actions, dev_flow, attr, error))
return -rte_errno;
- dev_flow->dv.actions[actions_n].type =
- MLX5DV_FLOW_ACTION_IBV_FLOW_ACTION;
- dev_flow->dv.actions[actions_n].action =
+ dev_flow->dv.actions[actions_n++] =
dev_flow->dv.encap_decap->verbs_action;
} else {
/* Handle encap without preceding decap. */
@@ -3005,12 +3092,9 @@ struct field_modify_info modify_tcp[] = {
dev_flow,
error))
return -rte_errno;
- dev_flow->dv.actions[actions_n].type =
- MLX5DV_FLOW_ACTION_IBV_FLOW_ACTION;
- dev_flow->dv.actions[actions_n].action =
+ dev_flow->dv.actions[actions_n++] =
dev_flow->dv.encap_decap->verbs_action;
}
- actions_n++;
action_flags |= MLX5_FLOW_ACTION_RAW_ENCAP;
break;
case RTE_FLOW_ACTION_TYPE_RAW_DECAP:
@@ -3025,11 +3109,8 @@ struct field_modify_info modify_tcp[] = {
dev_flow,
error))
return -rte_errno;
- dev_flow->dv.actions[actions_n].type =
- MLX5DV_FLOW_ACTION_IBV_FLOW_ACTION;
- dev_flow->dv.actions[actions_n].action =
+ dev_flow->dv.actions[actions_n++] =
dev_flow->dv.encap_decap->verbs_action;
- actions_n++;
}
/* If decap is followed by encap, handle it at encap. */
action_flags |= MLX5_FLOW_ACTION_RAW_DECAP;
@@ -3098,11 +3179,8 @@ struct field_modify_info modify_tcp[] = {
dev_flow,
error))
return -rte_errno;
- dev_flow->dv.actions[actions_n].type =
- MLX5DV_FLOW_ACTION_IBV_FLOW_ACTION;
- dev_flow->dv.actions[actions_n].action =
+ dev_flow->dv.actions[actions_n++] =
dev_flow->dv.modify_hdr->verbs_action;
- actions_n++;
}
break;
default:
@@ -3277,9 +3355,9 @@ struct field_modify_info modify_tcp[] = {
"cannot get drop hash queue");
goto error;
}
- dv->actions[n].type = MLX5DV_FLOW_ACTION_DEST_IBV_QP;
- dv->actions[n].qp = dv->hrxq->qp;
- n++;
+ dv->actions[n++] =
+ mlx5_glue->dv_create_flow_action_dest_ibv_qp
+ (dv->hrxq->qp);
} else if (flow->actions &
(MLX5_FLOW_ACTION_QUEUE | MLX5_FLOW_ACTION_RSS)) {
struct mlx5_hrxq *hrxq;
@@ -3304,9 +3382,9 @@ struct field_modify_info modify_tcp[] = {
goto error;
}
dv->hrxq = hrxq;
- dv->actions[n].type = MLX5DV_FLOW_ACTION_DEST_IBV_QP;
- dv->actions[n].qp = hrxq->qp;
- n++;
+ dv->actions[n++] =
+ mlx5_glue->dv_create_flow_action_dest_ibv_qp
+ (dv->hrxq->qp);
}
dv->flow =
mlx5_glue->dv_create_flow(dv->matcher->matcher_object,
@@ -3484,6 +3562,10 @@ struct field_modify_info modify_tcp[] = {
flow_dv_counter_release(flow->counter);
flow->counter = NULL;
}
+ if (flow->tag_resource) {
+ flow_dv_tag_release(dev, flow->tag_resource);
+ flow->tag_resource = NULL;
+ }
while (!LIST_EMPTY(&flow->dev_flows)) {
dev_flow = LIST_FIRST(&flow->dev_flows);
LIST_REMOVE(dev_flow, next);
diff --git a/drivers/net/mlx5/mlx5_glue.c b/drivers/net/mlx5/mlx5_glue.c
index c817d86..4b5aade 100644
--- a/drivers/net/mlx5/mlx5_glue.c
+++ b/drivers/net/mlx5/mlx5_glue.c
@@ -175,10 +175,20 @@
}
static int
-mlx5_glue_destroy_flow_action(struct ibv_flow_action *action)
+mlx5_glue_destroy_flow_action(void *action)
{
#ifdef HAVE_IBV_FLOW_DV_SUPPORT
- return ibv_destroy_flow_action(action);
+ struct mlx5dv_flow_action_attr *attr = action;
+ int res = 0;
+ switch (attr->type) {
+ case MLX5DV_FLOW_ACTION_TAG:
+ break;
+ default:
+ res = ibv_destroy_flow_action(attr->action);
+ break;
+ }
+ free(action);
+ return res;
#else
(void)action;
return ENOTSUP;
@@ -430,16 +440,23 @@
mlx5_glue_dv_create_flow(struct mlx5dv_flow_matcher *matcher,
struct mlx5dv_flow_match_parameters *match_value,
size_t num_actions,
- struct mlx5dv_flow_action_attr *actions_attr)
+ void *actions[])
{
#ifdef HAVE_IBV_FLOW_DV_SUPPORT
+ struct mlx5dv_flow_action_attr actions_attr[8];
+
+ if (num_actions > 8)
+ return NULL;
+ for (size_t i = 0; i < num_actions; i++)
+ actions_attr[i] =
+ *((struct mlx5dv_flow_action_attr *)(actions[i]));
return mlx5dv_create_flow(matcher, match_value,
num_actions, actions_attr);
#else
(void)matcher;
(void)match_value;
(void)num_actions;
- (void)actions_attr;
+ (void)actions;
return NULL;
#endif
}
@@ -455,31 +472,45 @@
#endif
}
-static struct ibv_flow_action *
-mlx5_glue_dv_create_flow_action_packet_reformat
- (struct ibv_context *ctx,
- size_t data_sz,
- void *data,
- enum mlx5dv_flow_action_packet_reformat_type reformat_type,
- enum mlx5dv_flow_table_type ft_type)
+static void *
+mlx5_glue_dv_create_flow_action_counter(void *counter_obj, uint32_t offset)
{
#ifdef HAVE_IBV_FLOW_DV_SUPPORT
- return mlx5dv_create_flow_action_packet_reformat(ctx,
- data_sz,
- data,
- reformat_type,
- ft_type);
+ struct mlx5dv_flow_action_attr *action;
+
+ (void)offset;
+ action = malloc(sizeof(*action));
+ if (!action)
+ return NULL;
+ action->type = MLX5DV_FLOW_ACTION_COUNTERS_DEVX;
+ action->obj = counter_obj;
+ return action;
#else
- (void)ctx;
- (void)data_sz;
- (void)data;
- (void)reformat_type;
- (void)ft_type;
+ (void)counter_obj;
+ (void)offset;
return NULL;
#endif
}
-static struct ibv_flow_action *
+static void *
+mlx5_glue_dv_create_flow_action_dest_ibv_qp(void *qp)
+{
+#ifdef HAVE_IBV_FLOW_DV_SUPPORT
+ struct mlx5dv_flow_action_attr *action;
+
+ action = malloc(sizeof(*action));
+ if (!action)
+ return NULL;
+ action->type = MLX5DV_FLOW_ACTION_DEST_IBV_QP;
+ action->obj = qp;
+ return action;
+#else
+ (void)qp;
+ return NULL;
+#endif
+}
+
+static void *
mlx5_glue_dv_create_flow_action_modify_header
(struct ibv_context *ctx,
size_t actions_sz,
@@ -487,8 +518,15 @@
enum mlx5dv_flow_table_type ft_type)
{
#ifdef HAVE_IBV_FLOW_DV_SUPPORT
- return mlx5dv_create_flow_action_modify_header(ctx, actions_sz,
- actions, ft_type);
+ struct mlx5dv_flow_action_attr *action;
+
+ action = malloc(sizeof(*action));
+ if (!action)
+ return NULL;
+ action->type = MLX5DV_FLOW_ACTION_IBV_FLOW_ACTION;
+ action->action = mlx5dv_create_flow_action_modify_header
+ (ctx, actions_sz, actions, ft_type);
+ return action;
#else
(void)ctx;
(void)actions_sz;
@@ -498,6 +536,50 @@
#endif
}
+static void *
+mlx5_glue_dv_create_flow_action_packet_reformat
+ (struct ibv_context *ctx,
+ size_t data_sz,
+ void *data,
+ enum mlx5dv_flow_action_packet_reformat_type reformat_type,
+ enum mlx5dv_flow_table_type ft_type)
+{
+#ifdef HAVE_IBV_FLOW_DV_SUPPORT
+ struct mlx5dv_flow_action_attr *action;
+
+ action = malloc(sizeof(*action));
+ if (!action)
+ return NULL;
+ action->type = MLX5DV_FLOW_ACTION_IBV_FLOW_ACTION;
+ action->action = mlx5dv_create_flow_action_packet_reformat
+ (ctx, data_sz, data, reformat_type, ft_type);
+ return action;
+#else
+ (void)ctx;
+ (void)data_sz;
+ (void)data;
+ (void)reformat_type;
+ (void)ft_type;
+ return NULL;
+#endif
+}
+
+static void *
+mlx5_glue_dv_create_flow_action_tag(uint32_t tag)
+{
+#ifdef HAVE_IBV_FLOW_DV_SUPPORT
+ struct mlx5dv_flow_action_attr *action;
+ action = malloc(sizeof(*action));
+ if (!action)
+ return NULL;
+ action->type = MLX5DV_FLOW_ACTION_TAG;
+ action->tag_value = tag;
+ return action;
+#endif
+ (void)tag;
+ return NULL;
+}
+
static struct ibv_context *
mlx5_glue_dv_open_device(struct ibv_device *device)
{
@@ -645,10 +727,15 @@
.dv_create_flow_matcher = mlx5_glue_dv_create_flow_matcher,
.dv_destroy_flow_matcher = mlx5_glue_dv_destroy_flow_matcher,
.dv_create_flow = mlx5_glue_dv_create_flow,
- .dv_create_flow_action_packet_reformat =
- mlx5_glue_dv_create_flow_action_packet_reformat,
+ .dv_create_flow_action_counter =
+ mlx5_glue_dv_create_flow_action_counter,
+ .dv_create_flow_action_dest_ibv_qp =
+ mlx5_glue_dv_create_flow_action_dest_ibv_qp,
.dv_create_flow_action_modify_header =
- mlx5_glue_dv_create_flow_action_modify_header,
+ mlx5_glue_dv_create_flow_action_modify_header,
+ .dv_create_flow_action_packet_reformat =
+ mlx5_glue_dv_create_flow_action_packet_reformat,
+ .dv_create_flow_action_tag = mlx5_glue_dv_create_flow_action_tag,
.dv_open_device = mlx5_glue_dv_open_device,
.devx_obj_create = mlx5_glue_devx_obj_create,
.devx_obj_destroy = mlx5_glue_devx_obj_destroy,
diff --git a/drivers/net/mlx5/mlx5_glue.h b/drivers/net/mlx5/mlx5_glue.h
index b118960..32487ea 100644
--- a/drivers/net/mlx5/mlx5_glue.h
+++ b/drivers/net/mlx5/mlx5_glue.h
@@ -55,6 +55,10 @@
enum mlx5dv_flow_table_type { flow_table_type = 0, };
#endif
+#ifndef HAVE_IBV_FLOW_DEVX_COUNTERS
+#define MLX5DV_FLOW_ACTION_COUNTERS_DEVX 0
+#endif
+
#ifndef HAVE_IBV_DEVX_OBJ
struct mlx5dv_devx_obj;
#endif
@@ -98,7 +102,7 @@ struct mlx5_glue {
struct ibv_flow *(*create_flow)(struct ibv_qp *qp,
struct ibv_flow_attr *flow);
int (*destroy_flow)(struct ibv_flow *flow_id);
- int (*destroy_flow_action)(struct ibv_flow_action *action);
+ int (*destroy_flow_action)(void *action);
struct ibv_qp *(*create_qp)(struct ibv_pd *pd,
struct ibv_qp_init_attr *qp_init_attr);
struct ibv_qp *(*create_qp_ex)
@@ -160,19 +164,17 @@ struct mlx5_glue {
int (*dv_destroy_flow_matcher)(struct mlx5dv_flow_matcher *matcher);
struct ibv_flow *(*dv_create_flow)(struct mlx5dv_flow_matcher *matcher,
struct mlx5dv_flow_match_parameters *match_value,
- size_t num_actions,
- struct mlx5dv_flow_action_attr *actions_attr);
- struct ibv_flow_action *(*dv_create_flow_action_packet_reformat)
- (struct ibv_context *ctx,
- size_t data_sz,
- void *data,
+ size_t num_actions, void *actions[]);
+ void *(*dv_create_flow_action_counter)(void *obj, uint32_t offset);
+ void *(*dv_create_flow_action_dest_ibv_qp)(void *qp);
+ void *(*dv_create_flow_action_modify_header)
+ (struct ibv_context *ctx, size_t actions_sz, uint64_t actions[],
+ enum mlx5dv_flow_table_type ft_type);
+ void *(*dv_create_flow_action_packet_reformat)
+ (struct ibv_context *ctx, size_t data_sz, void *data,
enum mlx5dv_flow_action_packet_reformat_type reformat_type,
enum mlx5dv_flow_table_type ft_type);
- struct ibv_flow_action *(*dv_create_flow_action_modify_header)
- (struct ibv_context *ctx,
- size_t actions_sz,
- uint64_t actions[],
- enum mlx5dv_flow_table_type ft_type);
+ void *(*dv_create_flow_action_tag)(uint32_t tag);
struct ibv_context *(*dv_open_device)(struct ibv_device *device);
struct mlx5dv_devx_obj *(*devx_obj_create)
(struct ibv_context *ctx,
--
1.8.3.1
^ permalink raw reply [flat|nested] 52+ messages in thread
* [dpdk-dev] [PATCH v3 1/3] net/mlx5: prepare Direct Verbs for Direct Rule
2019-04-03 13:21 ` [dpdk-dev] [PATCH v3 1/3] net/mlx5: prepare Direct Verbs for Direct Rule Ori Kam
@ 2019-04-03 13:21 ` Ori Kam
0 siblings, 0 replies; 52+ messages in thread
From: Ori Kam @ 2019-04-03 13:21 UTC (permalink / raw)
To: Matan Azrad, Yongseok Koh, Shahaf Shuler; +Cc: dev, Ori Kam, Slava Ovsiienko
This is the first patch of a series that is designed to enable the
Direct Rules API.
The main difference between Direct Verbs and Direct Rules from API
prespective, is that in Direct Rules each action has it's own create
function and the object itself is of type void.
In this patch I'm adding functions to generate actions that currenlty
are done without create action, and I'm changing the action type to be
void *, so in next patches only the glue functions will need to change.
Signed-off-by: Ori Kam <orika@mellanox.com>
Acked-by: Shahaf Shuler <shahafs@mellanox.com>
---
drivers/net/mlx5/Makefile | 2 +-
drivers/net/mlx5/meson.build | 2 +-
drivers/net/mlx5/mlx5.h | 2 +
drivers/net/mlx5/mlx5_flow.h | 17 +++-
drivers/net/mlx5/mlx5_flow_dv.c | 184 +++++++++++++++++++++++++++++-----------
drivers/net/mlx5/mlx5_glue.c | 141 ++++++++++++++++++++++++------
drivers/net/mlx5/mlx5_glue.h | 26 +++---
7 files changed, 280 insertions(+), 94 deletions(-)
diff --git a/drivers/net/mlx5/Makefile b/drivers/net/mlx5/Makefile
index ccd179c..0d20f0f 100644
--- a/drivers/net/mlx5/Makefile
+++ b/drivers/net/mlx5/Makefile
@@ -8,7 +8,7 @@ include $(RTE_SDK)/mk/rte.vars.mk
LIB = librte_pmd_mlx5.a
LIB_GLUE = $(LIB_GLUE_BASE).$(LIB_GLUE_VERSION)
LIB_GLUE_BASE = librte_pmd_mlx5_glue.so
-LIB_GLUE_VERSION = 19.02.0
+LIB_GLUE_VERSION = 19.05.0
# Sources.
SRCS-$(CONFIG_RTE_LIBRTE_MLX5_PMD) += mlx5.c
diff --git a/drivers/net/mlx5/meson.build b/drivers/net/mlx5/meson.build
index a429655..a4c684e 100644
--- a/drivers/net/mlx5/meson.build
+++ b/drivers/net/mlx5/meson.build
@@ -4,7 +4,7 @@
pmd_dlopen = (get_option('ibverbs_link') == 'dlopen')
LIB_GLUE_BASE = 'librte_pmd_mlx5_glue.so'
-LIB_GLUE_VERSION = '19.02.0'
+LIB_GLUE_VERSION = '19.05.0'
LIB_GLUE = LIB_GLUE_BASE + '.' + LIB_GLUE_VERSION
if pmd_dlopen
dpdk_conf.set('RTE_IBVERBS_LINK_DLOPEN', 1)
diff --git a/drivers/net/mlx5/mlx5.h b/drivers/net/mlx5/mlx5.h
index 47a7d75..784bf9b 100644
--- a/drivers/net/mlx5/mlx5.h
+++ b/drivers/net/mlx5/mlx5.h
@@ -309,6 +309,8 @@ struct mlx5_priv {
LIST_HEAD(matchers, mlx5_flow_dv_matcher) matchers;
LIST_HEAD(encap_decap, mlx5_flow_dv_encap_decap_resource) encaps_decaps;
LIST_HEAD(modify_cmd, mlx5_flow_dv_modify_hdr_resource) modify_cmds;
+ LIST_HEAD(tag, mlx5_flow_dv_tag_resource) tags;
+ /* Tags resources cache. */
uint32_t link_speed_capa; /* Link speed capabilities. */
struct mlx5_xstats_ctrl xstats_ctrl; /* Extended stats control. */
struct mlx5_stats_ctrl stats_ctrl; /* Stats control. */
diff --git a/drivers/net/mlx5/mlx5_flow.h b/drivers/net/mlx5/mlx5_flow.h
index e1e798b..4f69ae2 100644
--- a/drivers/net/mlx5/mlx5_flow.h
+++ b/drivers/net/mlx5/mlx5_flow.h
@@ -214,7 +214,7 @@ struct mlx5_flow_dv_encap_decap_resource {
LIST_ENTRY(mlx5_flow_dv_encap_decap_resource) next;
/* Pointer to next element. */
rte_atomic32_t refcnt; /**< Reference counter. */
- struct ibv_flow_action *verbs_action;
+ void *verbs_action;
/**< Verbs encap/decap action object. */
uint8_t buf[MLX5_ENCAP_MAX_LEN];
size_t size;
@@ -222,6 +222,16 @@ struct mlx5_flow_dv_encap_decap_resource {
uint8_t ft_type;
};
+/* Tag resource structure. */
+struct mlx5_flow_dv_tag_resource {
+ LIST_ENTRY(mlx5_flow_dv_tag_resource) next;
+ /* Pointer to next element. */
+ rte_atomic32_t refcnt; /**< Reference counter. */
+ void *action;
+ /**< Verbs tag action object. */
+ uint32_t tag; /**< the tag value. */
+};
+
/* Number of modification commands. */
#define MLX5_MODIFY_NUM 8
@@ -259,7 +269,7 @@ struct mlx5_flow_dv {
/**< Pointer to modify header resource in cache. */
struct ibv_flow *flow; /**< Installed flow. */
#ifdef HAVE_IBV_FLOW_DV_SUPPORT
- struct mlx5dv_flow_action_attr actions[MLX5_DV_MAX_NUMBER_OF_ACTIONS];
+ void *actions[MLX5_DV_MAX_NUMBER_OF_ACTIONS];
/**< Action list. */
#endif
int actions_n; /**< number of actions. */
@@ -332,6 +342,7 @@ struct mlx5_flow_counter {
};
uint64_t hits; /**< Number of packets matched by the rule. */
uint64_t bytes; /**< Number of bytes matched by the rule. */
+ void *action; /**< Pointer to the dv action. */
};
/* Flow structure. */
@@ -339,6 +350,8 @@ struct rte_flow {
TAILQ_ENTRY(rte_flow) next; /**< Pointer to the next flow structure. */
enum mlx5_flow_drv_type drv_type; /**< Drvier type. */
struct mlx5_flow_counter *counter; /**< Holds flow counter. */
+ struct mlx5_flow_dv_tag_resource *tag_resource;
+ /**< pointer to the tag action. */
struct rte_flow_action_rss rss;/**< RSS context. */
uint8_t key[MLX5_RSS_HASH_KEY_LEN]; /**< RSS hash key. */
uint16_t (*queue)[]; /**< Destination queues to redirect traffic to. */
diff --git a/drivers/net/mlx5/mlx5_flow_dv.c b/drivers/net/mlx5/mlx5_flow_dv.c
index ad84dea..d9e2ac9 100644
--- a/drivers/net/mlx5/mlx5_flow_dv.c
+++ b/drivers/net/mlx5/mlx5_flow_dv.c
@@ -32,7 +32,6 @@
#include "mlx5_prm.h"
#include "mlx5_glue.h"
#include "mlx5_flow.h"
-
#ifdef HAVE_IBV_FLOW_DV_SUPPORT
#ifndef HAVE_IBV_FLOW_DEVX_COUNTERS
@@ -1537,6 +1536,11 @@ struct field_modify_info modify_tcp[] = {
.id = id,
.dcs = dcs,
};
+ tmpl.action = mlx5_glue->dv_create_flow_action_counter(dcs->obj, 0);
+ if (!tmpl.action) {
+ ret = errno;
+ goto error_exit;
+ }
*cnt = tmpl;
LIST_INSERT_HEAD(&priv->flow_counters, cnt, next);
return cnt;
@@ -2828,6 +2832,97 @@ struct field_modify_info modify_tcp[] = {
}
/**
+ * Find existing tag resource or create and register a new one.
+ *
+ * @param dev[in, out]
+ * Pointer to rte_eth_dev structure.
+ * @param[in, out] resource
+ * Pointer to tag resource.
+ * @parm[in, out] dev_flow
+ * Pointer to the dev_flow.
+ * @param[out] error
+ * pointer to error structure.
+ *
+ * @return
+ * 0 on success otherwise -errno and errno is set.
+ */
+static int
+flow_dv_tag_resource_register
+ (struct rte_eth_dev *dev,
+ struct mlx5_flow_dv_tag_resource *resource,
+ struct mlx5_flow *dev_flow,
+ struct rte_flow_error *error)
+{
+ struct mlx5_priv *priv = dev->data->dev_private;
+ struct mlx5_flow_dv_tag_resource *cache_resource;
+
+ /* Lookup a matching resource from cache. */
+ LIST_FOREACH(cache_resource, &priv->tags, next) {
+ if (resource->tag == cache_resource->tag) {
+ DRV_LOG(DEBUG, "tag resource %p: refcnt %d++",
+ (void *)cache_resource,
+ rte_atomic32_read(&cache_resource->refcnt));
+ rte_atomic32_inc(&cache_resource->refcnt);
+ dev_flow->flow->tag_resource = cache_resource;
+ return 0;
+ }
+ }
+ /* Register new resource. */
+ cache_resource = rte_calloc(__func__, 1, sizeof(*cache_resource), 0);
+ if (!cache_resource)
+ return rte_flow_error_set(error, ENOMEM,
+ RTE_FLOW_ERROR_TYPE_UNSPECIFIED, NULL,
+ "cannot allocate resource memory");
+ *cache_resource = *resource;
+ cache_resource->action = mlx5_glue->dv_create_flow_action_tag
+ (resource->tag);
+ if (!cache_resource->action) {
+ rte_free(cache_resource);
+ return rte_flow_error_set(error, ENOMEM,
+ RTE_FLOW_ERROR_TYPE_UNSPECIFIED,
+ NULL, "cannot create action");
+ }
+ rte_atomic32_init(&cache_resource->refcnt);
+ rte_atomic32_inc(&cache_resource->refcnt);
+ LIST_INSERT_HEAD(&priv->tags, cache_resource, next);
+ dev_flow->flow->tag_resource = cache_resource;
+ DRV_LOG(DEBUG, "new tag resource %p: refcnt %d++",
+ (void *)cache_resource,
+ rte_atomic32_read(&cache_resource->refcnt));
+ return 0;
+}
+
+/**
+ * Release the tag.
+ *
+ * @param dev
+ * Pointer to Ethernet device.
+ * @param flow
+ * Pointer to mlx5_flow.
+ *
+ * @return
+ * 1 while a reference on it exists, 0 when freed.
+ */
+static int
+flow_dv_tag_release(struct rte_eth_dev *dev,
+ struct mlx5_flow_dv_tag_resource *tag)
+{
+ assert(tag);
+ DRV_LOG(DEBUG, "port %u tag %p: refcnt %d--",
+ dev->data->port_id, (void *)tag,
+ rte_atomic32_read(&tag->refcnt));
+ if (rte_atomic32_dec_and_test(&tag->refcnt)) {
+ claim_zero(mlx5_glue->destroy_flow_action(tag->action));
+ LIST_REMOVE(tag, next);
+ DRV_LOG(DEBUG, "port %u tag %p: removed",
+ dev->data->port_id, (void *)tag);
+ rte_free(tag);
+ return 0;
+ }
+ return 1;
+}
+
+/**
* Fill the flow with DV spec.
*
* @param[in] dev
@@ -2872,6 +2967,7 @@ struct field_modify_info modify_tcp[] = {
MLX5DV_FLOW_TABLE_TYPE_NIC_RX
};
union flow_dv_attr flow_attr = { .attr = 0 };
+ struct mlx5_flow_dv_tag_resource tag_resource;
if (priority == MLX5_FLOW_PRIO_RSVD)
priority = priv->config.flow_prio - 1;
@@ -2886,26 +2982,29 @@ struct field_modify_info modify_tcp[] = {
case RTE_FLOW_ACTION_TYPE_VOID:
break;
case RTE_FLOW_ACTION_TYPE_FLAG:
- dev_flow->dv.actions[actions_n].type =
- MLX5DV_FLOW_ACTION_TAG;
- dev_flow->dv.actions[actions_n].tag_value =
+ tag_resource.tag =
mlx5_flow_mark_set(MLX5_FLOW_MARK_DEFAULT);
- actions_n++;
+ if (!flow->tag_resource)
+ if (flow_dv_tag_resource_register
+ (dev, &tag_resource, dev_flow, error))
+ return errno;
+ dev_flow->dv.actions[actions_n++] =
+ flow->tag_resource->action;
action_flags |= MLX5_FLOW_ACTION_FLAG;
break;
case RTE_FLOW_ACTION_TYPE_MARK:
- dev_flow->dv.actions[actions_n].type =
- MLX5DV_FLOW_ACTION_TAG;
- dev_flow->dv.actions[actions_n].tag_value =
- mlx5_flow_mark_set
- (((const struct rte_flow_action_mark *)
- (actions->conf))->id);
- actions_n++;
+ tag_resource.tag = mlx5_flow_mark_set
+ (((const struct rte_flow_action_mark *)
+ (actions->conf))->id);
+ if (!flow->tag_resource)
+ if (flow_dv_tag_resource_register
+ (dev, &tag_resource, dev_flow, error))
+ return errno;
+ dev_flow->dv.actions[actions_n++] =
+ flow->tag_resource->action;
action_flags |= MLX5_FLOW_ACTION_MARK;
break;
case RTE_FLOW_ACTION_TYPE_DROP:
- dev_flow->dv.actions[actions_n].type =
- MLX5DV_FLOW_ACTION_DROP;
action_flags |= MLX5_FLOW_ACTION_DROP;
break;
case RTE_FLOW_ACTION_TYPE_QUEUE:
@@ -2933,17 +3032,13 @@ struct field_modify_info modify_tcp[] = {
rte_errno = ENOTSUP;
goto cnt_err;
}
- flow->counter =
- flow_dv_counter_new(dev,
- count->shared, count->id);
+ flow->counter = flow_dv_counter_new(dev, count->shared,
+ count->id);
if (flow->counter == NULL)
goto cnt_err;
- dev_flow->dv.actions[actions_n].type =
- MLX5DV_FLOW_ACTION_COUNTERS_DEVX;
- dev_flow->dv.actions[actions_n].obj =
- flow->counter->dcs->obj;
+ dev_flow->dv.actions[actions_n++] =
+ flow->counter->action;
action_flags |= MLX5_FLOW_ACTION_COUNT;
- ++actions_n;
break;
cnt_err:
if (rte_errno == ENOTSUP)
@@ -2964,11 +3059,8 @@ struct field_modify_info modify_tcp[] = {
if (flow_dv_create_action_l2_encap(dev, actions,
dev_flow, error))
return -rte_errno;
- dev_flow->dv.actions[actions_n].type =
- MLX5DV_FLOW_ACTION_IBV_FLOW_ACTION;
- dev_flow->dv.actions[actions_n].action =
+ dev_flow->dv.actions[actions_n++] =
dev_flow->dv.encap_decap->verbs_action;
- actions_n++;
action_flags |= actions->type ==
RTE_FLOW_ACTION_TYPE_VXLAN_ENCAP ?
MLX5_FLOW_ACTION_VXLAN_ENCAP :
@@ -2979,11 +3071,8 @@ struct field_modify_info modify_tcp[] = {
if (flow_dv_create_action_l2_decap(dev, dev_flow,
error))
return -rte_errno;
- dev_flow->dv.actions[actions_n].type =
- MLX5DV_FLOW_ACTION_IBV_FLOW_ACTION;
- dev_flow->dv.actions[actions_n].action =
+ dev_flow->dv.actions[actions_n++] =
dev_flow->dv.encap_decap->verbs_action;
- actions_n++;
action_flags |= actions->type ==
RTE_FLOW_ACTION_TYPE_VXLAN_DECAP ?
MLX5_FLOW_ACTION_VXLAN_DECAP :
@@ -2995,9 +3084,7 @@ struct field_modify_info modify_tcp[] = {
if (flow_dv_create_action_raw_encap
(dev, actions, dev_flow, attr, error))
return -rte_errno;
- dev_flow->dv.actions[actions_n].type =
- MLX5DV_FLOW_ACTION_IBV_FLOW_ACTION;
- dev_flow->dv.actions[actions_n].action =
+ dev_flow->dv.actions[actions_n++] =
dev_flow->dv.encap_decap->verbs_action;
} else {
/* Handle encap without preceding decap. */
@@ -3005,12 +3092,9 @@ struct field_modify_info modify_tcp[] = {
dev_flow,
error))
return -rte_errno;
- dev_flow->dv.actions[actions_n].type =
- MLX5DV_FLOW_ACTION_IBV_FLOW_ACTION;
- dev_flow->dv.actions[actions_n].action =
+ dev_flow->dv.actions[actions_n++] =
dev_flow->dv.encap_decap->verbs_action;
}
- actions_n++;
action_flags |= MLX5_FLOW_ACTION_RAW_ENCAP;
break;
case RTE_FLOW_ACTION_TYPE_RAW_DECAP:
@@ -3025,11 +3109,8 @@ struct field_modify_info modify_tcp[] = {
dev_flow,
error))
return -rte_errno;
- dev_flow->dv.actions[actions_n].type =
- MLX5DV_FLOW_ACTION_IBV_FLOW_ACTION;
- dev_flow->dv.actions[actions_n].action =
+ dev_flow->dv.actions[actions_n++] =
dev_flow->dv.encap_decap->verbs_action;
- actions_n++;
}
/* If decap is followed by encap, handle it at encap. */
action_flags |= MLX5_FLOW_ACTION_RAW_DECAP;
@@ -3098,11 +3179,8 @@ struct field_modify_info modify_tcp[] = {
dev_flow,
error))
return -rte_errno;
- dev_flow->dv.actions[actions_n].type =
- MLX5DV_FLOW_ACTION_IBV_FLOW_ACTION;
- dev_flow->dv.actions[actions_n].action =
+ dev_flow->dv.actions[actions_n++] =
dev_flow->dv.modify_hdr->verbs_action;
- actions_n++;
}
break;
default:
@@ -3277,9 +3355,9 @@ struct field_modify_info modify_tcp[] = {
"cannot get drop hash queue");
goto error;
}
- dv->actions[n].type = MLX5DV_FLOW_ACTION_DEST_IBV_QP;
- dv->actions[n].qp = dv->hrxq->qp;
- n++;
+ dv->actions[n++] =
+ mlx5_glue->dv_create_flow_action_dest_ibv_qp
+ (dv->hrxq->qp);
} else if (flow->actions &
(MLX5_FLOW_ACTION_QUEUE | MLX5_FLOW_ACTION_RSS)) {
struct mlx5_hrxq *hrxq;
@@ -3304,9 +3382,9 @@ struct field_modify_info modify_tcp[] = {
goto error;
}
dv->hrxq = hrxq;
- dv->actions[n].type = MLX5DV_FLOW_ACTION_DEST_IBV_QP;
- dv->actions[n].qp = hrxq->qp;
- n++;
+ dv->actions[n++] =
+ mlx5_glue->dv_create_flow_action_dest_ibv_qp
+ (dv->hrxq->qp);
}
dv->flow =
mlx5_glue->dv_create_flow(dv->matcher->matcher_object,
@@ -3484,6 +3562,10 @@ struct field_modify_info modify_tcp[] = {
flow_dv_counter_release(flow->counter);
flow->counter = NULL;
}
+ if (flow->tag_resource) {
+ flow_dv_tag_release(dev, flow->tag_resource);
+ flow->tag_resource = NULL;
+ }
while (!LIST_EMPTY(&flow->dev_flows)) {
dev_flow = LIST_FIRST(&flow->dev_flows);
LIST_REMOVE(dev_flow, next);
diff --git a/drivers/net/mlx5/mlx5_glue.c b/drivers/net/mlx5/mlx5_glue.c
index c817d86..4b5aade 100644
--- a/drivers/net/mlx5/mlx5_glue.c
+++ b/drivers/net/mlx5/mlx5_glue.c
@@ -175,10 +175,20 @@
}
static int
-mlx5_glue_destroy_flow_action(struct ibv_flow_action *action)
+mlx5_glue_destroy_flow_action(void *action)
{
#ifdef HAVE_IBV_FLOW_DV_SUPPORT
- return ibv_destroy_flow_action(action);
+ struct mlx5dv_flow_action_attr *attr = action;
+ int res = 0;
+ switch (attr->type) {
+ case MLX5DV_FLOW_ACTION_TAG:
+ break;
+ default:
+ res = ibv_destroy_flow_action(attr->action);
+ break;
+ }
+ free(action);
+ return res;
#else
(void)action;
return ENOTSUP;
@@ -430,16 +440,23 @@
mlx5_glue_dv_create_flow(struct mlx5dv_flow_matcher *matcher,
struct mlx5dv_flow_match_parameters *match_value,
size_t num_actions,
- struct mlx5dv_flow_action_attr *actions_attr)
+ void *actions[])
{
#ifdef HAVE_IBV_FLOW_DV_SUPPORT
+ struct mlx5dv_flow_action_attr actions_attr[8];
+
+ if (num_actions > 8)
+ return NULL;
+ for (size_t i = 0; i < num_actions; i++)
+ actions_attr[i] =
+ *((struct mlx5dv_flow_action_attr *)(actions[i]));
return mlx5dv_create_flow(matcher, match_value,
num_actions, actions_attr);
#else
(void)matcher;
(void)match_value;
(void)num_actions;
- (void)actions_attr;
+ (void)actions;
return NULL;
#endif
}
@@ -455,31 +472,45 @@
#endif
}
-static struct ibv_flow_action *
-mlx5_glue_dv_create_flow_action_packet_reformat
- (struct ibv_context *ctx,
- size_t data_sz,
- void *data,
- enum mlx5dv_flow_action_packet_reformat_type reformat_type,
- enum mlx5dv_flow_table_type ft_type)
+static void *
+mlx5_glue_dv_create_flow_action_counter(void *counter_obj, uint32_t offset)
{
#ifdef HAVE_IBV_FLOW_DV_SUPPORT
- return mlx5dv_create_flow_action_packet_reformat(ctx,
- data_sz,
- data,
- reformat_type,
- ft_type);
+ struct mlx5dv_flow_action_attr *action;
+
+ (void)offset;
+ action = malloc(sizeof(*action));
+ if (!action)
+ return NULL;
+ action->type = MLX5DV_FLOW_ACTION_COUNTERS_DEVX;
+ action->obj = counter_obj;
+ return action;
#else
- (void)ctx;
- (void)data_sz;
- (void)data;
- (void)reformat_type;
- (void)ft_type;
+ (void)counter_obj;
+ (void)offset;
return NULL;
#endif
}
-static struct ibv_flow_action *
+static void *
+mlx5_glue_dv_create_flow_action_dest_ibv_qp(void *qp)
+{
+#ifdef HAVE_IBV_FLOW_DV_SUPPORT
+ struct mlx5dv_flow_action_attr *action;
+
+ action = malloc(sizeof(*action));
+ if (!action)
+ return NULL;
+ action->type = MLX5DV_FLOW_ACTION_DEST_IBV_QP;
+ action->obj = qp;
+ return action;
+#else
+ (void)qp;
+ return NULL;
+#endif
+}
+
+static void *
mlx5_glue_dv_create_flow_action_modify_header
(struct ibv_context *ctx,
size_t actions_sz,
@@ -487,8 +518,15 @@
enum mlx5dv_flow_table_type ft_type)
{
#ifdef HAVE_IBV_FLOW_DV_SUPPORT
- return mlx5dv_create_flow_action_modify_header(ctx, actions_sz,
- actions, ft_type);
+ struct mlx5dv_flow_action_attr *action;
+
+ action = malloc(sizeof(*action));
+ if (!action)
+ return NULL;
+ action->type = MLX5DV_FLOW_ACTION_IBV_FLOW_ACTION;
+ action->action = mlx5dv_create_flow_action_modify_header
+ (ctx, actions_sz, actions, ft_type);
+ return action;
#else
(void)ctx;
(void)actions_sz;
@@ -498,6 +536,50 @@
#endif
}
+static void *
+mlx5_glue_dv_create_flow_action_packet_reformat
+ (struct ibv_context *ctx,
+ size_t data_sz,
+ void *data,
+ enum mlx5dv_flow_action_packet_reformat_type reformat_type,
+ enum mlx5dv_flow_table_type ft_type)
+{
+#ifdef HAVE_IBV_FLOW_DV_SUPPORT
+ struct mlx5dv_flow_action_attr *action;
+
+ action = malloc(sizeof(*action));
+ if (!action)
+ return NULL;
+ action->type = MLX5DV_FLOW_ACTION_IBV_FLOW_ACTION;
+ action->action = mlx5dv_create_flow_action_packet_reformat
+ (ctx, data_sz, data, reformat_type, ft_type);
+ return action;
+#else
+ (void)ctx;
+ (void)data_sz;
+ (void)data;
+ (void)reformat_type;
+ (void)ft_type;
+ return NULL;
+#endif
+}
+
+static void *
+mlx5_glue_dv_create_flow_action_tag(uint32_t tag)
+{
+#ifdef HAVE_IBV_FLOW_DV_SUPPORT
+ struct mlx5dv_flow_action_attr *action;
+ action = malloc(sizeof(*action));
+ if (!action)
+ return NULL;
+ action->type = MLX5DV_FLOW_ACTION_TAG;
+ action->tag_value = tag;
+ return action;
+#endif
+ (void)tag;
+ return NULL;
+}
+
static struct ibv_context *
mlx5_glue_dv_open_device(struct ibv_device *device)
{
@@ -645,10 +727,15 @@
.dv_create_flow_matcher = mlx5_glue_dv_create_flow_matcher,
.dv_destroy_flow_matcher = mlx5_glue_dv_destroy_flow_matcher,
.dv_create_flow = mlx5_glue_dv_create_flow,
- .dv_create_flow_action_packet_reformat =
- mlx5_glue_dv_create_flow_action_packet_reformat,
+ .dv_create_flow_action_counter =
+ mlx5_glue_dv_create_flow_action_counter,
+ .dv_create_flow_action_dest_ibv_qp =
+ mlx5_glue_dv_create_flow_action_dest_ibv_qp,
.dv_create_flow_action_modify_header =
- mlx5_glue_dv_create_flow_action_modify_header,
+ mlx5_glue_dv_create_flow_action_modify_header,
+ .dv_create_flow_action_packet_reformat =
+ mlx5_glue_dv_create_flow_action_packet_reformat,
+ .dv_create_flow_action_tag = mlx5_glue_dv_create_flow_action_tag,
.dv_open_device = mlx5_glue_dv_open_device,
.devx_obj_create = mlx5_glue_devx_obj_create,
.devx_obj_destroy = mlx5_glue_devx_obj_destroy,
diff --git a/drivers/net/mlx5/mlx5_glue.h b/drivers/net/mlx5/mlx5_glue.h
index b118960..32487ea 100644
--- a/drivers/net/mlx5/mlx5_glue.h
+++ b/drivers/net/mlx5/mlx5_glue.h
@@ -55,6 +55,10 @@
enum mlx5dv_flow_table_type { flow_table_type = 0, };
#endif
+#ifndef HAVE_IBV_FLOW_DEVX_COUNTERS
+#define MLX5DV_FLOW_ACTION_COUNTERS_DEVX 0
+#endif
+
#ifndef HAVE_IBV_DEVX_OBJ
struct mlx5dv_devx_obj;
#endif
@@ -98,7 +102,7 @@ struct mlx5_glue {
struct ibv_flow *(*create_flow)(struct ibv_qp *qp,
struct ibv_flow_attr *flow);
int (*destroy_flow)(struct ibv_flow *flow_id);
- int (*destroy_flow_action)(struct ibv_flow_action *action);
+ int (*destroy_flow_action)(void *action);
struct ibv_qp *(*create_qp)(struct ibv_pd *pd,
struct ibv_qp_init_attr *qp_init_attr);
struct ibv_qp *(*create_qp_ex)
@@ -160,19 +164,17 @@ struct mlx5_glue {
int (*dv_destroy_flow_matcher)(struct mlx5dv_flow_matcher *matcher);
struct ibv_flow *(*dv_create_flow)(struct mlx5dv_flow_matcher *matcher,
struct mlx5dv_flow_match_parameters *match_value,
- size_t num_actions,
- struct mlx5dv_flow_action_attr *actions_attr);
- struct ibv_flow_action *(*dv_create_flow_action_packet_reformat)
- (struct ibv_context *ctx,
- size_t data_sz,
- void *data,
+ size_t num_actions, void *actions[]);
+ void *(*dv_create_flow_action_counter)(void *obj, uint32_t offset);
+ void *(*dv_create_flow_action_dest_ibv_qp)(void *qp);
+ void *(*dv_create_flow_action_modify_header)
+ (struct ibv_context *ctx, size_t actions_sz, uint64_t actions[],
+ enum mlx5dv_flow_table_type ft_type);
+ void *(*dv_create_flow_action_packet_reformat)
+ (struct ibv_context *ctx, size_t data_sz, void *data,
enum mlx5dv_flow_action_packet_reformat_type reformat_type,
enum mlx5dv_flow_table_type ft_type);
- struct ibv_flow_action *(*dv_create_flow_action_modify_header)
- (struct ibv_context *ctx,
- size_t actions_sz,
- uint64_t actions[],
- enum mlx5dv_flow_table_type ft_type);
+ void *(*dv_create_flow_action_tag)(uint32_t tag);
struct ibv_context *(*dv_open_device)(struct ibv_device *device);
struct mlx5dv_devx_obj *(*devx_obj_create)
(struct ibv_context *ctx,
--
1.8.3.1
^ permalink raw reply [flat|nested] 52+ messages in thread
* [dpdk-dev] [PATCH v3 2/3] net/mlx5: add Direct Rules API
2019-04-03 13:21 ` [dpdk-dev] [PATCH v3 " Ori Kam
2019-04-03 13:21 ` Ori Kam
2019-04-03 13:21 ` [dpdk-dev] [PATCH v3 1/3] net/mlx5: prepare Direct Verbs for Direct Rule Ori Kam
@ 2019-04-03 13:21 ` Ori Kam
2019-04-03 13:21 ` Ori Kam
2019-04-03 13:21 ` [dpdk-dev] [PATCH v3 3/3] net/mlx5: add jump action support for NIC Ori Kam
3 siblings, 1 reply; 52+ messages in thread
From: Ori Kam @ 2019-04-03 13:21 UTC (permalink / raw)
To: Matan Azrad, Yongseok Koh, Shahaf Shuler; +Cc: dev, Ori Kam, Slava Ovsiienko
Adds calls to the Direct Rules API inside the glue functions.
Due to difference in parameters between the Direct Rules and Direct
Verbs some of the glue functions API was updated.
Signed-off-by: Ori Kam <orika@mellanox.com>
Acked-by: Shahaf Shuler <shahafs@mellanox.com>
---
drivers/net/mlx5/Makefile | 5 ++
drivers/net/mlx5/mlx5.c | 16 ++++
drivers/net/mlx5/mlx5.h | 15 ++++
drivers/net/mlx5/mlx5_flow.c | 1 +
drivers/net/mlx5/mlx5_flow.h | 6 +-
drivers/net/mlx5/mlx5_flow_dv.c | 103 ++++++++++++++++++++----
drivers/net/mlx5/mlx5_glue.c | 170 ++++++++++++++++++++++++++++++++++------
drivers/net/mlx5/mlx5_glue.h | 31 +++++---
drivers/net/mlx5/mlx5_prm.h | 24 +++++-
9 files changed, 318 insertions(+), 53 deletions(-)
diff --git a/drivers/net/mlx5/Makefile b/drivers/net/mlx5/Makefile
index 0d20f0f..93bc869 100644
--- a/drivers/net/mlx5/Makefile
+++ b/drivers/net/mlx5/Makefile
@@ -156,6 +156,11 @@ mlx5_autoconf.h.new: $(RTE_SDK)/buildtools/auto-config-h.sh
func mlx5dv_create_flow_action_packet_reformat \
$(AUTOCONF_OUTPUT)
$Q sh -- '$<' '$@' \
+ HAVE_MLX5DV_DR \
+ infiniband/mlx5dv.h \
+ enum MLX5DV_DR_NS_TYPE_TERMINATING \
+ $(AUTOCONF_OUTPUT)
+ $Q sh -- '$<' '$@' \
HAVE_IBV_DEVX_OBJ \
infiniband/mlx5dv.h \
func mlx5dv_devx_obj_create \
diff --git a/drivers/net/mlx5/mlx5.c b/drivers/net/mlx5/mlx5.c
index 4044505..65aa9cf 100644
--- a/drivers/net/mlx5/mlx5.c
+++ b/drivers/net/mlx5/mlx5.c
@@ -1483,6 +1483,22 @@ struct mlx5_dev_spawn_data {
priv->tcf_context = NULL;
}
}
+#ifdef HAVE_MLX5DV_DR
+ priv->rx_ns = mlx5dv_dr_create_ns
+ (sh->ctx, MLX5DV_DR_NS_DOMAIN_INGRESS_BYPASS);
+ if (priv->rx_ns == NULL) {
+ DRV_LOG(ERR, "mlx5dv_dr_create_ns failed");
+ err = errno;
+ goto error;
+ }
+ priv->tx_ns = mlx5dv_dr_create_ns(sh->ctx,
+ MLX5DV_DR_NS_DOMAIN_EGRESS_BYPASS);
+ if (priv->tx_ns == NULL) {
+ DRV_LOG(ERR, "mlx5dv_dr_create_ns failed");
+ err = errno;
+ goto error;
+ }
+#endif
TAILQ_INIT(&priv->flows);
TAILQ_INIT(&priv->ctrl_flows);
/* Hint libmlx5 to use PMD allocator for data plane resources */
diff --git a/drivers/net/mlx5/mlx5.h b/drivers/net/mlx5/mlx5.h
index 784bf9b..d4963cb 100644
--- a/drivers/net/mlx5/mlx5.h
+++ b/drivers/net/mlx5/mlx5.h
@@ -259,6 +259,15 @@ struct mlx5_ibv_shared {
struct mlx5_ibv_shared_port port[]; /* per device port data array. */
};
+/* Table structure. */
+struct mlx5_flow_tbl_resource {
+ void *obj; /**< Pointer to DR table object. */
+ rte_atomic32_t refcnt; /**< Reference counter. */
+};
+
+#define MLX5_MAX_TABLES 1024
+#define MLX5_GROUP_FACTOR 1
+
struct mlx5_priv {
LIST_ENTRY(mlx5_priv) mem_event_cb;
/**< Called by memory event callback. */
@@ -326,6 +335,12 @@ struct mlx5_priv {
/* UAR same-page access control required in 32bit implementations. */
#endif
struct mlx5_flow_tcf_context *tcf_context; /* TC flower context. */
+ void *rx_ns; /* RX Direct Rules name space handle. */
+ struct mlx5_flow_tbl_resource rx_tbl[MLX5_MAX_TABLES];
+ /* RX Direct Rules tables. */
+ void *tx_ns; /* TX Direct Rules name space handle. */
+ struct mlx5_flow_tbl_resource tx_tbl[MLX5_MAX_TABLES];
+ /* TX Direct Rules tables/ */
};
#define PORT_ID(priv) ((priv)->dev_data->port_id)
diff --git a/drivers/net/mlx5/mlx5_flow.c b/drivers/net/mlx5/mlx5_flow.c
index bc6a7c1..9dc492a 100644
--- a/drivers/net/mlx5/mlx5_flow.c
+++ b/drivers/net/mlx5/mlx5_flow.c
@@ -2084,6 +2084,7 @@ uint32_t mlx5_flow_adjust_priority(struct rte_eth_dev *dev, int32_t priority,
flow_size += RTE_ALIGN_CEIL(sizeof(uint16_t), sizeof(void *));
flow = rte_calloc(__func__, 1, flow_size, 0);
flow->drv_type = flow_get_drv_type(dev, attr);
+ flow->ingress = attr->ingress;
assert(flow->drv_type > MLX5_FLOW_TYPE_MIN &&
flow->drv_type < MLX5_FLOW_TYPE_MAX);
flow->queue = (void *)(flow + 1);
diff --git a/drivers/net/mlx5/mlx5_flow.h b/drivers/net/mlx5/mlx5_flow.h
index 4f69ae2..8ba37a0 100644
--- a/drivers/net/mlx5/mlx5_flow.h
+++ b/drivers/net/mlx5/mlx5_flow.h
@@ -204,6 +204,7 @@ struct mlx5_flow_dv_matcher {
uint16_t crc; /**< CRC of key. */
uint16_t priority; /**< Priority of matcher. */
uint8_t egress; /**< Egress matcher. */
+ uint32_t group; /**< The matcher group. */
struct mlx5_flow_dv_match_params mask; /**< Matcher mask. */
};
@@ -220,6 +221,7 @@ struct mlx5_flow_dv_encap_decap_resource {
size_t size;
uint8_t reformat_type;
uint8_t ft_type;
+ uint64_t flags; /**< Flags for RDMA API. */
};
/* Tag resource structure. */
@@ -348,7 +350,7 @@ struct mlx5_flow_counter {
/* Flow structure. */
struct rte_flow {
TAILQ_ENTRY(rte_flow) next; /**< Pointer to the next flow structure. */
- enum mlx5_flow_drv_type drv_type; /**< Drvier type. */
+ enum mlx5_flow_drv_type drv_type; /**< Driver type. */
struct mlx5_flow_counter *counter; /**< Holds flow counter. */
struct mlx5_flow_dv_tag_resource *tag_resource;
/**< pointer to the tag action. */
@@ -360,6 +362,8 @@ struct rte_flow {
uint64_t actions;
/**< Bit-fields of detected actions, see MLX5_FLOW_ACTION_*. */
struct mlx5_fdir *fdir; /**< Pointer to associated FDIR if any. */
+ uint8_t ingress; /**< 1 if the flow is ingress. */
+ uint32_t group; /**< The group index. */
};
typedef int (*mlx5_flow_validate_t)(struct rte_eth_dev *dev,
diff --git a/drivers/net/mlx5/mlx5_flow_dv.c b/drivers/net/mlx5/mlx5_flow_dv.c
index d9e2ac9..dd5b541 100644
--- a/drivers/net/mlx5/mlx5_flow_dv.c
+++ b/drivers/net/mlx5/mlx5_flow_dv.c
@@ -809,11 +809,20 @@ struct field_modify_info modify_tcp[] = {
{
struct mlx5_priv *priv = dev->data->dev_private;
struct mlx5_flow_dv_encap_decap_resource *cache_resource;
+ struct rte_flow *flow = dev_flow->flow;
+ struct mlx5dv_dr_ns *ns;
+
+ resource->flags = flow->group ? 0 : 1;
+ if (flow->ingress)
+ ns = priv->rx_ns;
+ else
+ ns = priv->tx_ns;
/* Lookup a matching resource from cache. */
LIST_FOREACH(cache_resource, &priv->encaps_decaps, next) {
if (resource->reformat_type == cache_resource->reformat_type &&
resource->ft_type == cache_resource->ft_type &&
+ resource->flags == cache_resource->flags &&
resource->size == cache_resource->size &&
!memcmp((const void *)resource->buf,
(const void *)cache_resource->buf,
@@ -835,10 +844,10 @@ struct field_modify_info modify_tcp[] = {
*cache_resource = *resource;
cache_resource->verbs_action =
mlx5_glue->dv_create_flow_action_packet_reformat
- (priv->sh->ctx, cache_resource->size,
- (cache_resource->size ? cache_resource->buf : NULL),
- cache_resource->reformat_type,
- cache_resource->ft_type);
+ (priv->sh->ctx, cache_resource->reformat_type,
+ cache_resource->ft_type, ns, cache_resource->flags,
+ cache_resource->size,
+ (cache_resource->size ? cache_resource->buf : NULL));
if (!cache_resource->verbs_action) {
rte_free(cache_resource);
return rte_flow_error_set(error, ENOMEM,
@@ -1442,6 +1451,10 @@ struct field_modify_info modify_tcp[] = {
struct mlx5_priv *priv = dev->data->dev_private;
struct mlx5_flow_dv_modify_hdr_resource *cache_resource;
+ struct mlx5dv_dr_ns *ns =
+ resource->ft_type == MLX5DV_FLOW_TABLE_TYPE_NIC_TX ?
+ priv->tx_ns : priv->rx_ns;
+
/* Lookup a matching resource from cache. */
LIST_FOREACH(cache_resource, &priv->modify_cmds, next) {
if (resource->ft_type == cache_resource->ft_type &&
@@ -1467,11 +1480,11 @@ struct field_modify_info modify_tcp[] = {
*cache_resource = *resource;
cache_resource->verbs_action =
mlx5_glue->dv_create_flow_action_modify_header
- (priv->sh->ctx,
+ (priv->sh->ctx, cache_resource->ft_type,
+ ns, 0,
cache_resource->actions_num *
sizeof(cache_resource->actions[0]),
- (uint64_t *)cache_resource->actions,
- cache_resource->ft_type);
+ (uint64_t *)cache_resource->actions);
if (!cache_resource->verbs_action) {
rte_free(cache_resource);
return rte_flow_error_set(error, ENOMEM,
@@ -1596,11 +1609,13 @@ struct field_modify_info modify_tcp[] = {
struct mlx5_priv *priv = dev->data->dev_private;
uint32_t priority_max = priv->config.flow_prio - 1;
+#ifdef HAVE_MLX5DV_DR
if (attributes->group)
return rte_flow_error_set(error, ENOTSUP,
RTE_FLOW_ERROR_TYPE_ATTR_GROUP,
NULL,
"groups is not supported");
+#endif
if (attributes->priority != MLX5_FLOW_PRIO_RSVD &&
attributes->priority >= priority_max)
return rte_flow_error_set(error, ENOTSUP,
@@ -2173,11 +2188,13 @@ struct field_modify_info modify_tcp[] = {
* Flow pattern to translate.
* @param[in] inner
* Item is inner pattern.
+ * @param[in] group
+ * The group to insert the rule.
*/
static void
flow_dv_translate_item_ipv4(void *matcher, void *key,
const struct rte_flow_item *item,
- int inner)
+ int inner, uint32_t group)
{
const struct rte_flow_item_ipv4 *ipv4_m = item->mask;
const struct rte_flow_item_ipv4 *ipv4_v = item->spec;
@@ -2204,7 +2221,10 @@ struct field_modify_info modify_tcp[] = {
outer_headers);
headers_v = MLX5_ADDR_OF(fte_match_param, key, outer_headers);
}
- MLX5_SET(fte_match_set_lyr_2_4, headers_m, ip_version, 0xf);
+ if (group == 0)
+ MLX5_SET(fte_match_set_lyr_2_4, headers_m, ip_version, 0xf);
+ else
+ MLX5_SET(fte_match_set_lyr_2_4, headers_m, ip_version, 0x4);
MLX5_SET(fte_match_set_lyr_2_4, headers_v, ip_version, 4);
if (!ipv4_v)
return;
@@ -2246,11 +2266,13 @@ struct field_modify_info modify_tcp[] = {
* Flow pattern to translate.
* @param[in] inner
* Item is inner pattern.
+ * @param[in] group
+ * The group to insert the rule.
*/
static void
flow_dv_translate_item_ipv6(void *matcher, void *key,
const struct rte_flow_item *item,
- int inner)
+ int inner, uint32_t group)
{
const struct rte_flow_item_ipv6 *ipv6_m = item->mask;
const struct rte_flow_item_ipv6 *ipv6_v = item->spec;
@@ -2287,7 +2309,10 @@ struct field_modify_info modify_tcp[] = {
outer_headers);
headers_v = MLX5_ADDR_OF(fte_match_param, key, outer_headers);
}
- MLX5_SET(fte_match_set_lyr_2_4, headers_m, ip_version, 0xf);
+ if (group == 0)
+ MLX5_SET(fte_match_set_lyr_2_4, headers_m, ip_version, 0xf);
+ else
+ MLX5_SET(fte_match_set_lyr_2_4, headers_m, ip_version, 0x6);
MLX5_SET(fte_match_set_lyr_2_4, headers_v, ip_version, 6);
if (!ipv6_v)
return;
@@ -2727,7 +2752,11 @@ struct field_modify_info modify_tcp[] = {
match_criteria_enable |=
(!HEADER_IS_ZERO(match_criteria, misc_parameters_2)) <<
MLX5_MATCH_CRITERIA_ENABLE_MISC2_BIT;
-
+#ifdef HAVE_MLX5DV_DR
+ match_criteria_enable |=
+ (!HEADER_IS_ZERO(match_criteria, misc_parameters_3)) <<
+ MLX5_MATCH_CRITERIA_ENABLE_MISC3_BIT;
+#endif
return match_criteria_enable;
}
@@ -2758,12 +2787,14 @@ struct field_modify_info modify_tcp[] = {
.type = IBV_FLOW_ATTR_NORMAL,
.match_mask = (void *)&matcher->mask,
};
+ struct mlx5_flow_tbl_resource *tbl = NULL;
/* Lookup from cache. */
LIST_FOREACH(cache_matcher, &priv->matchers, next) {
if (matcher->crc == cache_matcher->crc &&
matcher->priority == cache_matcher->priority &&
matcher->egress == cache_matcher->egress &&
+ matcher->group == cache_matcher->group &&
!memcmp((const void *)matcher->mask.buf,
(const void *)cache_matcher->mask.buf,
cache_matcher->mask.size)) {
@@ -2778,6 +2809,27 @@ struct field_modify_info modify_tcp[] = {
return 0;
}
}
+#ifdef HAVE_MLX5DV_DR
+ if (matcher->egress) {
+ tbl = &priv->tx_tbl[matcher->group];
+ if (!tbl->obj)
+ tbl->obj = mlx5_glue->dr_create_flow_tbl
+ (priv->tx_ns,
+ matcher->group * MLX5_GROUP_FACTOR);
+ } else {
+ tbl = &priv->rx_tbl[matcher->group];
+ if (!tbl->obj)
+ tbl->obj = mlx5_glue->dr_create_flow_tbl
+ (priv->rx_ns,
+ matcher->group * MLX5_GROUP_FACTOR);
+ }
+ if (!tbl->obj)
+ return rte_flow_error_set(error, ENOMEM,
+ RTE_FLOW_ERROR_TYPE_UNSPECIFIED,
+ NULL, "cannot create table");
+
+ rte_atomic32_inc(&tbl->refcnt);
+#endif
/* Register new matcher. */
cache_matcher = rte_calloc(__func__, 1, sizeof(*cache_matcher), 0);
if (!cache_matcher)
@@ -2791,9 +2843,16 @@ struct field_modify_info modify_tcp[] = {
if (matcher->egress)
dv_attr.flags |= IBV_FLOW_ATTR_FLAGS_EGRESS;
cache_matcher->matcher_object =
- mlx5_glue->dv_create_flow_matcher(priv->sh->ctx, &dv_attr);
+ mlx5_glue->dv_create_flow_matcher(priv->sh->ctx, &dv_attr,
+ tbl->obj);
if (!cache_matcher->matcher_object) {
rte_free(cache_matcher);
+#ifdef HAVE_MLX5DV_DR
+ if (rte_atomic32_dec_and_test(&tbl->refcnt)) {
+ mlx5_glue->dr_destroy_flow_tbl(tbl->obj);
+ tbl->obj = NULL;
+ }
+#endif
return rte_flow_error_set(error, ENOMEM,
RTE_FLOW_ERROR_TYPE_UNSPECIFIED,
NULL, "cannot create matcher");
@@ -2805,6 +2864,7 @@ struct field_modify_info modify_tcp[] = {
cache_matcher->priority,
cache_matcher->egress ? "tx" : "rx", (void *)cache_matcher,
rte_atomic32_read(&cache_matcher->refcnt));
+ rte_atomic32_inc(&tbl->refcnt);
return 0;
}
@@ -3226,7 +3286,7 @@ struct field_modify_info modify_tcp[] = {
break;
case RTE_FLOW_ITEM_TYPE_IPV4:
flow_dv_translate_item_ipv4(match_mask, match_value,
- items, tunnel);
+ items, tunnel, attr->group);
matcher.priority = MLX5_PRIORITY_MAP_L3;
dev_flow->dv.hash_fields |=
mlx5_flow_hashfields_adjust
@@ -3238,7 +3298,7 @@ struct field_modify_info modify_tcp[] = {
break;
case RTE_FLOW_ITEM_TYPE_IPV6:
flow_dv_translate_item_ipv6(match_mask, match_value,
- items, tunnel);
+ items, tunnel, attr->group);
matcher.priority = MLX5_PRIORITY_MAP_L3;
dev_flow->dv.hash_fields |=
mlx5_flow_hashfields_adjust
@@ -3316,6 +3376,7 @@ struct field_modify_info modify_tcp[] = {
matcher.priority = mlx5_flow_adjust_priority(dev, priority,
matcher.priority);
matcher.egress = attr->egress;
+ matcher.group = attr->group;
if (flow_dv_matcher_register(dev, &matcher, dev_flow, error))
return -rte_errno;
return 0;
@@ -3431,6 +3492,8 @@ struct field_modify_info modify_tcp[] = {
struct mlx5_flow *flow)
{
struct mlx5_flow_dv_matcher *matcher = flow->dv.matcher;
+ struct mlx5_priv *priv = dev->data->dev_private;
+ struct mlx5_flow_tbl_resource *tbl;
assert(matcher->matcher_object);
DRV_LOG(DEBUG, "port %u matcher %p: refcnt %d--",
@@ -3440,6 +3503,14 @@ struct field_modify_info modify_tcp[] = {
claim_zero(mlx5_glue->dv_destroy_flow_matcher
(matcher->matcher_object));
LIST_REMOVE(matcher, next);
+ if (matcher->egress)
+ tbl = &priv->tx_tbl[matcher->group];
+ else
+ tbl = &priv->rx_tbl[matcher->group];
+ if (rte_atomic32_dec_and_test(&tbl->refcnt)) {
+ mlx5_glue->dr_destroy_flow_tbl(tbl->obj);
+ tbl->obj = NULL;
+ }
rte_free(matcher);
DRV_LOG(DEBUG, "port %u matcher %p: removed",
dev->data->port_id, (void *)matcher);
@@ -3529,7 +3600,7 @@ struct field_modify_info modify_tcp[] = {
LIST_FOREACH(dev_flow, &flow->dev_flows, next) {
dv = &dev_flow->dv;
if (dv->flow) {
- claim_zero(mlx5_glue->destroy_flow(dv->flow));
+ claim_zero(mlx5_glue->dv_destroy_flow(dv->flow));
dv->flow = NULL;
}
if (dv->hrxq) {
diff --git a/drivers/net/mlx5/mlx5_glue.c b/drivers/net/mlx5/mlx5_glue.c
index 4b5aade..b0b144c 100644
--- a/drivers/net/mlx5/mlx5_glue.c
+++ b/drivers/net/mlx5/mlx5_glue.c
@@ -178,6 +178,9 @@
mlx5_glue_destroy_flow_action(void *action)
{
#ifdef HAVE_IBV_FLOW_DV_SUPPORT
+#ifdef HAVE_MLX5DV_DR
+ return mlx5dv_dr_destroy_action(action);
+#else
struct mlx5dv_flow_action_attr *attr = action;
int res = 0;
switch (attr->type) {
@@ -189,6 +192,7 @@
}
free(action);
return res;
+#endif
#else
(void)action;
return ENOTSUP;
@@ -365,6 +369,53 @@
return ibv_cq_ex_to_cq(cq);
}
+static void *
+mlx5_glue_dr_create_flow_tbl(void *ns, uint32_t level)
+{
+#ifdef HAVE_MLX5DV_DR
+ return mlx5dv_dr_create_ft(ns, level);
+#else
+ (void)ns;
+ (void)level;
+ return NULL;
+#endif
+}
+
+static int
+mlx5_glue_dr_destroy_flow_tbl(void *tbl)
+{
+#ifdef HAVE_MLX5DV_DR
+ return mlx5dv_dr_destroy_ft(tbl);
+#else
+ (void)tbl;
+ return 0;
+#endif
+}
+
+static void *
+mlx5_glue_dr_create_ns(struct ibv_context *ctx,
+ enum mlx5dv_dr_ns_domain domain)
+{
+#ifdef HAVE_MLX5DV_DR
+ return mlx5dv_dr_create_ns(ctx, domain);
+#else
+ (void)ctx;
+ (void)domain;
+ return NULL;
+#endif
+}
+
+static int
+mlx5_glue_dr_destroy_ns(void *ns)
+{
+#ifdef HAVE_MLX5DV_DR
+ return mlx5dv_dr_destroy_ns(ns);
+#else
+ (void)ns;
+ return 0;
+#endif
+}
+
static struct ibv_cq_ex *
mlx5_glue_dv_create_cq(struct ibv_context *context,
struct ibv_cq_init_attr_ex *cq_attr,
@@ -423,26 +474,40 @@
#endif
}
-static struct mlx5dv_flow_matcher *
+static void *
mlx5_glue_dv_create_flow_matcher(struct ibv_context *context,
- struct mlx5dv_flow_matcher_attr *matcher_attr)
+ struct mlx5dv_flow_matcher_attr *matcher_attr,
+ void *tbl)
{
#ifdef HAVE_IBV_FLOW_DV_SUPPORT
+#ifdef HAVE_MLX5DV_DR
+ (void)context;
+ return mlx5dv_dr_create_matcher(tbl, matcher_attr->priority,
+ matcher_attr->match_criteria_enable,
+ matcher_attr->match_mask);
+#else
+ (void)tbl;
return mlx5dv_create_flow_matcher(context, matcher_attr);
+#endif
#else
(void)context;
(void)matcher_attr;
+ (void)tbl;
return NULL;
#endif
}
-static struct ibv_flow *
-mlx5_glue_dv_create_flow(struct mlx5dv_flow_matcher *matcher,
- struct mlx5dv_flow_match_parameters *match_value,
+static void *
+mlx5_glue_dv_create_flow(void *matcher,
+ void *match_value,
size_t num_actions,
void *actions[])
{
#ifdef HAVE_IBV_FLOW_DV_SUPPORT
+#ifdef HAVE_MLX5DV_DR
+ return mlx5dv_dr_create_rule(matcher, match_value, num_actions,
+ (struct mlx5dv_dr_action **)actions);
+#else
struct mlx5dv_flow_action_attr actions_attr[8];
if (num_actions > 8)
@@ -452,6 +517,7 @@
*((struct mlx5dv_flow_action_attr *)(actions[i]));
return mlx5dv_create_flow(matcher, match_value,
num_actions, actions_attr);
+#endif
#else
(void)matcher;
(void)match_value;
@@ -461,21 +527,13 @@
#endif
}
-static int
-mlx5_glue_dv_destroy_flow_matcher(struct mlx5dv_flow_matcher *matcher)
-{
-#ifdef HAVE_IBV_FLOW_DV_SUPPORT
- return mlx5dv_destroy_flow_matcher(matcher);
-#else
- (void)matcher;
- return 0;
-#endif
-}
-
static void *
mlx5_glue_dv_create_flow_action_counter(void *counter_obj, uint32_t offset)
{
#ifdef HAVE_IBV_FLOW_DV_SUPPORT
+#ifdef HAVE_MLX5DV_DR
+ return mlx5dv_dr_create_action_devx_counter(counter_obj, offset);
+#else
struct mlx5dv_flow_action_attr *action;
(void)offset;
@@ -485,6 +543,7 @@
action->type = MLX5DV_FLOW_ACTION_COUNTERS_DEVX;
action->obj = counter_obj;
return action;
+#endif
#else
(void)counter_obj;
(void)offset;
@@ -496,6 +555,9 @@
mlx5_glue_dv_create_flow_action_dest_ibv_qp(void *qp)
{
#ifdef HAVE_IBV_FLOW_DV_SUPPORT
+#ifdef HAVE_MLX5DV_DR
+ return mlx5dv_dr_create_action_dest_ibv_qp(qp);
+#else
struct mlx5dv_flow_action_attr *action;
action = malloc(sizeof(*action));
@@ -504,6 +566,7 @@
action->type = MLX5DV_FLOW_ACTION_DEST_IBV_QP;
action->obj = qp;
return action;
+#endif
#else
(void)qp;
return NULL;
@@ -513,13 +576,22 @@
static void *
mlx5_glue_dv_create_flow_action_modify_header
(struct ibv_context *ctx,
+ enum mlx5dv_flow_table_type ft_type,
+ void *ns, uint64_t flags,
size_t actions_sz,
- uint64_t actions[],
- enum mlx5dv_flow_table_type ft_type)
+ uint64_t actions[])
{
#ifdef HAVE_IBV_FLOW_DV_SUPPORT
+#ifdef HAVE_MLX5DV_DR
+ (void)ctx;
+ (void)ft_type;
+ return mlx5dv_dr_create_action_modify_header(ns, flags, actions_sz,
+ actions);
+#else
struct mlx5dv_flow_action_attr *action;
+ (void)ns;
+ (void)flags;
action = malloc(sizeof(*action));
if (!action)
return NULL;
@@ -527,11 +599,14 @@
action->action = mlx5dv_create_flow_action_modify_header
(ctx, actions_sz, actions, ft_type);
return action;
+#endif
#else
(void)ctx;
+ (void)ft_type;
+ (void)ns;
+ (void)flags;
(void)actions_sz;
(void)actions;
- (void)ft_type;
return NULL;
#endif
}
@@ -539,12 +614,20 @@
static void *
mlx5_glue_dv_create_flow_action_packet_reformat
(struct ibv_context *ctx,
- size_t data_sz,
- void *data,
enum mlx5dv_flow_action_packet_reformat_type reformat_type,
- enum mlx5dv_flow_table_type ft_type)
+ enum mlx5dv_flow_table_type ft_type, struct mlx5dv_dr_ns *ns,
+ uint32_t flags, size_t data_sz, void *data)
{
#ifdef HAVE_IBV_FLOW_DV_SUPPORT
+#ifdef HAVE_MLX5DV_DR
+ (void)ctx;
+ (void)ft_type;
+ return mlx5dv_dr_create_action_packet_reformat(ns, flags,
+ reformat_type, data_sz,
+ data);
+#else
+ (void)ns;
+ (void)flags;
struct mlx5dv_flow_action_attr *action;
action = malloc(sizeof(*action));
@@ -554,12 +637,15 @@
action->action = mlx5dv_create_flow_action_packet_reformat
(ctx, data_sz, data, reformat_type, ft_type);
return action;
+#endif
#else
(void)ctx;
- (void)data_sz;
- (void)data;
(void)reformat_type;
(void)ft_type;
+ (void)ns;
+ (void)flags;
+ (void)data_sz;
+ (void)data;
return NULL;
#endif
}
@@ -568,6 +654,9 @@
mlx5_glue_dv_create_flow_action_tag(uint32_t tag)
{
#ifdef HAVE_IBV_FLOW_DV_SUPPORT
+#ifdef HAVE_MLX5DV_DR
+ return mlx5dv_dr_create_action_tag(tag);
+#else
struct mlx5dv_flow_action_attr *action;
action = malloc(sizeof(*action));
if (!action)
@@ -576,10 +665,36 @@
action->tag_value = tag;
return action;
#endif
+#endif
(void)tag;
return NULL;
}
+static int
+mlx5_glue_dv_destroy_flow(void *flow_id)
+{
+#ifdef HAVE_MLX5DV_DR
+ return mlx5dv_dr_destroy_rule(flow_id);
+#else
+ return ibv_destroy_flow(flow_id);
+#endif
+}
+
+static int
+mlx5_glue_dv_destroy_flow_matcher(void *matcher)
+{
+#ifdef HAVE_IBV_FLOW_DV_SUPPORT
+#ifdef HAVE_MLX5DV_DR
+ return mlx5dv_dr_destroy_matcher(matcher);
+#else
+ return mlx5dv_destroy_flow_matcher(matcher);
+#endif
+#else
+ (void)matcher;
+ return 0;
+#endif
+}
+
static struct ibv_context *
mlx5_glue_dv_open_device(struct ibv_device *device)
{
@@ -718,6 +833,10 @@
.get_async_event = mlx5_glue_get_async_event,
.port_state_str = mlx5_glue_port_state_str,
.cq_ex_to_cq = mlx5_glue_cq_ex_to_cq,
+ .dr_create_flow_tbl = mlx5_glue_dr_create_flow_tbl,
+ .dr_destroy_flow_tbl = mlx5_glue_dr_destroy_flow_tbl,
+ .dr_create_ns = mlx5_glue_dr_create_ns,
+ .dr_destroy_ns = mlx5_glue_dr_destroy_ns,
.dv_create_cq = mlx5_glue_dv_create_cq,
.dv_create_wq = mlx5_glue_dv_create_wq,
.dv_query_device = mlx5_glue_dv_query_device,
@@ -725,7 +844,6 @@
.dv_init_obj = mlx5_glue_dv_init_obj,
.dv_create_qp = mlx5_glue_dv_create_qp,
.dv_create_flow_matcher = mlx5_glue_dv_create_flow_matcher,
- .dv_destroy_flow_matcher = mlx5_glue_dv_destroy_flow_matcher,
.dv_create_flow = mlx5_glue_dv_create_flow,
.dv_create_flow_action_counter =
mlx5_glue_dv_create_flow_action_counter,
@@ -736,6 +854,8 @@
.dv_create_flow_action_packet_reformat =
mlx5_glue_dv_create_flow_action_packet_reformat,
.dv_create_flow_action_tag = mlx5_glue_dv_create_flow_action_tag,
+ .dv_destroy_flow = mlx5_glue_dv_destroy_flow,
+ .dv_destroy_flow_matcher = mlx5_glue_dv_destroy_flow_matcher,
.dv_open_device = mlx5_glue_dv_open_device,
.devx_obj_create = mlx5_glue_devx_obj_create,
.devx_obj_destroy = mlx5_glue_devx_obj_destroy,
diff --git a/drivers/net/mlx5/mlx5_glue.h b/drivers/net/mlx5/mlx5_glue.h
index 32487ea..eb29ffa 100644
--- a/drivers/net/mlx5/mlx5_glue.h
+++ b/drivers/net/mlx5/mlx5_glue.h
@@ -63,6 +63,11 @@
struct mlx5dv_devx_obj;
#endif
+#ifndef HAVE_MLX5DV_DR
+struct mlx5dv_dr_ns;
+enum mlx5dv_dr_ns_domain { unused, };
+#endif
+
/* LIB_GLUE_VERSION must be updated every time this structure is modified. */
struct mlx5_glue {
const char *version;
@@ -140,6 +145,11 @@ struct mlx5_glue {
struct ibv_async_event *event);
const char *(*port_state_str)(enum ibv_port_state port_state);
struct ibv_cq *(*cq_ex_to_cq)(struct ibv_cq_ex *cq);
+ void *(*dr_create_flow_tbl)(void *ns, uint32_t level);
+ int (*dr_destroy_flow_tbl)(void *tbl);
+ void *(*dr_create_ns)(struct ibv_context *ctx,
+ enum mlx5dv_dr_ns_domain domain);
+ int (*dr_destroy_ns)(void *ns);
struct ibv_cq_ex *(*dv_create_cq)
(struct ibv_context *context,
struct ibv_cq_init_attr_ex *cq_attr,
@@ -158,23 +168,26 @@ struct mlx5_glue {
(struct ibv_context *context,
struct ibv_qp_init_attr_ex *qp_init_attr_ex,
struct mlx5dv_qp_init_attr *dv_qp_init_attr);
- struct mlx5dv_flow_matcher *(*dv_create_flow_matcher)
+ void *(*dv_create_flow_matcher)
(struct ibv_context *context,
- struct mlx5dv_flow_matcher_attr *matcher_attr);
- int (*dv_destroy_flow_matcher)(struct mlx5dv_flow_matcher *matcher);
- struct ibv_flow *(*dv_create_flow)(struct mlx5dv_flow_matcher *matcher,
- struct mlx5dv_flow_match_parameters *match_value,
+ struct mlx5dv_flow_matcher_attr *matcher_attr,
+ void *tbl);
+ void *(*dv_create_flow)(void *matcher, void *match_value,
size_t num_actions, void *actions[]);
void *(*dv_create_flow_action_counter)(void *obj, uint32_t offset);
void *(*dv_create_flow_action_dest_ibv_qp)(void *qp);
void *(*dv_create_flow_action_modify_header)
- (struct ibv_context *ctx, size_t actions_sz, uint64_t actions[],
- enum mlx5dv_flow_table_type ft_type);
+ (struct ibv_context *ctx, enum mlx5dv_flow_table_type ft_type,
+ void *ns, uint64_t flags, size_t actions_sz,
+ uint64_t actions[]);
void *(*dv_create_flow_action_packet_reformat)
- (struct ibv_context *ctx, size_t data_sz, void *data,
+ (struct ibv_context *ctx,
enum mlx5dv_flow_action_packet_reformat_type reformat_type,
- enum mlx5dv_flow_table_type ft_type);
+ enum mlx5dv_flow_table_type ft_type, struct mlx5dv_dr_ns *ns,
+ uint32_t flags, size_t data_sz, void *data);
void *(*dv_create_flow_action_tag)(uint32_t tag);
+ int (*dv_destroy_flow)(void *flow);
+ int (*dv_destroy_flow_matcher)(void *matcher);
struct ibv_context *(*dv_open_device)(struct ibv_device *device);
struct mlx5dv_devx_obj *(*devx_obj_create)
(struct ibv_context *ctx,
diff --git a/drivers/net/mlx5/mlx5_prm.h b/drivers/net/mlx5/mlx5_prm.h
index da1219e..b15266f 100644
--- a/drivers/net/mlx5/mlx5_prm.h
+++ b/drivers/net/mlx5/mlx5_prm.h
@@ -492,20 +492,40 @@ struct mlx5_ifc_fte_match_set_misc2_bits {
u8 reserved_at_1a0[0x60];
};
+struct mlx5_ifc_fte_match_set_misc3_bits {
+ u8 inner_tcp_seq_num[0x20];
+ u8 outer_tcp_seq_num[0x20];
+ u8 inner_tcp_ack_num[0x20];
+ u8 outer_tcp_ack_num[0x20];
+ u8 reserved_at_auto1[0x8];
+ u8 outer_vxlan_gpe_vni[0x18];
+ u8 outer_vxlan_gpe_next_protocol[0x8];
+ u8 outer_vxlan_gpe_flags[0x8];
+ u8 reserved_at_a8[0x10];
+ u8 icmp_header_data[0x20];
+ u8 icmpv6_header_data[0x20];
+ u8 icmp_type[0x8];
+ u8 icmp_code[0x8];
+ u8 icmpv6_type[0x8];
+ u8 icmpv6_code[0x8];
+ u8 reserved_at_1a0[0xe0];
+};
+
/* Flow matcher. */
struct mlx5_ifc_fte_match_param_bits {
struct mlx5_ifc_fte_match_set_lyr_2_4_bits outer_headers;
struct mlx5_ifc_fte_match_set_misc_bits misc_parameters;
struct mlx5_ifc_fte_match_set_lyr_2_4_bits inner_headers;
struct mlx5_ifc_fte_match_set_misc2_bits misc_parameters_2;
- u8 reserved_at_800[0x800];
+ struct mlx5_ifc_fte_match_set_misc3_bits misc_parameters_3;
};
enum {
MLX5_MATCH_CRITERIA_ENABLE_OUTER_BIT,
MLX5_MATCH_CRITERIA_ENABLE_MISC_BIT,
MLX5_MATCH_CRITERIA_ENABLE_INNER_BIT,
- MLX5_MATCH_CRITERIA_ENABLE_MISC2_BIT
+ MLX5_MATCH_CRITERIA_ENABLE_MISC2_BIT,
+ MLX5_MATCH_CRITERIA_ENABLE_MISC3_BIT
};
enum {
--
1.8.3.1
^ permalink raw reply [flat|nested] 52+ messages in thread
* [dpdk-dev] [PATCH v3 2/3] net/mlx5: add Direct Rules API
2019-04-03 13:21 ` [dpdk-dev] [PATCH v3 2/3] net/mlx5: add Direct Rules API Ori Kam
@ 2019-04-03 13:21 ` Ori Kam
0 siblings, 0 replies; 52+ messages in thread
From: Ori Kam @ 2019-04-03 13:21 UTC (permalink / raw)
To: Matan Azrad, Yongseok Koh, Shahaf Shuler; +Cc: dev, Ori Kam, Slava Ovsiienko
Adds calls to the Direct Rules API inside the glue functions.
Due to difference in parameters between the Direct Rules and Direct
Verbs some of the glue functions API was updated.
Signed-off-by: Ori Kam <orika@mellanox.com>
Acked-by: Shahaf Shuler <shahafs@mellanox.com>
---
drivers/net/mlx5/Makefile | 5 ++
drivers/net/mlx5/mlx5.c | 16 ++++
drivers/net/mlx5/mlx5.h | 15 ++++
drivers/net/mlx5/mlx5_flow.c | 1 +
drivers/net/mlx5/mlx5_flow.h | 6 +-
drivers/net/mlx5/mlx5_flow_dv.c | 103 ++++++++++++++++++++----
drivers/net/mlx5/mlx5_glue.c | 170 ++++++++++++++++++++++++++++++++++------
drivers/net/mlx5/mlx5_glue.h | 31 +++++---
drivers/net/mlx5/mlx5_prm.h | 24 +++++-
9 files changed, 318 insertions(+), 53 deletions(-)
diff --git a/drivers/net/mlx5/Makefile b/drivers/net/mlx5/Makefile
index 0d20f0f..93bc869 100644
--- a/drivers/net/mlx5/Makefile
+++ b/drivers/net/mlx5/Makefile
@@ -156,6 +156,11 @@ mlx5_autoconf.h.new: $(RTE_SDK)/buildtools/auto-config-h.sh
func mlx5dv_create_flow_action_packet_reformat \
$(AUTOCONF_OUTPUT)
$Q sh -- '$<' '$@' \
+ HAVE_MLX5DV_DR \
+ infiniband/mlx5dv.h \
+ enum MLX5DV_DR_NS_TYPE_TERMINATING \
+ $(AUTOCONF_OUTPUT)
+ $Q sh -- '$<' '$@' \
HAVE_IBV_DEVX_OBJ \
infiniband/mlx5dv.h \
func mlx5dv_devx_obj_create \
diff --git a/drivers/net/mlx5/mlx5.c b/drivers/net/mlx5/mlx5.c
index 4044505..65aa9cf 100644
--- a/drivers/net/mlx5/mlx5.c
+++ b/drivers/net/mlx5/mlx5.c
@@ -1483,6 +1483,22 @@ struct mlx5_dev_spawn_data {
priv->tcf_context = NULL;
}
}
+#ifdef HAVE_MLX5DV_DR
+ priv->rx_ns = mlx5dv_dr_create_ns
+ (sh->ctx, MLX5DV_DR_NS_DOMAIN_INGRESS_BYPASS);
+ if (priv->rx_ns == NULL) {
+ DRV_LOG(ERR, "mlx5dv_dr_create_ns failed");
+ err = errno;
+ goto error;
+ }
+ priv->tx_ns = mlx5dv_dr_create_ns(sh->ctx,
+ MLX5DV_DR_NS_DOMAIN_EGRESS_BYPASS);
+ if (priv->tx_ns == NULL) {
+ DRV_LOG(ERR, "mlx5dv_dr_create_ns failed");
+ err = errno;
+ goto error;
+ }
+#endif
TAILQ_INIT(&priv->flows);
TAILQ_INIT(&priv->ctrl_flows);
/* Hint libmlx5 to use PMD allocator for data plane resources */
diff --git a/drivers/net/mlx5/mlx5.h b/drivers/net/mlx5/mlx5.h
index 784bf9b..d4963cb 100644
--- a/drivers/net/mlx5/mlx5.h
+++ b/drivers/net/mlx5/mlx5.h
@@ -259,6 +259,15 @@ struct mlx5_ibv_shared {
struct mlx5_ibv_shared_port port[]; /* per device port data array. */
};
+/* Table structure. */
+struct mlx5_flow_tbl_resource {
+ void *obj; /**< Pointer to DR table object. */
+ rte_atomic32_t refcnt; /**< Reference counter. */
+};
+
+#define MLX5_MAX_TABLES 1024
+#define MLX5_GROUP_FACTOR 1
+
struct mlx5_priv {
LIST_ENTRY(mlx5_priv) mem_event_cb;
/**< Called by memory event callback. */
@@ -326,6 +335,12 @@ struct mlx5_priv {
/* UAR same-page access control required in 32bit implementations. */
#endif
struct mlx5_flow_tcf_context *tcf_context; /* TC flower context. */
+ void *rx_ns; /* RX Direct Rules name space handle. */
+ struct mlx5_flow_tbl_resource rx_tbl[MLX5_MAX_TABLES];
+ /* RX Direct Rules tables. */
+ void *tx_ns; /* TX Direct Rules name space handle. */
+ struct mlx5_flow_tbl_resource tx_tbl[MLX5_MAX_TABLES];
+ /* TX Direct Rules tables/ */
};
#define PORT_ID(priv) ((priv)->dev_data->port_id)
diff --git a/drivers/net/mlx5/mlx5_flow.c b/drivers/net/mlx5/mlx5_flow.c
index bc6a7c1..9dc492a 100644
--- a/drivers/net/mlx5/mlx5_flow.c
+++ b/drivers/net/mlx5/mlx5_flow.c
@@ -2084,6 +2084,7 @@ uint32_t mlx5_flow_adjust_priority(struct rte_eth_dev *dev, int32_t priority,
flow_size += RTE_ALIGN_CEIL(sizeof(uint16_t), sizeof(void *));
flow = rte_calloc(__func__, 1, flow_size, 0);
flow->drv_type = flow_get_drv_type(dev, attr);
+ flow->ingress = attr->ingress;
assert(flow->drv_type > MLX5_FLOW_TYPE_MIN &&
flow->drv_type < MLX5_FLOW_TYPE_MAX);
flow->queue = (void *)(flow + 1);
diff --git a/drivers/net/mlx5/mlx5_flow.h b/drivers/net/mlx5/mlx5_flow.h
index 4f69ae2..8ba37a0 100644
--- a/drivers/net/mlx5/mlx5_flow.h
+++ b/drivers/net/mlx5/mlx5_flow.h
@@ -204,6 +204,7 @@ struct mlx5_flow_dv_matcher {
uint16_t crc; /**< CRC of key. */
uint16_t priority; /**< Priority of matcher. */
uint8_t egress; /**< Egress matcher. */
+ uint32_t group; /**< The matcher group. */
struct mlx5_flow_dv_match_params mask; /**< Matcher mask. */
};
@@ -220,6 +221,7 @@ struct mlx5_flow_dv_encap_decap_resource {
size_t size;
uint8_t reformat_type;
uint8_t ft_type;
+ uint64_t flags; /**< Flags for RDMA API. */
};
/* Tag resource structure. */
@@ -348,7 +350,7 @@ struct mlx5_flow_counter {
/* Flow structure. */
struct rte_flow {
TAILQ_ENTRY(rte_flow) next; /**< Pointer to the next flow structure. */
- enum mlx5_flow_drv_type drv_type; /**< Drvier type. */
+ enum mlx5_flow_drv_type drv_type; /**< Driver type. */
struct mlx5_flow_counter *counter; /**< Holds flow counter. */
struct mlx5_flow_dv_tag_resource *tag_resource;
/**< pointer to the tag action. */
@@ -360,6 +362,8 @@ struct rte_flow {
uint64_t actions;
/**< Bit-fields of detected actions, see MLX5_FLOW_ACTION_*. */
struct mlx5_fdir *fdir; /**< Pointer to associated FDIR if any. */
+ uint8_t ingress; /**< 1 if the flow is ingress. */
+ uint32_t group; /**< The group index. */
};
typedef int (*mlx5_flow_validate_t)(struct rte_eth_dev *dev,
diff --git a/drivers/net/mlx5/mlx5_flow_dv.c b/drivers/net/mlx5/mlx5_flow_dv.c
index d9e2ac9..dd5b541 100644
--- a/drivers/net/mlx5/mlx5_flow_dv.c
+++ b/drivers/net/mlx5/mlx5_flow_dv.c
@@ -809,11 +809,20 @@ struct field_modify_info modify_tcp[] = {
{
struct mlx5_priv *priv = dev->data->dev_private;
struct mlx5_flow_dv_encap_decap_resource *cache_resource;
+ struct rte_flow *flow = dev_flow->flow;
+ struct mlx5dv_dr_ns *ns;
+
+ resource->flags = flow->group ? 0 : 1;
+ if (flow->ingress)
+ ns = priv->rx_ns;
+ else
+ ns = priv->tx_ns;
/* Lookup a matching resource from cache. */
LIST_FOREACH(cache_resource, &priv->encaps_decaps, next) {
if (resource->reformat_type == cache_resource->reformat_type &&
resource->ft_type == cache_resource->ft_type &&
+ resource->flags == cache_resource->flags &&
resource->size == cache_resource->size &&
!memcmp((const void *)resource->buf,
(const void *)cache_resource->buf,
@@ -835,10 +844,10 @@ struct field_modify_info modify_tcp[] = {
*cache_resource = *resource;
cache_resource->verbs_action =
mlx5_glue->dv_create_flow_action_packet_reformat
- (priv->sh->ctx, cache_resource->size,
- (cache_resource->size ? cache_resource->buf : NULL),
- cache_resource->reformat_type,
- cache_resource->ft_type);
+ (priv->sh->ctx, cache_resource->reformat_type,
+ cache_resource->ft_type, ns, cache_resource->flags,
+ cache_resource->size,
+ (cache_resource->size ? cache_resource->buf : NULL));
if (!cache_resource->verbs_action) {
rte_free(cache_resource);
return rte_flow_error_set(error, ENOMEM,
@@ -1442,6 +1451,10 @@ struct field_modify_info modify_tcp[] = {
struct mlx5_priv *priv = dev->data->dev_private;
struct mlx5_flow_dv_modify_hdr_resource *cache_resource;
+ struct mlx5dv_dr_ns *ns =
+ resource->ft_type == MLX5DV_FLOW_TABLE_TYPE_NIC_TX ?
+ priv->tx_ns : priv->rx_ns;
+
/* Lookup a matching resource from cache. */
LIST_FOREACH(cache_resource, &priv->modify_cmds, next) {
if (resource->ft_type == cache_resource->ft_type &&
@@ -1467,11 +1480,11 @@ struct field_modify_info modify_tcp[] = {
*cache_resource = *resource;
cache_resource->verbs_action =
mlx5_glue->dv_create_flow_action_modify_header
- (priv->sh->ctx,
+ (priv->sh->ctx, cache_resource->ft_type,
+ ns, 0,
cache_resource->actions_num *
sizeof(cache_resource->actions[0]),
- (uint64_t *)cache_resource->actions,
- cache_resource->ft_type);
+ (uint64_t *)cache_resource->actions);
if (!cache_resource->verbs_action) {
rte_free(cache_resource);
return rte_flow_error_set(error, ENOMEM,
@@ -1596,11 +1609,13 @@ struct field_modify_info modify_tcp[] = {
struct mlx5_priv *priv = dev->data->dev_private;
uint32_t priority_max = priv->config.flow_prio - 1;
+#ifdef HAVE_MLX5DV_DR
if (attributes->group)
return rte_flow_error_set(error, ENOTSUP,
RTE_FLOW_ERROR_TYPE_ATTR_GROUP,
NULL,
"groups is not supported");
+#endif
if (attributes->priority != MLX5_FLOW_PRIO_RSVD &&
attributes->priority >= priority_max)
return rte_flow_error_set(error, ENOTSUP,
@@ -2173,11 +2188,13 @@ struct field_modify_info modify_tcp[] = {
* Flow pattern to translate.
* @param[in] inner
* Item is inner pattern.
+ * @param[in] group
+ * The group to insert the rule.
*/
static void
flow_dv_translate_item_ipv4(void *matcher, void *key,
const struct rte_flow_item *item,
- int inner)
+ int inner, uint32_t group)
{
const struct rte_flow_item_ipv4 *ipv4_m = item->mask;
const struct rte_flow_item_ipv4 *ipv4_v = item->spec;
@@ -2204,7 +2221,10 @@ struct field_modify_info modify_tcp[] = {
outer_headers);
headers_v = MLX5_ADDR_OF(fte_match_param, key, outer_headers);
}
- MLX5_SET(fte_match_set_lyr_2_4, headers_m, ip_version, 0xf);
+ if (group == 0)
+ MLX5_SET(fte_match_set_lyr_2_4, headers_m, ip_version, 0xf);
+ else
+ MLX5_SET(fte_match_set_lyr_2_4, headers_m, ip_version, 0x4);
MLX5_SET(fte_match_set_lyr_2_4, headers_v, ip_version, 4);
if (!ipv4_v)
return;
@@ -2246,11 +2266,13 @@ struct field_modify_info modify_tcp[] = {
* Flow pattern to translate.
* @param[in] inner
* Item is inner pattern.
+ * @param[in] group
+ * The group to insert the rule.
*/
static void
flow_dv_translate_item_ipv6(void *matcher, void *key,
const struct rte_flow_item *item,
- int inner)
+ int inner, uint32_t group)
{
const struct rte_flow_item_ipv6 *ipv6_m = item->mask;
const struct rte_flow_item_ipv6 *ipv6_v = item->spec;
@@ -2287,7 +2309,10 @@ struct field_modify_info modify_tcp[] = {
outer_headers);
headers_v = MLX5_ADDR_OF(fte_match_param, key, outer_headers);
}
- MLX5_SET(fte_match_set_lyr_2_4, headers_m, ip_version, 0xf);
+ if (group == 0)
+ MLX5_SET(fte_match_set_lyr_2_4, headers_m, ip_version, 0xf);
+ else
+ MLX5_SET(fte_match_set_lyr_2_4, headers_m, ip_version, 0x6);
MLX5_SET(fte_match_set_lyr_2_4, headers_v, ip_version, 6);
if (!ipv6_v)
return;
@@ -2727,7 +2752,11 @@ struct field_modify_info modify_tcp[] = {
match_criteria_enable |=
(!HEADER_IS_ZERO(match_criteria, misc_parameters_2)) <<
MLX5_MATCH_CRITERIA_ENABLE_MISC2_BIT;
-
+#ifdef HAVE_MLX5DV_DR
+ match_criteria_enable |=
+ (!HEADER_IS_ZERO(match_criteria, misc_parameters_3)) <<
+ MLX5_MATCH_CRITERIA_ENABLE_MISC3_BIT;
+#endif
return match_criteria_enable;
}
@@ -2758,12 +2787,14 @@ struct field_modify_info modify_tcp[] = {
.type = IBV_FLOW_ATTR_NORMAL,
.match_mask = (void *)&matcher->mask,
};
+ struct mlx5_flow_tbl_resource *tbl = NULL;
/* Lookup from cache. */
LIST_FOREACH(cache_matcher, &priv->matchers, next) {
if (matcher->crc == cache_matcher->crc &&
matcher->priority == cache_matcher->priority &&
matcher->egress == cache_matcher->egress &&
+ matcher->group == cache_matcher->group &&
!memcmp((const void *)matcher->mask.buf,
(const void *)cache_matcher->mask.buf,
cache_matcher->mask.size)) {
@@ -2778,6 +2809,27 @@ struct field_modify_info modify_tcp[] = {
return 0;
}
}
+#ifdef HAVE_MLX5DV_DR
+ if (matcher->egress) {
+ tbl = &priv->tx_tbl[matcher->group];
+ if (!tbl->obj)
+ tbl->obj = mlx5_glue->dr_create_flow_tbl
+ (priv->tx_ns,
+ matcher->group * MLX5_GROUP_FACTOR);
+ } else {
+ tbl = &priv->rx_tbl[matcher->group];
+ if (!tbl->obj)
+ tbl->obj = mlx5_glue->dr_create_flow_tbl
+ (priv->rx_ns,
+ matcher->group * MLX5_GROUP_FACTOR);
+ }
+ if (!tbl->obj)
+ return rte_flow_error_set(error, ENOMEM,
+ RTE_FLOW_ERROR_TYPE_UNSPECIFIED,
+ NULL, "cannot create table");
+
+ rte_atomic32_inc(&tbl->refcnt);
+#endif
/* Register new matcher. */
cache_matcher = rte_calloc(__func__, 1, sizeof(*cache_matcher), 0);
if (!cache_matcher)
@@ -2791,9 +2843,16 @@ struct field_modify_info modify_tcp[] = {
if (matcher->egress)
dv_attr.flags |= IBV_FLOW_ATTR_FLAGS_EGRESS;
cache_matcher->matcher_object =
- mlx5_glue->dv_create_flow_matcher(priv->sh->ctx, &dv_attr);
+ mlx5_glue->dv_create_flow_matcher(priv->sh->ctx, &dv_attr,
+ tbl->obj);
if (!cache_matcher->matcher_object) {
rte_free(cache_matcher);
+#ifdef HAVE_MLX5DV_DR
+ if (rte_atomic32_dec_and_test(&tbl->refcnt)) {
+ mlx5_glue->dr_destroy_flow_tbl(tbl->obj);
+ tbl->obj = NULL;
+ }
+#endif
return rte_flow_error_set(error, ENOMEM,
RTE_FLOW_ERROR_TYPE_UNSPECIFIED,
NULL, "cannot create matcher");
@@ -2805,6 +2864,7 @@ struct field_modify_info modify_tcp[] = {
cache_matcher->priority,
cache_matcher->egress ? "tx" : "rx", (void *)cache_matcher,
rte_atomic32_read(&cache_matcher->refcnt));
+ rte_atomic32_inc(&tbl->refcnt);
return 0;
}
@@ -3226,7 +3286,7 @@ struct field_modify_info modify_tcp[] = {
break;
case RTE_FLOW_ITEM_TYPE_IPV4:
flow_dv_translate_item_ipv4(match_mask, match_value,
- items, tunnel);
+ items, tunnel, attr->group);
matcher.priority = MLX5_PRIORITY_MAP_L3;
dev_flow->dv.hash_fields |=
mlx5_flow_hashfields_adjust
@@ -3238,7 +3298,7 @@ struct field_modify_info modify_tcp[] = {
break;
case RTE_FLOW_ITEM_TYPE_IPV6:
flow_dv_translate_item_ipv6(match_mask, match_value,
- items, tunnel);
+ items, tunnel, attr->group);
matcher.priority = MLX5_PRIORITY_MAP_L3;
dev_flow->dv.hash_fields |=
mlx5_flow_hashfields_adjust
@@ -3316,6 +3376,7 @@ struct field_modify_info modify_tcp[] = {
matcher.priority = mlx5_flow_adjust_priority(dev, priority,
matcher.priority);
matcher.egress = attr->egress;
+ matcher.group = attr->group;
if (flow_dv_matcher_register(dev, &matcher, dev_flow, error))
return -rte_errno;
return 0;
@@ -3431,6 +3492,8 @@ struct field_modify_info modify_tcp[] = {
struct mlx5_flow *flow)
{
struct mlx5_flow_dv_matcher *matcher = flow->dv.matcher;
+ struct mlx5_priv *priv = dev->data->dev_private;
+ struct mlx5_flow_tbl_resource *tbl;
assert(matcher->matcher_object);
DRV_LOG(DEBUG, "port %u matcher %p: refcnt %d--",
@@ -3440,6 +3503,14 @@ struct field_modify_info modify_tcp[] = {
claim_zero(mlx5_glue->dv_destroy_flow_matcher
(matcher->matcher_object));
LIST_REMOVE(matcher, next);
+ if (matcher->egress)
+ tbl = &priv->tx_tbl[matcher->group];
+ else
+ tbl = &priv->rx_tbl[matcher->group];
+ if (rte_atomic32_dec_and_test(&tbl->refcnt)) {
+ mlx5_glue->dr_destroy_flow_tbl(tbl->obj);
+ tbl->obj = NULL;
+ }
rte_free(matcher);
DRV_LOG(DEBUG, "port %u matcher %p: removed",
dev->data->port_id, (void *)matcher);
@@ -3529,7 +3600,7 @@ struct field_modify_info modify_tcp[] = {
LIST_FOREACH(dev_flow, &flow->dev_flows, next) {
dv = &dev_flow->dv;
if (dv->flow) {
- claim_zero(mlx5_glue->destroy_flow(dv->flow));
+ claim_zero(mlx5_glue->dv_destroy_flow(dv->flow));
dv->flow = NULL;
}
if (dv->hrxq) {
diff --git a/drivers/net/mlx5/mlx5_glue.c b/drivers/net/mlx5/mlx5_glue.c
index 4b5aade..b0b144c 100644
--- a/drivers/net/mlx5/mlx5_glue.c
+++ b/drivers/net/mlx5/mlx5_glue.c
@@ -178,6 +178,9 @@
mlx5_glue_destroy_flow_action(void *action)
{
#ifdef HAVE_IBV_FLOW_DV_SUPPORT
+#ifdef HAVE_MLX5DV_DR
+ return mlx5dv_dr_destroy_action(action);
+#else
struct mlx5dv_flow_action_attr *attr = action;
int res = 0;
switch (attr->type) {
@@ -189,6 +192,7 @@
}
free(action);
return res;
+#endif
#else
(void)action;
return ENOTSUP;
@@ -365,6 +369,53 @@
return ibv_cq_ex_to_cq(cq);
}
+static void *
+mlx5_glue_dr_create_flow_tbl(void *ns, uint32_t level)
+{
+#ifdef HAVE_MLX5DV_DR
+ return mlx5dv_dr_create_ft(ns, level);
+#else
+ (void)ns;
+ (void)level;
+ return NULL;
+#endif
+}
+
+static int
+mlx5_glue_dr_destroy_flow_tbl(void *tbl)
+{
+#ifdef HAVE_MLX5DV_DR
+ return mlx5dv_dr_destroy_ft(tbl);
+#else
+ (void)tbl;
+ return 0;
+#endif
+}
+
+static void *
+mlx5_glue_dr_create_ns(struct ibv_context *ctx,
+ enum mlx5dv_dr_ns_domain domain)
+{
+#ifdef HAVE_MLX5DV_DR
+ return mlx5dv_dr_create_ns(ctx, domain);
+#else
+ (void)ctx;
+ (void)domain;
+ return NULL;
+#endif
+}
+
+static int
+mlx5_glue_dr_destroy_ns(void *ns)
+{
+#ifdef HAVE_MLX5DV_DR
+ return mlx5dv_dr_destroy_ns(ns);
+#else
+ (void)ns;
+ return 0;
+#endif
+}
+
static struct ibv_cq_ex *
mlx5_glue_dv_create_cq(struct ibv_context *context,
struct ibv_cq_init_attr_ex *cq_attr,
@@ -423,26 +474,40 @@
#endif
}
-static struct mlx5dv_flow_matcher *
+static void *
mlx5_glue_dv_create_flow_matcher(struct ibv_context *context,
- struct mlx5dv_flow_matcher_attr *matcher_attr)
+ struct mlx5dv_flow_matcher_attr *matcher_attr,
+ void *tbl)
{
#ifdef HAVE_IBV_FLOW_DV_SUPPORT
+#ifdef HAVE_MLX5DV_DR
+ (void)context;
+ return mlx5dv_dr_create_matcher(tbl, matcher_attr->priority,
+ matcher_attr->match_criteria_enable,
+ matcher_attr->match_mask);
+#else
+ (void)tbl;
return mlx5dv_create_flow_matcher(context, matcher_attr);
+#endif
#else
(void)context;
(void)matcher_attr;
+ (void)tbl;
return NULL;
#endif
}
-static struct ibv_flow *
-mlx5_glue_dv_create_flow(struct mlx5dv_flow_matcher *matcher,
- struct mlx5dv_flow_match_parameters *match_value,
+static void *
+mlx5_glue_dv_create_flow(void *matcher,
+ void *match_value,
size_t num_actions,
void *actions[])
{
#ifdef HAVE_IBV_FLOW_DV_SUPPORT
+#ifdef HAVE_MLX5DV_DR
+ return mlx5dv_dr_create_rule(matcher, match_value, num_actions,
+ (struct mlx5dv_dr_action **)actions);
+#else
struct mlx5dv_flow_action_attr actions_attr[8];
if (num_actions > 8)
@@ -452,6 +517,7 @@
*((struct mlx5dv_flow_action_attr *)(actions[i]));
return mlx5dv_create_flow(matcher, match_value,
num_actions, actions_attr);
+#endif
#else
(void)matcher;
(void)match_value;
@@ -461,21 +527,13 @@
#endif
}
-static int
-mlx5_glue_dv_destroy_flow_matcher(struct mlx5dv_flow_matcher *matcher)
-{
-#ifdef HAVE_IBV_FLOW_DV_SUPPORT
- return mlx5dv_destroy_flow_matcher(matcher);
-#else
- (void)matcher;
- return 0;
-#endif
-}
-
static void *
mlx5_glue_dv_create_flow_action_counter(void *counter_obj, uint32_t offset)
{
#ifdef HAVE_IBV_FLOW_DV_SUPPORT
+#ifdef HAVE_MLX5DV_DR
+ return mlx5dv_dr_create_action_devx_counter(counter_obj, offset);
+#else
struct mlx5dv_flow_action_attr *action;
(void)offset;
@@ -485,6 +543,7 @@
action->type = MLX5DV_FLOW_ACTION_COUNTERS_DEVX;
action->obj = counter_obj;
return action;
+#endif
#else
(void)counter_obj;
(void)offset;
@@ -496,6 +555,9 @@
mlx5_glue_dv_create_flow_action_dest_ibv_qp(void *qp)
{
#ifdef HAVE_IBV_FLOW_DV_SUPPORT
+#ifdef HAVE_MLX5DV_DR
+ return mlx5dv_dr_create_action_dest_ibv_qp(qp);
+#else
struct mlx5dv_flow_action_attr *action;
action = malloc(sizeof(*action));
@@ -504,6 +566,7 @@
action->type = MLX5DV_FLOW_ACTION_DEST_IBV_QP;
action->obj = qp;
return action;
+#endif
#else
(void)qp;
return NULL;
@@ -513,13 +576,22 @@
static void *
mlx5_glue_dv_create_flow_action_modify_header
(struct ibv_context *ctx,
+ enum mlx5dv_flow_table_type ft_type,
+ void *ns, uint64_t flags,
size_t actions_sz,
- uint64_t actions[],
- enum mlx5dv_flow_table_type ft_type)
+ uint64_t actions[])
{
#ifdef HAVE_IBV_FLOW_DV_SUPPORT
+#ifdef HAVE_MLX5DV_DR
+ (void)ctx;
+ (void)ft_type;
+ return mlx5dv_dr_create_action_modify_header(ns, flags, actions_sz,
+ actions);
+#else
struct mlx5dv_flow_action_attr *action;
+ (void)ns;
+ (void)flags;
action = malloc(sizeof(*action));
if (!action)
return NULL;
@@ -527,11 +599,14 @@
action->action = mlx5dv_create_flow_action_modify_header
(ctx, actions_sz, actions, ft_type);
return action;
+#endif
#else
(void)ctx;
+ (void)ft_type;
+ (void)ns;
+ (void)flags;
(void)actions_sz;
(void)actions;
- (void)ft_type;
return NULL;
#endif
}
@@ -539,12 +614,20 @@
static void *
mlx5_glue_dv_create_flow_action_packet_reformat
(struct ibv_context *ctx,
- size_t data_sz,
- void *data,
enum mlx5dv_flow_action_packet_reformat_type reformat_type,
- enum mlx5dv_flow_table_type ft_type)
+ enum mlx5dv_flow_table_type ft_type, struct mlx5dv_dr_ns *ns,
+ uint32_t flags, size_t data_sz, void *data)
{
#ifdef HAVE_IBV_FLOW_DV_SUPPORT
+#ifdef HAVE_MLX5DV_DR
+ (void)ctx;
+ (void)ft_type;
+ return mlx5dv_dr_create_action_packet_reformat(ns, flags,
+ reformat_type, data_sz,
+ data);
+#else
+ (void)ns;
+ (void)flags;
struct mlx5dv_flow_action_attr *action;
action = malloc(sizeof(*action));
@@ -554,12 +637,15 @@
action->action = mlx5dv_create_flow_action_packet_reformat
(ctx, data_sz, data, reformat_type, ft_type);
return action;
+#endif
#else
(void)ctx;
- (void)data_sz;
- (void)data;
(void)reformat_type;
(void)ft_type;
+ (void)ns;
+ (void)flags;
+ (void)data_sz;
+ (void)data;
return NULL;
#endif
}
@@ -568,6 +654,9 @@
mlx5_glue_dv_create_flow_action_tag(uint32_t tag)
{
#ifdef HAVE_IBV_FLOW_DV_SUPPORT
+#ifdef HAVE_MLX5DV_DR
+ return mlx5dv_dr_create_action_tag(tag);
+#else
struct mlx5dv_flow_action_attr *action;
action = malloc(sizeof(*action));
if (!action)
@@ -576,10 +665,36 @@
action->tag_value = tag;
return action;
#endif
+#endif
(void)tag;
return NULL;
}
+static int
+mlx5_glue_dv_destroy_flow(void *flow_id)
+{
+#ifdef HAVE_MLX5DV_DR
+ return mlx5dv_dr_destroy_rule(flow_id);
+#else
+ return ibv_destroy_flow(flow_id);
+#endif
+}
+
+static int
+mlx5_glue_dv_destroy_flow_matcher(void *matcher)
+{
+#ifdef HAVE_IBV_FLOW_DV_SUPPORT
+#ifdef HAVE_MLX5DV_DR
+ return mlx5dv_dr_destroy_matcher(matcher);
+#else
+ return mlx5dv_destroy_flow_matcher(matcher);
+#endif
+#else
+ (void)matcher;
+ return 0;
+#endif
+}
+
static struct ibv_context *
mlx5_glue_dv_open_device(struct ibv_device *device)
{
@@ -718,6 +833,10 @@
.get_async_event = mlx5_glue_get_async_event,
.port_state_str = mlx5_glue_port_state_str,
.cq_ex_to_cq = mlx5_glue_cq_ex_to_cq,
+ .dr_create_flow_tbl = mlx5_glue_dr_create_flow_tbl,
+ .dr_destroy_flow_tbl = mlx5_glue_dr_destroy_flow_tbl,
+ .dr_create_ns = mlx5_glue_dr_create_ns,
+ .dr_destroy_ns = mlx5_glue_dr_destroy_ns,
.dv_create_cq = mlx5_glue_dv_create_cq,
.dv_create_wq = mlx5_glue_dv_create_wq,
.dv_query_device = mlx5_glue_dv_query_device,
@@ -725,7 +844,6 @@
.dv_init_obj = mlx5_glue_dv_init_obj,
.dv_create_qp = mlx5_glue_dv_create_qp,
.dv_create_flow_matcher = mlx5_glue_dv_create_flow_matcher,
- .dv_destroy_flow_matcher = mlx5_glue_dv_destroy_flow_matcher,
.dv_create_flow = mlx5_glue_dv_create_flow,
.dv_create_flow_action_counter =
mlx5_glue_dv_create_flow_action_counter,
@@ -736,6 +854,8 @@
.dv_create_flow_action_packet_reformat =
mlx5_glue_dv_create_flow_action_packet_reformat,
.dv_create_flow_action_tag = mlx5_glue_dv_create_flow_action_tag,
+ .dv_destroy_flow = mlx5_glue_dv_destroy_flow,
+ .dv_destroy_flow_matcher = mlx5_glue_dv_destroy_flow_matcher,
.dv_open_device = mlx5_glue_dv_open_device,
.devx_obj_create = mlx5_glue_devx_obj_create,
.devx_obj_destroy = mlx5_glue_devx_obj_destroy,
diff --git a/drivers/net/mlx5/mlx5_glue.h b/drivers/net/mlx5/mlx5_glue.h
index 32487ea..eb29ffa 100644
--- a/drivers/net/mlx5/mlx5_glue.h
+++ b/drivers/net/mlx5/mlx5_glue.h
@@ -63,6 +63,11 @@
struct mlx5dv_devx_obj;
#endif
+#ifndef HAVE_MLX5DV_DR
+struct mlx5dv_dr_ns;
+enum mlx5dv_dr_ns_domain { unused, };
+#endif
+
/* LIB_GLUE_VERSION must be updated every time this structure is modified. */
struct mlx5_glue {
const char *version;
@@ -140,6 +145,11 @@ struct mlx5_glue {
struct ibv_async_event *event);
const char *(*port_state_str)(enum ibv_port_state port_state);
struct ibv_cq *(*cq_ex_to_cq)(struct ibv_cq_ex *cq);
+ void *(*dr_create_flow_tbl)(void *ns, uint32_t level);
+ int (*dr_destroy_flow_tbl)(void *tbl);
+ void *(*dr_create_ns)(struct ibv_context *ctx,
+ enum mlx5dv_dr_ns_domain domain);
+ int (*dr_destroy_ns)(void *ns);
struct ibv_cq_ex *(*dv_create_cq)
(struct ibv_context *context,
struct ibv_cq_init_attr_ex *cq_attr,
@@ -158,23 +168,26 @@ struct mlx5_glue {
(struct ibv_context *context,
struct ibv_qp_init_attr_ex *qp_init_attr_ex,
struct mlx5dv_qp_init_attr *dv_qp_init_attr);
- struct mlx5dv_flow_matcher *(*dv_create_flow_matcher)
+ void *(*dv_create_flow_matcher)
(struct ibv_context *context,
- struct mlx5dv_flow_matcher_attr *matcher_attr);
- int (*dv_destroy_flow_matcher)(struct mlx5dv_flow_matcher *matcher);
- struct ibv_flow *(*dv_create_flow)(struct mlx5dv_flow_matcher *matcher,
- struct mlx5dv_flow_match_parameters *match_value,
+ struct mlx5dv_flow_matcher_attr *matcher_attr,
+ void *tbl);
+ void *(*dv_create_flow)(void *matcher, void *match_value,
size_t num_actions, void *actions[]);
void *(*dv_create_flow_action_counter)(void *obj, uint32_t offset);
void *(*dv_create_flow_action_dest_ibv_qp)(void *qp);
void *(*dv_create_flow_action_modify_header)
- (struct ibv_context *ctx, size_t actions_sz, uint64_t actions[],
- enum mlx5dv_flow_table_type ft_type);
+ (struct ibv_context *ctx, enum mlx5dv_flow_table_type ft_type,
+ void *ns, uint64_t flags, size_t actions_sz,
+ uint64_t actions[]);
void *(*dv_create_flow_action_packet_reformat)
- (struct ibv_context *ctx, size_t data_sz, void *data,
+ (struct ibv_context *ctx,
enum mlx5dv_flow_action_packet_reformat_type reformat_type,
- enum mlx5dv_flow_table_type ft_type);
+ enum mlx5dv_flow_table_type ft_type, struct mlx5dv_dr_ns *ns,
+ uint32_t flags, size_t data_sz, void *data);
void *(*dv_create_flow_action_tag)(uint32_t tag);
+ int (*dv_destroy_flow)(void *flow);
+ int (*dv_destroy_flow_matcher)(void *matcher);
struct ibv_context *(*dv_open_device)(struct ibv_device *device);
struct mlx5dv_devx_obj *(*devx_obj_create)
(struct ibv_context *ctx,
diff --git a/drivers/net/mlx5/mlx5_prm.h b/drivers/net/mlx5/mlx5_prm.h
index da1219e..b15266f 100644
--- a/drivers/net/mlx5/mlx5_prm.h
+++ b/drivers/net/mlx5/mlx5_prm.h
@@ -492,20 +492,40 @@ struct mlx5_ifc_fte_match_set_misc2_bits {
u8 reserved_at_1a0[0x60];
};
+struct mlx5_ifc_fte_match_set_misc3_bits {
+ u8 inner_tcp_seq_num[0x20];
+ u8 outer_tcp_seq_num[0x20];
+ u8 inner_tcp_ack_num[0x20];
+ u8 outer_tcp_ack_num[0x20];
+ u8 reserved_at_auto1[0x8];
+ u8 outer_vxlan_gpe_vni[0x18];
+ u8 outer_vxlan_gpe_next_protocol[0x8];
+ u8 outer_vxlan_gpe_flags[0x8];
+ u8 reserved_at_a8[0x10];
+ u8 icmp_header_data[0x20];
+ u8 icmpv6_header_data[0x20];
+ u8 icmp_type[0x8];
+ u8 icmp_code[0x8];
+ u8 icmpv6_type[0x8];
+ u8 icmpv6_code[0x8];
+ u8 reserved_at_1a0[0xe0];
+};
+
/* Flow matcher. */
struct mlx5_ifc_fte_match_param_bits {
struct mlx5_ifc_fte_match_set_lyr_2_4_bits outer_headers;
struct mlx5_ifc_fte_match_set_misc_bits misc_parameters;
struct mlx5_ifc_fte_match_set_lyr_2_4_bits inner_headers;
struct mlx5_ifc_fte_match_set_misc2_bits misc_parameters_2;
- u8 reserved_at_800[0x800];
+ struct mlx5_ifc_fte_match_set_misc3_bits misc_parameters_3;
};
enum {
MLX5_MATCH_CRITERIA_ENABLE_OUTER_BIT,
MLX5_MATCH_CRITERIA_ENABLE_MISC_BIT,
MLX5_MATCH_CRITERIA_ENABLE_INNER_BIT,
- MLX5_MATCH_CRITERIA_ENABLE_MISC2_BIT
+ MLX5_MATCH_CRITERIA_ENABLE_MISC2_BIT,
+ MLX5_MATCH_CRITERIA_ENABLE_MISC3_BIT
};
enum {
--
1.8.3.1
^ permalink raw reply [flat|nested] 52+ messages in thread
* [dpdk-dev] [PATCH v3 3/3] net/mlx5: add jump action support for NIC
2019-04-03 13:21 ` [dpdk-dev] [PATCH v3 " Ori Kam
` (2 preceding siblings ...)
2019-04-03 13:21 ` [dpdk-dev] [PATCH v3 2/3] net/mlx5: add Direct Rules API Ori Kam
@ 2019-04-03 13:21 ` Ori Kam
2019-04-03 13:21 ` Ori Kam
2019-04-04 5:26 ` Shahaf Shuler
3 siblings, 2 replies; 52+ messages in thread
From: Ori Kam @ 2019-04-03 13:21 UTC (permalink / raw)
To: Matan Azrad, Yongseok Koh, Shahaf Shuler; +Cc: dev, Ori Kam, Slava Ovsiienko
When using Direct Rules we can add actions to jump between tables.
This is extra useful since rule insertion rate is much higher on other
tables compared to table zero.
if no group is selected the rule is added to group 0.
Signed-off-by: Ori Kam <orika@mellanox.com>
Acked-by: Shahaf Shuler <shahafs@mellanox.com>
---
drivers/net/mlx5/mlx5.h | 6 +
drivers/net/mlx5/mlx5_flow.h | 15 ++-
drivers/net/mlx5/mlx5_flow_dv.c | 279 +++++++++++++++++++++++++++++++++++-----
drivers/net/mlx5/mlx5_glue.c | 13 ++
drivers/net/mlx5/mlx5_glue.h | 1 +
5 files changed, 283 insertions(+), 31 deletions(-)
diff --git a/drivers/net/mlx5/mlx5.h b/drivers/net/mlx5/mlx5.h
index d4963cb..5a7597e 100644
--- a/drivers/net/mlx5/mlx5.h
+++ b/drivers/net/mlx5/mlx5.h
@@ -319,6 +319,12 @@ struct mlx5_priv {
LIST_HEAD(encap_decap, mlx5_flow_dv_encap_decap_resource) encaps_decaps;
LIST_HEAD(modify_cmd, mlx5_flow_dv_modify_hdr_resource) modify_cmds;
LIST_HEAD(tag, mlx5_flow_dv_tag_resource) tags;
+ LIST_HEAD(jump, mlx5_flow_dv_jump_tbl_resource) jump_tbl;
+ /* Pointer to next element. */
+ rte_atomic32_t refcnt; /**< Reference counter. */
+ struct ibv_flow_action *verbs_action;
+ /**< Verbs modify header action object. */
+ uint8_t ft_type; /**< Flow table type, Rx or Tx. */
/* Tags resources cache. */
uint32_t link_speed_capa; /* Link speed capabilities. */
struct mlx5_xstats_ctrl xstats_ctrl; /* Extended stats control. */
diff --git a/drivers/net/mlx5/mlx5_flow.h b/drivers/net/mlx5/mlx5_flow.h
index 8ba37a0..622e305 100644
--- a/drivers/net/mlx5/mlx5_flow.h
+++ b/drivers/net/mlx5/mlx5_flow.h
@@ -115,7 +115,8 @@
#define MLX5_FLOW_ACTION_RAW_DECAP (1u << 27)
#define MLX5_FLOW_FATE_ACTIONS \
- (MLX5_FLOW_ACTION_DROP | MLX5_FLOW_ACTION_QUEUE | MLX5_FLOW_ACTION_RSS)
+ (MLX5_FLOW_ACTION_DROP | MLX5_FLOW_ACTION_QUEUE | \
+ MLX5_FLOW_ACTION_RSS | MLX5_FLOW_ACTION_JUMP)
#define MLX5_FLOW_ENCAP_ACTIONS (MLX5_FLOW_ACTION_VXLAN_ENCAP | \
MLX5_FLOW_ACTION_NVGRE_ENCAP | \
@@ -250,6 +251,16 @@ struct mlx5_flow_dv_modify_hdr_resource {
/**< Modification actions. */
};
+/* Jump action resource structure. */
+struct mlx5_flow_dv_jump_tbl_resource {
+ LIST_ENTRY(mlx5_flow_dv_jump_tbl_resource) next;
+ /* Pointer to next element. */
+ rte_atomic32_t refcnt; /**< Reference counter. */
+ void *action; /**< Pointer to the rdma core action. */
+ uint8_t ft_type; /**< Flow table type, Rx or Tx. */
+ struct mlx5_flow_tbl_resource *tbl; /**< The target table. */
+};
+
/*
* Max number of actions per DV flow.
* See CREATE_FLOW_MAX_FLOW_ACTIONS_SUPPORTED
@@ -270,6 +281,8 @@ struct mlx5_flow_dv {
struct mlx5_flow_dv_modify_hdr_resource *modify_hdr;
/**< Pointer to modify header resource in cache. */
struct ibv_flow *flow; /**< Installed flow. */
+ struct mlx5_flow_dv_jump_tbl_resource *jump;
+ /**< Pointer to the jump action resource. */
#ifdef HAVE_IBV_FLOW_DV_SUPPORT
void *actions[MLX5_DV_MAX_NUMBER_OF_ACTIONS];
/**< Action list. */
diff --git a/drivers/net/mlx5/mlx5_flow_dv.c b/drivers/net/mlx5/mlx5_flow_dv.c
index dd5b541..2c17491 100644
--- a/drivers/net/mlx5/mlx5_flow_dv.c
+++ b/drivers/net/mlx5/mlx5_flow_dv.c
@@ -865,6 +865,68 @@ struct field_modify_info modify_tcp[] = {
}
/**
+ * Find existing table jump resource or create and register a new one.
+ *
+ * @param dev[in, out]
+ * Pointer to rte_eth_dev structure.
+ * @param[in, out] resource
+ * Pointer to jump table resource.
+ * @parm[in, out] dev_flow
+ * Pointer to the dev_flow.
+ * @param[out] error
+ * pointer to error structure.
+ *
+ * @return
+ * 0 on success otherwise -errno and errno is set.
+ */
+static int
+flow_dv_jump_tbl_resource_register
+ (struct rte_eth_dev *dev,
+ struct mlx5_flow_dv_jump_tbl_resource *resource,
+ struct mlx5_flow *dev_flow,
+ struct rte_flow_error *error)
+{
+ struct mlx5_priv *priv = dev->data->dev_private;
+ struct mlx5_flow_dv_jump_tbl_resource *cache_resource;
+
+ /* Lookup a matching resource from cache. */
+ LIST_FOREACH(cache_resource, &priv->jump_tbl, next) {
+ if (resource->tbl == cache_resource->tbl) {
+ DRV_LOG(DEBUG, "jump table resource resource %p: refcnt %d++",
+ (void *)cache_resource,
+ rte_atomic32_read(&cache_resource->refcnt));
+ rte_atomic32_inc(&cache_resource->refcnt);
+ dev_flow->dv.jump = cache_resource;
+ return 0;
+ }
+ }
+ /* Register new jump table resource. */
+ cache_resource = rte_calloc(__func__, 1, sizeof(*cache_resource), 0);
+ if (!cache_resource)
+ return rte_flow_error_set(error, ENOMEM,
+ RTE_FLOW_ERROR_TYPE_UNSPECIFIED, NULL,
+ "cannot allocate resource memory");
+ *cache_resource = *resource;
+ cache_resource->action =
+ mlx5_glue->dr_create_flow_action_dest_flow_tbl
+ (resource->tbl->obj);
+ if (!cache_resource->action) {
+ rte_free(cache_resource);
+ return rte_flow_error_set(error, ENOMEM,
+ RTE_FLOW_ERROR_TYPE_UNSPECIFIED,
+ NULL, "cannot create action");
+ }
+ rte_atomic32_init(&cache_resource->refcnt);
+ rte_atomic32_inc(&cache_resource->refcnt);
+ LIST_INSERT_HEAD(&priv->jump_tbl, cache_resource, next);
+ dev_flow->dv.jump = cache_resource;
+ DRV_LOG(DEBUG, "new jump table resource %p: refcnt %d++",
+ (void *)cache_resource,
+ rte_atomic32_read(&cache_resource->refcnt));
+ return 0;
+}
+
+/**
* Get the size of specific rte_flow_item_type
*
* @param[in] item_type
@@ -1427,6 +1489,37 @@ struct field_modify_info modify_tcp[] = {
}
/**
+ * Validate jump action.
+ *
+ * @param[in] action
+ * Pointer to the modify action.
+ * @param[in] group
+ * The group of the current flow.
+ * @param[out] error
+ * Pointer to error structure.
+ *
+ * @return
+ * 0 on success, a negative errno value otherwise and rte_errno is set.
+ */
+static int
+flow_dv_validate_action_jump(const struct rte_flow_action *action,
+ uint32_t group,
+ struct rte_flow_error *error)
+{
+ if (action->type != RTE_FLOW_ACTION_TYPE_JUMP && !action->conf)
+ return rte_flow_error_set(error, EINVAL,
+ RTE_FLOW_ERROR_TYPE_ACTION_CONF,
+ NULL, "action configuration not set");
+ if (group >= ((const struct rte_flow_action_jump *)action->conf)->group)
+ return rte_flow_error_set(error, EINVAL,
+ RTE_FLOW_ERROR_TYPE_ACTION, NULL,
+ "target group must be higher then"
+ " the current flow group");
+ return 0;
+}
+
+
+/**
* Find existing modify-header resource or create and register a new one.
*
* @param dev[in, out]
@@ -1609,7 +1702,7 @@ struct field_modify_info modify_tcp[] = {
struct mlx5_priv *priv = dev->data->dev_private;
uint32_t priority_max = priv->config.flow_prio - 1;
-#ifdef HAVE_MLX5DV_DR
+#ifndef HAVE_MLX5DV_DR
if (attributes->group)
return rte_flow_error_set(error, ENOTSUP,
RTE_FLOW_ERROR_TYPE_ATTR_GROUP,
@@ -1981,6 +2074,14 @@ struct field_modify_info modify_tcp[] = {
MLX5_FLOW_ACTION_SET_TTL :
MLX5_FLOW_ACTION_DEC_TTL;
break;
+ case RTE_FLOW_ACTION_TYPE_JUMP:
+ ret = flow_dv_validate_action_jump(actions,
+ attr->group, error);
+ if (ret)
+ return ret;
+ ++actions_n;
+ action_flags |= MLX5_FLOW_ACTION_JUMP;
+ break;
default:
return rte_flow_error_set(error, ENOTSUP,
RTE_FLOW_ERROR_TYPE_ACTION,
@@ -2760,6 +2861,81 @@ struct field_modify_info modify_tcp[] = {
return match_criteria_enable;
}
+
+/**
+ * Get a flow table.
+ *
+ * @param dev[in, out]
+ * Pointer to rte_eth_dev structure.
+ * @param[in] table_id
+ * Table id to use.
+ * @param[in] egress
+ * Direction of the table.
+ * @param[out] error
+ * pointer to error structure.
+ *
+ * @return
+ * Returns tables resource based on the index, NULL in case of failed.
+ */
+static struct mlx5_flow_tbl_resource *
+flow_dv_tbl_resource_get(struct rte_eth_dev *dev,
+ uint32_t table_id, uint8_t egress,
+ struct rte_flow_error *error)
+{
+ struct mlx5_priv *priv = dev->data->dev_private;
+ struct mlx5_flow_tbl_resource *tbl;
+
+#ifdef HAVE_MLX5DV_DR
+ if (egress) {
+ tbl = &priv->tx_tbl[table_id];
+ if (!tbl->obj)
+ tbl->obj = mlx5_glue->dr_create_flow_tbl
+ (priv->tx_ns, table_id);
+ } else {
+ tbl = &priv->rx_tbl[table_id];
+ if (!tbl->obj)
+ tbl->obj = mlx5_glue->dr_create_flow_tbl
+ (priv->rx_ns, table_id);
+ }
+ if (!tbl->obj) {
+ rte_flow_error_set(error, ENOMEM,
+ RTE_FLOW_ERROR_TYPE_UNSPECIFIED,
+ NULL, "cannot create table");
+ return NULL;
+ }
+ rte_atomic32_inc(&tbl->refcnt);
+ return tbl;
+#else
+ (void)error;
+ if (egress)
+ return &priv->tx_tbl[table_id];
+ else
+ return &priv->rx_tbl[table_id];
+#endif
+}
+
+/**
+ * Release a flow table.
+ *
+ * @param[in] tbl
+ * Table resource to be released.
+ *
+ * @return
+ * Returns 0 if table was released, else return 1;
+ */
+static int
+flow_dv_tbl_resource_release(struct mlx5_flow_tbl_resource *tbl)
+{
+ if (!tbl)
+ return 0;
+ if (rte_atomic32_dec_and_test(&tbl->refcnt)) {
+ mlx5_glue->dr_destroy_flow_tbl(tbl->obj);
+ tbl->obj = NULL;
+ return 0;
+ }
+ return 1;
+}
+
/**
* Register the flow matcher.
*
@@ -2809,33 +2985,20 @@ struct field_modify_info modify_tcp[] = {
return 0;
}
}
-#ifdef HAVE_MLX5DV_DR
- if (matcher->egress) {
- tbl = &priv->tx_tbl[matcher->group];
- if (!tbl->obj)
- tbl->obj = mlx5_glue->dr_create_flow_tbl
- (priv->tx_ns,
- matcher->group * MLX5_GROUP_FACTOR);
- } else {
- tbl = &priv->rx_tbl[matcher->group];
- if (!tbl->obj)
- tbl->obj = mlx5_glue->dr_create_flow_tbl
- (priv->rx_ns,
- matcher->group * MLX5_GROUP_FACTOR);
- }
- if (!tbl->obj)
- return rte_flow_error_set(error, ENOMEM,
- RTE_FLOW_ERROR_TYPE_UNSPECIFIED,
- NULL, "cannot create table");
-
- rte_atomic32_inc(&tbl->refcnt);
-#endif
/* Register new matcher. */
cache_matcher = rte_calloc(__func__, 1, sizeof(*cache_matcher), 0);
if (!cache_matcher)
return rte_flow_error_set(error, ENOMEM,
RTE_FLOW_ERROR_TYPE_UNSPECIFIED, NULL,
"cannot allocate matcher memory");
+ tbl = flow_dv_tbl_resource_get(dev, matcher->group * MLX5_GROUP_FACTOR,
+ matcher->egress, error);
+ if (!tbl) {
+ rte_free(cache_matcher);
+ return rte_flow_error_set(error, ENOMEM,
+ RTE_FLOW_ERROR_TYPE_UNSPECIFIED,
+ NULL, "cannot create table");
+ }
*cache_matcher = *matcher;
dv_attr.match_criteria_enable =
flow_dv_matcher_enable(cache_matcher->mask.buf);
@@ -2848,10 +3011,7 @@ struct field_modify_info modify_tcp[] = {
if (!cache_matcher->matcher_object) {
rte_free(cache_matcher);
#ifdef HAVE_MLX5DV_DR
- if (rte_atomic32_dec_and_test(&tbl->refcnt)) {
- mlx5_glue->dr_destroy_flow_tbl(tbl->obj);
- tbl->obj = NULL;
- }
+ flow_dv_tbl_resource_release(tbl);
#endif
return rte_flow_error_set(error, ENOMEM,
RTE_FLOW_ERROR_TYPE_UNSPECIFIED,
@@ -3037,6 +3197,9 @@ struct field_modify_info modify_tcp[] = {
const struct rte_flow_action *action = actions;
const struct rte_flow_action_count *count = action->conf;
const uint8_t *rss_key;
+ const struct rte_flow_action_jump *jump_data;
+ struct mlx5_flow_dv_jump_tbl_resource jump_tbl_resource;
+ struct mlx5_flow_tbl_resource *tbl;
switch (actions->type) {
case RTE_FLOW_ACTION_TYPE_VOID:
@@ -3175,6 +3338,31 @@ struct field_modify_info modify_tcp[] = {
/* If decap is followed by encap, handle it at encap. */
action_flags |= MLX5_FLOW_ACTION_RAW_DECAP;
break;
+ case RTE_FLOW_ACTION_TYPE_JUMP:
+ jump_data = action->conf;
+ tbl = flow_dv_tbl_resource_get(dev, jump_data->group *
+ MLX5_GROUP_FACTOR,
+ attr->egress, error);
+ if (!tbl)
+ return rte_flow_error_set
+ (error, errno,
+ RTE_FLOW_ERROR_TYPE_ACTION,
+ NULL,
+ "cannot create jump action.");
+ jump_tbl_resource.tbl = tbl;
+ if (flow_dv_jump_tbl_resource_register
+ (dev, &jump_tbl_resource, dev_flow, error)) {
+ flow_dv_tbl_resource_release(tbl);
+ return rte_flow_error_set
+ (error, errno,
+ RTE_FLOW_ERROR_TYPE_ACTION,
+ NULL,
+ "cannot create jump action.");
+ }
+ dev_flow->dv.actions[actions_n++] =
+ dev_flow->dv.jump->action;
+ action_flags |= MLX5_FLOW_ACTION_JUMP;
+ break;
case RTE_FLOW_ACTION_TYPE_SET_MAC_SRC:
case RTE_FLOW_ACTION_TYPE_SET_MAC_DST:
if (flow_dv_convert_action_modify_mac(&res, actions,
@@ -3507,10 +3695,7 @@ struct field_modify_info modify_tcp[] = {
tbl = &priv->tx_tbl[matcher->group];
else
tbl = &priv->rx_tbl[matcher->group];
- if (rte_atomic32_dec_and_test(&tbl->refcnt)) {
- mlx5_glue->dr_destroy_flow_tbl(tbl->obj);
- tbl->obj = NULL;
- }
+ flow_dv_tbl_resource_release(tbl);
rte_free(matcher);
DRV_LOG(DEBUG, "port %u matcher %p: removed",
dev->data->port_id, (void *)matcher);
@@ -3551,6 +3736,38 @@ struct field_modify_info modify_tcp[] = {
}
/**
+ * Release an jump to table action resource.
+ *
+ * @param flow
+ * Pointer to mlx5_flow.
+ *
+ * @return
+ * 1 while a reference on it exists, 0 when freed.
+ */
+static int
+flow_dv_jump_tbl_resource_release(struct mlx5_flow *flow)
+{
+ struct mlx5_flow_dv_jump_tbl_resource *cache_resource =
+ flow->dv.jump;
+
+ assert(cache_resource->action);
+ DRV_LOG(DEBUG, "jump table resource %p: refcnt %d--",
+ (void *)cache_resource,
+ rte_atomic32_read(&cache_resource->refcnt));
+ if (rte_atomic32_dec_and_test(&cache_resource->refcnt)) {
+ claim_zero(mlx5_glue->destroy_flow_action
+ (cache_resource->action));
+ LIST_REMOVE(cache_resource, next);
+ flow_dv_tbl_resource_release(cache_resource->tbl);
+ rte_free(cache_resource);
+ DRV_LOG(DEBUG, "jump table resource %p: removed",
+ (void *)cache_resource);
+ return 0;
+ }
+ return 1;
+}
+
+/**
* Release a modify-header resource.
*
* @param flow
@@ -3646,6 +3863,8 @@ struct field_modify_info modify_tcp[] = {
flow_dv_encap_decap_resource_release(dev_flow);
if (dev_flow->dv.modify_hdr)
flow_dv_modify_hdr_resource_release(dev_flow);
+ if (dev_flow->dv.jump)
+ flow_dv_jump_tbl_resource_release(dev_flow);
rte_free(dev_flow);
}
}
diff --git a/drivers/net/mlx5/mlx5_glue.c b/drivers/net/mlx5/mlx5_glue.c
index b0b144c..f5a6c2e 100644
--- a/drivers/net/mlx5/mlx5_glue.c
+++ b/drivers/net/mlx5/mlx5_glue.c
@@ -370,6 +370,17 @@
}
static void *
+mlx5_glue_dr_create_flow_action_dest_flow_tbl(void *tbl)
+{
+#ifdef HAVE_MLX5DV_DR
+ return mlx5dv_dr_create_action_dest_flow_table(tbl);
+#else
+ (void)tbl;
+ return NULL;
+#endif
+}
+
+static void *
mlx5_glue_dr_create_flow_tbl(void *ns, uint32_t level)
{
#ifdef HAVE_MLX5DV_DR
@@ -833,6 +844,8 @@
.get_async_event = mlx5_glue_get_async_event,
.port_state_str = mlx5_glue_port_state_str,
.cq_ex_to_cq = mlx5_glue_cq_ex_to_cq,
+ .dr_create_flow_action_dest_flow_tbl =
+ mlx5_glue_dr_create_flow_action_dest_flow_tbl,
.dr_create_flow_tbl = mlx5_glue_dr_create_flow_tbl,
.dr_destroy_flow_tbl = mlx5_glue_dr_destroy_flow_tbl,
.dr_create_ns = mlx5_glue_dr_create_ns,
diff --git a/drivers/net/mlx5/mlx5_glue.h b/drivers/net/mlx5/mlx5_glue.h
index eb29ffa..058e9b1 100644
--- a/drivers/net/mlx5/mlx5_glue.h
+++ b/drivers/net/mlx5/mlx5_glue.h
@@ -145,6 +145,7 @@ struct mlx5_glue {
struct ibv_async_event *event);
const char *(*port_state_str)(enum ibv_port_state port_state);
struct ibv_cq *(*cq_ex_to_cq)(struct ibv_cq_ex *cq);
+ void *(*dr_create_flow_action_dest_flow_tbl)(void *tbl);
void *(*dr_create_flow_tbl)(void *ns, uint32_t level);
int (*dr_destroy_flow_tbl)(void *tbl);
void *(*dr_create_ns)(struct ibv_context *ctx,
--
1.8.3.1
^ permalink raw reply [flat|nested] 52+ messages in thread
* [dpdk-dev] [PATCH v3 3/3] net/mlx5: add jump action support for NIC
2019-04-03 13:21 ` [dpdk-dev] [PATCH v3 3/3] net/mlx5: add jump action support for NIC Ori Kam
@ 2019-04-03 13:21 ` Ori Kam
2019-04-04 5:26 ` Shahaf Shuler
1 sibling, 0 replies; 52+ messages in thread
From: Ori Kam @ 2019-04-03 13:21 UTC (permalink / raw)
To: Matan Azrad, Yongseok Koh, Shahaf Shuler; +Cc: dev, Ori Kam, Slava Ovsiienko
When using Direct Rules we can add actions to jump between tables.
This is extra useful since rule insertion rate is much higher on other
tables compared to table zero.
if no group is selected the rule is added to group 0.
Signed-off-by: Ori Kam <orika@mellanox.com>
Acked-by: Shahaf Shuler <shahafs@mellanox.com>
---
drivers/net/mlx5/mlx5.h | 6 +
drivers/net/mlx5/mlx5_flow.h | 15 ++-
drivers/net/mlx5/mlx5_flow_dv.c | 279 +++++++++++++++++++++++++++++++++++-----
drivers/net/mlx5/mlx5_glue.c | 13 ++
drivers/net/mlx5/mlx5_glue.h | 1 +
5 files changed, 283 insertions(+), 31 deletions(-)
diff --git a/drivers/net/mlx5/mlx5.h b/drivers/net/mlx5/mlx5.h
index d4963cb..5a7597e 100644
--- a/drivers/net/mlx5/mlx5.h
+++ b/drivers/net/mlx5/mlx5.h
@@ -319,6 +319,12 @@ struct mlx5_priv {
LIST_HEAD(encap_decap, mlx5_flow_dv_encap_decap_resource) encaps_decaps;
LIST_HEAD(modify_cmd, mlx5_flow_dv_modify_hdr_resource) modify_cmds;
LIST_HEAD(tag, mlx5_flow_dv_tag_resource) tags;
+ LIST_HEAD(jump, mlx5_flow_dv_jump_tbl_resource) jump_tbl;
+ /* Pointer to next element. */
+ rte_atomic32_t refcnt; /**< Reference counter. */
+ struct ibv_flow_action *verbs_action;
+ /**< Verbs modify header action object. */
+ uint8_t ft_type; /**< Flow table type, Rx or Tx. */
/* Tags resources cache. */
uint32_t link_speed_capa; /* Link speed capabilities. */
struct mlx5_xstats_ctrl xstats_ctrl; /* Extended stats control. */
diff --git a/drivers/net/mlx5/mlx5_flow.h b/drivers/net/mlx5/mlx5_flow.h
index 8ba37a0..622e305 100644
--- a/drivers/net/mlx5/mlx5_flow.h
+++ b/drivers/net/mlx5/mlx5_flow.h
@@ -115,7 +115,8 @@
#define MLX5_FLOW_ACTION_RAW_DECAP (1u << 27)
#define MLX5_FLOW_FATE_ACTIONS \
- (MLX5_FLOW_ACTION_DROP | MLX5_FLOW_ACTION_QUEUE | MLX5_FLOW_ACTION_RSS)
+ (MLX5_FLOW_ACTION_DROP | MLX5_FLOW_ACTION_QUEUE | \
+ MLX5_FLOW_ACTION_RSS | MLX5_FLOW_ACTION_JUMP)
#define MLX5_FLOW_ENCAP_ACTIONS (MLX5_FLOW_ACTION_VXLAN_ENCAP | \
MLX5_FLOW_ACTION_NVGRE_ENCAP | \
@@ -250,6 +251,16 @@ struct mlx5_flow_dv_modify_hdr_resource {
/**< Modification actions. */
};
+/* Jump action resource structure. */
+struct mlx5_flow_dv_jump_tbl_resource {
+ LIST_ENTRY(mlx5_flow_dv_jump_tbl_resource) next;
+ /* Pointer to next element. */
+ rte_atomic32_t refcnt; /**< Reference counter. */
+ void *action; /**< Pointer to the rdma core action. */
+ uint8_t ft_type; /**< Flow table type, Rx or Tx. */
+ struct mlx5_flow_tbl_resource *tbl; /**< The target table. */
+};
+
/*
* Max number of actions per DV flow.
* See CREATE_FLOW_MAX_FLOW_ACTIONS_SUPPORTED
@@ -270,6 +281,8 @@ struct mlx5_flow_dv {
struct mlx5_flow_dv_modify_hdr_resource *modify_hdr;
/**< Pointer to modify header resource in cache. */
struct ibv_flow *flow; /**< Installed flow. */
+ struct mlx5_flow_dv_jump_tbl_resource *jump;
+ /**< Pointer to the jump action resource. */
#ifdef HAVE_IBV_FLOW_DV_SUPPORT
void *actions[MLX5_DV_MAX_NUMBER_OF_ACTIONS];
/**< Action list. */
diff --git a/drivers/net/mlx5/mlx5_flow_dv.c b/drivers/net/mlx5/mlx5_flow_dv.c
index dd5b541..2c17491 100644
--- a/drivers/net/mlx5/mlx5_flow_dv.c
+++ b/drivers/net/mlx5/mlx5_flow_dv.c
@@ -865,6 +865,68 @@ struct field_modify_info modify_tcp[] = {
}
/**
+ * Find existing table jump resource or create and register a new one.
+ *
+ * @param dev[in, out]
+ * Pointer to rte_eth_dev structure.
+ * @param[in, out] resource
+ * Pointer to jump table resource.
+ * @parm[in, out] dev_flow
+ * Pointer to the dev_flow.
+ * @param[out] error
+ * pointer to error structure.
+ *
+ * @return
+ * 0 on success otherwise -errno and errno is set.
+ */
+static int
+flow_dv_jump_tbl_resource_register
+ (struct rte_eth_dev *dev,
+ struct mlx5_flow_dv_jump_tbl_resource *resource,
+ struct mlx5_flow *dev_flow,
+ struct rte_flow_error *error)
+{
+ struct mlx5_priv *priv = dev->data->dev_private;
+ struct mlx5_flow_dv_jump_tbl_resource *cache_resource;
+
+ /* Lookup a matching resource from cache. */
+ LIST_FOREACH(cache_resource, &priv->jump_tbl, next) {
+ if (resource->tbl == cache_resource->tbl) {
+ DRV_LOG(DEBUG, "jump table resource resource %p: refcnt %d++",
+ (void *)cache_resource,
+ rte_atomic32_read(&cache_resource->refcnt));
+ rte_atomic32_inc(&cache_resource->refcnt);
+ dev_flow->dv.jump = cache_resource;
+ return 0;
+ }
+ }
+ /* Register new jump table resource. */
+ cache_resource = rte_calloc(__func__, 1, sizeof(*cache_resource), 0);
+ if (!cache_resource)
+ return rte_flow_error_set(error, ENOMEM,
+ RTE_FLOW_ERROR_TYPE_UNSPECIFIED, NULL,
+ "cannot allocate resource memory");
+ *cache_resource = *resource;
+ cache_resource->action =
+ mlx5_glue->dr_create_flow_action_dest_flow_tbl
+ (resource->tbl->obj);
+ if (!cache_resource->action) {
+ rte_free(cache_resource);
+ return rte_flow_error_set(error, ENOMEM,
+ RTE_FLOW_ERROR_TYPE_UNSPECIFIED,
+ NULL, "cannot create action");
+ }
+ rte_atomic32_init(&cache_resource->refcnt);
+ rte_atomic32_inc(&cache_resource->refcnt);
+ LIST_INSERT_HEAD(&priv->jump_tbl, cache_resource, next);
+ dev_flow->dv.jump = cache_resource;
+ DRV_LOG(DEBUG, "new jump table resource %p: refcnt %d++",
+ (void *)cache_resource,
+ rte_atomic32_read(&cache_resource->refcnt));
+ return 0;
+}
+
+/**
* Get the size of specific rte_flow_item_type
*
* @param[in] item_type
@@ -1427,6 +1489,37 @@ struct field_modify_info modify_tcp[] = {
}
/**
+ * Validate jump action.
+ *
+ * @param[in] action
+ * Pointer to the modify action.
+ * @param[in] group
+ * The group of the current flow.
+ * @param[out] error
+ * Pointer to error structure.
+ *
+ * @return
+ * 0 on success, a negative errno value otherwise and rte_errno is set.
+ */
+static int
+flow_dv_validate_action_jump(const struct rte_flow_action *action,
+ uint32_t group,
+ struct rte_flow_error *error)
+{
+ if (action->type != RTE_FLOW_ACTION_TYPE_JUMP && !action->conf)
+ return rte_flow_error_set(error, EINVAL,
+ RTE_FLOW_ERROR_TYPE_ACTION_CONF,
+ NULL, "action configuration not set");
+ if (group >= ((const struct rte_flow_action_jump *)action->conf)->group)
+ return rte_flow_error_set(error, EINVAL,
+ RTE_FLOW_ERROR_TYPE_ACTION, NULL,
+ "target group must be higher then"
+ " the current flow group");
+ return 0;
+}
+
+
+/**
* Find existing modify-header resource or create and register a new one.
*
* @param dev[in, out]
@@ -1609,7 +1702,7 @@ struct field_modify_info modify_tcp[] = {
struct mlx5_priv *priv = dev->data->dev_private;
uint32_t priority_max = priv->config.flow_prio - 1;
-#ifdef HAVE_MLX5DV_DR
+#ifndef HAVE_MLX5DV_DR
if (attributes->group)
return rte_flow_error_set(error, ENOTSUP,
RTE_FLOW_ERROR_TYPE_ATTR_GROUP,
@@ -1981,6 +2074,14 @@ struct field_modify_info modify_tcp[] = {
MLX5_FLOW_ACTION_SET_TTL :
MLX5_FLOW_ACTION_DEC_TTL;
break;
+ case RTE_FLOW_ACTION_TYPE_JUMP:
+ ret = flow_dv_validate_action_jump(actions,
+ attr->group, error);
+ if (ret)
+ return ret;
+ ++actions_n;
+ action_flags |= MLX5_FLOW_ACTION_JUMP;
+ break;
default:
return rte_flow_error_set(error, ENOTSUP,
RTE_FLOW_ERROR_TYPE_ACTION,
@@ -2760,6 +2861,81 @@ struct field_modify_info modify_tcp[] = {
return match_criteria_enable;
}
+
+/**
+ * Get a flow table.
+ *
+ * @param dev[in, out]
+ * Pointer to rte_eth_dev structure.
+ * @param[in] table_id
+ * Table id to use.
+ * @param[in] egress
+ * Direction of the table.
+ * @param[out] error
+ * pointer to error structure.
+ *
+ * @return
+ * Returns tables resource based on the index, NULL in case of failed.
+ */
+static struct mlx5_flow_tbl_resource *
+flow_dv_tbl_resource_get(struct rte_eth_dev *dev,
+ uint32_t table_id, uint8_t egress,
+ struct rte_flow_error *error)
+{
+ struct mlx5_priv *priv = dev->data->dev_private;
+ struct mlx5_flow_tbl_resource *tbl;
+
+#ifdef HAVE_MLX5DV_DR
+ if (egress) {
+ tbl = &priv->tx_tbl[table_id];
+ if (!tbl->obj)
+ tbl->obj = mlx5_glue->dr_create_flow_tbl
+ (priv->tx_ns, table_id);
+ } else {
+ tbl = &priv->rx_tbl[table_id];
+ if (!tbl->obj)
+ tbl->obj = mlx5_glue->dr_create_flow_tbl
+ (priv->rx_ns, table_id);
+ }
+ if (!tbl->obj) {
+ rte_flow_error_set(error, ENOMEM,
+ RTE_FLOW_ERROR_TYPE_UNSPECIFIED,
+ NULL, "cannot create table");
+ return NULL;
+ }
+ rte_atomic32_inc(&tbl->refcnt);
+ return tbl;
+#else
+ (void)error;
+ if (egress)
+ return &priv->tx_tbl[table_id];
+ else
+ return &priv->rx_tbl[table_id];
+#endif
+}
+
+/**
+ * Release a flow table.
+ *
+ * @param[in] tbl
+ * Table resource to be released.
+ *
+ * @return
+ * Returns 0 if table was released, else return 1;
+ */
+static int
+flow_dv_tbl_resource_release(struct mlx5_flow_tbl_resource *tbl)
+{
+ if (!tbl)
+ return 0;
+ if (rte_atomic32_dec_and_test(&tbl->refcnt)) {
+ mlx5_glue->dr_destroy_flow_tbl(tbl->obj);
+ tbl->obj = NULL;
+ return 0;
+ }
+ return 1;
+}
+
/**
* Register the flow matcher.
*
@@ -2809,33 +2985,20 @@ struct field_modify_info modify_tcp[] = {
return 0;
}
}
-#ifdef HAVE_MLX5DV_DR
- if (matcher->egress) {
- tbl = &priv->tx_tbl[matcher->group];
- if (!tbl->obj)
- tbl->obj = mlx5_glue->dr_create_flow_tbl
- (priv->tx_ns,
- matcher->group * MLX5_GROUP_FACTOR);
- } else {
- tbl = &priv->rx_tbl[matcher->group];
- if (!tbl->obj)
- tbl->obj = mlx5_glue->dr_create_flow_tbl
- (priv->rx_ns,
- matcher->group * MLX5_GROUP_FACTOR);
- }
- if (!tbl->obj)
- return rte_flow_error_set(error, ENOMEM,
- RTE_FLOW_ERROR_TYPE_UNSPECIFIED,
- NULL, "cannot create table");
-
- rte_atomic32_inc(&tbl->refcnt);
-#endif
/* Register new matcher. */
cache_matcher = rte_calloc(__func__, 1, sizeof(*cache_matcher), 0);
if (!cache_matcher)
return rte_flow_error_set(error, ENOMEM,
RTE_FLOW_ERROR_TYPE_UNSPECIFIED, NULL,
"cannot allocate matcher memory");
+ tbl = flow_dv_tbl_resource_get(dev, matcher->group * MLX5_GROUP_FACTOR,
+ matcher->egress, error);
+ if (!tbl) {
+ rte_free(cache_matcher);
+ return rte_flow_error_set(error, ENOMEM,
+ RTE_FLOW_ERROR_TYPE_UNSPECIFIED,
+ NULL, "cannot create table");
+ }
*cache_matcher = *matcher;
dv_attr.match_criteria_enable =
flow_dv_matcher_enable(cache_matcher->mask.buf);
@@ -2848,10 +3011,7 @@ struct field_modify_info modify_tcp[] = {
if (!cache_matcher->matcher_object) {
rte_free(cache_matcher);
#ifdef HAVE_MLX5DV_DR
- if (rte_atomic32_dec_and_test(&tbl->refcnt)) {
- mlx5_glue->dr_destroy_flow_tbl(tbl->obj);
- tbl->obj = NULL;
- }
+ flow_dv_tbl_resource_release(tbl);
#endif
return rte_flow_error_set(error, ENOMEM,
RTE_FLOW_ERROR_TYPE_UNSPECIFIED,
@@ -3037,6 +3197,9 @@ struct field_modify_info modify_tcp[] = {
const struct rte_flow_action *action = actions;
const struct rte_flow_action_count *count = action->conf;
const uint8_t *rss_key;
+ const struct rte_flow_action_jump *jump_data;
+ struct mlx5_flow_dv_jump_tbl_resource jump_tbl_resource;
+ struct mlx5_flow_tbl_resource *tbl;
switch (actions->type) {
case RTE_FLOW_ACTION_TYPE_VOID:
@@ -3175,6 +3338,31 @@ struct field_modify_info modify_tcp[] = {
/* If decap is followed by encap, handle it at encap. */
action_flags |= MLX5_FLOW_ACTION_RAW_DECAP;
break;
+ case RTE_FLOW_ACTION_TYPE_JUMP:
+ jump_data = action->conf;
+ tbl = flow_dv_tbl_resource_get(dev, jump_data->group *
+ MLX5_GROUP_FACTOR,
+ attr->egress, error);
+ if (!tbl)
+ return rte_flow_error_set
+ (error, errno,
+ RTE_FLOW_ERROR_TYPE_ACTION,
+ NULL,
+ "cannot create jump action.");
+ jump_tbl_resource.tbl = tbl;
+ if (flow_dv_jump_tbl_resource_register
+ (dev, &jump_tbl_resource, dev_flow, error)) {
+ flow_dv_tbl_resource_release(tbl);
+ return rte_flow_error_set
+ (error, errno,
+ RTE_FLOW_ERROR_TYPE_ACTION,
+ NULL,
+ "cannot create jump action.");
+ }
+ dev_flow->dv.actions[actions_n++] =
+ dev_flow->dv.jump->action;
+ action_flags |= MLX5_FLOW_ACTION_JUMP;
+ break;
case RTE_FLOW_ACTION_TYPE_SET_MAC_SRC:
case RTE_FLOW_ACTION_TYPE_SET_MAC_DST:
if (flow_dv_convert_action_modify_mac(&res, actions,
@@ -3507,10 +3695,7 @@ struct field_modify_info modify_tcp[] = {
tbl = &priv->tx_tbl[matcher->group];
else
tbl = &priv->rx_tbl[matcher->group];
- if (rte_atomic32_dec_and_test(&tbl->refcnt)) {
- mlx5_glue->dr_destroy_flow_tbl(tbl->obj);
- tbl->obj = NULL;
- }
+ flow_dv_tbl_resource_release(tbl);
rte_free(matcher);
DRV_LOG(DEBUG, "port %u matcher %p: removed",
dev->data->port_id, (void *)matcher);
@@ -3551,6 +3736,38 @@ struct field_modify_info modify_tcp[] = {
}
/**
+ * Release an jump to table action resource.
+ *
+ * @param flow
+ * Pointer to mlx5_flow.
+ *
+ * @return
+ * 1 while a reference on it exists, 0 when freed.
+ */
+static int
+flow_dv_jump_tbl_resource_release(struct mlx5_flow *flow)
+{
+ struct mlx5_flow_dv_jump_tbl_resource *cache_resource =
+ flow->dv.jump;
+
+ assert(cache_resource->action);
+ DRV_LOG(DEBUG, "jump table resource %p: refcnt %d--",
+ (void *)cache_resource,
+ rte_atomic32_read(&cache_resource->refcnt));
+ if (rte_atomic32_dec_and_test(&cache_resource->refcnt)) {
+ claim_zero(mlx5_glue->destroy_flow_action
+ (cache_resource->action));
+ LIST_REMOVE(cache_resource, next);
+ flow_dv_tbl_resource_release(cache_resource->tbl);
+ rte_free(cache_resource);
+ DRV_LOG(DEBUG, "jump table resource %p: removed",
+ (void *)cache_resource);
+ return 0;
+ }
+ return 1;
+}
+
+/**
* Release a modify-header resource.
*
* @param flow
@@ -3646,6 +3863,8 @@ struct field_modify_info modify_tcp[] = {
flow_dv_encap_decap_resource_release(dev_flow);
if (dev_flow->dv.modify_hdr)
flow_dv_modify_hdr_resource_release(dev_flow);
+ if (dev_flow->dv.jump)
+ flow_dv_jump_tbl_resource_release(dev_flow);
rte_free(dev_flow);
}
}
diff --git a/drivers/net/mlx5/mlx5_glue.c b/drivers/net/mlx5/mlx5_glue.c
index b0b144c..f5a6c2e 100644
--- a/drivers/net/mlx5/mlx5_glue.c
+++ b/drivers/net/mlx5/mlx5_glue.c
@@ -370,6 +370,17 @@
}
static void *
+mlx5_glue_dr_create_flow_action_dest_flow_tbl(void *tbl)
+{
+#ifdef HAVE_MLX5DV_DR
+ return mlx5dv_dr_create_action_dest_flow_table(tbl);
+#else
+ (void)tbl;
+ return NULL;
+#endif
+}
+
+static void *
mlx5_glue_dr_create_flow_tbl(void *ns, uint32_t level)
{
#ifdef HAVE_MLX5DV_DR
@@ -833,6 +844,8 @@
.get_async_event = mlx5_glue_get_async_event,
.port_state_str = mlx5_glue_port_state_str,
.cq_ex_to_cq = mlx5_glue_cq_ex_to_cq,
+ .dr_create_flow_action_dest_flow_tbl =
+ mlx5_glue_dr_create_flow_action_dest_flow_tbl,
.dr_create_flow_tbl = mlx5_glue_dr_create_flow_tbl,
.dr_destroy_flow_tbl = mlx5_glue_dr_destroy_flow_tbl,
.dr_create_ns = mlx5_glue_dr_create_ns,
diff --git a/drivers/net/mlx5/mlx5_glue.h b/drivers/net/mlx5/mlx5_glue.h
index eb29ffa..058e9b1 100644
--- a/drivers/net/mlx5/mlx5_glue.h
+++ b/drivers/net/mlx5/mlx5_glue.h
@@ -145,6 +145,7 @@ struct mlx5_glue {
struct ibv_async_event *event);
const char *(*port_state_str)(enum ibv_port_state port_state);
struct ibv_cq *(*cq_ex_to_cq)(struct ibv_cq_ex *cq);
+ void *(*dr_create_flow_action_dest_flow_tbl)(void *tbl);
void *(*dr_create_flow_tbl)(void *ns, uint32_t level);
int (*dr_destroy_flow_tbl)(void *tbl);
void *(*dr_create_ns)(struct ibv_context *ctx,
--
1.8.3.1
^ permalink raw reply [flat|nested] 52+ messages in thread
* Re: [dpdk-dev] [PATCH v3 3/3] net/mlx5: add jump action support for NIC
2019-04-03 13:21 ` [dpdk-dev] [PATCH v3 3/3] net/mlx5: add jump action support for NIC Ori Kam
2019-04-03 13:21 ` Ori Kam
@ 2019-04-04 5:26 ` Shahaf Shuler
2019-04-04 5:26 ` Shahaf Shuler
1 sibling, 1 reply; 52+ messages in thread
From: Shahaf Shuler @ 2019-04-04 5:26 UTC (permalink / raw)
To: Ori Kam, Matan Azrad, Yongseok Koh; +Cc: dev, Slava Ovsiienko
Hi Ori,
Wednesday, April 3, 2019 4:22 PM, Ori Kam:
> Subject: [PATCH v3 3/3] net/mlx5: add jump action support for NIC
>
> When using Direct Rules we can add actions to jump between tables.
> This is extra useful since rule insertion rate is much higher on other tables
> compared to table zero.
>
> if no group is selected the rule is added to group 0.
>
> Signed-off-by: Ori Kam <orika@mellanox.com>
> Acked-by: Shahaf Shuler <shahafs@mellanox.com>
[...]
>
> +
> +/**
> + * Get a flow table.
> + *
> + * @param dev[in, out]
> + * Pointer to rte_eth_dev structure.
> + * @param[in] table_id
> + * Table id to use.
> + * @param[in] egress
> + * Direction of the table.
> + * @param[out] error
> + * pointer to error structure.
> + *
> + * @return
> + * Returns tables resource based on the index, NULL in case of failed.
> + */
> +static struct mlx5_flow_tbl_resource *
> +flow_dv_tbl_resource_get(struct rte_eth_dev *dev,
> + uint32_t table_id, uint8_t egress,
> + struct rte_flow_error *error)
> +{
> + struct mlx5_priv *priv = dev->data->dev_private;
> + struct mlx5_flow_tbl_resource *tbl;
Am getting compilation error here on latest rdma-core:
c: In function 'flow_dv_tbl_resource_get':
/.autodirect/swgwork/shahafs/workspace/dpdk.org/drivers/net/mlx5/mlx5_flow_dv.c:2886:33: error: unused variable 'tbl'
[-Werror=unused-variable]
struct mlx5_flow_tbl_resource *tbl;
^ permalink raw reply [flat|nested] 52+ messages in thread
* Re: [dpdk-dev] [PATCH v3 3/3] net/mlx5: add jump action support for NIC
2019-04-04 5:26 ` Shahaf Shuler
@ 2019-04-04 5:26 ` Shahaf Shuler
0 siblings, 0 replies; 52+ messages in thread
From: Shahaf Shuler @ 2019-04-04 5:26 UTC (permalink / raw)
To: Ori Kam, Matan Azrad, Yongseok Koh; +Cc: dev, Slava Ovsiienko
Hi Ori,
Wednesday, April 3, 2019 4:22 PM, Ori Kam:
> Subject: [PATCH v3 3/3] net/mlx5: add jump action support for NIC
>
> When using Direct Rules we can add actions to jump between tables.
> This is extra useful since rule insertion rate is much higher on other tables
> compared to table zero.
>
> if no group is selected the rule is added to group 0.
>
> Signed-off-by: Ori Kam <orika@mellanox.com>
> Acked-by: Shahaf Shuler <shahafs@mellanox.com>
[...]
>
> +
> +/**
> + * Get a flow table.
> + *
> + * @param dev[in, out]
> + * Pointer to rte_eth_dev structure.
> + * @param[in] table_id
> + * Table id to use.
> + * @param[in] egress
> + * Direction of the table.
> + * @param[out] error
> + * pointer to error structure.
> + *
> + * @return
> + * Returns tables resource based on the index, NULL in case of failed.
> + */
> +static struct mlx5_flow_tbl_resource *
> +flow_dv_tbl_resource_get(struct rte_eth_dev *dev,
> + uint32_t table_id, uint8_t egress,
> + struct rte_flow_error *error)
> +{
> + struct mlx5_priv *priv = dev->data->dev_private;
> + struct mlx5_flow_tbl_resource *tbl;
Am getting compilation error here on latest rdma-core:
c: In function 'flow_dv_tbl_resource_get':
/.autodirect/swgwork/shahafs/workspace/dpdk.org/drivers/net/mlx5/mlx5_flow_dv.c:2886:33: error: unused variable 'tbl'
[-Werror=unused-variable]
struct mlx5_flow_tbl_resource *tbl;
^ permalink raw reply [flat|nested] 52+ messages in thread
* [dpdk-dev] [PATCH v4 0/3] net/mlx5: Add Direct Rule support
2019-03-28 16:32 ` [dpdk-dev] [PATCH v2 0/3] net/mlx5: Add Direct Rule support Ori Kam
` (5 preceding siblings ...)
2019-04-03 13:21 ` [dpdk-dev] [PATCH v3 " Ori Kam
@ 2019-04-04 9:54 ` Ori Kam
2019-04-04 9:54 ` Ori Kam
` (4 more replies)
6 siblings, 5 replies; 52+ messages in thread
From: Ori Kam @ 2019-04-04 9:54 UTC (permalink / raw)
To: Matan Azrad, Yongseok Koh, Shahaf Shuler; +Cc: dev, Ori Kam, Slava Ovsiienko
This patch set adds support for Direct Rules API for MLX5 driver.
As more and more applications like OVS are inserting and removing large
number of flows, the flow insertion rate become more critical.
In current MLX5 PMD rules are inserted to the device using the FW,
this limits the max insertion rate. The solution is using a new set of API
that are part of RDMA-Core. which enables bypassing the FW and writing
the flows directly to the device, which results in a much faster
insertion rate.
v4:
* Fix small compilation issue.
v3:
* Address comments from ML.
* rebase.
v2:
* Small issues fixed.
* Rebased.
Ori Kam (3):
net/mlx5: prepare Direct Verbs for Direct Rule
net/mlx5: add Direct Rules API
net/mlx5: add jump action support for NIC
drivers/net/mlx5/Makefile | 7 +-
drivers/net/mlx5/meson.build | 2 +-
drivers/net/mlx5/mlx5.c | 16 ++
drivers/net/mlx5/mlx5.h | 23 ++
drivers/net/mlx5/mlx5_flow.c | 1 +
drivers/net/mlx5/mlx5_flow.h | 38 ++-
drivers/net/mlx5/mlx5_flow_dv.c | 507 ++++++++++++++++++++++++++++++++++------
drivers/net/mlx5/mlx5_glue.c | 300 ++++++++++++++++++++----
drivers/net/mlx5/mlx5_glue.h | 50 ++--
drivers/net/mlx5/mlx5_prm.h | 24 +-
10 files changed, 836 insertions(+), 132 deletions(-)
--
1.8.3.1
^ permalink raw reply [flat|nested] 52+ messages in thread
* [dpdk-dev] [PATCH v4 0/3] net/mlx5: Add Direct Rule support
2019-04-04 9:54 ` [dpdk-dev] [PATCH v4 0/3] net/mlx5: Add Direct Rule support Ori Kam
@ 2019-04-04 9:54 ` Ori Kam
2019-04-04 9:54 ` [dpdk-dev] [PATCH v4 1/3] net/mlx5: prepare Direct Verbs for Direct Rule Ori Kam
` (3 subsequent siblings)
4 siblings, 0 replies; 52+ messages in thread
From: Ori Kam @ 2019-04-04 9:54 UTC (permalink / raw)
To: Matan Azrad, Yongseok Koh, Shahaf Shuler; +Cc: dev, Ori Kam, Slava Ovsiienko
This patch set adds support for Direct Rules API for MLX5 driver.
As more and more applications like OVS are inserting and removing large
number of flows, the flow insertion rate become more critical.
In current MLX5 PMD rules are inserted to the device using the FW,
this limits the max insertion rate. The solution is using a new set of API
that are part of RDMA-Core. which enables bypassing the FW and writing
the flows directly to the device, which results in a much faster
insertion rate.
v4:
* Fix small compilation issue.
v3:
* Address comments from ML.
* rebase.
v2:
* Small issues fixed.
* Rebased.
Ori Kam (3):
net/mlx5: prepare Direct Verbs for Direct Rule
net/mlx5: add Direct Rules API
net/mlx5: add jump action support for NIC
drivers/net/mlx5/Makefile | 7 +-
drivers/net/mlx5/meson.build | 2 +-
drivers/net/mlx5/mlx5.c | 16 ++
drivers/net/mlx5/mlx5.h | 23 ++
drivers/net/mlx5/mlx5_flow.c | 1 +
drivers/net/mlx5/mlx5_flow.h | 38 ++-
drivers/net/mlx5/mlx5_flow_dv.c | 507 ++++++++++++++++++++++++++++++++++------
drivers/net/mlx5/mlx5_glue.c | 300 ++++++++++++++++++++----
drivers/net/mlx5/mlx5_glue.h | 50 ++--
drivers/net/mlx5/mlx5_prm.h | 24 +-
10 files changed, 836 insertions(+), 132 deletions(-)
--
1.8.3.1
^ permalink raw reply [flat|nested] 52+ messages in thread
* [dpdk-dev] [PATCH v4 1/3] net/mlx5: prepare Direct Verbs for Direct Rule
2019-04-04 9:54 ` [dpdk-dev] [PATCH v4 0/3] net/mlx5: Add Direct Rule support Ori Kam
2019-04-04 9:54 ` Ori Kam
@ 2019-04-04 9:54 ` Ori Kam
2019-04-04 9:54 ` Ori Kam
2019-04-12 23:51 ` dwilder
2019-04-04 9:54 ` [dpdk-dev] [PATCH v4 2/3] net/mlx5: add Direct Rules API Ori Kam
` (2 subsequent siblings)
4 siblings, 2 replies; 52+ messages in thread
From: Ori Kam @ 2019-04-04 9:54 UTC (permalink / raw)
To: Matan Azrad, Yongseok Koh, Shahaf Shuler; +Cc: dev, Ori Kam, Slava Ovsiienko
This is the first patch of a series that is designed to enable the
Direct Rules API.
The main difference between Direct Verbs and Direct Rules from API
prespective, is that in Direct Rules each action has it's own create
function and the object itself is of type void.
In this patch I'm adding functions to generate actions that currenlty
are done without create action, and I'm changing the action type to be
void *, so in next patches only the glue functions will need to change.
Signed-off-by: Ori Kam <orika@mellanox.com>
Acked-by: Shahaf Shuler <shahafs@mellanox.com>
---
drivers/net/mlx5/Makefile | 2 +-
drivers/net/mlx5/meson.build | 2 +-
drivers/net/mlx5/mlx5.h | 2 +
drivers/net/mlx5/mlx5_flow.h | 17 +++-
drivers/net/mlx5/mlx5_flow_dv.c | 184 +++++++++++++++++++++++++++++-----------
drivers/net/mlx5/mlx5_glue.c | 141 ++++++++++++++++++++++++------
drivers/net/mlx5/mlx5_glue.h | 26 +++---
7 files changed, 280 insertions(+), 94 deletions(-)
diff --git a/drivers/net/mlx5/Makefile b/drivers/net/mlx5/Makefile
index ccd179c..0d20f0f 100644
--- a/drivers/net/mlx5/Makefile
+++ b/drivers/net/mlx5/Makefile
@@ -8,7 +8,7 @@ include $(RTE_SDK)/mk/rte.vars.mk
LIB = librte_pmd_mlx5.a
LIB_GLUE = $(LIB_GLUE_BASE).$(LIB_GLUE_VERSION)
LIB_GLUE_BASE = librte_pmd_mlx5_glue.so
-LIB_GLUE_VERSION = 19.02.0
+LIB_GLUE_VERSION = 19.05.0
# Sources.
SRCS-$(CONFIG_RTE_LIBRTE_MLX5_PMD) += mlx5.c
diff --git a/drivers/net/mlx5/meson.build b/drivers/net/mlx5/meson.build
index a429655..a4c684e 100644
--- a/drivers/net/mlx5/meson.build
+++ b/drivers/net/mlx5/meson.build
@@ -4,7 +4,7 @@
pmd_dlopen = (get_option('ibverbs_link') == 'dlopen')
LIB_GLUE_BASE = 'librte_pmd_mlx5_glue.so'
-LIB_GLUE_VERSION = '19.02.0'
+LIB_GLUE_VERSION = '19.05.0'
LIB_GLUE = LIB_GLUE_BASE + '.' + LIB_GLUE_VERSION
if pmd_dlopen
dpdk_conf.set('RTE_IBVERBS_LINK_DLOPEN', 1)
diff --git a/drivers/net/mlx5/mlx5.h b/drivers/net/mlx5/mlx5.h
index 47a7d75..784bf9b 100644
--- a/drivers/net/mlx5/mlx5.h
+++ b/drivers/net/mlx5/mlx5.h
@@ -309,6 +309,8 @@ struct mlx5_priv {
LIST_HEAD(matchers, mlx5_flow_dv_matcher) matchers;
LIST_HEAD(encap_decap, mlx5_flow_dv_encap_decap_resource) encaps_decaps;
LIST_HEAD(modify_cmd, mlx5_flow_dv_modify_hdr_resource) modify_cmds;
+ LIST_HEAD(tag, mlx5_flow_dv_tag_resource) tags;
+ /* Tags resources cache. */
uint32_t link_speed_capa; /* Link speed capabilities. */
struct mlx5_xstats_ctrl xstats_ctrl; /* Extended stats control. */
struct mlx5_stats_ctrl stats_ctrl; /* Stats control. */
diff --git a/drivers/net/mlx5/mlx5_flow.h b/drivers/net/mlx5/mlx5_flow.h
index e1e798b..4f69ae2 100644
--- a/drivers/net/mlx5/mlx5_flow.h
+++ b/drivers/net/mlx5/mlx5_flow.h
@@ -214,7 +214,7 @@ struct mlx5_flow_dv_encap_decap_resource {
LIST_ENTRY(mlx5_flow_dv_encap_decap_resource) next;
/* Pointer to next element. */
rte_atomic32_t refcnt; /**< Reference counter. */
- struct ibv_flow_action *verbs_action;
+ void *verbs_action;
/**< Verbs encap/decap action object. */
uint8_t buf[MLX5_ENCAP_MAX_LEN];
size_t size;
@@ -222,6 +222,16 @@ struct mlx5_flow_dv_encap_decap_resource {
uint8_t ft_type;
};
+/* Tag resource structure. */
+struct mlx5_flow_dv_tag_resource {
+ LIST_ENTRY(mlx5_flow_dv_tag_resource) next;
+ /* Pointer to next element. */
+ rte_atomic32_t refcnt; /**< Reference counter. */
+ void *action;
+ /**< Verbs tag action object. */
+ uint32_t tag; /**< the tag value. */
+};
+
/* Number of modification commands. */
#define MLX5_MODIFY_NUM 8
@@ -259,7 +269,7 @@ struct mlx5_flow_dv {
/**< Pointer to modify header resource in cache. */
struct ibv_flow *flow; /**< Installed flow. */
#ifdef HAVE_IBV_FLOW_DV_SUPPORT
- struct mlx5dv_flow_action_attr actions[MLX5_DV_MAX_NUMBER_OF_ACTIONS];
+ void *actions[MLX5_DV_MAX_NUMBER_OF_ACTIONS];
/**< Action list. */
#endif
int actions_n; /**< number of actions. */
@@ -332,6 +342,7 @@ struct mlx5_flow_counter {
};
uint64_t hits; /**< Number of packets matched by the rule. */
uint64_t bytes; /**< Number of bytes matched by the rule. */
+ void *action; /**< Pointer to the dv action. */
};
/* Flow structure. */
@@ -339,6 +350,8 @@ struct rte_flow {
TAILQ_ENTRY(rte_flow) next; /**< Pointer to the next flow structure. */
enum mlx5_flow_drv_type drv_type; /**< Drvier type. */
struct mlx5_flow_counter *counter; /**< Holds flow counter. */
+ struct mlx5_flow_dv_tag_resource *tag_resource;
+ /**< pointer to the tag action. */
struct rte_flow_action_rss rss;/**< RSS context. */
uint8_t key[MLX5_RSS_HASH_KEY_LEN]; /**< RSS hash key. */
uint16_t (*queue)[]; /**< Destination queues to redirect traffic to. */
diff --git a/drivers/net/mlx5/mlx5_flow_dv.c b/drivers/net/mlx5/mlx5_flow_dv.c
index ad84dea..d9e2ac9 100644
--- a/drivers/net/mlx5/mlx5_flow_dv.c
+++ b/drivers/net/mlx5/mlx5_flow_dv.c
@@ -32,7 +32,6 @@
#include "mlx5_prm.h"
#include "mlx5_glue.h"
#include "mlx5_flow.h"
-
#ifdef HAVE_IBV_FLOW_DV_SUPPORT
#ifndef HAVE_IBV_FLOW_DEVX_COUNTERS
@@ -1537,6 +1536,11 @@ struct field_modify_info modify_tcp[] = {
.id = id,
.dcs = dcs,
};
+ tmpl.action = mlx5_glue->dv_create_flow_action_counter(dcs->obj, 0);
+ if (!tmpl.action) {
+ ret = errno;
+ goto error_exit;
+ }
*cnt = tmpl;
LIST_INSERT_HEAD(&priv->flow_counters, cnt, next);
return cnt;
@@ -2828,6 +2832,97 @@ struct field_modify_info modify_tcp[] = {
}
/**
+ * Find existing tag resource or create and register a new one.
+ *
+ * @param dev[in, out]
+ * Pointer to rte_eth_dev structure.
+ * @param[in, out] resource
+ * Pointer to tag resource.
+ * @parm[in, out] dev_flow
+ * Pointer to the dev_flow.
+ * @param[out] error
+ * pointer to error structure.
+ *
+ * @return
+ * 0 on success otherwise -errno and errno is set.
+ */
+static int
+flow_dv_tag_resource_register
+ (struct rte_eth_dev *dev,
+ struct mlx5_flow_dv_tag_resource *resource,
+ struct mlx5_flow *dev_flow,
+ struct rte_flow_error *error)
+{
+ struct mlx5_priv *priv = dev->data->dev_private;
+ struct mlx5_flow_dv_tag_resource *cache_resource;
+
+ /* Lookup a matching resource from cache. */
+ LIST_FOREACH(cache_resource, &priv->tags, next) {
+ if (resource->tag == cache_resource->tag) {
+ DRV_LOG(DEBUG, "tag resource %p: refcnt %d++",
+ (void *)cache_resource,
+ rte_atomic32_read(&cache_resource->refcnt));
+ rte_atomic32_inc(&cache_resource->refcnt);
+ dev_flow->flow->tag_resource = cache_resource;
+ return 0;
+ }
+ }
+ /* Register new resource. */
+ cache_resource = rte_calloc(__func__, 1, sizeof(*cache_resource), 0);
+ if (!cache_resource)
+ return rte_flow_error_set(error, ENOMEM,
+ RTE_FLOW_ERROR_TYPE_UNSPECIFIED, NULL,
+ "cannot allocate resource memory");
+ *cache_resource = *resource;
+ cache_resource->action = mlx5_glue->dv_create_flow_action_tag
+ (resource->tag);
+ if (!cache_resource->action) {
+ rte_free(cache_resource);
+ return rte_flow_error_set(error, ENOMEM,
+ RTE_FLOW_ERROR_TYPE_UNSPECIFIED,
+ NULL, "cannot create action");
+ }
+ rte_atomic32_init(&cache_resource->refcnt);
+ rte_atomic32_inc(&cache_resource->refcnt);
+ LIST_INSERT_HEAD(&priv->tags, cache_resource, next);
+ dev_flow->flow->tag_resource = cache_resource;
+ DRV_LOG(DEBUG, "new tag resource %p: refcnt %d++",
+ (void *)cache_resource,
+ rte_atomic32_read(&cache_resource->refcnt));
+ return 0;
+}
+
+/**
+ * Release the tag.
+ *
+ * @param dev
+ * Pointer to Ethernet device.
+ * @param flow
+ * Pointer to mlx5_flow.
+ *
+ * @return
+ * 1 while a reference on it exists, 0 when freed.
+ */
+static int
+flow_dv_tag_release(struct rte_eth_dev *dev,
+ struct mlx5_flow_dv_tag_resource *tag)
+{
+ assert(tag);
+ DRV_LOG(DEBUG, "port %u tag %p: refcnt %d--",
+ dev->data->port_id, (void *)tag,
+ rte_atomic32_read(&tag->refcnt));
+ if (rte_atomic32_dec_and_test(&tag->refcnt)) {
+ claim_zero(mlx5_glue->destroy_flow_action(tag->action));
+ LIST_REMOVE(tag, next);
+ DRV_LOG(DEBUG, "port %u tag %p: removed",
+ dev->data->port_id, (void *)tag);
+ rte_free(tag);
+ return 0;
+ }
+ return 1;
+}
+
+/**
* Fill the flow with DV spec.
*
* @param[in] dev
@@ -2872,6 +2967,7 @@ struct field_modify_info modify_tcp[] = {
MLX5DV_FLOW_TABLE_TYPE_NIC_RX
};
union flow_dv_attr flow_attr = { .attr = 0 };
+ struct mlx5_flow_dv_tag_resource tag_resource;
if (priority == MLX5_FLOW_PRIO_RSVD)
priority = priv->config.flow_prio - 1;
@@ -2886,26 +2982,29 @@ struct field_modify_info modify_tcp[] = {
case RTE_FLOW_ACTION_TYPE_VOID:
break;
case RTE_FLOW_ACTION_TYPE_FLAG:
- dev_flow->dv.actions[actions_n].type =
- MLX5DV_FLOW_ACTION_TAG;
- dev_flow->dv.actions[actions_n].tag_value =
+ tag_resource.tag =
mlx5_flow_mark_set(MLX5_FLOW_MARK_DEFAULT);
- actions_n++;
+ if (!flow->tag_resource)
+ if (flow_dv_tag_resource_register
+ (dev, &tag_resource, dev_flow, error))
+ return errno;
+ dev_flow->dv.actions[actions_n++] =
+ flow->tag_resource->action;
action_flags |= MLX5_FLOW_ACTION_FLAG;
break;
case RTE_FLOW_ACTION_TYPE_MARK:
- dev_flow->dv.actions[actions_n].type =
- MLX5DV_FLOW_ACTION_TAG;
- dev_flow->dv.actions[actions_n].tag_value =
- mlx5_flow_mark_set
- (((const struct rte_flow_action_mark *)
- (actions->conf))->id);
- actions_n++;
+ tag_resource.tag = mlx5_flow_mark_set
+ (((const struct rte_flow_action_mark *)
+ (actions->conf))->id);
+ if (!flow->tag_resource)
+ if (flow_dv_tag_resource_register
+ (dev, &tag_resource, dev_flow, error))
+ return errno;
+ dev_flow->dv.actions[actions_n++] =
+ flow->tag_resource->action;
action_flags |= MLX5_FLOW_ACTION_MARK;
break;
case RTE_FLOW_ACTION_TYPE_DROP:
- dev_flow->dv.actions[actions_n].type =
- MLX5DV_FLOW_ACTION_DROP;
action_flags |= MLX5_FLOW_ACTION_DROP;
break;
case RTE_FLOW_ACTION_TYPE_QUEUE:
@@ -2933,17 +3032,13 @@ struct field_modify_info modify_tcp[] = {
rte_errno = ENOTSUP;
goto cnt_err;
}
- flow->counter =
- flow_dv_counter_new(dev,
- count->shared, count->id);
+ flow->counter = flow_dv_counter_new(dev, count->shared,
+ count->id);
if (flow->counter == NULL)
goto cnt_err;
- dev_flow->dv.actions[actions_n].type =
- MLX5DV_FLOW_ACTION_COUNTERS_DEVX;
- dev_flow->dv.actions[actions_n].obj =
- flow->counter->dcs->obj;
+ dev_flow->dv.actions[actions_n++] =
+ flow->counter->action;
action_flags |= MLX5_FLOW_ACTION_COUNT;
- ++actions_n;
break;
cnt_err:
if (rte_errno == ENOTSUP)
@@ -2964,11 +3059,8 @@ struct field_modify_info modify_tcp[] = {
if (flow_dv_create_action_l2_encap(dev, actions,
dev_flow, error))
return -rte_errno;
- dev_flow->dv.actions[actions_n].type =
- MLX5DV_FLOW_ACTION_IBV_FLOW_ACTION;
- dev_flow->dv.actions[actions_n].action =
+ dev_flow->dv.actions[actions_n++] =
dev_flow->dv.encap_decap->verbs_action;
- actions_n++;
action_flags |= actions->type ==
RTE_FLOW_ACTION_TYPE_VXLAN_ENCAP ?
MLX5_FLOW_ACTION_VXLAN_ENCAP :
@@ -2979,11 +3071,8 @@ struct field_modify_info modify_tcp[] = {
if (flow_dv_create_action_l2_decap(dev, dev_flow,
error))
return -rte_errno;
- dev_flow->dv.actions[actions_n].type =
- MLX5DV_FLOW_ACTION_IBV_FLOW_ACTION;
- dev_flow->dv.actions[actions_n].action =
+ dev_flow->dv.actions[actions_n++] =
dev_flow->dv.encap_decap->verbs_action;
- actions_n++;
action_flags |= actions->type ==
RTE_FLOW_ACTION_TYPE_VXLAN_DECAP ?
MLX5_FLOW_ACTION_VXLAN_DECAP :
@@ -2995,9 +3084,7 @@ struct field_modify_info modify_tcp[] = {
if (flow_dv_create_action_raw_encap
(dev, actions, dev_flow, attr, error))
return -rte_errno;
- dev_flow->dv.actions[actions_n].type =
- MLX5DV_FLOW_ACTION_IBV_FLOW_ACTION;
- dev_flow->dv.actions[actions_n].action =
+ dev_flow->dv.actions[actions_n++] =
dev_flow->dv.encap_decap->verbs_action;
} else {
/* Handle encap without preceding decap. */
@@ -3005,12 +3092,9 @@ struct field_modify_info modify_tcp[] = {
dev_flow,
error))
return -rte_errno;
- dev_flow->dv.actions[actions_n].type =
- MLX5DV_FLOW_ACTION_IBV_FLOW_ACTION;
- dev_flow->dv.actions[actions_n].action =
+ dev_flow->dv.actions[actions_n++] =
dev_flow->dv.encap_decap->verbs_action;
}
- actions_n++;
action_flags |= MLX5_FLOW_ACTION_RAW_ENCAP;
break;
case RTE_FLOW_ACTION_TYPE_RAW_DECAP:
@@ -3025,11 +3109,8 @@ struct field_modify_info modify_tcp[] = {
dev_flow,
error))
return -rte_errno;
- dev_flow->dv.actions[actions_n].type =
- MLX5DV_FLOW_ACTION_IBV_FLOW_ACTION;
- dev_flow->dv.actions[actions_n].action =
+ dev_flow->dv.actions[actions_n++] =
dev_flow->dv.encap_decap->verbs_action;
- actions_n++;
}
/* If decap is followed by encap, handle it at encap. */
action_flags |= MLX5_FLOW_ACTION_RAW_DECAP;
@@ -3098,11 +3179,8 @@ struct field_modify_info modify_tcp[] = {
dev_flow,
error))
return -rte_errno;
- dev_flow->dv.actions[actions_n].type =
- MLX5DV_FLOW_ACTION_IBV_FLOW_ACTION;
- dev_flow->dv.actions[actions_n].action =
+ dev_flow->dv.actions[actions_n++] =
dev_flow->dv.modify_hdr->verbs_action;
- actions_n++;
}
break;
default:
@@ -3277,9 +3355,9 @@ struct field_modify_info modify_tcp[] = {
"cannot get drop hash queue");
goto error;
}
- dv->actions[n].type = MLX5DV_FLOW_ACTION_DEST_IBV_QP;
- dv->actions[n].qp = dv->hrxq->qp;
- n++;
+ dv->actions[n++] =
+ mlx5_glue->dv_create_flow_action_dest_ibv_qp
+ (dv->hrxq->qp);
} else if (flow->actions &
(MLX5_FLOW_ACTION_QUEUE | MLX5_FLOW_ACTION_RSS)) {
struct mlx5_hrxq *hrxq;
@@ -3304,9 +3382,9 @@ struct field_modify_info modify_tcp[] = {
goto error;
}
dv->hrxq = hrxq;
- dv->actions[n].type = MLX5DV_FLOW_ACTION_DEST_IBV_QP;
- dv->actions[n].qp = hrxq->qp;
- n++;
+ dv->actions[n++] =
+ mlx5_glue->dv_create_flow_action_dest_ibv_qp
+ (dv->hrxq->qp);
}
dv->flow =
mlx5_glue->dv_create_flow(dv->matcher->matcher_object,
@@ -3484,6 +3562,10 @@ struct field_modify_info modify_tcp[] = {
flow_dv_counter_release(flow->counter);
flow->counter = NULL;
}
+ if (flow->tag_resource) {
+ flow_dv_tag_release(dev, flow->tag_resource);
+ flow->tag_resource = NULL;
+ }
while (!LIST_EMPTY(&flow->dev_flows)) {
dev_flow = LIST_FIRST(&flow->dev_flows);
LIST_REMOVE(dev_flow, next);
diff --git a/drivers/net/mlx5/mlx5_glue.c b/drivers/net/mlx5/mlx5_glue.c
index c817d86..4b5aade 100644
--- a/drivers/net/mlx5/mlx5_glue.c
+++ b/drivers/net/mlx5/mlx5_glue.c
@@ -175,10 +175,20 @@
}
static int
-mlx5_glue_destroy_flow_action(struct ibv_flow_action *action)
+mlx5_glue_destroy_flow_action(void *action)
{
#ifdef HAVE_IBV_FLOW_DV_SUPPORT
- return ibv_destroy_flow_action(action);
+ struct mlx5dv_flow_action_attr *attr = action;
+ int res = 0;
+ switch (attr->type) {
+ case MLX5DV_FLOW_ACTION_TAG:
+ break;
+ default:
+ res = ibv_destroy_flow_action(attr->action);
+ break;
+ }
+ free(action);
+ return res;
#else
(void)action;
return ENOTSUP;
@@ -430,16 +440,23 @@
mlx5_glue_dv_create_flow(struct mlx5dv_flow_matcher *matcher,
struct mlx5dv_flow_match_parameters *match_value,
size_t num_actions,
- struct mlx5dv_flow_action_attr *actions_attr)
+ void *actions[])
{
#ifdef HAVE_IBV_FLOW_DV_SUPPORT
+ struct mlx5dv_flow_action_attr actions_attr[8];
+
+ if (num_actions > 8)
+ return NULL;
+ for (size_t i = 0; i < num_actions; i++)
+ actions_attr[i] =
+ *((struct mlx5dv_flow_action_attr *)(actions[i]));
return mlx5dv_create_flow(matcher, match_value,
num_actions, actions_attr);
#else
(void)matcher;
(void)match_value;
(void)num_actions;
- (void)actions_attr;
+ (void)actions;
return NULL;
#endif
}
@@ -455,31 +472,45 @@
#endif
}
-static struct ibv_flow_action *
-mlx5_glue_dv_create_flow_action_packet_reformat
- (struct ibv_context *ctx,
- size_t data_sz,
- void *data,
- enum mlx5dv_flow_action_packet_reformat_type reformat_type,
- enum mlx5dv_flow_table_type ft_type)
+static void *
+mlx5_glue_dv_create_flow_action_counter(void *counter_obj, uint32_t offset)
{
#ifdef HAVE_IBV_FLOW_DV_SUPPORT
- return mlx5dv_create_flow_action_packet_reformat(ctx,
- data_sz,
- data,
- reformat_type,
- ft_type);
+ struct mlx5dv_flow_action_attr *action;
+
+ (void)offset;
+ action = malloc(sizeof(*action));
+ if (!action)
+ return NULL;
+ action->type = MLX5DV_FLOW_ACTION_COUNTERS_DEVX;
+ action->obj = counter_obj;
+ return action;
#else
- (void)ctx;
- (void)data_sz;
- (void)data;
- (void)reformat_type;
- (void)ft_type;
+ (void)counter_obj;
+ (void)offset;
return NULL;
#endif
}
-static struct ibv_flow_action *
+static void *
+mlx5_glue_dv_create_flow_action_dest_ibv_qp(void *qp)
+{
+#ifdef HAVE_IBV_FLOW_DV_SUPPORT
+ struct mlx5dv_flow_action_attr *action;
+
+ action = malloc(sizeof(*action));
+ if (!action)
+ return NULL;
+ action->type = MLX5DV_FLOW_ACTION_DEST_IBV_QP;
+ action->obj = qp;
+ return action;
+#else
+ (void)qp;
+ return NULL;
+#endif
+}
+
+static void *
mlx5_glue_dv_create_flow_action_modify_header
(struct ibv_context *ctx,
size_t actions_sz,
@@ -487,8 +518,15 @@
enum mlx5dv_flow_table_type ft_type)
{
#ifdef HAVE_IBV_FLOW_DV_SUPPORT
- return mlx5dv_create_flow_action_modify_header(ctx, actions_sz,
- actions, ft_type);
+ struct mlx5dv_flow_action_attr *action;
+
+ action = malloc(sizeof(*action));
+ if (!action)
+ return NULL;
+ action->type = MLX5DV_FLOW_ACTION_IBV_FLOW_ACTION;
+ action->action = mlx5dv_create_flow_action_modify_header
+ (ctx, actions_sz, actions, ft_type);
+ return action;
#else
(void)ctx;
(void)actions_sz;
@@ -498,6 +536,50 @@
#endif
}
+static void *
+mlx5_glue_dv_create_flow_action_packet_reformat
+ (struct ibv_context *ctx,
+ size_t data_sz,
+ void *data,
+ enum mlx5dv_flow_action_packet_reformat_type reformat_type,
+ enum mlx5dv_flow_table_type ft_type)
+{
+#ifdef HAVE_IBV_FLOW_DV_SUPPORT
+ struct mlx5dv_flow_action_attr *action;
+
+ action = malloc(sizeof(*action));
+ if (!action)
+ return NULL;
+ action->type = MLX5DV_FLOW_ACTION_IBV_FLOW_ACTION;
+ action->action = mlx5dv_create_flow_action_packet_reformat
+ (ctx, data_sz, data, reformat_type, ft_type);
+ return action;
+#else
+ (void)ctx;
+ (void)data_sz;
+ (void)data;
+ (void)reformat_type;
+ (void)ft_type;
+ return NULL;
+#endif
+}
+
+static void *
+mlx5_glue_dv_create_flow_action_tag(uint32_t tag)
+{
+#ifdef HAVE_IBV_FLOW_DV_SUPPORT
+ struct mlx5dv_flow_action_attr *action;
+ action = malloc(sizeof(*action));
+ if (!action)
+ return NULL;
+ action->type = MLX5DV_FLOW_ACTION_TAG;
+ action->tag_value = tag;
+ return action;
+#endif
+ (void)tag;
+ return NULL;
+}
+
static struct ibv_context *
mlx5_glue_dv_open_device(struct ibv_device *device)
{
@@ -645,10 +727,15 @@
.dv_create_flow_matcher = mlx5_glue_dv_create_flow_matcher,
.dv_destroy_flow_matcher = mlx5_glue_dv_destroy_flow_matcher,
.dv_create_flow = mlx5_glue_dv_create_flow,
- .dv_create_flow_action_packet_reformat =
- mlx5_glue_dv_create_flow_action_packet_reformat,
+ .dv_create_flow_action_counter =
+ mlx5_glue_dv_create_flow_action_counter,
+ .dv_create_flow_action_dest_ibv_qp =
+ mlx5_glue_dv_create_flow_action_dest_ibv_qp,
.dv_create_flow_action_modify_header =
- mlx5_glue_dv_create_flow_action_modify_header,
+ mlx5_glue_dv_create_flow_action_modify_header,
+ .dv_create_flow_action_packet_reformat =
+ mlx5_glue_dv_create_flow_action_packet_reformat,
+ .dv_create_flow_action_tag = mlx5_glue_dv_create_flow_action_tag,
.dv_open_device = mlx5_glue_dv_open_device,
.devx_obj_create = mlx5_glue_devx_obj_create,
.devx_obj_destroy = mlx5_glue_devx_obj_destroy,
diff --git a/drivers/net/mlx5/mlx5_glue.h b/drivers/net/mlx5/mlx5_glue.h
index b118960..32487ea 100644
--- a/drivers/net/mlx5/mlx5_glue.h
+++ b/drivers/net/mlx5/mlx5_glue.h
@@ -55,6 +55,10 @@
enum mlx5dv_flow_table_type { flow_table_type = 0, };
#endif
+#ifndef HAVE_IBV_FLOW_DEVX_COUNTERS
+#define MLX5DV_FLOW_ACTION_COUNTERS_DEVX 0
+#endif
+
#ifndef HAVE_IBV_DEVX_OBJ
struct mlx5dv_devx_obj;
#endif
@@ -98,7 +102,7 @@ struct mlx5_glue {
struct ibv_flow *(*create_flow)(struct ibv_qp *qp,
struct ibv_flow_attr *flow);
int (*destroy_flow)(struct ibv_flow *flow_id);
- int (*destroy_flow_action)(struct ibv_flow_action *action);
+ int (*destroy_flow_action)(void *action);
struct ibv_qp *(*create_qp)(struct ibv_pd *pd,
struct ibv_qp_init_attr *qp_init_attr);
struct ibv_qp *(*create_qp_ex)
@@ -160,19 +164,17 @@ struct mlx5_glue {
int (*dv_destroy_flow_matcher)(struct mlx5dv_flow_matcher *matcher);
struct ibv_flow *(*dv_create_flow)(struct mlx5dv_flow_matcher *matcher,
struct mlx5dv_flow_match_parameters *match_value,
- size_t num_actions,
- struct mlx5dv_flow_action_attr *actions_attr);
- struct ibv_flow_action *(*dv_create_flow_action_packet_reformat)
- (struct ibv_context *ctx,
- size_t data_sz,
- void *data,
+ size_t num_actions, void *actions[]);
+ void *(*dv_create_flow_action_counter)(void *obj, uint32_t offset);
+ void *(*dv_create_flow_action_dest_ibv_qp)(void *qp);
+ void *(*dv_create_flow_action_modify_header)
+ (struct ibv_context *ctx, size_t actions_sz, uint64_t actions[],
+ enum mlx5dv_flow_table_type ft_type);
+ void *(*dv_create_flow_action_packet_reformat)
+ (struct ibv_context *ctx, size_t data_sz, void *data,
enum mlx5dv_flow_action_packet_reformat_type reformat_type,
enum mlx5dv_flow_table_type ft_type);
- struct ibv_flow_action *(*dv_create_flow_action_modify_header)
- (struct ibv_context *ctx,
- size_t actions_sz,
- uint64_t actions[],
- enum mlx5dv_flow_table_type ft_type);
+ void *(*dv_create_flow_action_tag)(uint32_t tag);
struct ibv_context *(*dv_open_device)(struct ibv_device *device);
struct mlx5dv_devx_obj *(*devx_obj_create)
(struct ibv_context *ctx,
--
1.8.3.1
^ permalink raw reply [flat|nested] 52+ messages in thread
* [dpdk-dev] [PATCH v4 1/3] net/mlx5: prepare Direct Verbs for Direct Rule
2019-04-04 9:54 ` [dpdk-dev] [PATCH v4 1/3] net/mlx5: prepare Direct Verbs for Direct Rule Ori Kam
@ 2019-04-04 9:54 ` Ori Kam
2019-04-12 23:51 ` dwilder
1 sibling, 0 replies; 52+ messages in thread
From: Ori Kam @ 2019-04-04 9:54 UTC (permalink / raw)
To: Matan Azrad, Yongseok Koh, Shahaf Shuler; +Cc: dev, Ori Kam, Slava Ovsiienko
This is the first patch of a series that is designed to enable the
Direct Rules API.
The main difference between Direct Verbs and Direct Rules from API
prespective, is that in Direct Rules each action has it's own create
function and the object itself is of type void.
In this patch I'm adding functions to generate actions that currenlty
are done without create action, and I'm changing the action type to be
void *, so in next patches only the glue functions will need to change.
Signed-off-by: Ori Kam <orika@mellanox.com>
Acked-by: Shahaf Shuler <shahafs@mellanox.com>
---
drivers/net/mlx5/Makefile | 2 +-
drivers/net/mlx5/meson.build | 2 +-
drivers/net/mlx5/mlx5.h | 2 +
drivers/net/mlx5/mlx5_flow.h | 17 +++-
drivers/net/mlx5/mlx5_flow_dv.c | 184 +++++++++++++++++++++++++++++-----------
drivers/net/mlx5/mlx5_glue.c | 141 ++++++++++++++++++++++++------
drivers/net/mlx5/mlx5_glue.h | 26 +++---
7 files changed, 280 insertions(+), 94 deletions(-)
diff --git a/drivers/net/mlx5/Makefile b/drivers/net/mlx5/Makefile
index ccd179c..0d20f0f 100644
--- a/drivers/net/mlx5/Makefile
+++ b/drivers/net/mlx5/Makefile
@@ -8,7 +8,7 @@ include $(RTE_SDK)/mk/rte.vars.mk
LIB = librte_pmd_mlx5.a
LIB_GLUE = $(LIB_GLUE_BASE).$(LIB_GLUE_VERSION)
LIB_GLUE_BASE = librte_pmd_mlx5_glue.so
-LIB_GLUE_VERSION = 19.02.0
+LIB_GLUE_VERSION = 19.05.0
# Sources.
SRCS-$(CONFIG_RTE_LIBRTE_MLX5_PMD) += mlx5.c
diff --git a/drivers/net/mlx5/meson.build b/drivers/net/mlx5/meson.build
index a429655..a4c684e 100644
--- a/drivers/net/mlx5/meson.build
+++ b/drivers/net/mlx5/meson.build
@@ -4,7 +4,7 @@
pmd_dlopen = (get_option('ibverbs_link') == 'dlopen')
LIB_GLUE_BASE = 'librte_pmd_mlx5_glue.so'
-LIB_GLUE_VERSION = '19.02.0'
+LIB_GLUE_VERSION = '19.05.0'
LIB_GLUE = LIB_GLUE_BASE + '.' + LIB_GLUE_VERSION
if pmd_dlopen
dpdk_conf.set('RTE_IBVERBS_LINK_DLOPEN', 1)
diff --git a/drivers/net/mlx5/mlx5.h b/drivers/net/mlx5/mlx5.h
index 47a7d75..784bf9b 100644
--- a/drivers/net/mlx5/mlx5.h
+++ b/drivers/net/mlx5/mlx5.h
@@ -309,6 +309,8 @@ struct mlx5_priv {
LIST_HEAD(matchers, mlx5_flow_dv_matcher) matchers;
LIST_HEAD(encap_decap, mlx5_flow_dv_encap_decap_resource) encaps_decaps;
LIST_HEAD(modify_cmd, mlx5_flow_dv_modify_hdr_resource) modify_cmds;
+ LIST_HEAD(tag, mlx5_flow_dv_tag_resource) tags;
+ /* Tags resources cache. */
uint32_t link_speed_capa; /* Link speed capabilities. */
struct mlx5_xstats_ctrl xstats_ctrl; /* Extended stats control. */
struct mlx5_stats_ctrl stats_ctrl; /* Stats control. */
diff --git a/drivers/net/mlx5/mlx5_flow.h b/drivers/net/mlx5/mlx5_flow.h
index e1e798b..4f69ae2 100644
--- a/drivers/net/mlx5/mlx5_flow.h
+++ b/drivers/net/mlx5/mlx5_flow.h
@@ -214,7 +214,7 @@ struct mlx5_flow_dv_encap_decap_resource {
LIST_ENTRY(mlx5_flow_dv_encap_decap_resource) next;
/* Pointer to next element. */
rte_atomic32_t refcnt; /**< Reference counter. */
- struct ibv_flow_action *verbs_action;
+ void *verbs_action;
/**< Verbs encap/decap action object. */
uint8_t buf[MLX5_ENCAP_MAX_LEN];
size_t size;
@@ -222,6 +222,16 @@ struct mlx5_flow_dv_encap_decap_resource {
uint8_t ft_type;
};
+/* Tag resource structure. */
+struct mlx5_flow_dv_tag_resource {
+ LIST_ENTRY(mlx5_flow_dv_tag_resource) next;
+ /* Pointer to next element. */
+ rte_atomic32_t refcnt; /**< Reference counter. */
+ void *action;
+ /**< Verbs tag action object. */
+ uint32_t tag; /**< the tag value. */
+};
+
/* Number of modification commands. */
#define MLX5_MODIFY_NUM 8
@@ -259,7 +269,7 @@ struct mlx5_flow_dv {
/**< Pointer to modify header resource in cache. */
struct ibv_flow *flow; /**< Installed flow. */
#ifdef HAVE_IBV_FLOW_DV_SUPPORT
- struct mlx5dv_flow_action_attr actions[MLX5_DV_MAX_NUMBER_OF_ACTIONS];
+ void *actions[MLX5_DV_MAX_NUMBER_OF_ACTIONS];
/**< Action list. */
#endif
int actions_n; /**< number of actions. */
@@ -332,6 +342,7 @@ struct mlx5_flow_counter {
};
uint64_t hits; /**< Number of packets matched by the rule. */
uint64_t bytes; /**< Number of bytes matched by the rule. */
+ void *action; /**< Pointer to the dv action. */
};
/* Flow structure. */
@@ -339,6 +350,8 @@ struct rte_flow {
TAILQ_ENTRY(rte_flow) next; /**< Pointer to the next flow structure. */
enum mlx5_flow_drv_type drv_type; /**< Drvier type. */
struct mlx5_flow_counter *counter; /**< Holds flow counter. */
+ struct mlx5_flow_dv_tag_resource *tag_resource;
+ /**< pointer to the tag action. */
struct rte_flow_action_rss rss;/**< RSS context. */
uint8_t key[MLX5_RSS_HASH_KEY_LEN]; /**< RSS hash key. */
uint16_t (*queue)[]; /**< Destination queues to redirect traffic to. */
diff --git a/drivers/net/mlx5/mlx5_flow_dv.c b/drivers/net/mlx5/mlx5_flow_dv.c
index ad84dea..d9e2ac9 100644
--- a/drivers/net/mlx5/mlx5_flow_dv.c
+++ b/drivers/net/mlx5/mlx5_flow_dv.c
@@ -32,7 +32,6 @@
#include "mlx5_prm.h"
#include "mlx5_glue.h"
#include "mlx5_flow.h"
-
#ifdef HAVE_IBV_FLOW_DV_SUPPORT
#ifndef HAVE_IBV_FLOW_DEVX_COUNTERS
@@ -1537,6 +1536,11 @@ struct field_modify_info modify_tcp[] = {
.id = id,
.dcs = dcs,
};
+ tmpl.action = mlx5_glue->dv_create_flow_action_counter(dcs->obj, 0);
+ if (!tmpl.action) {
+ ret = errno;
+ goto error_exit;
+ }
*cnt = tmpl;
LIST_INSERT_HEAD(&priv->flow_counters, cnt, next);
return cnt;
@@ -2828,6 +2832,97 @@ struct field_modify_info modify_tcp[] = {
}
/**
+ * Find existing tag resource or create and register a new one.
+ *
+ * @param dev[in, out]
+ * Pointer to rte_eth_dev structure.
+ * @param[in, out] resource
+ * Pointer to tag resource.
+ * @parm[in, out] dev_flow
+ * Pointer to the dev_flow.
+ * @param[out] error
+ * pointer to error structure.
+ *
+ * @return
+ * 0 on success otherwise -errno and errno is set.
+ */
+static int
+flow_dv_tag_resource_register
+ (struct rte_eth_dev *dev,
+ struct mlx5_flow_dv_tag_resource *resource,
+ struct mlx5_flow *dev_flow,
+ struct rte_flow_error *error)
+{
+ struct mlx5_priv *priv = dev->data->dev_private;
+ struct mlx5_flow_dv_tag_resource *cache_resource;
+
+ /* Lookup a matching resource from cache. */
+ LIST_FOREACH(cache_resource, &priv->tags, next) {
+ if (resource->tag == cache_resource->tag) {
+ DRV_LOG(DEBUG, "tag resource %p: refcnt %d++",
+ (void *)cache_resource,
+ rte_atomic32_read(&cache_resource->refcnt));
+ rte_atomic32_inc(&cache_resource->refcnt);
+ dev_flow->flow->tag_resource = cache_resource;
+ return 0;
+ }
+ }
+ /* Register new resource. */
+ cache_resource = rte_calloc(__func__, 1, sizeof(*cache_resource), 0);
+ if (!cache_resource)
+ return rte_flow_error_set(error, ENOMEM,
+ RTE_FLOW_ERROR_TYPE_UNSPECIFIED, NULL,
+ "cannot allocate resource memory");
+ *cache_resource = *resource;
+ cache_resource->action = mlx5_glue->dv_create_flow_action_tag
+ (resource->tag);
+ if (!cache_resource->action) {
+ rte_free(cache_resource);
+ return rte_flow_error_set(error, ENOMEM,
+ RTE_FLOW_ERROR_TYPE_UNSPECIFIED,
+ NULL, "cannot create action");
+ }
+ rte_atomic32_init(&cache_resource->refcnt);
+ rte_atomic32_inc(&cache_resource->refcnt);
+ LIST_INSERT_HEAD(&priv->tags, cache_resource, next);
+ dev_flow->flow->tag_resource = cache_resource;
+ DRV_LOG(DEBUG, "new tag resource %p: refcnt %d++",
+ (void *)cache_resource,
+ rte_atomic32_read(&cache_resource->refcnt));
+ return 0;
+}
+
+/**
+ * Release the tag.
+ *
+ * @param dev
+ * Pointer to Ethernet device.
+ * @param flow
+ * Pointer to mlx5_flow.
+ *
+ * @return
+ * 1 while a reference on it exists, 0 when freed.
+ */
+static int
+flow_dv_tag_release(struct rte_eth_dev *dev,
+ struct mlx5_flow_dv_tag_resource *tag)
+{
+ assert(tag);
+ DRV_LOG(DEBUG, "port %u tag %p: refcnt %d--",
+ dev->data->port_id, (void *)tag,
+ rte_atomic32_read(&tag->refcnt));
+ if (rte_atomic32_dec_and_test(&tag->refcnt)) {
+ claim_zero(mlx5_glue->destroy_flow_action(tag->action));
+ LIST_REMOVE(tag, next);
+ DRV_LOG(DEBUG, "port %u tag %p: removed",
+ dev->data->port_id, (void *)tag);
+ rte_free(tag);
+ return 0;
+ }
+ return 1;
+}
+
+/**
* Fill the flow with DV spec.
*
* @param[in] dev
@@ -2872,6 +2967,7 @@ struct field_modify_info modify_tcp[] = {
MLX5DV_FLOW_TABLE_TYPE_NIC_RX
};
union flow_dv_attr flow_attr = { .attr = 0 };
+ struct mlx5_flow_dv_tag_resource tag_resource;
if (priority == MLX5_FLOW_PRIO_RSVD)
priority = priv->config.flow_prio - 1;
@@ -2886,26 +2982,29 @@ struct field_modify_info modify_tcp[] = {
case RTE_FLOW_ACTION_TYPE_VOID:
break;
case RTE_FLOW_ACTION_TYPE_FLAG:
- dev_flow->dv.actions[actions_n].type =
- MLX5DV_FLOW_ACTION_TAG;
- dev_flow->dv.actions[actions_n].tag_value =
+ tag_resource.tag =
mlx5_flow_mark_set(MLX5_FLOW_MARK_DEFAULT);
- actions_n++;
+ if (!flow->tag_resource)
+ if (flow_dv_tag_resource_register
+ (dev, &tag_resource, dev_flow, error))
+ return errno;
+ dev_flow->dv.actions[actions_n++] =
+ flow->tag_resource->action;
action_flags |= MLX5_FLOW_ACTION_FLAG;
break;
case RTE_FLOW_ACTION_TYPE_MARK:
- dev_flow->dv.actions[actions_n].type =
- MLX5DV_FLOW_ACTION_TAG;
- dev_flow->dv.actions[actions_n].tag_value =
- mlx5_flow_mark_set
- (((const struct rte_flow_action_mark *)
- (actions->conf))->id);
- actions_n++;
+ tag_resource.tag = mlx5_flow_mark_set
+ (((const struct rte_flow_action_mark *)
+ (actions->conf))->id);
+ if (!flow->tag_resource)
+ if (flow_dv_tag_resource_register
+ (dev, &tag_resource, dev_flow, error))
+ return errno;
+ dev_flow->dv.actions[actions_n++] =
+ flow->tag_resource->action;
action_flags |= MLX5_FLOW_ACTION_MARK;
break;
case RTE_FLOW_ACTION_TYPE_DROP:
- dev_flow->dv.actions[actions_n].type =
- MLX5DV_FLOW_ACTION_DROP;
action_flags |= MLX5_FLOW_ACTION_DROP;
break;
case RTE_FLOW_ACTION_TYPE_QUEUE:
@@ -2933,17 +3032,13 @@ struct field_modify_info modify_tcp[] = {
rte_errno = ENOTSUP;
goto cnt_err;
}
- flow->counter =
- flow_dv_counter_new(dev,
- count->shared, count->id);
+ flow->counter = flow_dv_counter_new(dev, count->shared,
+ count->id);
if (flow->counter == NULL)
goto cnt_err;
- dev_flow->dv.actions[actions_n].type =
- MLX5DV_FLOW_ACTION_COUNTERS_DEVX;
- dev_flow->dv.actions[actions_n].obj =
- flow->counter->dcs->obj;
+ dev_flow->dv.actions[actions_n++] =
+ flow->counter->action;
action_flags |= MLX5_FLOW_ACTION_COUNT;
- ++actions_n;
break;
cnt_err:
if (rte_errno == ENOTSUP)
@@ -2964,11 +3059,8 @@ struct field_modify_info modify_tcp[] = {
if (flow_dv_create_action_l2_encap(dev, actions,
dev_flow, error))
return -rte_errno;
- dev_flow->dv.actions[actions_n].type =
- MLX5DV_FLOW_ACTION_IBV_FLOW_ACTION;
- dev_flow->dv.actions[actions_n].action =
+ dev_flow->dv.actions[actions_n++] =
dev_flow->dv.encap_decap->verbs_action;
- actions_n++;
action_flags |= actions->type ==
RTE_FLOW_ACTION_TYPE_VXLAN_ENCAP ?
MLX5_FLOW_ACTION_VXLAN_ENCAP :
@@ -2979,11 +3071,8 @@ struct field_modify_info modify_tcp[] = {
if (flow_dv_create_action_l2_decap(dev, dev_flow,
error))
return -rte_errno;
- dev_flow->dv.actions[actions_n].type =
- MLX5DV_FLOW_ACTION_IBV_FLOW_ACTION;
- dev_flow->dv.actions[actions_n].action =
+ dev_flow->dv.actions[actions_n++] =
dev_flow->dv.encap_decap->verbs_action;
- actions_n++;
action_flags |= actions->type ==
RTE_FLOW_ACTION_TYPE_VXLAN_DECAP ?
MLX5_FLOW_ACTION_VXLAN_DECAP :
@@ -2995,9 +3084,7 @@ struct field_modify_info modify_tcp[] = {
if (flow_dv_create_action_raw_encap
(dev, actions, dev_flow, attr, error))
return -rte_errno;
- dev_flow->dv.actions[actions_n].type =
- MLX5DV_FLOW_ACTION_IBV_FLOW_ACTION;
- dev_flow->dv.actions[actions_n].action =
+ dev_flow->dv.actions[actions_n++] =
dev_flow->dv.encap_decap->verbs_action;
} else {
/* Handle encap without preceding decap. */
@@ -3005,12 +3092,9 @@ struct field_modify_info modify_tcp[] = {
dev_flow,
error))
return -rte_errno;
- dev_flow->dv.actions[actions_n].type =
- MLX5DV_FLOW_ACTION_IBV_FLOW_ACTION;
- dev_flow->dv.actions[actions_n].action =
+ dev_flow->dv.actions[actions_n++] =
dev_flow->dv.encap_decap->verbs_action;
}
- actions_n++;
action_flags |= MLX5_FLOW_ACTION_RAW_ENCAP;
break;
case RTE_FLOW_ACTION_TYPE_RAW_DECAP:
@@ -3025,11 +3109,8 @@ struct field_modify_info modify_tcp[] = {
dev_flow,
error))
return -rte_errno;
- dev_flow->dv.actions[actions_n].type =
- MLX5DV_FLOW_ACTION_IBV_FLOW_ACTION;
- dev_flow->dv.actions[actions_n].action =
+ dev_flow->dv.actions[actions_n++] =
dev_flow->dv.encap_decap->verbs_action;
- actions_n++;
}
/* If decap is followed by encap, handle it at encap. */
action_flags |= MLX5_FLOW_ACTION_RAW_DECAP;
@@ -3098,11 +3179,8 @@ struct field_modify_info modify_tcp[] = {
dev_flow,
error))
return -rte_errno;
- dev_flow->dv.actions[actions_n].type =
- MLX5DV_FLOW_ACTION_IBV_FLOW_ACTION;
- dev_flow->dv.actions[actions_n].action =
+ dev_flow->dv.actions[actions_n++] =
dev_flow->dv.modify_hdr->verbs_action;
- actions_n++;
}
break;
default:
@@ -3277,9 +3355,9 @@ struct field_modify_info modify_tcp[] = {
"cannot get drop hash queue");
goto error;
}
- dv->actions[n].type = MLX5DV_FLOW_ACTION_DEST_IBV_QP;
- dv->actions[n].qp = dv->hrxq->qp;
- n++;
+ dv->actions[n++] =
+ mlx5_glue->dv_create_flow_action_dest_ibv_qp
+ (dv->hrxq->qp);
} else if (flow->actions &
(MLX5_FLOW_ACTION_QUEUE | MLX5_FLOW_ACTION_RSS)) {
struct mlx5_hrxq *hrxq;
@@ -3304,9 +3382,9 @@ struct field_modify_info modify_tcp[] = {
goto error;
}
dv->hrxq = hrxq;
- dv->actions[n].type = MLX5DV_FLOW_ACTION_DEST_IBV_QP;
- dv->actions[n].qp = hrxq->qp;
- n++;
+ dv->actions[n++] =
+ mlx5_glue->dv_create_flow_action_dest_ibv_qp
+ (dv->hrxq->qp);
}
dv->flow =
mlx5_glue->dv_create_flow(dv->matcher->matcher_object,
@@ -3484,6 +3562,10 @@ struct field_modify_info modify_tcp[] = {
flow_dv_counter_release(flow->counter);
flow->counter = NULL;
}
+ if (flow->tag_resource) {
+ flow_dv_tag_release(dev, flow->tag_resource);
+ flow->tag_resource = NULL;
+ }
while (!LIST_EMPTY(&flow->dev_flows)) {
dev_flow = LIST_FIRST(&flow->dev_flows);
LIST_REMOVE(dev_flow, next);
diff --git a/drivers/net/mlx5/mlx5_glue.c b/drivers/net/mlx5/mlx5_glue.c
index c817d86..4b5aade 100644
--- a/drivers/net/mlx5/mlx5_glue.c
+++ b/drivers/net/mlx5/mlx5_glue.c
@@ -175,10 +175,20 @@
}
static int
-mlx5_glue_destroy_flow_action(struct ibv_flow_action *action)
+mlx5_glue_destroy_flow_action(void *action)
{
#ifdef HAVE_IBV_FLOW_DV_SUPPORT
- return ibv_destroy_flow_action(action);
+ struct mlx5dv_flow_action_attr *attr = action;
+ int res = 0;
+ switch (attr->type) {
+ case MLX5DV_FLOW_ACTION_TAG:
+ break;
+ default:
+ res = ibv_destroy_flow_action(attr->action);
+ break;
+ }
+ free(action);
+ return res;
#else
(void)action;
return ENOTSUP;
@@ -430,16 +440,23 @@
mlx5_glue_dv_create_flow(struct mlx5dv_flow_matcher *matcher,
struct mlx5dv_flow_match_parameters *match_value,
size_t num_actions,
- struct mlx5dv_flow_action_attr *actions_attr)
+ void *actions[])
{
#ifdef HAVE_IBV_FLOW_DV_SUPPORT
+ struct mlx5dv_flow_action_attr actions_attr[8];
+
+ if (num_actions > 8)
+ return NULL;
+ for (size_t i = 0; i < num_actions; i++)
+ actions_attr[i] =
+ *((struct mlx5dv_flow_action_attr *)(actions[i]));
return mlx5dv_create_flow(matcher, match_value,
num_actions, actions_attr);
#else
(void)matcher;
(void)match_value;
(void)num_actions;
- (void)actions_attr;
+ (void)actions;
return NULL;
#endif
}
@@ -455,31 +472,45 @@
#endif
}
-static struct ibv_flow_action *
-mlx5_glue_dv_create_flow_action_packet_reformat
- (struct ibv_context *ctx,
- size_t data_sz,
- void *data,
- enum mlx5dv_flow_action_packet_reformat_type reformat_type,
- enum mlx5dv_flow_table_type ft_type)
+static void *
+mlx5_glue_dv_create_flow_action_counter(void *counter_obj, uint32_t offset)
{
#ifdef HAVE_IBV_FLOW_DV_SUPPORT
- return mlx5dv_create_flow_action_packet_reformat(ctx,
- data_sz,
- data,
- reformat_type,
- ft_type);
+ struct mlx5dv_flow_action_attr *action;
+
+ (void)offset;
+ action = malloc(sizeof(*action));
+ if (!action)
+ return NULL;
+ action->type = MLX5DV_FLOW_ACTION_COUNTERS_DEVX;
+ action->obj = counter_obj;
+ return action;
#else
- (void)ctx;
- (void)data_sz;
- (void)data;
- (void)reformat_type;
- (void)ft_type;
+ (void)counter_obj;
+ (void)offset;
return NULL;
#endif
}
-static struct ibv_flow_action *
+static void *
+mlx5_glue_dv_create_flow_action_dest_ibv_qp(void *qp)
+{
+#ifdef HAVE_IBV_FLOW_DV_SUPPORT
+ struct mlx5dv_flow_action_attr *action;
+
+ action = malloc(sizeof(*action));
+ if (!action)
+ return NULL;
+ action->type = MLX5DV_FLOW_ACTION_DEST_IBV_QP;
+ action->obj = qp;
+ return action;
+#else
+ (void)qp;
+ return NULL;
+#endif
+}
+
+static void *
mlx5_glue_dv_create_flow_action_modify_header
(struct ibv_context *ctx,
size_t actions_sz,
@@ -487,8 +518,15 @@
enum mlx5dv_flow_table_type ft_type)
{
#ifdef HAVE_IBV_FLOW_DV_SUPPORT
- return mlx5dv_create_flow_action_modify_header(ctx, actions_sz,
- actions, ft_type);
+ struct mlx5dv_flow_action_attr *action;
+
+ action = malloc(sizeof(*action));
+ if (!action)
+ return NULL;
+ action->type = MLX5DV_FLOW_ACTION_IBV_FLOW_ACTION;
+ action->action = mlx5dv_create_flow_action_modify_header
+ (ctx, actions_sz, actions, ft_type);
+ return action;
#else
(void)ctx;
(void)actions_sz;
@@ -498,6 +536,50 @@
#endif
}
+static void *
+mlx5_glue_dv_create_flow_action_packet_reformat
+ (struct ibv_context *ctx,
+ size_t data_sz,
+ void *data,
+ enum mlx5dv_flow_action_packet_reformat_type reformat_type,
+ enum mlx5dv_flow_table_type ft_type)
+{
+#ifdef HAVE_IBV_FLOW_DV_SUPPORT
+ struct mlx5dv_flow_action_attr *action;
+
+ action = malloc(sizeof(*action));
+ if (!action)
+ return NULL;
+ action->type = MLX5DV_FLOW_ACTION_IBV_FLOW_ACTION;
+ action->action = mlx5dv_create_flow_action_packet_reformat
+ (ctx, data_sz, data, reformat_type, ft_type);
+ return action;
+#else
+ (void)ctx;
+ (void)data_sz;
+ (void)data;
+ (void)reformat_type;
+ (void)ft_type;
+ return NULL;
+#endif
+}
+
+static void *
+mlx5_glue_dv_create_flow_action_tag(uint32_t tag)
+{
+#ifdef HAVE_IBV_FLOW_DV_SUPPORT
+ struct mlx5dv_flow_action_attr *action;
+ action = malloc(sizeof(*action));
+ if (!action)
+ return NULL;
+ action->type = MLX5DV_FLOW_ACTION_TAG;
+ action->tag_value = tag;
+ return action;
+#endif
+ (void)tag;
+ return NULL;
+}
+
static struct ibv_context *
mlx5_glue_dv_open_device(struct ibv_device *device)
{
@@ -645,10 +727,15 @@
.dv_create_flow_matcher = mlx5_glue_dv_create_flow_matcher,
.dv_destroy_flow_matcher = mlx5_glue_dv_destroy_flow_matcher,
.dv_create_flow = mlx5_glue_dv_create_flow,
- .dv_create_flow_action_packet_reformat =
- mlx5_glue_dv_create_flow_action_packet_reformat,
+ .dv_create_flow_action_counter =
+ mlx5_glue_dv_create_flow_action_counter,
+ .dv_create_flow_action_dest_ibv_qp =
+ mlx5_glue_dv_create_flow_action_dest_ibv_qp,
.dv_create_flow_action_modify_header =
- mlx5_glue_dv_create_flow_action_modify_header,
+ mlx5_glue_dv_create_flow_action_modify_header,
+ .dv_create_flow_action_packet_reformat =
+ mlx5_glue_dv_create_flow_action_packet_reformat,
+ .dv_create_flow_action_tag = mlx5_glue_dv_create_flow_action_tag,
.dv_open_device = mlx5_glue_dv_open_device,
.devx_obj_create = mlx5_glue_devx_obj_create,
.devx_obj_destroy = mlx5_glue_devx_obj_destroy,
diff --git a/drivers/net/mlx5/mlx5_glue.h b/drivers/net/mlx5/mlx5_glue.h
index b118960..32487ea 100644
--- a/drivers/net/mlx5/mlx5_glue.h
+++ b/drivers/net/mlx5/mlx5_glue.h
@@ -55,6 +55,10 @@
enum mlx5dv_flow_table_type { flow_table_type = 0, };
#endif
+#ifndef HAVE_IBV_FLOW_DEVX_COUNTERS
+#define MLX5DV_FLOW_ACTION_COUNTERS_DEVX 0
+#endif
+
#ifndef HAVE_IBV_DEVX_OBJ
struct mlx5dv_devx_obj;
#endif
@@ -98,7 +102,7 @@ struct mlx5_glue {
struct ibv_flow *(*create_flow)(struct ibv_qp *qp,
struct ibv_flow_attr *flow);
int (*destroy_flow)(struct ibv_flow *flow_id);
- int (*destroy_flow_action)(struct ibv_flow_action *action);
+ int (*destroy_flow_action)(void *action);
struct ibv_qp *(*create_qp)(struct ibv_pd *pd,
struct ibv_qp_init_attr *qp_init_attr);
struct ibv_qp *(*create_qp_ex)
@@ -160,19 +164,17 @@ struct mlx5_glue {
int (*dv_destroy_flow_matcher)(struct mlx5dv_flow_matcher *matcher);
struct ibv_flow *(*dv_create_flow)(struct mlx5dv_flow_matcher *matcher,
struct mlx5dv_flow_match_parameters *match_value,
- size_t num_actions,
- struct mlx5dv_flow_action_attr *actions_attr);
- struct ibv_flow_action *(*dv_create_flow_action_packet_reformat)
- (struct ibv_context *ctx,
- size_t data_sz,
- void *data,
+ size_t num_actions, void *actions[]);
+ void *(*dv_create_flow_action_counter)(void *obj, uint32_t offset);
+ void *(*dv_create_flow_action_dest_ibv_qp)(void *qp);
+ void *(*dv_create_flow_action_modify_header)
+ (struct ibv_context *ctx, size_t actions_sz, uint64_t actions[],
+ enum mlx5dv_flow_table_type ft_type);
+ void *(*dv_create_flow_action_packet_reformat)
+ (struct ibv_context *ctx, size_t data_sz, void *data,
enum mlx5dv_flow_action_packet_reformat_type reformat_type,
enum mlx5dv_flow_table_type ft_type);
- struct ibv_flow_action *(*dv_create_flow_action_modify_header)
- (struct ibv_context *ctx,
- size_t actions_sz,
- uint64_t actions[],
- enum mlx5dv_flow_table_type ft_type);
+ void *(*dv_create_flow_action_tag)(uint32_t tag);
struct ibv_context *(*dv_open_device)(struct ibv_device *device);
struct mlx5dv_devx_obj *(*devx_obj_create)
(struct ibv_context *ctx,
--
1.8.3.1
^ permalink raw reply [flat|nested] 52+ messages in thread
* [dpdk-dev] [PATCH v4 2/3] net/mlx5: add Direct Rules API
2019-04-04 9:54 ` [dpdk-dev] [PATCH v4 0/3] net/mlx5: Add Direct Rule support Ori Kam
2019-04-04 9:54 ` Ori Kam
2019-04-04 9:54 ` [dpdk-dev] [PATCH v4 1/3] net/mlx5: prepare Direct Verbs for Direct Rule Ori Kam
@ 2019-04-04 9:54 ` Ori Kam
2019-04-04 9:54 ` Ori Kam
2019-04-04 9:54 ` [dpdk-dev] [PATCH v4 3/3] net/mlx5: add jump action support for NIC Ori Kam
2019-04-04 11:01 ` [dpdk-dev] [PATCH v4 0/3] net/mlx5: Add Direct Rule support Shahaf Shuler
4 siblings, 1 reply; 52+ messages in thread
From: Ori Kam @ 2019-04-04 9:54 UTC (permalink / raw)
To: Matan Azrad, Yongseok Koh, Shahaf Shuler; +Cc: dev, Ori Kam, Slava Ovsiienko
Adds calls to the Direct Rules API inside the glue functions.
Due to difference in parameters between the Direct Rules and Direct
Verbs some of the glue functions API was updated.
Signed-off-by: Ori Kam <orika@mellanox.com>
Acked-by: Shahaf Shuler <shahafs@mellanox.com>
---
drivers/net/mlx5/Makefile | 5 ++
drivers/net/mlx5/mlx5.c | 16 ++++
drivers/net/mlx5/mlx5.h | 15 ++++
drivers/net/mlx5/mlx5_flow.c | 1 +
drivers/net/mlx5/mlx5_flow.h | 6 +-
drivers/net/mlx5/mlx5_flow_dv.c | 103 ++++++++++++++++++++----
drivers/net/mlx5/mlx5_glue.c | 170 ++++++++++++++++++++++++++++++++++------
drivers/net/mlx5/mlx5_glue.h | 31 +++++---
drivers/net/mlx5/mlx5_prm.h | 24 +++++-
9 files changed, 318 insertions(+), 53 deletions(-)
diff --git a/drivers/net/mlx5/Makefile b/drivers/net/mlx5/Makefile
index 0d20f0f..93bc869 100644
--- a/drivers/net/mlx5/Makefile
+++ b/drivers/net/mlx5/Makefile
@@ -156,6 +156,11 @@ mlx5_autoconf.h.new: $(RTE_SDK)/buildtools/auto-config-h.sh
func mlx5dv_create_flow_action_packet_reformat \
$(AUTOCONF_OUTPUT)
$Q sh -- '$<' '$@' \
+ HAVE_MLX5DV_DR \
+ infiniband/mlx5dv.h \
+ enum MLX5DV_DR_NS_TYPE_TERMINATING \
+ $(AUTOCONF_OUTPUT)
+ $Q sh -- '$<' '$@' \
HAVE_IBV_DEVX_OBJ \
infiniband/mlx5dv.h \
func mlx5dv_devx_obj_create \
diff --git a/drivers/net/mlx5/mlx5.c b/drivers/net/mlx5/mlx5.c
index 4044505..65aa9cf 100644
--- a/drivers/net/mlx5/mlx5.c
+++ b/drivers/net/mlx5/mlx5.c
@@ -1483,6 +1483,22 @@ struct mlx5_dev_spawn_data {
priv->tcf_context = NULL;
}
}
+#ifdef HAVE_MLX5DV_DR
+ priv->rx_ns = mlx5dv_dr_create_ns
+ (sh->ctx, MLX5DV_DR_NS_DOMAIN_INGRESS_BYPASS);
+ if (priv->rx_ns == NULL) {
+ DRV_LOG(ERR, "mlx5dv_dr_create_ns failed");
+ err = errno;
+ goto error;
+ }
+ priv->tx_ns = mlx5dv_dr_create_ns(sh->ctx,
+ MLX5DV_DR_NS_DOMAIN_EGRESS_BYPASS);
+ if (priv->tx_ns == NULL) {
+ DRV_LOG(ERR, "mlx5dv_dr_create_ns failed");
+ err = errno;
+ goto error;
+ }
+#endif
TAILQ_INIT(&priv->flows);
TAILQ_INIT(&priv->ctrl_flows);
/* Hint libmlx5 to use PMD allocator for data plane resources */
diff --git a/drivers/net/mlx5/mlx5.h b/drivers/net/mlx5/mlx5.h
index 784bf9b..d4963cb 100644
--- a/drivers/net/mlx5/mlx5.h
+++ b/drivers/net/mlx5/mlx5.h
@@ -259,6 +259,15 @@ struct mlx5_ibv_shared {
struct mlx5_ibv_shared_port port[]; /* per device port data array. */
};
+/* Table structure. */
+struct mlx5_flow_tbl_resource {
+ void *obj; /**< Pointer to DR table object. */
+ rte_atomic32_t refcnt; /**< Reference counter. */
+};
+
+#define MLX5_MAX_TABLES 1024
+#define MLX5_GROUP_FACTOR 1
+
struct mlx5_priv {
LIST_ENTRY(mlx5_priv) mem_event_cb;
/**< Called by memory event callback. */
@@ -326,6 +335,12 @@ struct mlx5_priv {
/* UAR same-page access control required in 32bit implementations. */
#endif
struct mlx5_flow_tcf_context *tcf_context; /* TC flower context. */
+ void *rx_ns; /* RX Direct Rules name space handle. */
+ struct mlx5_flow_tbl_resource rx_tbl[MLX5_MAX_TABLES];
+ /* RX Direct Rules tables. */
+ void *tx_ns; /* TX Direct Rules name space handle. */
+ struct mlx5_flow_tbl_resource tx_tbl[MLX5_MAX_TABLES];
+ /* TX Direct Rules tables/ */
};
#define PORT_ID(priv) ((priv)->dev_data->port_id)
diff --git a/drivers/net/mlx5/mlx5_flow.c b/drivers/net/mlx5/mlx5_flow.c
index bc6a7c1..9dc492a 100644
--- a/drivers/net/mlx5/mlx5_flow.c
+++ b/drivers/net/mlx5/mlx5_flow.c
@@ -2084,6 +2084,7 @@ uint32_t mlx5_flow_adjust_priority(struct rte_eth_dev *dev, int32_t priority,
flow_size += RTE_ALIGN_CEIL(sizeof(uint16_t), sizeof(void *));
flow = rte_calloc(__func__, 1, flow_size, 0);
flow->drv_type = flow_get_drv_type(dev, attr);
+ flow->ingress = attr->ingress;
assert(flow->drv_type > MLX5_FLOW_TYPE_MIN &&
flow->drv_type < MLX5_FLOW_TYPE_MAX);
flow->queue = (void *)(flow + 1);
diff --git a/drivers/net/mlx5/mlx5_flow.h b/drivers/net/mlx5/mlx5_flow.h
index 4f69ae2..8ba37a0 100644
--- a/drivers/net/mlx5/mlx5_flow.h
+++ b/drivers/net/mlx5/mlx5_flow.h
@@ -204,6 +204,7 @@ struct mlx5_flow_dv_matcher {
uint16_t crc; /**< CRC of key. */
uint16_t priority; /**< Priority of matcher. */
uint8_t egress; /**< Egress matcher. */
+ uint32_t group; /**< The matcher group. */
struct mlx5_flow_dv_match_params mask; /**< Matcher mask. */
};
@@ -220,6 +221,7 @@ struct mlx5_flow_dv_encap_decap_resource {
size_t size;
uint8_t reformat_type;
uint8_t ft_type;
+ uint64_t flags; /**< Flags for RDMA API. */
};
/* Tag resource structure. */
@@ -348,7 +350,7 @@ struct mlx5_flow_counter {
/* Flow structure. */
struct rte_flow {
TAILQ_ENTRY(rte_flow) next; /**< Pointer to the next flow structure. */
- enum mlx5_flow_drv_type drv_type; /**< Drvier type. */
+ enum mlx5_flow_drv_type drv_type; /**< Driver type. */
struct mlx5_flow_counter *counter; /**< Holds flow counter. */
struct mlx5_flow_dv_tag_resource *tag_resource;
/**< pointer to the tag action. */
@@ -360,6 +362,8 @@ struct rte_flow {
uint64_t actions;
/**< Bit-fields of detected actions, see MLX5_FLOW_ACTION_*. */
struct mlx5_fdir *fdir; /**< Pointer to associated FDIR if any. */
+ uint8_t ingress; /**< 1 if the flow is ingress. */
+ uint32_t group; /**< The group index. */
};
typedef int (*mlx5_flow_validate_t)(struct rte_eth_dev *dev,
diff --git a/drivers/net/mlx5/mlx5_flow_dv.c b/drivers/net/mlx5/mlx5_flow_dv.c
index d9e2ac9..dd5b541 100644
--- a/drivers/net/mlx5/mlx5_flow_dv.c
+++ b/drivers/net/mlx5/mlx5_flow_dv.c
@@ -809,11 +809,20 @@ struct field_modify_info modify_tcp[] = {
{
struct mlx5_priv *priv = dev->data->dev_private;
struct mlx5_flow_dv_encap_decap_resource *cache_resource;
+ struct rte_flow *flow = dev_flow->flow;
+ struct mlx5dv_dr_ns *ns;
+
+ resource->flags = flow->group ? 0 : 1;
+ if (flow->ingress)
+ ns = priv->rx_ns;
+ else
+ ns = priv->tx_ns;
/* Lookup a matching resource from cache. */
LIST_FOREACH(cache_resource, &priv->encaps_decaps, next) {
if (resource->reformat_type == cache_resource->reformat_type &&
resource->ft_type == cache_resource->ft_type &&
+ resource->flags == cache_resource->flags &&
resource->size == cache_resource->size &&
!memcmp((const void *)resource->buf,
(const void *)cache_resource->buf,
@@ -835,10 +844,10 @@ struct field_modify_info modify_tcp[] = {
*cache_resource = *resource;
cache_resource->verbs_action =
mlx5_glue->dv_create_flow_action_packet_reformat
- (priv->sh->ctx, cache_resource->size,
- (cache_resource->size ? cache_resource->buf : NULL),
- cache_resource->reformat_type,
- cache_resource->ft_type);
+ (priv->sh->ctx, cache_resource->reformat_type,
+ cache_resource->ft_type, ns, cache_resource->flags,
+ cache_resource->size,
+ (cache_resource->size ? cache_resource->buf : NULL));
if (!cache_resource->verbs_action) {
rte_free(cache_resource);
return rte_flow_error_set(error, ENOMEM,
@@ -1442,6 +1451,10 @@ struct field_modify_info modify_tcp[] = {
struct mlx5_priv *priv = dev->data->dev_private;
struct mlx5_flow_dv_modify_hdr_resource *cache_resource;
+ struct mlx5dv_dr_ns *ns =
+ resource->ft_type == MLX5DV_FLOW_TABLE_TYPE_NIC_TX ?
+ priv->tx_ns : priv->rx_ns;
+
/* Lookup a matching resource from cache. */
LIST_FOREACH(cache_resource, &priv->modify_cmds, next) {
if (resource->ft_type == cache_resource->ft_type &&
@@ -1467,11 +1480,11 @@ struct field_modify_info modify_tcp[] = {
*cache_resource = *resource;
cache_resource->verbs_action =
mlx5_glue->dv_create_flow_action_modify_header
- (priv->sh->ctx,
+ (priv->sh->ctx, cache_resource->ft_type,
+ ns, 0,
cache_resource->actions_num *
sizeof(cache_resource->actions[0]),
- (uint64_t *)cache_resource->actions,
- cache_resource->ft_type);
+ (uint64_t *)cache_resource->actions);
if (!cache_resource->verbs_action) {
rte_free(cache_resource);
return rte_flow_error_set(error, ENOMEM,
@@ -1596,11 +1609,13 @@ struct field_modify_info modify_tcp[] = {
struct mlx5_priv *priv = dev->data->dev_private;
uint32_t priority_max = priv->config.flow_prio - 1;
+#ifdef HAVE_MLX5DV_DR
if (attributes->group)
return rte_flow_error_set(error, ENOTSUP,
RTE_FLOW_ERROR_TYPE_ATTR_GROUP,
NULL,
"groups is not supported");
+#endif
if (attributes->priority != MLX5_FLOW_PRIO_RSVD &&
attributes->priority >= priority_max)
return rte_flow_error_set(error, ENOTSUP,
@@ -2173,11 +2188,13 @@ struct field_modify_info modify_tcp[] = {
* Flow pattern to translate.
* @param[in] inner
* Item is inner pattern.
+ * @param[in] group
+ * The group to insert the rule.
*/
static void
flow_dv_translate_item_ipv4(void *matcher, void *key,
const struct rte_flow_item *item,
- int inner)
+ int inner, uint32_t group)
{
const struct rte_flow_item_ipv4 *ipv4_m = item->mask;
const struct rte_flow_item_ipv4 *ipv4_v = item->spec;
@@ -2204,7 +2221,10 @@ struct field_modify_info modify_tcp[] = {
outer_headers);
headers_v = MLX5_ADDR_OF(fte_match_param, key, outer_headers);
}
- MLX5_SET(fte_match_set_lyr_2_4, headers_m, ip_version, 0xf);
+ if (group == 0)
+ MLX5_SET(fte_match_set_lyr_2_4, headers_m, ip_version, 0xf);
+ else
+ MLX5_SET(fte_match_set_lyr_2_4, headers_m, ip_version, 0x4);
MLX5_SET(fte_match_set_lyr_2_4, headers_v, ip_version, 4);
if (!ipv4_v)
return;
@@ -2246,11 +2266,13 @@ struct field_modify_info modify_tcp[] = {
* Flow pattern to translate.
* @param[in] inner
* Item is inner pattern.
+ * @param[in] group
+ * The group to insert the rule.
*/
static void
flow_dv_translate_item_ipv6(void *matcher, void *key,
const struct rte_flow_item *item,
- int inner)
+ int inner, uint32_t group)
{
const struct rte_flow_item_ipv6 *ipv6_m = item->mask;
const struct rte_flow_item_ipv6 *ipv6_v = item->spec;
@@ -2287,7 +2309,10 @@ struct field_modify_info modify_tcp[] = {
outer_headers);
headers_v = MLX5_ADDR_OF(fte_match_param, key, outer_headers);
}
- MLX5_SET(fte_match_set_lyr_2_4, headers_m, ip_version, 0xf);
+ if (group == 0)
+ MLX5_SET(fte_match_set_lyr_2_4, headers_m, ip_version, 0xf);
+ else
+ MLX5_SET(fte_match_set_lyr_2_4, headers_m, ip_version, 0x6);
MLX5_SET(fte_match_set_lyr_2_4, headers_v, ip_version, 6);
if (!ipv6_v)
return;
@@ -2727,7 +2752,11 @@ struct field_modify_info modify_tcp[] = {
match_criteria_enable |=
(!HEADER_IS_ZERO(match_criteria, misc_parameters_2)) <<
MLX5_MATCH_CRITERIA_ENABLE_MISC2_BIT;
-
+#ifdef HAVE_MLX5DV_DR
+ match_criteria_enable |=
+ (!HEADER_IS_ZERO(match_criteria, misc_parameters_3)) <<
+ MLX5_MATCH_CRITERIA_ENABLE_MISC3_BIT;
+#endif
return match_criteria_enable;
}
@@ -2758,12 +2787,14 @@ struct field_modify_info modify_tcp[] = {
.type = IBV_FLOW_ATTR_NORMAL,
.match_mask = (void *)&matcher->mask,
};
+ struct mlx5_flow_tbl_resource *tbl = NULL;
/* Lookup from cache. */
LIST_FOREACH(cache_matcher, &priv->matchers, next) {
if (matcher->crc == cache_matcher->crc &&
matcher->priority == cache_matcher->priority &&
matcher->egress == cache_matcher->egress &&
+ matcher->group == cache_matcher->group &&
!memcmp((const void *)matcher->mask.buf,
(const void *)cache_matcher->mask.buf,
cache_matcher->mask.size)) {
@@ -2778,6 +2809,27 @@ struct field_modify_info modify_tcp[] = {
return 0;
}
}
+#ifdef HAVE_MLX5DV_DR
+ if (matcher->egress) {
+ tbl = &priv->tx_tbl[matcher->group];
+ if (!tbl->obj)
+ tbl->obj = mlx5_glue->dr_create_flow_tbl
+ (priv->tx_ns,
+ matcher->group * MLX5_GROUP_FACTOR);
+ } else {
+ tbl = &priv->rx_tbl[matcher->group];
+ if (!tbl->obj)
+ tbl->obj = mlx5_glue->dr_create_flow_tbl
+ (priv->rx_ns,
+ matcher->group * MLX5_GROUP_FACTOR);
+ }
+ if (!tbl->obj)
+ return rte_flow_error_set(error, ENOMEM,
+ RTE_FLOW_ERROR_TYPE_UNSPECIFIED,
+ NULL, "cannot create table");
+
+ rte_atomic32_inc(&tbl->refcnt);
+#endif
/* Register new matcher. */
cache_matcher = rte_calloc(__func__, 1, sizeof(*cache_matcher), 0);
if (!cache_matcher)
@@ -2791,9 +2843,16 @@ struct field_modify_info modify_tcp[] = {
if (matcher->egress)
dv_attr.flags |= IBV_FLOW_ATTR_FLAGS_EGRESS;
cache_matcher->matcher_object =
- mlx5_glue->dv_create_flow_matcher(priv->sh->ctx, &dv_attr);
+ mlx5_glue->dv_create_flow_matcher(priv->sh->ctx, &dv_attr,
+ tbl->obj);
if (!cache_matcher->matcher_object) {
rte_free(cache_matcher);
+#ifdef HAVE_MLX5DV_DR
+ if (rte_atomic32_dec_and_test(&tbl->refcnt)) {
+ mlx5_glue->dr_destroy_flow_tbl(tbl->obj);
+ tbl->obj = NULL;
+ }
+#endif
return rte_flow_error_set(error, ENOMEM,
RTE_FLOW_ERROR_TYPE_UNSPECIFIED,
NULL, "cannot create matcher");
@@ -2805,6 +2864,7 @@ struct field_modify_info modify_tcp[] = {
cache_matcher->priority,
cache_matcher->egress ? "tx" : "rx", (void *)cache_matcher,
rte_atomic32_read(&cache_matcher->refcnt));
+ rte_atomic32_inc(&tbl->refcnt);
return 0;
}
@@ -3226,7 +3286,7 @@ struct field_modify_info modify_tcp[] = {
break;
case RTE_FLOW_ITEM_TYPE_IPV4:
flow_dv_translate_item_ipv4(match_mask, match_value,
- items, tunnel);
+ items, tunnel, attr->group);
matcher.priority = MLX5_PRIORITY_MAP_L3;
dev_flow->dv.hash_fields |=
mlx5_flow_hashfields_adjust
@@ -3238,7 +3298,7 @@ struct field_modify_info modify_tcp[] = {
break;
case RTE_FLOW_ITEM_TYPE_IPV6:
flow_dv_translate_item_ipv6(match_mask, match_value,
- items, tunnel);
+ items, tunnel, attr->group);
matcher.priority = MLX5_PRIORITY_MAP_L3;
dev_flow->dv.hash_fields |=
mlx5_flow_hashfields_adjust
@@ -3316,6 +3376,7 @@ struct field_modify_info modify_tcp[] = {
matcher.priority = mlx5_flow_adjust_priority(dev, priority,
matcher.priority);
matcher.egress = attr->egress;
+ matcher.group = attr->group;
if (flow_dv_matcher_register(dev, &matcher, dev_flow, error))
return -rte_errno;
return 0;
@@ -3431,6 +3492,8 @@ struct field_modify_info modify_tcp[] = {
struct mlx5_flow *flow)
{
struct mlx5_flow_dv_matcher *matcher = flow->dv.matcher;
+ struct mlx5_priv *priv = dev->data->dev_private;
+ struct mlx5_flow_tbl_resource *tbl;
assert(matcher->matcher_object);
DRV_LOG(DEBUG, "port %u matcher %p: refcnt %d--",
@@ -3440,6 +3503,14 @@ struct field_modify_info modify_tcp[] = {
claim_zero(mlx5_glue->dv_destroy_flow_matcher
(matcher->matcher_object));
LIST_REMOVE(matcher, next);
+ if (matcher->egress)
+ tbl = &priv->tx_tbl[matcher->group];
+ else
+ tbl = &priv->rx_tbl[matcher->group];
+ if (rte_atomic32_dec_and_test(&tbl->refcnt)) {
+ mlx5_glue->dr_destroy_flow_tbl(tbl->obj);
+ tbl->obj = NULL;
+ }
rte_free(matcher);
DRV_LOG(DEBUG, "port %u matcher %p: removed",
dev->data->port_id, (void *)matcher);
@@ -3529,7 +3600,7 @@ struct field_modify_info modify_tcp[] = {
LIST_FOREACH(dev_flow, &flow->dev_flows, next) {
dv = &dev_flow->dv;
if (dv->flow) {
- claim_zero(mlx5_glue->destroy_flow(dv->flow));
+ claim_zero(mlx5_glue->dv_destroy_flow(dv->flow));
dv->flow = NULL;
}
if (dv->hrxq) {
diff --git a/drivers/net/mlx5/mlx5_glue.c b/drivers/net/mlx5/mlx5_glue.c
index 4b5aade..b0b144c 100644
--- a/drivers/net/mlx5/mlx5_glue.c
+++ b/drivers/net/mlx5/mlx5_glue.c
@@ -178,6 +178,9 @@
mlx5_glue_destroy_flow_action(void *action)
{
#ifdef HAVE_IBV_FLOW_DV_SUPPORT
+#ifdef HAVE_MLX5DV_DR
+ return mlx5dv_dr_destroy_action(action);
+#else
struct mlx5dv_flow_action_attr *attr = action;
int res = 0;
switch (attr->type) {
@@ -189,6 +192,7 @@
}
free(action);
return res;
+#endif
#else
(void)action;
return ENOTSUP;
@@ -365,6 +369,53 @@
return ibv_cq_ex_to_cq(cq);
}
+static void *
+mlx5_glue_dr_create_flow_tbl(void *ns, uint32_t level)
+{
+#ifdef HAVE_MLX5DV_DR
+ return mlx5dv_dr_create_ft(ns, level);
+#else
+ (void)ns;
+ (void)level;
+ return NULL;
+#endif
+}
+
+static int
+mlx5_glue_dr_destroy_flow_tbl(void *tbl)
+{
+#ifdef HAVE_MLX5DV_DR
+ return mlx5dv_dr_destroy_ft(tbl);
+#else
+ (void)tbl;
+ return 0;
+#endif
+}
+
+static void *
+mlx5_glue_dr_create_ns(struct ibv_context *ctx,
+ enum mlx5dv_dr_ns_domain domain)
+{
+#ifdef HAVE_MLX5DV_DR
+ return mlx5dv_dr_create_ns(ctx, domain);
+#else
+ (void)ctx;
+ (void)domain;
+ return NULL;
+#endif
+}
+
+static int
+mlx5_glue_dr_destroy_ns(void *ns)
+{
+#ifdef HAVE_MLX5DV_DR
+ return mlx5dv_dr_destroy_ns(ns);
+#else
+ (void)ns;
+ return 0;
+#endif
+}
+
static struct ibv_cq_ex *
mlx5_glue_dv_create_cq(struct ibv_context *context,
struct ibv_cq_init_attr_ex *cq_attr,
@@ -423,26 +474,40 @@
#endif
}
-static struct mlx5dv_flow_matcher *
+static void *
mlx5_glue_dv_create_flow_matcher(struct ibv_context *context,
- struct mlx5dv_flow_matcher_attr *matcher_attr)
+ struct mlx5dv_flow_matcher_attr *matcher_attr,
+ void *tbl)
{
#ifdef HAVE_IBV_FLOW_DV_SUPPORT
+#ifdef HAVE_MLX5DV_DR
+ (void)context;
+ return mlx5dv_dr_create_matcher(tbl, matcher_attr->priority,
+ matcher_attr->match_criteria_enable,
+ matcher_attr->match_mask);
+#else
+ (void)tbl;
return mlx5dv_create_flow_matcher(context, matcher_attr);
+#endif
#else
(void)context;
(void)matcher_attr;
+ (void)tbl;
return NULL;
#endif
}
-static struct ibv_flow *
-mlx5_glue_dv_create_flow(struct mlx5dv_flow_matcher *matcher,
- struct mlx5dv_flow_match_parameters *match_value,
+static void *
+mlx5_glue_dv_create_flow(void *matcher,
+ void *match_value,
size_t num_actions,
void *actions[])
{
#ifdef HAVE_IBV_FLOW_DV_SUPPORT
+#ifdef HAVE_MLX5DV_DR
+ return mlx5dv_dr_create_rule(matcher, match_value, num_actions,
+ (struct mlx5dv_dr_action **)actions);
+#else
struct mlx5dv_flow_action_attr actions_attr[8];
if (num_actions > 8)
@@ -452,6 +517,7 @@
*((struct mlx5dv_flow_action_attr *)(actions[i]));
return mlx5dv_create_flow(matcher, match_value,
num_actions, actions_attr);
+#endif
#else
(void)matcher;
(void)match_value;
@@ -461,21 +527,13 @@
#endif
}
-static int
-mlx5_glue_dv_destroy_flow_matcher(struct mlx5dv_flow_matcher *matcher)
-{
-#ifdef HAVE_IBV_FLOW_DV_SUPPORT
- return mlx5dv_destroy_flow_matcher(matcher);
-#else
- (void)matcher;
- return 0;
-#endif
-}
-
static void *
mlx5_glue_dv_create_flow_action_counter(void *counter_obj, uint32_t offset)
{
#ifdef HAVE_IBV_FLOW_DV_SUPPORT
+#ifdef HAVE_MLX5DV_DR
+ return mlx5dv_dr_create_action_devx_counter(counter_obj, offset);
+#else
struct mlx5dv_flow_action_attr *action;
(void)offset;
@@ -485,6 +543,7 @@
action->type = MLX5DV_FLOW_ACTION_COUNTERS_DEVX;
action->obj = counter_obj;
return action;
+#endif
#else
(void)counter_obj;
(void)offset;
@@ -496,6 +555,9 @@
mlx5_glue_dv_create_flow_action_dest_ibv_qp(void *qp)
{
#ifdef HAVE_IBV_FLOW_DV_SUPPORT
+#ifdef HAVE_MLX5DV_DR
+ return mlx5dv_dr_create_action_dest_ibv_qp(qp);
+#else
struct mlx5dv_flow_action_attr *action;
action = malloc(sizeof(*action));
@@ -504,6 +566,7 @@
action->type = MLX5DV_FLOW_ACTION_DEST_IBV_QP;
action->obj = qp;
return action;
+#endif
#else
(void)qp;
return NULL;
@@ -513,13 +576,22 @@
static void *
mlx5_glue_dv_create_flow_action_modify_header
(struct ibv_context *ctx,
+ enum mlx5dv_flow_table_type ft_type,
+ void *ns, uint64_t flags,
size_t actions_sz,
- uint64_t actions[],
- enum mlx5dv_flow_table_type ft_type)
+ uint64_t actions[])
{
#ifdef HAVE_IBV_FLOW_DV_SUPPORT
+#ifdef HAVE_MLX5DV_DR
+ (void)ctx;
+ (void)ft_type;
+ return mlx5dv_dr_create_action_modify_header(ns, flags, actions_sz,
+ actions);
+#else
struct mlx5dv_flow_action_attr *action;
+ (void)ns;
+ (void)flags;
action = malloc(sizeof(*action));
if (!action)
return NULL;
@@ -527,11 +599,14 @@
action->action = mlx5dv_create_flow_action_modify_header
(ctx, actions_sz, actions, ft_type);
return action;
+#endif
#else
(void)ctx;
+ (void)ft_type;
+ (void)ns;
+ (void)flags;
(void)actions_sz;
(void)actions;
- (void)ft_type;
return NULL;
#endif
}
@@ -539,12 +614,20 @@
static void *
mlx5_glue_dv_create_flow_action_packet_reformat
(struct ibv_context *ctx,
- size_t data_sz,
- void *data,
enum mlx5dv_flow_action_packet_reformat_type reformat_type,
- enum mlx5dv_flow_table_type ft_type)
+ enum mlx5dv_flow_table_type ft_type, struct mlx5dv_dr_ns *ns,
+ uint32_t flags, size_t data_sz, void *data)
{
#ifdef HAVE_IBV_FLOW_DV_SUPPORT
+#ifdef HAVE_MLX5DV_DR
+ (void)ctx;
+ (void)ft_type;
+ return mlx5dv_dr_create_action_packet_reformat(ns, flags,
+ reformat_type, data_sz,
+ data);
+#else
+ (void)ns;
+ (void)flags;
struct mlx5dv_flow_action_attr *action;
action = malloc(sizeof(*action));
@@ -554,12 +637,15 @@
action->action = mlx5dv_create_flow_action_packet_reformat
(ctx, data_sz, data, reformat_type, ft_type);
return action;
+#endif
#else
(void)ctx;
- (void)data_sz;
- (void)data;
(void)reformat_type;
(void)ft_type;
+ (void)ns;
+ (void)flags;
+ (void)data_sz;
+ (void)data;
return NULL;
#endif
}
@@ -568,6 +654,9 @@
mlx5_glue_dv_create_flow_action_tag(uint32_t tag)
{
#ifdef HAVE_IBV_FLOW_DV_SUPPORT
+#ifdef HAVE_MLX5DV_DR
+ return mlx5dv_dr_create_action_tag(tag);
+#else
struct mlx5dv_flow_action_attr *action;
action = malloc(sizeof(*action));
if (!action)
@@ -576,10 +665,36 @@
action->tag_value = tag;
return action;
#endif
+#endif
(void)tag;
return NULL;
}
+static int
+mlx5_glue_dv_destroy_flow(void *flow_id)
+{
+#ifdef HAVE_MLX5DV_DR
+ return mlx5dv_dr_destroy_rule(flow_id);
+#else
+ return ibv_destroy_flow(flow_id);
+#endif
+}
+
+static int
+mlx5_glue_dv_destroy_flow_matcher(void *matcher)
+{
+#ifdef HAVE_IBV_FLOW_DV_SUPPORT
+#ifdef HAVE_MLX5DV_DR
+ return mlx5dv_dr_destroy_matcher(matcher);
+#else
+ return mlx5dv_destroy_flow_matcher(matcher);
+#endif
+#else
+ (void)matcher;
+ return 0;
+#endif
+}
+
static struct ibv_context *
mlx5_glue_dv_open_device(struct ibv_device *device)
{
@@ -718,6 +833,10 @@
.get_async_event = mlx5_glue_get_async_event,
.port_state_str = mlx5_glue_port_state_str,
.cq_ex_to_cq = mlx5_glue_cq_ex_to_cq,
+ .dr_create_flow_tbl = mlx5_glue_dr_create_flow_tbl,
+ .dr_destroy_flow_tbl = mlx5_glue_dr_destroy_flow_tbl,
+ .dr_create_ns = mlx5_glue_dr_create_ns,
+ .dr_destroy_ns = mlx5_glue_dr_destroy_ns,
.dv_create_cq = mlx5_glue_dv_create_cq,
.dv_create_wq = mlx5_glue_dv_create_wq,
.dv_query_device = mlx5_glue_dv_query_device,
@@ -725,7 +844,6 @@
.dv_init_obj = mlx5_glue_dv_init_obj,
.dv_create_qp = mlx5_glue_dv_create_qp,
.dv_create_flow_matcher = mlx5_glue_dv_create_flow_matcher,
- .dv_destroy_flow_matcher = mlx5_glue_dv_destroy_flow_matcher,
.dv_create_flow = mlx5_glue_dv_create_flow,
.dv_create_flow_action_counter =
mlx5_glue_dv_create_flow_action_counter,
@@ -736,6 +854,8 @@
.dv_create_flow_action_packet_reformat =
mlx5_glue_dv_create_flow_action_packet_reformat,
.dv_create_flow_action_tag = mlx5_glue_dv_create_flow_action_tag,
+ .dv_destroy_flow = mlx5_glue_dv_destroy_flow,
+ .dv_destroy_flow_matcher = mlx5_glue_dv_destroy_flow_matcher,
.dv_open_device = mlx5_glue_dv_open_device,
.devx_obj_create = mlx5_glue_devx_obj_create,
.devx_obj_destroy = mlx5_glue_devx_obj_destroy,
diff --git a/drivers/net/mlx5/mlx5_glue.h b/drivers/net/mlx5/mlx5_glue.h
index 32487ea..eb29ffa 100644
--- a/drivers/net/mlx5/mlx5_glue.h
+++ b/drivers/net/mlx5/mlx5_glue.h
@@ -63,6 +63,11 @@
struct mlx5dv_devx_obj;
#endif
+#ifndef HAVE_MLX5DV_DR
+struct mlx5dv_dr_ns;
+enum mlx5dv_dr_ns_domain { unused, };
+#endif
+
/* LIB_GLUE_VERSION must be updated every time this structure is modified. */
struct mlx5_glue {
const char *version;
@@ -140,6 +145,11 @@ struct mlx5_glue {
struct ibv_async_event *event);
const char *(*port_state_str)(enum ibv_port_state port_state);
struct ibv_cq *(*cq_ex_to_cq)(struct ibv_cq_ex *cq);
+ void *(*dr_create_flow_tbl)(void *ns, uint32_t level);
+ int (*dr_destroy_flow_tbl)(void *tbl);
+ void *(*dr_create_ns)(struct ibv_context *ctx,
+ enum mlx5dv_dr_ns_domain domain);
+ int (*dr_destroy_ns)(void *ns);
struct ibv_cq_ex *(*dv_create_cq)
(struct ibv_context *context,
struct ibv_cq_init_attr_ex *cq_attr,
@@ -158,23 +168,26 @@ struct mlx5_glue {
(struct ibv_context *context,
struct ibv_qp_init_attr_ex *qp_init_attr_ex,
struct mlx5dv_qp_init_attr *dv_qp_init_attr);
- struct mlx5dv_flow_matcher *(*dv_create_flow_matcher)
+ void *(*dv_create_flow_matcher)
(struct ibv_context *context,
- struct mlx5dv_flow_matcher_attr *matcher_attr);
- int (*dv_destroy_flow_matcher)(struct mlx5dv_flow_matcher *matcher);
- struct ibv_flow *(*dv_create_flow)(struct mlx5dv_flow_matcher *matcher,
- struct mlx5dv_flow_match_parameters *match_value,
+ struct mlx5dv_flow_matcher_attr *matcher_attr,
+ void *tbl);
+ void *(*dv_create_flow)(void *matcher, void *match_value,
size_t num_actions, void *actions[]);
void *(*dv_create_flow_action_counter)(void *obj, uint32_t offset);
void *(*dv_create_flow_action_dest_ibv_qp)(void *qp);
void *(*dv_create_flow_action_modify_header)
- (struct ibv_context *ctx, size_t actions_sz, uint64_t actions[],
- enum mlx5dv_flow_table_type ft_type);
+ (struct ibv_context *ctx, enum mlx5dv_flow_table_type ft_type,
+ void *ns, uint64_t flags, size_t actions_sz,
+ uint64_t actions[]);
void *(*dv_create_flow_action_packet_reformat)
- (struct ibv_context *ctx, size_t data_sz, void *data,
+ (struct ibv_context *ctx,
enum mlx5dv_flow_action_packet_reformat_type reformat_type,
- enum mlx5dv_flow_table_type ft_type);
+ enum mlx5dv_flow_table_type ft_type, struct mlx5dv_dr_ns *ns,
+ uint32_t flags, size_t data_sz, void *data);
void *(*dv_create_flow_action_tag)(uint32_t tag);
+ int (*dv_destroy_flow)(void *flow);
+ int (*dv_destroy_flow_matcher)(void *matcher);
struct ibv_context *(*dv_open_device)(struct ibv_device *device);
struct mlx5dv_devx_obj *(*devx_obj_create)
(struct ibv_context *ctx,
diff --git a/drivers/net/mlx5/mlx5_prm.h b/drivers/net/mlx5/mlx5_prm.h
index da1219e..b15266f 100644
--- a/drivers/net/mlx5/mlx5_prm.h
+++ b/drivers/net/mlx5/mlx5_prm.h
@@ -492,20 +492,40 @@ struct mlx5_ifc_fte_match_set_misc2_bits {
u8 reserved_at_1a0[0x60];
};
+struct mlx5_ifc_fte_match_set_misc3_bits {
+ u8 inner_tcp_seq_num[0x20];
+ u8 outer_tcp_seq_num[0x20];
+ u8 inner_tcp_ack_num[0x20];
+ u8 outer_tcp_ack_num[0x20];
+ u8 reserved_at_auto1[0x8];
+ u8 outer_vxlan_gpe_vni[0x18];
+ u8 outer_vxlan_gpe_next_protocol[0x8];
+ u8 outer_vxlan_gpe_flags[0x8];
+ u8 reserved_at_a8[0x10];
+ u8 icmp_header_data[0x20];
+ u8 icmpv6_header_data[0x20];
+ u8 icmp_type[0x8];
+ u8 icmp_code[0x8];
+ u8 icmpv6_type[0x8];
+ u8 icmpv6_code[0x8];
+ u8 reserved_at_1a0[0xe0];
+};
+
/* Flow matcher. */
struct mlx5_ifc_fte_match_param_bits {
struct mlx5_ifc_fte_match_set_lyr_2_4_bits outer_headers;
struct mlx5_ifc_fte_match_set_misc_bits misc_parameters;
struct mlx5_ifc_fte_match_set_lyr_2_4_bits inner_headers;
struct mlx5_ifc_fte_match_set_misc2_bits misc_parameters_2;
- u8 reserved_at_800[0x800];
+ struct mlx5_ifc_fte_match_set_misc3_bits misc_parameters_3;
};
enum {
MLX5_MATCH_CRITERIA_ENABLE_OUTER_BIT,
MLX5_MATCH_CRITERIA_ENABLE_MISC_BIT,
MLX5_MATCH_CRITERIA_ENABLE_INNER_BIT,
- MLX5_MATCH_CRITERIA_ENABLE_MISC2_BIT
+ MLX5_MATCH_CRITERIA_ENABLE_MISC2_BIT,
+ MLX5_MATCH_CRITERIA_ENABLE_MISC3_BIT
};
enum {
--
1.8.3.1
^ permalink raw reply [flat|nested] 52+ messages in thread
* [dpdk-dev] [PATCH v4 2/3] net/mlx5: add Direct Rules API
2019-04-04 9:54 ` [dpdk-dev] [PATCH v4 2/3] net/mlx5: add Direct Rules API Ori Kam
@ 2019-04-04 9:54 ` Ori Kam
0 siblings, 0 replies; 52+ messages in thread
From: Ori Kam @ 2019-04-04 9:54 UTC (permalink / raw)
To: Matan Azrad, Yongseok Koh, Shahaf Shuler; +Cc: dev, Ori Kam, Slava Ovsiienko
Adds calls to the Direct Rules API inside the glue functions.
Due to difference in parameters between the Direct Rules and Direct
Verbs some of the glue functions API was updated.
Signed-off-by: Ori Kam <orika@mellanox.com>
Acked-by: Shahaf Shuler <shahafs@mellanox.com>
---
drivers/net/mlx5/Makefile | 5 ++
drivers/net/mlx5/mlx5.c | 16 ++++
drivers/net/mlx5/mlx5.h | 15 ++++
drivers/net/mlx5/mlx5_flow.c | 1 +
drivers/net/mlx5/mlx5_flow.h | 6 +-
drivers/net/mlx5/mlx5_flow_dv.c | 103 ++++++++++++++++++++----
drivers/net/mlx5/mlx5_glue.c | 170 ++++++++++++++++++++++++++++++++++------
drivers/net/mlx5/mlx5_glue.h | 31 +++++---
drivers/net/mlx5/mlx5_prm.h | 24 +++++-
9 files changed, 318 insertions(+), 53 deletions(-)
diff --git a/drivers/net/mlx5/Makefile b/drivers/net/mlx5/Makefile
index 0d20f0f..93bc869 100644
--- a/drivers/net/mlx5/Makefile
+++ b/drivers/net/mlx5/Makefile
@@ -156,6 +156,11 @@ mlx5_autoconf.h.new: $(RTE_SDK)/buildtools/auto-config-h.sh
func mlx5dv_create_flow_action_packet_reformat \
$(AUTOCONF_OUTPUT)
$Q sh -- '$<' '$@' \
+ HAVE_MLX5DV_DR \
+ infiniband/mlx5dv.h \
+ enum MLX5DV_DR_NS_TYPE_TERMINATING \
+ $(AUTOCONF_OUTPUT)
+ $Q sh -- '$<' '$@' \
HAVE_IBV_DEVX_OBJ \
infiniband/mlx5dv.h \
func mlx5dv_devx_obj_create \
diff --git a/drivers/net/mlx5/mlx5.c b/drivers/net/mlx5/mlx5.c
index 4044505..65aa9cf 100644
--- a/drivers/net/mlx5/mlx5.c
+++ b/drivers/net/mlx5/mlx5.c
@@ -1483,6 +1483,22 @@ struct mlx5_dev_spawn_data {
priv->tcf_context = NULL;
}
}
+#ifdef HAVE_MLX5DV_DR
+ priv->rx_ns = mlx5dv_dr_create_ns
+ (sh->ctx, MLX5DV_DR_NS_DOMAIN_INGRESS_BYPASS);
+ if (priv->rx_ns == NULL) {
+ DRV_LOG(ERR, "mlx5dv_dr_create_ns failed");
+ err = errno;
+ goto error;
+ }
+ priv->tx_ns = mlx5dv_dr_create_ns(sh->ctx,
+ MLX5DV_DR_NS_DOMAIN_EGRESS_BYPASS);
+ if (priv->tx_ns == NULL) {
+ DRV_LOG(ERR, "mlx5dv_dr_create_ns failed");
+ err = errno;
+ goto error;
+ }
+#endif
TAILQ_INIT(&priv->flows);
TAILQ_INIT(&priv->ctrl_flows);
/* Hint libmlx5 to use PMD allocator for data plane resources */
diff --git a/drivers/net/mlx5/mlx5.h b/drivers/net/mlx5/mlx5.h
index 784bf9b..d4963cb 100644
--- a/drivers/net/mlx5/mlx5.h
+++ b/drivers/net/mlx5/mlx5.h
@@ -259,6 +259,15 @@ struct mlx5_ibv_shared {
struct mlx5_ibv_shared_port port[]; /* per device port data array. */
};
+/* Table structure. */
+struct mlx5_flow_tbl_resource {
+ void *obj; /**< Pointer to DR table object. */
+ rte_atomic32_t refcnt; /**< Reference counter. */
+};
+
+#define MLX5_MAX_TABLES 1024
+#define MLX5_GROUP_FACTOR 1
+
struct mlx5_priv {
LIST_ENTRY(mlx5_priv) mem_event_cb;
/**< Called by memory event callback. */
@@ -326,6 +335,12 @@ struct mlx5_priv {
/* UAR same-page access control required in 32bit implementations. */
#endif
struct mlx5_flow_tcf_context *tcf_context; /* TC flower context. */
+ void *rx_ns; /* RX Direct Rules name space handle. */
+ struct mlx5_flow_tbl_resource rx_tbl[MLX5_MAX_TABLES];
+ /* RX Direct Rules tables. */
+ void *tx_ns; /* TX Direct Rules name space handle. */
+ struct mlx5_flow_tbl_resource tx_tbl[MLX5_MAX_TABLES];
+ /* TX Direct Rules tables/ */
};
#define PORT_ID(priv) ((priv)->dev_data->port_id)
diff --git a/drivers/net/mlx5/mlx5_flow.c b/drivers/net/mlx5/mlx5_flow.c
index bc6a7c1..9dc492a 100644
--- a/drivers/net/mlx5/mlx5_flow.c
+++ b/drivers/net/mlx5/mlx5_flow.c
@@ -2084,6 +2084,7 @@ uint32_t mlx5_flow_adjust_priority(struct rte_eth_dev *dev, int32_t priority,
flow_size += RTE_ALIGN_CEIL(sizeof(uint16_t), sizeof(void *));
flow = rte_calloc(__func__, 1, flow_size, 0);
flow->drv_type = flow_get_drv_type(dev, attr);
+ flow->ingress = attr->ingress;
assert(flow->drv_type > MLX5_FLOW_TYPE_MIN &&
flow->drv_type < MLX5_FLOW_TYPE_MAX);
flow->queue = (void *)(flow + 1);
diff --git a/drivers/net/mlx5/mlx5_flow.h b/drivers/net/mlx5/mlx5_flow.h
index 4f69ae2..8ba37a0 100644
--- a/drivers/net/mlx5/mlx5_flow.h
+++ b/drivers/net/mlx5/mlx5_flow.h
@@ -204,6 +204,7 @@ struct mlx5_flow_dv_matcher {
uint16_t crc; /**< CRC of key. */
uint16_t priority; /**< Priority of matcher. */
uint8_t egress; /**< Egress matcher. */
+ uint32_t group; /**< The matcher group. */
struct mlx5_flow_dv_match_params mask; /**< Matcher mask. */
};
@@ -220,6 +221,7 @@ struct mlx5_flow_dv_encap_decap_resource {
size_t size;
uint8_t reformat_type;
uint8_t ft_type;
+ uint64_t flags; /**< Flags for RDMA API. */
};
/* Tag resource structure. */
@@ -348,7 +350,7 @@ struct mlx5_flow_counter {
/* Flow structure. */
struct rte_flow {
TAILQ_ENTRY(rte_flow) next; /**< Pointer to the next flow structure. */
- enum mlx5_flow_drv_type drv_type; /**< Drvier type. */
+ enum mlx5_flow_drv_type drv_type; /**< Driver type. */
struct mlx5_flow_counter *counter; /**< Holds flow counter. */
struct mlx5_flow_dv_tag_resource *tag_resource;
/**< pointer to the tag action. */
@@ -360,6 +362,8 @@ struct rte_flow {
uint64_t actions;
/**< Bit-fields of detected actions, see MLX5_FLOW_ACTION_*. */
struct mlx5_fdir *fdir; /**< Pointer to associated FDIR if any. */
+ uint8_t ingress; /**< 1 if the flow is ingress. */
+ uint32_t group; /**< The group index. */
};
typedef int (*mlx5_flow_validate_t)(struct rte_eth_dev *dev,
diff --git a/drivers/net/mlx5/mlx5_flow_dv.c b/drivers/net/mlx5/mlx5_flow_dv.c
index d9e2ac9..dd5b541 100644
--- a/drivers/net/mlx5/mlx5_flow_dv.c
+++ b/drivers/net/mlx5/mlx5_flow_dv.c
@@ -809,11 +809,20 @@ struct field_modify_info modify_tcp[] = {
{
struct mlx5_priv *priv = dev->data->dev_private;
struct mlx5_flow_dv_encap_decap_resource *cache_resource;
+ struct rte_flow *flow = dev_flow->flow;
+ struct mlx5dv_dr_ns *ns;
+
+ resource->flags = flow->group ? 0 : 1;
+ if (flow->ingress)
+ ns = priv->rx_ns;
+ else
+ ns = priv->tx_ns;
/* Lookup a matching resource from cache. */
LIST_FOREACH(cache_resource, &priv->encaps_decaps, next) {
if (resource->reformat_type == cache_resource->reformat_type &&
resource->ft_type == cache_resource->ft_type &&
+ resource->flags == cache_resource->flags &&
resource->size == cache_resource->size &&
!memcmp((const void *)resource->buf,
(const void *)cache_resource->buf,
@@ -835,10 +844,10 @@ struct field_modify_info modify_tcp[] = {
*cache_resource = *resource;
cache_resource->verbs_action =
mlx5_glue->dv_create_flow_action_packet_reformat
- (priv->sh->ctx, cache_resource->size,
- (cache_resource->size ? cache_resource->buf : NULL),
- cache_resource->reformat_type,
- cache_resource->ft_type);
+ (priv->sh->ctx, cache_resource->reformat_type,
+ cache_resource->ft_type, ns, cache_resource->flags,
+ cache_resource->size,
+ (cache_resource->size ? cache_resource->buf : NULL));
if (!cache_resource->verbs_action) {
rte_free(cache_resource);
return rte_flow_error_set(error, ENOMEM,
@@ -1442,6 +1451,10 @@ struct field_modify_info modify_tcp[] = {
struct mlx5_priv *priv = dev->data->dev_private;
struct mlx5_flow_dv_modify_hdr_resource *cache_resource;
+ struct mlx5dv_dr_ns *ns =
+ resource->ft_type == MLX5DV_FLOW_TABLE_TYPE_NIC_TX ?
+ priv->tx_ns : priv->rx_ns;
+
/* Lookup a matching resource from cache. */
LIST_FOREACH(cache_resource, &priv->modify_cmds, next) {
if (resource->ft_type == cache_resource->ft_type &&
@@ -1467,11 +1480,11 @@ struct field_modify_info modify_tcp[] = {
*cache_resource = *resource;
cache_resource->verbs_action =
mlx5_glue->dv_create_flow_action_modify_header
- (priv->sh->ctx,
+ (priv->sh->ctx, cache_resource->ft_type,
+ ns, 0,
cache_resource->actions_num *
sizeof(cache_resource->actions[0]),
- (uint64_t *)cache_resource->actions,
- cache_resource->ft_type);
+ (uint64_t *)cache_resource->actions);
if (!cache_resource->verbs_action) {
rte_free(cache_resource);
return rte_flow_error_set(error, ENOMEM,
@@ -1596,11 +1609,13 @@ struct field_modify_info modify_tcp[] = {
struct mlx5_priv *priv = dev->data->dev_private;
uint32_t priority_max = priv->config.flow_prio - 1;
+#ifdef HAVE_MLX5DV_DR
if (attributes->group)
return rte_flow_error_set(error, ENOTSUP,
RTE_FLOW_ERROR_TYPE_ATTR_GROUP,
NULL,
"groups is not supported");
+#endif
if (attributes->priority != MLX5_FLOW_PRIO_RSVD &&
attributes->priority >= priority_max)
return rte_flow_error_set(error, ENOTSUP,
@@ -2173,11 +2188,13 @@ struct field_modify_info modify_tcp[] = {
* Flow pattern to translate.
* @param[in] inner
* Item is inner pattern.
+ * @param[in] group
+ * The group to insert the rule.
*/
static void
flow_dv_translate_item_ipv4(void *matcher, void *key,
const struct rte_flow_item *item,
- int inner)
+ int inner, uint32_t group)
{
const struct rte_flow_item_ipv4 *ipv4_m = item->mask;
const struct rte_flow_item_ipv4 *ipv4_v = item->spec;
@@ -2204,7 +2221,10 @@ struct field_modify_info modify_tcp[] = {
outer_headers);
headers_v = MLX5_ADDR_OF(fte_match_param, key, outer_headers);
}
- MLX5_SET(fte_match_set_lyr_2_4, headers_m, ip_version, 0xf);
+ if (group == 0)
+ MLX5_SET(fte_match_set_lyr_2_4, headers_m, ip_version, 0xf);
+ else
+ MLX5_SET(fte_match_set_lyr_2_4, headers_m, ip_version, 0x4);
MLX5_SET(fte_match_set_lyr_2_4, headers_v, ip_version, 4);
if (!ipv4_v)
return;
@@ -2246,11 +2266,13 @@ struct field_modify_info modify_tcp[] = {
* Flow pattern to translate.
* @param[in] inner
* Item is inner pattern.
+ * @param[in] group
+ * The group to insert the rule.
*/
static void
flow_dv_translate_item_ipv6(void *matcher, void *key,
const struct rte_flow_item *item,
- int inner)
+ int inner, uint32_t group)
{
const struct rte_flow_item_ipv6 *ipv6_m = item->mask;
const struct rte_flow_item_ipv6 *ipv6_v = item->spec;
@@ -2287,7 +2309,10 @@ struct field_modify_info modify_tcp[] = {
outer_headers);
headers_v = MLX5_ADDR_OF(fte_match_param, key, outer_headers);
}
- MLX5_SET(fte_match_set_lyr_2_4, headers_m, ip_version, 0xf);
+ if (group == 0)
+ MLX5_SET(fte_match_set_lyr_2_4, headers_m, ip_version, 0xf);
+ else
+ MLX5_SET(fte_match_set_lyr_2_4, headers_m, ip_version, 0x6);
MLX5_SET(fte_match_set_lyr_2_4, headers_v, ip_version, 6);
if (!ipv6_v)
return;
@@ -2727,7 +2752,11 @@ struct field_modify_info modify_tcp[] = {
match_criteria_enable |=
(!HEADER_IS_ZERO(match_criteria, misc_parameters_2)) <<
MLX5_MATCH_CRITERIA_ENABLE_MISC2_BIT;
-
+#ifdef HAVE_MLX5DV_DR
+ match_criteria_enable |=
+ (!HEADER_IS_ZERO(match_criteria, misc_parameters_3)) <<
+ MLX5_MATCH_CRITERIA_ENABLE_MISC3_BIT;
+#endif
return match_criteria_enable;
}
@@ -2758,12 +2787,14 @@ struct field_modify_info modify_tcp[] = {
.type = IBV_FLOW_ATTR_NORMAL,
.match_mask = (void *)&matcher->mask,
};
+ struct mlx5_flow_tbl_resource *tbl = NULL;
/* Lookup from cache. */
LIST_FOREACH(cache_matcher, &priv->matchers, next) {
if (matcher->crc == cache_matcher->crc &&
matcher->priority == cache_matcher->priority &&
matcher->egress == cache_matcher->egress &&
+ matcher->group == cache_matcher->group &&
!memcmp((const void *)matcher->mask.buf,
(const void *)cache_matcher->mask.buf,
cache_matcher->mask.size)) {
@@ -2778,6 +2809,27 @@ struct field_modify_info modify_tcp[] = {
return 0;
}
}
+#ifdef HAVE_MLX5DV_DR
+ if (matcher->egress) {
+ tbl = &priv->tx_tbl[matcher->group];
+ if (!tbl->obj)
+ tbl->obj = mlx5_glue->dr_create_flow_tbl
+ (priv->tx_ns,
+ matcher->group * MLX5_GROUP_FACTOR);
+ } else {
+ tbl = &priv->rx_tbl[matcher->group];
+ if (!tbl->obj)
+ tbl->obj = mlx5_glue->dr_create_flow_tbl
+ (priv->rx_ns,
+ matcher->group * MLX5_GROUP_FACTOR);
+ }
+ if (!tbl->obj)
+ return rte_flow_error_set(error, ENOMEM,
+ RTE_FLOW_ERROR_TYPE_UNSPECIFIED,
+ NULL, "cannot create table");
+
+ rte_atomic32_inc(&tbl->refcnt);
+#endif
/* Register new matcher. */
cache_matcher = rte_calloc(__func__, 1, sizeof(*cache_matcher), 0);
if (!cache_matcher)
@@ -2791,9 +2843,16 @@ struct field_modify_info modify_tcp[] = {
if (matcher->egress)
dv_attr.flags |= IBV_FLOW_ATTR_FLAGS_EGRESS;
cache_matcher->matcher_object =
- mlx5_glue->dv_create_flow_matcher(priv->sh->ctx, &dv_attr);
+ mlx5_glue->dv_create_flow_matcher(priv->sh->ctx, &dv_attr,
+ tbl->obj);
if (!cache_matcher->matcher_object) {
rte_free(cache_matcher);
+#ifdef HAVE_MLX5DV_DR
+ if (rte_atomic32_dec_and_test(&tbl->refcnt)) {
+ mlx5_glue->dr_destroy_flow_tbl(tbl->obj);
+ tbl->obj = NULL;
+ }
+#endif
return rte_flow_error_set(error, ENOMEM,
RTE_FLOW_ERROR_TYPE_UNSPECIFIED,
NULL, "cannot create matcher");
@@ -2805,6 +2864,7 @@ struct field_modify_info modify_tcp[] = {
cache_matcher->priority,
cache_matcher->egress ? "tx" : "rx", (void *)cache_matcher,
rte_atomic32_read(&cache_matcher->refcnt));
+ rte_atomic32_inc(&tbl->refcnt);
return 0;
}
@@ -3226,7 +3286,7 @@ struct field_modify_info modify_tcp[] = {
break;
case RTE_FLOW_ITEM_TYPE_IPV4:
flow_dv_translate_item_ipv4(match_mask, match_value,
- items, tunnel);
+ items, tunnel, attr->group);
matcher.priority = MLX5_PRIORITY_MAP_L3;
dev_flow->dv.hash_fields |=
mlx5_flow_hashfields_adjust
@@ -3238,7 +3298,7 @@ struct field_modify_info modify_tcp[] = {
break;
case RTE_FLOW_ITEM_TYPE_IPV6:
flow_dv_translate_item_ipv6(match_mask, match_value,
- items, tunnel);
+ items, tunnel, attr->group);
matcher.priority = MLX5_PRIORITY_MAP_L3;
dev_flow->dv.hash_fields |=
mlx5_flow_hashfields_adjust
@@ -3316,6 +3376,7 @@ struct field_modify_info modify_tcp[] = {
matcher.priority = mlx5_flow_adjust_priority(dev, priority,
matcher.priority);
matcher.egress = attr->egress;
+ matcher.group = attr->group;
if (flow_dv_matcher_register(dev, &matcher, dev_flow, error))
return -rte_errno;
return 0;
@@ -3431,6 +3492,8 @@ struct field_modify_info modify_tcp[] = {
struct mlx5_flow *flow)
{
struct mlx5_flow_dv_matcher *matcher = flow->dv.matcher;
+ struct mlx5_priv *priv = dev->data->dev_private;
+ struct mlx5_flow_tbl_resource *tbl;
assert(matcher->matcher_object);
DRV_LOG(DEBUG, "port %u matcher %p: refcnt %d--",
@@ -3440,6 +3503,14 @@ struct field_modify_info modify_tcp[] = {
claim_zero(mlx5_glue->dv_destroy_flow_matcher
(matcher->matcher_object));
LIST_REMOVE(matcher, next);
+ if (matcher->egress)
+ tbl = &priv->tx_tbl[matcher->group];
+ else
+ tbl = &priv->rx_tbl[matcher->group];
+ if (rte_atomic32_dec_and_test(&tbl->refcnt)) {
+ mlx5_glue->dr_destroy_flow_tbl(tbl->obj);
+ tbl->obj = NULL;
+ }
rte_free(matcher);
DRV_LOG(DEBUG, "port %u matcher %p: removed",
dev->data->port_id, (void *)matcher);
@@ -3529,7 +3600,7 @@ struct field_modify_info modify_tcp[] = {
LIST_FOREACH(dev_flow, &flow->dev_flows, next) {
dv = &dev_flow->dv;
if (dv->flow) {
- claim_zero(mlx5_glue->destroy_flow(dv->flow));
+ claim_zero(mlx5_glue->dv_destroy_flow(dv->flow));
dv->flow = NULL;
}
if (dv->hrxq) {
diff --git a/drivers/net/mlx5/mlx5_glue.c b/drivers/net/mlx5/mlx5_glue.c
index 4b5aade..b0b144c 100644
--- a/drivers/net/mlx5/mlx5_glue.c
+++ b/drivers/net/mlx5/mlx5_glue.c
@@ -178,6 +178,9 @@
mlx5_glue_destroy_flow_action(void *action)
{
#ifdef HAVE_IBV_FLOW_DV_SUPPORT
+#ifdef HAVE_MLX5DV_DR
+ return mlx5dv_dr_destroy_action(action);
+#else
struct mlx5dv_flow_action_attr *attr = action;
int res = 0;
switch (attr->type) {
@@ -189,6 +192,7 @@
}
free(action);
return res;
+#endif
#else
(void)action;
return ENOTSUP;
@@ -365,6 +369,53 @@
return ibv_cq_ex_to_cq(cq);
}
+static void *
+mlx5_glue_dr_create_flow_tbl(void *ns, uint32_t level)
+{
+#ifdef HAVE_MLX5DV_DR
+ return mlx5dv_dr_create_ft(ns, level);
+#else
+ (void)ns;
+ (void)level;
+ return NULL;
+#endif
+}
+
+static int
+mlx5_glue_dr_destroy_flow_tbl(void *tbl)
+{
+#ifdef HAVE_MLX5DV_DR
+ return mlx5dv_dr_destroy_ft(tbl);
+#else
+ (void)tbl;
+ return 0;
+#endif
+}
+
+static void *
+mlx5_glue_dr_create_ns(struct ibv_context *ctx,
+ enum mlx5dv_dr_ns_domain domain)
+{
+#ifdef HAVE_MLX5DV_DR
+ return mlx5dv_dr_create_ns(ctx, domain);
+#else
+ (void)ctx;
+ (void)domain;
+ return NULL;
+#endif
+}
+
+static int
+mlx5_glue_dr_destroy_ns(void *ns)
+{
+#ifdef HAVE_MLX5DV_DR
+ return mlx5dv_dr_destroy_ns(ns);
+#else
+ (void)ns;
+ return 0;
+#endif
+}
+
static struct ibv_cq_ex *
mlx5_glue_dv_create_cq(struct ibv_context *context,
struct ibv_cq_init_attr_ex *cq_attr,
@@ -423,26 +474,40 @@
#endif
}
-static struct mlx5dv_flow_matcher *
+static void *
mlx5_glue_dv_create_flow_matcher(struct ibv_context *context,
- struct mlx5dv_flow_matcher_attr *matcher_attr)
+ struct mlx5dv_flow_matcher_attr *matcher_attr,
+ void *tbl)
{
#ifdef HAVE_IBV_FLOW_DV_SUPPORT
+#ifdef HAVE_MLX5DV_DR
+ (void)context;
+ return mlx5dv_dr_create_matcher(tbl, matcher_attr->priority,
+ matcher_attr->match_criteria_enable,
+ matcher_attr->match_mask);
+#else
+ (void)tbl;
return mlx5dv_create_flow_matcher(context, matcher_attr);
+#endif
#else
(void)context;
(void)matcher_attr;
+ (void)tbl;
return NULL;
#endif
}
-static struct ibv_flow *
-mlx5_glue_dv_create_flow(struct mlx5dv_flow_matcher *matcher,
- struct mlx5dv_flow_match_parameters *match_value,
+static void *
+mlx5_glue_dv_create_flow(void *matcher,
+ void *match_value,
size_t num_actions,
void *actions[])
{
#ifdef HAVE_IBV_FLOW_DV_SUPPORT
+#ifdef HAVE_MLX5DV_DR
+ return mlx5dv_dr_create_rule(matcher, match_value, num_actions,
+ (struct mlx5dv_dr_action **)actions);
+#else
struct mlx5dv_flow_action_attr actions_attr[8];
if (num_actions > 8)
@@ -452,6 +517,7 @@
*((struct mlx5dv_flow_action_attr *)(actions[i]));
return mlx5dv_create_flow(matcher, match_value,
num_actions, actions_attr);
+#endif
#else
(void)matcher;
(void)match_value;
@@ -461,21 +527,13 @@
#endif
}
-static int
-mlx5_glue_dv_destroy_flow_matcher(struct mlx5dv_flow_matcher *matcher)
-{
-#ifdef HAVE_IBV_FLOW_DV_SUPPORT
- return mlx5dv_destroy_flow_matcher(matcher);
-#else
- (void)matcher;
- return 0;
-#endif
-}
-
static void *
mlx5_glue_dv_create_flow_action_counter(void *counter_obj, uint32_t offset)
{
#ifdef HAVE_IBV_FLOW_DV_SUPPORT
+#ifdef HAVE_MLX5DV_DR
+ return mlx5dv_dr_create_action_devx_counter(counter_obj, offset);
+#else
struct mlx5dv_flow_action_attr *action;
(void)offset;
@@ -485,6 +543,7 @@
action->type = MLX5DV_FLOW_ACTION_COUNTERS_DEVX;
action->obj = counter_obj;
return action;
+#endif
#else
(void)counter_obj;
(void)offset;
@@ -496,6 +555,9 @@
mlx5_glue_dv_create_flow_action_dest_ibv_qp(void *qp)
{
#ifdef HAVE_IBV_FLOW_DV_SUPPORT
+#ifdef HAVE_MLX5DV_DR
+ return mlx5dv_dr_create_action_dest_ibv_qp(qp);
+#else
struct mlx5dv_flow_action_attr *action;
action = malloc(sizeof(*action));
@@ -504,6 +566,7 @@
action->type = MLX5DV_FLOW_ACTION_DEST_IBV_QP;
action->obj = qp;
return action;
+#endif
#else
(void)qp;
return NULL;
@@ -513,13 +576,22 @@
static void *
mlx5_glue_dv_create_flow_action_modify_header
(struct ibv_context *ctx,
+ enum mlx5dv_flow_table_type ft_type,
+ void *ns, uint64_t flags,
size_t actions_sz,
- uint64_t actions[],
- enum mlx5dv_flow_table_type ft_type)
+ uint64_t actions[])
{
#ifdef HAVE_IBV_FLOW_DV_SUPPORT
+#ifdef HAVE_MLX5DV_DR
+ (void)ctx;
+ (void)ft_type;
+ return mlx5dv_dr_create_action_modify_header(ns, flags, actions_sz,
+ actions);
+#else
struct mlx5dv_flow_action_attr *action;
+ (void)ns;
+ (void)flags;
action = malloc(sizeof(*action));
if (!action)
return NULL;
@@ -527,11 +599,14 @@
action->action = mlx5dv_create_flow_action_modify_header
(ctx, actions_sz, actions, ft_type);
return action;
+#endif
#else
(void)ctx;
+ (void)ft_type;
+ (void)ns;
+ (void)flags;
(void)actions_sz;
(void)actions;
- (void)ft_type;
return NULL;
#endif
}
@@ -539,12 +614,20 @@
static void *
mlx5_glue_dv_create_flow_action_packet_reformat
(struct ibv_context *ctx,
- size_t data_sz,
- void *data,
enum mlx5dv_flow_action_packet_reformat_type reformat_type,
- enum mlx5dv_flow_table_type ft_type)
+ enum mlx5dv_flow_table_type ft_type, struct mlx5dv_dr_ns *ns,
+ uint32_t flags, size_t data_sz, void *data)
{
#ifdef HAVE_IBV_FLOW_DV_SUPPORT
+#ifdef HAVE_MLX5DV_DR
+ (void)ctx;
+ (void)ft_type;
+ return mlx5dv_dr_create_action_packet_reformat(ns, flags,
+ reformat_type, data_sz,
+ data);
+#else
+ (void)ns;
+ (void)flags;
struct mlx5dv_flow_action_attr *action;
action = malloc(sizeof(*action));
@@ -554,12 +637,15 @@
action->action = mlx5dv_create_flow_action_packet_reformat
(ctx, data_sz, data, reformat_type, ft_type);
return action;
+#endif
#else
(void)ctx;
- (void)data_sz;
- (void)data;
(void)reformat_type;
(void)ft_type;
+ (void)ns;
+ (void)flags;
+ (void)data_sz;
+ (void)data;
return NULL;
#endif
}
@@ -568,6 +654,9 @@
mlx5_glue_dv_create_flow_action_tag(uint32_t tag)
{
#ifdef HAVE_IBV_FLOW_DV_SUPPORT
+#ifdef HAVE_MLX5DV_DR
+ return mlx5dv_dr_create_action_tag(tag);
+#else
struct mlx5dv_flow_action_attr *action;
action = malloc(sizeof(*action));
if (!action)
@@ -576,10 +665,36 @@
action->tag_value = tag;
return action;
#endif
+#endif
(void)tag;
return NULL;
}
+static int
+mlx5_glue_dv_destroy_flow(void *flow_id)
+{
+#ifdef HAVE_MLX5DV_DR
+ return mlx5dv_dr_destroy_rule(flow_id);
+#else
+ return ibv_destroy_flow(flow_id);
+#endif
+}
+
+static int
+mlx5_glue_dv_destroy_flow_matcher(void *matcher)
+{
+#ifdef HAVE_IBV_FLOW_DV_SUPPORT
+#ifdef HAVE_MLX5DV_DR
+ return mlx5dv_dr_destroy_matcher(matcher);
+#else
+ return mlx5dv_destroy_flow_matcher(matcher);
+#endif
+#else
+ (void)matcher;
+ return 0;
+#endif
+}
+
static struct ibv_context *
mlx5_glue_dv_open_device(struct ibv_device *device)
{
@@ -718,6 +833,10 @@
.get_async_event = mlx5_glue_get_async_event,
.port_state_str = mlx5_glue_port_state_str,
.cq_ex_to_cq = mlx5_glue_cq_ex_to_cq,
+ .dr_create_flow_tbl = mlx5_glue_dr_create_flow_tbl,
+ .dr_destroy_flow_tbl = mlx5_glue_dr_destroy_flow_tbl,
+ .dr_create_ns = mlx5_glue_dr_create_ns,
+ .dr_destroy_ns = mlx5_glue_dr_destroy_ns,
.dv_create_cq = mlx5_glue_dv_create_cq,
.dv_create_wq = mlx5_glue_dv_create_wq,
.dv_query_device = mlx5_glue_dv_query_device,
@@ -725,7 +844,6 @@
.dv_init_obj = mlx5_glue_dv_init_obj,
.dv_create_qp = mlx5_glue_dv_create_qp,
.dv_create_flow_matcher = mlx5_glue_dv_create_flow_matcher,
- .dv_destroy_flow_matcher = mlx5_glue_dv_destroy_flow_matcher,
.dv_create_flow = mlx5_glue_dv_create_flow,
.dv_create_flow_action_counter =
mlx5_glue_dv_create_flow_action_counter,
@@ -736,6 +854,8 @@
.dv_create_flow_action_packet_reformat =
mlx5_glue_dv_create_flow_action_packet_reformat,
.dv_create_flow_action_tag = mlx5_glue_dv_create_flow_action_tag,
+ .dv_destroy_flow = mlx5_glue_dv_destroy_flow,
+ .dv_destroy_flow_matcher = mlx5_glue_dv_destroy_flow_matcher,
.dv_open_device = mlx5_glue_dv_open_device,
.devx_obj_create = mlx5_glue_devx_obj_create,
.devx_obj_destroy = mlx5_glue_devx_obj_destroy,
diff --git a/drivers/net/mlx5/mlx5_glue.h b/drivers/net/mlx5/mlx5_glue.h
index 32487ea..eb29ffa 100644
--- a/drivers/net/mlx5/mlx5_glue.h
+++ b/drivers/net/mlx5/mlx5_glue.h
@@ -63,6 +63,11 @@
struct mlx5dv_devx_obj;
#endif
+#ifndef HAVE_MLX5DV_DR
+struct mlx5dv_dr_ns;
+enum mlx5dv_dr_ns_domain { unused, };
+#endif
+
/* LIB_GLUE_VERSION must be updated every time this structure is modified. */
struct mlx5_glue {
const char *version;
@@ -140,6 +145,11 @@ struct mlx5_glue {
struct ibv_async_event *event);
const char *(*port_state_str)(enum ibv_port_state port_state);
struct ibv_cq *(*cq_ex_to_cq)(struct ibv_cq_ex *cq);
+ void *(*dr_create_flow_tbl)(void *ns, uint32_t level);
+ int (*dr_destroy_flow_tbl)(void *tbl);
+ void *(*dr_create_ns)(struct ibv_context *ctx,
+ enum mlx5dv_dr_ns_domain domain);
+ int (*dr_destroy_ns)(void *ns);
struct ibv_cq_ex *(*dv_create_cq)
(struct ibv_context *context,
struct ibv_cq_init_attr_ex *cq_attr,
@@ -158,23 +168,26 @@ struct mlx5_glue {
(struct ibv_context *context,
struct ibv_qp_init_attr_ex *qp_init_attr_ex,
struct mlx5dv_qp_init_attr *dv_qp_init_attr);
- struct mlx5dv_flow_matcher *(*dv_create_flow_matcher)
+ void *(*dv_create_flow_matcher)
(struct ibv_context *context,
- struct mlx5dv_flow_matcher_attr *matcher_attr);
- int (*dv_destroy_flow_matcher)(struct mlx5dv_flow_matcher *matcher);
- struct ibv_flow *(*dv_create_flow)(struct mlx5dv_flow_matcher *matcher,
- struct mlx5dv_flow_match_parameters *match_value,
+ struct mlx5dv_flow_matcher_attr *matcher_attr,
+ void *tbl);
+ void *(*dv_create_flow)(void *matcher, void *match_value,
size_t num_actions, void *actions[]);
void *(*dv_create_flow_action_counter)(void *obj, uint32_t offset);
void *(*dv_create_flow_action_dest_ibv_qp)(void *qp);
void *(*dv_create_flow_action_modify_header)
- (struct ibv_context *ctx, size_t actions_sz, uint64_t actions[],
- enum mlx5dv_flow_table_type ft_type);
+ (struct ibv_context *ctx, enum mlx5dv_flow_table_type ft_type,
+ void *ns, uint64_t flags, size_t actions_sz,
+ uint64_t actions[]);
void *(*dv_create_flow_action_packet_reformat)
- (struct ibv_context *ctx, size_t data_sz, void *data,
+ (struct ibv_context *ctx,
enum mlx5dv_flow_action_packet_reformat_type reformat_type,
- enum mlx5dv_flow_table_type ft_type);
+ enum mlx5dv_flow_table_type ft_type, struct mlx5dv_dr_ns *ns,
+ uint32_t flags, size_t data_sz, void *data);
void *(*dv_create_flow_action_tag)(uint32_t tag);
+ int (*dv_destroy_flow)(void *flow);
+ int (*dv_destroy_flow_matcher)(void *matcher);
struct ibv_context *(*dv_open_device)(struct ibv_device *device);
struct mlx5dv_devx_obj *(*devx_obj_create)
(struct ibv_context *ctx,
diff --git a/drivers/net/mlx5/mlx5_prm.h b/drivers/net/mlx5/mlx5_prm.h
index da1219e..b15266f 100644
--- a/drivers/net/mlx5/mlx5_prm.h
+++ b/drivers/net/mlx5/mlx5_prm.h
@@ -492,20 +492,40 @@ struct mlx5_ifc_fte_match_set_misc2_bits {
u8 reserved_at_1a0[0x60];
};
+struct mlx5_ifc_fte_match_set_misc3_bits {
+ u8 inner_tcp_seq_num[0x20];
+ u8 outer_tcp_seq_num[0x20];
+ u8 inner_tcp_ack_num[0x20];
+ u8 outer_tcp_ack_num[0x20];
+ u8 reserved_at_auto1[0x8];
+ u8 outer_vxlan_gpe_vni[0x18];
+ u8 outer_vxlan_gpe_next_protocol[0x8];
+ u8 outer_vxlan_gpe_flags[0x8];
+ u8 reserved_at_a8[0x10];
+ u8 icmp_header_data[0x20];
+ u8 icmpv6_header_data[0x20];
+ u8 icmp_type[0x8];
+ u8 icmp_code[0x8];
+ u8 icmpv6_type[0x8];
+ u8 icmpv6_code[0x8];
+ u8 reserved_at_1a0[0xe0];
+};
+
/* Flow matcher. */
struct mlx5_ifc_fte_match_param_bits {
struct mlx5_ifc_fte_match_set_lyr_2_4_bits outer_headers;
struct mlx5_ifc_fte_match_set_misc_bits misc_parameters;
struct mlx5_ifc_fte_match_set_lyr_2_4_bits inner_headers;
struct mlx5_ifc_fte_match_set_misc2_bits misc_parameters_2;
- u8 reserved_at_800[0x800];
+ struct mlx5_ifc_fte_match_set_misc3_bits misc_parameters_3;
};
enum {
MLX5_MATCH_CRITERIA_ENABLE_OUTER_BIT,
MLX5_MATCH_CRITERIA_ENABLE_MISC_BIT,
MLX5_MATCH_CRITERIA_ENABLE_INNER_BIT,
- MLX5_MATCH_CRITERIA_ENABLE_MISC2_BIT
+ MLX5_MATCH_CRITERIA_ENABLE_MISC2_BIT,
+ MLX5_MATCH_CRITERIA_ENABLE_MISC3_BIT
};
enum {
--
1.8.3.1
^ permalink raw reply [flat|nested] 52+ messages in thread
* [dpdk-dev] [PATCH v4 3/3] net/mlx5: add jump action support for NIC
2019-04-04 9:54 ` [dpdk-dev] [PATCH v4 0/3] net/mlx5: Add Direct Rule support Ori Kam
` (2 preceding siblings ...)
2019-04-04 9:54 ` [dpdk-dev] [PATCH v4 2/3] net/mlx5: add Direct Rules API Ori Kam
@ 2019-04-04 9:54 ` Ori Kam
2019-04-04 9:54 ` Ori Kam
2019-04-04 11:01 ` [dpdk-dev] [PATCH v4 0/3] net/mlx5: Add Direct Rule support Shahaf Shuler
4 siblings, 1 reply; 52+ messages in thread
From: Ori Kam @ 2019-04-04 9:54 UTC (permalink / raw)
To: Matan Azrad, Yongseok Koh, Shahaf Shuler; +Cc: dev, Ori Kam, Slava Ovsiienko
When using Direct Rules we can add actions to jump between tables.
This is extra useful since rule insertion rate is much higher on other
tables compared to table zero.
if no group is selected the rule is added to group 0.
Signed-off-by: Ori Kam <orika@mellanox.com>
Acked-by: Shahaf Shuler <shahafs@mellanox.com>
---
v4:
* Fix compilation issue.
---
drivers/net/mlx5/mlx5.h | 6 +
drivers/net/mlx5/mlx5_flow.h | 15 ++-
drivers/net/mlx5/mlx5_flow_dv.c | 280 +++++++++++++++++++++++++++++++++++-----
drivers/net/mlx5/mlx5_glue.c | 13 ++
drivers/net/mlx5/mlx5_glue.h | 1 +
5 files changed, 284 insertions(+), 31 deletions(-)
diff --git a/drivers/net/mlx5/mlx5.h b/drivers/net/mlx5/mlx5.h
index d4963cb..5a7597e 100644
--- a/drivers/net/mlx5/mlx5.h
+++ b/drivers/net/mlx5/mlx5.h
@@ -319,6 +319,12 @@ struct mlx5_priv {
LIST_HEAD(encap_decap, mlx5_flow_dv_encap_decap_resource) encaps_decaps;
LIST_HEAD(modify_cmd, mlx5_flow_dv_modify_hdr_resource) modify_cmds;
LIST_HEAD(tag, mlx5_flow_dv_tag_resource) tags;
+ LIST_HEAD(jump, mlx5_flow_dv_jump_tbl_resource) jump_tbl;
+ /* Pointer to next element. */
+ rte_atomic32_t refcnt; /**< Reference counter. */
+ struct ibv_flow_action *verbs_action;
+ /**< Verbs modify header action object. */
+ uint8_t ft_type; /**< Flow table type, Rx or Tx. */
/* Tags resources cache. */
uint32_t link_speed_capa; /* Link speed capabilities. */
struct mlx5_xstats_ctrl xstats_ctrl; /* Extended stats control. */
diff --git a/drivers/net/mlx5/mlx5_flow.h b/drivers/net/mlx5/mlx5_flow.h
index 8ba37a0..622e305 100644
--- a/drivers/net/mlx5/mlx5_flow.h
+++ b/drivers/net/mlx5/mlx5_flow.h
@@ -115,7 +115,8 @@
#define MLX5_FLOW_ACTION_RAW_DECAP (1u << 27)
#define MLX5_FLOW_FATE_ACTIONS \
- (MLX5_FLOW_ACTION_DROP | MLX5_FLOW_ACTION_QUEUE | MLX5_FLOW_ACTION_RSS)
+ (MLX5_FLOW_ACTION_DROP | MLX5_FLOW_ACTION_QUEUE | \
+ MLX5_FLOW_ACTION_RSS | MLX5_FLOW_ACTION_JUMP)
#define MLX5_FLOW_ENCAP_ACTIONS (MLX5_FLOW_ACTION_VXLAN_ENCAP | \
MLX5_FLOW_ACTION_NVGRE_ENCAP | \
@@ -250,6 +251,16 @@ struct mlx5_flow_dv_modify_hdr_resource {
/**< Modification actions. */
};
+/* Jump action resource structure. */
+struct mlx5_flow_dv_jump_tbl_resource {
+ LIST_ENTRY(mlx5_flow_dv_jump_tbl_resource) next;
+ /* Pointer to next element. */
+ rte_atomic32_t refcnt; /**< Reference counter. */
+ void *action; /**< Pointer to the rdma core action. */
+ uint8_t ft_type; /**< Flow table type, Rx or Tx. */
+ struct mlx5_flow_tbl_resource *tbl; /**< The target table. */
+};
+
/*
* Max number of actions per DV flow.
* See CREATE_FLOW_MAX_FLOW_ACTIONS_SUPPORTED
@@ -270,6 +281,8 @@ struct mlx5_flow_dv {
struct mlx5_flow_dv_modify_hdr_resource *modify_hdr;
/**< Pointer to modify header resource in cache. */
struct ibv_flow *flow; /**< Installed flow. */
+ struct mlx5_flow_dv_jump_tbl_resource *jump;
+ /**< Pointer to the jump action resource. */
#ifdef HAVE_IBV_FLOW_DV_SUPPORT
void *actions[MLX5_DV_MAX_NUMBER_OF_ACTIONS];
/**< Action list. */
diff --git a/drivers/net/mlx5/mlx5_flow_dv.c b/drivers/net/mlx5/mlx5_flow_dv.c
index dd5b541..10c6eee 100644
--- a/drivers/net/mlx5/mlx5_flow_dv.c
+++ b/drivers/net/mlx5/mlx5_flow_dv.c
@@ -865,6 +865,68 @@ struct field_modify_info modify_tcp[] = {
}
/**
+ * Find existing table jump resource or create and register a new one.
+ *
+ * @param dev[in, out]
+ * Pointer to rte_eth_dev structure.
+ * @param[in, out] resource
+ * Pointer to jump table resource.
+ * @parm[in, out] dev_flow
+ * Pointer to the dev_flow.
+ * @param[out] error
+ * pointer to error structure.
+ *
+ * @return
+ * 0 on success otherwise -errno and errno is set.
+ */
+static int
+flow_dv_jump_tbl_resource_register
+ (struct rte_eth_dev *dev,
+ struct mlx5_flow_dv_jump_tbl_resource *resource,
+ struct mlx5_flow *dev_flow,
+ struct rte_flow_error *error)
+{
+ struct mlx5_priv *priv = dev->data->dev_private;
+ struct mlx5_flow_dv_jump_tbl_resource *cache_resource;
+
+ /* Lookup a matching resource from cache. */
+ LIST_FOREACH(cache_resource, &priv->jump_tbl, next) {
+ if (resource->tbl == cache_resource->tbl) {
+ DRV_LOG(DEBUG, "jump table resource resource %p: refcnt %d++",
+ (void *)cache_resource,
+ rte_atomic32_read(&cache_resource->refcnt));
+ rte_atomic32_inc(&cache_resource->refcnt);
+ dev_flow->dv.jump = cache_resource;
+ return 0;
+ }
+ }
+ /* Register new jump table resource. */
+ cache_resource = rte_calloc(__func__, 1, sizeof(*cache_resource), 0);
+ if (!cache_resource)
+ return rte_flow_error_set(error, ENOMEM,
+ RTE_FLOW_ERROR_TYPE_UNSPECIFIED, NULL,
+ "cannot allocate resource memory");
+ *cache_resource = *resource;
+ cache_resource->action =
+ mlx5_glue->dr_create_flow_action_dest_flow_tbl
+ (resource->tbl->obj);
+ if (!cache_resource->action) {
+ rte_free(cache_resource);
+ return rte_flow_error_set(error, ENOMEM,
+ RTE_FLOW_ERROR_TYPE_UNSPECIFIED,
+ NULL, "cannot create action");
+ }
+ rte_atomic32_init(&cache_resource->refcnt);
+ rte_atomic32_inc(&cache_resource->refcnt);
+ LIST_INSERT_HEAD(&priv->jump_tbl, cache_resource, next);
+ dev_flow->dv.jump = cache_resource;
+ DRV_LOG(DEBUG, "new jump table resource %p: refcnt %d++",
+ (void *)cache_resource,
+ rte_atomic32_read(&cache_resource->refcnt));
+ return 0;
+}
+
+/**
* Get the size of specific rte_flow_item_type
*
* @param[in] item_type
@@ -1427,6 +1489,37 @@ struct field_modify_info modify_tcp[] = {
}
/**
+ * Validate jump action.
+ *
+ * @param[in] action
+ * Pointer to the modify action.
+ * @param[in] group
+ * The group of the current flow.
+ * @param[out] error
+ * Pointer to error structure.
+ *
+ * @return
+ * 0 on success, a negative errno value otherwise and rte_errno is set.
+ */
+static int
+flow_dv_validate_action_jump(const struct rte_flow_action *action,
+ uint32_t group,
+ struct rte_flow_error *error)
+{
+ if (action->type != RTE_FLOW_ACTION_TYPE_JUMP && !action->conf)
+ return rte_flow_error_set(error, EINVAL,
+ RTE_FLOW_ERROR_TYPE_ACTION_CONF,
+ NULL, "action configuration not set");
+ if (group >= ((const struct rte_flow_action_jump *)action->conf)->group)
+ return rte_flow_error_set(error, EINVAL,
+ RTE_FLOW_ERROR_TYPE_ACTION, NULL,
+ "target group must be higher then"
+ " the current flow group");
+ return 0;
+}
+
+
+/**
* Find existing modify-header resource or create and register a new one.
*
* @param dev[in, out]
@@ -1609,7 +1702,7 @@ struct field_modify_info modify_tcp[] = {
struct mlx5_priv *priv = dev->data->dev_private;
uint32_t priority_max = priv->config.flow_prio - 1;
-#ifdef HAVE_MLX5DV_DR
+#ifndef HAVE_MLX5DV_DR
if (attributes->group)
return rte_flow_error_set(error, ENOTSUP,
RTE_FLOW_ERROR_TYPE_ATTR_GROUP,
@@ -1981,6 +2074,14 @@ struct field_modify_info modify_tcp[] = {
MLX5_FLOW_ACTION_SET_TTL :
MLX5_FLOW_ACTION_DEC_TTL;
break;
+ case RTE_FLOW_ACTION_TYPE_JUMP:
+ ret = flow_dv_validate_action_jump(actions,
+ attr->group, error);
+ if (ret)
+ return ret;
+ ++actions_n;
+ action_flags |= MLX5_FLOW_ACTION_JUMP;
+ break;
default:
return rte_flow_error_set(error, ENOTSUP,
RTE_FLOW_ERROR_TYPE_ACTION,
@@ -2760,6 +2861,82 @@ struct field_modify_info modify_tcp[] = {
return match_criteria_enable;
}
+
+/**
+ * Get a flow table.
+ *
+ * @param dev[in, out]
+ * Pointer to rte_eth_dev structure.
+ * @param[in] table_id
+ * Table id to use.
+ * @param[in] egress
+ * Direction of the table.
+ * @param[out] error
+ * pointer to error structure.
+ *
+ * @return
+ * Returns tables resource based on the index, NULL in case of failed.
+ */
+static struct mlx5_flow_tbl_resource *
+flow_dv_tbl_resource_get(struct rte_eth_dev *dev,
+ uint32_t table_id, uint8_t egress,
+ struct rte_flow_error *error)
+{
+ struct mlx5_priv *priv = dev->data->dev_private;
+ struct mlx5_flow_tbl_resource *tbl;
+
+#ifdef HAVE_MLX5DV_DR
+ if (egress) {
+ tbl = &priv->tx_tbl[table_id];
+ if (!tbl->obj)
+ tbl->obj = mlx5_glue->dr_create_flow_tbl
+ (priv->tx_ns, table_id);
+ } else {
+ tbl = &priv->rx_tbl[table_id];
+ if (!tbl->obj)
+ tbl->obj = mlx5_glue->dr_create_flow_tbl
+ (priv->rx_ns, table_id);
+ }
+ if (!tbl->obj) {
+ rte_flow_error_set(error, ENOMEM,
+ RTE_FLOW_ERROR_TYPE_UNSPECIFIED,
+ NULL, "cannot create table");
+ return NULL;
+ }
+ rte_atomic32_inc(&tbl->refcnt);
+ return tbl;
+#else
+ (void)error;
+ (void)tbl;
+ if (egress)
+ return &priv->tx_tbl[table_id];
+ else
+ return &priv->rx_tbl[table_id];
+#endif
+}
+
+/**
+ * Release a flow table.
+ *
+ * @param[in] tbl
+ * Table resource to be released.
+ *
+ * @return
+ * Returns 0 if table was released, else return 1;
+ */
+static int
+flow_dv_tbl_resource_release(struct mlx5_flow_tbl_resource *tbl)
+{
+ if (!tbl)
+ return 0;
+ if (rte_atomic32_dec_and_test(&tbl->refcnt)) {
+ mlx5_glue->dr_destroy_flow_tbl(tbl->obj);
+ tbl->obj = NULL;
+ return 0;
+ }
+ return 1;
+}
+
/**
* Register the flow matcher.
*
@@ -2809,33 +2986,20 @@ struct field_modify_info modify_tcp[] = {
return 0;
}
}
-#ifdef HAVE_MLX5DV_DR
- if (matcher->egress) {
- tbl = &priv->tx_tbl[matcher->group];
- if (!tbl->obj)
- tbl->obj = mlx5_glue->dr_create_flow_tbl
- (priv->tx_ns,
- matcher->group * MLX5_GROUP_FACTOR);
- } else {
- tbl = &priv->rx_tbl[matcher->group];
- if (!tbl->obj)
- tbl->obj = mlx5_glue->dr_create_flow_tbl
- (priv->rx_ns,
- matcher->group * MLX5_GROUP_FACTOR);
- }
- if (!tbl->obj)
- return rte_flow_error_set(error, ENOMEM,
- RTE_FLOW_ERROR_TYPE_UNSPECIFIED,
- NULL, "cannot create table");
-
- rte_atomic32_inc(&tbl->refcnt);
-#endif
/* Register new matcher. */
cache_matcher = rte_calloc(__func__, 1, sizeof(*cache_matcher), 0);
if (!cache_matcher)
return rte_flow_error_set(error, ENOMEM,
RTE_FLOW_ERROR_TYPE_UNSPECIFIED, NULL,
"cannot allocate matcher memory");
+ tbl = flow_dv_tbl_resource_get(dev, matcher->group * MLX5_GROUP_FACTOR,
+ matcher->egress, error);
+ if (!tbl) {
+ rte_free(cache_matcher);
+ return rte_flow_error_set(error, ENOMEM,
+ RTE_FLOW_ERROR_TYPE_UNSPECIFIED,
+ NULL, "cannot create table");
+ }
*cache_matcher = *matcher;
dv_attr.match_criteria_enable =
flow_dv_matcher_enable(cache_matcher->mask.buf);
@@ -2848,10 +3012,7 @@ struct field_modify_info modify_tcp[] = {
if (!cache_matcher->matcher_object) {
rte_free(cache_matcher);
#ifdef HAVE_MLX5DV_DR
- if (rte_atomic32_dec_and_test(&tbl->refcnt)) {
- mlx5_glue->dr_destroy_flow_tbl(tbl->obj);
- tbl->obj = NULL;
- }
+ flow_dv_tbl_resource_release(tbl);
#endif
return rte_flow_error_set(error, ENOMEM,
RTE_FLOW_ERROR_TYPE_UNSPECIFIED,
@@ -3037,6 +3198,9 @@ struct field_modify_info modify_tcp[] = {
const struct rte_flow_action *action = actions;
const struct rte_flow_action_count *count = action->conf;
const uint8_t *rss_key;
+ const struct rte_flow_action_jump *jump_data;
+ struct mlx5_flow_dv_jump_tbl_resource jump_tbl_resource;
+ struct mlx5_flow_tbl_resource *tbl;
switch (actions->type) {
case RTE_FLOW_ACTION_TYPE_VOID:
@@ -3175,6 +3339,31 @@ struct field_modify_info modify_tcp[] = {
/* If decap is followed by encap, handle it at encap. */
action_flags |= MLX5_FLOW_ACTION_RAW_DECAP;
break;
+ case RTE_FLOW_ACTION_TYPE_JUMP:
+ jump_data = action->conf;
+ tbl = flow_dv_tbl_resource_get(dev, jump_data->group *
+ MLX5_GROUP_FACTOR,
+ attr->egress, error);
+ if (!tbl)
+ return rte_flow_error_set
+ (error, errno,
+ RTE_FLOW_ERROR_TYPE_ACTION,
+ NULL,
+ "cannot create jump action.");
+ jump_tbl_resource.tbl = tbl;
+ if (flow_dv_jump_tbl_resource_register
+ (dev, &jump_tbl_resource, dev_flow, error)) {
+ flow_dv_tbl_resource_release(tbl);
+ return rte_flow_error_set
+ (error, errno,
+ RTE_FLOW_ERROR_TYPE_ACTION,
+ NULL,
+ "cannot create jump action.");
+ }
+ dev_flow->dv.actions[actions_n++] =
+ dev_flow->dv.jump->action;
+ action_flags |= MLX5_FLOW_ACTION_JUMP;
+ break;
case RTE_FLOW_ACTION_TYPE_SET_MAC_SRC:
case RTE_FLOW_ACTION_TYPE_SET_MAC_DST:
if (flow_dv_convert_action_modify_mac(&res, actions,
@@ -3507,10 +3696,7 @@ struct field_modify_info modify_tcp[] = {
tbl = &priv->tx_tbl[matcher->group];
else
tbl = &priv->rx_tbl[matcher->group];
- if (rte_atomic32_dec_and_test(&tbl->refcnt)) {
- mlx5_glue->dr_destroy_flow_tbl(tbl->obj);
- tbl->obj = NULL;
- }
+ flow_dv_tbl_resource_release(tbl);
rte_free(matcher);
DRV_LOG(DEBUG, "port %u matcher %p: removed",
dev->data->port_id, (void *)matcher);
@@ -3551,6 +3737,38 @@ struct field_modify_info modify_tcp[] = {
}
/**
+ * Release an jump to table action resource.
+ *
+ * @param flow
+ * Pointer to mlx5_flow.
+ *
+ * @return
+ * 1 while a reference on it exists, 0 when freed.
+ */
+static int
+flow_dv_jump_tbl_resource_release(struct mlx5_flow *flow)
+{
+ struct mlx5_flow_dv_jump_tbl_resource *cache_resource =
+ flow->dv.jump;
+
+ assert(cache_resource->action);
+ DRV_LOG(DEBUG, "jump table resource %p: refcnt %d--",
+ (void *)cache_resource,
+ rte_atomic32_read(&cache_resource->refcnt));
+ if (rte_atomic32_dec_and_test(&cache_resource->refcnt)) {
+ claim_zero(mlx5_glue->destroy_flow_action
+ (cache_resource->action));
+ LIST_REMOVE(cache_resource, next);
+ flow_dv_tbl_resource_release(cache_resource->tbl);
+ rte_free(cache_resource);
+ DRV_LOG(DEBUG, "jump table resource %p: removed",
+ (void *)cache_resource);
+ return 0;
+ }
+ return 1;
+}
+
+/**
* Release a modify-header resource.
*
* @param flow
@@ -3646,6 +3864,8 @@ struct field_modify_info modify_tcp[] = {
flow_dv_encap_decap_resource_release(dev_flow);
if (dev_flow->dv.modify_hdr)
flow_dv_modify_hdr_resource_release(dev_flow);
+ if (dev_flow->dv.jump)
+ flow_dv_jump_tbl_resource_release(dev_flow);
rte_free(dev_flow);
}
}
diff --git a/drivers/net/mlx5/mlx5_glue.c b/drivers/net/mlx5/mlx5_glue.c
index b0b144c..f5a6c2e 100644
--- a/drivers/net/mlx5/mlx5_glue.c
+++ b/drivers/net/mlx5/mlx5_glue.c
@@ -370,6 +370,17 @@
}
static void *
+mlx5_glue_dr_create_flow_action_dest_flow_tbl(void *tbl)
+{
+#ifdef HAVE_MLX5DV_DR
+ return mlx5dv_dr_create_action_dest_flow_table(tbl);
+#else
+ (void)tbl;
+ return NULL;
+#endif
+}
+
+static void *
mlx5_glue_dr_create_flow_tbl(void *ns, uint32_t level)
{
#ifdef HAVE_MLX5DV_DR
@@ -833,6 +844,8 @@
.get_async_event = mlx5_glue_get_async_event,
.port_state_str = mlx5_glue_port_state_str,
.cq_ex_to_cq = mlx5_glue_cq_ex_to_cq,
+ .dr_create_flow_action_dest_flow_tbl =
+ mlx5_glue_dr_create_flow_action_dest_flow_tbl,
.dr_create_flow_tbl = mlx5_glue_dr_create_flow_tbl,
.dr_destroy_flow_tbl = mlx5_glue_dr_destroy_flow_tbl,
.dr_create_ns = mlx5_glue_dr_create_ns,
diff --git a/drivers/net/mlx5/mlx5_glue.h b/drivers/net/mlx5/mlx5_glue.h
index eb29ffa..058e9b1 100644
--- a/drivers/net/mlx5/mlx5_glue.h
+++ b/drivers/net/mlx5/mlx5_glue.h
@@ -145,6 +145,7 @@ struct mlx5_glue {
struct ibv_async_event *event);
const char *(*port_state_str)(enum ibv_port_state port_state);
struct ibv_cq *(*cq_ex_to_cq)(struct ibv_cq_ex *cq);
+ void *(*dr_create_flow_action_dest_flow_tbl)(void *tbl);
void *(*dr_create_flow_tbl)(void *ns, uint32_t level);
int (*dr_destroy_flow_tbl)(void *tbl);
void *(*dr_create_ns)(struct ibv_context *ctx,
--
1.8.3.1
^ permalink raw reply [flat|nested] 52+ messages in thread
* [dpdk-dev] [PATCH v4 3/3] net/mlx5: add jump action support for NIC
2019-04-04 9:54 ` [dpdk-dev] [PATCH v4 3/3] net/mlx5: add jump action support for NIC Ori Kam
@ 2019-04-04 9:54 ` Ori Kam
0 siblings, 0 replies; 52+ messages in thread
From: Ori Kam @ 2019-04-04 9:54 UTC (permalink / raw)
To: Matan Azrad, Yongseok Koh, Shahaf Shuler; +Cc: dev, Ori Kam, Slava Ovsiienko
When using Direct Rules we can add actions to jump between tables.
This is extra useful since rule insertion rate is much higher on other
tables compared to table zero.
if no group is selected the rule is added to group 0.
Signed-off-by: Ori Kam <orika@mellanox.com>
Acked-by: Shahaf Shuler <shahafs@mellanox.com>
---
v4:
* Fix compilation issue.
---
drivers/net/mlx5/mlx5.h | 6 +
drivers/net/mlx5/mlx5_flow.h | 15 ++-
drivers/net/mlx5/mlx5_flow_dv.c | 280 +++++++++++++++++++++++++++++++++++-----
drivers/net/mlx5/mlx5_glue.c | 13 ++
drivers/net/mlx5/mlx5_glue.h | 1 +
5 files changed, 284 insertions(+), 31 deletions(-)
diff --git a/drivers/net/mlx5/mlx5.h b/drivers/net/mlx5/mlx5.h
index d4963cb..5a7597e 100644
--- a/drivers/net/mlx5/mlx5.h
+++ b/drivers/net/mlx5/mlx5.h
@@ -319,6 +319,12 @@ struct mlx5_priv {
LIST_HEAD(encap_decap, mlx5_flow_dv_encap_decap_resource) encaps_decaps;
LIST_HEAD(modify_cmd, mlx5_flow_dv_modify_hdr_resource) modify_cmds;
LIST_HEAD(tag, mlx5_flow_dv_tag_resource) tags;
+ LIST_HEAD(jump, mlx5_flow_dv_jump_tbl_resource) jump_tbl;
+ /* Pointer to next element. */
+ rte_atomic32_t refcnt; /**< Reference counter. */
+ struct ibv_flow_action *verbs_action;
+ /**< Verbs modify header action object. */
+ uint8_t ft_type; /**< Flow table type, Rx or Tx. */
/* Tags resources cache. */
uint32_t link_speed_capa; /* Link speed capabilities. */
struct mlx5_xstats_ctrl xstats_ctrl; /* Extended stats control. */
diff --git a/drivers/net/mlx5/mlx5_flow.h b/drivers/net/mlx5/mlx5_flow.h
index 8ba37a0..622e305 100644
--- a/drivers/net/mlx5/mlx5_flow.h
+++ b/drivers/net/mlx5/mlx5_flow.h
@@ -115,7 +115,8 @@
#define MLX5_FLOW_ACTION_RAW_DECAP (1u << 27)
#define MLX5_FLOW_FATE_ACTIONS \
- (MLX5_FLOW_ACTION_DROP | MLX5_FLOW_ACTION_QUEUE | MLX5_FLOW_ACTION_RSS)
+ (MLX5_FLOW_ACTION_DROP | MLX5_FLOW_ACTION_QUEUE | \
+ MLX5_FLOW_ACTION_RSS | MLX5_FLOW_ACTION_JUMP)
#define MLX5_FLOW_ENCAP_ACTIONS (MLX5_FLOW_ACTION_VXLAN_ENCAP | \
MLX5_FLOW_ACTION_NVGRE_ENCAP | \
@@ -250,6 +251,16 @@ struct mlx5_flow_dv_modify_hdr_resource {
/**< Modification actions. */
};
+/* Jump action resource structure. */
+struct mlx5_flow_dv_jump_tbl_resource {
+ LIST_ENTRY(mlx5_flow_dv_jump_tbl_resource) next;
+ /* Pointer to next element. */
+ rte_atomic32_t refcnt; /**< Reference counter. */
+ void *action; /**< Pointer to the rdma core action. */
+ uint8_t ft_type; /**< Flow table type, Rx or Tx. */
+ struct mlx5_flow_tbl_resource *tbl; /**< The target table. */
+};
+
/*
* Max number of actions per DV flow.
* See CREATE_FLOW_MAX_FLOW_ACTIONS_SUPPORTED
@@ -270,6 +281,8 @@ struct mlx5_flow_dv {
struct mlx5_flow_dv_modify_hdr_resource *modify_hdr;
/**< Pointer to modify header resource in cache. */
struct ibv_flow *flow; /**< Installed flow. */
+ struct mlx5_flow_dv_jump_tbl_resource *jump;
+ /**< Pointer to the jump action resource. */
#ifdef HAVE_IBV_FLOW_DV_SUPPORT
void *actions[MLX5_DV_MAX_NUMBER_OF_ACTIONS];
/**< Action list. */
diff --git a/drivers/net/mlx5/mlx5_flow_dv.c b/drivers/net/mlx5/mlx5_flow_dv.c
index dd5b541..10c6eee 100644
--- a/drivers/net/mlx5/mlx5_flow_dv.c
+++ b/drivers/net/mlx5/mlx5_flow_dv.c
@@ -865,6 +865,68 @@ struct field_modify_info modify_tcp[] = {
}
/**
+ * Find existing table jump resource or create and register a new one.
+ *
+ * @param dev[in, out]
+ * Pointer to rte_eth_dev structure.
+ * @param[in, out] resource
+ * Pointer to jump table resource.
+ * @parm[in, out] dev_flow
+ * Pointer to the dev_flow.
+ * @param[out] error
+ * pointer to error structure.
+ *
+ * @return
+ * 0 on success otherwise -errno and errno is set.
+ */
+static int
+flow_dv_jump_tbl_resource_register
+ (struct rte_eth_dev *dev,
+ struct mlx5_flow_dv_jump_tbl_resource *resource,
+ struct mlx5_flow *dev_flow,
+ struct rte_flow_error *error)
+{
+ struct mlx5_priv *priv = dev->data->dev_private;
+ struct mlx5_flow_dv_jump_tbl_resource *cache_resource;
+
+ /* Lookup a matching resource from cache. */
+ LIST_FOREACH(cache_resource, &priv->jump_tbl, next) {
+ if (resource->tbl == cache_resource->tbl) {
+ DRV_LOG(DEBUG, "jump table resource resource %p: refcnt %d++",
+ (void *)cache_resource,
+ rte_atomic32_read(&cache_resource->refcnt));
+ rte_atomic32_inc(&cache_resource->refcnt);
+ dev_flow->dv.jump = cache_resource;
+ return 0;
+ }
+ }
+ /* Register new jump table resource. */
+ cache_resource = rte_calloc(__func__, 1, sizeof(*cache_resource), 0);
+ if (!cache_resource)
+ return rte_flow_error_set(error, ENOMEM,
+ RTE_FLOW_ERROR_TYPE_UNSPECIFIED, NULL,
+ "cannot allocate resource memory");
+ *cache_resource = *resource;
+ cache_resource->action =
+ mlx5_glue->dr_create_flow_action_dest_flow_tbl
+ (resource->tbl->obj);
+ if (!cache_resource->action) {
+ rte_free(cache_resource);
+ return rte_flow_error_set(error, ENOMEM,
+ RTE_FLOW_ERROR_TYPE_UNSPECIFIED,
+ NULL, "cannot create action");
+ }
+ rte_atomic32_init(&cache_resource->refcnt);
+ rte_atomic32_inc(&cache_resource->refcnt);
+ LIST_INSERT_HEAD(&priv->jump_tbl, cache_resource, next);
+ dev_flow->dv.jump = cache_resource;
+ DRV_LOG(DEBUG, "new jump table resource %p: refcnt %d++",
+ (void *)cache_resource,
+ rte_atomic32_read(&cache_resource->refcnt));
+ return 0;
+}
+
+/**
* Get the size of specific rte_flow_item_type
*
* @param[in] item_type
@@ -1427,6 +1489,37 @@ struct field_modify_info modify_tcp[] = {
}
/**
+ * Validate jump action.
+ *
+ * @param[in] action
+ * Pointer to the modify action.
+ * @param[in] group
+ * The group of the current flow.
+ * @param[out] error
+ * Pointer to error structure.
+ *
+ * @return
+ * 0 on success, a negative errno value otherwise and rte_errno is set.
+ */
+static int
+flow_dv_validate_action_jump(const struct rte_flow_action *action,
+ uint32_t group,
+ struct rte_flow_error *error)
+{
+ if (action->type != RTE_FLOW_ACTION_TYPE_JUMP && !action->conf)
+ return rte_flow_error_set(error, EINVAL,
+ RTE_FLOW_ERROR_TYPE_ACTION_CONF,
+ NULL, "action configuration not set");
+ if (group >= ((const struct rte_flow_action_jump *)action->conf)->group)
+ return rte_flow_error_set(error, EINVAL,
+ RTE_FLOW_ERROR_TYPE_ACTION, NULL,
+ "target group must be higher then"
+ " the current flow group");
+ return 0;
+}
+
+
+/**
* Find existing modify-header resource or create and register a new one.
*
* @param dev[in, out]
@@ -1609,7 +1702,7 @@ struct field_modify_info modify_tcp[] = {
struct mlx5_priv *priv = dev->data->dev_private;
uint32_t priority_max = priv->config.flow_prio - 1;
-#ifdef HAVE_MLX5DV_DR
+#ifndef HAVE_MLX5DV_DR
if (attributes->group)
return rte_flow_error_set(error, ENOTSUP,
RTE_FLOW_ERROR_TYPE_ATTR_GROUP,
@@ -1981,6 +2074,14 @@ struct field_modify_info modify_tcp[] = {
MLX5_FLOW_ACTION_SET_TTL :
MLX5_FLOW_ACTION_DEC_TTL;
break;
+ case RTE_FLOW_ACTION_TYPE_JUMP:
+ ret = flow_dv_validate_action_jump(actions,
+ attr->group, error);
+ if (ret)
+ return ret;
+ ++actions_n;
+ action_flags |= MLX5_FLOW_ACTION_JUMP;
+ break;
default:
return rte_flow_error_set(error, ENOTSUP,
RTE_FLOW_ERROR_TYPE_ACTION,
@@ -2760,6 +2861,82 @@ struct field_modify_info modify_tcp[] = {
return match_criteria_enable;
}
+
+/**
+ * Get a flow table.
+ *
+ * @param dev[in, out]
+ * Pointer to rte_eth_dev structure.
+ * @param[in] table_id
+ * Table id to use.
+ * @param[in] egress
+ * Direction of the table.
+ * @param[out] error
+ * pointer to error structure.
+ *
+ * @return
+ * Returns tables resource based on the index, NULL in case of failed.
+ */
+static struct mlx5_flow_tbl_resource *
+flow_dv_tbl_resource_get(struct rte_eth_dev *dev,
+ uint32_t table_id, uint8_t egress,
+ struct rte_flow_error *error)
+{
+ struct mlx5_priv *priv = dev->data->dev_private;
+ struct mlx5_flow_tbl_resource *tbl;
+
+#ifdef HAVE_MLX5DV_DR
+ if (egress) {
+ tbl = &priv->tx_tbl[table_id];
+ if (!tbl->obj)
+ tbl->obj = mlx5_glue->dr_create_flow_tbl
+ (priv->tx_ns, table_id);
+ } else {
+ tbl = &priv->rx_tbl[table_id];
+ if (!tbl->obj)
+ tbl->obj = mlx5_glue->dr_create_flow_tbl
+ (priv->rx_ns, table_id);
+ }
+ if (!tbl->obj) {
+ rte_flow_error_set(error, ENOMEM,
+ RTE_FLOW_ERROR_TYPE_UNSPECIFIED,
+ NULL, "cannot create table");
+ return NULL;
+ }
+ rte_atomic32_inc(&tbl->refcnt);
+ return tbl;
+#else
+ (void)error;
+ (void)tbl;
+ if (egress)
+ return &priv->tx_tbl[table_id];
+ else
+ return &priv->rx_tbl[table_id];
+#endif
+}
+
+/**
+ * Release a flow table.
+ *
+ * @param[in] tbl
+ * Table resource to be released.
+ *
+ * @return
+ * Returns 0 if table was released, else return 1;
+ */
+static int
+flow_dv_tbl_resource_release(struct mlx5_flow_tbl_resource *tbl)
+{
+ if (!tbl)
+ return 0;
+ if (rte_atomic32_dec_and_test(&tbl->refcnt)) {
+ mlx5_glue->dr_destroy_flow_tbl(tbl->obj);
+ tbl->obj = NULL;
+ return 0;
+ }
+ return 1;
+}
+
/**
* Register the flow matcher.
*
@@ -2809,33 +2986,20 @@ struct field_modify_info modify_tcp[] = {
return 0;
}
}
-#ifdef HAVE_MLX5DV_DR
- if (matcher->egress) {
- tbl = &priv->tx_tbl[matcher->group];
- if (!tbl->obj)
- tbl->obj = mlx5_glue->dr_create_flow_tbl
- (priv->tx_ns,
- matcher->group * MLX5_GROUP_FACTOR);
- } else {
- tbl = &priv->rx_tbl[matcher->group];
- if (!tbl->obj)
- tbl->obj = mlx5_glue->dr_create_flow_tbl
- (priv->rx_ns,
- matcher->group * MLX5_GROUP_FACTOR);
- }
- if (!tbl->obj)
- return rte_flow_error_set(error, ENOMEM,
- RTE_FLOW_ERROR_TYPE_UNSPECIFIED,
- NULL, "cannot create table");
-
- rte_atomic32_inc(&tbl->refcnt);
-#endif
/* Register new matcher. */
cache_matcher = rte_calloc(__func__, 1, sizeof(*cache_matcher), 0);
if (!cache_matcher)
return rte_flow_error_set(error, ENOMEM,
RTE_FLOW_ERROR_TYPE_UNSPECIFIED, NULL,
"cannot allocate matcher memory");
+ tbl = flow_dv_tbl_resource_get(dev, matcher->group * MLX5_GROUP_FACTOR,
+ matcher->egress, error);
+ if (!tbl) {
+ rte_free(cache_matcher);
+ return rte_flow_error_set(error, ENOMEM,
+ RTE_FLOW_ERROR_TYPE_UNSPECIFIED,
+ NULL, "cannot create table");
+ }
*cache_matcher = *matcher;
dv_attr.match_criteria_enable =
flow_dv_matcher_enable(cache_matcher->mask.buf);
@@ -2848,10 +3012,7 @@ struct field_modify_info modify_tcp[] = {
if (!cache_matcher->matcher_object) {
rte_free(cache_matcher);
#ifdef HAVE_MLX5DV_DR
- if (rte_atomic32_dec_and_test(&tbl->refcnt)) {
- mlx5_glue->dr_destroy_flow_tbl(tbl->obj);
- tbl->obj = NULL;
- }
+ flow_dv_tbl_resource_release(tbl);
#endif
return rte_flow_error_set(error, ENOMEM,
RTE_FLOW_ERROR_TYPE_UNSPECIFIED,
@@ -3037,6 +3198,9 @@ struct field_modify_info modify_tcp[] = {
const struct rte_flow_action *action = actions;
const struct rte_flow_action_count *count = action->conf;
const uint8_t *rss_key;
+ const struct rte_flow_action_jump *jump_data;
+ struct mlx5_flow_dv_jump_tbl_resource jump_tbl_resource;
+ struct mlx5_flow_tbl_resource *tbl;
switch (actions->type) {
case RTE_FLOW_ACTION_TYPE_VOID:
@@ -3175,6 +3339,31 @@ struct field_modify_info modify_tcp[] = {
/* If decap is followed by encap, handle it at encap. */
action_flags |= MLX5_FLOW_ACTION_RAW_DECAP;
break;
+ case RTE_FLOW_ACTION_TYPE_JUMP:
+ jump_data = action->conf;
+ tbl = flow_dv_tbl_resource_get(dev, jump_data->group *
+ MLX5_GROUP_FACTOR,
+ attr->egress, error);
+ if (!tbl)
+ return rte_flow_error_set
+ (error, errno,
+ RTE_FLOW_ERROR_TYPE_ACTION,
+ NULL,
+ "cannot create jump action.");
+ jump_tbl_resource.tbl = tbl;
+ if (flow_dv_jump_tbl_resource_register
+ (dev, &jump_tbl_resource, dev_flow, error)) {
+ flow_dv_tbl_resource_release(tbl);
+ return rte_flow_error_set
+ (error, errno,
+ RTE_FLOW_ERROR_TYPE_ACTION,
+ NULL,
+ "cannot create jump action.");
+ }
+ dev_flow->dv.actions[actions_n++] =
+ dev_flow->dv.jump->action;
+ action_flags |= MLX5_FLOW_ACTION_JUMP;
+ break;
case RTE_FLOW_ACTION_TYPE_SET_MAC_SRC:
case RTE_FLOW_ACTION_TYPE_SET_MAC_DST:
if (flow_dv_convert_action_modify_mac(&res, actions,
@@ -3507,10 +3696,7 @@ struct field_modify_info modify_tcp[] = {
tbl = &priv->tx_tbl[matcher->group];
else
tbl = &priv->rx_tbl[matcher->group];
- if (rte_atomic32_dec_and_test(&tbl->refcnt)) {
- mlx5_glue->dr_destroy_flow_tbl(tbl->obj);
- tbl->obj = NULL;
- }
+ flow_dv_tbl_resource_release(tbl);
rte_free(matcher);
DRV_LOG(DEBUG, "port %u matcher %p: removed",
dev->data->port_id, (void *)matcher);
@@ -3551,6 +3737,38 @@ struct field_modify_info modify_tcp[] = {
}
/**
+ * Release an jump to table action resource.
+ *
+ * @param flow
+ * Pointer to mlx5_flow.
+ *
+ * @return
+ * 1 while a reference on it exists, 0 when freed.
+ */
+static int
+flow_dv_jump_tbl_resource_release(struct mlx5_flow *flow)
+{
+ struct mlx5_flow_dv_jump_tbl_resource *cache_resource =
+ flow->dv.jump;
+
+ assert(cache_resource->action);
+ DRV_LOG(DEBUG, "jump table resource %p: refcnt %d--",
+ (void *)cache_resource,
+ rte_atomic32_read(&cache_resource->refcnt));
+ if (rte_atomic32_dec_and_test(&cache_resource->refcnt)) {
+ claim_zero(mlx5_glue->destroy_flow_action
+ (cache_resource->action));
+ LIST_REMOVE(cache_resource, next);
+ flow_dv_tbl_resource_release(cache_resource->tbl);
+ rte_free(cache_resource);
+ DRV_LOG(DEBUG, "jump table resource %p: removed",
+ (void *)cache_resource);
+ return 0;
+ }
+ return 1;
+}
+
+/**
* Release a modify-header resource.
*
* @param flow
@@ -3646,6 +3864,8 @@ struct field_modify_info modify_tcp[] = {
flow_dv_encap_decap_resource_release(dev_flow);
if (dev_flow->dv.modify_hdr)
flow_dv_modify_hdr_resource_release(dev_flow);
+ if (dev_flow->dv.jump)
+ flow_dv_jump_tbl_resource_release(dev_flow);
rte_free(dev_flow);
}
}
diff --git a/drivers/net/mlx5/mlx5_glue.c b/drivers/net/mlx5/mlx5_glue.c
index b0b144c..f5a6c2e 100644
--- a/drivers/net/mlx5/mlx5_glue.c
+++ b/drivers/net/mlx5/mlx5_glue.c
@@ -370,6 +370,17 @@
}
static void *
+mlx5_glue_dr_create_flow_action_dest_flow_tbl(void *tbl)
+{
+#ifdef HAVE_MLX5DV_DR
+ return mlx5dv_dr_create_action_dest_flow_table(tbl);
+#else
+ (void)tbl;
+ return NULL;
+#endif
+}
+
+static void *
mlx5_glue_dr_create_flow_tbl(void *ns, uint32_t level)
{
#ifdef HAVE_MLX5DV_DR
@@ -833,6 +844,8 @@
.get_async_event = mlx5_glue_get_async_event,
.port_state_str = mlx5_glue_port_state_str,
.cq_ex_to_cq = mlx5_glue_cq_ex_to_cq,
+ .dr_create_flow_action_dest_flow_tbl =
+ mlx5_glue_dr_create_flow_action_dest_flow_tbl,
.dr_create_flow_tbl = mlx5_glue_dr_create_flow_tbl,
.dr_destroy_flow_tbl = mlx5_glue_dr_destroy_flow_tbl,
.dr_create_ns = mlx5_glue_dr_create_ns,
diff --git a/drivers/net/mlx5/mlx5_glue.h b/drivers/net/mlx5/mlx5_glue.h
index eb29ffa..058e9b1 100644
--- a/drivers/net/mlx5/mlx5_glue.h
+++ b/drivers/net/mlx5/mlx5_glue.h
@@ -145,6 +145,7 @@ struct mlx5_glue {
struct ibv_async_event *event);
const char *(*port_state_str)(enum ibv_port_state port_state);
struct ibv_cq *(*cq_ex_to_cq)(struct ibv_cq_ex *cq);
+ void *(*dr_create_flow_action_dest_flow_tbl)(void *tbl);
void *(*dr_create_flow_tbl)(void *ns, uint32_t level);
int (*dr_destroy_flow_tbl)(void *tbl);
void *(*dr_create_ns)(struct ibv_context *ctx,
--
1.8.3.1
^ permalink raw reply [flat|nested] 52+ messages in thread
* Re: [dpdk-dev] [PATCH v4 0/3] net/mlx5: Add Direct Rule support
2019-04-04 9:54 ` [dpdk-dev] [PATCH v4 0/3] net/mlx5: Add Direct Rule support Ori Kam
` (3 preceding siblings ...)
2019-04-04 9:54 ` [dpdk-dev] [PATCH v4 3/3] net/mlx5: add jump action support for NIC Ori Kam
@ 2019-04-04 11:01 ` Shahaf Shuler
2019-04-04 11:01 ` Shahaf Shuler
4 siblings, 1 reply; 52+ messages in thread
From: Shahaf Shuler @ 2019-04-04 11:01 UTC (permalink / raw)
To: Ori Kam, Matan Azrad, Yongseok Koh; +Cc: dev, Slava Ovsiienko
Thursday, April 4, 2019 12:54 PM, Ori Kam:
> Subject: [PATCH v4 0/3] net/mlx5: Add Direct Rule support
>
> This patch set adds support for Direct Rules API for MLX5 driver.
>
> As more and more applications like OVS are inserting and removing large
> number of flows, the flow insertion rate become more critical.
>
> In current MLX5 PMD rules are inserted to the device using the FW, this limits
> the max insertion rate. The solution is using a new set of API that are part of
> RDMA-Core. which enables bypassing the FW and writing the flows directly
> to the device, which results in a much faster insertion rate.
>
Series applied to next-net-mlx, thanks.
> v4:
> * Fix small compilation issue.
>
> v3:
> * Address comments from ML.
> * rebase.
>
> v2:
> * Small issues fixed.
> * Rebased.
>
> Ori Kam (3):
> net/mlx5: prepare Direct Verbs for Direct Rule
> net/mlx5: add Direct Rules API
> net/mlx5: add jump action support for NIC
>
> drivers/net/mlx5/Makefile | 7 +-
> drivers/net/mlx5/meson.build | 2 +-
> drivers/net/mlx5/mlx5.c | 16 ++
> drivers/net/mlx5/mlx5.h | 23 ++
> drivers/net/mlx5/mlx5_flow.c | 1 +
> drivers/net/mlx5/mlx5_flow.h | 38 ++-
> drivers/net/mlx5/mlx5_flow_dv.c | 507
> ++++++++++++++++++++++++++++++++++------
> drivers/net/mlx5/mlx5_glue.c | 300 ++++++++++++++++++++----
> drivers/net/mlx5/mlx5_glue.h | 50 ++--
> drivers/net/mlx5/mlx5_prm.h | 24 +-
> 10 files changed, 836 insertions(+), 132 deletions(-)
>
> --
> 1.8.3.1
^ permalink raw reply [flat|nested] 52+ messages in thread
* Re: [dpdk-dev] [PATCH v4 0/3] net/mlx5: Add Direct Rule support
2019-04-04 11:01 ` [dpdk-dev] [PATCH v4 0/3] net/mlx5: Add Direct Rule support Shahaf Shuler
@ 2019-04-04 11:01 ` Shahaf Shuler
0 siblings, 0 replies; 52+ messages in thread
From: Shahaf Shuler @ 2019-04-04 11:01 UTC (permalink / raw)
To: Ori Kam, Matan Azrad, Yongseok Koh; +Cc: dev, Slava Ovsiienko
Thursday, April 4, 2019 12:54 PM, Ori Kam:
> Subject: [PATCH v4 0/3] net/mlx5: Add Direct Rule support
>
> This patch set adds support for Direct Rules API for MLX5 driver.
>
> As more and more applications like OVS are inserting and removing large
> number of flows, the flow insertion rate become more critical.
>
> In current MLX5 PMD rules are inserted to the device using the FW, this limits
> the max insertion rate. The solution is using a new set of API that are part of
> RDMA-Core. which enables bypassing the FW and writing the flows directly
> to the device, which results in a much faster insertion rate.
>
Series applied to next-net-mlx, thanks.
> v4:
> * Fix small compilation issue.
>
> v3:
> * Address comments from ML.
> * rebase.
>
> v2:
> * Small issues fixed.
> * Rebased.
>
> Ori Kam (3):
> net/mlx5: prepare Direct Verbs for Direct Rule
> net/mlx5: add Direct Rules API
> net/mlx5: add jump action support for NIC
>
> drivers/net/mlx5/Makefile | 7 +-
> drivers/net/mlx5/meson.build | 2 +-
> drivers/net/mlx5/mlx5.c | 16 ++
> drivers/net/mlx5/mlx5.h | 23 ++
> drivers/net/mlx5/mlx5_flow.c | 1 +
> drivers/net/mlx5/mlx5_flow.h | 38 ++-
> drivers/net/mlx5/mlx5_flow_dv.c | 507
> ++++++++++++++++++++++++++++++++++------
> drivers/net/mlx5/mlx5_glue.c | 300 ++++++++++++++++++++----
> drivers/net/mlx5/mlx5_glue.h | 50 ++--
> drivers/net/mlx5/mlx5_prm.h | 24 +-
> 10 files changed, 836 insertions(+), 132 deletions(-)
>
> --
> 1.8.3.1
^ permalink raw reply [flat|nested] 52+ messages in thread
* Re: [dpdk-dev] [PATCH v4 1/3] net/mlx5: prepare Direct Verbs for Direct Rule
2019-04-04 9:54 ` [dpdk-dev] [PATCH v4 1/3] net/mlx5: prepare Direct Verbs for Direct Rule Ori Kam
2019-04-04 9:54 ` Ori Kam
@ 2019-04-12 23:51 ` dwilder
2019-04-12 23:51 ` dwilder
2019-04-13 0:16 ` Yongseok Koh
1 sibling, 2 replies; 52+ messages in thread
From: dwilder @ 2019-04-12 23:51 UTC (permalink / raw)
To: Ori Kam
Cc: Matan Azrad, Yongseok Koh, Shahaf Shuler, dev, Slava Ovsiienko, pradeep
I am seeing a build break when building on ubuntu 19.04 beta after this
commit (cbb66daa3).
The build produces a number of the following errors.
drivers/net/mlx5/mlx5_glue.c:194:2: error: implicit declaration of
function ‘free’
....
drivers/net/mlx5/mlx5_glue.c:552:11: error: incompatible implicit
declaration of built-in function ‘malloc’ [-Werror]
....
Adding "#include <stdlib.h>" to drivers/net/mlx5/mlx5_glue.c fixed it
for me.
I do not see this problem on Ubuntu 18.04, maybe some headers have
changed. I have only tested on ppc64le.
On 2019-04-04 02:54, Ori Kam wrote:
> This is the first patch of a series that is designed to enable the
> Direct Rules API.
>
> The main difference between Direct Verbs and Direct Rules from API
> prespective, is that in Direct Rules each action has it's own create
> function and the object itself is of type void.
>
> In this patch I'm adding functions to generate actions that currenlty
> are done without create action, and I'm changing the action type to be
> void *, so in next patches only the glue functions will need to change.
>
> Signed-off-by: Ori Kam <orika@mellanox.com>
> Acked-by: Shahaf Shuler <shahafs@mellanox.com>
> ---
> drivers/net/mlx5/Makefile | 2 +-
> drivers/net/mlx5/meson.build | 2 +-
> drivers/net/mlx5/mlx5.h | 2 +
> drivers/net/mlx5/mlx5_flow.h | 17 +++-
> drivers/net/mlx5/mlx5_flow_dv.c | 184
> +++++++++++++++++++++++++++++-----------
> drivers/net/mlx5/mlx5_glue.c | 141 ++++++++++++++++++++++++------
> drivers/net/mlx5/mlx5_glue.h | 26 +++---
> 7 files changed, 280 insertions(+), 94 deletions(-)
>
> diff --git a/drivers/net/mlx5/Makefile b/drivers/net/mlx5/Makefile
> index ccd179c..0d20f0f 100644
> --- a/drivers/net/mlx5/Makefile
> +++ b/drivers/net/mlx5/Makefile
> @@ -8,7 +8,7 @@ include $(RTE_SDK)/mk/rte.vars.mk
> LIB = librte_pmd_mlx5.a
> LIB_GLUE = $(LIB_GLUE_BASE).$(LIB_GLUE_VERSION)
> LIB_GLUE_BASE = librte_pmd_mlx5_glue.so
> -LIB_GLUE_VERSION = 19.02.0
> +LIB_GLUE_VERSION = 19.05.0
>
> # Sources.
> SRCS-$(CONFIG_RTE_LIBRTE_MLX5_PMD) += mlx5.c
> diff --git a/drivers/net/mlx5/meson.build
> b/drivers/net/mlx5/meson.build
> index a429655..a4c684e 100644
> --- a/drivers/net/mlx5/meson.build
> +++ b/drivers/net/mlx5/meson.build
> @@ -4,7 +4,7 @@
>
> pmd_dlopen = (get_option('ibverbs_link') == 'dlopen')
> LIB_GLUE_BASE = 'librte_pmd_mlx5_glue.so'
> -LIB_GLUE_VERSION = '19.02.0'
> +LIB_GLUE_VERSION = '19.05.0'
> LIB_GLUE = LIB_GLUE_BASE + '.' + LIB_GLUE_VERSION
> if pmd_dlopen
> dpdk_conf.set('RTE_IBVERBS_LINK_DLOPEN', 1)
> diff --git a/drivers/net/mlx5/mlx5.h b/drivers/net/mlx5/mlx5.h
> index 47a7d75..784bf9b 100644
> --- a/drivers/net/mlx5/mlx5.h
> +++ b/drivers/net/mlx5/mlx5.h
> @@ -309,6 +309,8 @@ struct mlx5_priv {
> LIST_HEAD(matchers, mlx5_flow_dv_matcher) matchers;
> LIST_HEAD(encap_decap, mlx5_flow_dv_encap_decap_resource)
> encaps_decaps;
> LIST_HEAD(modify_cmd, mlx5_flow_dv_modify_hdr_resource) modify_cmds;
> + LIST_HEAD(tag, mlx5_flow_dv_tag_resource) tags;
> + /* Tags resources cache. */
> uint32_t link_speed_capa; /* Link speed capabilities. */
> struct mlx5_xstats_ctrl xstats_ctrl; /* Extended stats control. */
> struct mlx5_stats_ctrl stats_ctrl; /* Stats control. */
> diff --git a/drivers/net/mlx5/mlx5_flow.h
> b/drivers/net/mlx5/mlx5_flow.h
> index e1e798b..4f69ae2 100644
> --- a/drivers/net/mlx5/mlx5_flow.h
> +++ b/drivers/net/mlx5/mlx5_flow.h
> @@ -214,7 +214,7 @@ struct mlx5_flow_dv_encap_decap_resource {
> LIST_ENTRY(mlx5_flow_dv_encap_decap_resource) next;
> /* Pointer to next element. */
> rte_atomic32_t refcnt; /**< Reference counter. */
> - struct ibv_flow_action *verbs_action;
> + void *verbs_action;
> /**< Verbs encap/decap action object. */
> uint8_t buf[MLX5_ENCAP_MAX_LEN];
> size_t size;
> @@ -222,6 +222,16 @@ struct mlx5_flow_dv_encap_decap_resource {
> uint8_t ft_type;
> };
>
> +/* Tag resource structure. */
> +struct mlx5_flow_dv_tag_resource {
> + LIST_ENTRY(mlx5_flow_dv_tag_resource) next;
> + /* Pointer to next element. */
> + rte_atomic32_t refcnt; /**< Reference counter. */
> + void *action;
> + /**< Verbs tag action object. */
> + uint32_t tag; /**< the tag value. */
> +};
> +
> /* Number of modification commands. */
> #define MLX5_MODIFY_NUM 8
>
> @@ -259,7 +269,7 @@ struct mlx5_flow_dv {
> /**< Pointer to modify header resource in cache. */
> struct ibv_flow *flow; /**< Installed flow. */
> #ifdef HAVE_IBV_FLOW_DV_SUPPORT
> - struct mlx5dv_flow_action_attr
> actions[MLX5_DV_MAX_NUMBER_OF_ACTIONS];
> + void *actions[MLX5_DV_MAX_NUMBER_OF_ACTIONS];
> /**< Action list. */
> #endif
> int actions_n; /**< number of actions. */
> @@ -332,6 +342,7 @@ struct mlx5_flow_counter {
> };
> uint64_t hits; /**< Number of packets matched by the rule. */
> uint64_t bytes; /**< Number of bytes matched by the rule. */
> + void *action; /**< Pointer to the dv action. */
> };
>
> /* Flow structure. */
> @@ -339,6 +350,8 @@ struct rte_flow {
> TAILQ_ENTRY(rte_flow) next; /**< Pointer to the next flow structure.
> */
> enum mlx5_flow_drv_type drv_type; /**< Drvier type. */
> struct mlx5_flow_counter *counter; /**< Holds flow counter. */
> + struct mlx5_flow_dv_tag_resource *tag_resource;
> + /**< pointer to the tag action. */
> struct rte_flow_action_rss rss;/**< RSS context. */
> uint8_t key[MLX5_RSS_HASH_KEY_LEN]; /**< RSS hash key. */
> uint16_t (*queue)[]; /**< Destination queues to redirect traffic to.
> */
> diff --git a/drivers/net/mlx5/mlx5_flow_dv.c
> b/drivers/net/mlx5/mlx5_flow_dv.c
> index ad84dea..d9e2ac9 100644
> --- a/drivers/net/mlx5/mlx5_flow_dv.c
> +++ b/drivers/net/mlx5/mlx5_flow_dv.c
> @@ -32,7 +32,6 @@
> #include "mlx5_prm.h"
> #include "mlx5_glue.h"
> #include "mlx5_flow.h"
> -
> #ifdef HAVE_IBV_FLOW_DV_SUPPORT
>
> #ifndef HAVE_IBV_FLOW_DEVX_COUNTERS
> @@ -1537,6 +1536,11 @@ struct field_modify_info modify_tcp[] = {
> .id = id,
> .dcs = dcs,
> };
> + tmpl.action = mlx5_glue->dv_create_flow_action_counter(dcs->obj, 0);
> + if (!tmpl.action) {
> + ret = errno;
> + goto error_exit;
> + }
> *cnt = tmpl;
> LIST_INSERT_HEAD(&priv->flow_counters, cnt, next);
> return cnt;
> @@ -2828,6 +2832,97 @@ struct field_modify_info modify_tcp[] = {
> }
>
> /**
> + * Find existing tag resource or create and register a new one.
> + *
> + * @param dev[in, out]
> + * Pointer to rte_eth_dev structure.
> + * @param[in, out] resource
> + * Pointer to tag resource.
> + * @parm[in, out] dev_flow
> + * Pointer to the dev_flow.
> + * @param[out] error
> + * pointer to error structure.
> + *
> + * @return
> + * 0 on success otherwise -errno and errno is set.
> + */
> +static int
> +flow_dv_tag_resource_register
> + (struct rte_eth_dev *dev,
> + struct mlx5_flow_dv_tag_resource *resource,
> + struct mlx5_flow *dev_flow,
> + struct rte_flow_error *error)
> +{
> + struct mlx5_priv *priv = dev->data->dev_private;
> + struct mlx5_flow_dv_tag_resource *cache_resource;
> +
> + /* Lookup a matching resource from cache. */
> + LIST_FOREACH(cache_resource, &priv->tags, next) {
> + if (resource->tag == cache_resource->tag) {
> + DRV_LOG(DEBUG, "tag resource %p: refcnt %d++",
> + (void *)cache_resource,
> + rte_atomic32_read(&cache_resource->refcnt));
> + rte_atomic32_inc(&cache_resource->refcnt);
> + dev_flow->flow->tag_resource = cache_resource;
> + return 0;
> + }
> + }
> + /* Register new resource. */
> + cache_resource = rte_calloc(__func__, 1, sizeof(*cache_resource), 0);
> + if (!cache_resource)
> + return rte_flow_error_set(error, ENOMEM,
> + RTE_FLOW_ERROR_TYPE_UNSPECIFIED, NULL,
> + "cannot allocate resource memory");
> + *cache_resource = *resource;
> + cache_resource->action = mlx5_glue->dv_create_flow_action_tag
> + (resource->tag);
> + if (!cache_resource->action) {
> + rte_free(cache_resource);
> + return rte_flow_error_set(error, ENOMEM,
> + RTE_FLOW_ERROR_TYPE_UNSPECIFIED,
> + NULL, "cannot create action");
> + }
> + rte_atomic32_init(&cache_resource->refcnt);
> + rte_atomic32_inc(&cache_resource->refcnt);
> + LIST_INSERT_HEAD(&priv->tags, cache_resource, next);
> + dev_flow->flow->tag_resource = cache_resource;
> + DRV_LOG(DEBUG, "new tag resource %p: refcnt %d++",
> + (void *)cache_resource,
> + rte_atomic32_read(&cache_resource->refcnt));
> + return 0;
> +}
> +
> +/**
> + * Release the tag.
> + *
> + * @param dev
> + * Pointer to Ethernet device.
> + * @param flow
> + * Pointer to mlx5_flow.
> + *
> + * @return
> + * 1 while a reference on it exists, 0 when freed.
> + */
> +static int
> +flow_dv_tag_release(struct rte_eth_dev *dev,
> + struct mlx5_flow_dv_tag_resource *tag)
> +{
> + assert(tag);
> + DRV_LOG(DEBUG, "port %u tag %p: refcnt %d--",
> + dev->data->port_id, (void *)tag,
> + rte_atomic32_read(&tag->refcnt));
> + if (rte_atomic32_dec_and_test(&tag->refcnt)) {
> + claim_zero(mlx5_glue->destroy_flow_action(tag->action));
> + LIST_REMOVE(tag, next);
> + DRV_LOG(DEBUG, "port %u tag %p: removed",
> + dev->data->port_id, (void *)tag);
> + rte_free(tag);
> + return 0;
> + }
> + return 1;
> +}
> +
> +/**
> * Fill the flow with DV spec.
> *
> * @param[in] dev
> @@ -2872,6 +2967,7 @@ struct field_modify_info modify_tcp[] = {
> MLX5DV_FLOW_TABLE_TYPE_NIC_RX
> };
> union flow_dv_attr flow_attr = { .attr = 0 };
> + struct mlx5_flow_dv_tag_resource tag_resource;
>
> if (priority == MLX5_FLOW_PRIO_RSVD)
> priority = priv->config.flow_prio - 1;
> @@ -2886,26 +2982,29 @@ struct field_modify_info modify_tcp[] = {
> case RTE_FLOW_ACTION_TYPE_VOID:
> break;
> case RTE_FLOW_ACTION_TYPE_FLAG:
> - dev_flow->dv.actions[actions_n].type =
> - MLX5DV_FLOW_ACTION_TAG;
> - dev_flow->dv.actions[actions_n].tag_value =
> + tag_resource.tag =
> mlx5_flow_mark_set(MLX5_FLOW_MARK_DEFAULT);
> - actions_n++;
> + if (!flow->tag_resource)
> + if (flow_dv_tag_resource_register
> + (dev, &tag_resource, dev_flow, error))
> + return errno;
> + dev_flow->dv.actions[actions_n++] =
> + flow->tag_resource->action;
> action_flags |= MLX5_FLOW_ACTION_FLAG;
> break;
> case RTE_FLOW_ACTION_TYPE_MARK:
> - dev_flow->dv.actions[actions_n].type =
> - MLX5DV_FLOW_ACTION_TAG;
> - dev_flow->dv.actions[actions_n].tag_value =
> - mlx5_flow_mark_set
> - (((const struct rte_flow_action_mark *)
> - (actions->conf))->id);
> - actions_n++;
> + tag_resource.tag = mlx5_flow_mark_set
> + (((const struct rte_flow_action_mark *)
> + (actions->conf))->id);
> + if (!flow->tag_resource)
> + if (flow_dv_tag_resource_register
> + (dev, &tag_resource, dev_flow, error))
> + return errno;
> + dev_flow->dv.actions[actions_n++] =
> + flow->tag_resource->action;
> action_flags |= MLX5_FLOW_ACTION_MARK;
> break;
> case RTE_FLOW_ACTION_TYPE_DROP:
> - dev_flow->dv.actions[actions_n].type =
> - MLX5DV_FLOW_ACTION_DROP;
> action_flags |= MLX5_FLOW_ACTION_DROP;
> break;
> case RTE_FLOW_ACTION_TYPE_QUEUE:
> @@ -2933,17 +3032,13 @@ struct field_modify_info modify_tcp[] = {
> rte_errno = ENOTSUP;
> goto cnt_err;
> }
> - flow->counter =
> - flow_dv_counter_new(dev,
> - count->shared, count->id);
> + flow->counter = flow_dv_counter_new(dev, count->shared,
> + count->id);
> if (flow->counter == NULL)
> goto cnt_err;
> - dev_flow->dv.actions[actions_n].type =
> - MLX5DV_FLOW_ACTION_COUNTERS_DEVX;
> - dev_flow->dv.actions[actions_n].obj =
> - flow->counter->dcs->obj;
> + dev_flow->dv.actions[actions_n++] =
> + flow->counter->action;
> action_flags |= MLX5_FLOW_ACTION_COUNT;
> - ++actions_n;
> break;
> cnt_err:
> if (rte_errno == ENOTSUP)
> @@ -2964,11 +3059,8 @@ struct field_modify_info modify_tcp[] = {
> if (flow_dv_create_action_l2_encap(dev, actions,
> dev_flow, error))
> return -rte_errno;
> - dev_flow->dv.actions[actions_n].type =
> - MLX5DV_FLOW_ACTION_IBV_FLOW_ACTION;
> - dev_flow->dv.actions[actions_n].action =
> + dev_flow->dv.actions[actions_n++] =
> dev_flow->dv.encap_decap->verbs_action;
> - actions_n++;
> action_flags |= actions->type ==
> RTE_FLOW_ACTION_TYPE_VXLAN_ENCAP ?
> MLX5_FLOW_ACTION_VXLAN_ENCAP :
> @@ -2979,11 +3071,8 @@ struct field_modify_info modify_tcp[] = {
> if (flow_dv_create_action_l2_decap(dev, dev_flow,
> error))
> return -rte_errno;
> - dev_flow->dv.actions[actions_n].type =
> - MLX5DV_FLOW_ACTION_IBV_FLOW_ACTION;
> - dev_flow->dv.actions[actions_n].action =
> + dev_flow->dv.actions[actions_n++] =
> dev_flow->dv.encap_decap->verbs_action;
> - actions_n++;
> action_flags |= actions->type ==
> RTE_FLOW_ACTION_TYPE_VXLAN_DECAP ?
> MLX5_FLOW_ACTION_VXLAN_DECAP :
> @@ -2995,9 +3084,7 @@ struct field_modify_info modify_tcp[] = {
> if (flow_dv_create_action_raw_encap
> (dev, actions, dev_flow, attr, error))
> return -rte_errno;
> - dev_flow->dv.actions[actions_n].type =
> - MLX5DV_FLOW_ACTION_IBV_FLOW_ACTION;
> - dev_flow->dv.actions[actions_n].action =
> + dev_flow->dv.actions[actions_n++] =
> dev_flow->dv.encap_decap->verbs_action;
> } else {
> /* Handle encap without preceding decap. */
> @@ -3005,12 +3092,9 @@ struct field_modify_info modify_tcp[] = {
> dev_flow,
> error))
> return -rte_errno;
> - dev_flow->dv.actions[actions_n].type =
> - MLX5DV_FLOW_ACTION_IBV_FLOW_ACTION;
> - dev_flow->dv.actions[actions_n].action =
> + dev_flow->dv.actions[actions_n++] =
> dev_flow->dv.encap_decap->verbs_action;
> }
> - actions_n++;
> action_flags |= MLX5_FLOW_ACTION_RAW_ENCAP;
> break;
> case RTE_FLOW_ACTION_TYPE_RAW_DECAP:
> @@ -3025,11 +3109,8 @@ struct field_modify_info modify_tcp[] = {
> dev_flow,
> error))
> return -rte_errno;
> - dev_flow->dv.actions[actions_n].type =
> - MLX5DV_FLOW_ACTION_IBV_FLOW_ACTION;
> - dev_flow->dv.actions[actions_n].action =
> + dev_flow->dv.actions[actions_n++] =
> dev_flow->dv.encap_decap->verbs_action;
> - actions_n++;
> }
> /* If decap is followed by encap, handle it at encap. */
> action_flags |= MLX5_FLOW_ACTION_RAW_DECAP;
> @@ -3098,11 +3179,8 @@ struct field_modify_info modify_tcp[] = {
> dev_flow,
> error))
> return -rte_errno;
> - dev_flow->dv.actions[actions_n].type =
> - MLX5DV_FLOW_ACTION_IBV_FLOW_ACTION;
> - dev_flow->dv.actions[actions_n].action =
> + dev_flow->dv.actions[actions_n++] =
> dev_flow->dv.modify_hdr->verbs_action;
> - actions_n++;
> }
> break;
> default:
> @@ -3277,9 +3355,9 @@ struct field_modify_info modify_tcp[] = {
> "cannot get drop hash queue");
> goto error;
> }
> - dv->actions[n].type = MLX5DV_FLOW_ACTION_DEST_IBV_QP;
> - dv->actions[n].qp = dv->hrxq->qp;
> - n++;
> + dv->actions[n++] =
> + mlx5_glue->dv_create_flow_action_dest_ibv_qp
> + (dv->hrxq->qp);
> } else if (flow->actions &
> (MLX5_FLOW_ACTION_QUEUE | MLX5_FLOW_ACTION_RSS)) {
> struct mlx5_hrxq *hrxq;
> @@ -3304,9 +3382,9 @@ struct field_modify_info modify_tcp[] = {
> goto error;
> }
> dv->hrxq = hrxq;
> - dv->actions[n].type = MLX5DV_FLOW_ACTION_DEST_IBV_QP;
> - dv->actions[n].qp = hrxq->qp;
> - n++;
> + dv->actions[n++] =
> + mlx5_glue->dv_create_flow_action_dest_ibv_qp
> + (dv->hrxq->qp);
> }
> dv->flow =
> mlx5_glue->dv_create_flow(dv->matcher->matcher_object,
> @@ -3484,6 +3562,10 @@ struct field_modify_info modify_tcp[] = {
> flow_dv_counter_release(flow->counter);
> flow->counter = NULL;
> }
> + if (flow->tag_resource) {
> + flow_dv_tag_release(dev, flow->tag_resource);
> + flow->tag_resource = NULL;
> + }
> while (!LIST_EMPTY(&flow->dev_flows)) {
> dev_flow = LIST_FIRST(&flow->dev_flows);
> LIST_REMOVE(dev_flow, next);
> diff --git a/drivers/net/mlx5/mlx5_glue.c
> b/drivers/net/mlx5/mlx5_glue.c
> index c817d86..4b5aade 100644
> --- a/drivers/net/mlx5/mlx5_glue.c
> +++ b/drivers/net/mlx5/mlx5_glue.c
> @@ -175,10 +175,20 @@
> }
>
> static int
> -mlx5_glue_destroy_flow_action(struct ibv_flow_action *action)
> +mlx5_glue_destroy_flow_action(void *action)
> {
> #ifdef HAVE_IBV_FLOW_DV_SUPPORT
> - return ibv_destroy_flow_action(action);
> + struct mlx5dv_flow_action_attr *attr = action;
> + int res = 0;
> + switch (attr->type) {
> + case MLX5DV_FLOW_ACTION_TAG:
> + break;
> + default:
> + res = ibv_destroy_flow_action(attr->action);
> + break;
> + }
> + free(action);
> + return res;
> #else
> (void)action;
> return ENOTSUP;
> @@ -430,16 +440,23 @@
> mlx5_glue_dv_create_flow(struct mlx5dv_flow_matcher *matcher,
> struct mlx5dv_flow_match_parameters *match_value,
> size_t num_actions,
> - struct mlx5dv_flow_action_attr *actions_attr)
> + void *actions[])
> {
> #ifdef HAVE_IBV_FLOW_DV_SUPPORT
> + struct mlx5dv_flow_action_attr actions_attr[8];
> +
> + if (num_actions > 8)
> + return NULL;
> + for (size_t i = 0; i < num_actions; i++)
> + actions_attr[i] =
> + *((struct mlx5dv_flow_action_attr *)(actions[i]));
> return mlx5dv_create_flow(matcher, match_value,
> num_actions, actions_attr);
> #else
> (void)matcher;
> (void)match_value;
> (void)num_actions;
> - (void)actions_attr;
> + (void)actions;
> return NULL;
> #endif
> }
> @@ -455,31 +472,45 @@
> #endif
> }
>
> -static struct ibv_flow_action *
> -mlx5_glue_dv_create_flow_action_packet_reformat
> - (struct ibv_context *ctx,
> - size_t data_sz,
> - void *data,
> - enum mlx5dv_flow_action_packet_reformat_type reformat_type,
> - enum mlx5dv_flow_table_type ft_type)
> +static void *
> +mlx5_glue_dv_create_flow_action_counter(void *counter_obj, uint32_t
> offset)
> {
> #ifdef HAVE_IBV_FLOW_DV_SUPPORT
> - return mlx5dv_create_flow_action_packet_reformat(ctx,
> - data_sz,
> - data,
> - reformat_type,
> - ft_type);
> + struct mlx5dv_flow_action_attr *action;
> +
> + (void)offset;
> + action = malloc(sizeof(*action));
> + if (!action)
> + return NULL;
> + action->type = MLX5DV_FLOW_ACTION_COUNTERS_DEVX;
> + action->obj = counter_obj;
> + return action;
> #else
> - (void)ctx;
> - (void)data_sz;
> - (void)data;
> - (void)reformat_type;
> - (void)ft_type;
> + (void)counter_obj;
> + (void)offset;
> return NULL;
> #endif
> }
>
> -static struct ibv_flow_action *
> +static void *
> +mlx5_glue_dv_create_flow_action_dest_ibv_qp(void *qp)
> +{
> +#ifdef HAVE_IBV_FLOW_DV_SUPPORT
> + struct mlx5dv_flow_action_attr *action;
> +
> + action = malloc(sizeof(*action));
> + if (!action)
> + return NULL;
> + action->type = MLX5DV_FLOW_ACTION_DEST_IBV_QP;
> + action->obj = qp;
> + return action;
> +#else
> + (void)qp;
> + return NULL;
> +#endif
> +}
> +
> +static void *
> mlx5_glue_dv_create_flow_action_modify_header
> (struct ibv_context *ctx,
> size_t actions_sz,
> @@ -487,8 +518,15 @@
> enum mlx5dv_flow_table_type ft_type)
> {
> #ifdef HAVE_IBV_FLOW_DV_SUPPORT
> - return mlx5dv_create_flow_action_modify_header(ctx, actions_sz,
> - actions, ft_type);
> + struct mlx5dv_flow_action_attr *action;
> +
> + action = malloc(sizeof(*action));
> + if (!action)
> + return NULL;
> + action->type = MLX5DV_FLOW_ACTION_IBV_FLOW_ACTION;
> + action->action = mlx5dv_create_flow_action_modify_header
> + (ctx, actions_sz, actions, ft_type);
> + return action;
> #else
> (void)ctx;
> (void)actions_sz;
> @@ -498,6 +536,50 @@
> #endif
> }
>
> +static void *
> +mlx5_glue_dv_create_flow_action_packet_reformat
> + (struct ibv_context *ctx,
> + size_t data_sz,
> + void *data,
> + enum mlx5dv_flow_action_packet_reformat_type reformat_type,
> + enum mlx5dv_flow_table_type ft_type)
> +{
> +#ifdef HAVE_IBV_FLOW_DV_SUPPORT
> + struct mlx5dv_flow_action_attr *action;
> +
> + action = malloc(sizeof(*action));
> + if (!action)
> + return NULL;
> + action->type = MLX5DV_FLOW_ACTION_IBV_FLOW_ACTION;
> + action->action = mlx5dv_create_flow_action_packet_reformat
> + (ctx, data_sz, data, reformat_type, ft_type);
> + return action;
> +#else
> + (void)ctx;
> + (void)data_sz;
> + (void)data;
> + (void)reformat_type;
> + (void)ft_type;
> + return NULL;
> +#endif
> +}
> +
> +static void *
> +mlx5_glue_dv_create_flow_action_tag(uint32_t tag)
> +{
> +#ifdef HAVE_IBV_FLOW_DV_SUPPORT
> + struct mlx5dv_flow_action_attr *action;
> + action = malloc(sizeof(*action));
> + if (!action)
> + return NULL;
> + action->type = MLX5DV_FLOW_ACTION_TAG;
> + action->tag_value = tag;
> + return action;
> +#endif
> + (void)tag;
> + return NULL;
> +}
> +
> static struct ibv_context *
> mlx5_glue_dv_open_device(struct ibv_device *device)
> {
> @@ -645,10 +727,15 @@
> .dv_create_flow_matcher = mlx5_glue_dv_create_flow_matcher,
> .dv_destroy_flow_matcher = mlx5_glue_dv_destroy_flow_matcher,
> .dv_create_flow = mlx5_glue_dv_create_flow,
> - .dv_create_flow_action_packet_reformat =
> - mlx5_glue_dv_create_flow_action_packet_reformat,
> + .dv_create_flow_action_counter =
> + mlx5_glue_dv_create_flow_action_counter,
> + .dv_create_flow_action_dest_ibv_qp =
> + mlx5_glue_dv_create_flow_action_dest_ibv_qp,
> .dv_create_flow_action_modify_header =
> - mlx5_glue_dv_create_flow_action_modify_header,
> + mlx5_glue_dv_create_flow_action_modify_header,
> + .dv_create_flow_action_packet_reformat =
> + mlx5_glue_dv_create_flow_action_packet_reformat,
> + .dv_create_flow_action_tag = mlx5_glue_dv_create_flow_action_tag,
> .dv_open_device = mlx5_glue_dv_open_device,
> .devx_obj_create = mlx5_glue_devx_obj_create,
> .devx_obj_destroy = mlx5_glue_devx_obj_destroy,
> diff --git a/drivers/net/mlx5/mlx5_glue.h
> b/drivers/net/mlx5/mlx5_glue.h
> index b118960..32487ea 100644
> --- a/drivers/net/mlx5/mlx5_glue.h
> +++ b/drivers/net/mlx5/mlx5_glue.h
> @@ -55,6 +55,10 @@
> enum mlx5dv_flow_table_type { flow_table_type = 0, };
> #endif
>
> +#ifndef HAVE_IBV_FLOW_DEVX_COUNTERS
> +#define MLX5DV_FLOW_ACTION_COUNTERS_DEVX 0
> +#endif
> +
> #ifndef HAVE_IBV_DEVX_OBJ
> struct mlx5dv_devx_obj;
> #endif
> @@ -98,7 +102,7 @@ struct mlx5_glue {
> struct ibv_flow *(*create_flow)(struct ibv_qp *qp,
> struct ibv_flow_attr *flow);
> int (*destroy_flow)(struct ibv_flow *flow_id);
> - int (*destroy_flow_action)(struct ibv_flow_action *action);
> + int (*destroy_flow_action)(void *action);
> struct ibv_qp *(*create_qp)(struct ibv_pd *pd,
> struct ibv_qp_init_attr *qp_init_attr);
> struct ibv_qp *(*create_qp_ex)
> @@ -160,19 +164,17 @@ struct mlx5_glue {
> int (*dv_destroy_flow_matcher)(struct mlx5dv_flow_matcher *matcher);
> struct ibv_flow *(*dv_create_flow)(struct mlx5dv_flow_matcher
> *matcher,
> struct mlx5dv_flow_match_parameters *match_value,
> - size_t num_actions,
> - struct mlx5dv_flow_action_attr *actions_attr);
> - struct ibv_flow_action *(*dv_create_flow_action_packet_reformat)
> - (struct ibv_context *ctx,
> - size_t data_sz,
> - void *data,
> + size_t num_actions, void *actions[]);
> + void *(*dv_create_flow_action_counter)(void *obj, uint32_t offset);
> + void *(*dv_create_flow_action_dest_ibv_qp)(void *qp);
> + void *(*dv_create_flow_action_modify_header)
> + (struct ibv_context *ctx, size_t actions_sz, uint64_t actions[],
> + enum mlx5dv_flow_table_type ft_type);
> + void *(*dv_create_flow_action_packet_reformat)
> + (struct ibv_context *ctx, size_t data_sz, void *data,
> enum mlx5dv_flow_action_packet_reformat_type reformat_type,
> enum mlx5dv_flow_table_type ft_type);
> - struct ibv_flow_action *(*dv_create_flow_action_modify_header)
> - (struct ibv_context *ctx,
> - size_t actions_sz,
> - uint64_t actions[],
> - enum mlx5dv_flow_table_type ft_type);
> + void *(*dv_create_flow_action_tag)(uint32_t tag);
> struct ibv_context *(*dv_open_device)(struct ibv_device *device);
> struct mlx5dv_devx_obj *(*devx_obj_create)
> (struct ibv_context *ctx,
^ permalink raw reply [flat|nested] 52+ messages in thread
* Re: [dpdk-dev] [PATCH v4 1/3] net/mlx5: prepare Direct Verbs for Direct Rule
2019-04-12 23:51 ` dwilder
@ 2019-04-12 23:51 ` dwilder
2019-04-13 0:16 ` Yongseok Koh
1 sibling, 0 replies; 52+ messages in thread
From: dwilder @ 2019-04-12 23:51 UTC (permalink / raw)
To: Ori Kam
Cc: Matan Azrad, Yongseok Koh, Shahaf Shuler, dev, Slava Ovsiienko, pradeep
I am seeing a build break when building on ubuntu 19.04 beta after this
commit (cbb66daa3).
The build produces a number of the following errors.
drivers/net/mlx5/mlx5_glue.c:194:2: error: implicit declaration of
function ‘free’
....
drivers/net/mlx5/mlx5_glue.c:552:11: error: incompatible implicit
declaration of built-in function ‘malloc’ [-Werror]
....
Adding "#include <stdlib.h>" to drivers/net/mlx5/mlx5_glue.c fixed it
for me.
I do not see this problem on Ubuntu 18.04, maybe some headers have
changed. I have only tested on ppc64le.
On 2019-04-04 02:54, Ori Kam wrote:
> This is the first patch of a series that is designed to enable the
> Direct Rules API.
>
> The main difference between Direct Verbs and Direct Rules from API
> prespective, is that in Direct Rules each action has it's own create
> function and the object itself is of type void.
>
> In this patch I'm adding functions to generate actions that currenlty
> are done without create action, and I'm changing the action type to be
> void *, so in next patches only the glue functions will need to change.
>
> Signed-off-by: Ori Kam <orika@mellanox.com>
> Acked-by: Shahaf Shuler <shahafs@mellanox.com>
> ---
> drivers/net/mlx5/Makefile | 2 +-
> drivers/net/mlx5/meson.build | 2 +-
> drivers/net/mlx5/mlx5.h | 2 +
> drivers/net/mlx5/mlx5_flow.h | 17 +++-
> drivers/net/mlx5/mlx5_flow_dv.c | 184
> +++++++++++++++++++++++++++++-----------
> drivers/net/mlx5/mlx5_glue.c | 141 ++++++++++++++++++++++++------
> drivers/net/mlx5/mlx5_glue.h | 26 +++---
> 7 files changed, 280 insertions(+), 94 deletions(-)
>
> diff --git a/drivers/net/mlx5/Makefile b/drivers/net/mlx5/Makefile
> index ccd179c..0d20f0f 100644
> --- a/drivers/net/mlx5/Makefile
> +++ b/drivers/net/mlx5/Makefile
> @@ -8,7 +8,7 @@ include $(RTE_SDK)/mk/rte.vars.mk
> LIB = librte_pmd_mlx5.a
> LIB_GLUE = $(LIB_GLUE_BASE).$(LIB_GLUE_VERSION)
> LIB_GLUE_BASE = librte_pmd_mlx5_glue.so
> -LIB_GLUE_VERSION = 19.02.0
> +LIB_GLUE_VERSION = 19.05.0
>
> # Sources.
> SRCS-$(CONFIG_RTE_LIBRTE_MLX5_PMD) += mlx5.c
> diff --git a/drivers/net/mlx5/meson.build
> b/drivers/net/mlx5/meson.build
> index a429655..a4c684e 100644
> --- a/drivers/net/mlx5/meson.build
> +++ b/drivers/net/mlx5/meson.build
> @@ -4,7 +4,7 @@
>
> pmd_dlopen = (get_option('ibverbs_link') == 'dlopen')
> LIB_GLUE_BASE = 'librte_pmd_mlx5_glue.so'
> -LIB_GLUE_VERSION = '19.02.0'
> +LIB_GLUE_VERSION = '19.05.0'
> LIB_GLUE = LIB_GLUE_BASE + '.' + LIB_GLUE_VERSION
> if pmd_dlopen
> dpdk_conf.set('RTE_IBVERBS_LINK_DLOPEN', 1)
> diff --git a/drivers/net/mlx5/mlx5.h b/drivers/net/mlx5/mlx5.h
> index 47a7d75..784bf9b 100644
> --- a/drivers/net/mlx5/mlx5.h
> +++ b/drivers/net/mlx5/mlx5.h
> @@ -309,6 +309,8 @@ struct mlx5_priv {
> LIST_HEAD(matchers, mlx5_flow_dv_matcher) matchers;
> LIST_HEAD(encap_decap, mlx5_flow_dv_encap_decap_resource)
> encaps_decaps;
> LIST_HEAD(modify_cmd, mlx5_flow_dv_modify_hdr_resource) modify_cmds;
> + LIST_HEAD(tag, mlx5_flow_dv_tag_resource) tags;
> + /* Tags resources cache. */
> uint32_t link_speed_capa; /* Link speed capabilities. */
> struct mlx5_xstats_ctrl xstats_ctrl; /* Extended stats control. */
> struct mlx5_stats_ctrl stats_ctrl; /* Stats control. */
> diff --git a/drivers/net/mlx5/mlx5_flow.h
> b/drivers/net/mlx5/mlx5_flow.h
> index e1e798b..4f69ae2 100644
> --- a/drivers/net/mlx5/mlx5_flow.h
> +++ b/drivers/net/mlx5/mlx5_flow.h
> @@ -214,7 +214,7 @@ struct mlx5_flow_dv_encap_decap_resource {
> LIST_ENTRY(mlx5_flow_dv_encap_decap_resource) next;
> /* Pointer to next element. */
> rte_atomic32_t refcnt; /**< Reference counter. */
> - struct ibv_flow_action *verbs_action;
> + void *verbs_action;
> /**< Verbs encap/decap action object. */
> uint8_t buf[MLX5_ENCAP_MAX_LEN];
> size_t size;
> @@ -222,6 +222,16 @@ struct mlx5_flow_dv_encap_decap_resource {
> uint8_t ft_type;
> };
>
> +/* Tag resource structure. */
> +struct mlx5_flow_dv_tag_resource {
> + LIST_ENTRY(mlx5_flow_dv_tag_resource) next;
> + /* Pointer to next element. */
> + rte_atomic32_t refcnt; /**< Reference counter. */
> + void *action;
> + /**< Verbs tag action object. */
> + uint32_t tag; /**< the tag value. */
> +};
> +
> /* Number of modification commands. */
> #define MLX5_MODIFY_NUM 8
>
> @@ -259,7 +269,7 @@ struct mlx5_flow_dv {
> /**< Pointer to modify header resource in cache. */
> struct ibv_flow *flow; /**< Installed flow. */
> #ifdef HAVE_IBV_FLOW_DV_SUPPORT
> - struct mlx5dv_flow_action_attr
> actions[MLX5_DV_MAX_NUMBER_OF_ACTIONS];
> + void *actions[MLX5_DV_MAX_NUMBER_OF_ACTIONS];
> /**< Action list. */
> #endif
> int actions_n; /**< number of actions. */
> @@ -332,6 +342,7 @@ struct mlx5_flow_counter {
> };
> uint64_t hits; /**< Number of packets matched by the rule. */
> uint64_t bytes; /**< Number of bytes matched by the rule. */
> + void *action; /**< Pointer to the dv action. */
> };
>
> /* Flow structure. */
> @@ -339,6 +350,8 @@ struct rte_flow {
> TAILQ_ENTRY(rte_flow) next; /**< Pointer to the next flow structure.
> */
> enum mlx5_flow_drv_type drv_type; /**< Drvier type. */
> struct mlx5_flow_counter *counter; /**< Holds flow counter. */
> + struct mlx5_flow_dv_tag_resource *tag_resource;
> + /**< pointer to the tag action. */
> struct rte_flow_action_rss rss;/**< RSS context. */
> uint8_t key[MLX5_RSS_HASH_KEY_LEN]; /**< RSS hash key. */
> uint16_t (*queue)[]; /**< Destination queues to redirect traffic to.
> */
> diff --git a/drivers/net/mlx5/mlx5_flow_dv.c
> b/drivers/net/mlx5/mlx5_flow_dv.c
> index ad84dea..d9e2ac9 100644
> --- a/drivers/net/mlx5/mlx5_flow_dv.c
> +++ b/drivers/net/mlx5/mlx5_flow_dv.c
> @@ -32,7 +32,6 @@
> #include "mlx5_prm.h"
> #include "mlx5_glue.h"
> #include "mlx5_flow.h"
> -
> #ifdef HAVE_IBV_FLOW_DV_SUPPORT
>
> #ifndef HAVE_IBV_FLOW_DEVX_COUNTERS
> @@ -1537,6 +1536,11 @@ struct field_modify_info modify_tcp[] = {
> .id = id,
> .dcs = dcs,
> };
> + tmpl.action = mlx5_glue->dv_create_flow_action_counter(dcs->obj, 0);
> + if (!tmpl.action) {
> + ret = errno;
> + goto error_exit;
> + }
> *cnt = tmpl;
> LIST_INSERT_HEAD(&priv->flow_counters, cnt, next);
> return cnt;
> @@ -2828,6 +2832,97 @@ struct field_modify_info modify_tcp[] = {
> }
>
> /**
> + * Find existing tag resource or create and register a new one.
> + *
> + * @param dev[in, out]
> + * Pointer to rte_eth_dev structure.
> + * @param[in, out] resource
> + * Pointer to tag resource.
> + * @parm[in, out] dev_flow
> + * Pointer to the dev_flow.
> + * @param[out] error
> + * pointer to error structure.
> + *
> + * @return
> + * 0 on success otherwise -errno and errno is set.
> + */
> +static int
> +flow_dv_tag_resource_register
> + (struct rte_eth_dev *dev,
> + struct mlx5_flow_dv_tag_resource *resource,
> + struct mlx5_flow *dev_flow,
> + struct rte_flow_error *error)
> +{
> + struct mlx5_priv *priv = dev->data->dev_private;
> + struct mlx5_flow_dv_tag_resource *cache_resource;
> +
> + /* Lookup a matching resource from cache. */
> + LIST_FOREACH(cache_resource, &priv->tags, next) {
> + if (resource->tag == cache_resource->tag) {
> + DRV_LOG(DEBUG, "tag resource %p: refcnt %d++",
> + (void *)cache_resource,
> + rte_atomic32_read(&cache_resource->refcnt));
> + rte_atomic32_inc(&cache_resource->refcnt);
> + dev_flow->flow->tag_resource = cache_resource;
> + return 0;
> + }
> + }
> + /* Register new resource. */
> + cache_resource = rte_calloc(__func__, 1, sizeof(*cache_resource), 0);
> + if (!cache_resource)
> + return rte_flow_error_set(error, ENOMEM,
> + RTE_FLOW_ERROR_TYPE_UNSPECIFIED, NULL,
> + "cannot allocate resource memory");
> + *cache_resource = *resource;
> + cache_resource->action = mlx5_glue->dv_create_flow_action_tag
> + (resource->tag);
> + if (!cache_resource->action) {
> + rte_free(cache_resource);
> + return rte_flow_error_set(error, ENOMEM,
> + RTE_FLOW_ERROR_TYPE_UNSPECIFIED,
> + NULL, "cannot create action");
> + }
> + rte_atomic32_init(&cache_resource->refcnt);
> + rte_atomic32_inc(&cache_resource->refcnt);
> + LIST_INSERT_HEAD(&priv->tags, cache_resource, next);
> + dev_flow->flow->tag_resource = cache_resource;
> + DRV_LOG(DEBUG, "new tag resource %p: refcnt %d++",
> + (void *)cache_resource,
> + rte_atomic32_read(&cache_resource->refcnt));
> + return 0;
> +}
> +
> +/**
> + * Release the tag.
> + *
> + * @param dev
> + * Pointer to Ethernet device.
> + * @param flow
> + * Pointer to mlx5_flow.
> + *
> + * @return
> + * 1 while a reference on it exists, 0 when freed.
> + */
> +static int
> +flow_dv_tag_release(struct rte_eth_dev *dev,
> + struct mlx5_flow_dv_tag_resource *tag)
> +{
> + assert(tag);
> + DRV_LOG(DEBUG, "port %u tag %p: refcnt %d--",
> + dev->data->port_id, (void *)tag,
> + rte_atomic32_read(&tag->refcnt));
> + if (rte_atomic32_dec_and_test(&tag->refcnt)) {
> + claim_zero(mlx5_glue->destroy_flow_action(tag->action));
> + LIST_REMOVE(tag, next);
> + DRV_LOG(DEBUG, "port %u tag %p: removed",
> + dev->data->port_id, (void *)tag);
> + rte_free(tag);
> + return 0;
> + }
> + return 1;
> +}
> +
> +/**
> * Fill the flow with DV spec.
> *
> * @param[in] dev
> @@ -2872,6 +2967,7 @@ struct field_modify_info modify_tcp[] = {
> MLX5DV_FLOW_TABLE_TYPE_NIC_RX
> };
> union flow_dv_attr flow_attr = { .attr = 0 };
> + struct mlx5_flow_dv_tag_resource tag_resource;
>
> if (priority == MLX5_FLOW_PRIO_RSVD)
> priority = priv->config.flow_prio - 1;
> @@ -2886,26 +2982,29 @@ struct field_modify_info modify_tcp[] = {
> case RTE_FLOW_ACTION_TYPE_VOID:
> break;
> case RTE_FLOW_ACTION_TYPE_FLAG:
> - dev_flow->dv.actions[actions_n].type =
> - MLX5DV_FLOW_ACTION_TAG;
> - dev_flow->dv.actions[actions_n].tag_value =
> + tag_resource.tag =
> mlx5_flow_mark_set(MLX5_FLOW_MARK_DEFAULT);
> - actions_n++;
> + if (!flow->tag_resource)
> + if (flow_dv_tag_resource_register
> + (dev, &tag_resource, dev_flow, error))
> + return errno;
> + dev_flow->dv.actions[actions_n++] =
> + flow->tag_resource->action;
> action_flags |= MLX5_FLOW_ACTION_FLAG;
> break;
> case RTE_FLOW_ACTION_TYPE_MARK:
> - dev_flow->dv.actions[actions_n].type =
> - MLX5DV_FLOW_ACTION_TAG;
> - dev_flow->dv.actions[actions_n].tag_value =
> - mlx5_flow_mark_set
> - (((const struct rte_flow_action_mark *)
> - (actions->conf))->id);
> - actions_n++;
> + tag_resource.tag = mlx5_flow_mark_set
> + (((const struct rte_flow_action_mark *)
> + (actions->conf))->id);
> + if (!flow->tag_resource)
> + if (flow_dv_tag_resource_register
> + (dev, &tag_resource, dev_flow, error))
> + return errno;
> + dev_flow->dv.actions[actions_n++] =
> + flow->tag_resource->action;
> action_flags |= MLX5_FLOW_ACTION_MARK;
> break;
> case RTE_FLOW_ACTION_TYPE_DROP:
> - dev_flow->dv.actions[actions_n].type =
> - MLX5DV_FLOW_ACTION_DROP;
> action_flags |= MLX5_FLOW_ACTION_DROP;
> break;
> case RTE_FLOW_ACTION_TYPE_QUEUE:
> @@ -2933,17 +3032,13 @@ struct field_modify_info modify_tcp[] = {
> rte_errno = ENOTSUP;
> goto cnt_err;
> }
> - flow->counter =
> - flow_dv_counter_new(dev,
> - count->shared, count->id);
> + flow->counter = flow_dv_counter_new(dev, count->shared,
> + count->id);
> if (flow->counter == NULL)
> goto cnt_err;
> - dev_flow->dv.actions[actions_n].type =
> - MLX5DV_FLOW_ACTION_COUNTERS_DEVX;
> - dev_flow->dv.actions[actions_n].obj =
> - flow->counter->dcs->obj;
> + dev_flow->dv.actions[actions_n++] =
> + flow->counter->action;
> action_flags |= MLX5_FLOW_ACTION_COUNT;
> - ++actions_n;
> break;
> cnt_err:
> if (rte_errno == ENOTSUP)
> @@ -2964,11 +3059,8 @@ struct field_modify_info modify_tcp[] = {
> if (flow_dv_create_action_l2_encap(dev, actions,
> dev_flow, error))
> return -rte_errno;
> - dev_flow->dv.actions[actions_n].type =
> - MLX5DV_FLOW_ACTION_IBV_FLOW_ACTION;
> - dev_flow->dv.actions[actions_n].action =
> + dev_flow->dv.actions[actions_n++] =
> dev_flow->dv.encap_decap->verbs_action;
> - actions_n++;
> action_flags |= actions->type ==
> RTE_FLOW_ACTION_TYPE_VXLAN_ENCAP ?
> MLX5_FLOW_ACTION_VXLAN_ENCAP :
> @@ -2979,11 +3071,8 @@ struct field_modify_info modify_tcp[] = {
> if (flow_dv_create_action_l2_decap(dev, dev_flow,
> error))
> return -rte_errno;
> - dev_flow->dv.actions[actions_n].type =
> - MLX5DV_FLOW_ACTION_IBV_FLOW_ACTION;
> - dev_flow->dv.actions[actions_n].action =
> + dev_flow->dv.actions[actions_n++] =
> dev_flow->dv.encap_decap->verbs_action;
> - actions_n++;
> action_flags |= actions->type ==
> RTE_FLOW_ACTION_TYPE_VXLAN_DECAP ?
> MLX5_FLOW_ACTION_VXLAN_DECAP :
> @@ -2995,9 +3084,7 @@ struct field_modify_info modify_tcp[] = {
> if (flow_dv_create_action_raw_encap
> (dev, actions, dev_flow, attr, error))
> return -rte_errno;
> - dev_flow->dv.actions[actions_n].type =
> - MLX5DV_FLOW_ACTION_IBV_FLOW_ACTION;
> - dev_flow->dv.actions[actions_n].action =
> + dev_flow->dv.actions[actions_n++] =
> dev_flow->dv.encap_decap->verbs_action;
> } else {
> /* Handle encap without preceding decap. */
> @@ -3005,12 +3092,9 @@ struct field_modify_info modify_tcp[] = {
> dev_flow,
> error))
> return -rte_errno;
> - dev_flow->dv.actions[actions_n].type =
> - MLX5DV_FLOW_ACTION_IBV_FLOW_ACTION;
> - dev_flow->dv.actions[actions_n].action =
> + dev_flow->dv.actions[actions_n++] =
> dev_flow->dv.encap_decap->verbs_action;
> }
> - actions_n++;
> action_flags |= MLX5_FLOW_ACTION_RAW_ENCAP;
> break;
> case RTE_FLOW_ACTION_TYPE_RAW_DECAP:
> @@ -3025,11 +3109,8 @@ struct field_modify_info modify_tcp[] = {
> dev_flow,
> error))
> return -rte_errno;
> - dev_flow->dv.actions[actions_n].type =
> - MLX5DV_FLOW_ACTION_IBV_FLOW_ACTION;
> - dev_flow->dv.actions[actions_n].action =
> + dev_flow->dv.actions[actions_n++] =
> dev_flow->dv.encap_decap->verbs_action;
> - actions_n++;
> }
> /* If decap is followed by encap, handle it at encap. */
> action_flags |= MLX5_FLOW_ACTION_RAW_DECAP;
> @@ -3098,11 +3179,8 @@ struct field_modify_info modify_tcp[] = {
> dev_flow,
> error))
> return -rte_errno;
> - dev_flow->dv.actions[actions_n].type =
> - MLX5DV_FLOW_ACTION_IBV_FLOW_ACTION;
> - dev_flow->dv.actions[actions_n].action =
> + dev_flow->dv.actions[actions_n++] =
> dev_flow->dv.modify_hdr->verbs_action;
> - actions_n++;
> }
> break;
> default:
> @@ -3277,9 +3355,9 @@ struct field_modify_info modify_tcp[] = {
> "cannot get drop hash queue");
> goto error;
> }
> - dv->actions[n].type = MLX5DV_FLOW_ACTION_DEST_IBV_QP;
> - dv->actions[n].qp = dv->hrxq->qp;
> - n++;
> + dv->actions[n++] =
> + mlx5_glue->dv_create_flow_action_dest_ibv_qp
> + (dv->hrxq->qp);
> } else if (flow->actions &
> (MLX5_FLOW_ACTION_QUEUE | MLX5_FLOW_ACTION_RSS)) {
> struct mlx5_hrxq *hrxq;
> @@ -3304,9 +3382,9 @@ struct field_modify_info modify_tcp[] = {
> goto error;
> }
> dv->hrxq = hrxq;
> - dv->actions[n].type = MLX5DV_FLOW_ACTION_DEST_IBV_QP;
> - dv->actions[n].qp = hrxq->qp;
> - n++;
> + dv->actions[n++] =
> + mlx5_glue->dv_create_flow_action_dest_ibv_qp
> + (dv->hrxq->qp);
> }
> dv->flow =
> mlx5_glue->dv_create_flow(dv->matcher->matcher_object,
> @@ -3484,6 +3562,10 @@ struct field_modify_info modify_tcp[] = {
> flow_dv_counter_release(flow->counter);
> flow->counter = NULL;
> }
> + if (flow->tag_resource) {
> + flow_dv_tag_release(dev, flow->tag_resource);
> + flow->tag_resource = NULL;
> + }
> while (!LIST_EMPTY(&flow->dev_flows)) {
> dev_flow = LIST_FIRST(&flow->dev_flows);
> LIST_REMOVE(dev_flow, next);
> diff --git a/drivers/net/mlx5/mlx5_glue.c
> b/drivers/net/mlx5/mlx5_glue.c
> index c817d86..4b5aade 100644
> --- a/drivers/net/mlx5/mlx5_glue.c
> +++ b/drivers/net/mlx5/mlx5_glue.c
> @@ -175,10 +175,20 @@
> }
>
> static int
> -mlx5_glue_destroy_flow_action(struct ibv_flow_action *action)
> +mlx5_glue_destroy_flow_action(void *action)
> {
> #ifdef HAVE_IBV_FLOW_DV_SUPPORT
> - return ibv_destroy_flow_action(action);
> + struct mlx5dv_flow_action_attr *attr = action;
> + int res = 0;
> + switch (attr->type) {
> + case MLX5DV_FLOW_ACTION_TAG:
> + break;
> + default:
> + res = ibv_destroy_flow_action(attr->action);
> + break;
> + }
> + free(action);
> + return res;
> #else
> (void)action;
> return ENOTSUP;
> @@ -430,16 +440,23 @@
> mlx5_glue_dv_create_flow(struct mlx5dv_flow_matcher *matcher,
> struct mlx5dv_flow_match_parameters *match_value,
> size_t num_actions,
> - struct mlx5dv_flow_action_attr *actions_attr)
> + void *actions[])
> {
> #ifdef HAVE_IBV_FLOW_DV_SUPPORT
> + struct mlx5dv_flow_action_attr actions_attr[8];
> +
> + if (num_actions > 8)
> + return NULL;
> + for (size_t i = 0; i < num_actions; i++)
> + actions_attr[i] =
> + *((struct mlx5dv_flow_action_attr *)(actions[i]));
> return mlx5dv_create_flow(matcher, match_value,
> num_actions, actions_attr);
> #else
> (void)matcher;
> (void)match_value;
> (void)num_actions;
> - (void)actions_attr;
> + (void)actions;
> return NULL;
> #endif
> }
> @@ -455,31 +472,45 @@
> #endif
> }
>
> -static struct ibv_flow_action *
> -mlx5_glue_dv_create_flow_action_packet_reformat
> - (struct ibv_context *ctx,
> - size_t data_sz,
> - void *data,
> - enum mlx5dv_flow_action_packet_reformat_type reformat_type,
> - enum mlx5dv_flow_table_type ft_type)
> +static void *
> +mlx5_glue_dv_create_flow_action_counter(void *counter_obj, uint32_t
> offset)
> {
> #ifdef HAVE_IBV_FLOW_DV_SUPPORT
> - return mlx5dv_create_flow_action_packet_reformat(ctx,
> - data_sz,
> - data,
> - reformat_type,
> - ft_type);
> + struct mlx5dv_flow_action_attr *action;
> +
> + (void)offset;
> + action = malloc(sizeof(*action));
> + if (!action)
> + return NULL;
> + action->type = MLX5DV_FLOW_ACTION_COUNTERS_DEVX;
> + action->obj = counter_obj;
> + return action;
> #else
> - (void)ctx;
> - (void)data_sz;
> - (void)data;
> - (void)reformat_type;
> - (void)ft_type;
> + (void)counter_obj;
> + (void)offset;
> return NULL;
> #endif
> }
>
> -static struct ibv_flow_action *
> +static void *
> +mlx5_glue_dv_create_flow_action_dest_ibv_qp(void *qp)
> +{
> +#ifdef HAVE_IBV_FLOW_DV_SUPPORT
> + struct mlx5dv_flow_action_attr *action;
> +
> + action = malloc(sizeof(*action));
> + if (!action)
> + return NULL;
> + action->type = MLX5DV_FLOW_ACTION_DEST_IBV_QP;
> + action->obj = qp;
> + return action;
> +#else
> + (void)qp;
> + return NULL;
> +#endif
> +}
> +
> +static void *
> mlx5_glue_dv_create_flow_action_modify_header
> (struct ibv_context *ctx,
> size_t actions_sz,
> @@ -487,8 +518,15 @@
> enum mlx5dv_flow_table_type ft_type)
> {
> #ifdef HAVE_IBV_FLOW_DV_SUPPORT
> - return mlx5dv_create_flow_action_modify_header(ctx, actions_sz,
> - actions, ft_type);
> + struct mlx5dv_flow_action_attr *action;
> +
> + action = malloc(sizeof(*action));
> + if (!action)
> + return NULL;
> + action->type = MLX5DV_FLOW_ACTION_IBV_FLOW_ACTION;
> + action->action = mlx5dv_create_flow_action_modify_header
> + (ctx, actions_sz, actions, ft_type);
> + return action;
> #else
> (void)ctx;
> (void)actions_sz;
> @@ -498,6 +536,50 @@
> #endif
> }
>
> +static void *
> +mlx5_glue_dv_create_flow_action_packet_reformat
> + (struct ibv_context *ctx,
> + size_t data_sz,
> + void *data,
> + enum mlx5dv_flow_action_packet_reformat_type reformat_type,
> + enum mlx5dv_flow_table_type ft_type)
> +{
> +#ifdef HAVE_IBV_FLOW_DV_SUPPORT
> + struct mlx5dv_flow_action_attr *action;
> +
> + action = malloc(sizeof(*action));
> + if (!action)
> + return NULL;
> + action->type = MLX5DV_FLOW_ACTION_IBV_FLOW_ACTION;
> + action->action = mlx5dv_create_flow_action_packet_reformat
> + (ctx, data_sz, data, reformat_type, ft_type);
> + return action;
> +#else
> + (void)ctx;
> + (void)data_sz;
> + (void)data;
> + (void)reformat_type;
> + (void)ft_type;
> + return NULL;
> +#endif
> +}
> +
> +static void *
> +mlx5_glue_dv_create_flow_action_tag(uint32_t tag)
> +{
> +#ifdef HAVE_IBV_FLOW_DV_SUPPORT
> + struct mlx5dv_flow_action_attr *action;
> + action = malloc(sizeof(*action));
> + if (!action)
> + return NULL;
> + action->type = MLX5DV_FLOW_ACTION_TAG;
> + action->tag_value = tag;
> + return action;
> +#endif
> + (void)tag;
> + return NULL;
> +}
> +
> static struct ibv_context *
> mlx5_glue_dv_open_device(struct ibv_device *device)
> {
> @@ -645,10 +727,15 @@
> .dv_create_flow_matcher = mlx5_glue_dv_create_flow_matcher,
> .dv_destroy_flow_matcher = mlx5_glue_dv_destroy_flow_matcher,
> .dv_create_flow = mlx5_glue_dv_create_flow,
> - .dv_create_flow_action_packet_reformat =
> - mlx5_glue_dv_create_flow_action_packet_reformat,
> + .dv_create_flow_action_counter =
> + mlx5_glue_dv_create_flow_action_counter,
> + .dv_create_flow_action_dest_ibv_qp =
> + mlx5_glue_dv_create_flow_action_dest_ibv_qp,
> .dv_create_flow_action_modify_header =
> - mlx5_glue_dv_create_flow_action_modify_header,
> + mlx5_glue_dv_create_flow_action_modify_header,
> + .dv_create_flow_action_packet_reformat =
> + mlx5_glue_dv_create_flow_action_packet_reformat,
> + .dv_create_flow_action_tag = mlx5_glue_dv_create_flow_action_tag,
> .dv_open_device = mlx5_glue_dv_open_device,
> .devx_obj_create = mlx5_glue_devx_obj_create,
> .devx_obj_destroy = mlx5_glue_devx_obj_destroy,
> diff --git a/drivers/net/mlx5/mlx5_glue.h
> b/drivers/net/mlx5/mlx5_glue.h
> index b118960..32487ea 100644
> --- a/drivers/net/mlx5/mlx5_glue.h
> +++ b/drivers/net/mlx5/mlx5_glue.h
> @@ -55,6 +55,10 @@
> enum mlx5dv_flow_table_type { flow_table_type = 0, };
> #endif
>
> +#ifndef HAVE_IBV_FLOW_DEVX_COUNTERS
> +#define MLX5DV_FLOW_ACTION_COUNTERS_DEVX 0
> +#endif
> +
> #ifndef HAVE_IBV_DEVX_OBJ
> struct mlx5dv_devx_obj;
> #endif
> @@ -98,7 +102,7 @@ struct mlx5_glue {
> struct ibv_flow *(*create_flow)(struct ibv_qp *qp,
> struct ibv_flow_attr *flow);
> int (*destroy_flow)(struct ibv_flow *flow_id);
> - int (*destroy_flow_action)(struct ibv_flow_action *action);
> + int (*destroy_flow_action)(void *action);
> struct ibv_qp *(*create_qp)(struct ibv_pd *pd,
> struct ibv_qp_init_attr *qp_init_attr);
> struct ibv_qp *(*create_qp_ex)
> @@ -160,19 +164,17 @@ struct mlx5_glue {
> int (*dv_destroy_flow_matcher)(struct mlx5dv_flow_matcher *matcher);
> struct ibv_flow *(*dv_create_flow)(struct mlx5dv_flow_matcher
> *matcher,
> struct mlx5dv_flow_match_parameters *match_value,
> - size_t num_actions,
> - struct mlx5dv_flow_action_attr *actions_attr);
> - struct ibv_flow_action *(*dv_create_flow_action_packet_reformat)
> - (struct ibv_context *ctx,
> - size_t data_sz,
> - void *data,
> + size_t num_actions, void *actions[]);
> + void *(*dv_create_flow_action_counter)(void *obj, uint32_t offset);
> + void *(*dv_create_flow_action_dest_ibv_qp)(void *qp);
> + void *(*dv_create_flow_action_modify_header)
> + (struct ibv_context *ctx, size_t actions_sz, uint64_t actions[],
> + enum mlx5dv_flow_table_type ft_type);
> + void *(*dv_create_flow_action_packet_reformat)
> + (struct ibv_context *ctx, size_t data_sz, void *data,
> enum mlx5dv_flow_action_packet_reformat_type reformat_type,
> enum mlx5dv_flow_table_type ft_type);
> - struct ibv_flow_action *(*dv_create_flow_action_modify_header)
> - (struct ibv_context *ctx,
> - size_t actions_sz,
> - uint64_t actions[],
> - enum mlx5dv_flow_table_type ft_type);
> + void *(*dv_create_flow_action_tag)(uint32_t tag);
> struct ibv_context *(*dv_open_device)(struct ibv_device *device);
> struct mlx5dv_devx_obj *(*devx_obj_create)
> (struct ibv_context *ctx,
^ permalink raw reply [flat|nested] 52+ messages in thread
* Re: [dpdk-dev] [PATCH v4 1/3] net/mlx5: prepare Direct Verbs for Direct Rule
2019-04-12 23:51 ` dwilder
2019-04-12 23:51 ` dwilder
@ 2019-04-13 0:16 ` Yongseok Koh
2019-04-13 0:16 ` Yongseok Koh
1 sibling, 1 reply; 52+ messages in thread
From: Yongseok Koh @ 2019-04-13 0:16 UTC (permalink / raw)
To: dwilder
Cc: Ori Kam, Matan Azrad, Shahaf Shuler, dev, Slava Ovsiienko, pradeep
> On Apr 12, 2019, at 4:51 PM, dwilder <dwilder@us.ibm.com> wrote:
>
> I am seeing a build break when building on ubuntu 19.04 beta after this commit (cbb66daa3).
>
> The build produces a number of the following errors.
>
> drivers/net/mlx5/mlx5_glue.c:194:2: error: implicit declaration of function ‘free’
> ....
> drivers/net/mlx5/mlx5_glue.c:552:11: error: incompatible implicit declaration of built-in function ‘malloc’ [-Werror]
> ....
>
> Adding "#include <stdlib.h>" to drivers/net/mlx5/mlx5_glue.c fixed it for me.
>
> I do not see this problem on Ubuntu 18.04, maybe some headers have changed. I have only tested on ppc64le.
Thanks for the notice.
That patch's been merged on to the PMD branch and will be merged on to the master soon
http://git.dpdk.org/next/dpdk-next-net-mlx/commit/?id=4d6b192ccc15a4adfe575900519ba90059e2abd2
Yongseok
> On 2019-04-04 02:54, Ori Kam wrote:
>> This is the first patch of a series that is designed to enable the
>> Direct Rules API.
>> The main difference between Direct Verbs and Direct Rules from API
>> prespective, is that in Direct Rules each action has it's own create
>> function and the object itself is of type void.
>> In this patch I'm adding functions to generate actions that currenlty
>> are done without create action, and I'm changing the action type to be
>> void *, so in next patches only the glue functions will need to change.
>> Signed-off-by: Ori Kam <orika@mellanox.com>
>> Acked-by: Shahaf Shuler <shahafs@mellanox.com>
>> ---
>> drivers/net/mlx5/Makefile | 2 +-
>> drivers/net/mlx5/meson.build | 2 +-
>> drivers/net/mlx5/mlx5.h | 2 +
>> drivers/net/mlx5/mlx5_flow.h | 17 +++-
>> drivers/net/mlx5/mlx5_flow_dv.c | 184 +++++++++++++++++++++++++++++-----------
>> drivers/net/mlx5/mlx5_glue.c | 141 ++++++++++++++++++++++++------
>> drivers/net/mlx5/mlx5_glue.h | 26 +++---
>> 7 files changed, 280 insertions(+), 94 deletions(-)
>> diff --git a/drivers/net/mlx5/Makefile b/drivers/net/mlx5/Makefile
>> index ccd179c..0d20f0f 100644
>> --- a/drivers/net/mlx5/Makefile
>> +++ b/drivers/net/mlx5/Makefile
>> @@ -8,7 +8,7 @@ include $(RTE_SDK)/mk/rte.vars.mk
>> LIB = librte_pmd_mlx5.a
>> LIB_GLUE = $(LIB_GLUE_BASE).$(LIB_GLUE_VERSION)
>> LIB_GLUE_BASE = librte_pmd_mlx5_glue.so
>> -LIB_GLUE_VERSION = 19.02.0
>> +LIB_GLUE_VERSION = 19.05.0
>> # Sources.
>> SRCS-$(CONFIG_RTE_LIBRTE_MLX5_PMD) += mlx5.c
>> diff --git a/drivers/net/mlx5/meson.build b/drivers/net/mlx5/meson.build
>> index a429655..a4c684e 100644
>> --- a/drivers/net/mlx5/meson.build
>> +++ b/drivers/net/mlx5/meson.build
>> @@ -4,7 +4,7 @@
>> pmd_dlopen = (get_option('ibverbs_link') == 'dlopen')
>> LIB_GLUE_BASE = 'librte_pmd_mlx5_glue.so'
>> -LIB_GLUE_VERSION = '19.02.0'
>> +LIB_GLUE_VERSION = '19.05.0'
>> LIB_GLUE = LIB_GLUE_BASE + '.' + LIB_GLUE_VERSION
>> if pmd_dlopen
>> dpdk_conf.set('RTE_IBVERBS_LINK_DLOPEN', 1)
>> diff --git a/drivers/net/mlx5/mlx5.h b/drivers/net/mlx5/mlx5.h
>> index 47a7d75..784bf9b 100644
>> --- a/drivers/net/mlx5/mlx5.h
>> +++ b/drivers/net/mlx5/mlx5.h
>> @@ -309,6 +309,8 @@ struct mlx5_priv {
>> LIST_HEAD(matchers, mlx5_flow_dv_matcher) matchers;
>> LIST_HEAD(encap_decap, mlx5_flow_dv_encap_decap_resource) encaps_decaps;
>> LIST_HEAD(modify_cmd, mlx5_flow_dv_modify_hdr_resource) modify_cmds;
>> + LIST_HEAD(tag, mlx5_flow_dv_tag_resource) tags;
>> + /* Tags resources cache. */
>> uint32_t link_speed_capa; /* Link speed capabilities. */
>> struct mlx5_xstats_ctrl xstats_ctrl; /* Extended stats control. */
>> struct mlx5_stats_ctrl stats_ctrl; /* Stats control. */
>> diff --git a/drivers/net/mlx5/mlx5_flow.h b/drivers/net/mlx5/mlx5_flow.h
>> index e1e798b..4f69ae2 100644
>> --- a/drivers/net/mlx5/mlx5_flow.h
>> +++ b/drivers/net/mlx5/mlx5_flow.h
>> @@ -214,7 +214,7 @@ struct mlx5_flow_dv_encap_decap_resource {
>> LIST_ENTRY(mlx5_flow_dv_encap_decap_resource) next;
>> /* Pointer to next element. */
>> rte_atomic32_t refcnt; /**< Reference counter. */
>> - struct ibv_flow_action *verbs_action;
>> + void *verbs_action;
>> /**< Verbs encap/decap action object. */
>> uint8_t buf[MLX5_ENCAP_MAX_LEN];
>> size_t size;
>> @@ -222,6 +222,16 @@ struct mlx5_flow_dv_encap_decap_resource {
>> uint8_t ft_type;
>> };
>> +/* Tag resource structure. */
>> +struct mlx5_flow_dv_tag_resource {
>> + LIST_ENTRY(mlx5_flow_dv_tag_resource) next;
>> + /* Pointer to next element. */
>> + rte_atomic32_t refcnt; /**< Reference counter. */
>> + void *action;
>> + /**< Verbs tag action object. */
>> + uint32_t tag; /**< the tag value. */
>> +};
>> +
>> /* Number of modification commands. */
>> #define MLX5_MODIFY_NUM 8
>> @@ -259,7 +269,7 @@ struct mlx5_flow_dv {
>> /**< Pointer to modify header resource in cache. */
>> struct ibv_flow *flow; /**< Installed flow. */
>> #ifdef HAVE_IBV_FLOW_DV_SUPPORT
>> - struct mlx5dv_flow_action_attr actions[MLX5_DV_MAX_NUMBER_OF_ACTIONS];
>> + void *actions[MLX5_DV_MAX_NUMBER_OF_ACTIONS];
>> /**< Action list. */
>> #endif
>> int actions_n; /**< number of actions. */
>> @@ -332,6 +342,7 @@ struct mlx5_flow_counter {
>> };
>> uint64_t hits; /**< Number of packets matched by the rule. */
>> uint64_t bytes; /**< Number of bytes matched by the rule. */
>> + void *action; /**< Pointer to the dv action. */
>> };
>> /* Flow structure. */
>> @@ -339,6 +350,8 @@ struct rte_flow {
>> TAILQ_ENTRY(rte_flow) next; /**< Pointer to the next flow structure. */
>> enum mlx5_flow_drv_type drv_type; /**< Drvier type. */
>> struct mlx5_flow_counter *counter; /**< Holds flow counter. */
>> + struct mlx5_flow_dv_tag_resource *tag_resource;
>> + /**< pointer to the tag action. */
>> struct rte_flow_action_rss rss;/**< RSS context. */
>> uint8_t key[MLX5_RSS_HASH_KEY_LEN]; /**< RSS hash key. */
>> uint16_t (*queue)[]; /**< Destination queues to redirect traffic to. */
>> diff --git a/drivers/net/mlx5/mlx5_flow_dv.c b/drivers/net/mlx5/mlx5_flow_dv.c
>> index ad84dea..d9e2ac9 100644
>> --- a/drivers/net/mlx5/mlx5_flow_dv.c
>> +++ b/drivers/net/mlx5/mlx5_flow_dv.c
>> @@ -32,7 +32,6 @@
>> #include "mlx5_prm.h"
>> #include "mlx5_glue.h"
>> #include "mlx5_flow.h"
>> -
>> #ifdef HAVE_IBV_FLOW_DV_SUPPORT
>> #ifndef HAVE_IBV_FLOW_DEVX_COUNTERS
>> @@ -1537,6 +1536,11 @@ struct field_modify_info modify_tcp[] = {
>> .id = id,
>> .dcs = dcs,
>> };
>> + tmpl.action = mlx5_glue->dv_create_flow_action_counter(dcs->obj, 0);
>> + if (!tmpl.action) {
>> + ret = errno;
>> + goto error_exit;
>> + }
>> *cnt = tmpl;
>> LIST_INSERT_HEAD(&priv->flow_counters, cnt, next);
>> return cnt;
>> @@ -2828,6 +2832,97 @@ struct field_modify_info modify_tcp[] = {
>> }
>> /**
>> + * Find existing tag resource or create and register a new one.
>> + *
>> + * @param dev[in, out]
>> + * Pointer to rte_eth_dev structure.
>> + * @param[in, out] resource
>> + * Pointer to tag resource.
>> + * @parm[in, out] dev_flow
>> + * Pointer to the dev_flow.
>> + * @param[out] error
>> + * pointer to error structure.
>> + *
>> + * @return
>> + * 0 on success otherwise -errno and errno is set.
>> + */
>> +static int
>> +flow_dv_tag_resource_register
>> + (struct rte_eth_dev *dev,
>> + struct mlx5_flow_dv_tag_resource *resource,
>> + struct mlx5_flow *dev_flow,
>> + struct rte_flow_error *error)
>> +{
>> + struct mlx5_priv *priv = dev->data->dev_private;
>> + struct mlx5_flow_dv_tag_resource *cache_resource;
>> +
>> + /* Lookup a matching resource from cache. */
>> + LIST_FOREACH(cache_resource, &priv->tags, next) {
>> + if (resource->tag == cache_resource->tag) {
>> + DRV_LOG(DEBUG, "tag resource %p: refcnt %d++",
>> + (void *)cache_resource,
>> + rte_atomic32_read(&cache_resource->refcnt));
>> + rte_atomic32_inc(&cache_resource->refcnt);
>> + dev_flow->flow->tag_resource = cache_resource;
>> + return 0;
>> + }
>> + }
>> + /* Register new resource. */
>> + cache_resource = rte_calloc(__func__, 1, sizeof(*cache_resource), 0);
>> + if (!cache_resource)
>> + return rte_flow_error_set(error, ENOMEM,
>> + RTE_FLOW_ERROR_TYPE_UNSPECIFIED, NULL,
>> + "cannot allocate resource memory");
>> + *cache_resource = *resource;
>> + cache_resource->action = mlx5_glue->dv_create_flow_action_tag
>> + (resource->tag);
>> + if (!cache_resource->action) {
>> + rte_free(cache_resource);
>> + return rte_flow_error_set(error, ENOMEM,
>> + RTE_FLOW_ERROR_TYPE_UNSPECIFIED,
>> + NULL, "cannot create action");
>> + }
>> + rte_atomic32_init(&cache_resource->refcnt);
>> + rte_atomic32_inc(&cache_resource->refcnt);
>> + LIST_INSERT_HEAD(&priv->tags, cache_resource, next);
>> + dev_flow->flow->tag_resource = cache_resource;
>> + DRV_LOG(DEBUG, "new tag resource %p: refcnt %d++",
>> + (void *)cache_resource,
>> + rte_atomic32_read(&cache_resource->refcnt));
>> + return 0;
>> +}
>> +
>> +/**
>> + * Release the tag.
>> + *
>> + * @param dev
>> + * Pointer to Ethernet device.
>> + * @param flow
>> + * Pointer to mlx5_flow.
>> + *
>> + * @return
>> + * 1 while a reference on it exists, 0 when freed.
>> + */
>> +static int
>> +flow_dv_tag_release(struct rte_eth_dev *dev,
>> + struct mlx5_flow_dv_tag_resource *tag)
>> +{
>> + assert(tag);
>> + DRV_LOG(DEBUG, "port %u tag %p: refcnt %d--",
>> + dev->data->port_id, (void *)tag,
>> + rte_atomic32_read(&tag->refcnt));
>> + if (rte_atomic32_dec_and_test(&tag->refcnt)) {
>> + claim_zero(mlx5_glue->destroy_flow_action(tag->action));
>> + LIST_REMOVE(tag, next);
>> + DRV_LOG(DEBUG, "port %u tag %p: removed",
>> + dev->data->port_id, (void *)tag);
>> + rte_free(tag);
>> + return 0;
>> + }
>> + return 1;
>> +}
>> +
>> +/**
>> * Fill the flow with DV spec.
>> *
>> * @param[in] dev
>> @@ -2872,6 +2967,7 @@ struct field_modify_info modify_tcp[] = {
>> MLX5DV_FLOW_TABLE_TYPE_NIC_RX
>> };
>> union flow_dv_attr flow_attr = { .attr = 0 };
>> + struct mlx5_flow_dv_tag_resource tag_resource;
>> if (priority == MLX5_FLOW_PRIO_RSVD)
>> priority = priv->config.flow_prio - 1;
>> @@ -2886,26 +2982,29 @@ struct field_modify_info modify_tcp[] = {
>> case RTE_FLOW_ACTION_TYPE_VOID:
>> break;
>> case RTE_FLOW_ACTION_TYPE_FLAG:
>> - dev_flow->dv.actions[actions_n].type =
>> - MLX5DV_FLOW_ACTION_TAG;
>> - dev_flow->dv.actions[actions_n].tag_value =
>> + tag_resource.tag =
>> mlx5_flow_mark_set(MLX5_FLOW_MARK_DEFAULT);
>> - actions_n++;
>> + if (!flow->tag_resource)
>> + if (flow_dv_tag_resource_register
>> + (dev, &tag_resource, dev_flow, error))
>> + return errno;
>> + dev_flow->dv.actions[actions_n++] =
>> + flow->tag_resource->action;
>> action_flags |= MLX5_FLOW_ACTION_FLAG;
>> break;
>> case RTE_FLOW_ACTION_TYPE_MARK:
>> - dev_flow->dv.actions[actions_n].type =
>> - MLX5DV_FLOW_ACTION_TAG;
>> - dev_flow->dv.actions[actions_n].tag_value =
>> - mlx5_flow_mark_set
>> - (((const struct rte_flow_action_mark *)
>> - (actions->conf))->id);
>> - actions_n++;
>> + tag_resource.tag = mlx5_flow_mark_set
>> + (((const struct rte_flow_action_mark *)
>> + (actions->conf))->id);
>> + if (!flow->tag_resource)
>> + if (flow_dv_tag_resource_register
>> + (dev, &tag_resource, dev_flow, error))
>> + return errno;
>> + dev_flow->dv.actions[actions_n++] =
>> + flow->tag_resource->action;
>> action_flags |= MLX5_FLOW_ACTION_MARK;
>> break;
>> case RTE_FLOW_ACTION_TYPE_DROP:
>> - dev_flow->dv.actions[actions_n].type =
>> - MLX5DV_FLOW_ACTION_DROP;
>> action_flags |= MLX5_FLOW_ACTION_DROP;
>> break;
>> case RTE_FLOW_ACTION_TYPE_QUEUE:
>> @@ -2933,17 +3032,13 @@ struct field_modify_info modify_tcp[] = {
>> rte_errno = ENOTSUP;
>> goto cnt_err;
>> }
>> - flow->counter =
>> - flow_dv_counter_new(dev,
>> - count->shared, count->id);
>> + flow->counter = flow_dv_counter_new(dev, count->shared,
>> + count->id);
>> if (flow->counter == NULL)
>> goto cnt_err;
>> - dev_flow->dv.actions[actions_n].type =
>> - MLX5DV_FLOW_ACTION_COUNTERS_DEVX;
>> - dev_flow->dv.actions[actions_n].obj =
>> - flow->counter->dcs->obj;
>> + dev_flow->dv.actions[actions_n++] =
>> + flow->counter->action;
>> action_flags |= MLX5_FLOW_ACTION_COUNT;
>> - ++actions_n;
>> break;
>> cnt_err:
>> if (rte_errno == ENOTSUP)
>> @@ -2964,11 +3059,8 @@ struct field_modify_info modify_tcp[] = {
>> if (flow_dv_create_action_l2_encap(dev, actions,
>> dev_flow, error))
>> return -rte_errno;
>> - dev_flow->dv.actions[actions_n].type =
>> - MLX5DV_FLOW_ACTION_IBV_FLOW_ACTION;
>> - dev_flow->dv.actions[actions_n].action =
>> + dev_flow->dv.actions[actions_n++] =
>> dev_flow->dv.encap_decap->verbs_action;
>> - actions_n++;
>> action_flags |= actions->type ==
>> RTE_FLOW_ACTION_TYPE_VXLAN_ENCAP ?
>> MLX5_FLOW_ACTION_VXLAN_ENCAP :
>> @@ -2979,11 +3071,8 @@ struct field_modify_info modify_tcp[] = {
>> if (flow_dv_create_action_l2_decap(dev, dev_flow,
>> error))
>> return -rte_errno;
>> - dev_flow->dv.actions[actions_n].type =
>> - MLX5DV_FLOW_ACTION_IBV_FLOW_ACTION;
>> - dev_flow->dv.actions[actions_n].action =
>> + dev_flow->dv.actions[actions_n++] =
>> dev_flow->dv.encap_decap->verbs_action;
>> - actions_n++;
>> action_flags |= actions->type ==
>> RTE_FLOW_ACTION_TYPE_VXLAN_DECAP ?
>> MLX5_FLOW_ACTION_VXLAN_DECAP :
>> @@ -2995,9 +3084,7 @@ struct field_modify_info modify_tcp[] = {
>> if (flow_dv_create_action_raw_encap
>> (dev, actions, dev_flow, attr, error))
>> return -rte_errno;
>> - dev_flow->dv.actions[actions_n].type =
>> - MLX5DV_FLOW_ACTION_IBV_FLOW_ACTION;
>> - dev_flow->dv.actions[actions_n].action =
>> + dev_flow->dv.actions[actions_n++] =
>> dev_flow->dv.encap_decap->verbs_action;
>> } else {
>> /* Handle encap without preceding decap. */
>> @@ -3005,12 +3092,9 @@ struct field_modify_info modify_tcp[] = {
>> dev_flow,
>> error))
>> return -rte_errno;
>> - dev_flow->dv.actions[actions_n].type =
>> - MLX5DV_FLOW_ACTION_IBV_FLOW_ACTION;
>> - dev_flow->dv.actions[actions_n].action =
>> + dev_flow->dv.actions[actions_n++] =
>> dev_flow->dv.encap_decap->verbs_action;
>> }
>> - actions_n++;
>> action_flags |= MLX5_FLOW_ACTION_RAW_ENCAP;
>> break;
>> case RTE_FLOW_ACTION_TYPE_RAW_DECAP:
>> @@ -3025,11 +3109,8 @@ struct field_modify_info modify_tcp[] = {
>> dev_flow,
>> error))
>> return -rte_errno;
>> - dev_flow->dv.actions[actions_n].type =
>> - MLX5DV_FLOW_ACTION_IBV_FLOW_ACTION;
>> - dev_flow->dv.actions[actions_n].action =
>> + dev_flow->dv.actions[actions_n++] =
>> dev_flow->dv.encap_decap->verbs_action;
>> - actions_n++;
>> }
>> /* If decap is followed by encap, handle it at encap. */
>> action_flags |= MLX5_FLOW_ACTION_RAW_DECAP;
>> @@ -3098,11 +3179,8 @@ struct field_modify_info modify_tcp[] = {
>> dev_flow,
>> error))
>> return -rte_errno;
>> - dev_flow->dv.actions[actions_n].type =
>> - MLX5DV_FLOW_ACTION_IBV_FLOW_ACTION;
>> - dev_flow->dv.actions[actions_n].action =
>> + dev_flow->dv.actions[actions_n++] =
>> dev_flow->dv.modify_hdr->verbs_action;
>> - actions_n++;
>> }
>> break;
>> default:
>> @@ -3277,9 +3355,9 @@ struct field_modify_info modify_tcp[] = {
>> "cannot get drop hash queue");
>> goto error;
>> }
>> - dv->actions[n].type = MLX5DV_FLOW_ACTION_DEST_IBV_QP;
>> - dv->actions[n].qp = dv->hrxq->qp;
>> - n++;
>> + dv->actions[n++] =
>> + mlx5_glue->dv_create_flow_action_dest_ibv_qp
>> + (dv->hrxq->qp);
>> } else if (flow->actions &
>> (MLX5_FLOW_ACTION_QUEUE | MLX5_FLOW_ACTION_RSS)) {
>> struct mlx5_hrxq *hrxq;
>> @@ -3304,9 +3382,9 @@ struct field_modify_info modify_tcp[] = {
>> goto error;
>> }
>> dv->hrxq = hrxq;
>> - dv->actions[n].type = MLX5DV_FLOW_ACTION_DEST_IBV_QP;
>> - dv->actions[n].qp = hrxq->qp;
>> - n++;
>> + dv->actions[n++] =
>> + mlx5_glue->dv_create_flow_action_dest_ibv_qp
>> + (dv->hrxq->qp);
>> }
>> dv->flow =
>> mlx5_glue->dv_create_flow(dv->matcher->matcher_object,
>> @@ -3484,6 +3562,10 @@ struct field_modify_info modify_tcp[] = {
>> flow_dv_counter_release(flow->counter);
>> flow->counter = NULL;
>> }
>> + if (flow->tag_resource) {
>> + flow_dv_tag_release(dev, flow->tag_resource);
>> + flow->tag_resource = NULL;
>> + }
>> while (!LIST_EMPTY(&flow->dev_flows)) {
>> dev_flow = LIST_FIRST(&flow->dev_flows);
>> LIST_REMOVE(dev_flow, next);
>> diff --git a/drivers/net/mlx5/mlx5_glue.c b/drivers/net/mlx5/mlx5_glue.c
>> index c817d86..4b5aade 100644
>> --- a/drivers/net/mlx5/mlx5_glue.c
>> +++ b/drivers/net/mlx5/mlx5_glue.c
>> @@ -175,10 +175,20 @@
>> }
>> static int
>> -mlx5_glue_destroy_flow_action(struct ibv_flow_action *action)
>> +mlx5_glue_destroy_flow_action(void *action)
>> {
>> #ifdef HAVE_IBV_FLOW_DV_SUPPORT
>> - return ibv_destroy_flow_action(action);
>> + struct mlx5dv_flow_action_attr *attr = action;
>> + int res = 0;
>> + switch (attr->type) {
>> + case MLX5DV_FLOW_ACTION_TAG:
>> + break;
>> + default:
>> + res = ibv_destroy_flow_action(attr->action);
>> + break;
>> + }
>> + free(action);
>> + return res;
>> #else
>> (void)action;
>> return ENOTSUP;
>> @@ -430,16 +440,23 @@
>> mlx5_glue_dv_create_flow(struct mlx5dv_flow_matcher *matcher,
>> struct mlx5dv_flow_match_parameters *match_value,
>> size_t num_actions,
>> - struct mlx5dv_flow_action_attr *actions_attr)
>> + void *actions[])
>> {
>> #ifdef HAVE_IBV_FLOW_DV_SUPPORT
>> + struct mlx5dv_flow_action_attr actions_attr[8];
>> +
>> + if (num_actions > 8)
>> + return NULL;
>> + for (size_t i = 0; i < num_actions; i++)
>> + actions_attr[i] =
>> + *((struct mlx5dv_flow_action_attr *)(actions[i]));
>> return mlx5dv_create_flow(matcher, match_value,
>> num_actions, actions_attr);
>> #else
>> (void)matcher;
>> (void)match_value;
>> (void)num_actions;
>> - (void)actions_attr;
>> + (void)actions;
>> return NULL;
>> #endif
>> }
>> @@ -455,31 +472,45 @@
>> #endif
>> }
>> -static struct ibv_flow_action *
>> -mlx5_glue_dv_create_flow_action_packet_reformat
>> - (struct ibv_context *ctx,
>> - size_t data_sz,
>> - void *data,
>> - enum mlx5dv_flow_action_packet_reformat_type reformat_type,
>> - enum mlx5dv_flow_table_type ft_type)
>> +static void *
>> +mlx5_glue_dv_create_flow_action_counter(void *counter_obj, uint32_t offset)
>> {
>> #ifdef HAVE_IBV_FLOW_DV_SUPPORT
>> - return mlx5dv_create_flow_action_packet_reformat(ctx,
>> - data_sz,
>> - data,
>> - reformat_type,
>> - ft_type);
>> + struct mlx5dv_flow_action_attr *action;
>> +
>> + (void)offset;
>> + action = malloc(sizeof(*action));
>> + if (!action)
>> + return NULL;
>> + action->type = MLX5DV_FLOW_ACTION_COUNTERS_DEVX;
>> + action->obj = counter_obj;
>> + return action;
>> #else
>> - (void)ctx;
>> - (void)data_sz;
>> - (void)data;
>> - (void)reformat_type;
>> - (void)ft_type;
>> + (void)counter_obj;
>> + (void)offset;
>> return NULL;
>> #endif
>> }
>> -static struct ibv_flow_action *
>> +static void *
>> +mlx5_glue_dv_create_flow_action_dest_ibv_qp(void *qp)
>> +{
>> +#ifdef HAVE_IBV_FLOW_DV_SUPPORT
>> + struct mlx5dv_flow_action_attr *action;
>> +
>> + action = malloc(sizeof(*action));
>> + if (!action)
>> + return NULL;
>> + action->type = MLX5DV_FLOW_ACTION_DEST_IBV_QP;
>> + action->obj = qp;
>> + return action;
>> +#else
>> + (void)qp;
>> + return NULL;
>> +#endif
>> +}
>> +
>> +static void *
>> mlx5_glue_dv_create_flow_action_modify_header
>> (struct ibv_context *ctx,
>> size_t actions_sz,
>> @@ -487,8 +518,15 @@
>> enum mlx5dv_flow_table_type ft_type)
>> {
>> #ifdef HAVE_IBV_FLOW_DV_SUPPORT
>> - return mlx5dv_create_flow_action_modify_header(ctx, actions_sz,
>> - actions, ft_type);
>> + struct mlx5dv_flow_action_attr *action;
>> +
>> + action = malloc(sizeof(*action));
>> + if (!action)
>> + return NULL;
>> + action->type = MLX5DV_FLOW_ACTION_IBV_FLOW_ACTION;
>> + action->action = mlx5dv_create_flow_action_modify_header
>> + (ctx, actions_sz, actions, ft_type);
>> + return action;
>> #else
>> (void)ctx;
>> (void)actions_sz;
>> @@ -498,6 +536,50 @@
>> #endif
>> }
>> +static void *
>> +mlx5_glue_dv_create_flow_action_packet_reformat
>> + (struct ibv_context *ctx,
>> + size_t data_sz,
>> + void *data,
>> + enum mlx5dv_flow_action_packet_reformat_type reformat_type,
>> + enum mlx5dv_flow_table_type ft_type)
>> +{
>> +#ifdef HAVE_IBV_FLOW_DV_SUPPORT
>> + struct mlx5dv_flow_action_attr *action;
>> +
>> + action = malloc(sizeof(*action));
>> + if (!action)
>> + return NULL;
>> + action->type = MLX5DV_FLOW_ACTION_IBV_FLOW_ACTION;
>> + action->action = mlx5dv_create_flow_action_packet_reformat
>> + (ctx, data_sz, data, reformat_type, ft_type);
>> + return action;
>> +#else
>> + (void)ctx;
>> + (void)data_sz;
>> + (void)data;
>> + (void)reformat_type;
>> + (void)ft_type;
>> + return NULL;
>> +#endif
>> +}
>> +
>> +static void *
>> +mlx5_glue_dv_create_flow_action_tag(uint32_t tag)
>> +{
>> +#ifdef HAVE_IBV_FLOW_DV_SUPPORT
>> + struct mlx5dv_flow_action_attr *action;
>> + action = malloc(sizeof(*action));
>> + if (!action)
>> + return NULL;
>> + action->type = MLX5DV_FLOW_ACTION_TAG;
>> + action->tag_value = tag;
>> + return action;
>> +#endif
>> + (void)tag;
>> + return NULL;
>> +}
>> +
>> static struct ibv_context *
>> mlx5_glue_dv_open_device(struct ibv_device *device)
>> {
>> @@ -645,10 +727,15 @@
>> .dv_create_flow_matcher = mlx5_glue_dv_create_flow_matcher,
>> .dv_destroy_flow_matcher = mlx5_glue_dv_destroy_flow_matcher,
>> .dv_create_flow = mlx5_glue_dv_create_flow,
>> - .dv_create_flow_action_packet_reformat =
>> - mlx5_glue_dv_create_flow_action_packet_reformat,
>> + .dv_create_flow_action_counter =
>> + mlx5_glue_dv_create_flow_action_counter,
>> + .dv_create_flow_action_dest_ibv_qp =
>> + mlx5_glue_dv_create_flow_action_dest_ibv_qp,
>> .dv_create_flow_action_modify_header =
>> - mlx5_glue_dv_create_flow_action_modify_header,
>> + mlx5_glue_dv_create_flow_action_modify_header,
>> + .dv_create_flow_action_packet_reformat =
>> + mlx5_glue_dv_create_flow_action_packet_reformat,
>> + .dv_create_flow_action_tag = mlx5_glue_dv_create_flow_action_tag,
>> .dv_open_device = mlx5_glue_dv_open_device,
>> .devx_obj_create = mlx5_glue_devx_obj_create,
>> .devx_obj_destroy = mlx5_glue_devx_obj_destroy,
>> diff --git a/drivers/net/mlx5/mlx5_glue.h b/drivers/net/mlx5/mlx5_glue.h
>> index b118960..32487ea 100644
>> --- a/drivers/net/mlx5/mlx5_glue.h
>> +++ b/drivers/net/mlx5/mlx5_glue.h
>> @@ -55,6 +55,10 @@
>> enum mlx5dv_flow_table_type { flow_table_type = 0, };
>> #endif
>> +#ifndef HAVE_IBV_FLOW_DEVX_COUNTERS
>> +#define MLX5DV_FLOW_ACTION_COUNTERS_DEVX 0
>> +#endif
>> +
>> #ifndef HAVE_IBV_DEVX_OBJ
>> struct mlx5dv_devx_obj;
>> #endif
>> @@ -98,7 +102,7 @@ struct mlx5_glue {
>> struct ibv_flow *(*create_flow)(struct ibv_qp *qp,
>> struct ibv_flow_attr *flow);
>> int (*destroy_flow)(struct ibv_flow *flow_id);
>> - int (*destroy_flow_action)(struct ibv_flow_action *action);
>> + int (*destroy_flow_action)(void *action);
>> struct ibv_qp *(*create_qp)(struct ibv_pd *pd,
>> struct ibv_qp_init_attr *qp_init_attr);
>> struct ibv_qp *(*create_qp_ex)
>> @@ -160,19 +164,17 @@ struct mlx5_glue {
>> int (*dv_destroy_flow_matcher)(struct mlx5dv_flow_matcher *matcher);
>> struct ibv_flow *(*dv_create_flow)(struct mlx5dv_flow_matcher *matcher,
>> struct mlx5dv_flow_match_parameters *match_value,
>> - size_t num_actions,
>> - struct mlx5dv_flow_action_attr *actions_attr);
>> - struct ibv_flow_action *(*dv_create_flow_action_packet_reformat)
>> - (struct ibv_context *ctx,
>> - size_t data_sz,
>> - void *data,
>> + size_t num_actions, void *actions[]);
>> + void *(*dv_create_flow_action_counter)(void *obj, uint32_t offset);
>> + void *(*dv_create_flow_action_dest_ibv_qp)(void *qp);
>> + void *(*dv_create_flow_action_modify_header)
>> + (struct ibv_context *ctx, size_t actions_sz, uint64_t actions[],
>> + enum mlx5dv_flow_table_type ft_type);
>> + void *(*dv_create_flow_action_packet_reformat)
>> + (struct ibv_context *ctx, size_t data_sz, void *data,
>> enum mlx5dv_flow_action_packet_reformat_type reformat_type,
>> enum mlx5dv_flow_table_type ft_type);
>> - struct ibv_flow_action *(*dv_create_flow_action_modify_header)
>> - (struct ibv_context *ctx,
>> - size_t actions_sz,
>> - uint64_t actions[],
>> - enum mlx5dv_flow_table_type ft_type);
>> + void *(*dv_create_flow_action_tag)(uint32_t tag);
>> struct ibv_context *(*dv_open_device)(struct ibv_device *device);
>> struct mlx5dv_devx_obj *(*devx_obj_create)
>> (struct ibv_context *ctx,
>
^ permalink raw reply [flat|nested] 52+ messages in thread
* Re: [dpdk-dev] [PATCH v4 1/3] net/mlx5: prepare Direct Verbs for Direct Rule
2019-04-13 0:16 ` Yongseok Koh
@ 2019-04-13 0:16 ` Yongseok Koh
0 siblings, 0 replies; 52+ messages in thread
From: Yongseok Koh @ 2019-04-13 0:16 UTC (permalink / raw)
To: dwilder
Cc: Ori Kam, Matan Azrad, Shahaf Shuler, dev, Slava Ovsiienko, pradeep
> On Apr 12, 2019, at 4:51 PM, dwilder <dwilder@us.ibm.com> wrote:
>
> I am seeing a build break when building on ubuntu 19.04 beta after this commit (cbb66daa3).
>
> The build produces a number of the following errors.
>
> drivers/net/mlx5/mlx5_glue.c:194:2: error: implicit declaration of function ‘free’
> ....
> drivers/net/mlx5/mlx5_glue.c:552:11: error: incompatible implicit declaration of built-in function ‘malloc’ [-Werror]
> ....
>
> Adding "#include <stdlib.h>" to drivers/net/mlx5/mlx5_glue.c fixed it for me.
>
> I do not see this problem on Ubuntu 18.04, maybe some headers have changed. I have only tested on ppc64le.
Thanks for the notice.
That patch's been merged on to the PMD branch and will be merged on to the master soon
http://git.dpdk.org/next/dpdk-next-net-mlx/commit/?id=4d6b192ccc15a4adfe575900519ba90059e2abd2
Yongseok
> On 2019-04-04 02:54, Ori Kam wrote:
>> This is the first patch of a series that is designed to enable the
>> Direct Rules API.
>> The main difference between Direct Verbs and Direct Rules from API
>> prespective, is that in Direct Rules each action has it's own create
>> function and the object itself is of type void.
>> In this patch I'm adding functions to generate actions that currenlty
>> are done without create action, and I'm changing the action type to be
>> void *, so in next patches only the glue functions will need to change.
>> Signed-off-by: Ori Kam <orika@mellanox.com>
>> Acked-by: Shahaf Shuler <shahafs@mellanox.com>
>> ---
>> drivers/net/mlx5/Makefile | 2 +-
>> drivers/net/mlx5/meson.build | 2 +-
>> drivers/net/mlx5/mlx5.h | 2 +
>> drivers/net/mlx5/mlx5_flow.h | 17 +++-
>> drivers/net/mlx5/mlx5_flow_dv.c | 184 +++++++++++++++++++++++++++++-----------
>> drivers/net/mlx5/mlx5_glue.c | 141 ++++++++++++++++++++++++------
>> drivers/net/mlx5/mlx5_glue.h | 26 +++---
>> 7 files changed, 280 insertions(+), 94 deletions(-)
>> diff --git a/drivers/net/mlx5/Makefile b/drivers/net/mlx5/Makefile
>> index ccd179c..0d20f0f 100644
>> --- a/drivers/net/mlx5/Makefile
>> +++ b/drivers/net/mlx5/Makefile
>> @@ -8,7 +8,7 @@ include $(RTE_SDK)/mk/rte.vars.mk
>> LIB = librte_pmd_mlx5.a
>> LIB_GLUE = $(LIB_GLUE_BASE).$(LIB_GLUE_VERSION)
>> LIB_GLUE_BASE = librte_pmd_mlx5_glue.so
>> -LIB_GLUE_VERSION = 19.02.0
>> +LIB_GLUE_VERSION = 19.05.0
>> # Sources.
>> SRCS-$(CONFIG_RTE_LIBRTE_MLX5_PMD) += mlx5.c
>> diff --git a/drivers/net/mlx5/meson.build b/drivers/net/mlx5/meson.build
>> index a429655..a4c684e 100644
>> --- a/drivers/net/mlx5/meson.build
>> +++ b/drivers/net/mlx5/meson.build
>> @@ -4,7 +4,7 @@
>> pmd_dlopen = (get_option('ibverbs_link') == 'dlopen')
>> LIB_GLUE_BASE = 'librte_pmd_mlx5_glue.so'
>> -LIB_GLUE_VERSION = '19.02.0'
>> +LIB_GLUE_VERSION = '19.05.0'
>> LIB_GLUE = LIB_GLUE_BASE + '.' + LIB_GLUE_VERSION
>> if pmd_dlopen
>> dpdk_conf.set('RTE_IBVERBS_LINK_DLOPEN', 1)
>> diff --git a/drivers/net/mlx5/mlx5.h b/drivers/net/mlx5/mlx5.h
>> index 47a7d75..784bf9b 100644
>> --- a/drivers/net/mlx5/mlx5.h
>> +++ b/drivers/net/mlx5/mlx5.h
>> @@ -309,6 +309,8 @@ struct mlx5_priv {
>> LIST_HEAD(matchers, mlx5_flow_dv_matcher) matchers;
>> LIST_HEAD(encap_decap, mlx5_flow_dv_encap_decap_resource) encaps_decaps;
>> LIST_HEAD(modify_cmd, mlx5_flow_dv_modify_hdr_resource) modify_cmds;
>> + LIST_HEAD(tag, mlx5_flow_dv_tag_resource) tags;
>> + /* Tags resources cache. */
>> uint32_t link_speed_capa; /* Link speed capabilities. */
>> struct mlx5_xstats_ctrl xstats_ctrl; /* Extended stats control. */
>> struct mlx5_stats_ctrl stats_ctrl; /* Stats control. */
>> diff --git a/drivers/net/mlx5/mlx5_flow.h b/drivers/net/mlx5/mlx5_flow.h
>> index e1e798b..4f69ae2 100644
>> --- a/drivers/net/mlx5/mlx5_flow.h
>> +++ b/drivers/net/mlx5/mlx5_flow.h
>> @@ -214,7 +214,7 @@ struct mlx5_flow_dv_encap_decap_resource {
>> LIST_ENTRY(mlx5_flow_dv_encap_decap_resource) next;
>> /* Pointer to next element. */
>> rte_atomic32_t refcnt; /**< Reference counter. */
>> - struct ibv_flow_action *verbs_action;
>> + void *verbs_action;
>> /**< Verbs encap/decap action object. */
>> uint8_t buf[MLX5_ENCAP_MAX_LEN];
>> size_t size;
>> @@ -222,6 +222,16 @@ struct mlx5_flow_dv_encap_decap_resource {
>> uint8_t ft_type;
>> };
>> +/* Tag resource structure. */
>> +struct mlx5_flow_dv_tag_resource {
>> + LIST_ENTRY(mlx5_flow_dv_tag_resource) next;
>> + /* Pointer to next element. */
>> + rte_atomic32_t refcnt; /**< Reference counter. */
>> + void *action;
>> + /**< Verbs tag action object. */
>> + uint32_t tag; /**< the tag value. */
>> +};
>> +
>> /* Number of modification commands. */
>> #define MLX5_MODIFY_NUM 8
>> @@ -259,7 +269,7 @@ struct mlx5_flow_dv {
>> /**< Pointer to modify header resource in cache. */
>> struct ibv_flow *flow; /**< Installed flow. */
>> #ifdef HAVE_IBV_FLOW_DV_SUPPORT
>> - struct mlx5dv_flow_action_attr actions[MLX5_DV_MAX_NUMBER_OF_ACTIONS];
>> + void *actions[MLX5_DV_MAX_NUMBER_OF_ACTIONS];
>> /**< Action list. */
>> #endif
>> int actions_n; /**< number of actions. */
>> @@ -332,6 +342,7 @@ struct mlx5_flow_counter {
>> };
>> uint64_t hits; /**< Number of packets matched by the rule. */
>> uint64_t bytes; /**< Number of bytes matched by the rule. */
>> + void *action; /**< Pointer to the dv action. */
>> };
>> /* Flow structure. */
>> @@ -339,6 +350,8 @@ struct rte_flow {
>> TAILQ_ENTRY(rte_flow) next; /**< Pointer to the next flow structure. */
>> enum mlx5_flow_drv_type drv_type; /**< Drvier type. */
>> struct mlx5_flow_counter *counter; /**< Holds flow counter. */
>> + struct mlx5_flow_dv_tag_resource *tag_resource;
>> + /**< pointer to the tag action. */
>> struct rte_flow_action_rss rss;/**< RSS context. */
>> uint8_t key[MLX5_RSS_HASH_KEY_LEN]; /**< RSS hash key. */
>> uint16_t (*queue)[]; /**< Destination queues to redirect traffic to. */
>> diff --git a/drivers/net/mlx5/mlx5_flow_dv.c b/drivers/net/mlx5/mlx5_flow_dv.c
>> index ad84dea..d9e2ac9 100644
>> --- a/drivers/net/mlx5/mlx5_flow_dv.c
>> +++ b/drivers/net/mlx5/mlx5_flow_dv.c
>> @@ -32,7 +32,6 @@
>> #include "mlx5_prm.h"
>> #include "mlx5_glue.h"
>> #include "mlx5_flow.h"
>> -
>> #ifdef HAVE_IBV_FLOW_DV_SUPPORT
>> #ifndef HAVE_IBV_FLOW_DEVX_COUNTERS
>> @@ -1537,6 +1536,11 @@ struct field_modify_info modify_tcp[] = {
>> .id = id,
>> .dcs = dcs,
>> };
>> + tmpl.action = mlx5_glue->dv_create_flow_action_counter(dcs->obj, 0);
>> + if (!tmpl.action) {
>> + ret = errno;
>> + goto error_exit;
>> + }
>> *cnt = tmpl;
>> LIST_INSERT_HEAD(&priv->flow_counters, cnt, next);
>> return cnt;
>> @@ -2828,6 +2832,97 @@ struct field_modify_info modify_tcp[] = {
>> }
>> /**
>> + * Find existing tag resource or create and register a new one.
>> + *
>> + * @param dev[in, out]
>> + * Pointer to rte_eth_dev structure.
>> + * @param[in, out] resource
>> + * Pointer to tag resource.
>> + * @parm[in, out] dev_flow
>> + * Pointer to the dev_flow.
>> + * @param[out] error
>> + * pointer to error structure.
>> + *
>> + * @return
>> + * 0 on success otherwise -errno and errno is set.
>> + */
>> +static int
>> +flow_dv_tag_resource_register
>> + (struct rte_eth_dev *dev,
>> + struct mlx5_flow_dv_tag_resource *resource,
>> + struct mlx5_flow *dev_flow,
>> + struct rte_flow_error *error)
>> +{
>> + struct mlx5_priv *priv = dev->data->dev_private;
>> + struct mlx5_flow_dv_tag_resource *cache_resource;
>> +
>> + /* Lookup a matching resource from cache. */
>> + LIST_FOREACH(cache_resource, &priv->tags, next) {
>> + if (resource->tag == cache_resource->tag) {
>> + DRV_LOG(DEBUG, "tag resource %p: refcnt %d++",
>> + (void *)cache_resource,
>> + rte_atomic32_read(&cache_resource->refcnt));
>> + rte_atomic32_inc(&cache_resource->refcnt);
>> + dev_flow->flow->tag_resource = cache_resource;
>> + return 0;
>> + }
>> + }
>> + /* Register new resource. */
>> + cache_resource = rte_calloc(__func__, 1, sizeof(*cache_resource), 0);
>> + if (!cache_resource)
>> + return rte_flow_error_set(error, ENOMEM,
>> + RTE_FLOW_ERROR_TYPE_UNSPECIFIED, NULL,
>> + "cannot allocate resource memory");
>> + *cache_resource = *resource;
>> + cache_resource->action = mlx5_glue->dv_create_flow_action_tag
>> + (resource->tag);
>> + if (!cache_resource->action) {
>> + rte_free(cache_resource);
>> + return rte_flow_error_set(error, ENOMEM,
>> + RTE_FLOW_ERROR_TYPE_UNSPECIFIED,
>> + NULL, "cannot create action");
>> + }
>> + rte_atomic32_init(&cache_resource->refcnt);
>> + rte_atomic32_inc(&cache_resource->refcnt);
>> + LIST_INSERT_HEAD(&priv->tags, cache_resource, next);
>> + dev_flow->flow->tag_resource = cache_resource;
>> + DRV_LOG(DEBUG, "new tag resource %p: refcnt %d++",
>> + (void *)cache_resource,
>> + rte_atomic32_read(&cache_resource->refcnt));
>> + return 0;
>> +}
>> +
>> +/**
>> + * Release the tag.
>> + *
>> + * @param dev
>> + * Pointer to Ethernet device.
>> + * @param flow
>> + * Pointer to mlx5_flow.
>> + *
>> + * @return
>> + * 1 while a reference on it exists, 0 when freed.
>> + */
>> +static int
>> +flow_dv_tag_release(struct rte_eth_dev *dev,
>> + struct mlx5_flow_dv_tag_resource *tag)
>> +{
>> + assert(tag);
>> + DRV_LOG(DEBUG, "port %u tag %p: refcnt %d--",
>> + dev->data->port_id, (void *)tag,
>> + rte_atomic32_read(&tag->refcnt));
>> + if (rte_atomic32_dec_and_test(&tag->refcnt)) {
>> + claim_zero(mlx5_glue->destroy_flow_action(tag->action));
>> + LIST_REMOVE(tag, next);
>> + DRV_LOG(DEBUG, "port %u tag %p: removed",
>> + dev->data->port_id, (void *)tag);
>> + rte_free(tag);
>> + return 0;
>> + }
>> + return 1;
>> +}
>> +
>> +/**
>> * Fill the flow with DV spec.
>> *
>> * @param[in] dev
>> @@ -2872,6 +2967,7 @@ struct field_modify_info modify_tcp[] = {
>> MLX5DV_FLOW_TABLE_TYPE_NIC_RX
>> };
>> union flow_dv_attr flow_attr = { .attr = 0 };
>> + struct mlx5_flow_dv_tag_resource tag_resource;
>> if (priority == MLX5_FLOW_PRIO_RSVD)
>> priority = priv->config.flow_prio - 1;
>> @@ -2886,26 +2982,29 @@ struct field_modify_info modify_tcp[] = {
>> case RTE_FLOW_ACTION_TYPE_VOID:
>> break;
>> case RTE_FLOW_ACTION_TYPE_FLAG:
>> - dev_flow->dv.actions[actions_n].type =
>> - MLX5DV_FLOW_ACTION_TAG;
>> - dev_flow->dv.actions[actions_n].tag_value =
>> + tag_resource.tag =
>> mlx5_flow_mark_set(MLX5_FLOW_MARK_DEFAULT);
>> - actions_n++;
>> + if (!flow->tag_resource)
>> + if (flow_dv_tag_resource_register
>> + (dev, &tag_resource, dev_flow, error))
>> + return errno;
>> + dev_flow->dv.actions[actions_n++] =
>> + flow->tag_resource->action;
>> action_flags |= MLX5_FLOW_ACTION_FLAG;
>> break;
>> case RTE_FLOW_ACTION_TYPE_MARK:
>> - dev_flow->dv.actions[actions_n].type =
>> - MLX5DV_FLOW_ACTION_TAG;
>> - dev_flow->dv.actions[actions_n].tag_value =
>> - mlx5_flow_mark_set
>> - (((const struct rte_flow_action_mark *)
>> - (actions->conf))->id);
>> - actions_n++;
>> + tag_resource.tag = mlx5_flow_mark_set
>> + (((const struct rte_flow_action_mark *)
>> + (actions->conf))->id);
>> + if (!flow->tag_resource)
>> + if (flow_dv_tag_resource_register
>> + (dev, &tag_resource, dev_flow, error))
>> + return errno;
>> + dev_flow->dv.actions[actions_n++] =
>> + flow->tag_resource->action;
>> action_flags |= MLX5_FLOW_ACTION_MARK;
>> break;
>> case RTE_FLOW_ACTION_TYPE_DROP:
>> - dev_flow->dv.actions[actions_n].type =
>> - MLX5DV_FLOW_ACTION_DROP;
>> action_flags |= MLX5_FLOW_ACTION_DROP;
>> break;
>> case RTE_FLOW_ACTION_TYPE_QUEUE:
>> @@ -2933,17 +3032,13 @@ struct field_modify_info modify_tcp[] = {
>> rte_errno = ENOTSUP;
>> goto cnt_err;
>> }
>> - flow->counter =
>> - flow_dv_counter_new(dev,
>> - count->shared, count->id);
>> + flow->counter = flow_dv_counter_new(dev, count->shared,
>> + count->id);
>> if (flow->counter == NULL)
>> goto cnt_err;
>> - dev_flow->dv.actions[actions_n].type =
>> - MLX5DV_FLOW_ACTION_COUNTERS_DEVX;
>> - dev_flow->dv.actions[actions_n].obj =
>> - flow->counter->dcs->obj;
>> + dev_flow->dv.actions[actions_n++] =
>> + flow->counter->action;
>> action_flags |= MLX5_FLOW_ACTION_COUNT;
>> - ++actions_n;
>> break;
>> cnt_err:
>> if (rte_errno == ENOTSUP)
>> @@ -2964,11 +3059,8 @@ struct field_modify_info modify_tcp[] = {
>> if (flow_dv_create_action_l2_encap(dev, actions,
>> dev_flow, error))
>> return -rte_errno;
>> - dev_flow->dv.actions[actions_n].type =
>> - MLX5DV_FLOW_ACTION_IBV_FLOW_ACTION;
>> - dev_flow->dv.actions[actions_n].action =
>> + dev_flow->dv.actions[actions_n++] =
>> dev_flow->dv.encap_decap->verbs_action;
>> - actions_n++;
>> action_flags |= actions->type ==
>> RTE_FLOW_ACTION_TYPE_VXLAN_ENCAP ?
>> MLX5_FLOW_ACTION_VXLAN_ENCAP :
>> @@ -2979,11 +3071,8 @@ struct field_modify_info modify_tcp[] = {
>> if (flow_dv_create_action_l2_decap(dev, dev_flow,
>> error))
>> return -rte_errno;
>> - dev_flow->dv.actions[actions_n].type =
>> - MLX5DV_FLOW_ACTION_IBV_FLOW_ACTION;
>> - dev_flow->dv.actions[actions_n].action =
>> + dev_flow->dv.actions[actions_n++] =
>> dev_flow->dv.encap_decap->verbs_action;
>> - actions_n++;
>> action_flags |= actions->type ==
>> RTE_FLOW_ACTION_TYPE_VXLAN_DECAP ?
>> MLX5_FLOW_ACTION_VXLAN_DECAP :
>> @@ -2995,9 +3084,7 @@ struct field_modify_info modify_tcp[] = {
>> if (flow_dv_create_action_raw_encap
>> (dev, actions, dev_flow, attr, error))
>> return -rte_errno;
>> - dev_flow->dv.actions[actions_n].type =
>> - MLX5DV_FLOW_ACTION_IBV_FLOW_ACTION;
>> - dev_flow->dv.actions[actions_n].action =
>> + dev_flow->dv.actions[actions_n++] =
>> dev_flow->dv.encap_decap->verbs_action;
>> } else {
>> /* Handle encap without preceding decap. */
>> @@ -3005,12 +3092,9 @@ struct field_modify_info modify_tcp[] = {
>> dev_flow,
>> error))
>> return -rte_errno;
>> - dev_flow->dv.actions[actions_n].type =
>> - MLX5DV_FLOW_ACTION_IBV_FLOW_ACTION;
>> - dev_flow->dv.actions[actions_n].action =
>> + dev_flow->dv.actions[actions_n++] =
>> dev_flow->dv.encap_decap->verbs_action;
>> }
>> - actions_n++;
>> action_flags |= MLX5_FLOW_ACTION_RAW_ENCAP;
>> break;
>> case RTE_FLOW_ACTION_TYPE_RAW_DECAP:
>> @@ -3025,11 +3109,8 @@ struct field_modify_info modify_tcp[] = {
>> dev_flow,
>> error))
>> return -rte_errno;
>> - dev_flow->dv.actions[actions_n].type =
>> - MLX5DV_FLOW_ACTION_IBV_FLOW_ACTION;
>> - dev_flow->dv.actions[actions_n].action =
>> + dev_flow->dv.actions[actions_n++] =
>> dev_flow->dv.encap_decap->verbs_action;
>> - actions_n++;
>> }
>> /* If decap is followed by encap, handle it at encap. */
>> action_flags |= MLX5_FLOW_ACTION_RAW_DECAP;
>> @@ -3098,11 +3179,8 @@ struct field_modify_info modify_tcp[] = {
>> dev_flow,
>> error))
>> return -rte_errno;
>> - dev_flow->dv.actions[actions_n].type =
>> - MLX5DV_FLOW_ACTION_IBV_FLOW_ACTION;
>> - dev_flow->dv.actions[actions_n].action =
>> + dev_flow->dv.actions[actions_n++] =
>> dev_flow->dv.modify_hdr->verbs_action;
>> - actions_n++;
>> }
>> break;
>> default:
>> @@ -3277,9 +3355,9 @@ struct field_modify_info modify_tcp[] = {
>> "cannot get drop hash queue");
>> goto error;
>> }
>> - dv->actions[n].type = MLX5DV_FLOW_ACTION_DEST_IBV_QP;
>> - dv->actions[n].qp = dv->hrxq->qp;
>> - n++;
>> + dv->actions[n++] =
>> + mlx5_glue->dv_create_flow_action_dest_ibv_qp
>> + (dv->hrxq->qp);
>> } else if (flow->actions &
>> (MLX5_FLOW_ACTION_QUEUE | MLX5_FLOW_ACTION_RSS)) {
>> struct mlx5_hrxq *hrxq;
>> @@ -3304,9 +3382,9 @@ struct field_modify_info modify_tcp[] = {
>> goto error;
>> }
>> dv->hrxq = hrxq;
>> - dv->actions[n].type = MLX5DV_FLOW_ACTION_DEST_IBV_QP;
>> - dv->actions[n].qp = hrxq->qp;
>> - n++;
>> + dv->actions[n++] =
>> + mlx5_glue->dv_create_flow_action_dest_ibv_qp
>> + (dv->hrxq->qp);
>> }
>> dv->flow =
>> mlx5_glue->dv_create_flow(dv->matcher->matcher_object,
>> @@ -3484,6 +3562,10 @@ struct field_modify_info modify_tcp[] = {
>> flow_dv_counter_release(flow->counter);
>> flow->counter = NULL;
>> }
>> + if (flow->tag_resource) {
>> + flow_dv_tag_release(dev, flow->tag_resource);
>> + flow->tag_resource = NULL;
>> + }
>> while (!LIST_EMPTY(&flow->dev_flows)) {
>> dev_flow = LIST_FIRST(&flow->dev_flows);
>> LIST_REMOVE(dev_flow, next);
>> diff --git a/drivers/net/mlx5/mlx5_glue.c b/drivers/net/mlx5/mlx5_glue.c
>> index c817d86..4b5aade 100644
>> --- a/drivers/net/mlx5/mlx5_glue.c
>> +++ b/drivers/net/mlx5/mlx5_glue.c
>> @@ -175,10 +175,20 @@
>> }
>> static int
>> -mlx5_glue_destroy_flow_action(struct ibv_flow_action *action)
>> +mlx5_glue_destroy_flow_action(void *action)
>> {
>> #ifdef HAVE_IBV_FLOW_DV_SUPPORT
>> - return ibv_destroy_flow_action(action);
>> + struct mlx5dv_flow_action_attr *attr = action;
>> + int res = 0;
>> + switch (attr->type) {
>> + case MLX5DV_FLOW_ACTION_TAG:
>> + break;
>> + default:
>> + res = ibv_destroy_flow_action(attr->action);
>> + break;
>> + }
>> + free(action);
>> + return res;
>> #else
>> (void)action;
>> return ENOTSUP;
>> @@ -430,16 +440,23 @@
>> mlx5_glue_dv_create_flow(struct mlx5dv_flow_matcher *matcher,
>> struct mlx5dv_flow_match_parameters *match_value,
>> size_t num_actions,
>> - struct mlx5dv_flow_action_attr *actions_attr)
>> + void *actions[])
>> {
>> #ifdef HAVE_IBV_FLOW_DV_SUPPORT
>> + struct mlx5dv_flow_action_attr actions_attr[8];
>> +
>> + if (num_actions > 8)
>> + return NULL;
>> + for (size_t i = 0; i < num_actions; i++)
>> + actions_attr[i] =
>> + *((struct mlx5dv_flow_action_attr *)(actions[i]));
>> return mlx5dv_create_flow(matcher, match_value,
>> num_actions, actions_attr);
>> #else
>> (void)matcher;
>> (void)match_value;
>> (void)num_actions;
>> - (void)actions_attr;
>> + (void)actions;
>> return NULL;
>> #endif
>> }
>> @@ -455,31 +472,45 @@
>> #endif
>> }
>> -static struct ibv_flow_action *
>> -mlx5_glue_dv_create_flow_action_packet_reformat
>> - (struct ibv_context *ctx,
>> - size_t data_sz,
>> - void *data,
>> - enum mlx5dv_flow_action_packet_reformat_type reformat_type,
>> - enum mlx5dv_flow_table_type ft_type)
>> +static void *
>> +mlx5_glue_dv_create_flow_action_counter(void *counter_obj, uint32_t offset)
>> {
>> #ifdef HAVE_IBV_FLOW_DV_SUPPORT
>> - return mlx5dv_create_flow_action_packet_reformat(ctx,
>> - data_sz,
>> - data,
>> - reformat_type,
>> - ft_type);
>> + struct mlx5dv_flow_action_attr *action;
>> +
>> + (void)offset;
>> + action = malloc(sizeof(*action));
>> + if (!action)
>> + return NULL;
>> + action->type = MLX5DV_FLOW_ACTION_COUNTERS_DEVX;
>> + action->obj = counter_obj;
>> + return action;
>> #else
>> - (void)ctx;
>> - (void)data_sz;
>> - (void)data;
>> - (void)reformat_type;
>> - (void)ft_type;
>> + (void)counter_obj;
>> + (void)offset;
>> return NULL;
>> #endif
>> }
>> -static struct ibv_flow_action *
>> +static void *
>> +mlx5_glue_dv_create_flow_action_dest_ibv_qp(void *qp)
>> +{
>> +#ifdef HAVE_IBV_FLOW_DV_SUPPORT
>> + struct mlx5dv_flow_action_attr *action;
>> +
>> + action = malloc(sizeof(*action));
>> + if (!action)
>> + return NULL;
>> + action->type = MLX5DV_FLOW_ACTION_DEST_IBV_QP;
>> + action->obj = qp;
>> + return action;
>> +#else
>> + (void)qp;
>> + return NULL;
>> +#endif
>> +}
>> +
>> +static void *
>> mlx5_glue_dv_create_flow_action_modify_header
>> (struct ibv_context *ctx,
>> size_t actions_sz,
>> @@ -487,8 +518,15 @@
>> enum mlx5dv_flow_table_type ft_type)
>> {
>> #ifdef HAVE_IBV_FLOW_DV_SUPPORT
>> - return mlx5dv_create_flow_action_modify_header(ctx, actions_sz,
>> - actions, ft_type);
>> + struct mlx5dv_flow_action_attr *action;
>> +
>> + action = malloc(sizeof(*action));
>> + if (!action)
>> + return NULL;
>> + action->type = MLX5DV_FLOW_ACTION_IBV_FLOW_ACTION;
>> + action->action = mlx5dv_create_flow_action_modify_header
>> + (ctx, actions_sz, actions, ft_type);
>> + return action;
>> #else
>> (void)ctx;
>> (void)actions_sz;
>> @@ -498,6 +536,50 @@
>> #endif
>> }
>> +static void *
>> +mlx5_glue_dv_create_flow_action_packet_reformat
>> + (struct ibv_context *ctx,
>> + size_t data_sz,
>> + void *data,
>> + enum mlx5dv_flow_action_packet_reformat_type reformat_type,
>> + enum mlx5dv_flow_table_type ft_type)
>> +{
>> +#ifdef HAVE_IBV_FLOW_DV_SUPPORT
>> + struct mlx5dv_flow_action_attr *action;
>> +
>> + action = malloc(sizeof(*action));
>> + if (!action)
>> + return NULL;
>> + action->type = MLX5DV_FLOW_ACTION_IBV_FLOW_ACTION;
>> + action->action = mlx5dv_create_flow_action_packet_reformat
>> + (ctx, data_sz, data, reformat_type, ft_type);
>> + return action;
>> +#else
>> + (void)ctx;
>> + (void)data_sz;
>> + (void)data;
>> + (void)reformat_type;
>> + (void)ft_type;
>> + return NULL;
>> +#endif
>> +}
>> +
>> +static void *
>> +mlx5_glue_dv_create_flow_action_tag(uint32_t tag)
>> +{
>> +#ifdef HAVE_IBV_FLOW_DV_SUPPORT
>> + struct mlx5dv_flow_action_attr *action;
>> + action = malloc(sizeof(*action));
>> + if (!action)
>> + return NULL;
>> + action->type = MLX5DV_FLOW_ACTION_TAG;
>> + action->tag_value = tag;
>> + return action;
>> +#endif
>> + (void)tag;
>> + return NULL;
>> +}
>> +
>> static struct ibv_context *
>> mlx5_glue_dv_open_device(struct ibv_device *device)
>> {
>> @@ -645,10 +727,15 @@
>> .dv_create_flow_matcher = mlx5_glue_dv_create_flow_matcher,
>> .dv_destroy_flow_matcher = mlx5_glue_dv_destroy_flow_matcher,
>> .dv_create_flow = mlx5_glue_dv_create_flow,
>> - .dv_create_flow_action_packet_reformat =
>> - mlx5_glue_dv_create_flow_action_packet_reformat,
>> + .dv_create_flow_action_counter =
>> + mlx5_glue_dv_create_flow_action_counter,
>> + .dv_create_flow_action_dest_ibv_qp =
>> + mlx5_glue_dv_create_flow_action_dest_ibv_qp,
>> .dv_create_flow_action_modify_header =
>> - mlx5_glue_dv_create_flow_action_modify_header,
>> + mlx5_glue_dv_create_flow_action_modify_header,
>> + .dv_create_flow_action_packet_reformat =
>> + mlx5_glue_dv_create_flow_action_packet_reformat,
>> + .dv_create_flow_action_tag = mlx5_glue_dv_create_flow_action_tag,
>> .dv_open_device = mlx5_glue_dv_open_device,
>> .devx_obj_create = mlx5_glue_devx_obj_create,
>> .devx_obj_destroy = mlx5_glue_devx_obj_destroy,
>> diff --git a/drivers/net/mlx5/mlx5_glue.h b/drivers/net/mlx5/mlx5_glue.h
>> index b118960..32487ea 100644
>> --- a/drivers/net/mlx5/mlx5_glue.h
>> +++ b/drivers/net/mlx5/mlx5_glue.h
>> @@ -55,6 +55,10 @@
>> enum mlx5dv_flow_table_type { flow_table_type = 0, };
>> #endif
>> +#ifndef HAVE_IBV_FLOW_DEVX_COUNTERS
>> +#define MLX5DV_FLOW_ACTION_COUNTERS_DEVX 0
>> +#endif
>> +
>> #ifndef HAVE_IBV_DEVX_OBJ
>> struct mlx5dv_devx_obj;
>> #endif
>> @@ -98,7 +102,7 @@ struct mlx5_glue {
>> struct ibv_flow *(*create_flow)(struct ibv_qp *qp,
>> struct ibv_flow_attr *flow);
>> int (*destroy_flow)(struct ibv_flow *flow_id);
>> - int (*destroy_flow_action)(struct ibv_flow_action *action);
>> + int (*destroy_flow_action)(void *action);
>> struct ibv_qp *(*create_qp)(struct ibv_pd *pd,
>> struct ibv_qp_init_attr *qp_init_attr);
>> struct ibv_qp *(*create_qp_ex)
>> @@ -160,19 +164,17 @@ struct mlx5_glue {
>> int (*dv_destroy_flow_matcher)(struct mlx5dv_flow_matcher *matcher);
>> struct ibv_flow *(*dv_create_flow)(struct mlx5dv_flow_matcher *matcher,
>> struct mlx5dv_flow_match_parameters *match_value,
>> - size_t num_actions,
>> - struct mlx5dv_flow_action_attr *actions_attr);
>> - struct ibv_flow_action *(*dv_create_flow_action_packet_reformat)
>> - (struct ibv_context *ctx,
>> - size_t data_sz,
>> - void *data,
>> + size_t num_actions, void *actions[]);
>> + void *(*dv_create_flow_action_counter)(void *obj, uint32_t offset);
>> + void *(*dv_create_flow_action_dest_ibv_qp)(void *qp);
>> + void *(*dv_create_flow_action_modify_header)
>> + (struct ibv_context *ctx, size_t actions_sz, uint64_t actions[],
>> + enum mlx5dv_flow_table_type ft_type);
>> + void *(*dv_create_flow_action_packet_reformat)
>> + (struct ibv_context *ctx, size_t data_sz, void *data,
>> enum mlx5dv_flow_action_packet_reformat_type reformat_type,
>> enum mlx5dv_flow_table_type ft_type);
>> - struct ibv_flow_action *(*dv_create_flow_action_modify_header)
>> - (struct ibv_context *ctx,
>> - size_t actions_sz,
>> - uint64_t actions[],
>> - enum mlx5dv_flow_table_type ft_type);
>> + void *(*dv_create_flow_action_tag)(uint32_t tag);
>> struct ibv_context *(*dv_open_device)(struct ibv_device *device);
>> struct mlx5dv_devx_obj *(*devx_obj_create)
>> (struct ibv_context *ctx,
>
^ permalink raw reply [flat|nested] 52+ messages in thread
end of thread, other threads:[~2019-04-13 0:16 UTC | newest]
Thread overview: 52+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2019-03-20 15:38 [dpdk-dev] [PATCH 0/3]net/mlx5: Add Direct Rule support Ori Kam
2019-03-20 15:38 ` Ori Kam
2019-03-20 15:39 ` [dpdk-dev] [PATCH 1/3] net/mlx5: prepare Direct Verbs for Direct Rule Ori Kam
2019-03-20 15:39 ` Ori Kam
2019-03-20 15:39 ` [dpdk-dev] [PATCH 2/3] net/mlx5: add Direct Rules API Ori Kam
2019-03-20 15:39 ` Ori Kam
2019-03-20 15:39 ` [dpdk-dev] [PATCH 3/3] net/mlx5: add jump action support for NIC Ori Kam
2019-03-20 15:39 ` Ori Kam
2019-03-28 16:32 ` [dpdk-dev] [PATCH v2 0/3] net/mlx5: Add Direct Rule support Ori Kam
2019-03-28 16:32 ` Ori Kam
2019-03-28 16:32 ` [dpdk-dev] [PATCH v2 1/3] net/mlx5: prepare Direct Verbs for Direct Rule Ori Kam
2019-03-28 16:32 ` Ori Kam
2019-04-01 14:38 ` Slava Ovsiienko
2019-04-01 14:38 ` Slava Ovsiienko
2019-04-03 10:15 ` Shahaf Shuler
2019-04-03 10:15 ` Shahaf Shuler
2019-03-28 16:32 ` [dpdk-dev] [PATCH v2 2/3] net/mlx5: add Direct Rules API Ori Kam
2019-03-28 16:32 ` Ori Kam
2019-04-01 14:38 ` Slava Ovsiienko
2019-04-01 14:38 ` Slava Ovsiienko
2019-03-28 16:32 ` [dpdk-dev] [PATCH v2 3/3] net/mlx5: add jump action support for NIC Ori Kam
2019-03-28 16:32 ` Ori Kam
2019-04-01 14:38 ` Slava Ovsiienko
2019-04-01 14:38 ` Slava Ovsiienko
2019-04-03 10:16 ` Shahaf Shuler
2019-04-03 10:16 ` Shahaf Shuler
2019-04-03 10:17 ` [dpdk-dev] [PATCH v2 0/3] net/mlx5: Add Direct Rule support Shahaf Shuler
2019-04-03 10:17 ` Shahaf Shuler
2019-04-03 13:21 ` [dpdk-dev] [PATCH v3 " Ori Kam
2019-04-03 13:21 ` Ori Kam
2019-04-03 13:21 ` [dpdk-dev] [PATCH v3 1/3] net/mlx5: prepare Direct Verbs for Direct Rule Ori Kam
2019-04-03 13:21 ` Ori Kam
2019-04-03 13:21 ` [dpdk-dev] [PATCH v3 2/3] net/mlx5: add Direct Rules API Ori Kam
2019-04-03 13:21 ` Ori Kam
2019-04-03 13:21 ` [dpdk-dev] [PATCH v3 3/3] net/mlx5: add jump action support for NIC Ori Kam
2019-04-03 13:21 ` Ori Kam
2019-04-04 5:26 ` Shahaf Shuler
2019-04-04 5:26 ` Shahaf Shuler
2019-04-04 9:54 ` [dpdk-dev] [PATCH v4 0/3] net/mlx5: Add Direct Rule support Ori Kam
2019-04-04 9:54 ` Ori Kam
2019-04-04 9:54 ` [dpdk-dev] [PATCH v4 1/3] net/mlx5: prepare Direct Verbs for Direct Rule Ori Kam
2019-04-04 9:54 ` Ori Kam
2019-04-12 23:51 ` dwilder
2019-04-12 23:51 ` dwilder
2019-04-13 0:16 ` Yongseok Koh
2019-04-13 0:16 ` Yongseok Koh
2019-04-04 9:54 ` [dpdk-dev] [PATCH v4 2/3] net/mlx5: add Direct Rules API Ori Kam
2019-04-04 9:54 ` Ori Kam
2019-04-04 9:54 ` [dpdk-dev] [PATCH v4 3/3] net/mlx5: add jump action support for NIC Ori Kam
2019-04-04 9:54 ` Ori Kam
2019-04-04 11:01 ` [dpdk-dev] [PATCH v4 0/3] net/mlx5: Add Direct Rule support Shahaf Shuler
2019-04-04 11:01 ` Shahaf Shuler
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).