From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from dpdk.org (dpdk.org [92.243.14.124]) by inbox.dpdk.org (Postfix) with ESMTP id 4101BA04B1; Wed, 30 Sep 2020 08:37:01 +0200 (CEST) Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id 6F9C81DAF0; Wed, 30 Sep 2020 08:34:41 +0200 (CEST) Received: from mga09.intel.com (mga09.intel.com [134.134.136.24]) by dpdk.org (Postfix) with ESMTP id E5A231DA70 for ; Wed, 30 Sep 2020 08:34:28 +0200 (CEST) IronPort-SDR: 67lLcjb1gDyLF3ws20I2wBByeCrTDUcLQKzuqxzXoB2/NlJ60QH9igbFkLdhKgToCgCyY0xpeS hbExbTPhRRwg== X-IronPort-AV: E=McAfee;i="6000,8403,9759"; a="163238676" X-IronPort-AV: E=Sophos;i="5.77,321,1596524400"; d="scan'208";a="163238676" X-Amp-Result: SKIPPED(no attachment in message) X-Amp-File-Uploaded: False Received: from orsmga005.jf.intel.com ([10.7.209.41]) by orsmga102.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 29 Sep 2020 23:34:27 -0700 IronPort-SDR: axKjzlPOnG3mdr3lXgqLR5KQujW8XuF47BDIuEqaM1rUz13g3L1JG+TDYKozSPEdH5LaQ6A3uU c2GEFIIZbpKg== X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.77,321,1596524400"; d="scan'208";a="495881516" Received: from silpixa00400573.ir.intel.com (HELO silpixa00400573.ger.corp.intel.com) ([10.237.223.107]) by orsmga005.jf.intel.com with ESMTP; 29 Sep 2020 23:34:27 -0700 From: Cristian Dumitrescu To: dev@dpdk.org Cc: thomas@monjalon.net, david.marchand@redhat.com Date: Wed, 30 Sep 2020 07:33:41 +0100 Message-Id: <20200930063416.68428-8-cristian.dumitrescu@intel.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20200930063416.68428-1-cristian.dumitrescu@intel.com> References: <20200923180645.55852-2-cristian.dumitrescu@intel.com> <20200930063416.68428-1-cristian.dumitrescu@intel.com> Subject: [dpdk-dev] [PATCH v6 07/42] pipeline: add SWX pipeline tables X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" Add tables to the SWX pipeline. The match fields are flexibly selected from the headers and meta-data. The set of table actions is flexibly selected for each table from the set of pipeline actions. Signed-off-by: Cristian Dumitrescu --- lib/librte_pipeline/meson.build | 3 +- lib/librte_pipeline/rte_pipeline_version.map | 4 + lib/librte_pipeline/rte_swx_ctl.h | 85 +++ lib/librte_pipeline/rte_swx_pipeline.c | 700 +++++++++++++++++++ lib/librte_pipeline/rte_swx_pipeline.h | 118 ++++ lib/librte_table/meson.build | 3 +- lib/librte_table/rte_swx_table.h | 295 ++++++++ 7 files changed, 1206 insertions(+), 2 deletions(-) create mode 100644 lib/librte_pipeline/rte_swx_ctl.h create mode 100644 lib/librte_table/rte_swx_table.h diff --git a/lib/librte_pipeline/meson.build b/lib/librte_pipeline/meson.build index bea406848..d5f4d16e5 100644 --- a/lib/librte_pipeline/meson.build +++ b/lib/librte_pipeline/meson.build @@ -9,5 +9,6 @@ headers = files('rte_pipeline.h', 'rte_port_in_action.h', 'rte_table_action.h', 'rte_swx_pipeline.h', - 'rte_swx_extern.h',) + 'rte_swx_extern.h', + 'rte_swx_ctl.h',) deps += ['port', 'table', 'meter', 'sched', 'cryptodev'] diff --git a/lib/librte_pipeline/rte_pipeline_version.map b/lib/librte_pipeline/rte_pipeline_version.map index c701f158d..b9e59bce2 100644 --- a/lib/librte_pipeline/rte_pipeline_version.map +++ b/lib/librte_pipeline/rte_pipeline_version.map @@ -68,6 +68,10 @@ EXPERIMENTAL { rte_swx_pipeline_packet_header_register; rte_swx_pipeline_packet_metadata_register; rte_swx_pipeline_action_config; + rte_swx_pipeline_table_type_register; + rte_swx_pipeline_table_config; rte_swx_pipeline_build; rte_swx_pipeline_free; + rte_swx_pipeline_table_state_get; + rte_swx_pipeline_table_state_set; }; diff --git a/lib/librte_pipeline/rte_swx_ctl.h b/lib/librte_pipeline/rte_swx_ctl.h new file mode 100644 index 000000000..c824ab56f --- /dev/null +++ b/lib/librte_pipeline/rte_swx_ctl.h @@ -0,0 +1,85 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright(c) 2020 Intel Corporation + */ +#ifndef __INCLUDE_RTE_SWX_CTL_H__ +#define __INCLUDE_RTE_SWX_CTL_H__ + +#ifdef __cplusplus +extern "C" { +#endif + +/** + * @file + * RTE SWX Pipeline Control + */ + +#include +#include + +#include + +#include "rte_swx_table.h" + +/* + * Table Update API. + */ + +/** Table state. */ +struct rte_swx_table_state { + /** Table object. */ + void *obj; + + /** Action ID of the table default action. */ + uint64_t default_action_id; + + /** Action data of the table default action. Ignored when the action + * data size is zero; otherwise, action data size bytes are meaningful. + */ + uint8_t *default_action_data; +}; + +/** + * Pipeline table state get + * + * @param[in] p + * Pipeline handle. + * @param[out] table_state + * After successful execution, the *table_state* contains the pointer to the + * current pipeline table state, which is an array of *n_tables* elements, + * with array element i containing the state of the i-th pipeline table. The + * pipeline continues to own all the data structures directly or indirectly + * referenced by the *table_state* until the subsequent successful invocation + * of function *rte_swx_pipeline_table_state_set*. + * @return + * 0 on success or the following error codes otherwise: + * -EINVAL: Invalid argument. + */ +__rte_experimental +int +rte_swx_pipeline_table_state_get(struct rte_swx_pipeline *p, + struct rte_swx_table_state **table_state); + +/** + * Pipeline table state set + * + * @param[in] p + * Pipeline handle. + * @param[out] table_state + * After successful execution, the pipeline table state is updated to this + * *table_state*. The ownership of all the data structures directly or + * indirectly referenced by this *table_state* is passed from the caller to + * the pipeline. + * @return + * 0 on success or the following error codes otherwise: + * -EINVAL: Invalid argument. + */ +__rte_experimental +int +rte_swx_pipeline_table_state_set(struct rte_swx_pipeline *p, + struct rte_swx_table_state *table_state); + +#ifdef __cplusplus +} +#endif + +#endif diff --git a/lib/librte_pipeline/rte_swx_pipeline.c b/lib/librte_pipeline/rte_swx_pipeline.c index 678700050..eb5b327e8 100644 --- a/lib/librte_pipeline/rte_swx_pipeline.c +++ b/lib/librte_pipeline/rte_swx_pipeline.c @@ -10,6 +10,7 @@ #include #include "rte_swx_pipeline.h" +#include "rte_swx_ctl.h" #define CHECK(condition, err_code) \ do { \ @@ -197,6 +198,55 @@ struct action { TAILQ_HEAD(action_tailq, action); +/* + * Table. + */ +struct table_type { + TAILQ_ENTRY(table_type) node; + char name[RTE_SWX_NAME_SIZE]; + enum rte_swx_table_match_type match_type; + struct rte_swx_table_ops ops; +}; + +TAILQ_HEAD(table_type_tailq, table_type); + +struct match_field { + enum rte_swx_table_match_type match_type; + struct field *field; +}; + +struct table { + TAILQ_ENTRY(table) node; + char name[RTE_SWX_NAME_SIZE]; + char args[RTE_SWX_NAME_SIZE]; + struct table_type *type; /* NULL when n_fields == 0. */ + + /* Match. */ + struct match_field *fields; + uint32_t n_fields; + int is_header; /* Only valid when n_fields > 0. */ + struct header *header; /* Only valid when n_fields > 0. */ + + /* Action. */ + struct action **actions; + struct action *default_action; + uint8_t *default_action_data; + uint32_t n_actions; + int default_action_is_const; + uint32_t action_data_size_max; + + uint32_t size; + uint32_t id; +}; + +TAILQ_HEAD(table_tailq, table); + +struct table_runtime { + rte_swx_table_lookup_t func; + void *mailbox; + uint8_t **key; +}; + /* * Pipeline. */ @@ -215,6 +265,12 @@ struct thread { /* Packet meta-data. */ uint8_t *metadata; + /* Tables. */ + struct table_runtime *tables; + struct rte_swx_table_state *table_state; + uint64_t action_id; + int hit; /* 0 = Miss, 1 = Hit. */ + /* Extern objects and functions. */ struct extern_obj_runtime *extern_objs; struct extern_func_runtime *extern_funcs; @@ -237,10 +293,13 @@ struct rte_swx_pipeline { struct struct_type *metadata_st; uint32_t metadata_struct_id; struct action_tailq actions; + struct table_type_tailq table_types; + struct table_tailq tables; struct port_in_runtime *in; struct port_out_runtime *out; struct instruction **action_instructions; + struct rte_swx_table_state *table_state; struct thread threads[RTE_SWX_PIPELINE_THREADS_MAX]; uint32_t n_structs; @@ -249,6 +308,7 @@ struct rte_swx_pipeline { uint32_t n_extern_objs; uint32_t n_extern_funcs; uint32_t n_actions; + uint32_t n_tables; uint32_t n_headers; int build_done; int numa_node; @@ -269,6 +329,21 @@ struct_type_find(struct rte_swx_pipeline *p, const char *name) return NULL; } +static struct field * +struct_type_field_find(struct struct_type *st, const char *name) +{ + uint32_t i; + + for (i = 0; i < st->n_fields; i++) { + struct field *f = &st->fields[i]; + + if (strcmp(f->name, name) == 0) + return f; + } + + return NULL; +} + int rte_swx_pipeline_struct_type_register(struct rte_swx_pipeline *p, const char *name, @@ -1106,6 +1181,50 @@ header_find(struct rte_swx_pipeline *p, const char *name) return NULL; } +static struct field * +header_field_parse(struct rte_swx_pipeline *p, + const char *name, + struct header **header) +{ + struct header *h; + struct field *f; + char *header_name, *field_name; + + if ((name[0] != 'h') || (name[1] != '.')) + return NULL; + + header_name = strdup(&name[2]); + if (!header_name) + return NULL; + + field_name = strchr(header_name, '.'); + if (!field_name) { + free(header_name); + return NULL; + } + + *field_name = 0; + field_name++; + + h = header_find(p, header_name); + if (!h) { + free(header_name); + return NULL; + } + + f = struct_type_field_find(h->st, field_name); + if (!f) { + free(header_name); + return NULL; + } + + if (header) + *header = h; + + free(header_name); + return f; +} + int rte_swx_pipeline_packet_header_register(struct rte_swx_pipeline *p, const char *name, @@ -1229,6 +1348,18 @@ header_free(struct rte_swx_pipeline *p) /* * Meta-data. */ +static struct field * +metadata_field_parse(struct rte_swx_pipeline *p, const char *name) +{ + if (!p->metadata_st) + return NULL; + + if (name[0] != 'm' || name[1] != '.') + return NULL; + + return struct_type_field_find(p->metadata_st, &name[2]); +} + int rte_swx_pipeline_packet_metadata_register(struct rte_swx_pipeline *p, const char *struct_type_name) @@ -1407,6 +1538,536 @@ action_free(struct rte_swx_pipeline *p) } } +/* + * Table. + */ +static struct table_type * +table_type_find(struct rte_swx_pipeline *p, const char *name) +{ + struct table_type *elem; + + TAILQ_FOREACH(elem, &p->table_types, node) + if (strcmp(elem->name, name) == 0) + return elem; + + return NULL; +} + +static struct table_type * +table_type_resolve(struct rte_swx_pipeline *p, + const char *recommended_type_name, + enum rte_swx_table_match_type match_type) +{ + struct table_type *elem; + + /* Only consider the recommended type if the match type is correct. */ + if (recommended_type_name) + TAILQ_FOREACH(elem, &p->table_types, node) + if (!strcmp(elem->name, recommended_type_name) && + (elem->match_type == match_type)) + return elem; + + /* Ignore the recommended type and get the first element with this match + * type. + */ + TAILQ_FOREACH(elem, &p->table_types, node) + if (elem->match_type == match_type) + return elem; + + return NULL; +} + +static struct table * +table_find(struct rte_swx_pipeline *p, const char *name) +{ + struct table *elem; + + TAILQ_FOREACH(elem, &p->tables, node) + if (strcmp(elem->name, name) == 0) + return elem; + + return NULL; +} + +static struct table * +table_find_by_id(struct rte_swx_pipeline *p, uint32_t id) +{ + struct table *table = NULL; + + TAILQ_FOREACH(table, &p->tables, node) + if (table->id == id) + return table; + + return NULL; +} + +int +rte_swx_pipeline_table_type_register(struct rte_swx_pipeline *p, + const char *name, + enum rte_swx_table_match_type match_type, + struct rte_swx_table_ops *ops) +{ + struct table_type *elem; + + CHECK(p, EINVAL); + + CHECK_NAME(name, EINVAL); + CHECK(!table_type_find(p, name), EEXIST); + + CHECK(ops, EINVAL); + CHECK(ops->create, EINVAL); + CHECK(ops->lkp, EINVAL); + CHECK(ops->free, EINVAL); + + /* Node allocation. */ + elem = calloc(1, sizeof(struct table_type)); + CHECK(elem, ENOMEM); + + /* Node initialization. */ + strcpy(elem->name, name); + elem->match_type = match_type; + memcpy(&elem->ops, ops, sizeof(*ops)); + + /* Node add to tailq. */ + TAILQ_INSERT_TAIL(&p->table_types, elem, node); + + return 0; +} + +static enum rte_swx_table_match_type +table_match_type_resolve(struct rte_swx_match_field_params *fields, + uint32_t n_fields) +{ + uint32_t i; + + for (i = 0; i < n_fields; i++) + if (fields[i].match_type != RTE_SWX_TABLE_MATCH_EXACT) + break; + + if (i == n_fields) + return RTE_SWX_TABLE_MATCH_EXACT; + + if ((i == n_fields - 1) && + (fields[i].match_type == RTE_SWX_TABLE_MATCH_LPM)) + return RTE_SWX_TABLE_MATCH_LPM; + + return RTE_SWX_TABLE_MATCH_WILDCARD; +} + +int +rte_swx_pipeline_table_config(struct rte_swx_pipeline *p, + const char *name, + struct rte_swx_pipeline_table_params *params, + const char *recommended_table_type_name, + const char *args, + uint32_t size) +{ + struct table_type *type; + struct table *t; + struct action *default_action; + struct header *header = NULL; + int is_header = 0; + uint32_t offset_prev = 0, action_data_size_max = 0, i; + + CHECK(p, EINVAL); + + CHECK_NAME(name, EINVAL); + CHECK(!table_find(p, name), EEXIST); + + CHECK(params, EINVAL); + + /* Match checks. */ + CHECK(!params->n_fields || params->fields, EINVAL); + for (i = 0; i < params->n_fields; i++) { + struct rte_swx_match_field_params *field = ¶ms->fields[i]; + struct header *h; + struct field *hf, *mf; + uint32_t offset; + + CHECK_NAME(field->name, EINVAL); + + hf = header_field_parse(p, field->name, &h); + mf = metadata_field_parse(p, field->name); + CHECK(hf || mf, EINVAL); + + offset = hf ? hf->offset : mf->offset; + + if (i == 0) { + is_header = hf ? 1 : 0; + header = hf ? h : NULL; + offset_prev = offset; + + continue; + } + + CHECK((is_header && hf && (h->id == header->id)) || + (!is_header && mf), EINVAL); + + CHECK(offset > offset_prev, EINVAL); + offset_prev = offset; + } + + /* Action checks. */ + CHECK(params->n_actions, EINVAL); + CHECK(params->action_names, EINVAL); + for (i = 0; i < params->n_actions; i++) { + const char *action_name = params->action_names[i]; + struct action *a; + uint32_t action_data_size; + + CHECK(action_name, EINVAL); + + a = action_find(p, action_name); + CHECK(a, EINVAL); + + action_data_size = a->st ? a->st->n_bits / 8 : 0; + if (action_data_size > action_data_size_max) + action_data_size_max = action_data_size; + } + + CHECK(params->default_action_name, EINVAL); + for (i = 0; i < p->n_actions; i++) + if (!strcmp(params->action_names[i], + params->default_action_name)) + break; + CHECK(i < params->n_actions, EINVAL); + default_action = action_find(p, params->default_action_name); + CHECK((default_action->st && params->default_action_data) || + !params->default_action_data, EINVAL); + + /* Table type checks. */ + if (params->n_fields) { + enum rte_swx_table_match_type match_type; + + match_type = table_match_type_resolve(params->fields, + params->n_fields); + type = table_type_resolve(p, + recommended_table_type_name, + match_type); + CHECK(type, EINVAL); + } else { + type = NULL; + } + + /* Memory allocation. */ + t = calloc(1, sizeof(struct table)); + CHECK(t, ENOMEM); + + t->fields = calloc(params->n_fields, sizeof(struct match_field)); + if (!t->fields) { + free(t); + CHECK(0, ENOMEM); + } + + t->actions = calloc(params->n_actions, sizeof(struct action *)); + if (!t->actions) { + free(t->fields); + free(t); + CHECK(0, ENOMEM); + } + + if (action_data_size_max) { + t->default_action_data = calloc(1, action_data_size_max); + if (!t->default_action_data) { + free(t->actions); + free(t->fields); + free(t); + CHECK(0, ENOMEM); + } + } + + /* Node initialization. */ + strcpy(t->name, name); + if (args && args[0]) + strcpy(t->args, args); + t->type = type; + + for (i = 0; i < params->n_fields; i++) { + struct rte_swx_match_field_params *field = ¶ms->fields[i]; + struct match_field *f = &t->fields[i]; + + f->match_type = field->match_type; + f->field = is_header ? + header_field_parse(p, field->name, NULL) : + metadata_field_parse(p, field->name); + } + t->n_fields = params->n_fields; + t->is_header = is_header; + t->header = header; + + for (i = 0; i < params->n_actions; i++) + t->actions[i] = action_find(p, params->action_names[i]); + t->default_action = default_action; + if (default_action->st) + memcpy(t->default_action_data, + params->default_action_data, + default_action->st->n_bits / 8); + t->n_actions = params->n_actions; + t->default_action_is_const = params->default_action_is_const; + t->action_data_size_max = action_data_size_max; + + t->size = size; + t->id = p->n_tables; + + /* Node add to tailq. */ + TAILQ_INSERT_TAIL(&p->tables, t, node); + p->n_tables++; + + return 0; +} + +static struct rte_swx_table_params * +table_params_get(struct table *table) +{ + struct rte_swx_table_params *params; + struct field *first, *last; + uint8_t *key_mask; + uint32_t key_size, key_offset, action_data_size, i; + + /* Memory allocation. */ + params = calloc(1, sizeof(struct rte_swx_table_params)); + if (!params) + return NULL; + + /* Key offset and size. */ + first = table->fields[0].field; + last = table->fields[table->n_fields - 1].field; + key_offset = first->offset / 8; + key_size = (last->offset + last->n_bits - first->offset) / 8; + + /* Memory allocation. */ + key_mask = calloc(1, key_size); + if (!key_mask) { + free(params); + return NULL; + } + + /* Key mask. */ + for (i = 0; i < table->n_fields; i++) { + struct field *f = table->fields[i].field; + uint32_t start = (f->offset - first->offset) / 8; + size_t size = f->n_bits / 8; + + memset(&key_mask[start], 0xFF, size); + } + + /* Action data size. */ + action_data_size = 0; + for (i = 0; i < table->n_actions; i++) { + struct action *action = table->actions[i]; + uint32_t ads = action->st ? action->st->n_bits / 8 : 0; + + if (ads > action_data_size) + action_data_size = ads; + } + + /* Fill in. */ + params->match_type = table->type->match_type; + params->key_size = key_size; + params->key_offset = key_offset; + params->key_mask0 = key_mask; + params->action_data_size = action_data_size; + params->n_keys_max = table->size; + + return params; +} + +static void +table_params_free(struct rte_swx_table_params *params) +{ + if (!params) + return; + + free(params->key_mask0); + free(params); +} + +static int +table_state_build(struct rte_swx_pipeline *p) +{ + struct table *table; + + p->table_state = calloc(p->n_tables, + sizeof(struct rte_swx_table_state)); + CHECK(p->table_state, ENOMEM); + + TAILQ_FOREACH(table, &p->tables, node) { + struct rte_swx_table_state *ts = &p->table_state[table->id]; + + if (table->type) { + struct rte_swx_table_params *params; + + /* ts->obj. */ + params = table_params_get(table); + CHECK(params, ENOMEM); + + ts->obj = table->type->ops.create(params, + NULL, + table->args, + p->numa_node); + + table_params_free(params); + CHECK(ts->obj, ENODEV); + } + + /* ts->default_action_data. */ + if (table->action_data_size_max) { + ts->default_action_data = + malloc(table->action_data_size_max); + CHECK(ts->default_action_data, ENOMEM); + + memcpy(ts->default_action_data, + table->default_action_data, + table->action_data_size_max); + } + + /* ts->default_action_id. */ + ts->default_action_id = table->default_action->id; + } + + return 0; +} + +static void +table_state_build_free(struct rte_swx_pipeline *p) +{ + uint32_t i; + + if (!p->table_state) + return; + + for (i = 0; i < p->n_tables; i++) { + struct rte_swx_table_state *ts = &p->table_state[i]; + struct table *table = table_find_by_id(p, i); + + /* ts->obj. */ + if (table->type && ts->obj) + table->type->ops.free(ts->obj); + + /* ts->default_action_data. */ + free(ts->default_action_data); + } + + free(p->table_state); + p->table_state = NULL; +} + +static void +table_state_free(struct rte_swx_pipeline *p) +{ + table_state_build_free(p); +} + +static int +table_stub_lkp(void *table __rte_unused, + void *mailbox __rte_unused, + uint8_t **key __rte_unused, + uint64_t *action_id __rte_unused, + uint8_t **action_data __rte_unused, + int *hit) +{ + *hit = 0; + return 1; /* DONE. */ +} + +static int +table_build(struct rte_swx_pipeline *p) +{ + uint32_t i; + + for (i = 0; i < RTE_SWX_PIPELINE_THREADS_MAX; i++) { + struct thread *t = &p->threads[i]; + struct table *table; + + t->tables = calloc(p->n_tables, sizeof(struct table_runtime)); + CHECK(t->tables, ENOMEM); + + TAILQ_FOREACH(table, &p->tables, node) { + struct table_runtime *r = &t->tables[table->id]; + + if (table->type) { + uint64_t size; + + size = table->type->ops.mailbox_size_get(); + + /* r->func. */ + r->func = table->type->ops.lkp; + + /* r->mailbox. */ + if (size) { + r->mailbox = calloc(1, size); + CHECK(r->mailbox, ENOMEM); + } + + /* r->key. */ + r->key = table->is_header ? + &t->structs[table->header->struct_id] : + &t->structs[p->metadata_struct_id]; + } else { + r->func = table_stub_lkp; + } + } + } + + return 0; +} + +static void +table_build_free(struct rte_swx_pipeline *p) +{ + uint32_t i; + + for (i = 0; i < RTE_SWX_PIPELINE_THREADS_MAX; i++) { + struct thread *t = &p->threads[i]; + uint32_t j; + + if (!t->tables) + continue; + + for (j = 0; j < p->n_tables; j++) { + struct table_runtime *r = &t->tables[j]; + + free(r->mailbox); + } + + free(t->tables); + t->tables = NULL; + } +} + +static void +table_free(struct rte_swx_pipeline *p) +{ + table_build_free(p); + + /* Tables. */ + for ( ; ; ) { + struct table *elem; + + elem = TAILQ_FIRST(&p->tables); + if (!elem) + break; + + TAILQ_REMOVE(&p->tables, elem, node); + free(elem->fields); + free(elem->actions); + free(elem->default_action_data); + free(elem); + } + + /* Table types. */ + for ( ; ; ) { + struct table_type *elem; + + elem = TAILQ_FIRST(&p->table_types); + if (!elem) + break; + + TAILQ_REMOVE(&p->table_types, elem, node); + free(elem); + } +} + /* * Pipeline. */ @@ -1433,6 +2094,8 @@ rte_swx_pipeline_config(struct rte_swx_pipeline **p, int numa_node) TAILQ_INIT(&pipeline->extern_funcs); TAILQ_INIT(&pipeline->headers); TAILQ_INIT(&pipeline->actions); + TAILQ_INIT(&pipeline->table_types); + TAILQ_INIT(&pipeline->tables); pipeline->n_structs = 1; /* Struct 0 is reserved for action_data. */ pipeline->numa_node = numa_node; @@ -1447,6 +2110,8 @@ rte_swx_pipeline_free(struct rte_swx_pipeline *p) if (!p) return; + table_state_free(p); + table_free(p); action_free(p); metadata_free(p); header_free(p); @@ -1499,10 +2164,20 @@ rte_swx_pipeline_build(struct rte_swx_pipeline *p) if (status) goto error; + status = table_build(p); + if (status) + goto error; + + status = table_state_build(p); + if (status) + goto error; + p->build_done = 1; return 0; error: + table_state_build_free(p); + table_build_free(p); action_build_free(p); metadata_build_free(p); header_build_free(p); @@ -1514,3 +2189,28 @@ rte_swx_pipeline_build(struct rte_swx_pipeline *p) return status; } + +/* + * Control. + */ +int +rte_swx_pipeline_table_state_get(struct rte_swx_pipeline *p, + struct rte_swx_table_state **table_state) +{ + if (!p || !table_state || !p->build_done) + return -EINVAL; + + *table_state = p->table_state; + return 0; +} + +int +rte_swx_pipeline_table_state_set(struct rte_swx_pipeline *p, + struct rte_swx_table_state *table_state) +{ + if (!p || !table_state || !p->build_done) + return -EINVAL; + + p->table_state = table_state; + return 0; +} diff --git a/lib/librte_pipeline/rte_swx_pipeline.h b/lib/librte_pipeline/rte_swx_pipeline.h index 1b20293cb..d7e3ba1ec 100644 --- a/lib/librte_pipeline/rte_swx_pipeline.h +++ b/lib/librte_pipeline/rte_swx_pipeline.h @@ -19,6 +19,7 @@ extern "C" { #include #include "rte_swx_port.h" +#include "rte_swx_table.h" #include "rte_swx_extern.h" /** Name size. */ @@ -377,6 +378,123 @@ rte_swx_pipeline_action_config(struct rte_swx_pipeline *p, const char **instructions, uint32_t n_instructions); +/* + * Pipeline table + */ + +/** + * Pipeline table type register + * + * @param[in] p + * Pipeline handle. + * @param[in] name + * Table type name. + * @param[in] match type + * Match type implemented by the new table type. + * @param[in] ops + * Table type operations. + * @return + * 0 on success or the following error codes otherwise: + * -EINVAL: Invalid argument; + * -ENOMEM: Not enough space/cannot allocate memory; + * -EEXIST: Table type with this name already exists. + */ +__rte_experimental +int +rte_swx_pipeline_table_type_register(struct rte_swx_pipeline *p, + const char *name, + enum rte_swx_table_match_type match_type, + struct rte_swx_table_ops *ops); + +/** Match field parameters. */ +struct rte_swx_match_field_params { + /** Match field name. Must be either a field of one of the registered + * packet headers ("h.header.field") or a field of the registered + * meta-data ("m.field"). + */ + const char *name; + + /** Match type of the field. */ + enum rte_swx_table_match_type match_type; +}; + +/** Pipeline table parameters. */ +struct rte_swx_pipeline_table_params { + /** The set of match fields for the current table. + * Restriction: All the match fields of the current table need to be + * part of the same struct, i.e. either all the match fields are part of + * the same header or all the match fields are part of the meta-data. + */ + struct rte_swx_match_field_params *fields; + + /** The number of match fields for the current table. If set to zero, no + * "regular" entries (i.e. entries other than the default entry) can be + * added to the current table and the match process always results in + * lookup miss. + */ + uint32_t n_fields; + + /** The set of actions for the current table. */ + const char **action_names; + + /** The number of actions for the current table. Must be at least one. + */ + uint32_t n_actions; + + /** The default table action that gets executed on lookup miss. Must be + * one of the table actions included in the *action_names*. + */ + const char *default_action_name; + + /** Default action data. The size of this array is the action data size + * of the default action. Must be NULL if the default action data size + * is zero. + */ + uint8_t *default_action_data; + + /** If non-zero (true), then the default action of the current table + * cannot be changed. If zero (false), then the default action can be + * changed in the future with another action from the *action_names* + * list. + */ + int default_action_is_const; +}; + +/** + * Pipeline table configure + * + * @param[out] p + * Pipeline handle. + * @param[in] name + * Table name. + * @param[in] params + * Table parameters. + * @param[in] recommended_table_type_name + * Recommended table type. Typically set to NULL. Useful as guidance when + * there are multiple table types registered for the match type of the table, + * as determined from the table match fields specification. Silently ignored + * if the recommended table type does not exist or it serves a different match + * type. + * @param[in] args + * Table creation arguments. + * @param[in] size + * Guideline on maximum number of table entries. + * @return + * 0 on success or the following error codes otherwise: + * -EINVAL: Invalid argument; + * -ENOMEM: Not enough space/cannot allocate memory; + * -EEXIST: Table with this name already exists; + * -ENODEV: Table creation error. + */ +__rte_experimental +int +rte_swx_pipeline_table_config(struct rte_swx_pipeline *p, + const char *name, + struct rte_swx_pipeline_table_params *params, + const char *recommended_table_type_name, + const char *args, + uint32_t size); + /** * Pipeline build * diff --git a/lib/librte_table/meson.build b/lib/librte_table/meson.build index 71d134768..b9d4fe3dc 100644 --- a/lib/librte_table/meson.build +++ b/lib/librte_table/meson.build @@ -22,7 +22,8 @@ headers = files('rte_table.h', 'rte_table_hash_func_arm64.h', 'rte_lru.h', 'rte_table_array.h', - 'rte_table_stub.h') + 'rte_table_stub.h', + 'rte_swx_table.h',) deps += ['mbuf', 'port', 'lpm', 'hash', 'acl'] if arch_subdir == 'x86' diff --git a/lib/librte_table/rte_swx_table.h b/lib/librte_table/rte_swx_table.h new file mode 100644 index 000000000..dc434b72e --- /dev/null +++ b/lib/librte_table/rte_swx_table.h @@ -0,0 +1,295 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright(c) 2020 Intel Corporation + */ +#ifndef __INCLUDE_RTE_SWX_TABLE_H__ +#define __INCLUDE_RTE_SWX_TABLE_H__ + +#ifdef __cplusplus +extern "C" { +#endif + +/** + * @file + * RTE SWX Table + * + * Table interface. + */ + +#include +#include + +/** Match type. */ +enum rte_swx_table_match_type { + /** Wildcard Match (WM). */ + RTE_SWX_TABLE_MATCH_WILDCARD, + + /** Longest Prefix Match (LPM). */ + RTE_SWX_TABLE_MATCH_LPM, + + /** Exact Match (EM). */ + RTE_SWX_TABLE_MATCH_EXACT, +}; + +/** Table creation parameters. */ +struct rte_swx_table_params { + /** Table match type. */ + enum rte_swx_table_match_type match_type; + + /** Key size in bytes. */ + uint32_t key_size; + + /** Offset of the first byte of the key within the key buffer. */ + uint32_t key_offset; + + /** Mask of *key_size* bytes logically laid over the bytes at positions + * *key_offset* .. (*key_offset* + *key_size* - 1) of the key buffer in + * order to specify which bits from the key buffer are part of the key + * and which ones are not. A bit value of 1 in the *key_mask0* means the + * respective bit in the key buffer is part of the key, while a bit + * value of 0 means the opposite. A NULL value means that all the bits + * are part of the key, i.e. the *key_mask0* is an all-ones mask. + */ + uint8_t *key_mask0; + + /** Maximum size (in bytes) of the action data. The data stored in the + * table for each entry is equal to *action_data_size* plus 8 bytes, + * which are used to store the action ID. + */ + uint32_t action_data_size; + + /** Maximum number of keys to be stored in the table together with their + * associated data. + */ + uint32_t n_keys_max; +}; + +/** Table entry. */ +struct rte_swx_table_entry { + /** Used to facilitate the membership of this table entry to a + * linked list. + */ + TAILQ_ENTRY(rte_swx_table_entry) node; + + /** Key value for the current entry. Array of *key_size* bytes or NULL + * if the *key_size* for the current table is 0. + */ + uint8_t *key; + + /** Key mask for the current entry. Array of *key_size* bytes that is + * logically and'ed with *key_mask0* of the current table. A NULL value + * means that all the key bits already enabled by *key_mask0* are part + * of the key of the current entry. + */ + uint8_t *key_mask; + + /** Placeholder for a possible compressed version of the *key* and + * *key_mask* of the current entry. Typically a hash signature, its main + * purpose is to the linked list search operation. Should be ignored by + * the API functions below. + */ + uint64_t key_signature; + + /** Action ID for the current entry. */ + uint64_t action_id; + + /** Action data for the current entry. Its size is defined by the action + * specified by the *action_id*. It must be NULL when the action data + * size of the *action_id* action is NULL. It must never exceed the + * *action_data_size* of the table. + */ + uint8_t *action_data; +}; + +/** List of table entries. */ +TAILQ_HEAD(rte_swx_table_entry_list, rte_swx_table_entry); + +/** + * Table memory footprint get + * + * @param[in] params + * Table create parameters. + * @param[in] entries + * Table entries. + * @param[in] args + * Any additional table create arguments. It may be NULL. + * @return + * Table memory footprint in bytes, if successful, or zero, on error. + */ +typedef uint64_t +(*rte_swx_table_footprint_get_t)(struct rte_swx_table_params *params, + struct rte_swx_table_entry_list *entries, + const char *args); + +/** + * Table mailbox size get + * + * The mailbox is used to store the context of a lookup operation that is in + * progress and it is passed as a parameter to the lookup operation. This allows + * for multiple concurrent lookup operations into the same table. + * + * @param[in] params + * Table creation parameters. + * @param[in] entries + * Entries to be added to the table at creation time. + * @param[in] args + * Any additional table create arguments. It may be NULL. + * @return + * Table memory footprint in bytes, on success, or zero, on error. + */ +typedef uint64_t +(*rte_swx_table_mailbox_size_get_t)(void); + +/** + * Table create + * + * @param[in] params + * Table creation parameters. + * @param[in] entries + * Entries to be added to the table at creation time. + * @param[in] args + * Any additional table create arguments. It may be NULL. + * @param[in] numa_node + * Non-Uniform Memory Access (NUMA) node. + * @return + * Table handle, on success, or NULL, on error. + */ +typedef void * +(*rte_swx_table_create_t)(struct rte_swx_table_params *params, + struct rte_swx_table_entry_list *entries, + const char *args, + int numa_node); + +/** + * Table entry add + * + * @param[in] table + * Table handle. + * @param[in] entry + * Entry to be added to the table. + * @return + * 0 on success or the following error codes otherwise: + * -EINVAL: Invalid table handle, entry or entry field; + * -ENOSPC: Table full. + */ +typedef int +(*rte_swx_table_add_t)(void *table, + struct rte_swx_table_entry *entry); + +/** + * Table entry delete + * + * @param[in] table + * Table handle. + * @param[in] entry + * Entry to be deleted from the table. The entry *action_id* and *action_data* + * fields are ignored. + * @return + * 0 on success or the following error codes otherwise: + * -EINVAL: Invalid table handle, entry or entry field; + * -ENOSPC: Table full. + */ +typedef int +(*rte_swx_table_delete_t)(void *table, + struct rte_swx_table_entry *entry); + +/** + * Table lookup + * + * The table lookup operation searches a given key in the table and upon its + * completion it returns an indication of whether the key is found in the table + * (lookup hit) or not (lookup miss). In case of lookup hit, the action_id and + * the action_data associated with the key are also returned. + * + * Multiple invocations of this function may be required in order to complete a + * single table lookup operation for a given table and a given lookup key. The + * completion of the table lookup operation is flagged by a return value of 1; + * in case of a return value of 0, the function must be invoked again with + * exactly the same arguments. + * + * The mailbox argument is used to store the context of an on-going table lookup + * operation. The mailbox mechanism allows for multiple concurrent table lookup + * operations into the same table. + * + * The typical reason an implementation may choose to split the table lookup + * operation into multiple steps is to hide the latency of the inherrent memory + * read operations: before a read operation with the source data likely not in + * the CPU cache, the source data prefetch is issued and the table lookup + * operation is postponed in favor of some other unrelated work, which the CPU + * executes in parallel with the source data being fetched into the CPU cache; + * later on, the table lookup operation is resumed, this time with the source + * data likely to be read from the CPU cache with no CPU pipeline stall, which + * significantly improves the table lookup performance. + * + * @param[in] table + * Table handle. + * @param[in] mailbox + * Mailbox for the current table lookup operation. + * @param[in] key + * Lookup key. Its size mult be equal to the table *key_size*. If the latter + * is zero, then the lookup key must be NULL. + * @param[out] action_id + * ID of the action associated with the *key*. Must point to a valid 64-bit + * variable. Only valid when the function returns 1 and *hit* is set to true. + * @param[out] action_data + * Action data for the *action_id* action. Must point to a valid array of + * table *action_data_size* bytes. Only valid when the function returns 1 and + * *hit* is set to true. + * @param[out] hit + * Only valid when the function returns 1. Set to non-zero (true) on table + * lookup hit and to zero (false) on table lookup miss. + * @return + * 0 when the table lookup operation is not yet completed, and 1 when the + * table lookup operation is completed. No other return values are allowed. + */ +typedef int +(*rte_swx_table_lookup_t)(void *table, + void *mailbox, + uint8_t **key, + uint64_t *action_id, + uint8_t **action_data, + int *hit); + +/** + * Table free + * + * @param[in] table + * Table handle. + */ +typedef void +(*rte_swx_table_free_t)(void *table); + +/** Table operations. */ +struct rte_swx_table_ops { + /** Table memory footprint get. Set to NULL when not supported. */ + rte_swx_table_footprint_get_t footprint_get; + + /** Table mailbox size get. When NULL, the mailbox size is 0. */ + rte_swx_table_mailbox_size_get_t mailbox_size_get; + + /** Table create. Must be non-NULL. */ + rte_swx_table_create_t create; + + /** Incremental table entry add. Set to NULL when not supported, in + * which case the existing table has to be destroyed and a new table + * built from scratch with the new entry included. + */ + rte_swx_table_add_t add; + + /** Incremental table entry delete. Set to NULL when not supported, in + * which case the existing table has to be destroyed and a new table + * built from scratch with the entry excluded. + */ + rte_swx_table_delete_t del; + + /** Table lookup. Must be non-NULL. */ + rte_swx_table_lookup_t lkp; + + /** Table free. Must be non-NULL. */ + rte_swx_table_free_t free; +}; + +#ifdef __cplusplus +} +#endif + +#endif -- 2.17.1